[2026-07-02 23:56:17] Waiting for PID 999999999. [2026-07-02 23:56:17] PID 999999999 finished. Starting Qwen3 generalization queue. [2026-07-02 23:56:17] Model: Qwen/Qwen3-8B [2026-07-02 23:56:17] Datasets: chemistry physics biology material tooluse [2026-07-02 23:56:17] Methods: grpo rlsd_tr [2026-07-02 23:56:17] Starting Qwen/Qwen3-8B chemistry grpo. Experiment: qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: chemistry Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet Ray tmp: /tmp/ray_q3g_chemistry_grpo_Qwen3_8B 2026-07-02 23:56:19,612 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/chemistry Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_grpo_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_grpo_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 23:56:29,407 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2224264) TaskRunner hostname: mole-workspace-rw, PID: 2224264 (TaskRunner pid=2224264) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2224264) 'calculate_entropy': False, (TaskRunner pid=2224264) 'calculate_sum_pi_squared': False, (TaskRunner pid=2224264) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2224264) 'async_save': False, (TaskRunner pid=2224264) 'load_contents': ['model', (TaskRunner pid=2224264) 'optimizer', (TaskRunner pid=2224264) 'extra'], (TaskRunner pid=2224264) 'save_contents': ['model', (TaskRunner pid=2224264) 'optimizer', (TaskRunner pid=2224264) 'extra']}, (TaskRunner pid=2224264) 'clip_ratio': 0.2, (TaskRunner pid=2224264) 'clip_ratio_c': 3.0, (TaskRunner pid=2224264) 'clip_ratio_high': 0.28, (TaskRunner pid=2224264) 'clip_ratio_low': 0.2, (TaskRunner pid=2224264) 'data_loader_seed': 42, (TaskRunner pid=2224264) 'entropy_checkpointing': False, (TaskRunner pid=2224264) 'entropy_coeff': 0, (TaskRunner pid=2224264) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2224264) 'freeze_vision_tower': False, (TaskRunner pid=2224264) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2224264) 'dtype': 'bfloat16', (TaskRunner pid=2224264) 'entropy_checkpointing': False, (TaskRunner pid=2224264) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2224264) 'forward_only': False, (TaskRunner pid=2224264) 'forward_prefetch': False, (TaskRunner pid=2224264) 'fsdp_size': -1, (TaskRunner pid=2224264) 'full_determinism': False, (TaskRunner pid=2224264) 'model_dtype': 'fp32', (TaskRunner pid=2224264) 'offload_policy': False, (TaskRunner pid=2224264) 'optimizer_offload': False, (TaskRunner pid=2224264) 'param_offload': False, (TaskRunner pid=2224264) 'reshard_after_forward': True, (TaskRunner pid=2224264) 'seed': 42, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_orig_params': False, (TaskRunner pid=2224264) 'use_torch_compile': True, (TaskRunner pid=2224264) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2224264) 'grad_clip': 1.0, (TaskRunner pid=2224264) 'kl_loss_coef': 0.0, (TaskRunner pid=2224264) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2224264) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2224264) 'loss_scale_factor': None, (TaskRunner pid=2224264) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2224264) 'betas': [0.9, 0.999], (TaskRunner pid=2224264) 'clip_grad': 1.0, (TaskRunner pid=2224264) 'lr': 1e-06, (TaskRunner pid=2224264) 'lr_scheduler_type': 'constant', (TaskRunner pid=2224264) 'lr_warmup_steps': 10, (TaskRunner pid=2224264) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2224264) 'min_lr_ratio': 0.0, (TaskRunner pid=2224264) 'num_cycles': 0.5, (TaskRunner pid=2224264) 'optimizer': 'AdamW', (TaskRunner pid=2224264) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2224264) 'override_optimizer_config': None, (TaskRunner pid=2224264) 'total_training_steps': -1, (TaskRunner pid=2224264) 'warmup_style': None, (TaskRunner pid=2224264) 'weight_decay': 0.01}, (TaskRunner pid=2224264) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2224264) 'clip_cov_lb': 1.0, (TaskRunner pid=2224264) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2224264) 'clip_cov_ub': 5.0, (TaskRunner pid=2224264) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2224264) 'loss_mode': 'vanilla', (TaskRunner pid=2224264) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2224264) 'ppo_epochs': 1, (TaskRunner pid=2224264) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2224264) 'ppo_micro_batch_size': None, (TaskRunner pid=2224264) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2224264) 'ppo_mini_batch_size': 8, (TaskRunner pid=2224264) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'all_ranks': False, (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'ranks': [], (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'tool': None, (TaskRunner pid=2224264) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2224264) 'analysis': True, (TaskRunner pid=2224264) 'contents': [], (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'level': 'level0'}, (TaskRunner pid=2224264) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'discrete': False}, (TaskRunner pid=2224264) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2224264) 'step_end': None, (TaskRunner pid=2224264) 'step_start': 0}, (TaskRunner pid=2224264) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2224264) 'rollout_n': 8, (TaskRunner pid=2224264) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2224264) 'mode': 'disabled', (TaskRunner pid=2224264) 'record_file': None, (TaskRunner pid=2224264) 'replay_file': None}, (TaskRunner pid=2224264) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2224264) 'alpha': 0.5, (TaskRunner pid=2224264) 'distillation_add_tail': True, (TaskRunner pid=2224264) 'distillation_topk': 100, (TaskRunner pid=2224264) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2224264) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2224264) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'loss_weight': 0.0, (TaskRunner pid=2224264) 'mask': 'all'}, (TaskRunner pid=2224264) 'feedback_template': '\n' (TaskRunner pid=2224264) 'The ' (TaskRunner pid=2224264) 'following ' (TaskRunner pid=2224264) 'is ' (TaskRunner pid=2224264) 'feedback ' (TaskRunner pid=2224264) 'from ' (TaskRunner pid=2224264) 'your ' (TaskRunner pid=2224264) 'unsuccessful ' (TaskRunner pid=2224264) 'earlier ' (TaskRunner pid=2224264) 'attempt:\n' (TaskRunner pid=2224264) '\n' (TaskRunner pid=2224264) '{feedback_raw}', (TaskRunner pid=2224264) 'full_logit_distillation': True, (TaskRunner pid=2224264) 'include_environment_feedback': True, (TaskRunner pid=2224264) 'is_clip': 2, (TaskRunner pid=2224264) 'max_reprompt_len': 22528, (TaskRunner pid=2224264) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2224264) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2224264) '\n' (TaskRunner pid=2224264) 'Correctly ' (TaskRunner pid=2224264) 'solve ' (TaskRunner pid=2224264) 'the ' (TaskRunner pid=2224264) 'original ' (TaskRunner pid=2224264) 'question.', (TaskRunner pid=2224264) 'reprompt_truncation': 'right', (TaskRunner pid=2224264) 'solution_template': '\n' (TaskRunner pid=2224264) 'Correct ' (TaskRunner pid=2224264) 'solution:\n' (TaskRunner pid=2224264) '\n' (TaskRunner pid=2224264) '{successful_previous_attempt}', (TaskRunner pid=2224264) 'srpo_dynamic_weighting': False, (TaskRunner pid=2224264) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2224264) 'success_reward_threshold': 0.5, (TaskRunner pid=2224264) 'teacher_regularization': 'ema', (TaskRunner pid=2224264) 'teacher_update_rate': 0.0, (TaskRunner pid=2224264) 'token_reweight_decay_steps': None, (TaskRunner pid=2224264) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2224264) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2224264) 'shuffle': False, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2224264) 'tau_neg': 1.05, (TaskRunner pid=2224264) 'tau_pos': 1.0, (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_dynamic_bsz': False, (TaskRunner pid=2224264) 'use_fused_kernels': False, (TaskRunner pid=2224264) 'use_kl_loss': False, (TaskRunner pid=2224264) 'use_prefix_grouper': False, (TaskRunner pid=2224264) 'use_remove_padding': True, (TaskRunner pid=2224264) 'use_torch_compile': True}, (TaskRunner pid=2224264) 'hybrid_engine': True, (TaskRunner pid=2224264) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2224264) 'custom_chat_template': None, (TaskRunner pid=2224264) 'enable_activation_offload': False, (TaskRunner pid=2224264) 'enable_gradient_checkpointing': True, (TaskRunner pid=2224264) 'exclude_modules': None, (TaskRunner pid=2224264) 'external_lib': None, (TaskRunner pid=2224264) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2224264) 'hf_config_path': None, (TaskRunner pid=2224264) 'lora_adapter_path': None, (TaskRunner pid=2224264) 'lora_alpha': 16, (TaskRunner pid=2224264) 'lora_rank': 0, (TaskRunner pid=2224264) 'override_config': {}, (TaskRunner pid=2224264) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2224264) 'target_modules': 'all-linear', (TaskRunner pid=2224264) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2224264) 'num_shards': 4}, (TaskRunner pid=2224264) 'tokenizer_path': None, (TaskRunner pid=2224264) 'trust_remote_code': True, (TaskRunner pid=2224264) 'use_fused_kernels': False, (TaskRunner pid=2224264) 'use_liger': False, (TaskRunner pid=2224264) 'use_remove_padding': True, (TaskRunner pid=2224264) 'use_shm': False}, (TaskRunner pid=2224264) 'nccl_timeout': 600, (TaskRunner pid=2224264) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2224264) 'entropy_checkpointing': False, (TaskRunner pid=2224264) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2224264) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2224264) 'dtype': 'bfloat16', (TaskRunner pid=2224264) 'entropy_checkpointing': False, (TaskRunner pid=2224264) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2224264) 'forward_only': True, (TaskRunner pid=2224264) 'forward_prefetch': False, (TaskRunner pid=2224264) 'fsdp_size': -1, (TaskRunner pid=2224264) 'full_determinism': False, (TaskRunner pid=2224264) 'model_dtype': 'fp32', (TaskRunner pid=2224264) 'offload_policy': False, (TaskRunner pid=2224264) 'optimizer_offload': False, (TaskRunner pid=2224264) 'param_offload': False, (TaskRunner pid=2224264) 'reshard_after_forward': True, (TaskRunner pid=2224264) 'seed': 42, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_orig_params': False, (TaskRunner pid=2224264) 'use_torch_compile': True, (TaskRunner pid=2224264) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2224264) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2224264) 'log_prob_micro_batch_size': None, (TaskRunner pid=2224264) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2224264) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2224264) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'all_ranks': False, (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'ranks': [], (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'tool': None, (TaskRunner pid=2224264) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2224264) 'analysis': True, (TaskRunner pid=2224264) 'contents': [], (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'level': 'level0'}, (TaskRunner pid=2224264) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'discrete': False}, (TaskRunner pid=2224264) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2224264) 'step_end': None, (TaskRunner pid=2224264) 'step_start': 0}, (TaskRunner pid=2224264) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2224264) 'rollout_n': 8, (TaskRunner pid=2224264) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2224264) 'mode': 'disabled', (TaskRunner pid=2224264) 'record_file': None, (TaskRunner pid=2224264) 'replay_file': None}, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_torch_compile': True}, (TaskRunner pid=2224264) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2224264) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2224264) 'agent_loop_config_path': None, (TaskRunner pid=2224264) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2224264) 'name': None, (TaskRunner pid=2224264) 'path': None}, (TaskRunner pid=2224264) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2224264) 'num_workers': 8}, (TaskRunner pid=2224264) 'calculate_log_probs': True, (TaskRunner pid=2224264) 'cudagraph_capture_sizes': None, (TaskRunner pid=2224264) 'data_parallel_size': 1, (TaskRunner pid=2224264) 'disable_log_stats': True, (TaskRunner pid=2224264) 'do_sample': True, (TaskRunner pid=2224264) 'dtype': 'bfloat16', (TaskRunner pid=2224264) 'enable_chunked_prefill': True, (TaskRunner pid=2224264) 'enable_prefix_caching': True, (TaskRunner pid=2224264) 'enable_rollout_routing_replay': False, (TaskRunner pid=2224264) 'enforce_eager': False, (TaskRunner pid=2224264) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2224264) 'expert_parallel_size': 1, (TaskRunner pid=2224264) 'free_cache_engine': True, (TaskRunner pid=2224264) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2224264) 'ignore_eos': False, (TaskRunner pid=2224264) 'layered_summon': False, (TaskRunner pid=2224264) 'load_format': 'dummy', (TaskRunner pid=2224264) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2224264) 'log_prob_micro_batch_size': None, (TaskRunner pid=2224264) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2224264) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2224264) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2224264) 'max_model_len': 10240, (TaskRunner pid=2224264) 'max_num_batched_tokens': 10240, (TaskRunner pid=2224264) 'max_num_seqs': 1024, (TaskRunner pid=2224264) 'mode': 'async', (TaskRunner pid=2224264) 'multi_stage_wake_up': False, (TaskRunner pid=2224264) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'format': 'hermes', (TaskRunner pid=2224264) 'interaction_config_path': None, (TaskRunner pid=2224264) 'max_assistant_turns': None, (TaskRunner pid=2224264) 'max_parallel_calls': 1, (TaskRunner pid=2224264) 'max_tool_response_length': 256, (TaskRunner pid=2224264) 'max_user_turns': None, (TaskRunner pid=2224264) 'num_repeat_rollouts': None, (TaskRunner pid=2224264) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2224264) 'tool_config_path': None, (TaskRunner pid=2224264) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2224264) 'use_inference_chat_template': False}, (TaskRunner pid=2224264) 'n': 8, (TaskRunner pid=2224264) 'name': 'vllm', (TaskRunner pid=2224264) 'over_sample_rate': 0, (TaskRunner pid=2224264) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2224264) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'all_ranks': False, (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'ranks': [], (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'tool': None, (TaskRunner pid=2224264) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2224264) 'analysis': True, (TaskRunner pid=2224264) 'contents': [], (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'level': 'level0'}, (TaskRunner pid=2224264) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'discrete': False}, (TaskRunner pid=2224264) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2224264) 'step_end': None, (TaskRunner pid=2224264) 'step_start': 0}, (TaskRunner pid=2224264) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2224264) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2224264) 'port': 9090, (TaskRunner pid=2224264) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2224264) 'prompt_length': 2048, (TaskRunner pid=2224264) 'quantization': None, (TaskRunner pid=2224264) 'quantization_config_file': None, (TaskRunner pid=2224264) 'response_length': 8192, (TaskRunner pid=2224264) 'scheduling_policy': 'fcfs', (TaskRunner pid=2224264) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2224264) 'skip_rollout': False, (TaskRunner pid=2224264) 'skip_tokenizer_init': True, (TaskRunner pid=2224264) 'temperature': 1.0, (TaskRunner pid=2224264) 'tensor_model_parallel_size': 2, (TaskRunner pid=2224264) 'top_k': -1, (TaskRunner pid=2224264) 'top_p': 1.0, (TaskRunner pid=2224264) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2224264) 'backend': None, (TaskRunner pid=2224264) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2224264) 'token2text': False}, (TaskRunner pid=2224264) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2224264) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2224264) 'do_sample': True, (TaskRunner pid=2224264) 'n': 16, (TaskRunner pid=2224264) 'temperature': 0.6, (TaskRunner pid=2224264) 'top_k': -1, (TaskRunner pid=2224264) 'top_p': 0.95}}}, (TaskRunner pid=2224264) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2224264) 'adv_estimator': 'grpo', (TaskRunner pid=2224264) 'gamma': 1.0, (TaskRunner pid=2224264) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2224264) 'horizon': 10000, (TaskRunner pid=2224264) 'kl_coef': 0.001, (TaskRunner pid=2224264) 'target_kl': 0.1, (TaskRunner pid=2224264) 'type': 'fixed'}, (TaskRunner pid=2224264) 'kl_penalty': 'kl', (TaskRunner pid=2224264) 'lam': 1.0, (TaskRunner pid=2224264) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2224264) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2224264) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2224264) 'loss_type': 'ppo_clip', (TaskRunner pid=2224264) 'rollout_is': 'token', (TaskRunner pid=2224264) 'rollout_is_batch_normalize': False, (TaskRunner pid=2224264) 'rollout_is_threshold': 2.0, (TaskRunner pid=2224264) 'rollout_rs': None, (TaskRunner pid=2224264) 'rollout_rs_threshold': None}, (TaskRunner pid=2224264) 'use_kl_in_reward': False, (TaskRunner pid=2224264) 'use_pf_ppo': False}, (TaskRunner pid=2224264) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2224264) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2224264) 'async_save': False, (TaskRunner pid=2224264) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2224264) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2224264) 'cliprange_value': 0.5, (TaskRunner pid=2224264) 'data_loader_seed': 42, (TaskRunner pid=2224264) 'enable': None, (TaskRunner pid=2224264) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2224264) 'forward_micro_batch_size': None, (TaskRunner pid=2224264) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2224264) 'grad_clip': 1.0, (TaskRunner pid=2224264) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2224264) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2224264) 'enable_activation_offload': False, (TaskRunner pid=2224264) 'enable_gradient_checkpointing': True, (TaskRunner pid=2224264) 'external_lib': None, (TaskRunner pid=2224264) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2224264) 'dtype': 'bfloat16', (TaskRunner pid=2224264) 'entropy_checkpointing': False, (TaskRunner pid=2224264) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2224264) 'forward_only': False, (TaskRunner pid=2224264) 'forward_prefetch': False, (TaskRunner pid=2224264) 'fsdp_size': -1, (TaskRunner pid=2224264) 'full_determinism': False, (TaskRunner pid=2224264) 'model_dtype': 'fp32', (TaskRunner pid=2224264) 'offload_policy': False, (TaskRunner pid=2224264) 'optimizer_offload': False, (TaskRunner pid=2224264) 'param_offload': False, (TaskRunner pid=2224264) 'reshard_after_forward': True, (TaskRunner pid=2224264) 'seed': 42, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_orig_params': False, (TaskRunner pid=2224264) 'use_torch_compile': True, (TaskRunner pid=2224264) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2224264) 'lora_alpha': 16, (TaskRunner pid=2224264) 'lora_rank': 0, (TaskRunner pid=2224264) 'override_config': {}, (TaskRunner pid=2224264) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2224264) 'target_modules': 'all-linear', (TaskRunner pid=2224264) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2224264) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2224264) 'trust_remote_code': True, (TaskRunner pid=2224264) 'use_remove_padding': False, (TaskRunner pid=2224264) 'use_shm': False}, (TaskRunner pid=2224264) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2224264) 'betas': [0.9, 0.999], (TaskRunner pid=2224264) 'clip_grad': 1.0, (TaskRunner pid=2224264) 'lr': 1e-05, (TaskRunner pid=2224264) 'lr_scheduler_type': 'constant', (TaskRunner pid=2224264) 'lr_warmup_steps': -1, (TaskRunner pid=2224264) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2224264) 'min_lr_ratio': 0.0, (TaskRunner pid=2224264) 'num_cycles': 0.5, (TaskRunner pid=2224264) 'optimizer': 'AdamW', (TaskRunner pid=2224264) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2224264) 'override_optimizer_config': None, (TaskRunner pid=2224264) 'total_training_steps': -1, (TaskRunner pid=2224264) 'warmup_style': None, (TaskRunner pid=2224264) 'weight_decay': 0.01}, (TaskRunner pid=2224264) 'ppo_epochs': 1, (TaskRunner pid=2224264) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2224264) 'ppo_micro_batch_size': None, (TaskRunner pid=2224264) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2224264) 'ppo_mini_batch_size': 8, (TaskRunner pid=2224264) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'all_ranks': False, (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'ranks': [], (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'tool': None, (TaskRunner pid=2224264) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2224264) 'analysis': True, (TaskRunner pid=2224264) 'contents': [], (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'level': 'level0'}, (TaskRunner pid=2224264) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'discrete': False}, (TaskRunner pid=2224264) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2224264) 'step_end': None, (TaskRunner pid=2224264) 'step_start': 0}, (TaskRunner pid=2224264) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2224264) 'rollout_n': 8, (TaskRunner pid=2224264) 'shuffle': False, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_dynamic_bsz': False}, (TaskRunner pid=2224264) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2224264) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2224264) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2224264) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2224264) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2224264) 'dataloader_num_workers': 8, (TaskRunner pid=2224264) 'filter_overlong_prompts': True, (TaskRunner pid=2224264) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2224264) 'image_key': 'images', (TaskRunner pid=2224264) 'image_patch_size': 14, (TaskRunner pid=2224264) 'max_prompt_length': 2048, (TaskRunner pid=2224264) 'max_response_length': 8192, (TaskRunner pid=2224264) 'prompt_key': 'prompt', (TaskRunner pid=2224264) 'return_full_prompt': False, (TaskRunner pid=2224264) 'return_multi_modal_inputs': True, (TaskRunner pid=2224264) 'return_raw_chat': True, (TaskRunner pid=2224264) 'return_raw_input_ids': False, (TaskRunner pid=2224264) 'reward_fn_key': 'data_source', (TaskRunner pid=2224264) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2224264) 'seed': None, (TaskRunner pid=2224264) 'shuffle': True, (TaskRunner pid=2224264) 'tokenizer': None, (TaskRunner pid=2224264) 'tool_config_path': None, (TaskRunner pid=2224264) 'train_batch_size': 32, (TaskRunner pid=2224264) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'], (TaskRunner pid=2224264) 'train_max_samples': 3200, (TaskRunner pid=2224264) 'truncation': 'error', (TaskRunner pid=2224264) 'trust_remote_code': True, (TaskRunner pid=2224264) 'use_shm': False, (TaskRunner pid=2224264) 'val_batch_size': None, (TaskRunner pid=2224264) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'], (TaskRunner pid=2224264) 'val_max_samples': -1, (TaskRunner pid=2224264) 'validation_shuffle': False, (TaskRunner pid=2224264) 'video_key': 'videos'}, (TaskRunner pid=2224264) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2224264) 'cuda-memory-usage': 'true', (TaskRunner pid=2224264) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2224264) 'capture-range-end': None, (TaskRunner pid=2224264) 'cuda-graph-trace': 'graph', (TaskRunner pid=2224264) 'cuda-memory-usage': 'true', (TaskRunner pid=2224264) 'kill': 'none', (TaskRunner pid=2224264) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2224264) 'torch_memory': {'context': 'all', (TaskRunner pid=2224264) 'kw_args': {}, (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'stacks': 'all', (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2224264) 'profile_continuous_steps': False, (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'steps': None, (TaskRunner pid=2224264) 'tool': None}, (TaskRunner pid=2224264) 'max_model_len': 10240, (TaskRunner pid=2224264) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_grpo_Qwen3_8B', (TaskRunner pid=2224264) 'include_dashboard': False, (TaskRunner pid=2224264) 'num_cpus': None, (TaskRunner pid=2224264) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2224264) 'TASK': 'datasets/sciknoweval/chemistry', (TaskRunner pid=2224264) 'USER': 'root'}}}, (TaskRunner pid=2224264) 'timeline_json_file': None}, (TaskRunner pid=2224264) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2224264) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2224264) 'name': 'custom_reward_manager', (TaskRunner pid=2224264) 'path': None}, (TaskRunner pid=2224264) 'name': 'naive', (TaskRunner pid=2224264) 'source': 'register'}, (TaskRunner pid=2224264) 'reward_model': {'enable': False, (TaskRunner pid=2224264) 'enable_resource_pool': False, (TaskRunner pid=2224264) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2224264) 'launch_reward_fn_async': False, (TaskRunner pid=2224264) 'max_length': None, (TaskRunner pid=2224264) 'micro_batch_size': None, (TaskRunner pid=2224264) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2224264) 'model': {'external_lib': None, (TaskRunner pid=2224264) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2224264) 'forward_prefetch': False, (TaskRunner pid=2224264) 'fsdp_size': -1, (TaskRunner pid=2224264) 'param_offload': False, (TaskRunner pid=2224264) 'reshard_after_forward': True, (TaskRunner pid=2224264) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2224264) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2224264) 'override_config': {}, (TaskRunner pid=2224264) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2224264) 'trust_remote_code': False, (TaskRunner pid=2224264) 'use_fused_kernels': False, (TaskRunner pid=2224264) 'use_remove_padding': False, (TaskRunner pid=2224264) 'use_shm': False}, (TaskRunner pid=2224264) 'n_gpus_per_node': 8, (TaskRunner pid=2224264) 'nnodes': 0, (TaskRunner pid=2224264) 'num_workers': 1, (TaskRunner pid=2224264) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2224264) 'all_ranks': False, (TaskRunner pid=2224264) 'enable': False, (TaskRunner pid=2224264) 'ranks': [], (TaskRunner pid=2224264) 'save_path': 'outputs/profile', (TaskRunner pid=2224264) 'tool': None, (TaskRunner pid=2224264) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2224264) 'analysis': True, (TaskRunner pid=2224264) 'contents': [], (TaskRunner pid=2224264) 'discrete': False, (TaskRunner pid=2224264) 'level': 'level0'}, (TaskRunner pid=2224264) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2224264) 'discrete': False}, (TaskRunner pid=2224264) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2224264) 'step_end': None, (TaskRunner pid=2224264) 'step_start': 0}, (TaskRunner pid=2224264) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2224264) 'stack_depth': 32, (TaskRunner pid=2224264) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2224264) 'reward_loop_class_name': None, (TaskRunner pid=2224264) 'reward_loop_module_path': None, (TaskRunner pid=2224264) 'reward_loop_source': 'register', (TaskRunner pid=2224264) 'reward_manager': 'naive', (TaskRunner pid=2224264) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2224264) 'cudagraph_capture_sizes': None, (TaskRunner pid=2224264) 'data_parallel_size': 1, (TaskRunner pid=2224264) 'disable_log_stats': True, (TaskRunner pid=2224264) 'dtype': 'bfloat16', (TaskRunner pid=2224264) 'enable_chunked_prefill': True, (TaskRunner pid=2224264) 'enable_prefix_caching': True, (TaskRunner pid=2224264) 'enforce_eager': True, (TaskRunner pid=2224264) 'engine_kwargs': {}, (TaskRunner pid=2224264) 'expert_parallel_size': 1, (TaskRunner pid=2224264) 'free_cache_engine': True, (TaskRunner pid=2224264) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2224264) 'limit_images': None, (TaskRunner pid=2224264) 'load_format': 'auto', (TaskRunner pid=2224264) 'max_model_len': None, (TaskRunner pid=2224264) 'max_num_batched_tokens': 8192, (TaskRunner pid=2224264) 'max_num_seqs': 1024, (TaskRunner pid=2224264) 'name': '???', (TaskRunner pid=2224264) 'prompt_length': 2048, (TaskRunner pid=2224264) 'response_length': 2048, (TaskRunner pid=2224264) 'skip_tokenizer_init': False, (TaskRunner pid=2224264) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2224264) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2224264) 'memory_limit_mb': 1024, (TaskRunner pid=2224264) 'url': None}, (TaskRunner pid=2224264) 'strategy': 'fsdp', (TaskRunner pid=2224264) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2224264) 'use_dynamic_bsz': False, (TaskRunner pid=2224264) 'use_reward_loop': False}, (TaskRunner pid=2224264) 'trainer': {'balance_batch': True, (TaskRunner pid=2224264) 'critic_warmup': 0, (TaskRunner pid=2224264) 'default_hdfs_dir': None, (TaskRunner pid=2224264) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2224264) 'del_local_ckpt_after_load': False, (TaskRunner pid=2224264) 'device': 'cuda', (TaskRunner pid=2224264) 'esi_redundant_time': 0, (TaskRunner pid=2224264) 'experiment_name': 'qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2224264) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2224264) 'log_val_generations': 0, (TaskRunner pid=2224264) 'logger': ['console', 'wandb'], (TaskRunner pid=2224264) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2224264) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2224264) 'n_gpus_per_node': 8, (TaskRunner pid=2224264) 'nnodes': 1, (TaskRunner pid=2224264) 'project_name': 'SDPO-root', (TaskRunner pid=2224264) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2224264) 'resume_from_path': None, (TaskRunner pid=2224264) 'resume_mode': 'auto', (TaskRunner pid=2224264) 'rollout_data_dir': None, (TaskRunner pid=2224264) 'save_freq': 10, (TaskRunner pid=2224264) 'test_freq': 10, (TaskRunner pid=2224264) 'total_epochs': 30, (TaskRunner pid=2224264) 'total_training_steps': 100, (TaskRunner pid=2224264) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2224264) 'val_before_train': False, (TaskRunner pid=2224264) 'val_only': False, (TaskRunner pid=2224264) 'validation_data_dir': None}, (TaskRunner pid=2224264) 'transfer_queue': {'enable': False}, (TaskRunner pid=2224264) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry', (TaskRunner pid=2224264) 'dir': '/users/root/SDPO', (TaskRunner pid=2224264) 'log_dir': '/users/root/output', (TaskRunner pid=2224264) 'task': 'datasets/sciknoweval/chemistry'}} (TaskRunner pid=2224264) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2224264) use_critic=need_critic(config), (TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2224264) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2224264) Using dataset class: RLHFDataset (TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2224264) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2224264) dataset len: 1890 (TaskRunner pid=2224264) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2224264) WARNING:2026-07-02 23:56:38,560:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2224264) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/1890 [00:00 (TaskRunner pid=2224264) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2224264) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2224264) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2224264) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2224264) bind role actor_rollout method wake_up to class .WorkerDict'> (WorkerDict pid=2224678) [W702 23:56:48.468298935 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:38433 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2224673) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2224673) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2224680) [W702 23:56:48.948596036 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:38433 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2224673) Model config after override: Qwen3Config { (WorkerDict pid=2224673) "architectures": [ (WorkerDict pid=2224673) "Qwen3ForCausalLM" (WorkerDict pid=2224673) ], (WorkerDict pid=2224673) "attention_bias": false, (WorkerDict pid=2224673) "attention_dropout": 0.0, (WorkerDict pid=2224673) "dtype": "bfloat16", (WorkerDict pid=2224673) "eos_token_id": 151645, (WorkerDict pid=2224673) "head_dim": 128, (WorkerDict pid=2224673) "hidden_act": "silu", (WorkerDict pid=2224673) "hidden_size": 4096, (WorkerDict pid=2224673) "initializer_range": 0.02, (WorkerDict pid=2224673) "intermediate_size": 12288, (WorkerDict pid=2224673) "layer_types": [ (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention", (WorkerDict pid=2224673) "full_attention" (WorkerDict pid=2224673) ], (WorkerDict pid=2224673) "max_position_embeddings": 40960, (WorkerDict pid=2224673) "max_window_layers": 36, (WorkerDict pid=2224673) "model_type": "qwen3", (WorkerDict pid=2224673) "num_attention_heads": 32, (WorkerDict pid=2224673) "num_hidden_layers": 36, (WorkerDict pid=2224673) "num_key_value_heads": 8, (WorkerDict pid=2224673) "pad_token_id": 151643, (WorkerDict pid=2224673) "rms_norm_eps": 1e-06, (WorkerDict pid=2224673) "rope_scaling": null, (WorkerDict pid=2224673) "rope_theta": 1000000, (WorkerDict pid=2224673) "sliding_window": null, (WorkerDict pid=2224673) "tie_word_embeddings": false, (WorkerDict pid=2224673) "transformers_version": "4.57.1", (WorkerDict pid=2224673) "use_cache": true, (WorkerDict pid=2224673) "use_sliding_window": false, (WorkerDict pid=2224673) "vocab_size": 151936 (WorkerDict pid=2224673) } (WorkerDict pid=2224673) (WorkerDict pid=2224673) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2224673) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2224673) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2224673) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2224673) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2224674) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2224674) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2224680) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.93s/it] [repeated 15x across cluster] (WorkerDict pid=2224680) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.22s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.62s/it] [repeated 7x across cluster] (WorkerDict pid=2224673) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2224673) Actor use_remove_padding=True (WorkerDict pid=2224673) Actor use_fused_kernels=False (WorkerDict pid=2224673) Actor use_prefix_grouper=False (WorkerDict pid=2224680) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2224680) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2224674) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2224674) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2224674) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2224673) warnings.warn( (WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2224673) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2224673) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2224673) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2224680) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2224680) warnings.warn( [repeated 7x across cluster] (TaskRunner pid=2224264) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2224264) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (vLLMHttpServer pid=2225569) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2225569) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2225570) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2225570) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2225569) ['serve', (vLLMHttpServer pid=2225569) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2225569) '--dtype', (vLLMHttpServer pid=2225569) 'bfloat16', (vLLMHttpServer pid=2225569) '--load_format', (vLLMHttpServer pid=2225569) 'dummy', (vLLMHttpServer pid=2225569) '--trust_remote_code', (vLLMHttpServer pid=2225569) '--max_model_len', (vLLMHttpServer pid=2225569) '40960', (vLLMHttpServer pid=2225569) '--max_num_seqs', (vLLMHttpServer pid=2225569) '1024', (vLLMHttpServer pid=2225569) '--enable_chunked_prefill', (vLLMHttpServer pid=2225569) '--max_num_batched_tokens', (vLLMHttpServer pid=2225569) '10240', (vLLMHttpServer pid=2225569) '--enable_prefix_caching', (vLLMHttpServer pid=2225569) '--enable_sleep_mode', (vLLMHttpServer pid=2225569) '--logprobs_mode', (vLLMHttpServer pid=2225569) 'processed_logprobs', (vLLMHttpServer pid=2225569) '--disable_custom_all_reduce', (vLLMHttpServer pid=2225569) '--gpu_memory_utilization', (vLLMHttpServer pid=2225569) '0.8', (vLLMHttpServer pid=2225569) '--disable_log_stats', (vLLMHttpServer pid=2225569) '--tensor_parallel_size', (vLLMHttpServer pid=2225569) '2', (vLLMHttpServer pid=2225569) '--seed', (vLLMHttpServer pid=2225569) '0', (vLLMHttpServer pid=2225569) '--override_generation_config', (vLLMHttpServer pid=2225569) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2225569) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2225569) '--hf_overrides', (vLLMHttpServer pid=2225569) '{}', (vLLMHttpServer pid=2225569) '--scheduling_policy', (vLLMHttpServer pid=2225569) 'fcfs'] (WorkerDict pid=2224680) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2225569) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2225569) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2225569) WARNING 07-02 23:57:38 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2224673) WARNING 07-02 23:57:46 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2225571) WARNING 07-02 23:57:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2224675) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2224677) 2026-07-02 23:58:01,108 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (vLLMHttpServer pid=2225571) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2225571) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2225571) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2225571) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2224678) 2026-07-02 23:58:01,122 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2224673) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and identify what functional groups or atoms are present that must be introduced in the synthesis. (TaskRunner pid=2224264) (TaskRunner pid=2224264) The product has the following key features: (TaskRunner pid=2224264) - A nitrogen-containing ring (indole-like structure) (TaskRunner pid=2224264) - A chlorine atom attached to a benzene ring (TaskRunner pid=2224264) - A trifluoromethyl group (C(F)(F)F) attached to a benzene ring (TaskRunner pid=2224264) - A substituent group that includes a nitrogen atom bonded to a carbon chain (TaskRunner pid=2224264) (TaskRunner pid=2224264) We are given four options (A, B, C, D) and need to identify which one is the only correct reactant that can be used in the synthesis of this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Let’s analyze each option: (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option A:** (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) This option includes the same ring structure as the product, but it has an extra oxygen and a double bond. It seems to be a more complex derivative and may not be a direct reactant for the synthesis of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option B:** (TaskRunner pid=2224264) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) This option has a chlorine atom attached to a carbon that is also bonded to a nitrogen. It includes a ring structure and a chlorine atom, but it does not match the exact structure of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option C:** (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) This option includes a trifluoromethyl group (C(F)(F)F), a chlorine atom, and a ring structure. However, it has an oxygen atom attached to a carbon that is also bonded to a ring. This structure is more complex and seems to be a derivative of the product, not a direct reactant. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option D:** (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) This option contains a ring structure, a trifluoromethyl group (C(F)(F)F), and a carbon chain with a double bond. It matches the necessary components of the product, including the trifluoromethyl group and the ring structure. It is the only option that contains the trifluoromethyl group and a carbon chain that could be used to build the product's structure. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Thus, the only correct reactant that matches the necessary components of the product is **Option D**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) D (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 0.0 (TaskRunner pid=2224264) [acc] 0.0 (TaskRunner pid=2224264) [pred] D (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2224264) step:10 - global_seqlen/min:20418 - global_seqlen/max:28068 - global_seqlen/minmax_diff:7650 - global_seqlen/balanced_min:23873 - global_seqlen/balanced_max:23970 - global_seqlen/mean:23892.875 - actor/entropy:0.5462746024131775 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977370262146 - training/rollout_probs_diff_mean:0.004709707107394934 - training/rollout_probs_diff_std:0.009677772410213947 - training/rollout_actor_probs_pearson_corr:0.9994706511497498 - rollout_corr/rollout_is_mean:0.9998794198036194 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.18444459582679e-06 - rollout_corr/rollout_is_max:1.622496247291565 - rollout_corr/rollout_is_min:0.4166650176048279 - rollout_corr/rollout_is_std:0.034952208399772644 - rollout_corr/rollout_is_eff_sample_size:0.9987795959879308 - rollout_corr/rollout_is_seq_mean:0.9998565912246704 - rollout_corr/rollout_is_seq_std:0.0018025203607976437 - rollout_corr/rollout_is_seq_max:1.0046859979629517 - rollout_corr/rollout_is_seq_min:0.9944931268692017 - rollout_corr/rollout_is_seq_max_deviation:0.00550687313079834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7906524906866252) - rollout_corr/training_log_ppl:np.float64(0.5681954241008498) - rollout_corr/kl:np.float64(0.000816460652924178) - rollout_corr/k3_kl:np.float64(0.0006859729254529157) - rollout_corr/rollout_ppl:np.float64(1.789125895127654) - rollout_corr/rollout_log_ppl:np.float64(0.5673789661959745) - rollout_corr/log_ppl_diff:np.float64(0.0008164579048752785) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015062049496918917) - rollout_corr/log_ppl_diff_max:np.float64(0.007841110229492188) - rollout_corr/log_ppl_diff_min:np.float64(-0.004483282566070557) - rollout_corr/ppl_ratio:np.float64(1.0008185552433133) - rollout_corr/chi2_token:np.float64(0.0011271920520812273) - rollout_corr/chi2_seq:np.float64(0.35812679631635547) - actor/pg_loss:np.float64(-9.637698531150818e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003609476652854937) - actor/ppo_kl:np.float64(1.5247296355269668e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38365066051483154) - perf/mfu/actor:np.float64(0.09690410487686144) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(99.56629180908203) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.42410714285714285) - val-aux/sciknoweval/reward/std@16:np.float64(0.3064350931481341) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5519095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2843083193630651) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.2967190476190476) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.23547573506551264) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.42388571428571425) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3061951415792409) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6715952380952382) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.229658540263386) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.21003333333333332) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1505707856063924) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.43492857142857144) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.25799645877611216) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.765652380952381) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1562366777957225) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.1566142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09130497085868691) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.44051904761904753) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.20808730912319104) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8246428571428572) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0841254257162172) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.12242857142857143) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05650288713800214) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.44238571428571427) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1575830385066246) - val-aux/sciknoweval/score/mean@16:np.float64(0.42410714285714285) - val-aux/sciknoweval/score/std@16:np.float64(0.3064350931481341) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5519095238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2843083193630651) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.2967190476190476) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.23547573506551264) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.42388571428571425) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3061951415792409) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6715952380952382) - val-aux/sciknoweval/score/best@4/std:np.float64(0.229658540263386) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.21003333333333332) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1505707856063924) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.43492857142857144) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.25799645877611216) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.765652380952381) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1562366777957225) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.1566142857142857) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09130497085868691) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.44051904761904753) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.20808730912319104) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8246428571428572) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0841254257162172) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.12242857142857143) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05650288713800214) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.44238571428571427) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1575830385066246) - val-core/sciknoweval/acc/mean@16:np.float64(0.42410714285714285) - val-aux/sciknoweval/acc/std@16:np.float64(0.3064350931481341) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5519095238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2843083193630651) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.2967190476190476) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.23547573506551264) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.42388571428571425) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3061951415792409) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6715952380952382) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.229658540263386) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.21003333333333332) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1505707856063924) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.43492857142857144) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.25799645877611216) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.765652380952381) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1562366777957225) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.1566142857142857) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09130497085868691) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.44051904761904753) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.20808730912319104) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8246428571428572) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0841254257162172) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.12242857142857143) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05650288713800214) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.44238571428571427) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1575830385066246) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9955357142857143) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.017290104224140252) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.999647619047619) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.004935643403728794) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9915619047619048) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.02303424670029897) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9958761904761906) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.016639024114376826) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9841142857142856) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.02968846984091712) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9983333333333334) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.010724673876175908) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9717047619047618) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.03493004126518059) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9995761904761905) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005142725136555151) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9547523809523809) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03440111225850822) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999714285714284) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0007119567591960759) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003161241766065359 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003161241766065359 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:477.27734375 - response_length/max:1573.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:477.27734375 - response_length_non_aborted/max:1573.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.375 - prompt_length/max:701.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015650317072868347 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0671779178082943) - timing_s/agent_loop/generate_sequences/max:np.float64(11.94185172766447) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.342340253017028) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.94185172766447) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:1573 - timing_s/gen:18.750355437397957 - timing_s/reward:0.06060847267508507 - timing_s/old_log_prob:3.169859381392598 - timing_s/adv:0.15408215299248695 - timing_s/update_actor:12.863298328593373 - timing_s/step:35.09146223217249 - timing_s/testing:148.6573334839195 - timing_s/save_checkpoint:12.811893673613667 - timing_s/stop_profile:0.00013393908739089966 - timing_per_token_ms/adv:0.0008061093160224908 - timing_per_token_ms/gen:0.1534612461422453 - timing_per_token_ms/update_actor:0.0672967272073441 - perf/total_num_tokens:191143 - perf/time_per_step:35.09146223217249 - perf/throughput:680.8743061750952 (TaskRunner pid=2224264) Training Progress: 10%|█ | 10/100 [09:38<2:10:46, 87.18s/it] (TaskRunner pid=2224264) step:11 - global_seqlen/min:20294 - global_seqlen/max:26725 - global_seqlen/minmax_diff:6431 - global_seqlen/balanced_min:24058 - global_seqlen/balanced_max:24076 - global_seqlen/mean:24070.25 - actor/entropy:0.5913231372833252 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23963481187820435 - training/rollout_probs_diff_mean:0.004752492066472769 - training/rollout_probs_diff_std:0.009357922710478306 - training/rollout_actor_probs_pearson_corr:0.9995139241218567 - rollout_corr/rollout_is_mean:1.0000724792480469 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.7242271496797912e-05 - rollout_corr/rollout_is_max:1.639595627784729 - rollout_corr/rollout_is_min:0.41323140263557434 - rollout_corr/rollout_is_std:0.03530850633978844 - rollout_corr/rollout_is_eff_sample_size:0.9987550993777402 - rollout_corr/rollout_is_seq_mean:1.0001287460327148 - rollout_corr/rollout_is_seq_std:0.001979757333174348 - rollout_corr/rollout_is_seq_max:1.0079063177108765 - rollout_corr/rollout_is_seq_min:0.9954867959022522 - rollout_corr/rollout_is_seq_max_deviation:0.007906317710876465 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8813741342164576) - rollout_corr/training_log_ppl:np.float64(0.6172284478088841) - rollout_corr/kl:np.float64(0.0006792962190935725) - rollout_corr/k3_kl:np.float64(0.0007297945928712579) - rollout_corr/rollout_ppl:np.float64(1.8801453905180097) - rollout_corr/rollout_log_ppl:np.float64(0.6165491492720321) - rollout_corr/log_ppl_diff:np.float64(0.0006792985368520021) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017337198369204998) - rollout_corr/log_ppl_diff_max:np.float64(0.00989621877670288) - rollout_corr/log_ppl_diff_min:np.float64(-0.006999552249908447) - rollout_corr/ppl_ratio:np.float64(1.000681943492964) - rollout_corr/chi2_token:np.float64(0.0015983630437403917) - rollout_corr/chi2_seq:np.float64(0.8742716582491994) - actor/pg_loss:np.float64(-0.00019931478891521692) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003874804629049322) - actor/ppo_kl:np.float64(0.000144808165869037) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4361106902360916) - perf/mfu/actor:np.float64(0.09751332153344734) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(105.7747859954834) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005713657476007938 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005713657476007938 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.1015625 - response_length/max:1181.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.1015625 - response_length_non_aborted/max:1181.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.09375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.448345422744751e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.323524136096239) - timing_s/agent_loop/generate_sequences/max:np.float64(8.916998555883765) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.45590021499811) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.916998555883765) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:319 - timing_s/agent_loop/slowest/response_length:1181 - timing_s/gen:15.152160251513124 - timing_s/reward:0.057105498388409615 - timing_s/old_log_prob:3.2711402103304863 - timing_s/adv:0.15576941706240177 - timing_s/update_actor:13.000678863376379 - timing_s/step:31.738592451438308 - timing_s/stop_profile:0.00011883489787578583 - timing_per_token_ms/adv:0.0008089312380552849 - timing_per_token_ms/gen:0.13062882779724058 - timing_per_token_ms/update_actor:0.0675142492463538 - perf/total_num_tokens:192562 - perf/time_per_step:31.738592451438308 - perf/throughput:758.3905945680714 (TaskRunner pid=2224264) Training Progress: 11%|█ | 11/100 [10:09<1:44:10, 70.23s/it] (TaskRunner pid=2224264) step:12 - global_seqlen/min:22279 - global_seqlen/max:28767 - global_seqlen/minmax_diff:6488 - global_seqlen/balanced_min:25313 - global_seqlen/balanced_max:25329 - global_seqlen/mean:25325.625 - actor/entropy:0.5468142628669739 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30783188343048096 - training/rollout_probs_diff_mean:0.004490185528993607 - training/rollout_probs_diff_std:0.009273474104702473 - training/rollout_actor_probs_pearson_corr:0.9995062351226807 - rollout_corr/rollout_is_mean:1.0000845193862915 - rollout_corr/rollout_is_ratio_fraction_high:7.435773568431614e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.075246810913086 - rollout_corr/rollout_is_min:0.5100579261779785 - rollout_corr/rollout_is_std:0.03405739739537239 - rollout_corr/rollout_is_eff_sample_size:0.9988416753032981 - rollout_corr/rollout_is_seq_mean:1.0000801086425781 - rollout_corr/rollout_is_seq_std:0.0016866717487573624 - rollout_corr/rollout_is_seq_max:1.0045456886291504 - rollout_corr/rollout_is_seq_min:0.9936270713806152 - rollout_corr/rollout_is_seq_max_deviation:0.006372928619384766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7922574547119439) - rollout_corr/training_log_ppl:np.float64(0.5712134827044792) - rollout_corr/kl:np.float64(0.0005522373108419742) - rollout_corr/k3_kl:np.float64(0.0006720679998579726) - rollout_corr/rollout_ppl:np.float64(1.7912592180073261) - rollout_corr/rollout_log_ppl:np.float64(0.5706612397334538) - rollout_corr/log_ppl_diff:np.float64(0.0005522429710254073) - rollout_corr/log_ppl_abs_diff:np.float64(0.001352141029201448) - rollout_corr/log_ppl_diff_max:np.float64(0.007192671298980713) - rollout_corr/log_ppl_diff_min:np.float64(-0.003842592239379883) - rollout_corr/ppl_ratio:np.float64(1.0005537504330277) - rollout_corr/chi2_token:np.float64(0.0016048443503677845) - rollout_corr/chi2_seq:np.float64(0.7803806539159268) - actor/pg_loss:np.float64(-5.2671064622700214e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002636122353578685) - actor/ppo_kl:np.float64(2.2634695866941e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3025449775159359) - perf/mfu/actor:np.float64(0.09925995879832478) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.3304214477539) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003494813572615385 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003494813572615385 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:525.33203125 - response_length/max:1340.0 - response_length/min:219.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:525.33203125 - response_length_non_aborted/max:1340.0 - response_length_non_aborted/min:219.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.09375 - prompt_length/max:665.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001250188797712326 - timing_s/agent_loop/generate_sequences/min:np.float64(2.742388065904379) - timing_s/agent_loop/generate_sequences/max:np.float64(11.409071335569024) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.0465768424546695) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.409071335569024) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:1340 - timing_s/gen:18.710343282669783 - timing_s/reward:0.06669484823942184 - timing_s/old_log_prob:3.1704280357807875 - timing_s/adv:0.14182846806943417 - timing_s/update_actor:13.454730957746506 - timing_s/step:35.63072715699673 - timing_s/stop_profile:0.0001154579222202301 - timing_per_token_ms/adv:0.0007000245209616454 - timing_per_token_ms/gen:0.1391258748757838 - timing_per_token_ms/update_actor:0.06640868170946672 - perf/total_num_tokens:202605 - perf/time_per_step:35.63072715699673 - perf/throughput:710.7804701377491 (TaskRunner pid=2224264) Training Progress: 12%|█▏ | 12/100 [10:45<1:27:35, 59.72s/it] (TaskRunner pid=2224264) step:13 - global_seqlen/min:22139 - global_seqlen/max:26070 - global_seqlen/minmax_diff:3931 - global_seqlen/balanced_min:23703 - global_seqlen/balanced_max:23955 - global_seqlen/mean:23768.875 - actor/entropy:0.6082448959350586 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8117851614952087 - training/rollout_probs_diff_mean:0.004782064817845821 - training/rollout_probs_diff_std:0.009643850848078728 - training/rollout_actor_probs_pearson_corr:0.9995208978652954 - rollout_corr/rollout_is_mean:0.9998765587806702 - rollout_corr/rollout_is_ratio_fraction_high:8.387642992602196e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.677528598520439e-05 - rollout_corr/rollout_is_max:2.2560038566589355 - rollout_corr/rollout_is_min:0.1834239512681961 - rollout_corr/rollout_is_std:0.03520030155777931 - rollout_corr/rollout_is_eff_sample_size:0.9987622341887205 - rollout_corr/rollout_is_seq_mean:0.9998448491096497 - rollout_corr/rollout_is_seq_std:0.0017469775630161166 - rollout_corr/rollout_is_seq_max:1.0048439502716064 - rollout_corr/rollout_is_seq_min:0.9949353933334351 - rollout_corr/rollout_is_seq_max_deviation:0.005064606666564941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8870359733700752) - rollout_corr/training_log_ppl:np.float64(0.6238076707813889) - rollout_corr/kl:np.float64(0.0008689244228179405) - rollout_corr/k3_kl:np.float64(0.0006925447481762603) - rollout_corr/rollout_ppl:np.float64(1.8853362454101443) - rollout_corr/rollout_log_ppl:np.float64(0.6229387456551194) - rollout_corr/log_ppl_diff:np.float64(0.0008689251262694597) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016076937317848206) - rollout_corr/log_ppl_diff_max:np.float64(0.007483184337615967) - rollout_corr/log_ppl_diff_min:np.float64(-0.0051155686378479) - rollout_corr/ppl_ratio:np.float64(1.00087097636424) - rollout_corr/chi2_token:np.float64(0.001052779145538807) - rollout_corr/chi2_seq:np.float64(0.7514310674741864) - actor/pg_loss:np.float64(7.94560182839632e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002786977966025006) - actor/ppo_kl:np.float64(6.474458803040761e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.397162064909935) - perf/mfu/actor:np.float64(0.09482056304606831) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.62731170654297) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0025341168511658907 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0025341168511658907 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:465.71484375 - response_length/max:1517.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:465.71484375 - response_length_non_aborted/max:1517.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:612.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010670162737369537 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1883809324353933) - timing_s/agent_loop/generate_sequences/max:np.float64(11.769639683887362) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.302525144354149) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.769639683887362) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:1517 - timing_s/gen:18.91310457699001 - timing_s/reward:0.0668586827814579 - timing_s/old_log_prob:3.0675366707146168 - timing_s/adv:0.13432571291923523 - timing_s/update_actor:13.279257319867611 - timing_s/step:35.54763768427074 - timing_s/stop_profile:0.00011998414993286133 - timing_per_token_ms/adv:0.0007064160215788254 - timing_per_token_ms/gen:0.15863637533856728 - timing_per_token_ms/update_actor:0.06983532729182393 - perf/total_num_tokens:190151 - perf/time_per_step:35.54763768427074 - perf/throughput:668.6485107987174 (TaskRunner pid=2224264) Training Progress: 13%|█▎ | 13/100 [11:21<1:15:59, 52.41s/it] (TaskRunner pid=2224264) step:14 - global_seqlen/min:21108 - global_seqlen/max:28388 - global_seqlen/minmax_diff:7280 - global_seqlen/balanced_min:24288 - global_seqlen/balanced_max:24367 - global_seqlen/mean:24304.0 - actor/entropy:0.5927061438560486 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932337522506714 - training/rollout_probs_diff_mean:0.004692592658102512 - training/rollout_probs_diff_std:0.00925843883305788 - training/rollout_actor_probs_pearson_corr:0.9995315670967102 - rollout_corr/rollout_is_mean:0.9999940991401672 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.629692554473877 - rollout_corr/rollout_is_min:0.5129373669624329 - rollout_corr/rollout_is_std:0.03493344411253929 - rollout_corr/rollout_is_eff_sample_size:0.998781141930825 - rollout_corr/rollout_is_seq_mean:0.9999703168869019 - rollout_corr/rollout_is_seq_std:0.0018831780180335045 - rollout_corr/rollout_is_seq_max:1.007323145866394 - rollout_corr/rollout_is_seq_min:0.9952291250228882 - rollout_corr/rollout_is_seq_max_deviation:0.007323145866394043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8519614487886429) - rollout_corr/training_log_ppl:np.float64(0.6055547567666508) - rollout_corr/kl:np.float64(0.0006922458704323731) - rollout_corr/k3_kl:np.float64(0.0006847217604217803) - rollout_corr/rollout_ppl:np.float64(1.850690824445337) - rollout_corr/rollout_log_ppl:np.float64(0.604862505744677) - rollout_corr/log_ppl_diff:np.float64(0.0006922510219737887) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016473830910399556) - rollout_corr/log_ppl_diff_max:np.float64(0.005706071853637695) - rollout_corr/log_ppl_diff_min:np.float64(-0.006285727024078369) - rollout_corr/ppl_ratio:np.float64(1.00069451588206) - rollout_corr/chi2_token:np.float64(0.0013829697854816914) - rollout_corr/chi2_seq:np.float64(1.670509943505749) - actor/pg_loss:np.float64(4.7211069613695145e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002791377244193427) - actor/ppo_kl:np.float64(2.9361409389139226e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.39511965960264206) - perf/mfu/actor:np.float64(0.09905766411058545) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.6309814453125) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.39453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.39453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001839753007516265 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.001839753007516265 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:492.0625 - response_length/max:1506.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:492.0625 - response_length_non_aborted/max:1506.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.4375 - prompt_length/max:666.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010376237332820892 - timing_s/agent_loop/generate_sequences/min:np.float64(2.323818039149046) - timing_s/agent_loop/generate_sequences/max:np.float64(11.8414571788162) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.66649279979174) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.8414571788162) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:1506 - timing_s/gen:18.678927469998598 - timing_s/reward:0.09569268859922886 - timing_s/old_log_prob:3.05848809145391 - timing_s/adv:0.16165275499224663 - timing_s/update_actor:12.951634736731648 - timing_s/step:35.03177675232291 - timing_s/stop_profile:0.00011714547872543335 - timing_per_token_ms/adv:0.0008314102359295107 - timing_per_token_ms/gen:0.1482831153149895 - timing_per_token_ms/update_actor:0.06661267042838447 - perf/total_num_tokens:194432 - perf/time_per_step:35.03177675232291 - perf/throughput:693.7701211055027 (TaskRunner pid=2224264) Training Progress: 14%|█▍ | 14/100 [11:56<1:07:37, 47.17s/it] (TaskRunner pid=2224264) step:15 - global_seqlen/min:22401 - global_seqlen/max:28174 - global_seqlen/minmax_diff:5773 - global_seqlen/balanced_min:25422 - global_seqlen/balanced_max:25432 - global_seqlen/mean:25428.5 - actor/entropy:0.644464373588562 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21453610062599182 - training/rollout_probs_diff_mean:0.004607322160154581 - training/rollout_probs_diff_std:0.008982905186712742 - training/rollout_actor_probs_pearson_corr:0.9995591640472412 - rollout_corr/rollout_is_mean:0.9999948143959045 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.693491170357447e-06 - rollout_corr/rollout_is_max:1.5525251626968384 - rollout_corr/rollout_is_min:0.4232032895088196 - rollout_corr/rollout_is_std:0.0347040556371212 - rollout_corr/rollout_is_eff_sample_size:0.9987969583907628 - rollout_corr/rollout_is_seq_mean:1.000001072883606 - rollout_corr/rollout_is_seq_std:0.001659586327150464 - rollout_corr/rollout_is_seq_max:1.0077872276306152 - rollout_corr/rollout_is_seq_min:0.994013249874115 - rollout_corr/rollout_is_seq_max_deviation:0.007787227630615234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.001432800665498) - rollout_corr/training_log_ppl:np.float64(0.6742313308641315) - rollout_corr/kl:np.float64(0.0006003224210004987) - rollout_corr/k3_kl:np.float64(0.0006980515448731239) - rollout_corr/rollout_ppl:np.float64(2.0002612890675664) - rollout_corr/rollout_log_ppl:np.float64(0.673631009995006) - rollout_corr/log_ppl_diff:np.float64(0.0006003208691254258) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014080120017752051) - rollout_corr/log_ppl_diff_max:np.float64(0.00482177734375) - rollout_corr/log_ppl_diff_min:np.float64(-0.007164597511291504) - rollout_corr/ppl_ratio:np.float64(1.0006020197179168) - rollout_corr/chi2_token:np.float64(0.001624844502657652) - rollout_corr/chi2_seq:np.float64(1.5461109012831002) - actor/pg_loss:np.float64(0.00015092291869223118) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017774101524992147) - actor/ppo_kl:np.float64(-3.1671785084341764e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3353484757244587) - perf/mfu/actor:np.float64(0.10308922077230062) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.59129333496094) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006222110823728144 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006222110823728144 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:507.734375 - response_length/max:1102.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:507.734375 - response_length_non_aborted/max:1102.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.90625 - prompt_length/max:494.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000116739422082901 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3703849464654922) - timing_s/agent_loop/generate_sequences/max:np.float64(9.983016459271312) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.813961479689169) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.983016459271312) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:1093 - timing_s/gen:17.158777246251702 - timing_s/reward:0.06794460862874985 - timing_s/old_log_prob:3.08405601978302 - timing_s/adv:0.12691123597323895 - timing_s/update_actor:12.908528231084347 - timing_s/step:33.43038131669164 - timing_s/stop_profile:2.7742236852645874e-05 - timing_per_token_ms/adv:0.0006238631652144196 - timing_per_token_ms/gen:0.13201090357171644 - timing_per_token_ms/update_actor:0.06345502207702158 - perf/total_num_tokens:203428 - perf/time_per_step:33.43038131669164 - perf/throughput:760.6404413731191 (TaskRunner pid=2224264) Training Progress: 15%|█▌ | 15/100 [12:29<1:00:58, 43.04s/it] (TaskRunner pid=2224264) step:16 - global_seqlen/min:21410 - global_seqlen/max:32573 - global_seqlen/minmax_diff:11163 - global_seqlen/balanced_min:26094 - global_seqlen/balanced_max:26108 - global_seqlen/mean:26103.625 - actor/entropy:0.6026089787483215 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1841939091682434 - training/rollout_probs_diff_mean:0.004535277374088764 - training/rollout_probs_diff_std:0.008988428860902786 - training/rollout_actor_probs_pearson_corr:0.9995559453964233 - rollout_corr/rollout_is_mean:1.000097632408142 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6370495557785034 - rollout_corr/rollout_is_min:0.5366882681846619 - rollout_corr/rollout_is_std:0.034516338258981705 - rollout_corr/rollout_is_eff_sample_size:0.9988102779034601 - rollout_corr/rollout_is_seq_mean:1.0000861883163452 - rollout_corr/rollout_is_seq_std:0.00168060592841357 - rollout_corr/rollout_is_seq_max:1.0056310892105103 - rollout_corr/rollout_is_seq_min:0.9956023097038269 - rollout_corr/rollout_is_seq_max_deviation:0.005631089210510254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8813290460966527) - rollout_corr/training_log_ppl:np.float64(0.6181044758995995) - rollout_corr/kl:np.float64(0.0006577574873949388) - rollout_corr/k3_kl:np.float64(0.0006780388332572329) - rollout_corr/rollout_ppl:np.float64(1.880119965877384) - rollout_corr/rollout_log_ppl:np.float64(0.617446719086729) - rollout_corr/log_ppl_diff:np.float64(0.0006577568128705025) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014887393917888403) - rollout_corr/log_ppl_diff_max:np.float64(0.009718984365463257) - rollout_corr/log_ppl_diff_min:np.float64(-0.005425512790679932) - rollout_corr/ppl_ratio:np.float64(1.0006597633473575) - rollout_corr/chi2_token:np.float64(0.0014098691754043102) - rollout_corr/chi2_seq:np.float64(2.303532388759777) - actor/pg_loss:np.float64(-0.00024759862571954727) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005523854347302404) - actor/ppo_kl:np.float64(0.00011259556639942048) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4002230614423752) - perf/mfu/actor:np.float64(0.10440002990848575) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.71824264526367) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004673639312386513 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004673639312386513 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:535.01953125 - response_length/max:1208.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:535.01953125 - response_length_non_aborted/max:1208.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.71875 - prompt_length/max:859.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.0385330319404602e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0918461363762617) - timing_s/agent_loop/generate_sequences/max:np.float64(10.83283576183021) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.100973397937196) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.83283576183021) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:1163 - timing_s/gen:17.82087113149464 - timing_s/reward:0.06884737312793732 - timing_s/old_log_prob:3.0925582088530064 - timing_s/adv:0.11866064369678497 - timing_s/update_actor:13.111870618537068 - timing_s/step:34.29907225072384 - timing_s/stop_profile:0.00011087954044342041 - timing_per_token_ms/adv:0.0005682191826651709 - timing_per_token_ms/gen:0.1301125917679308 - timing_per_token_ms/update_actor:0.0627875947236115 - perf/total_num_tokens:208829 - perf/time_per_step:34.29907225072384 - perf/throughput:761.059214931072 (TaskRunner pid=2224264) Training Progress: 16%|█▌ | 16/100 [13:04<56:35, 40.42s/it] (TaskRunner pid=2224264) step:17 - global_seqlen/min:25658 - global_seqlen/max:31580 - global_seqlen/minmax_diff:5922 - global_seqlen/balanced_min:28427 - global_seqlen/balanced_max:28619 - global_seqlen/mean:28480.125 - actor/entropy:0.6307497024536133 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24447917938232422 - training/rollout_probs_diff_mean:0.004340663086622953 - training/rollout_probs_diff_std:0.008799507282674313 - training/rollout_actor_probs_pearson_corr:0.9995622634887695 - rollout_corr/rollout_is_mean:1.0000449419021606 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.015722076175734e-06 - rollout_corr/rollout_is_max:1.5821566581726074 - rollout_corr/rollout_is_min:0.4949151575565338 - rollout_corr/rollout_is_std:0.03397679701447487 - rollout_corr/rollout_is_eff_sample_size:0.998847027329727 - rollout_corr/rollout_is_seq_mean:1.0000169277191162 - rollout_corr/rollout_is_seq_std:0.0016468465328216553 - rollout_corr/rollout_is_seq_max:1.006962537765503 - rollout_corr/rollout_is_seq_min:0.9948194622993469 - rollout_corr/rollout_is_seq_max_deviation:0.00696253776550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0382777974009514) - rollout_corr/training_log_ppl:np.float64(0.6885525062098168) - rollout_corr/kl:np.float64(0.0007926652352825414) - rollout_corr/k3_kl:np.float64(0.000665645356832556) - rollout_corr/rollout_ppl:np.float64(2.0366465840488672) - rollout_corr/rollout_log_ppl:np.float64(0.687759839813225) - rollout_corr/log_ppl_diff:np.float64(0.0007926663965918124) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014466078137047589) - rollout_corr/log_ppl_diff_max:np.float64(0.0072629451751708984) - rollout_corr/log_ppl_diff_min:np.float64(-0.004837632179260254) - rollout_corr/ppl_ratio:np.float64(1.0007944458629936) - rollout_corr/chi2_token:np.float64(0.0010911454446613789) - rollout_corr/chi2_seq:np.float64(0.8875894017983228) - actor/pg_loss:np.float64(4.744785837829113e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019340229709996493) - actor/ppo_kl:np.float64(0.00017584012368843105) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3084394074976444) - perf/mfu/actor:np.float64(0.1127631964826109) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.56375503540039) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.3984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005952840205281973 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005952840205281973 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:556.78515625 - response_length/max:1943.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:556.78515625 - response_length_non_aborted/max:1943.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:333.21875 - prompt_length/max:617.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012286566197872162 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8024302516132593) - timing_s/agent_loop/generate_sequences/max:np.float64(15.304179856553674) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.37867253353761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.304179856553674) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1943 - timing_s/gen:21.94722494482994 - timing_s/reward:0.07538571581244469 - timing_s/old_log_prob:3.1670491863042116 - timing_s/adv:0.15483485721051693 - timing_s/update_actor:13.154093807563186 - timing_s/step:38.58247912488878 - timing_s/stop_profile:0.00011283159255981445 - timing_per_token_ms/adv:0.0006795741644853952 - timing_per_token_ms/gen:0.15397563400962516 - timing_per_token_ms/update_actor:0.057733655521013276 - perf/total_num_tokens:227841 - perf/time_per_step:38.58247912488878 - perf/throughput:738.1621307384586 (TaskRunner pid=2224264) Training Progress: 17%|█▋ | 17/100 [13:42<55:10, 39.88s/it] (TaskRunner pid=2224264) step:18 - global_seqlen/min:21346 - global_seqlen/max:28688 - global_seqlen/minmax_diff:7342 - global_seqlen/balanced_min:24408 - global_seqlen/balanced_max:24420 - global_seqlen/mean:24413.25 - actor/entropy:0.6875190734863281 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4119926691055298 - training/rollout_probs_diff_mean:0.0047507998533546925 - training/rollout_probs_diff_std:0.009137631393969059 - training/rollout_actor_probs_pearson_corr:0.9995434284210205 - rollout_corr/rollout_is_mean:0.9999079704284668 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.3933751435833983e-05 - rollout_corr/rollout_is_max:1.649957537651062 - rollout_corr/rollout_is_min:0.2244005799293518 - rollout_corr/rollout_is_std:0.03564453125 - rollout_corr/rollout_is_eff_sample_size:0.9987308417828792 - rollout_corr/rollout_is_seq_mean:0.9998347759246826 - rollout_corr/rollout_is_seq_std:0.0020229120273143053 - rollout_corr/rollout_is_seq_max:1.005630373954773 - rollout_corr/rollout_is_seq_min:0.9910240769386292 - rollout_corr/rollout_is_seq_max_deviation:0.00897592306137085 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0345947919413447) - rollout_corr/training_log_ppl:np.float64(0.6928353062248789) - rollout_corr/kl:np.float64(0.0008856774063996653) - rollout_corr/k3_kl:np.float64(0.0006838213192850162) - rollout_corr/rollout_ppl:np.float64(2.0328480429016054) - rollout_corr/rollout_log_ppl:np.float64(0.6919496294576675) - rollout_corr/log_ppl_diff:np.float64(0.0008856767672114074) - rollout_corr/log_ppl_abs_diff:np.float64(0.001704056456219405) - rollout_corr/log_ppl_diff_max:np.float64(0.00990024209022522) - rollout_corr/log_ppl_diff_min:np.float64(-0.004864394664764404) - rollout_corr/ppl_ratio:np.float64(1.0008882319089025) - rollout_corr/chi2_token:np.float64(0.0009777550585567951) - rollout_corr/chi2_seq:np.float64(0.8223780991975218) - actor/pg_loss:np.float64(0.00021487544290721416) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000207112968837464) - actor/ppo_kl:np.float64(5.649118667616637e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38337405771017075) - perf/mfu/actor:np.float64(0.09893896312961671) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.62813949584961) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0014918704982846975 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0014918704982846975 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:489.6328125 - response_length/max:1187.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:489.6328125 - response_length_non_aborted/max:1187.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.28125 - prompt_length/max:540.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001486111432313919 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7897587660700083) - timing_s/agent_loop/generate_sequences/max:np.float64(10.173580570146441) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.444523438432952) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.173580570146441) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:330 - timing_s/agent_loop/slowest/response_length:1187 - timing_s/gen:17.149094123393297 - timing_s/reward:0.10750813968479633 - timing_s/old_log_prob:3.131292313337326 - timing_s/adv:0.1582884006202221 - timing_s/update_actor:12.998078908771276 - timing_s/step:33.6287423055619 - timing_s/stop_profile:0.00010932423174381256 - timing_per_token_ms/adv:0.0008104635834035928 - timing_per_token_ms/gen:0.13681405169206276 - timing_per_token_ms/update_actor:0.06655237887607793 - perf/total_num_tokens:195306 - perf/time_per_step:33.6287423055619 - perf/throughput:725.9638132813031 (TaskRunner pid=2224264) Training Progress: 18%|█▊ | 18/100 [14:16<51:58, 38.03s/it] (TaskRunner pid=2224264) step:19 - global_seqlen/min:21987 - global_seqlen/max:30303 - global_seqlen/minmax_diff:8316 - global_seqlen/balanced_min:26121 - global_seqlen/balanced_max:26132 - global_seqlen/mean:26128.5 - actor/entropy:0.6984733939170837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413516640663147 - training/rollout_probs_diff_mean:0.004623448941856623 - training/rollout_probs_diff_std:0.008824565447866917 - training/rollout_actor_probs_pearson_corr:0.9995649456977844 - rollout_corr/rollout_is_mean:0.9998148083686829 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.593936970806681e-06 - rollout_corr/rollout_is_max:1.9665265083312988 - rollout_corr/rollout_is_min:0.4158303737640381 - rollout_corr/rollout_is_std:0.035356584936380386 - rollout_corr/rollout_is_eff_sample_size:0.9987510563634333 - rollout_corr/rollout_is_seq_mean:0.9998022317886353 - rollout_corr/rollout_is_seq_std:0.001719683175906539 - rollout_corr/rollout_is_seq_max:1.005003571510315 - rollout_corr/rollout_is_seq_min:0.9942381978034973 - rollout_corr/rollout_is_seq_max_deviation:0.0057618021965026855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1011477317661047) - rollout_corr/training_log_ppl:np.float64(0.7186182939331047) - rollout_corr/kl:np.float64(0.000854842338526618) - rollout_corr/k3_kl:np.float64(0.0006763917490957283) - rollout_corr/rollout_ppl:np.float64(2.0992823550477624) - rollout_corr/rollout_log_ppl:np.float64(0.7177634529653005) - rollout_corr/log_ppl_diff:np.float64(0.0008548409678041935) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015009507769718766) - rollout_corr/log_ppl_diff_max:np.float64(0.0058748722076416016) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035433173179626465) - rollout_corr/ppl_ratio:np.float64(1.000856626080349) - rollout_corr/chi2_token:np.float64(0.0010158277582377195) - rollout_corr/chi2_seq:np.float64(0.722900487948209) - actor/pg_loss:np.float64(0.00023403007071465254) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021148656901459617) - actor/ppo_kl:np.float64(1.1429972420096135e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.30054471269249916) - perf/mfu/actor:np.float64(0.10550384895416316) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.6054801940918) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012871723156422377 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012871723156422377 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:514.390625 - response_length/max:1276.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:514.390625 - response_length_non_aborted/max:1276.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.125 - prompt_length/max:836.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001385621726512909 - timing_s/agent_loop/generate_sequences/min:np.float64(1.989379409700632) - timing_s/agent_loop/generate_sequences/max:np.float64(10.74332383275032) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.9351765791507205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.74332383275032) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:1276 - timing_s/gen:17.415069840848446 - timing_s/reward:0.06816395185887814 - timing_s/old_log_prob:3.1230949088931084 - timing_s/adv:0.12103297747671604 - timing_s/update_actor:13.004424184560776 - timing_s/step:33.81597386114299 - timing_s/stop_profile:3.315135836601257e-05 - timing_per_token_ms/adv:0.0005790275823177567 - timing_per_token_ms/gen:0.13224894323417002 - timing_per_token_ms/update_actor:0.062213790423104924 - perf/total_num_tokens:209028 - perf/time_per_step:33.81597386114299 - perf/throughput:772.667382204939 (TaskRunner pid=2224264) Training Progress: 19%|█▉ | 19/100 [14:50<49:38, 36.77s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) To determine which molecule is the only correct reactant used in the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure and functional groups of the product and match it with the structure of the reactant. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 1: Analyze the product (TaskRunner pid=2224264) The product is: (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) This can be interpreted as: (TaskRunner pid=2224264) - A nitrogen atom (N) is attached to a ring. (TaskRunner pid=2224264) - The ring is a pyrrole-like structure (N in the ring). (TaskRunner pid=2224264) - The ring is substituted with a group: `Cc2ccc(Cl)cc2C(F)(F)F`, which is a benzene ring with a chlorine atom and a trifluoromethyl group. (TaskRunner pid=2224264) - The structure is a pyrrole derivative with a substituent on the ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 2: Analyze the options (TaskRunner pid=2224264) Let’s examine the options one by one to see which one matches the structure of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option A: (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) This structure contains: (TaskRunner pid=2224264) - A carbonyl group (O=C), which is not in the product. (TaskRunner pid=2224264) - Two carbonyl groups (C=O), which are not in the product. (TaskRunner pid=2224264) - The part `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` matches the pyrrole-like structure of the product, but the presence of two carbonyl groups makes this a different compound. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option B: (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) This contains: (TaskRunner pid=2224264) - A carbonyl group (O=C), which is not in the product. (TaskRunner pid=2224264) - A chlorine atom (Cl) attached to the ring. (TaskRunner pid=2224264) - The rest of the structure does not match the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option C: (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) This contains: (TaskRunner pid=2224264) - Two carbonyl groups (C=O), which are not in the product. (TaskRunner pid=2224264) - A hydroxyl group (O), which is not in the product. (TaskRunner pid=2224264) - The part `c2ccc(Cl)c(C(F)(F)F)c2` is similar to the product, but the overall structure is not matching the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option D: (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) ``` (TaskRunner pid=2224264) This contains: (TaskRunner pid=2224264) - Two carbonyl groups (C=O), which are not in the product. (TaskRunner pid=2224264) - A fluorinated group (C(F)(F)F), which is present in the product. (TaskRunner pid=2224264) - The structure is not matching the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 3: Conclusion (TaskRunner pid=2224264) None of the options exactly match the structure of the product. However, **Option A** is the only one that contains the core structure of the product: `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1`. The other groups in Option A (carbonyl groups) are not part of the product, but the rest of the structure is the closest match. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Thus, **Option A** is the best candidate as the correct reactant. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2224264) step:20 - global_seqlen/min:22848 - global_seqlen/max:30898 - global_seqlen/minmax_diff:8050 - global_seqlen/balanced_min:26847 - global_seqlen/balanced_max:26876 - global_seqlen/mean:26867.0 - actor/entropy:0.6798826456069946 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967540204524994 - training/rollout_probs_diff_mean:0.004492460284382105 - training/rollout_probs_diff_std:0.008851698599755764 - training/rollout_actor_probs_pearson_corr:0.9995761513710022 - rollout_corr/rollout_is_mean:0.9998908638954163 - rollout_corr/rollout_is_ratio_fraction_high:1.4051259313418996e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.54838228225708 - rollout_corr/rollout_is_min:0.5467735528945923 - rollout_corr/rollout_is_std:0.034748680889606476 - rollout_corr/rollout_is_eff_sample_size:0.9987937474899505 - rollout_corr/rollout_is_seq_mean:0.9998890161514282 - rollout_corr/rollout_is_seq_std:0.0017418391071259975 - rollout_corr/rollout_is_seq_max:1.0073792934417725 - rollout_corr/rollout_is_seq_min:0.9951199293136597 - rollout_corr/rollout_is_seq_max_deviation:0.007379293441772461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.033839966636151) - rollout_corr/training_log_ppl:np.float64(0.6936833750805818) - rollout_corr/kl:np.float64(0.0007494871477833343) - rollout_corr/k3_kl:np.float64(0.0006487431917321373) - rollout_corr/rollout_ppl:np.float64(2.0322980885393918) - rollout_corr/rollout_log_ppl:np.float64(0.6929338824702427) - rollout_corr/log_ppl_diff:np.float64(0.0007494926103390753) - rollout_corr/log_ppl_abs_diff:np.float64(0.001494129013735801) - rollout_corr/log_ppl_diff_max:np.float64(0.005956470966339111) - rollout_corr/log_ppl_diff_min:np.float64(-0.006281614303588867) - rollout_corr/ppl_ratio:np.float64(1.000751372659579) - rollout_corr/chi2_token:np.float64(0.0011060182005167007) - rollout_corr/chi2_seq:np.float64(0.7747026174329221) - actor/pg_loss:np.float64(-0.00010517542250454426) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014331985744320264) - actor/ppo_kl:np.float64(1.3654966693366077e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.31467055529356003) - perf/mfu/actor:np.float64(0.10809592356334449) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(102.8171272277832) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.48392857142857143) - val-aux/sciknoweval/reward/std@16:np.float64(0.3171121217029239) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6190095238095239) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2728610239160176) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3498904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2656159472576298) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.48431904761904765) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3171144172517289) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.729252380952381) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.20041976709665926) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.24425714285714284) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1861647179153658) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5091) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2646122540406198) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8068619047619048) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.12410804929329204) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.17493809523809525) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11508084398834663) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5237238095238095) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.20562351058186512) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8522714285714285) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06491422684529762) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.13282380952380954) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06387633477899576) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5326761904761904) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.15389547364520514) - val-aux/sciknoweval/score/mean@16:np.float64(0.48392857142857143) - val-aux/sciknoweval/score/std@16:np.float64(0.3171121217029239) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6190095238095239) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2728610239160176) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.3498904761904762) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2656159472576298) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.48431904761904765) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3171144172517289) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.729252380952381) - val-aux/sciknoweval/score/best@4/std:np.float64(0.20041976709665926) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.24425714285714284) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1861647179153658) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5091) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2646122540406198) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8068619047619048) - val-aux/sciknoweval/score/best@8/std:np.float64(0.12410804929329204) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.17493809523809525) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11508084398834663) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5237238095238095) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.20562351058186512) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8522714285714285) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06491422684529762) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.13282380952380954) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06387633477899576) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5326761904761904) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.15389547364520514) - val-core/sciknoweval/acc/mean@16:np.float64(0.48392857142857143) - val-aux/sciknoweval/acc/std@16:np.float64(0.3171121217029239) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6190095238095239) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2728610239160176) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.3498904761904762) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2656159472576298) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.48431904761904765) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3171144172517289) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.729252380952381) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.20041976709665926) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.24425714285714284) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1861647179153658) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5091) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2646122540406198) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8068619047619048) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.12410804929329204) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.17493809523809525) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11508084398834663) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5237238095238095) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.20562351058186512) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8522714285714285) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06491422684529762) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.13282380952380954) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06387633477899576) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5326761904761904) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.15389547364520514) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9931547619047619) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.023312712167358005) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9993809523809524) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006485809156183406) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9871333333333333) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.030787750802297818) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9933857142857144) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.02299957497117341) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9763380952380952) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03858134164350246) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9975285714285714) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.014190325800172833) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9601904761904761) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.043352249329006456) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9995333333333334) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005554643730532354) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9392047619047619) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04061806930568887) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999666666666667) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0009652924861912784) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017669458175078034 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017669458175078034 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:556.0 - response_length/max:1546.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:556.0 - response_length_non_aborted/max:1546.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:392.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.37429004907608e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.663187799975276) - timing_s/agent_loop/generate_sequences/max:np.float64(12.704742550849915) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.302203726881999) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.704742550849915) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:336 - timing_s/agent_loop/slowest/response_length:1546 - timing_s/gen:19.35746957361698 - timing_s/reward:0.06838529743254185 - timing_s/old_log_prob:3.1675357930362225 - timing_s/adv:0.14193121530115604 - timing_s/update_actor:13.063340313732624 - timing_s/step:35.8843690212816 - timing_s/testing:160.607242802158 - timing_s/save_checkpoint:12.75176297314465 - timing_s/stop_profile:0.0001437067985534668 - timing_per_token_ms/adv:0.0006603417542950275 - timing_per_token_ms/gen:0.1359984092121247 - timing_per_token_ms/update_actor:0.06077781439001668 - perf/total_num_tokens:214936 - perf/time_per_step:35.8843690212816 - perf/throughput:748.710392094849 (TaskRunner pid=2224264) Training Progress: 20%|██ | 20/100 [18:19<1:58:06, 88.58s/it] (TaskRunner pid=2224264) step:21 - global_seqlen/min:21817 - global_seqlen/max:33018 - global_seqlen/minmax_diff:11201 - global_seqlen/balanced_min:27703 - global_seqlen/balanced_max:27729 - global_seqlen/mean:27721.25 - actor/entropy:0.6702685952186584 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163779854774475 - training/rollout_probs_diff_mean:0.004457509610801935 - training/rollout_probs_diff_std:0.00877729244530201 - training/rollout_actor_probs_pearson_corr:0.9996225833892822 - rollout_corr/rollout_is_mean:1.0001065731048584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7989544868469238 - rollout_corr/rollout_is_min:0.6091647744178772 - rollout_corr/rollout_is_std:0.03465593233704567 - rollout_corr/rollout_is_eff_sample_size:0.9988006450060433 - rollout_corr/rollout_is_seq_mean:1.0000840425491333 - rollout_corr/rollout_is_seq_std:0.001599725685082376 - rollout_corr/rollout_is_seq_max:1.0058624744415283 - rollout_corr/rollout_is_seq_min:0.9920676350593567 - rollout_corr/rollout_is_seq_max_deviation:0.00793236494064331 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.009537471458316) - rollout_corr/training_log_ppl:np.float64(0.6828376608900726) - rollout_corr/kl:np.float64(0.000724996415337742) - rollout_corr/k3_kl:np.float64(0.0006654245569848172) - rollout_corr/rollout_ppl:np.float64(2.008096869569272) - rollout_corr/rollout_log_ppl:np.float64(0.6821126618888229) - rollout_corr/log_ppl_diff:np.float64(0.000724999001249671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013606392312794924) - rollout_corr/log_ppl_diff_max:np.float64(0.009048700332641602) - rollout_corr/log_ppl_diff_min:np.float64(-0.004755973815917969) - rollout_corr/ppl_ratio:np.float64(1.0007265841122717) - rollout_corr/chi2_token:np.float64(0.001218994613736868) - rollout_corr/chi2_seq:np.float64(0.7947670572903007) - actor/pg_loss:np.float64(-0.00015212141443043947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016183609022846213) - actor/ppo_kl:np.float64(0.00017459750991832834) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3394359387457371) - perf/mfu/actor:np.float64(0.10795117855645653) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.55794906616211) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003770750481635332 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003770750481635332 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:583.1015625 - response_length/max:1506.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:583.1015625 - response_length_non_aborted/max:1506.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.1875 - prompt_length/max:669.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011318549513816833 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5702421497553587) - timing_s/agent_loop/generate_sequences/max:np.float64(12.702512258663774) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.579333163579577) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.702512258663774) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:321 - timing_s/agent_loop/slowest/response_length:1501 - timing_s/gen:18.926135268062353 - timing_s/reward:0.06378746405243874 - timing_s/old_log_prob:3.246299659833312 - timing_s/adv:0.12370392866432667 - timing_s/update_actor:13.3603654820472 - timing_s/step:35.82531424425542 - timing_s/stop_profile:0.0001258011907339096 - timing_per_token_ms/adv:0.0005578028077031459 - timing_per_token_ms/gen:0.12678788850075937 - timing_per_token_ms/update_actor:0.060244241701074085 - perf/total_num_tokens:221770 - perf/time_per_step:35.82531424425542 - perf/throughput:773.7894442738933 (TaskRunner pid=2224264) Training Progress: 21%|██ | 21/100 [18:55<1:35:47, 72.76s/it] (TaskRunner pid=2224264) step:22 - global_seqlen/min:20941 - global_seqlen/max:26032 - global_seqlen/minmax_diff:5091 - global_seqlen/balanced_min:23791 - global_seqlen/balanced_max:27438 - global_seqlen/mean:24253.875 - actor/entropy:0.6328612565994263 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18033698201179504 - training/rollout_probs_diff_mean:0.004449577536433935 - training/rollout_probs_diff_std:0.00884386245161295 - training/rollout_actor_probs_pearson_corr:0.9996188879013062 - rollout_corr/rollout_is_mean:0.9999265074729919 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.831578016281128 - rollout_corr/rollout_is_min:0.5145671367645264 - rollout_corr/rollout_is_std:0.0340801440179348 - rollout_corr/rollout_is_eff_sample_size:0.9988397723743896 - rollout_corr/rollout_is_seq_mean:0.9999299049377441 - rollout_corr/rollout_is_seq_std:0.0016637897351756692 - rollout_corr/rollout_is_seq_max:1.00577974319458 - rollout_corr/rollout_is_seq_min:0.9933801293373108 - rollout_corr/rollout_is_seq_max_deviation:0.006619870662689209 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9823644189164042) - rollout_corr/training_log_ppl:np.float64(0.6632540230639279) - rollout_corr/kl:np.float64(0.0008493472823345449) - rollout_corr/k3_kl:np.float64(0.000727977444228145) - rollout_corr/rollout_ppl:np.float64(1.9806274939328432) - rollout_corr/rollout_log_ppl:np.float64(0.6624046702054329) - rollout_corr/log_ppl_diff:np.float64(0.0008493528584949672) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016184973646886647) - rollout_corr/log_ppl_diff_max:np.float64(0.007628798484802246) - rollout_corr/log_ppl_diff_min:np.float64(-0.00557255744934082) - rollout_corr/ppl_ratio:np.float64(1.000851531745866) - rollout_corr/chi2_token:np.float64(0.0012481804005801678) - rollout_corr/chi2_seq:np.float64(0.8115007842425257) - actor/pg_loss:np.float64(0.0003393040969967842) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005306760692747048) - actor/ppo_kl:np.float64(0.00014998488964579337) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.35529135540127754) - perf/mfu/actor:np.float64(0.09394398118409722) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.99365997314453) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01673314906656742 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01673314906656742 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:502.37109375 - response_length/max:4901.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:502.37109375 - response_length_non_aborted/max:4901.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:592.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013022124767303467 - timing_s/agent_loop/generate_sequences/min:np.float64(1.669778237119317) - timing_s/agent_loop/generate_sequences/max:np.float64(30.597046691924334) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.5186303148948355) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(30.597046691924334) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:447 - timing_s/agent_loop/slowest/response_length:4901 - timing_s/gen:37.434364441782236 - timing_s/reward:0.06758200004696846 - timing_s/old_log_prob:3.2071466892957687 - timing_s/adv:0.11781474389135838 - timing_s/update_actor:13.525701014325023 - timing_s/step:54.440226985141635 - timing_s/stop_profile:0.00011406838893890381 - timing_per_token_ms/adv:0.0006071954682053815 - timing_per_token_ms/gen:0.2910756369543045 - timing_per_token_ms/update_actor:0.0697089692591649 - perf/total_num_tokens:194031 - perf/time_per_step:54.440226985141635 - perf/throughput:445.51384781366926 (TaskRunner pid=2224264) Training Progress: 22%|██▏ | 22/100 [19:49<1:27:27, 67.28s/it] (TaskRunner pid=2224264) step:23 - global_seqlen/min:20573 - global_seqlen/max:31570 - global_seqlen/minmax_diff:10997 - global_seqlen/balanced_min:28209 - global_seqlen/balanced_max:28226 - global_seqlen/mean:28218.75 - actor/entropy:0.6778538823127747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7014397382736206 - training/rollout_probs_diff_mean:0.0045140404254198074 - training/rollout_probs_diff_std:0.008873715065419674 - training/rollout_actor_probs_pearson_corr:0.9996016025543213 - rollout_corr/rollout_is_mean:0.999861478805542 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2932093341078144e-05 - rollout_corr/rollout_is_max:1.5868637561798096 - rollout_corr/rollout_is_min:0.09759196639060974 - rollout_corr/rollout_is_std:0.0343623049557209 - rollout_corr/rollout_is_eff_sample_size:0.9988203866993153 - rollout_corr/rollout_is_seq_mean:0.9999100565910339 - rollout_corr/rollout_is_seq_std:0.0015114216366782784 - rollout_corr/rollout_is_seq_max:1.004460334777832 - rollout_corr/rollout_is_seq_min:0.9952100515365601 - rollout_corr/rollout_is_seq_max_deviation:0.004789948463439941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.013377800118178) - rollout_corr/training_log_ppl:np.float64(0.6853225293452851) - rollout_corr/kl:np.float64(0.0008045039580757418) - rollout_corr/k3_kl:np.float64(0.0007065492574156451) - rollout_corr/rollout_ppl:np.float64(2.0117388791404665) - rollout_corr/rollout_log_ppl:np.float64(0.684518019144889) - rollout_corr/log_ppl_diff:np.float64(0.0008045102003961802) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014397446066141129) - rollout_corr/log_ppl_diff_max:np.float64(0.007155239582061768) - rollout_corr/log_ppl_diff_min:np.float64(-0.004554927349090576) - rollout_corr/ppl_ratio:np.float64(1.0008063388522714) - rollout_corr/chi2_token:np.float64(0.0012174241710454226) - rollout_corr/chi2_seq:np.float64(1.7728085950948298) - actor/pg_loss:np.float64(0.0003834259696304798) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000311835868842536) - actor/ppo_kl:np.float64(9.214319616290823e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.32652221247553825) - perf/mfu/actor:np.float64(0.11250225764410283) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.84423065185547) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0047743674367666245 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0047743674367666245 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.1171875 - response_length/max:1391.0 - response_length/min:180.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.1171875 - response_length_non_aborted/max:1391.0 - response_length_non_aborted/min:180.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.71875 - prompt_length/max:766.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010936148464679718 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4273153580725193) - timing_s/agent_loop/generate_sequences/max:np.float64(12.947387943044305) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.094733837271633) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.947387943044305) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:1391 - timing_s/gen:19.754568798467517 - timing_s/reward:0.07217162102460861 - timing_s/old_log_prob:3.125368718057871 - timing_s/adv:0.1287905927747488 - timing_s/update_actor:13.216036595404148 - timing_s/step:36.38285103067756 - timing_s/stop_profile:0.00010989978909492493 - timing_per_token_ms/adv:0.0005705009646721985 - timing_per_token_ms/gen:0.12773396613386992 - timing_per_token_ms/update_actor:0.05854279776480243 - perf/total_num_tokens:225750 - perf/time_per_step:36.38285103067756 - perf/throughput:775.6057923060046 (TaskRunner pid=2224264) Training Progress: 23%|██▎ | 23/100 [20:26<1:14:27, 58.02s/it] (TaskRunner pid=2224264) step:24 - global_seqlen/min:25634 - global_seqlen/max:31704 - global_seqlen/minmax_diff:6070 - global_seqlen/balanced_min:29137 - global_seqlen/balanced_max:29209 - global_seqlen/mean:29156.5 - actor/entropy:0.712544858455658 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3460633456707001 - training/rollout_probs_diff_mean:0.004510445520281792 - training/rollout_probs_diff_std:0.008606033399701118 - training/rollout_actor_probs_pearson_corr:0.9996117949485779 - rollout_corr/rollout_is_mean:0.9999745488166809 - rollout_corr/rollout_is_ratio_fraction_high:6.192870841914555e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.238574168382911e-05 - rollout_corr/rollout_is_max:2.087355136871338 - rollout_corr/rollout_is_min:0.47689226269721985 - rollout_corr/rollout_is_std:0.03445065766572952 - rollout_corr/rollout_is_eff_sample_size:0.9988145592508434 - rollout_corr/rollout_is_seq_mean:0.999997615814209 - rollout_corr/rollout_is_seq_std:0.0016162468818947673 - rollout_corr/rollout_is_seq_max:1.0054336786270142 - rollout_corr/rollout_is_seq_min:0.9961015582084656 - rollout_corr/rollout_is_seq_max_deviation:0.00543367862701416 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.082198773045093) - rollout_corr/training_log_ppl:np.float64(0.7211795629700646) - rollout_corr/kl:np.float64(0.0007120691560231762) - rollout_corr/k3_kl:np.float64(0.0006404822171930391) - rollout_corr/rollout_ppl:np.float64(2.0806773733347654) - rollout_corr/rollout_log_ppl:np.float64(0.7204674915410578) - rollout_corr/log_ppl_diff:np.float64(0.0007120714290067554) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014570994535461068) - rollout_corr/log_ppl_diff_max:np.float64(0.005695939064025879) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035497546195983887) - rollout_corr/ppl_ratio:np.float64(1.000713731860742) - rollout_corr/chi2_token:np.float64(0.0011395812034606934) - rollout_corr/chi2_seq:np.float64(1.328517090762034) - actor/pg_loss:np.float64(-4.311627708375454e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001907837179260241) - actor/ppo_kl:np.float64(0.0001019216878037632) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3508700765669346) - perf/mfu/actor:np.float64(0.11424736196001828) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.2998194694519) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002073063515126705 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002073063515126705 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:630.765625 - response_length/max:1901.0 - response_length/min:207.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:630.765625 - response_length_non_aborted/max:1901.0 - response_length_non_aborted/min:207.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.375 - prompt_length/max:848.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011662021279335022 - timing_s/agent_loop/generate_sequences/min:np.float64(2.381055638194084) - timing_s/agent_loop/generate_sequences/max:np.float64(15.576675411313772) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.741719769590418) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.576675411313772) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1901 - timing_s/gen:22.96907013654709 - timing_s/reward:0.07352139614522457 - timing_s/old_log_prob:3.155397677794099 - timing_s/adv:0.11861182004213333 - timing_s/update_actor:13.56319702602923 - timing_s/step:39.965819630771875 - timing_s/stop_profile:0.00010657869279384613 - timing_per_token_ms/adv:0.0005085136249298326 - timing_per_token_ms/gen:0.14224448299776493 - timing_per_token_ms/update_actor:0.05814825607510002 - perf/total_num_tokens:233252 - perf/time_per_step:39.965819630771875 - perf/throughput:729.5358951565405 (TaskRunner pid=2224264) Training Progress: 24%|██▍ | 24/100 [21:06<1:06:38, 52.62s/it] (TaskRunner pid=2224264) step:25 - global_seqlen/min:23500 - global_seqlen/max:34274 - global_seqlen/minmax_diff:10774 - global_seqlen/balanced_min:27954 - global_seqlen/balanced_max:27980 - global_seqlen/mean:27971.25 - actor/entropy:0.7009389400482178 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18294647336006165 - training/rollout_probs_diff_mean:0.004573621321469545 - training/rollout_probs_diff_std:0.008723455481231213 - training/rollout_actor_probs_pearson_corr:0.999584436416626 - rollout_corr/rollout_is_mean:0.9999302625656128 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.836859029135667e-06 - rollout_corr/rollout_is_max:1.4950411319732666 - rollout_corr/rollout_is_min:0.4878106117248535 - rollout_corr/rollout_is_std:0.03484973683953285 - rollout_corr/rollout_is_eff_sample_size:0.9987866122287246 - rollout_corr/rollout_is_seq_mean:0.9999464154243469 - rollout_corr/rollout_is_seq_std:0.001561047974973917 - rollout_corr/rollout_is_seq_max:1.004056453704834 - rollout_corr/rollout_is_seq_min:0.9949167370796204 - rollout_corr/rollout_is_seq_max_deviation:0.005083262920379639 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1113617275841534) - rollout_corr/training_log_ppl:np.float64(0.7249491217080504) - rollout_corr/kl:np.float64(0.0006290591016089309) - rollout_corr/k3_kl:np.float64(0.0006558882103604446) - rollout_corr/rollout_ppl:np.float64(2.1101012001745403) - rollout_corr/rollout_log_ppl:np.float64(0.7243200563825667) - rollout_corr/log_ppl_diff:np.float64(0.0006290653254836798) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012821299023926258) - rollout_corr/log_ppl_diff_max:np.float64(0.005804777145385742) - rollout_corr/log_ppl_diff_min:np.float64(-0.004645586013793945) - rollout_corr/ppl_ratio:np.float64(1.0006304802373052) - rollout_corr/chi2_token:np.float64(0.00137246772646904) - rollout_corr/chi2_seq:np.float64(0.4346559832338244) - actor/pg_loss:np.float64(-0.00020399875938892365) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024524242849111033) - actor/ppo_kl:np.float64(-6.53956497700392e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.31579897925257683) - perf/mfu/actor:np.float64(0.10990411377383731) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(108.17280960083008) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002174121094867587 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002174121094867587 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:571.3515625 - response_length/max:1596.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.3515625 - response_length_non_aborted/max:1596.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.75 - prompt_length/max:646.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010717101395130157 - timing_s/agent_loop/generate_sequences/min:np.float64(3.3154151998460293) - timing_s/agent_loop/generate_sequences/max:np.float64(15.293862147256732) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.137247967235453) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.293862147256732) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:1596 - timing_s/gen:22.198232408612967 - timing_s/reward:0.06951954215765 - timing_s/old_log_prob:3.2000021431595087 - timing_s/adv:0.11978011205792427 - timing_s/update_actor:13.237928776070476 - timing_s/step:38.91434215568006 - timing_s/stop_profile:0.00012552551925182343 - timing_per_token_ms/adv:0.000535282263296797 - timing_per_token_ms/gen:0.15176618222015345 - timing_per_token_ms/update_actor:0.05915863956772791 - perf/total_num_tokens:223770 - perf/time_per_step:38.91434215568006 - perf/throughput:718.7902570239705 (TaskRunner pid=2224264) Training Progress: 25%|██▌ | 25/100 [21:45<1:00:38, 48.52s/it] (TaskRunner pid=2224264) step:26 - global_seqlen/min:22426 - global_seqlen/max:29951 - global_seqlen/minmax_diff:7525 - global_seqlen/balanced_min:26952 - global_seqlen/balanced_max:26964 - global_seqlen/mean:26960.375 - actor/entropy:0.7369614243507385 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31862038373947144 - training/rollout_probs_diff_mean:0.004758655559271574 - training/rollout_probs_diff_std:0.00890500657260418 - training/rollout_actor_probs_pearson_corr:0.9995867609977722 - rollout_corr/rollout_is_mean:0.9999259114265442 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4305845070339274e-05 - rollout_corr/rollout_is_max:1.8682441711425781 - rollout_corr/rollout_is_min:0.27226293087005615 - rollout_corr/rollout_is_std:0.03548027575016022 - rollout_corr/rollout_is_eff_sample_size:0.9987426137038081 - rollout_corr/rollout_is_seq_mean:0.9999041557312012 - rollout_corr/rollout_is_seq_std:0.001717878389172256 - rollout_corr/rollout_is_seq_max:1.005202293395996 - rollout_corr/rollout_is_seq_min:0.9952166080474854 - rollout_corr/rollout_is_seq_max_deviation:0.005202293395996094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1379281943663955) - rollout_corr/training_log_ppl:np.float64(0.7447412704350427) - rollout_corr/kl:np.float64(0.0007289938001822094) - rollout_corr/k3_kl:np.float64(0.0007074658968804215) - rollout_corr/rollout_ppl:np.float64(2.136324817314744) - rollout_corr/rollout_log_ppl:np.float64(0.7440122718689963) - rollout_corr/log_ppl_diff:np.float64(0.0007289985660463572) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015073230024427176) - rollout_corr/log_ppl_diff_max:np.float64(0.005815982818603516) - rollout_corr/log_ppl_diff_min:np.float64(-0.004653453826904297) - rollout_corr/ppl_ratio:np.float64(1.00073087005876) - rollout_corr/chi2_token:np.float64(0.0013713838998228312) - rollout_corr/chi2_seq:np.float64(0.9597642624285072) - actor/pg_loss:np.float64(0.00014449155423790216) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012465850477383356) - actor/ppo_kl:np.float64(-1.70074631995476e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2908688709139824) - perf/mfu/actor:np.float64(0.105823819821739) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.64182186126709) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029881333466619253 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0029881333466619253 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:546.10546875 - response_length/max:1208.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:546.10546875 - response_length_non_aborted/max:1208.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.40625 - prompt_length/max:709.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015214644372463226 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7672749813646078) - timing_s/agent_loop/generate_sequences/max:np.float64(10.418701702728868) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.990022340542055) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.418701702728868) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:329 - timing_s/agent_loop/slowest/response_length:1208 - timing_s/gen:18.300879038870335 - timing_s/reward:0.07066521607339382 - timing_s/old_log_prob:3.2463080435991287 - timing_s/adv:0.16146212071180344 - timing_s/update_actor:13.367159321904182 - timing_s/step:35.227726478129625 - timing_s/stop_profile:2.6691704988479614e-05 - timing_per_token_ms/adv:0.0007486084703560477 - timing_per_token_ms/gen:0.13090476627018258 - timing_per_token_ms/update_actor:0.06197595230919536 - perf/total_num_tokens:215683 - perf/time_per_step:35.227726478129625 - perf/throughput:765.3169163992961 (TaskRunner pid=2224264) Training Progress: 26%|██▌ | 26/100 [22:20<54:56, 44.54s/it] (TaskRunner pid=2224264) step:27 - global_seqlen/min:23952 - global_seqlen/max:34841 - global_seqlen/minmax_diff:10889 - global_seqlen/balanced_min:28921 - global_seqlen/balanced_max:28949 - global_seqlen/mean:28940.875 - actor/entropy:0.701714038848877 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17479756474494934 - training/rollout_probs_diff_mean:0.004477211274206638 - training/rollout_probs_diff_std:0.008556351996958256 - training/rollout_actor_probs_pearson_corr:0.9996283054351807 - rollout_corr/rollout_is_mean:0.9999887943267822 - rollout_corr/rollout_is_ratio_fraction_high:6.0401425798772834e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.0401425798772834e-06 - rollout_corr/rollout_is_max:2.401174545288086 - rollout_corr/rollout_is_min:0.40530839562416077 - rollout_corr/rollout_is_std:0.034324124455451965 - rollout_corr/rollout_is_eff_sample_size:0.9988231220557214 - rollout_corr/rollout_is_seq_mean:0.9999889135360718 - rollout_corr/rollout_is_seq_std:0.0014754357980564237 - rollout_corr/rollout_is_seq_max:1.0065122842788696 - rollout_corr/rollout_is_seq_min:0.9951512813568115 - rollout_corr/rollout_is_seq_max_deviation:0.006512284278869629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.07450102083385) - rollout_corr/training_log_ppl:np.float64(0.7144390830071643) - rollout_corr/kl:np.float64(0.0007389031823423764) - rollout_corr/k3_kl:np.float64(0.0006497853174778356) - rollout_corr/rollout_ppl:np.float64(2.0729061695747077) - rollout_corr/rollout_log_ppl:np.float64(0.713700178428553) - rollout_corr/log_ppl_diff:np.float64(0.0007389045786112547) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013226103037595749) - rollout_corr/log_ppl_diff_max:np.float64(0.0057825446128845215) - rollout_corr/log_ppl_diff_min:np.float64(-0.005777716636657715) - rollout_corr/ppl_ratio:np.float64(1.0007404030766338) - rollout_corr/chi2_token:np.float64(0.001133262412622571) - rollout_corr/chi2_seq:np.float64(0.8787905650679022) - actor/pg_loss:np.float64(-2.4800654500722885e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022700108638673555) - actor/ppo_kl:np.float64(0.00011755238243083) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.32230544835329056) - perf/mfu/actor:np.float64(0.11335348091929753) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.77265930175781) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006488623563200235 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006488623563200235 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:646.71484375 - response_length/max:1758.0 - response_length/min:222.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.71484375 - response_length_non_aborted/max:1758.0 - response_length_non_aborted/min:222.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.6875 - prompt_length/max:380.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.4581090807914734e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.452285857871175) - timing_s/agent_loop/generate_sequences/max:np.float64(14.636356772854924) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.195729885730543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.636356772854924) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:194 - timing_s/agent_loop/slowest/response_length:1758 - timing_s/gen:21.458807596936822 - timing_s/reward:0.06992848217487335 - timing_s/old_log_prob:3.163326393812895 - timing_s/adv:0.12315551564097404 - timing_s/update_actor:13.37716430425644 - timing_s/step:38.279962316155434 - timing_s/stop_profile:0.00011939182877540588 - timing_per_token_ms/adv:0.0005319272293986189 - timing_per_token_ms/gen:0.1296142619666513 - timing_per_token_ms/update_actor:0.057777988330762454 - perf/total_num_tokens:231527 - perf/time_per_step:38.279962316155434 - perf/throughput:756.0319616037337 (TaskRunner pid=2224264) Training Progress: 27%|██▋ | 27/100 [22:58<51:55, 42.68s/it] (TaskRunner pid=2224264) step:28 - global_seqlen/min:25216 - global_seqlen/max:34360 - global_seqlen/minmax_diff:9144 - global_seqlen/balanced_min:29787 - global_seqlen/balanced_max:30195 - global_seqlen/mean:29843.75 - actor/entropy:0.7055036425590515 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1455976366996765 - training/rollout_probs_diff_mean:0.004539164714515209 - training/rollout_probs_diff_std:0.008678900077939034 - training/rollout_actor_probs_pearson_corr:0.9996199011802673 - rollout_corr/rollout_is_mean:1.0000463724136353 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.916669579164591e-06 - rollout_corr/rollout_is_max:1.6682544946670532 - rollout_corr/rollout_is_min:0.4844903349876404 - rollout_corr/rollout_is_std:0.0349300317466259 - rollout_corr/rollout_is_eff_sample_size:0.9987814986875572 - rollout_corr/rollout_is_seq_mean:1.0001145601272583 - rollout_corr/rollout_is_seq_std:0.0015147364465519786 - rollout_corr/rollout_is_seq_max:1.0062413215637207 - rollout_corr/rollout_is_seq_min:0.9959177374839783 - rollout_corr/rollout_is_seq_max_deviation:0.006241321563720703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.081616838928312) - rollout_corr/training_log_ppl:np.float64(0.716681465273723) - rollout_corr/kl:np.float64(0.0007082077879232429) - rollout_corr/k3_kl:np.float64(0.0006868325160667155) - rollout_corr/rollout_ppl:np.float64(2.0801399582996964) - rollout_corr/rollout_log_ppl:np.float64(0.7159732520813122) - rollout_corr/log_ppl_diff:np.float64(0.0007082131924107671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013121537631377578) - rollout_corr/log_ppl_diff_max:np.float64(0.00504302978515625) - rollout_corr/log_ppl_diff_min:np.float64(-0.0041460394859313965) - rollout_corr/ppl_ratio:np.float64(1.0007096340414137) - rollout_corr/chi2_token:np.float64(0.0013347528874874115) - rollout_corr/chi2_seq:np.float64(4.891642920905724) - actor/pg_loss:np.float64(-6.637291517108679e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002813626040278905) - actor/ppo_kl:np.float64(0.00019132641280006357) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.33150169998407364) - perf/mfu/actor:np.float64(0.11585717187378697) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.45846176147461) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007486805785447359 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007486805785447359 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:660.2109375 - response_length/max:2519.0 - response_length/min:204.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:660.2109375 - response_length_non_aborted/max:2519.0 - response_length_non_aborted/min:204.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.40625 - prompt_length/max:443.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011588819324970245 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6954549103975296) - timing_s/agent_loop/generate_sequences/max:np.float64(18.969659883528948) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.485852991718275) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.969659883528948) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:2519 - timing_s/gen:25.569389276206493 - timing_s/reward:0.08602294698357582 - timing_s/old_log_prob:3.1977268513292074 - timing_s/adv:0.1315357405692339 - timing_s/update_actor:13.538917873054743 - timing_s/step:42.61257563531399 - timing_s/stop_profile:0.00011502020061016083 - timing_per_token_ms/adv:0.0005509350390334404 - timing_per_token_ms/gen:0.1512856288603695 - timing_per_token_ms/update_actor:0.056707509415936094 - perf/total_num_tokens:238750 - perf/time_per_step:42.61257563531399 - perf/throughput:700.3507662950987 (TaskRunner pid=2224264) Training Progress: 28%|██▊ | 28/100 [23:41<51:12, 42.67s/it] (TaskRunner pid=2224264) step:29 - global_seqlen/min:26637 - global_seqlen/max:40228 - global_seqlen/minmax_diff:13591 - global_seqlen/balanced_min:33532 - global_seqlen/balanced_max:33558 - global_seqlen/mean:33551.125 - actor/entropy:0.6751730442047119 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30162957310676575 - training/rollout_probs_diff_mean:0.004225059412419796 - training/rollout_probs_diff_std:0.008393223397433758 - training/rollout_actor_probs_pearson_corr:0.9996880292892456 - rollout_corr/rollout_is_mean:0.9999781847000122 - rollout_corr/rollout_is_ratio_fraction_high:5.061727733846055e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.012345546769211e-05 - rollout_corr/rollout_is_max:2.110851764678955 - rollout_corr/rollout_is_min:0.42815104126930237 - rollout_corr/rollout_is_std:0.033269479870796204 - rollout_corr/rollout_is_eff_sample_size:0.9988943655271626 - rollout_corr/rollout_is_seq_mean:1.0000176429748535 - rollout_corr/rollout_is_seq_std:0.001319664646871388 - rollout_corr/rollout_is_seq_max:1.0051316022872925 - rollout_corr/rollout_is_seq_min:0.9961720108985901 - rollout_corr/rollout_is_seq_max_deviation:0.0051316022872924805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.010434144642204) - rollout_corr/training_log_ppl:np.float64(0.6835667735431343) - rollout_corr/kl:np.float64(0.0005298591240814332) - rollout_corr/k3_kl:np.float64(0.0006146891930143283) - rollout_corr/rollout_ppl:np.float64(2.009353395085782) - rollout_corr/rollout_log_ppl:np.float64(0.6830369132512715) - rollout_corr/log_ppl_diff:np.float64(0.0005298602918628603) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011505558213684708) - rollout_corr/log_ppl_diff_max:np.float64(0.0041373372077941895) - rollout_corr/log_ppl_diff_min:np.float64(-0.005634099245071411) - rollout_corr/ppl_ratio:np.float64(1.0005309600383043) - rollout_corr/chi2_token:np.float64(0.0014236378483474255) - rollout_corr/chi2_seq:np.float64(0.9517913018353283) - actor/pg_loss:np.float64(7.968046702444553e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001298733541261754) - actor/ppo_kl:np.float64(-2.9945098688699545e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21987130120396614) - perf/mfu/actor:np.float64(0.11725733884439407) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(109.95436000823975) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00392979895696044 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00392979895696044 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:771.72265625 - response_length/max:1889.0 - response_length/min:208.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:771.72265625 - response_length_non_aborted/max:1889.0 - response_length_non_aborted/min:208.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.75 - prompt_length/max:492.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.993037045001984e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6769527215510607) - timing_s/agent_loop/generate_sequences/max:np.float64(16.69622828066349) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.844669617952604) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.69622828066349) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:330 - timing_s/agent_loop/slowest/response_length:1889 - timing_s/gen:23.39403291605413 - timing_s/reward:0.07856784015893936 - timing_s/old_log_prob:4.369897283613682 - timing_s/adv:0.31469788402318954 - timing_s/update_actor:15.082241535186768 - timing_s/step:43.32600663602352 - timing_s/stop_profile:0.0001105237752199173 - timing_per_token_ms/adv:0.001172456527252028 - timing_per_token_ms/gen:0.11841422606715966 - timing_per_token_ms/update_actor:0.05619126607225081 - perf/total_num_tokens:268409 - perf/time_per_step:43.32600663602352 - perf/throughput:774.387662400066 (TaskRunner pid=2224264) Training Progress: 29%|██▉ | 29/100 [24:24<50:44, 42.88s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) To determine which of the given options is the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure and functional groups involved. (TaskRunner pid=2224264) (TaskRunner pid=2224264) The target product is: (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This can be interpreted as: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - A **nitrile group (–CN)** attached to a ring (likely a pyrrole or similar heterocycle). (TaskRunner pid=2224264) - The ring has a **chloro group (–Cl)** on a carbon and a **trifluoromethyl group (–CF₃)** on another carbon. (TaskRunner pid=2224264) - The structure suggests that the molecule is a **pyrrole derivative** with a **chloro-substituted benzene ring** and a **trifluoromethyl group** attached to the same ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Now, let's examine the options one by one to see which one could be a **reactant** that leads to this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option A: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a **complex molecule**. It contains: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - A **carbonyl group (C=O)** and a **nitrile group (–CN)**. (TaskRunner pid=2224264) - A **benzene ring** with a **chloro group (–Cl)** and a **trifluoromethyl group (–CF₃)**. (TaskRunner pid=2224264) - A **pyrrole-like ring** (as suggested by the nitrile group and the ring structure). (TaskRunner pid=2224264) - The **nitrile group** is attached to the ring in a way that matches the target product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This option **matches the structure** of the target molecule. It is **likely the correct reactant** that could be used in the synthesis of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option B: (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This structure contains: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - A **carbonyl group (C=O)**. (TaskRunner pid=2224264) - A **chloro group (–Cl)**. (TaskRunner pid=2224264) - A **pyrrole-like ring**. (TaskRunner pid=2224264) - However, the **trifluoromethyl group (–CF₃)** is **missing**. (TaskRunner pid=2224264) - The **nitrile group (–CN)** is **not clearly present**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This molecule is **not a good match** for the target product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option C: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This structure contains: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - A **carbonyl group (C=O)**. (TaskRunner pid=2224264) - A **trifluoromethyl group (–CF₃)**. (TaskRunner pid=2224264) - A **benzene ring** with a **chloro group (–Cl)**. (TaskRunner pid=2224264) - A **pyrrole-like ring**. (TaskRunner pid=2224264) - However, the **nitrile group (–CN)** is **missing**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This molecule is **not a good match** for the target product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option D: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This structure contains: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - A **carbonyl group (C=O)**. (TaskRunner pid=2224264) - A **trifluoromethyl group (–CF₃)**. (TaskRunner pid=2224264) - A **benzene ring**. (TaskRunner pid=2224264) - A **pyrrole-like ring**. (TaskRunner pid=2224264) - However, the **nitrile group (–CN)** is **missing**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Again, this molecule is **not a good match** for the target product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Final Analysis: (TaskRunner pid=2224264) Only **Option A** contains all the necessary functional groups and structure to be a **valid reactant** for the synthesis of the target molecule **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2224264) step:30 - global_seqlen/min:28624 - global_seqlen/max:34380 - global_seqlen/minmax_diff:5756 - global_seqlen/balanced_min:31823 - global_seqlen/balanced_max:31938 - global_seqlen/mean:31860.75 - actor/entropy:0.7115141153335571 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5642241835594177 - training/rollout_probs_diff_mean:0.004424138925969601 - training/rollout_probs_diff_std:0.008687857538461685 - training/rollout_actor_probs_pearson_corr:0.999598503112793 - rollout_corr/rollout_is_mean:0.9999087452888489 - rollout_corr/rollout_is_ratio_fraction_high:5.690936632163357e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.1381873264326714e-05 - rollout_corr/rollout_is_max:2.0644965171813965 - rollout_corr/rollout_is_min:0.22554616630077362 - rollout_corr/rollout_is_std:0.034556034952402115 - rollout_corr/rollout_is_eff_sample_size:0.9988071858420651 - rollout_corr/rollout_is_seq_mean:0.9999098777770996 - rollout_corr/rollout_is_seq_std:0.0014933171914890409 - rollout_corr/rollout_is_seq_max:1.0051254034042358 - rollout_corr/rollout_is_seq_min:0.9926964044570923 - rollout_corr/rollout_is_seq_max_deviation:0.007303595542907715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1091170106083155) - rollout_corr/training_log_ppl:np.float64(0.7282795129576698) - rollout_corr/kl:np.float64(0.0007562485593410884) - rollout_corr/k3_kl:np.float64(0.0006585642515801737) - rollout_corr/rollout_ppl:np.float64(2.107493511401117) - rollout_corr/rollout_log_ppl:np.float64(0.7275232608662918) - rollout_corr/log_ppl_diff:np.float64(0.0007562520913779736) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013768195640295744) - rollout_corr/log_ppl_diff_max:np.float64(0.005323231220245361) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032753348350524902) - rollout_corr/ppl_ratio:np.float64(1.0007577680516988) - rollout_corr/chi2_token:np.float64(0.0011402126401662827) - rollout_corr/chi2_seq:np.float64(0.9514149494934827) - actor/pg_loss:np.float64(-0.00013444386422634125) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002210548966559145) - actor/ppo_kl:np.float64(4.4027712672978225e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2887660190463066) - perf/mfu/actor:np.float64(0.12035666503727542) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(107.33295822143555) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.549404761904762) - val-aux/sciknoweval/reward/std@16:np.float64(0.31608937376624846) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6822666666666667) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.25471779568655717) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.41630476190476196) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.28182671788474917) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5503333333333333) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3158685881058942) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7810047619047619) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1712884300404119) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.30183333333333334) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.21622513075595678) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.587) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2597657274415112) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8442666666666666) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10407977320648662) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.21562857142857142) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.14281142327250002) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.609352380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1968004873596571) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8824571428571428) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.056375240990812564) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.16108571428571428) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07902405626412515) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6248190476190476) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1453890445797554) - val-aux/sciknoweval/score/mean@16:np.float64(0.549404761904762) - val-aux/sciknoweval/score/std@16:np.float64(0.31608937376624846) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6822666666666667) - val-aux/sciknoweval/score/best@2/std:np.float64(0.25471779568655717) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.41630476190476196) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.28182671788474917) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5503333333333333) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3158685881058942) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7810047619047619) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1712884300404119) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.30183333333333334) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.21622513075595678) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.587) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2597657274415112) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8442666666666666) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10407977320648662) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.21562857142857142) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.14281142327250002) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.609352380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1968004873596571) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8824571428571428) - val-aux/sciknoweval/score/best@16/std:np.float64(0.056375240990812564) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.16108571428571428) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07902405626412515) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6248190476190476) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1453890445797554) - val-core/sciknoweval/acc/mean@16:np.float64(0.549404761904762) - val-aux/sciknoweval/acc/std@16:np.float64(0.31608937376624846) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6822666666666667) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.25471779568655717) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.41630476190476196) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.28182671788474917) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5503333333333333) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3158685881058942) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7810047619047619) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1712884300404119) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.30183333333333334) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.21622513075595678) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.587) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2597657274415112) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8442666666666666) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10407977320648662) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.21562857142857142) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.14281142327250002) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.609352380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1968004873596571) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8824571428571428) - val-core/sciknoweval/acc/best@16/std:np.float64(0.056375240990812564) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.16108571428571428) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07902405626412515) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6248190476190476) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1453890445797554) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9892857142857143) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03624438335470682) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9989476190476191) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010879631942208362) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9796380952380952) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04790227517330985) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9895285714285714) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03580241391826494) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0001505093393264677) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9626380952380952) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05990016890311522) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9957904761904761) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0229613769030037) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9365666666666667) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06682360250957863) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9989619047619047) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.011013375581110648) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9040238095238093) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.061018043502578855) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999142857142858) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0015394818506189996) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.41796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002168958308175206 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002168958308175206 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:686.3984375 - response_length/max:2205.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:686.3984375 - response_length_non_aborted/max:2205.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.25 - prompt_length/max:516.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.668455898761749e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4918374791741371) - timing_s/agent_loop/generate_sequences/max:np.float64(17.408627146854997) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.319493488706939) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.408627146854997) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:349 - timing_s/agent_loop/slowest/response_length:2205 - timing_s/gen:24.27543739043176 - timing_s/reward:0.07385276816785336 - timing_s/old_log_prob:3.287810454145074 - timing_s/adv:0.10759958252310753 - timing_s/update_actor:13.824590772390366 - timing_s/step:41.648208467289805 - timing_s/testing:186.15690745972097 - timing_s/save_checkpoint:15.06649879179895 - timing_s/stop_profile:0.00015422143042087555 - timing_per_token_ms/adv:0.00042214787208048904 - timing_per_token_ms/gen:0.13814997547452032 - timing_per_token_ms/update_actor:0.05423832918398957 - perf/total_num_tokens:254886 - perf/time_per_step:41.648208467289805 - perf/throughput:764.9968911633546 (TaskRunner pid=2224264) Training Progress: 30%|███ | 30/100 [28:27<2:00:03, 102.91s/it] (TaskRunner pid=2224264) step:31 - global_seqlen/min:30423 - global_seqlen/max:35721 - global_seqlen/minmax_diff:5298 - global_seqlen/balanced_min:33243 - global_seqlen/balanced_max:33352 - global_seqlen/mean:33261.875 - actor/entropy:0.7014114856719971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39983245730400085 - training/rollout_probs_diff_mean:0.004444092512130737 - training/rollout_probs_diff_std:0.008645395748317242 - training/rollout_actor_probs_pearson_corr:0.9995979070663452 - rollout_corr/rollout_is_mean:1.0000203847885132 - rollout_corr/rollout_is_ratio_fraction_high:5.1425781748548616e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.1425781748548616e-06 - rollout_corr/rollout_is_max:3.193486452102661 - rollout_corr/rollout_is_min:0.4723663926124573 - rollout_corr/rollout_is_std:0.03425285220146179 - rollout_corr/rollout_is_eff_sample_size:0.998828117093035 - rollout_corr/rollout_is_seq_mean:1.0000666379928589 - rollout_corr/rollout_is_seq_std:0.0014698212035000324 - rollout_corr/rollout_is_seq_max:1.0053932666778564 - rollout_corr/rollout_is_seq_min:0.9955719709396362 - rollout_corr/rollout_is_seq_max_deviation:0.005393266677856445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.08949265582487) - rollout_corr/training_log_ppl:np.float64(0.7218374547082931) - rollout_corr/kl:np.float64(0.0005589085460151466) - rollout_corr/k3_kl:np.float64(0.0006561674455269895) - rollout_corr/rollout_ppl:np.float64(2.088360784109682) - rollout_corr/rollout_log_ppl:np.float64(0.7212785450974479) - rollout_corr/log_ppl_diff:np.float64(0.0005589096108451486) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012190231354907155) - rollout_corr/log_ppl_diff_max:np.float64(0.004900336265563965) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036498308181762695) - rollout_corr/ppl_ratio:np.float64(1.0005601237062365) - rollout_corr/chi2_token:np.float64(0.0015174625441432) - rollout_corr/chi2_seq:np.float64(2.2517903256230056) - actor/pg_loss:np.float64(-9.307335130870342e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020832591962971492) - actor/ppo_kl:np.float64(1.1285746243316908e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2606422230601311) - perf/mfu/actor:np.float64(0.12484098624463014) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(113.83588361740112) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009717543609440327 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009717543609440327 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:759.58984375 - response_length/max:2514.0 - response_length/min:234.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:759.58984375 - response_length_non_aborted/max:2514.0 - response_length_non_aborted/min:234.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.84375 - prompt_length/max:635.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014492124319076538 - timing_s/agent_loop/generate_sequences/min:np.float64(3.011936265975237) - timing_s/agent_loop/generate_sequences/max:np.float64(20.60571327060461) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.018827380772564) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.60571327060461) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:2514 - timing_s/gen:27.48937368579209 - timing_s/reward:0.07477623783051968 - timing_s/old_log_prob:3.7115073055028915 - timing_s/adv:0.17105644941329956 - timing_s/update_actor:13.997227132320404 - timing_s/step:45.54523225687444 - timing_s/stop_profile:0.00011219456791877747 - timing_per_token_ms/adv:0.0006428397730633779 - timing_per_token_ms/gen:0.14136624764491573 - timing_per_token_ms/update_actor:0.05260236807275749 - perf/total_num_tokens:266095 - perf/time_per_step:45.54523225687444 - perf/throughput:730.3042130162717 (TaskRunner pid=2224264) Training Progress: 31%|███ | 31/100 [29:13<1:38:34, 85.71s/it] (TaskRunner pid=2224264) step:32 - global_seqlen/min:25482 - global_seqlen/max:38028 - global_seqlen/minmax_diff:12546 - global_seqlen/balanced_min:31743 - global_seqlen/balanced_max:31780 - global_seqlen/mean:31769.5 - actor/entropy:0.6990305185317993 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2773066759109497 - training/rollout_probs_diff_mean:0.004490962252020836 - training/rollout_probs_diff_std:0.00860312208533287 - training/rollout_actor_probs_pearson_corr:0.9996287226676941 - rollout_corr/rollout_is_mean:1.0000104904174805 - rollout_corr/rollout_is_ratio_fraction_high:1.0702741747081745e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.417339324951172 - rollout_corr/rollout_is_min:0.6783297061920166 - rollout_corr/rollout_is_std:0.03459395840764046 - rollout_corr/rollout_is_eff_sample_size:0.9988045694973763 - rollout_corr/rollout_is_seq_mean:0.9999669790267944 - rollout_corr/rollout_is_seq_std:0.0015490534715354443 - rollout_corr/rollout_is_seq_max:1.004462480545044 - rollout_corr/rollout_is_seq_min:0.9947811365127563 - rollout_corr/rollout_is_seq_max_deviation:0.005218863487243652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0522580044344068) - rollout_corr/training_log_ppl:np.float64(0.7055484902230091) - rollout_corr/kl:np.float64(0.0007327849301645983) - rollout_corr/k3_kl:np.float64(0.0006532650920689775) - rollout_corr/rollout_ppl:np.float64(2.0508041838184) - rollout_corr/rollout_log_ppl:np.float64(0.7048157050157897) - rollout_corr/log_ppl_diff:np.float64(0.0007327852072194219) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013509184354916215) - rollout_corr/log_ppl_diff_max:np.float64(0.006094217300415039) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032320618629455566) - rollout_corr/ppl_ratio:np.float64(1.0007343096658587) - rollout_corr/chi2_token:np.float64(0.0011649802327156067) - rollout_corr/chi2_seq:np.float64(0.8535998552106321) - actor/pg_loss:np.float64(-7.09133455529809e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026227998409922293) - actor/ppo_kl:np.float64(7.257984136543882e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28863443806767464) - perf/mfu/actor:np.float64(0.12142939169953111) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(106.63142013549805) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010431160219013691 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010431160219013691 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:729.953125 - response_length/max:1851.0 - response_length/min:205.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:729.953125 - response_length_non_aborted/max:1851.0 - response_length_non_aborted/min:205.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.84375 - prompt_length/max:404.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010944157838821411 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5309949908405542) - timing_s/agent_loop/generate_sequences/max:np.float64(15.264747710898519) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.378220335514925) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.264747710898519) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:312 - timing_s/agent_loop/slowest/response_length:1851 - timing_s/gen:22.15535177104175 - timing_s/reward:0.06814277172088623 - timing_s/old_log_prob:3.451232196763158 - timing_s/adv:0.157595532014966 - timing_s/update_actor:13.804202174767852 - timing_s/step:39.72742600180209 - timing_s/stop_profile:0.00011063739657402039 - timing_per_token_ms/adv:0.0006200740175914242 - timing_per_token_ms/gen:0.11856150743327777 - timing_per_token_ms/update_actor:0.054313894516627 - perf/total_num_tokens:254156 - perf/time_per_step:39.72742600180209 - perf/throughput:799.6868460231703 (TaskRunner pid=2224264) Training Progress: 32%|███▏ | 32/100 [29:53<1:21:31, 71.93s/it] (TaskRunner pid=2224264) step:33 - global_seqlen/min:26534 - global_seqlen/max:38525 - global_seqlen/minmax_diff:11991 - global_seqlen/balanced_min:31307 - global_seqlen/balanced_max:31349 - global_seqlen/mean:31335.875 - actor/entropy:0.7530014514923096 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40599530935287476 - training/rollout_probs_diff_mean:0.004578310996294022 - training/rollout_probs_diff_std:0.00858127698302269 - training/rollout_actor_probs_pearson_corr:0.9995956420898438 - rollout_corr/rollout_is_mean:0.9998608231544495 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.69842768527451e-06 - rollout_corr/rollout_is_max:1.6179550886154175 - rollout_corr/rollout_is_min:0.4878464341163635 - rollout_corr/rollout_is_std:0.03484460711479187 - rollout_corr/rollout_is_eff_sample_size:0.9987870879096347 - rollout_corr/rollout_is_seq_mean:0.9998835921287537 - rollout_corr/rollout_is_seq_std:0.0015449043130502105 - rollout_corr/rollout_is_seq_max:1.003616213798523 - rollout_corr/rollout_is_seq_min:0.9941043257713318 - rollout_corr/rollout_is_seq_max_deviation:0.005895674228668213 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1979672331362963) - rollout_corr/training_log_ppl:np.float64(0.7705774857313372) - rollout_corr/kl:np.float64(0.0007750543829923018) - rollout_corr/k3_kl:np.float64(0.0006546921346171075) - rollout_corr/rollout_ppl:np.float64(2.1962045463733375) - rollout_corr/rollout_log_ppl:np.float64(0.7698024294222705) - rollout_corr/log_ppl_diff:np.float64(0.0007750563090667129) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013818222796544433) - rollout_corr/log_ppl_diff_max:np.float64(0.006634533405303955) - rollout_corr/log_ppl_diff_min:np.float64(-0.004233121871948242) - rollout_corr/ppl_ratio:np.float64(1.0007766336202621) - rollout_corr/chi2_token:np.float64(0.0010767022613435984) - rollout_corr/chi2_seq:np.float64(0.8498944872990251) - actor/pg_loss:np.float64(-0.0003114178543910384) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014040332393960853) - actor/ppo_kl:np.float64(3.355153801543054e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2854798920452595) - perf/mfu/actor:np.float64(0.12016219905694546) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.70193099975586) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013369936496019363 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013369936496019363 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:685.49609375 - response_length/max:1829.0 - response_length/min:195.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:685.49609375 - response_length_non_aborted/max:1829.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.75 - prompt_length/max:474.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012979470193386078 - timing_s/agent_loop/generate_sequences/min:np.float64(2.423111606389284) - timing_s/agent_loop/generate_sequences/max:np.float64(15.327023109421134) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.73543391644489) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.327023109421134) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:1829 - timing_s/gen:22.37846451252699 - timing_s/reward:0.07666251063346863 - timing_s/old_log_prob:3.2687183786183596 - timing_s/adv:0.12963519990444183 - timing_s/update_actor:13.795639680698514 - timing_s/step:39.736858043819666 - timing_s/stop_profile:0.00010845810174942017 - timing_per_token_ms/adv:0.0005171197545323125 - timing_per_token_ms/gen:0.12752206438384034 - timing_per_token_ms/update_actor:0.05503133262075223 - perf/total_num_tokens:250687 - perf/time_per_step:39.736858043819666 - perf/throughput:788.5846174713785 (TaskRunner pid=2224264) Training Progress: 33%|███▎ | 33/100 [30:33<1:09:33, 62.29s/it] (TaskRunner pid=2224264) step:34 - global_seqlen/min:31672 - global_seqlen/max:34621 - global_seqlen/minmax_diff:2949 - global_seqlen/balanced_min:33347 - global_seqlen/balanced_max:33381 - global_seqlen/mean:33365.5 - actor/entropy:0.7147052884101868 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26073455810546875 - training/rollout_probs_diff_mean:0.004393857438117266 - training/rollout_probs_diff_std:0.008422989398241043 - training/rollout_actor_probs_pearson_corr:0.9996627569198608 - rollout_corr/rollout_is_mean:1.0000426769256592 - rollout_corr/rollout_is_ratio_fraction_high:1.0432097042212263e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:5.339309215545654 - rollout_corr/rollout_is_min:0.5086534023284912 - rollout_corr/rollout_is_std:0.03433627635240555 - rollout_corr/rollout_is_eff_sample_size:0.99882252741175 - rollout_corr/rollout_is_seq_mean:1.0001329183578491 - rollout_corr/rollout_is_seq_std:0.001656638691201806 - rollout_corr/rollout_is_seq_max:1.0120186805725098 - rollout_corr/rollout_is_seq_min:0.9953494668006897 - rollout_corr/rollout_is_seq_max_deviation:0.012018680572509766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.108749763574451) - rollout_corr/training_log_ppl:np.float64(0.7289996044710279) - rollout_corr/kl:np.float64(0.0005081289243759057) - rollout_corr/k3_kl:np.float64(0.0006798729952492977) - rollout_corr/rollout_ppl:np.float64(2.107720549684018) - rollout_corr/rollout_log_ppl:np.float64(0.7284914773190394) - rollout_corr/log_ppl_diff:np.float64(0.0005081271519884467) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012719499645754695) - rollout_corr/log_ppl_diff_max:np.float64(0.0048035383224487305) - rollout_corr/log_ppl_diff_min:np.float64(-0.006281495094299316) - rollout_corr/ppl_ratio:np.float64(1.0005095447413623) - rollout_corr/chi2_token:np.float64(0.0017949678003787994) - rollout_corr/chi2_seq:np.float64(4.14957369142212) - actor/pg_loss:np.float64(-5.571101792156696e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020079502564840368) - actor/ppo_kl:np.float64(8.411469075042532e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3010113313794136) - perf/mfu/actor:np.float64(0.12801561879083337) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.21452951431274) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007644771598279476 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007644771598279476 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:748.890625 - response_length/max:2008.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:748.890625 - response_length_non_aborted/max:2008.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.78125 - prompt_length/max:621.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014275126159191132 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1188270319253206) - timing_s/agent_loop/generate_sequences/max:np.float64(16.865083750337362) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.53478015192377) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.865083750337362) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:201 - timing_s/agent_loop/slowest/response_length:2008 - timing_s/gen:23.755706263706088 - timing_s/reward:0.08198409341275692 - timing_s/old_log_prob:3.3104164469987154 - timing_s/adv:0.14668767154216766 - timing_s/update_actor:13.706282814964652 - timing_s/step:41.08737375959754 - timing_s/stop_profile:0.00011901184916496277 - timing_per_token_ms/adv:0.0005495484540249946 - timing_per_token_ms/gen:0.12391092169514327 - timing_per_token_ms/update_actor:0.05134900876266148 - perf/total_num_tokens:266924 - perf/time_per_step:41.08737375959754 - perf/throughput:812.0621238831601 (TaskRunner pid=2224264) Training Progress: 34%|███▍ | 34/100 [31:14<1:01:32, 55.95s/it] (TaskRunner pid=2224264) step:35 - global_seqlen/min:28121 - global_seqlen/max:32901 - global_seqlen/minmax_diff:4780 - global_seqlen/balanced_min:30268 - global_seqlen/balanced_max:30287 - global_seqlen/mean:30280.375 - actor/entropy:0.7235326170921326 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21305981278419495 - training/rollout_probs_diff_mean:0.00439543928951025 - training/rollout_probs_diff_std:0.00852300226688385 - training/rollout_actor_probs_pearson_corr:0.9996048808097839 - rollout_corr/rollout_is_mean:1.0000625848770142 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.8470300245971885e-06 - rollout_corr/rollout_is_max:1.7555011510849 - rollout_corr/rollout_is_min:0.4614633619785309 - rollout_corr/rollout_is_std:0.03465937077999115 - rollout_corr/rollout_is_eff_sample_size:0.9988002882356332 - rollout_corr/rollout_is_seq_mean:1.0000625848770142 - rollout_corr/rollout_is_seq_std:0.001464530942030251 - rollout_corr/rollout_is_seq_max:1.0040720701217651 - rollout_corr/rollout_is_seq_min:0.9944659471511841 - rollout_corr/rollout_is_seq_max_deviation:0.005534052848815918 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0981541997753084) - rollout_corr/training_log_ppl:np.float64(0.7273534100968391) - rollout_corr/kl:np.float64(0.0005946585815870264) - rollout_corr/k3_kl:np.float64(0.0006194554455305479) - rollout_corr/rollout_ppl:np.float64(2.0969019741751254) - rollout_corr/rollout_log_ppl:np.float64(0.7267587499227375) - rollout_corr/log_ppl_diff:np.float64(0.0005946601741015911) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012836407404392958) - rollout_corr/log_ppl_diff_max:np.float64(0.005986064672470093) - rollout_corr/log_ppl_diff_min:np.float64(-0.003438234329223633) - rollout_corr/ppl_ratio:np.float64(1.000595951685682) - rollout_corr/chi2_token:np.float64(0.001301520736888051) - rollout_corr/chi2_seq:np.float64(1.3097328948788345) - actor/pg_loss:np.float64(0.00015541678294539452) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001370309196317976) - actor/ppo_kl:np.float64(4.7729094319493015e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2635045759379864) - perf/mfu/actor:np.float64(0.11788960362270617) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.97077560424805) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020808118861168623 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020808118861168623 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:668.07421875 - response_length/max:1568.0 - response_length/min:256.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:668.07421875 - response_length_non_aborted/max:1568.0 - response_length_non_aborted/min:256.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.1875 - prompt_length/max:514.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001488998532295227 - timing_s/agent_loop/generate_sequences/min:np.float64(3.3336482755839825) - timing_s/agent_loop/generate_sequences/max:np.float64(13.82805041782558) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.910963898211776) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.82805041782558) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:218 - timing_s/agent_loop/slowest/response_length:1568 - timing_s/gen:20.717208752408624 - timing_s/reward:0.07455052435398102 - timing_s/old_log_prob:3.3136789835989475 - timing_s/adv:0.15318860113620758 - timing_s/update_actor:13.571898808702826 - timing_s/step:37.9156019538641 - timing_s/stop_profile:0.00011204928159713745 - timing_per_token_ms/adv:0.0006323757596141378 - timing_per_token_ms/gen:0.12113414111461128 - timing_per_token_ms/update_actor:0.056025969000973515 - perf/total_num_tokens:242243 - perf/time_per_step:37.9156019538641 - perf/throughput:798.6257223832373 (TaskRunner pid=2224264) Training Progress: 35%|███▌ | 35/100 [31:52<54:46, 50.56s/it] (TaskRunner pid=2224264) step:36 - global_seqlen/min:23484 - global_seqlen/max:37627 - global_seqlen/minmax_diff:14143 - global_seqlen/balanced_min:32177 - global_seqlen/balanced_max:32196 - global_seqlen/mean:32190.375 - actor/entropy:0.7190540432929993 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9983694553375244 - training/rollout_probs_diff_mean:0.0044546606950461864 - training/rollout_probs_diff_std:0.008799256756901741 - training/rollout_actor_probs_pearson_corr:0.999652624130249 - rollout_corr/rollout_is_mean:0.9999595880508423 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.36610241397284e-06 - rollout_corr/rollout_is_max:1.631447434425354 - rollout_corr/rollout_is_min:0.00012358058302197605 - rollout_corr/rollout_is_std:0.03424588963389397 - rollout_corr/rollout_is_eff_sample_size:0.9988282360231038 - rollout_corr/rollout_is_seq_mean:0.9999497532844543 - rollout_corr/rollout_is_seq_std:0.0013012057170271873 - rollout_corr/rollout_is_seq_max:1.005060076713562 - rollout_corr/rollout_is_seq_min:0.9961336851119995 - rollout_corr/rollout_is_seq_max_deviation:0.005060076713562012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0887983799912035) - rollout_corr/training_log_ppl:np.float64(0.7234805303742178) - rollout_corr/kl:np.float64(0.0007222782085847257) - rollout_corr/k3_kl:np.float64(0.000665171026241751) - rollout_corr/rollout_ppl:np.float64(2.0872840681113303) - rollout_corr/rollout_log_ppl:np.float64(0.7227582528139465) - rollout_corr/log_ppl_diff:np.float64(0.0007222775602713227) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012800676049664617) - rollout_corr/log_ppl_diff_max:np.float64(0.014325737953186035) - rollout_corr/log_ppl_diff_min:np.float64(-0.004422605037689209) - rollout_corr/ppl_ratio:np.float64(1.0007238842081279) - rollout_corr/chi2_token:np.float64(0.001139611704275012) - rollout_corr/chi2_seq:np.float64(2.33724519261159) - actor/pg_loss:np.float64(7.578812073916197e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016148608483490534) - actor/ppo_kl:np.float64(3.428134950134165e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3038923889398575) - perf/mfu/actor:np.float64(0.12476615328489614) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.85765075683594) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017614230746403337 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017614230746403337 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:727.94921875 - response_length/max:1574.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:727.94921875 - response_length_non_aborted/max:1574.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.0 - prompt_length/max:704.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010665319859981537 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2873131558299065) - timing_s/agent_loop/generate_sequences/max:np.float64(14.295806081965566) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.435247752218856) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.295806081965566) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:193 - timing_s/agent_loop/slowest/response_length:1574 - timing_s/gen:20.98940148949623 - timing_s/reward:0.07632862776517868 - timing_s/old_log_prob:3.21400635689497 - timing_s/adv:0.11980516463518143 - timing_s/update_actor:13.706334153190255 - timing_s/step:38.190680116415024 - timing_s/stop_profile:0.0001220405101776123 - timing_per_token_ms/adv:0.0004652212215420814 - timing_per_token_ms/gen:0.11263127627107526 - timing_per_token_ms/update_actor:0.05322372818424084 - perf/total_num_tokens:257523 - perf/time_per_step:38.190680116415024 - perf/throughput:842.8856176919461 (TaskRunner pid=2224264) Training Progress: 36%|███▌ | 36/100 [32:30<49:59, 46.86s/it] (TaskRunner pid=2224264) step:37 - global_seqlen/min:27444 - global_seqlen/max:33696 - global_seqlen/minmax_diff:6252 - global_seqlen/balanced_min:31074 - global_seqlen/balanced_max:31120 - global_seqlen/mean:31108.25 - actor/entropy:0.749553382396698 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1951146274805069 - training/rollout_probs_diff_mean:0.004549640696495771 - training/rollout_probs_diff_std:0.008537894114851952 - training/rollout_actor_probs_pearson_corr:0.9996463060379028 - rollout_corr/rollout_is_mean:0.9998870491981506 - rollout_corr/rollout_is_ratio_fraction_high:5.924381184740923e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0695595741271973 - rollout_corr/rollout_is_min:0.5410090088844299 - rollout_corr/rollout_is_std:0.035030562430620193 - rollout_corr/rollout_is_eff_sample_size:0.9987741257668857 - rollout_corr/rollout_is_seq_mean:0.9999449849128723 - rollout_corr/rollout_is_seq_std:0.001592667424120009 - rollout_corr/rollout_is_seq_max:1.0082616806030273 - rollout_corr/rollout_is_seq_min:0.9951016306877136 - rollout_corr/rollout_is_seq_max_deviation:0.008261680603027344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2040511020459235) - rollout_corr/training_log_ppl:np.float64(0.7697590510360897) - rollout_corr/kl:np.float64(0.0007292848855877665) - rollout_corr/k3_kl:np.float64(0.0006632836551716537) - rollout_corr/rollout_ppl:np.float64(2.202401583082974) - rollout_corr/rollout_log_ppl:np.float64(0.769029765157029) - rollout_corr/log_ppl_diff:np.float64(0.000729285879060626) - rollout_corr/log_ppl_abs_diff:np.float64(0.001381295034661889) - rollout_corr/log_ppl_diff_max:np.float64(0.005849003791809082) - rollout_corr/log_ppl_diff_min:np.float64(-0.005055129528045654) - rollout_corr/ppl_ratio:np.float64(1.0007309019565582) - rollout_corr/chi2_token:np.float64(0.001207549823448062) - rollout_corr/chi2_seq:np.float64(0.7918412457220256) - actor/pg_loss:np.float64(-0.0002318680053576827) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019654674611047085) - actor/ppo_kl:np.float64(3.777838127683175e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3199927434325218) - perf/mfu/actor:np.float64(0.11945996157211544) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.98107528686523) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011995390988886356 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011995390988886356 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:659.3515625 - response_length/max:2064.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:659.3515625 - response_length_non_aborted/max:2064.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.78125 - prompt_length/max:1009.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015878677368164062 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0059274714440107) - timing_s/agent_loop/generate_sequences/max:np.float64(16.902138613164425) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.578634077712195) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.902138613164425) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:171 - timing_s/agent_loop/slowest/response_length:2064 - timing_s/gen:23.681104212999344 - timing_s/reward:0.08328941836953163 - timing_s/old_log_prob:3.3095543030649424 - timing_s/adv:0.13988415524363518 - timing_s/update_actor:13.724779484793544 - timing_s/step:41.0210815500468 - timing_s/stop_profile:0.0001230333000421524 - timing_per_token_ms/adv:0.0005620862441781327 - timing_per_token_ms/gen:0.14029588855646138 - timing_per_token_ms/update_actor:0.05514927505080462 - perf/total_num_tokens:248866 - perf/time_per_step:41.0210815500468 - perf/throughput:758.3478744227432 (TaskRunner pid=2224264) Training Progress: 37%|███▋ | 37/100 [33:11<47:22, 45.12s/it] (TaskRunner pid=2224264) step:38 - global_seqlen/min:22559 - global_seqlen/max:30637 - global_seqlen/minmax_diff:8078 - global_seqlen/balanced_min:27684 - global_seqlen/balanced_max:27696 - global_seqlen/mean:27690.75 - actor/entropy:0.7535861730575562 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36917543411254883 - training/rollout_probs_diff_mean:0.004689306020736694 - training/rollout_probs_diff_std:0.008696562610566616 - training/rollout_actor_probs_pearson_corr:0.9996323585510254 - rollout_corr/rollout_is_mean:1.0000420808792114 - rollout_corr/rollout_is_ratio_fraction_high:6.4868513618421275e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0322699546813965 - rollout_corr/rollout_is_min:0.5430662631988525 - rollout_corr/rollout_is_std:0.03519861027598381 - rollout_corr/rollout_is_eff_sample_size:0.9987627098464102 - rollout_corr/rollout_is_seq_mean:1.0000241994857788 - rollout_corr/rollout_is_seq_std:0.0014864748809486628 - rollout_corr/rollout_is_seq_max:1.0049471855163574 - rollout_corr/rollout_is_seq_min:0.995693564414978 - rollout_corr/rollout_is_seq_max_deviation:0.004947185516357422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1352647584863007) - rollout_corr/training_log_ppl:np.float64(0.7448823321610689) - rollout_corr/kl:np.float64(0.0005967357578882115) - rollout_corr/k3_kl:np.float64(0.0006872201306578063) - rollout_corr/rollout_ppl:np.float64(2.1338966507464647) - rollout_corr/rollout_log_ppl:np.float64(0.7442855968838558) - rollout_corr/log_ppl_diff:np.float64(0.0005967352772131562) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013833154225721955) - rollout_corr/log_ppl_diff_max:np.float64(0.0049326419830322266) - rollout_corr/log_ppl_diff_min:np.float64(-0.004489302635192871) - rollout_corr/ppl_ratio:np.float64(1.0005982399452478) - rollout_corr/chi2_token:np.float64(0.0015692522283643484) - rollout_corr/chi2_seq:np.float64(1.3063512451481074) - actor/pg_loss:np.float64(-0.00012253155000507832) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015133037595660426) - actor/ppo_kl:np.float64(-1.1263918693771302e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27262789383530617) - perf/mfu/actor:np.float64(0.10989664398239996) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.94292449951172) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031607181299477816 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031607181299477816 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:602.1796875 - response_length/max:1274.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:602.1796875 - response_length_non_aborted/max:1274.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.15625 - prompt_length/max:500.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.338022649288177e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9933321122080088) - timing_s/agent_loop/generate_sequences/max:np.float64(11.328456040471792) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.766713866491045) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.328456040471792) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:1274 - timing_s/gen:17.93396664969623 - timing_s/reward:0.07525189034640789 - timing_s/old_log_prob:3.114859651774168 - timing_s/adv:0.12139961495995522 - timing_s/update_actor:12.985273517668247 - timing_s/step:34.3180456738919 - timing_s/stop_profile:0.00011270120739936829 - timing_per_token_ms/adv:0.0005480151989380714 - timing_per_token_ms/gen:0.11633497223430656 - timing_per_token_ms/update_actor:0.05861737907815898 - perf/total_num_tokens:221526 - perf/time_per_step:34.3180456738919 - perf/throughput:806.8859824691666 (TaskRunner pid=2224264) Training Progress: 38%|███▊ | 38/100 [33:45<43:17, 41.90s/it] (TaskRunner pid=2224264) step:39 - global_seqlen/min:24673 - global_seqlen/max:36395 - global_seqlen/minmax_diff:11722 - global_seqlen/balanced_min:29976 - global_seqlen/balanced_max:30014 - global_seqlen/mean:30001.0 - actor/entropy:0.7436038255691528 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39769014716148376 - training/rollout_probs_diff_mean:0.004428871441632509 - training/rollout_probs_diff_std:0.008462125435471535 - training/rollout_actor_probs_pearson_corr:0.9996308088302612 - rollout_corr/rollout_is_mean:0.9998753666877747 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.558502197265625 - rollout_corr/rollout_is_min:0.5394280552864075 - rollout_corr/rollout_is_std:0.03459912911057472 - rollout_corr/rollout_is_eff_sample_size:0.9988039748754949 - rollout_corr/rollout_is_seq_mean:0.9998376965522766 - rollout_corr/rollout_is_seq_std:0.0016168290749192238 - rollout_corr/rollout_is_seq_max:1.0040711164474487 - rollout_corr/rollout_is_seq_min:0.9940407276153564 - rollout_corr/rollout_is_seq_max_deviation:0.005959272384643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.18053255463019) - rollout_corr/training_log_ppl:np.float64(0.7624029302969575) - rollout_corr/kl:np.float64(0.0008376638515699142) - rollout_corr/k3_kl:np.float64(0.0006715801446830483) - rollout_corr/rollout_ppl:np.float64(2.178553344681859) - rollout_corr/rollout_log_ppl:np.float64(0.7615652666427195) - rollout_corr/log_ppl_diff:np.float64(0.0008376636542379856) - rollout_corr/log_ppl_abs_diff:np.float64(0.001551654888316989) - rollout_corr/log_ppl_diff_max:np.float64(0.00657200813293457) - rollout_corr/log_ppl_diff_min:np.float64(-0.004390835762023926) - rollout_corr/ppl_ratio:np.float64(1.0008396471384913) - rollout_corr/chi2_token:np.float64(0.0010091052390635014) - rollout_corr/chi2_seq:np.float64(2.287320557516068) - actor/pg_loss:np.float64(-9.888317435979843e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001992107047499303) - actor/ppo_kl:np.float64(5.7301584355684554e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29527168720960617) - perf/mfu/actor:np.float64(0.11556263551770317) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.95692443847656) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002959122182801366 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002959122182801366 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:655.25 - response_length/max:1746.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:655.25 - response_length_non_aborted/max:1746.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:452.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001086648553609848 - timing_s/agent_loop/generate_sequences/min:np.float64(2.099978694692254) - timing_s/agent_loop/generate_sequences/max:np.float64(14.446734165772796) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.490971767700103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.446734165772796) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1746 - timing_s/gen:21.59979004599154 - timing_s/reward:0.07381308265030384 - timing_s/old_log_prob:3.1907936204224825 - timing_s/adv:0.1207788810133934 - timing_s/update_actor:13.672080148011446 - timing_s/step:38.74228004924953 - timing_s/stop_profile:0.00011139921844005585 - timing_per_token_ms/adv:0.0005032285632703635 - timing_per_token_ms/gen:0.12876639430317352 - timing_per_token_ms/update_actor:0.056965101779988356 - perf/total_num_tokens:240008 - perf/time_per_step:38.74228004924953 - perf/throughput:774.3736290652606 (TaskRunner pid=2224264) Training Progress: 39%|███▉ | 39/100 [34:24<41:38, 40.96s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a target product: (TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"** (TaskRunner pid=2224264) This is a nitrogen-containing compound with a specific substituent pattern. Let's break down the structure: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - The core is a **pyrrole ring** (N connected to a carbon ring). (TaskRunner pid=2224264) - The nitrogen is attached to a **carbon chain** that leads to: (TaskRunner pid=2224264) - A **chloro-substituted benzene ring** (from "Cc2ccc(Cl)cc2C(F)(F)F") (TaskRunner pid=2224264) - A **trifluoromethyl group** (C(F)(F)F) (TaskRunner pid=2224264) (TaskRunner pid=2224264) So the structure is: (TaskRunner pid=2224264) **N–C–(–CH₂–C(Cl)–C(F)(F)F–)** (TaskRunner pid=2224264) (TaskRunner pid=2224264) Now, we are to identify the **correct reactant** that could be used in the synthesis of this compound. Let's evaluate each option. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option A: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This appears to be a **pyrrole derivative** with a **carbonyl group** (O=C) and a **trifluoromethyl group**. The structure seems to match the target product. The **"c1ccn(...)"** part resembles the pyrrole ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a **strong candidate** for the correct reactant. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option B: (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **chlorine atom** on the ring and a **trifluoromethyl** group? Not directly. The structure is more complex and does not clearly match the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option C: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This includes a **trifluoromethyl group** and a **chloro-substituted benzene**, but the **N** is not directly attached to the **chloro-substituted benzene**. The structure is more complex than the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option D: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **trifluoromethyl group** and a **benzene ring**, but the **N** is not directly attached to the **chloro-substituted benzene**. The structure is also more complex than the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Final Evaluation: (TaskRunner pid=2224264) - **Option A** contains a **pyrrole ring** with a **chloro-substituted benzene** and a **trifluoromethyl group**, which matches the structure of the target product. (TaskRunner pid=2224264) - The other options either lack the correct substituent pattern or have more complex structures that do not align with the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### ✅ **Correct Answer: A** (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2224264) step:40 - global_seqlen/min:26780 - global_seqlen/max:38560 - global_seqlen/minmax_diff:11780 - global_seqlen/balanced_min:31398 - global_seqlen/balanced_max:31614 - global_seqlen/mean:31430.375 - actor/entropy:0.7148814797401428 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.353511780500412 - training/rollout_probs_diff_mean:0.004250292666256428 - training/rollout_probs_diff_std:0.008334573358297348 - training/rollout_actor_probs_pearson_corr:0.9996918439865112 - rollout_corr/rollout_is_mean:0.9999884963035583 - rollout_corr/rollout_is_ratio_fraction_high:5.834475814481266e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.834475814481266e-06 - rollout_corr/rollout_is_max:2.314491033554077 - rollout_corr/rollout_is_min:0.4158317446708679 - rollout_corr/rollout_is_std:0.034074895083904266 - rollout_corr/rollout_is_eff_sample_size:0.9988401291730077 - rollout_corr/rollout_is_seq_mean:0.9999082088470459 - rollout_corr/rollout_is_seq_std:0.0015310312155634165 - rollout_corr/rollout_is_seq_max:1.0046111345291138 - rollout_corr/rollout_is_seq_min:0.9931868314743042 - rollout_corr/rollout_is_seq_max_deviation:0.006813168525695801 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0881334473378956) - rollout_corr/training_log_ppl:np.float64(0.7168666583602317) - rollout_corr/kl:np.float64(0.0009168515699862212) - rollout_corr/k3_kl:np.float64(0.0007126086813400434) - rollout_corr/rollout_ppl:np.float64(2.0860950271598995) - rollout_corr/rollout_log_ppl:np.float64(0.7159498034743592) - rollout_corr/log_ppl_diff:np.float64(0.0009168548858724535) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013491588761098683) - rollout_corr/log_ppl_diff_max:np.float64(0.006253480911254883) - rollout_corr/log_ppl_diff_min:np.float64(-0.003960609436035156) - rollout_corr/ppl_ratio:np.float64(1.0009184894151986) - rollout_corr/chi2_token:np.float64(0.0010311168152838945) - rollout_corr/chi2_seq:np.float64(0.5281695865560323) - actor/pg_loss:np.float64(0.0002642440376803279) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030195120598364156) - actor/ppo_kl:np.float64(0.00023282111740385858) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26611069589853287) - perf/mfu/actor:np.float64(0.12153132959867792) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(103.97843170166016) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6053571428571428) - val-aux/sciknoweval/reward/std@16:np.float64(0.28468084712859354) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7211619047619049) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22140257031009367) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.4908571428571428) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.25871458688057813) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6054666666666666) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.28424403856079056) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8044809523809524) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15305264517858658) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.3869142857142857) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.2100138722335839) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6357761904761906) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22516406707067232) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8634285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09899214217926788) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3012904761904762) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.15568733164559595) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6521095238095237) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15983092148992065) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9010238095238096) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.056177528972713935) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.23781428571428573) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.09852453096057993) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6605380952380953) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11372781200461213) - val-aux/sciknoweval/score/mean@16:np.float64(0.6053571428571428) - val-aux/sciknoweval/score/std@16:np.float64(0.28468084712859354) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7211619047619049) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22140257031009367) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.4908571428571428) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.25871458688057813) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6054666666666666) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.28424403856079056) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8044809523809524) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15305264517858658) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.3869142857142857) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.2100138722335839) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6357761904761906) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.22516406707067232) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8634285714285714) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09899214217926788) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3012904761904762) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.15568733164559595) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6521095238095237) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15983092148992065) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9010238095238096) - val-aux/sciknoweval/score/best@16/std:np.float64(0.056177528972713935) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.23781428571428573) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.09852453096057993) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6605380952380953) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11372781200461213) - val-core/sciknoweval/acc/mean@16:np.float64(0.6053571428571428) - val-aux/sciknoweval/acc/std@16:np.float64(0.28468084712859354) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7211619047619049) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22140257031009367) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.4908571428571428) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.25871458688057813) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6054666666666666) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.28424403856079056) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8044809523809524) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15305264517858658) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.3869142857142857) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.2100138722335839) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6357761904761906) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.22516406707067232) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8634285714285714) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09899214217926788) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3012904761904762) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.15568733164559595) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6521095238095237) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15983092148992065) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9010238095238096) - val-core/sciknoweval/acc/best@16/std:np.float64(0.056177528972713935) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.23781428571428573) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.09852453096057993) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6605380952380953) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11372781200461213) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.986904761904762) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.036439037613581124) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9981857142857142) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0128589514254664) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9762047619047618) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04654134805312809) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9871476190476189) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.036109563011539794) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999333333333333) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0014234611985593974) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9578047619047619) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05537427217213782) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9950380952380953) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.022826156373882658) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9323095238095238) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05709074252726499) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9987190476190476) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010923127179237949) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9055857142857143) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04723161312584812) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998666666666666) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002542800942671283) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004572770558297634 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004572770558297634 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:669.51171875 - response_length/max:1961.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:669.51171875 - response_length_non_aborted/max:1961.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.6875 - prompt_length/max:501.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011765025556087494 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0561186894774437) - timing_s/agent_loop/generate_sequences/max:np.float64(16.069326899945736) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.7381278641551035) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.069326899945736) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:423 - timing_s/agent_loop/slowest/response_length:1961 - timing_s/gen:22.699881782755256 - timing_s/reward:0.07818994671106339 - timing_s/old_log_prob:3.2518085297197104 - timing_s/adv:0.13994908705353737 - timing_s/update_actor:13.636622594669461 - timing_s/step:39.892101377248764 - timing_s/testing:188.02605186402798 - timing_s/save_checkpoint:13.08927645906806 - timing_s/stop_profile:0.00019207783043384552 - timing_per_token_ms/adv:0.0005565837468274613 - timing_per_token_ms/gen:0.1324419136074871 - timing_per_token_ms/update_actor:0.054233454877127066 - perf/total_num_tokens:251443 - perf/time_per_step:39.892101377248764 - perf/throughput:787.8846667607576 (TaskRunner pid=2224264) Training Progress: 40%|████ | 40/100 [38:25<1:41:00, 101.00s/it] (TaskRunner pid=2224264) step:41 - global_seqlen/min:25184 - global_seqlen/max:36473 - global_seqlen/minmax_diff:11289 - global_seqlen/balanced_min:30315 - global_seqlen/balanced_max:30362 - global_seqlen/mean:30349.125 - actor/entropy:0.7456910610198975 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1853349208831787 - training/rollout_probs_diff_mean:0.004585789050906897 - training/rollout_probs_diff_std:0.008587997406721115 - training/rollout_actor_probs_pearson_corr:0.9996112585067749 - rollout_corr/rollout_is_mean:1.000074863433838 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1413375432312023e-05 - rollout_corr/rollout_is_max:1.7971372604370117 - rollout_corr/rollout_is_min:0.38024628162384033 - rollout_corr/rollout_is_std:0.0351342037320137 - rollout_corr/rollout_is_eff_sample_size:0.9987673475326293 - rollout_corr/rollout_is_seq_mean:1.000067949295044 - rollout_corr/rollout_is_seq_std:0.0015586130321025848 - rollout_corr/rollout_is_seq_max:1.005883812904358 - rollout_corr/rollout_is_seq_min:0.993992805480957 - rollout_corr/rollout_is_seq_max_deviation:0.006007194519042969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1489445390179753) - rollout_corr/training_log_ppl:np.float64(0.7491948831593618) - rollout_corr/kl:np.float64(0.0007140677866042111) - rollout_corr/k3_kl:np.float64(0.000693655981081065) - rollout_corr/rollout_ppl:np.float64(2.147350816987455) - rollout_corr/rollout_log_ppl:np.float64(0.7484808203298599) - rollout_corr/log_ppl_diff:np.float64(0.0007140628295019269) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013865955406799912) - rollout_corr/log_ppl_diff_max:np.float64(0.00584721565246582) - rollout_corr/log_ppl_diff_min:np.float64(-0.004841506481170654) - rollout_corr/ppl_ratio:np.float64(1.0007157281506807) - rollout_corr/chi2_token:np.float64(0.0013592832256108522) - rollout_corr/chi2_seq:np.float64(1.1909501447807997) - actor/pg_loss:np.float64(0.000211457721889019) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024292130228786846) - actor/ppo_kl:np.float64(0.00014254798117097067) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2715475298464298) - perf/mfu/actor:np.float64(0.11821237532599987) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(105.00201988220215) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00029318107408471406 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00029318107408471406 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:684.50390625 - response_length/max:1922.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:684.50390625 - response_length_non_aborted/max:1922.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.90625 - prompt_length/max:631.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.857250213623047e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0997980404645205) - timing_s/agent_loop/generate_sequences/max:np.float64(15.434562433511019) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.771742777309555) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.434562433511019) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:245 - timing_s/agent_loop/slowest/response_length:1922 - timing_s/gen:22.271852983161807 - timing_s/reward:0.07384450361132622 - timing_s/old_log_prob:3.3064878918230534 - timing_s/adv:0.13560799323022366 - timing_s/update_actor:13.53317659534514 - timing_s/step:39.414824433624744 - timing_s/stop_profile:0.00011513754725456238 - timing_per_token_ms/adv:0.0005585333729976715 - timing_per_token_ms/gen:0.12709850874642223 - timing_per_token_ms/update_actor:0.055739566607542805 - perf/total_num_tokens:242793 - perf/time_per_step:39.414824433624744 - perf/throughput:769.9926470840549 (TaskRunner pid=2224264) Training Progress: 41%|████ | 41/100 [39:05<1:21:09, 82.54s/it] (TaskRunner pid=2224264) step:42 - global_seqlen/min:23559 - global_seqlen/max:32248 - global_seqlen/minmax_diff:8689 - global_seqlen/balanced_min:29140 - global_seqlen/balanced_max:29158 - global_seqlen/mean:29150.625 - actor/entropy:0.7418811321258545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8146259784698486 - training/rollout_probs_diff_mean:0.004636453464627266 - training/rollout_probs_diff_std:0.008857984095811844 - training/rollout_actor_probs_pearson_corr:0.9996300339698792 - rollout_corr/rollout_is_mean:0.9999567270278931 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.9698286349885166e-06 - rollout_corr/rollout_is_max:1.5637112855911255 - rollout_corr/rollout_is_min:0.1537874937057495 - rollout_corr/rollout_is_std:0.03467312455177307 - rollout_corr/rollout_is_eff_sample_size:0.9987990990027735 - rollout_corr/rollout_is_seq_mean:0.9999449849128723 - rollout_corr/rollout_is_seq_std:0.0014816136099398136 - rollout_corr/rollout_is_seq_max:1.006678581237793 - rollout_corr/rollout_is_seq_min:0.9932494759559631 - rollout_corr/rollout_is_seq_max_deviation:0.006750524044036865 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.149658269248903) - rollout_corr/training_log_ppl:np.float64(0.750017037265934) - rollout_corr/kl:np.float64(0.0007465042794017052) - rollout_corr/k3_kl:np.float64(0.0006643551571414719) - rollout_corr/rollout_ppl:np.float64(2.148035450838506) - rollout_corr/rollout_log_ppl:np.float64(0.7492705312324688) - rollout_corr/log_ppl_diff:np.float64(0.0007465060334652662) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012925199698656797) - rollout_corr/log_ppl_diff_max:np.float64(0.005987584590911865) - rollout_corr/log_ppl_diff_min:np.float64(-0.00581049919128418) - rollout_corr/ppl_ratio:np.float64(1.0007479563355446) - rollout_corr/chi2_token:np.float64(0.0011864088010042906) - rollout_corr/chi2_seq:np.float64(0.5256321090273559) - actor/pg_loss:np.float64(-3.93550144508481e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024687639711373777) - actor/ppo_kl:np.float64(6.906396298589357e-05) - actor/pg_clipfrac_lower:np.float64(8.510348379786592e-06) - actor/grad_norm:np.float64(0.31687242165207863) - perf/mfu/actor:np.float64(0.11438361389153799) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.63388013839722) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007985391654074192 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007985391654074192 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:654.33203125 - response_length/max:1379.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:654.33203125 - response_length_non_aborted/max:1379.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:256.625 - prompt_length/max:557.0 - prompt_length/min:175.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015127286314964294 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8712534438818693) - timing_s/agent_loop/generate_sequences/max:np.float64(12.603244848549366) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.466689185537689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.603244848549366) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:1379 - timing_s/gen:20.526346927508712 - timing_s/reward:0.07452544383704662 - timing_s/old_log_prob:3.2816093750298023 - timing_s/adv:0.14070634730160236 - timing_s/update_actor:13.285876266658306 - timing_s/step:37.395900616422296 - timing_s/stop_profile:0.00011482276022434235 - timing_per_token_ms/adv:0.0006033590501987623 - timing_per_token_ms/gen:0.12253877061834714 - timing_per_token_ms/update_actor:0.056970803656260824 - perf/total_num_tokens:233205 - perf/time_per_step:37.395900616422296 - perf/throughput:779.5139178222811 (TaskRunner pid=2224264) Training Progress: 42%|████▏ | 42/100 [39:42<1:06:43, 69.02s/it] (TaskRunner pid=2224264) step:43 - global_seqlen/min:28157 - global_seqlen/max:35791 - global_seqlen/minmax_diff:7634 - global_seqlen/balanced_min:32488 - global_seqlen/balanced_max:32514 - global_seqlen/mean:32503.125 - actor/entropy:0.7455630302429199 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102365970611572 - training/rollout_probs_diff_mean:0.0044556111097335815 - training/rollout_probs_diff_std:0.008428295142948627 - training/rollout_actor_probs_pearson_corr:0.9996055960655212 - rollout_corr/rollout_is_mean:1.0000288486480713 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1921050574746914e-05 - rollout_corr/rollout_is_max:1.6338140964508057 - rollout_corr/rollout_is_min:0.3978634774684906 - rollout_corr/rollout_is_std:0.03456293046474457 - rollout_corr/rollout_is_eff_sample_size:0.9988067101420113 - rollout_corr/rollout_is_seq_mean:1.000025749206543 - rollout_corr/rollout_is_seq_std:0.0014121269341558218 - rollout_corr/rollout_is_seq_max:1.0047554969787598 - rollout_corr/rollout_is_seq_min:0.9933763146400452 - rollout_corr/rollout_is_seq_max_deviation:0.006623685359954834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1632221047766507) - rollout_corr/training_log_ppl:np.float64(0.7551403474062681) - rollout_corr/kl:np.float64(0.0007046414169096238) - rollout_corr/k3_kl:np.float64(0.0006673236663345961) - rollout_corr/rollout_ppl:np.float64(2.16164415422827) - rollout_corr/rollout_log_ppl:np.float64(0.7544357032165863) - rollout_corr/log_ppl_diff:np.float64(0.0007046441896818578) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013581206439994276) - rollout_corr/log_ppl_diff_max:np.float64(0.013478577136993408) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035314559936523438) - rollout_corr/ppl_ratio:np.float64(1.0007065150421113) - rollout_corr/chi2_token:np.float64(0.0012414606753736734) - rollout_corr/chi2_seq:np.float64(1.941952116554603) - actor/pg_loss:np.float64(6.042851600795984e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017739249074111285) - actor/ppo_kl:np.float64(0.00010932787842676817) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27033093199133873) - perf/mfu/actor:np.float64(0.12426556577467283) - perf/max_memory_allocated_gb:np.float64(131.90583896636963) - perf/max_memory_reserved_gb:np.float64(144.6015625) - perf/cpu_memory_used_gb:np.float64(104.3560562133789) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005291193723678589 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005291193723678589 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:712.78515625 - response_length/max:1722.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:712.78515625 - response_length_non_aborted/max:1722.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.9375 - prompt_length/max:813.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010757148265838623 - timing_s/agent_loop/generate_sequences/min:np.float64(2.216030417010188) - timing_s/agent_loop/generate_sequences/max:np.float64(14.333171149715781) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.09952915172471) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.333171149715781) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:1722 - timing_s/gen:21.01793995499611 - timing_s/reward:0.07846580259501934 - timing_s/old_log_prob:3.292416427284479 - timing_s/adv:0.14018669165670872 - timing_s/update_actor:13.660967510193586 - timing_s/step:38.27972553297877 - timing_s/stop_profile:0.00010936520993709564 - timing_per_token_ms/adv:0.0005391277440888712 - timing_per_token_ms/gen:0.11518383517011344 - timing_per_token_ms/update_actor:0.05253713108429415 - perf/total_num_tokens:260025 - perf/time_per_step:38.27972553297877 - perf/throughput:849.0950378418437 (TaskRunner pid=2224264) Training Progress: 43%|████▎ | 43/100 [40:21<56:49, 59.81s/it] (TaskRunner pid=2224264) step:44 - global_seqlen/min:28261 - global_seqlen/max:39735 - global_seqlen/minmax_diff:11474 - global_seqlen/balanced_min:33071 - global_seqlen/balanced_max:39250 - global_seqlen/mean:33860.375 - actor/entropy:0.7164559364318848 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3860778212547302 - training/rollout_probs_diff_mean:0.0043455152772367 - training/rollout_probs_diff_std:0.008481818251311779 - training/rollout_actor_probs_pearson_corr:0.9996494650840759 - rollout_corr/rollout_is_mean:1.0000731945037842 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.569669802847784e-05 - rollout_corr/rollout_is_max:1.6253478527069092 - rollout_corr/rollout_is_min:0.3264278769493103 - rollout_corr/rollout_is_std:0.03381501883268356 - rollout_corr/rollout_is_eff_sample_size:0.9988579694289216 - rollout_corr/rollout_is_seq_mean:1.0000576972961426 - rollout_corr/rollout_is_seq_std:0.0014711449621245265 - rollout_corr/rollout_is_seq_max:1.0052176713943481 - rollout_corr/rollout_is_seq_min:0.9947458505630493 - rollout_corr/rollout_is_seq_max_deviation:0.005254149436950684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1722452212125063) - rollout_corr/training_log_ppl:np.float64(0.7589113183676091) - rollout_corr/kl:np.float64(0.0006052850321509595) - rollout_corr/k3_kl:np.float64(0.0006508081007154942) - rollout_corr/rollout_ppl:np.float64(2.170979785732925) - rollout_corr/rollout_log_ppl:np.float64(0.7583060346714774) - rollout_corr/log_ppl_diff:np.float64(0.0006052836961316643) - rollout_corr/log_ppl_abs_diff:np.float64(0.001306259231569129) - rollout_corr/log_ppl_diff_max:np.float64(0.005559146404266357) - rollout_corr/log_ppl_diff_min:np.float64(-0.00449979305267334) - rollout_corr/ppl_ratio:np.float64(1.0006066909991205) - rollout_corr/chi2_token:np.float64(0.001387621508911252) - rollout_corr/chi2_seq:np.float64(1.5838370565325022) - actor/pg_loss:np.float64(8.792639710009098e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022318573451229895) - actor/ppo_kl:np.float64(5.0495547061046864e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.303693313151598) - perf/mfu/actor:np.float64(0.121345258264149) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.35682678222656) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03973030298948288 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03973030298948288 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:746.57421875 - response_length/max:8192.0 - response_length/min:178.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:746.57421875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:311.5625 - prompt_length/max:701.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.71539455652237e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3765512369573116) - timing_s/agent_loop/generate_sequences/max:np.float64(61.594920651987195) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.441452034945542) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(61.594920651987195) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:384 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:68.32033305615187 - timing_s/reward:0.07946610450744629 - timing_s/old_log_prob:3.648138355463743 - timing_s/adv:0.15895853005349636 - timing_s/update_actor:14.574448632076383 - timing_s/step:86.86667879670858 - timing_s/stop_profile:0.00011816807091236115 - timing_per_token_ms/adv:0.0005868161902131044 - timing_per_token_ms/gen:0.35746787700146954 - timing_per_token_ms/update_actor:0.0538034820644942 - perf/total_num_tokens:270883 - perf/time_per_step:86.86667879670858 - perf/throughput:389.7970484084282 (TaskRunner pid=2224264) Training Progress: 44%|████▍ | 44/100 [41:47<1:03:24, 67.94s/it] (TaskRunner pid=2224264) step:45 - global_seqlen/min:30046 - global_seqlen/max:36319 - global_seqlen/minmax_diff:6273 - global_seqlen/balanced_min:33079 - global_seqlen/balanced_max:33119 - global_seqlen/mean:33104.0 - actor/entropy:0.7044548392295837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.456476628780365 - training/rollout_probs_diff_mean:0.004338444676250219 - training/rollout_probs_diff_std:0.008459679782390594 - training/rollout_actor_probs_pearson_corr:0.999611496925354 - rollout_corr/rollout_is_mean:1.0000234842300415 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.426760253612883e-06 - rollout_corr/rollout_is_max:1.7352091073989868 - rollout_corr/rollout_is_min:0.412738174200058 - rollout_corr/rollout_is_std:0.03374973684549332 - rollout_corr/rollout_is_eff_sample_size:0.9988624890615934 - rollout_corr/rollout_is_seq_mean:0.9999771118164062 - rollout_corr/rollout_is_seq_std:0.0013954972382634878 - rollout_corr/rollout_is_seq_max:1.0044217109680176 - rollout_corr/rollout_is_seq_min:0.9954630732536316 - rollout_corr/rollout_is_seq_max_deviation:0.004536926746368408 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1041139638982713) - rollout_corr/training_log_ppl:np.float64(0.7206308286986314) - rollout_corr/kl:np.float64(0.0006204138266534542) - rollout_corr/k3_kl:np.float64(0.0006203657341075086) - rollout_corr/rollout_ppl:np.float64(2.102761774789542) - rollout_corr/rollout_log_ppl:np.float64(0.7200104069779627) - rollout_corr/log_ppl_diff:np.float64(0.0006204217206686735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011971264611929655) - rollout_corr/log_ppl_diff_max:np.float64(0.005265474319458008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036600232124328613) - rollout_corr/ppl_ratio:np.float64(1.0006216508336365) - rollout_corr/chi2_token:np.float64(0.0012450276408344507) - rollout_corr/chi2_seq:np.float64(1.1021782979369164) - actor/pg_loss:np.float64(-7.425120566040277e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015113081622075697) - actor/ppo_kl:np.float64(1.2432504128412347e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29480820521712303) - perf/mfu/actor:np.float64(0.1272612688514589) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.27553939819336) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005770002957433462 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005770002957433462 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:719.8125 - response_length/max:1962.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:719.8125 - response_length_non_aborted/max:1962.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.6875 - prompt_length/max:845.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.384138345718384e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.28562774322927) - timing_s/agent_loop/generate_sequences/max:np.float64(16.55846000276506) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.204205383684894) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.55846000276506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:201 - timing_s/agent_loop/slowest/response_length:1962 - timing_s/gen:23.205180127173662 - timing_s/reward:0.0775289349257946 - timing_s/old_log_prob:3.335056768730283 - timing_s/adv:0.130776172503829 - timing_s/update_actor:13.470531040802598 - timing_s/step:40.30686186812818 - timing_s/stop_profile:0.00012123957276344299 - timing_per_token_ms/adv:0.0004938080462475418 - timing_per_token_ms/gen:0.1259289535424463 - timing_per_token_ms/update_actor:0.05086443874155162 - perf/total_num_tokens:264832 - perf/time_per_step:40.30686186812818 - perf/throughput:821.2993635750221 (TaskRunner pid=2224264) Training Progress: 45%|████▌ | 45/100 [42:28<54:41, 59.66s/it] (TaskRunner pid=2224264) step:46 - global_seqlen/min:25951 - global_seqlen/max:37896 - global_seqlen/minmax_diff:11945 - global_seqlen/balanced_min:31356 - global_seqlen/balanced_max:31791 - global_seqlen/mean:31428.0 - actor/entropy:0.7460911870002747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.456478089094162 - training/rollout_probs_diff_mean:0.004542972426861525 - training/rollout_probs_diff_std:0.008595590479671955 - training/rollout_actor_probs_pearson_corr:0.9995923638343811 - rollout_corr/rollout_is_mean:1.0000001192092896 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1051189176214393e-05 - rollout_corr/rollout_is_max:1.62330961227417 - rollout_corr/rollout_is_min:0.39613932371139526 - rollout_corr/rollout_is_std:0.03452324494719505 - rollout_corr/rollout_is_eff_sample_size:0.9988095643491313 - rollout_corr/rollout_is_seq_mean:0.9999744892120361 - rollout_corr/rollout_is_seq_std:0.0015889799688011408 - rollout_corr/rollout_is_seq_max:1.0049422979354858 - rollout_corr/rollout_is_seq_min:0.9941216111183167 - rollout_corr/rollout_is_seq_max_deviation:0.00587838888168335 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2059524417854846) - rollout_corr/training_log_ppl:np.float64(0.7630574661307037) - rollout_corr/kl:np.float64(0.0006121348877812238) - rollout_corr/k3_kl:np.float64(0.000635629686428274) - rollout_corr/rollout_ppl:np.float64(2.204429803881794) - rollout_corr/rollout_log_ppl:np.float64(0.7624453329481184) - rollout_corr/log_ppl_diff:np.float64(0.0006121331825852394) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012938708532601595) - rollout_corr/log_ppl_diff_max:np.float64(0.006876647472381592) - rollout_corr/log_ppl_diff_min:np.float64(-0.004626870155334473) - rollout_corr/ppl_ratio:np.float64(1.0006135429721326) - rollout_corr/chi2_token:np.float64(0.0013200147077441216) - rollout_corr/chi2_seq:np.float64(1.2292231675237417) - actor/pg_loss:np.float64(-5.2339513786137104e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001717434528245576) - actor/ppo_kl:np.float64(-3.4712943731474155e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27383991330862045) - perf/mfu/actor:np.float64(0.11895889859672076) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.13016891479492) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014861086383461952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014861086383461952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:706.9375 - response_length/max:2609.0 - response_length/min:176.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:706.9375 - response_length_non_aborted/max:2609.0 - response_length_non_aborted/min:176.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.1875 - prompt_length/max:595.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.24672931432724e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7639765553176403) - timing_s/agent_loop/generate_sequences/max:np.float64(19.902556966990232) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.572092931586667) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.902556966990232) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:177 - timing_s/agent_loop/slowest/response_length:2609 - timing_s/gen:27.38719269633293 - timing_s/reward:0.0768952015787363 - timing_s/old_log_prob:3.2372986152768135 - timing_s/adv:0.13518765196204185 - timing_s/update_actor:14.003670601174235 - timing_s/step:44.92860447615385 - timing_s/stop_profile:0.00011640042066574097 - timing_per_token_ms/adv:0.0005376879373569821 - timing_per_token_ms/gen:0.15133052281149395 - timing_per_token_ms/update_actor:0.055697429844303786 - perf/total_num_tokens:251424 - perf/time_per_step:44.92860447615385 - perf/throughput:699.5098193330401 (TaskRunner pid=2224264) Training Progress: 46%|████▌ | 46/100 [43:13<49:43, 55.26s/it] (TaskRunner pid=2224264) step:47 - global_seqlen/min:28796 - global_seqlen/max:40193 - global_seqlen/minmax_diff:11397 - global_seqlen/balanced_min:34771 - global_seqlen/balanced_max:34835 - global_seqlen/mean:34821.25 - actor/entropy:0.7023751139640808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34761902689933777 - training/rollout_probs_diff_mean:0.004359257873147726 - training/rollout_probs_diff_std:0.008405433967709541 - training/rollout_actor_probs_pearson_corr:0.9996216297149658 - rollout_corr/rollout_is_mean:0.9998831152915955 - rollout_corr/rollout_is_ratio_fraction_high:4.914246346743312e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.914246346743312e-06 - rollout_corr/rollout_is_max:2.393986701965332 - rollout_corr/rollout_is_min:0.404596209526062 - rollout_corr/rollout_is_std:0.03408888727426529 - rollout_corr/rollout_is_eff_sample_size:0.9988389398452722 - rollout_corr/rollout_is_seq_mean:0.9998857975006104 - rollout_corr/rollout_is_seq_std:0.0014544243458658457 - rollout_corr/rollout_is_seq_max:1.0058720111846924 - rollout_corr/rollout_is_seq_min:0.9959080219268799 - rollout_corr/rollout_is_seq_max_deviation:0.005872011184692383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.088344756513834) - rollout_corr/training_log_ppl:np.float64(0.71887747416622) - rollout_corr/kl:np.float64(0.0006769733809974809) - rollout_corr/k3_kl:np.float64(0.0006363334275647503) - rollout_corr/rollout_ppl:np.float64(2.086956723127514) - rollout_corr/rollout_log_ppl:np.float64(0.7182004956412129) - rollout_corr/log_ppl_diff:np.float64(0.0006769785250071436) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012075564882252365) - rollout_corr/log_ppl_diff_max:np.float64(0.006249547004699707) - rollout_corr/log_ppl_diff_min:np.float64(-0.004652619361877441) - rollout_corr/ppl_ratio:np.float64(1.0006782109849155) - rollout_corr/chi2_token:np.float64(0.0012049346696585417) - rollout_corr/chi2_seq:np.float64(0.8489019055850804) - actor/pg_loss:np.float64(0.00018669036217033863) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019566034120543918) - actor/ppo_kl:np.float64(-4.604769495308858e-05) - actor/pg_clipfrac_lower:np.float64(4.4389203139871825e-06) - actor/grad_norm:np.float64(0.2251530885696411) - perf/mfu/actor:np.float64(0.1320186966648972) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.04901123046875) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006613961420953274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006613961420953274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:794.8828125 - response_length/max:2347.0 - response_length/min:195.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:794.8828125 - response_length_non_aborted/max:2347.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.28125 - prompt_length/max:950.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010444037616252899 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4994081798940897) - timing_s/agent_loop/generate_sequences/max:np.float64(18.286521969363093) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.572983121870493) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.286521969363093) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:2347 - timing_s/gen:25.14034183882177 - timing_s/reward:0.07981901802122593 - timing_s/old_log_prob:3.3178899493068457 - timing_s/adv:0.1422868575900793 - timing_s/update_actor:13.743424396961927 - timing_s/step:42.512125335633755 - timing_s/stop_profile:0.00011054240167140961 - timing_per_token_ms/adv:0.0005107759543026145 - timing_per_token_ms/gen:0.12354583438410618 - timing_per_token_ms/update_actor:0.04933562263331273 - perf/total_num_tokens:278570 - perf/time_per_step:42.512125335633755 - perf/throughput:819.0898414296109 (TaskRunner pid=2224264) Training Progress: 47%|████▋ | 47/100 [43:55<45:26, 51.45s/it] (TaskRunner pid=2224264) step:48 - global_seqlen/min:21182 - global_seqlen/max:34146 - global_seqlen/minmax_diff:12964 - global_seqlen/balanced_min:29830 - global_seqlen/balanced_max:29848 - global_seqlen/mean:29841.125 - actor/entropy:0.7442194819450378 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4606970250606537 - training/rollout_probs_diff_mean:0.004590690601617098 - training/rollout_probs_diff_std:0.008653250522911549 - training/rollout_actor_probs_pearson_corr:0.999643087387085 - rollout_corr/rollout_is_mean:0.9999340772628784 - rollout_corr/rollout_is_ratio_fraction_high:1.1716118024196476e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.565861463546753 - rollout_corr/rollout_is_min:0.6271895170211792 - rollout_corr/rollout_is_std:0.03491979092359543 - rollout_corr/rollout_is_eff_sample_size:0.9987818554445442 - rollout_corr/rollout_is_seq_mean:0.9998950362205505 - rollout_corr/rollout_is_seq_std:0.0015674727037549019 - rollout_corr/rollout_is_seq_max:1.0053834915161133 - rollout_corr/rollout_is_seq_min:0.9940774440765381 - rollout_corr/rollout_is_seq_max_deviation:0.005922555923461914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.153257429599762) - rollout_corr/training_log_ppl:np.float64(0.7468871437013149) - rollout_corr/kl:np.float64(0.0006051990699234011) - rollout_corr/k3_kl:np.float64(0.0006705579712615872) - rollout_corr/rollout_ppl:np.float64(2.1520667281001806) - rollout_corr/rollout_log_ppl:np.float64(0.7462819424690679) - rollout_corr/log_ppl_diff:np.float64(0.0006052012322470546) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013254067162051797) - rollout_corr/log_ppl_diff_max:np.float64(0.006841838359832764) - rollout_corr/log_ppl_diff_min:np.float64(-0.004772186279296875) - rollout_corr/ppl_ratio:np.float64(1.0006067838985473) - rollout_corr/chi2_token:np.float64(0.0014893666375428438) - rollout_corr/chi2_seq:np.float64(1.7015499353874475) - actor/pg_loss:np.float64(-4.240649286657572e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019128616168018198) - actor/ppo_kl:np.float64(-0.00013246856422000075) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3110559768974781) - perf/mfu/actor:np.float64(0.11562384315159505) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.0516357421875) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004874637350440025 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004874637350440025 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:666.81640625 - response_length/max:1727.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:666.81640625 - response_length_non_aborted/max:1727.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.71875 - prompt_length/max:563.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011752359569072723 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0456913970410824) - timing_s/agent_loop/generate_sequences/max:np.float64(14.015774877741933) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.352618356359017) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.015774877741933) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1727 - timing_s/gen:21.136593686416745 - timing_s/reward:0.07471263594925404 - timing_s/old_log_prob:3.2577999029308558 - timing_s/adv:0.11815616860985756 - timing_s/update_actor:13.682143926620483 - timing_s/step:38.35702735558152 - timing_s/stop_profile:0.00011106766760349274 - timing_per_token_ms/adv:0.0004949384809129078 - timing_per_token_ms/gen:0.12381941762934152 - timing_per_token_ms/update_actor:0.05731245021183218 - perf/total_num_tokens:238729 - perf/time_per_step:38.35702735558152 - perf/throughput:777.9832551507063 (TaskRunner pid=2224264) Training Progress: 48%|████▊ | 48/100 [44:34<41:11, 47.54s/it] (TaskRunner pid=2224264) step:49 - global_seqlen/min:27565 - global_seqlen/max:33967 - global_seqlen/minmax_diff:6402 - global_seqlen/balanced_min:31660 - global_seqlen/balanced_max:31677 - global_seqlen/mean:31668.625 - actor/entropy:0.7485659122467041 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8766915202140808 - training/rollout_probs_diff_mean:0.004531657788902521 - training/rollout_probs_diff_std:0.008790009655058384 - training/rollout_actor_probs_pearson_corr:0.9996523857116699 - rollout_corr/rollout_is_mean:1.0000706911087036 - rollout_corr/rollout_is_ratio_fraction_high:5.495320692716632e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.495320692716632e-06 - rollout_corr/rollout_is_max:8.354974746704102 - rollout_corr/rollout_is_min:0.0006339381216093898 - rollout_corr/rollout_is_std:0.03499140590429306 - rollout_corr/rollout_is_eff_sample_size:0.99877733654154 - rollout_corr/rollout_is_seq_mean:1.0000840425491333 - rollout_corr/rollout_is_seq_std:0.0015765647403895855 - rollout_corr/rollout_is_seq_max:1.0120781660079956 - rollout_corr/rollout_is_seq_min:0.9965959787368774 - rollout_corr/rollout_is_seq_max_deviation:0.012078166007995605 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.135033040307462) - rollout_corr/training_log_ppl:np.float64(0.7435363659169525) - rollout_corr/kl:np.float64(0.000531884074939093) - rollout_corr/k3_kl:np.float64(0.0007136939013889787) - rollout_corr/rollout_ppl:np.float64(2.1338718575425446) - rollout_corr/rollout_log_ppl:np.float64(0.7430044782813638) - rollout_corr/log_ppl_diff:np.float64(0.0005318876355886459) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011775929015129805) - rollout_corr/log_ppl_diff_max:np.float64(0.011757910251617432) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038299560546875) - rollout_corr/ppl_ratio:np.float64(1.0005332818254828) - rollout_corr/chi2_token:np.float64(0.00199099606834352) - rollout_corr/chi2_seq:np.float64(1.2359290518797934) - actor/pg_loss:np.float64(-0.00026974163483828306) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014913909058122954) - actor/ppo_kl:np.float64(-6.670435310951461e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24813814461231232) - perf/mfu/actor:np.float64(0.12164250497550971) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.1574478149414) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006872585508972406 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006872585508972406 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:710.83203125 - response_length/max:1348.0 - response_length/min:200.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:710.83203125 - response_length_non_aborted/max:1348.0 - response_length_non_aborted/min:200.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.8125 - prompt_length/max:418.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015538372099399567 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5202769823372364) - timing_s/agent_loop/generate_sequences/max:np.float64(13.002393383532763) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.453353088778385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.002393383532763) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:191 - timing_s/agent_loop/slowest/response_length:1348 - timing_s/gen:20.19887711107731 - timing_s/reward:0.077494902536273 - timing_s/old_log_prob:3.275420231744647 - timing_s/adv:0.1729813516139984 - timing_s/update_actor:13.726705972105265 - timing_s/step:37.536205399781466 - timing_s/stop_profile:0.00011497922241687775 - timing_per_token_ms/adv:0.0006827789003074747 - timing_per_token_ms/gen:0.1109993081999929 - timing_per_token_ms/update_actor:0.05418101501132929 - perf/total_num_tokens:253349 - perf/time_per_step:37.536205399781466 - perf/throughput:843.6821107171471 (TaskRunner pid=2224264) Training Progress: 49%|████▉ | 49/100 [45:11<37:52, 44.56s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a product molecule: (TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"** (TaskRunner pid=2224264) This is a **nitrogen-containing heterocyclic compound** (likely a pyrrole or similar), with a substituent on the nitrogen: (TaskRunner pid=2224264) - A **benzene ring** with a **chloro group** on one carbon (TaskRunner pid=2224264) - A **trifluoromethyl group** (C(F)(F)F) on another carbon (TaskRunner pid=2224264) - A **nitrile group** (the N in the ring is bonded to a carbon chain) (TaskRunner pid=2224264) (TaskRunner pid=2224264) Let's break it down: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Product structure: (TaskRunner pid=2224264) - The core is a **pyrrole-like ring** (N in ring) (TaskRunner pid=2224264) - Attached to the N is a **benzene ring** with: (TaskRunner pid=2224264) - A **chloro group** (Cl) on one carbon (TaskRunner pid=2224264) - A **trifluoromethyl group** (C(F)(F)F) on another carbon (TaskRunner pid=2224264) - The ring also has a **nitrile group** (C≡N) (TaskRunner pid=2224264) (TaskRunner pid=2224264) We are to determine which of the four options is the **only correct reactant** in the synthesis of this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Let’s analyze each option: (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### **Option A:** (TaskRunner pid=2224264) `O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a **nitrile group** (C≡N) (TaskRunner pid=2224264) - Contains a **benzene ring** with Cl and C(F)(F)F (TaskRunner pid=2224264) - The structure matches the **product** closely (TaskRunner pid=2224264) - The **nitrogen ring** is connected to the same substituents (TaskRunner pid=2224264) - Seems to be a **correct structure** for a possible reactant (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### **Option B:** (TaskRunner pid=2224264) `O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1` (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a **Cl group** on the ring (TaskRunner pid=2224264) - The ring has **nitrogen** and **chlorine** (TaskRunner pid=2224264) - However, the **trifluoromethyl group** is missing (TaskRunner pid=2224264) - Structure does **not match** the product (TaskRunner pid=2224264) - Likely incorrect (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### **Option C:** (TaskRunner pid=2224264) `O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1` (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a **benzene ring** with Cl and C(F)(F)F (TaskRunner pid=2224264) - Contains **nitrogen and oxygen** in the ring (TaskRunner pid=2224264) - However, the **nitrile group** is not present (TaskRunner pid=2224264) - The **structure is not the same** as the product (TaskRunner pid=2224264) - Likely incorrect (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### **Option D:** (TaskRunner pid=2224264) `O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F` (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a **trifluoromethyl group** (TaskRunner pid=2224264) - Contains **double bonds and rings** (TaskRunner pid=2224264) - But **no benzene ring with Cl** (TaskRunner pid=2224264) - No **nitrogen ring** with the correct substituents (TaskRunner pid=2224264) - Does not match the product (TaskRunner pid=2224264) - Incorrect (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### **Conclusion:** (TaskRunner pid=2224264) Only **Option A** contains all the necessary **functional groups and substituents** to be the correct reactant in the synthesis of the given product. It matches the **structure and substituents** of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2224264) step:50 - global_seqlen/min:25774 - global_seqlen/max:40010 - global_seqlen/minmax_diff:14236 - global_seqlen/balanced_min:30199 - global_seqlen/balanced_max:30236 - global_seqlen/mean:30221.125 - actor/entropy:0.6831239461898804 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18094828724861145 - training/rollout_probs_diff_mean:0.00435651233419776 - training/rollout_probs_diff_std:0.008434131741523743 - training/rollout_actor_probs_pearson_corr:0.9996002316474915 - rollout_corr/rollout_is_mean:0.9999532699584961 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.877063358639134e-06 - rollout_corr/rollout_is_max:1.8264771699905396 - rollout_corr/rollout_is_min:0.4736720621585846 - rollout_corr/rollout_is_std:0.03403113782405853 - rollout_corr/rollout_is_eff_sample_size:0.998842864637548 - rollout_corr/rollout_is_seq_mean:0.9999036192893982 - rollout_corr/rollout_is_seq_std:0.0016001641051843762 - rollout_corr/rollout_is_seq_max:1.0056898593902588 - rollout_corr/rollout_is_seq_min:0.9943985939025879 - rollout_corr/rollout_is_seq_max_deviation:0.005689859390258789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0370463258586824) - rollout_corr/training_log_ppl:np.float64(0.6895827453117818) - rollout_corr/kl:np.float64(0.0007133520028830276) - rollout_corr/k3_kl:np.float64(0.0006415439404463541) - rollout_corr/rollout_ppl:np.float64(2.035565957427025) - rollout_corr/rollout_log_ppl:np.float64(0.6888693935470656) - rollout_corr/log_ppl_diff:np.float64(0.000713351764716208) - rollout_corr/log_ppl_abs_diff:np.float64(0.001375065534375608) - rollout_corr/log_ppl_diff_max:np.float64(0.010087192058563232) - rollout_corr/log_ppl_diff_min:np.float64(-0.004953622817993164) - rollout_corr/ppl_ratio:np.float64(1.0007150897290558) - rollout_corr/chi2_token:np.float64(0.0011441940441727638) - rollout_corr/chi2_seq:np.float64(1.4973946234676987) - actor/pg_loss:np.float64(0.00014200282748788595) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030153528336995805) - actor/ppo_kl:np.float64(1.1964239081230232e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3060915023088455) - perf/mfu/actor:np.float64(0.11810264114100524) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.11758041381836) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6223214285714286) - val-aux/sciknoweval/reward/std@16:np.float64(0.2647595134884485) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7291095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19877560636838412) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5156142857142857) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.24700525701136392) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6215809523809525) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2629386719576279) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8044857142857144) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12973275443729954) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4167428571428572) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.20684059335831737) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6544095238095238) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20967583799172174) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8527809523809524) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07656020696044112) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3329095238095238) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.16042430484607695) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6712285714285714) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15275032646284742) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8799809523809523) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.041193057541303105) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.2648619047619048) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.10865600566624517) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6785476190476191) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11458915286528892) - val-aux/sciknoweval/score/mean@16:np.float64(0.6223214285714286) - val-aux/sciknoweval/score/std@16:np.float64(0.2647595134884485) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7291095238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.19877560636838412) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5156142857142857) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.24700525701136392) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6215809523809525) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2629386719576279) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8044857142857144) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12973275443729954) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4167428571428572) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.20684059335831737) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6544095238095238) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20967583799172174) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8527809523809524) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07656020696044112) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3329095238095238) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.16042430484607695) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6712285714285714) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15275032646284742) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8799809523809523) - val-aux/sciknoweval/score/best@16/std:np.float64(0.041193057541303105) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.2648619047619048) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.10865600566624517) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6785476190476191) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11458915286528892) - val-core/sciknoweval/acc/mean@16:np.float64(0.6223214285714286) - val-aux/sciknoweval/acc/std@16:np.float64(0.2647595134884485) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7291095238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.19877560636838412) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5156142857142857) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.24700525701136392) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6215809523809525) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2629386719576279) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8044857142857144) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12973275443729954) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4167428571428572) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.20684059335831737) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6544095238095238) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20967583799172174) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8527809523809524) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07656020696044112) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3329095238095238) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.16042430484607695) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6712285714285714) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15275032646284742) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8799809523809523) - val-core/sciknoweval/acc/best@16/std:np.float64(0.041193057541303105) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.2648619047619048) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.10865600566624517) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6785476190476191) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11458915286528892) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.986904761904762) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03936087691755317) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.998252380952381) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.013004708934596106) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9755047619047619) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0512654983800644) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9868380952380952) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.039356229004372134) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999285714285715) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0010041883502657082) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9567476190476191) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.06208854001798514) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9947333333333334) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.024839640974003308) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9297809523809525) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06686724116120492) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9986714285714285) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.011269482933237476) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8981190476190476) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05941598517810314) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998619047619047) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0028135293122891663) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002585907932370901 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002585907932370901 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:664.66015625 - response_length/max:1839.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:664.66015625 - response_length_non_aborted/max:1839.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.75 - prompt_length/max:900.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018392875790596008 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0269889794290066) - timing_s/agent_loop/generate_sequences/max:np.float64(14.812706418335438) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.506310426571872) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.812706418335438) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:165 - timing_s/agent_loop/slowest/response_length:1763 - timing_s/gen:21.724913926795125 - timing_s/reward:0.07297280617058277 - timing_s/old_log_prob:3.25269278883934 - timing_s/adv:0.11958634853363037 - timing_s/update_actor:13.451122742146254 - timing_s/step:38.707119492813945 - timing_s/testing:191.03151337988675 - timing_s/save_checkpoint:12.945183955132961 - timing_s/stop_profile:0.00014170818030834198 - timing_per_token_ms/adv:0.000494630612417764 - timing_per_token_ms/gen:0.1276787005036357 - timing_per_token_ms/update_actor:0.05563625916534483 - perf/total_num_tokens:241769 - perf/time_per_step:38.707119492813945 - perf/throughput:780.7639885373184 (TaskRunner pid=2224264) Training Progress: 50%|█████ | 50/100 [49:14<1:26:40, 104.01s/it] (TaskRunner pid=2224264) step:51 - global_seqlen/min:30736 - global_seqlen/max:37371 - global_seqlen/minmax_diff:6635 - global_seqlen/balanced_min:34292 - global_seqlen/balanced_max:34302 - global_seqlen/mean:34297.75 - actor/entropy:0.762580394744873 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294635772705078 - training/rollout_probs_diff_mean:0.004454846493899822 - training/rollout_probs_diff_std:0.008279886096715927 - training/rollout_actor_probs_pearson_corr:0.9996731281280518 - rollout_corr/rollout_is_mean:1.0000529289245605 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.811339259147644 - rollout_corr/rollout_is_min:0.5012558698654175 - rollout_corr/rollout_is_std:0.03488221764564514 - rollout_corr/rollout_is_eff_sample_size:0.9987848284293472 - rollout_corr/rollout_is_seq_mean:1.0000650882720947 - rollout_corr/rollout_is_seq_std:0.0014846812700852752 - rollout_corr/rollout_is_seq_max:1.0048807859420776 - rollout_corr/rollout_is_seq_min:0.9954461455345154 - rollout_corr/rollout_is_seq_max_deviation:0.004880785942077637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.208350053522736) - rollout_corr/training_log_ppl:np.float64(0.7750773588195443) - rollout_corr/kl:np.float64(0.0005686312180488429) - rollout_corr/k3_kl:np.float64(0.0006700464483628821) - rollout_corr/rollout_ppl:np.float64(2.207024629227817) - rollout_corr/rollout_log_ppl:np.float64(0.7745087213115767) - rollout_corr/log_ppl_diff:np.float64(0.0005686375079676509) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012317007640376687) - rollout_corr/log_ppl_diff_max:np.float64(0.006791532039642334) - rollout_corr/log_ppl_diff_min:np.float64(-0.006045103073120117) - rollout_corr/ppl_ratio:np.float64(1.000570009695366) - rollout_corr/chi2_token:np.float64(0.0015524374321103096) - rollout_corr/chi2_seq:np.float64(2.341128587955609) - actor/pg_loss:np.float64(-0.00025634258054196835) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016806365397314948) - actor/ppo_kl:np.float64(7.1047582039796e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24387626722455025) - perf/mfu/actor:np.float64(0.13125870218702868) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.71975231170654) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033099849242717028 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033099849242717028 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:759.2734375 - response_length/max:1708.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:759.2734375 - response_length_non_aborted/max:1708.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.53125 - prompt_length/max:608.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.316764771938324e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.95208290964365) - timing_s/agent_loop/generate_sequences/max:np.float64(15.170884735882282) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.576293181155052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.170884735882282) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:1708 - timing_s/gen:21.975822178646922 - timing_s/reward:0.07666116580367088 - timing_s/old_log_prob:3.373812895268202 - timing_s/adv:0.12022814154624939 - timing_s/update_actor:13.922087775543332 - timing_s/step:39.56273279339075 - timing_s/stop_profile:0.00014520063996315002 - timing_per_token_ms/adv:0.0004381779473371044 - timing_per_token_ms/gen:0.11305947389386915 - timing_per_token_ms/update_actor:0.05073979989774596 - perf/total_num_tokens:274382 - perf/time_per_step:39.56273279339075 - perf/throughput:866.920649266415 (TaskRunner pid=2224264) Training Progress: 51%|█████ | 51/100 [49:54<1:09:09, 84.69s/it] (TaskRunner pid=2224264) step:52 - global_seqlen/min:25163 - global_seqlen/max:39354 - global_seqlen/minmax_diff:14191 - global_seqlen/balanced_min:31304 - global_seqlen/balanced_max:31326 - global_seqlen/mean:31315.625 - actor/entropy:0.7275306582450867 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846359670162201 - training/rollout_probs_diff_mean:0.0044688270427286625 - training/rollout_probs_diff_std:0.008479300886392593 - training/rollout_actor_probs_pearson_corr:0.9996131658554077 - rollout_corr/rollout_is_mean:0.9999827146530151 - rollout_corr/rollout_is_ratio_fraction_high:5.457889528770465e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.091577905754093e-05 - rollout_corr/rollout_is_max:2.513453245162964 - rollout_corr/rollout_is_min:0.2490382194519043 - rollout_corr/rollout_is_std:0.03466796875 - rollout_corr/rollout_is_eff_sample_size:0.9987993368491189 - rollout_corr/rollout_is_seq_mean:0.9999496936798096 - rollout_corr/rollout_is_seq_std:0.0013713843654841185 - rollout_corr/rollout_is_seq_max:1.0041810274124146 - rollout_corr/rollout_is_seq_min:0.9949362277984619 - rollout_corr/rollout_is_seq_max_deviation:0.005063772201538086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0971961123868823) - rollout_corr/training_log_ppl:np.float64(0.7249851016094908) - rollout_corr/kl:np.float64(0.000624122314107467) - rollout_corr/k3_kl:np.float64(0.0006732126169026742) - rollout_corr/rollout_ppl:np.float64(2.095865353010595) - rollout_corr/rollout_log_ppl:np.float64(0.7243609796278179) - rollout_corr/log_ppl_diff:np.float64(0.0006241219816729426) - rollout_corr/log_ppl_abs_diff:np.float64(0.001236821641214192) - rollout_corr/log_ppl_diff_max:np.float64(0.00571441650390625) - rollout_corr/log_ppl_diff_min:np.float64(-0.005221307277679443) - rollout_corr/ppl_ratio:np.float64(1.0006253765895963) - rollout_corr/chi2_token:np.float64(0.0014572420623153448) - rollout_corr/chi2_seq:np.float64(2.134200472617522) - actor/pg_loss:np.float64(0.00022022065240889788) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021542626041082258) - actor/ppo_kl:np.float64(-4.695473637639225e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25628167018294334) - perf/mfu/actor:np.float64(0.1214559238638076) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.17374038696289) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011097254231572151 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011097254231572151 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:715.70703125 - response_length/max:1611.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:715.70703125 - response_length_non_aborted/max:1611.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.90625 - prompt_length/max:453.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.15229320526123e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3523097801953554) - timing_s/agent_loop/generate_sequences/max:np.float64(14.437729807570577) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.303848396521062) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.437729807570577) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:1611 - timing_s/gen:21.366347413510084 - timing_s/reward:0.07527691684663296 - timing_s/old_log_prob:3.179901920258999 - timing_s/adv:0.12300918810069561 - timing_s/update_actor:13.513995960354805 - timing_s/step:38.342299580574036 - timing_s/stop_profile:0.00011920370161533356 - timing_per_token_ms/adv:0.0004910056405576115 - timing_per_token_ms/gen:0.1166151664575026 - timing_per_token_ms/update_actor:0.053942704162677595 - perf/total_num_tokens:250525 - perf/time_per_step:38.342299580574036 - perf/throughput:816.7383110184119 (TaskRunner pid=2224264) Training Progress: 52%|█████▏ | 52/100 [50:32<56:38, 70.80s/it] (TaskRunner pid=2224264) step:53 - global_seqlen/min:29023 - global_seqlen/max:38170 - global_seqlen/minmax_diff:9147 - global_seqlen/balanced_min:32089 - global_seqlen/balanced_max:32125 - global_seqlen/mean:32112.5 - actor/entropy:0.7673122882843018 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2821834683418274 - training/rollout_probs_diff_mean:0.004418913275003433 - training/rollout_probs_diff_std:0.008327017538249493 - training/rollout_actor_probs_pearson_corr:0.9996277093887329 - rollout_corr/rollout_is_mean:1.0000674724578857 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6528507471084595 - rollout_corr/rollout_is_min:0.6141930818557739 - rollout_corr/rollout_is_std:0.035042472183704376 - rollout_corr/rollout_is_eff_sample_size:0.9987737690154207 - rollout_corr/rollout_is_seq_mean:1.0000889301300049 - rollout_corr/rollout_is_seq_std:0.0013838375452905893 - rollout_corr/rollout_is_seq_max:1.005210518836975 - rollout_corr/rollout_is_seq_min:0.9962599873542786 - rollout_corr/rollout_is_seq_max_deviation:0.005210518836975098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2401722352951765) - rollout_corr/training_log_ppl:np.float64(0.7886160378111526) - rollout_corr/kl:np.float64(0.0006792797128305494) - rollout_corr/k3_kl:np.float64(0.0006657197256458858) - rollout_corr/rollout_ppl:np.float64(2.238586582709104) - rollout_corr/rollout_log_ppl:np.float64(0.7879367553978227) - rollout_corr/log_ppl_diff:np.float64(0.0006792824133299291) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012466523912735283) - rollout_corr/log_ppl_diff_max:np.float64(0.0049315690994262695) - rollout_corr/log_ppl_diff_min:np.float64(-0.004511535167694092) - rollout_corr/ppl_ratio:np.float64(1.000680540688336) - rollout_corr/chi2_token:np.float64(0.00131767219863832) - rollout_corr/chi2_seq:np.float64(0.9375523473136127) - actor/pg_loss:np.float64(-5.056627560406923e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015928275297483196) - actor/ppo_kl:np.float64(0.00014284484250026708) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2610822059214115) - perf/mfu/actor:np.float64(0.12400191776249886) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.9232177734375) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009051043540239334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009051043540239334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:707.953125 - response_length/max:1922.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:707.953125 - response_length_non_aborted/max:1922.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.5625 - prompt_length/max:532.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.531714022159576e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6737753115594387) - timing_s/agent_loop/generate_sequences/max:np.float64(16.309401037171483) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.246375748174614) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.309401037171483) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:1922 - timing_s/gen:22.928913297131658 - timing_s/reward:0.07421708479523659 - timing_s/old_log_prob:3.3302101399749517 - timing_s/adv:0.1337327789515257 - timing_s/update_actor:13.578472383320332 - timing_s/step:40.13115260377526 - timing_s/stop_profile:0.0001122206449508667 - timing_per_token_ms/adv:0.000520563561508469 - timing_per_token_ms/gen:0.12651412135078935 - timing_per_token_ms/update_actor:0.0528550890748164 - perf/total_num_tokens:256900 - perf/time_per_step:40.13115260377526 - perf/throughput:800.1888287898084 (TaskRunner pid=2224264) Training Progress: 53%|█████▎ | 53/100 [51:12<48:15, 61.61s/it] (TaskRunner pid=2224264) step:54 - global_seqlen/min:24046 - global_seqlen/max:37658 - global_seqlen/minmax_diff:13612 - global_seqlen/balanced_min:31667 - global_seqlen/balanced_max:31732 - global_seqlen/mean:31711.0 - actor/entropy:0.7221550345420837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564734995365143 - training/rollout_probs_diff_mean:0.0042567262426018715 - training/rollout_probs_diff_std:0.008278296329081059 - training/rollout_actor_probs_pearson_corr:0.9996740221977234 - rollout_corr/rollout_is_mean:0.9999973773956299 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.423934908321826e-06 - rollout_corr/rollout_is_max:1.8484896421432495 - rollout_corr/rollout_is_min:0.4127388894557953 - rollout_corr/rollout_is_std:0.03383616358041763 - rollout_corr/rollout_is_eff_sample_size:0.9988563043114053 - rollout_corr/rollout_is_seq_mean:0.9999293088912964 - rollout_corr/rollout_is_seq_std:0.0013600483071058989 - rollout_corr/rollout_is_seq_max:1.003616213798523 - rollout_corr/rollout_is_seq_min:0.9944389462471008 - rollout_corr/rollout_is_seq_max_deviation:0.00556105375289917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1455902410671115) - rollout_corr/training_log_ppl:np.float64(0.7426831858465448) - rollout_corr/kl:np.float64(0.0007347915104665148) - rollout_corr/k3_kl:np.float64(0.0006544318975443275) - rollout_corr/rollout_ppl:np.float64(2.14404917974025) - rollout_corr/rollout_log_ppl:np.float64(0.7419483949197456) - rollout_corr/log_ppl_diff:np.float64(0.0007347909267991781) - rollout_corr/log_ppl_abs_diff:np.float64(0.00128568010404706) - rollout_corr/log_ppl_diff_max:np.float64(0.007453739643096924) - rollout_corr/log_ppl_diff_min:np.float64(-0.0039052367210388184) - rollout_corr/ppl_ratio:np.float64(1.000736182089895) - rollout_corr/chi2_token:np.float64(0.0011647003702819347) - rollout_corr/chi2_seq:np.float64(1.3221846474334598) - actor/pg_loss:np.float64(-0.0001485362881794572) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030488177571896813) - actor/ppo_kl:np.float64(6.0147084293804554e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3080219365656376) - perf/mfu/actor:np.float64(0.12202071329811175) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.63494873046875) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.016614189371466637 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.016614189371466637 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:720.1875 - response_length/max:2286.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:720.1875 - response_length_non_aborted/max:2286.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.78125 - prompt_length/max:597.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.760074317455292e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9534575715661049) - timing_s/agent_loop/generate_sequences/max:np.float64(17.89197232015431) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.935718771463144) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.89197232015431) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:2286 - timing_s/gen:25.277695801109076 - timing_s/reward:0.08129468001425266 - timing_s/old_log_prob:3.2715712431818247 - timing_s/adv:0.1201015543192625 - timing_s/update_actor:13.721273077651858 - timing_s/step:42.558674819767475 - timing_s/stop_profile:0.00013141706585884094 - timing_per_token_ms/adv:0.0004734222916309108 - timing_per_token_ms/gen:0.13710457238300072 - timing_per_token_ms/update_actor:0.05408719796620991 - perf/total_num_tokens:253688 - perf/time_per_step:42.558674819767475 - perf/throughput:745.1124860981575 (TaskRunner pid=2224264) Training Progress: 54%|█████▍ | 54/100 [51:55<42:51, 55.91s/it] (TaskRunner pid=2224264) step:55 - global_seqlen/min:29461 - global_seqlen/max:36701 - global_seqlen/minmax_diff:7240 - global_seqlen/balanced_min:33720 - global_seqlen/balanced_max:34509 - global_seqlen/mean:33827.375 - actor/entropy:0.8024973273277283 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23547464609146118 - training/rollout_probs_diff_mean:0.0043955049477517605 - training/rollout_probs_diff_std:0.008106191642582417 - training/rollout_actor_probs_pearson_corr:0.999698281288147 - rollout_corr/rollout_is_mean:1.0000178813934326 - rollout_corr/rollout_is_ratio_fraction_high:5.190302545088343e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0640006065368652 - rollout_corr/rollout_is_min:0.5909958481788635 - rollout_corr/rollout_is_std:0.03487367555499077 - rollout_corr/rollout_is_eff_sample_size:0.9987854230284318 - rollout_corr/rollout_is_seq_mean:0.999980092048645 - rollout_corr/rollout_is_seq_std:0.0013782148016616702 - rollout_corr/rollout_is_seq_max:1.0047253370285034 - rollout_corr/rollout_is_seq_min:0.9960294961929321 - rollout_corr/rollout_is_seq_max_deviation:0.004725337028503418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.3131857896223664) - rollout_corr/training_log_ppl:np.float64(0.8200820172205567) - rollout_corr/kl:np.float64(0.0007159894126873567) - rollout_corr/k3_kl:np.float64(0.0006568527235231159) - rollout_corr/rollout_ppl:np.float64(2.311495006084442) - rollout_corr/rollout_log_ppl:np.float64(0.8193660224787891) - rollout_corr/log_ppl_diff:np.float64(0.0007159947417676449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012078655418008566) - rollout_corr/log_ppl_diff_max:np.float64(0.004928708076477051) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037838220596313477) - rollout_corr/ppl_ratio:np.float64(1.0007172094192356) - rollout_corr/chi2_token:np.float64(0.0011943182907998562) - rollout_corr/chi2_seq:np.float64(2.1422022855840623) - actor/pg_loss:np.float64(6.403832230716944e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014861452268633002) - actor/ppo_kl:np.float64(7.01219851411139e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2634842284023762) - perf/mfu/actor:np.float64(0.12720097629374041) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.60932922363281) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000431443884735927 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000431443884735927 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:752.60546875 - response_length/max:2800.0 - response_length/min:187.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:752.60546875 - response_length_non_aborted/max:2800.0 - response_length_non_aborted/min:187.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.5 - prompt_length/max:625.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.310990571975708e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7985909432172775) - timing_s/agent_loop/generate_sequences/max:np.float64(20.643996415659785) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.059685343854653) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.643996415659785) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:2800 - timing_s/gen:27.551894007250667 - timing_s/reward:0.07855446077883244 - timing_s/old_log_prob:3.331155611202121 - timing_s/adv:0.1180018950253725 - timing_s/update_actor:14.12481613457203 - timing_s/step:45.289689207449555 - timing_s/stop_profile:3.606826066970825e-05 - timing_per_token_ms/adv:0.0004360443835258149 - timing_per_token_ms/gen:0.14300266266278433 - timing_per_token_ms/update_actor:0.05219447316918631 - perf/total_num_tokens:270619 - perf/time_per_step:45.289689207449555 - perf/throughput:746.9111754124345 (TaskRunner pid=2224264) Training Progress: 55%|█████▌ | 55/100 [52:40<39:32, 52.73s/it] (TaskRunner pid=2224264) step:56 - global_seqlen/min:25792 - global_seqlen/max:39171 - global_seqlen/minmax_diff:13379 - global_seqlen/balanced_min:31985 - global_seqlen/balanced_max:31996 - global_seqlen/mean:31991.875 - actor/entropy:0.8143277764320374 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2929895520210266 - training/rollout_probs_diff_mean:0.004465053323656321 - training/rollout_probs_diff_std:0.008296534419059753 - training/rollout_actor_probs_pearson_corr:0.9996805787086487 - rollout_corr/rollout_is_mean:0.9999870657920837 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5952778994687833e-05 - rollout_corr/rollout_is_max:1.629705786705017 - rollout_corr/rollout_is_min:0.2926250994205475 - rollout_corr/rollout_is_std:0.034940268844366074 - rollout_corr/rollout_is_eff_sample_size:0.9987806662555786 - rollout_corr/rollout_is_seq_mean:1.0000478029251099 - rollout_corr/rollout_is_seq_std:0.0013941883808001876 - rollout_corr/rollout_is_seq_max:1.0048588514328003 - rollout_corr/rollout_is_seq_min:0.9960804581642151 - rollout_corr/rollout_is_seq_max_deviation:0.004858851432800293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.281407651025802) - rollout_corr/training_log_ppl:np.float64(0.804402201261837) - rollout_corr/kl:np.float64(0.0005219981214015235) - rollout_corr/k3_kl:np.float64(0.0006556217045954327) - rollout_corr/rollout_ppl:np.float64(2.2802593959495425) - rollout_corr/rollout_log_ppl:np.float64(0.803880195133388) - rollout_corr/log_ppl_diff:np.float64(0.000522006128448993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012433496885932982) - rollout_corr/log_ppl_diff_max:np.float64(0.0066702961921691895) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035393238067626953) - rollout_corr/ppl_ratio:np.float64(1.0005232789553702) - rollout_corr/chi2_token:np.float64(0.0015740657690912485) - rollout_corr/chi2_seq:np.float64(5.73605796508491) - actor/pg_loss:np.float64(-4.861317574977875e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018069532825393253) - actor/ppo_kl:np.float64(-5.679699522431747e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2360125184059143) - perf/mfu/actor:np.float64(0.12372654611791109) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.54317474365234) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002341070445254445 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002341070445254445 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:734.58984375 - response_length/max:1632.0 - response_length/min:182.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:734.58984375 - response_length_non_aborted/max:1632.0 - response_length_non_aborted/min:182.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.15625 - prompt_length/max:458.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.9824674129486084e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.308333108201623) - timing_s/agent_loop/generate_sequences/max:np.float64(14.604689434170723) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.345092790725175) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.604689434170723) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:1632 - timing_s/gen:21.219629768282175 - timing_s/reward:0.07743704505264759 - timing_s/old_log_prob:3.259379044175148 - timing_s/adv:0.11571713909506798 - timing_s/update_actor:13.64204434491694 - timing_s/step:38.399653954431415 - timing_s/stop_profile:0.00011725537478923798 - timing_per_token_ms/adv:0.0004521348744605778 - timing_per_token_ms/gen:0.11283736017804459 - timing_per_token_ms/update_actor:0.05330276962868283 - perf/total_num_tokens:255935 - perf/time_per_step:38.399653954431415 - perf/throughput:833.1292526220294 (TaskRunner pid=2224264) Training Progress: 56%|█████▌ | 56/100 [53:19<35:31, 48.45s/it] (TaskRunner pid=2224264) step:57 - global_seqlen/min:27687 - global_seqlen/max:34593 - global_seqlen/minmax_diff:6906 - global_seqlen/balanced_min:31298 - global_seqlen/balanced_max:31311 - global_seqlen/mean:31305.125 - actor/entropy:0.7938913106918335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.680421769618988 - training/rollout_probs_diff_mean:0.004562178626656532 - training/rollout_probs_diff_std:0.008708939887583256 - training/rollout_actor_probs_pearson_corr:0.9995999932289124 - rollout_corr/rollout_is_mean:0.9999580979347229 - rollout_corr/rollout_is_ratio_fraction_high:5.648919795930851e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6946758478297852e-05 - rollout_corr/rollout_is_max:2.2787086963653564 - rollout_corr/rollout_is_min:0.24652595818042755 - rollout_corr/rollout_is_std:0.035597678273916245 - rollout_corr/rollout_is_eff_sample_size:0.9987344090023405 - rollout_corr/rollout_is_seq_mean:0.9999576807022095 - rollout_corr/rollout_is_seq_std:0.001491630100645125 - rollout_corr/rollout_is_seq_max:1.0051113367080688 - rollout_corr/rollout_is_seq_min:0.9956648349761963 - rollout_corr/rollout_is_seq_max_deviation:0.005111336708068848 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.283419963438064) - rollout_corr/training_log_ppl:np.float64(0.8058918532915413) - rollout_corr/kl:np.float64(0.0006797224436070337) - rollout_corr/k3_kl:np.float64(0.0007055759461991329) - rollout_corr/rollout_ppl:np.float64(2.281822293996811) - rollout_corr/rollout_log_ppl:np.float64(0.80521212273743) - rollout_corr/log_ppl_diff:np.float64(0.0006797305541113019) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013461158378049731) - rollout_corr/log_ppl_diff_max:np.float64(0.0058408379554748535) - rollout_corr/log_ppl_diff_min:np.float64(-0.004617094993591309) - rollout_corr/ppl_ratio:np.float64(1.0006813008803874) - rollout_corr/chi2_token:np.float64(0.0014507242012768984) - rollout_corr/chi2_seq:np.float64(1.6866442584432662) - actor/pg_loss:np.float64(1.2409058399498463e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002567762589933409) - actor/ppo_kl:np.float64(-1.4622721524304438e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2836088426411152) - perf/mfu/actor:np.float64(0.12280457724250629) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.51215362548828) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012533540138974786 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012533540138974786 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:691.50390625 - response_length/max:1421.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:691.50390625 - response_length_non_aborted/max:1421.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.78125 - prompt_length/max:660.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012706033885478973 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5593209099024534) - timing_s/agent_loop/generate_sequences/max:np.float64(12.716230429708958) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.830025155824842) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.716230429708958) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1334 - timing_s/gen:19.76116858050227 - timing_s/reward:0.08693374507129192 - timing_s/old_log_prob:3.292963979765773 - timing_s/adv:0.1231906358152628 - timing_s/update_actor:13.490689365193248 - timing_s/step:36.84112096019089 - timing_s/stop_profile:0.00011323578655719757 - timing_per_token_ms/adv:0.0004918948407619471 - timing_per_token_ms/gen:0.11162925338512793 - timing_per_token_ms/update_actor:0.05386773477662702 - perf/total_num_tokens:250441 - perf/time_per_step:36.84112096019089 - perf/throughput:849.7332378628523 (TaskRunner pid=2224264) Training Progress: 57%|█████▋ | 57/100 [53:56<32:14, 44.98s/it] (TaskRunner pid=2224264) step:58 - global_seqlen/min:26170 - global_seqlen/max:34255 - global_seqlen/minmax_diff:8085 - global_seqlen/balanced_min:30104 - global_seqlen/balanced_max:30420 - global_seqlen/mean:30161.5 - actor/entropy:0.7656847238540649 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16260474920272827 - training/rollout_probs_diff_mean:0.00442152377218008 - training/rollout_probs_diff_std:0.00836502481251955 - training/rollout_actor_probs_pearson_corr:0.9997066259384155 - rollout_corr/rollout_is_mean:0.9998526573181152 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.867856089025736e-06 - rollout_corr/rollout_is_max:1.6352856159210205 - rollout_corr/rollout_is_min:0.49183452129364014 - rollout_corr/rollout_is_std:0.034748680889606476 - rollout_corr/rollout_is_eff_sample_size:0.9987937474899505 - rollout_corr/rollout_is_seq_mean:0.9999055862426758 - rollout_corr/rollout_is_seq_std:0.0014970327029004693 - rollout_corr/rollout_is_seq_max:1.0064585208892822 - rollout_corr/rollout_is_seq_min:0.9960758090019226 - rollout_corr/rollout_is_seq_max_deviation:0.0064585208892822266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.18733112514019) - rollout_corr/training_log_ppl:np.float64(0.7628000770928338) - rollout_corr/kl:np.float64(0.0007255446162326429) - rollout_corr/k3_kl:np.float64(0.0006535338600883733) - rollout_corr/rollout_ppl:np.float64(2.185690295882523) - rollout_corr/rollout_log_ppl:np.float64(0.7620745294261724) - rollout_corr/log_ppl_diff:np.float64(0.0007255476666614413) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013074638554826379) - rollout_corr/log_ppl_diff_max:np.float64(0.005662679672241211) - rollout_corr/log_ppl_diff_min:np.float64(-0.00701594352722168) - rollout_corr/ppl_ratio:np.float64(1.000726986443624) - rollout_corr/chi2_token:np.float64(0.0011695893481373787) - rollout_corr/chi2_seq:np.float64(1.832958000479266) - actor/pg_loss:np.float64(1.464621163904667e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012746554830300738) - actor/ppo_kl:np.float64(9.933237482329105e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2319943569600582) - perf/mfu/actor:np.float64(0.11497644796674522) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.57088470458984) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002008273731917143 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002008273731917143 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:665.703125 - response_length/max:2537.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:665.703125 - response_length_non_aborted/max:2537.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:598.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001398194581270218 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0110350400209427) - timing_s/agent_loop/generate_sequences/max:np.float64(19.426406839862466) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.346314342365076) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.426406839862466) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:199 - timing_s/agent_loop/slowest/response_length:2537 - timing_s/gen:26.27677822113037 - timing_s/reward:0.07973587512969971 - timing_s/old_log_prob:3.217642493546009 - timing_s/adv:0.12296506203711033 - timing_s/update_actor:13.712465612217784 - timing_s/step:43.495522832497954 - timing_s/stop_profile:0.00010489486157894135 - timing_per_token_ms/adv:0.0005096110191681047 - timing_per_token_ms/gen:0.15418834773577264 - timing_per_token_ms/update_actor:0.056829342092642045 - perf/total_num_tokens:241292 - perf/time_per_step:43.495522832497954 - perf/throughput:693.4391872044505 (TaskRunner pid=2224264) Training Progress: 58%|█████▊ | 58/100 [54:39<31:11, 44.55s/it] (TaskRunner pid=2224264) step:59 - global_seqlen/min:27515 - global_seqlen/max:33648 - global_seqlen/minmax_diff:6133 - global_seqlen/balanced_min:30606 - global_seqlen/balanced_max:31185 - global_seqlen/mean:30683.625 - actor/entropy:0.7695986032485962 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4610300064086914 - training/rollout_probs_diff_mean:0.004400559701025486 - training/rollout_probs_diff_std:0.008376654237508774 - training/rollout_actor_probs_pearson_corr:0.9996274709701538 - rollout_corr/rollout_is_mean:1.0000059604644775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.658765076077543e-06 - rollout_corr/rollout_is_max:1.8190242052078247 - rollout_corr/rollout_is_min:0.3681882917881012 - rollout_corr/rollout_is_std:0.03469890356063843 - rollout_corr/rollout_is_eff_sample_size:0.9987974340815278 - rollout_corr/rollout_is_seq_mean:0.9999397397041321 - rollout_corr/rollout_is_seq_std:0.0016430045943707228 - rollout_corr/rollout_is_seq_max:1.0059504508972168 - rollout_corr/rollout_is_seq_min:0.9930161237716675 - rollout_corr/rollout_is_seq_max_deviation:0.0069838762283325195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2372520426288247) - rollout_corr/training_log_ppl:np.float64(0.7862228258745745) - rollout_corr/kl:np.float64(0.0008618131305837551) - rollout_corr/k3_kl:np.float64(0.0006665370408427407) - rollout_corr/rollout_ppl:np.float64(2.2352824225090444) - rollout_corr/rollout_log_ppl:np.float64(0.7853610130259767) - rollout_corr/log_ppl_diff:np.float64(0.000861812848597765) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014634081162512302) - rollout_corr/log_ppl_diff_max:np.float64(0.008165895938873291) - rollout_corr/log_ppl_diff_min:np.float64(-0.005269348621368408) - rollout_corr/ppl_ratio:np.float64(1.0008637136779726) - rollout_corr/chi2_token:np.float64(0.000947984866797924) - rollout_corr/chi2_seq:np.float64(1.44492331543006) - actor/pg_loss:np.float64(-2.2010761313140392e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016003822292987024) - actor/ppo_kl:np.float64(0.00017181198663673847) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20559313893318176) - perf/mfu/actor:np.float64(0.11867857354144504) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(104.51495361328125) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009747223230078816 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009747223230078816 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:690.30078125 - response_length/max:2398.0 - response_length/min:212.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:690.30078125 - response_length_non_aborted/max:2398.0 - response_length_non_aborted/min:212.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:615.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012865476310253143 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7066904306411743) - timing_s/agent_loop/generate_sequences/max:np.float64(18.255667094141245) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.844594236710691) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.255667094141245) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:2398 - timing_s/gen:25.205973524600267 - timing_s/reward:0.07148762419819832 - timing_s/old_log_prob:3.309969963505864 - timing_s/adv:0.11456507630646229 - timing_s/update_actor:13.433156419545412 - timing_s/step:42.23662491515279 - timing_s/stop_profile:0.00011372379958629608 - timing_per_token_ms/adv:0.00046671912260392266 - timing_per_token_ms/gen:0.1426346844084059 - timing_per_token_ms/update_actor:0.054724451639699564 - perf/total_num_tokens:245469 - perf/time_per_step:42.23662491515279 - perf/throughput:726.4696234994847 (TaskRunner pid=2224264) Training Progress: 59%|█████▉ | 59/100 [55:21<29:58, 43.87s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a product molecule: (TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a condensed chemical formula. Let's interpret it step by step. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 1: Parse the product formula (TaskRunner pid=2224264) The formula: (TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a shorthand for a benzene ring with substituents. Let's break it down: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - **Nc1ccn(...)**: This is a nitrogen-containing ring. (TaskRunner pid=2224264) - **Cc2ccc(Cl)cc2C(F)(F)F**: This is a side chain attached to the ring via the nitrogen. It contains a benzene ring with a Cl substituent and a trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2224264) (TaskRunner pid=2224264) So the product is likely a **nitrogen-containing heterocycle (e.g., pyridine or pyrrole)**, with a **benzene ring bearing a Cl and a CF₃ group** attached to it. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 2: Understand what we are looking for (TaskRunner pid=2224264) We are to find the correct **reactant** that would be used in the synthesis of this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) The options are all chemical formulas that may be reactants in a synthesis. We must find the one that matches the **structural features of the product**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 3: Analyze each option (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option A: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a **diacyl compound** (two C=O groups), and it includes a **nitrogen ring** with the same side chain as the product: (TaskRunner pid=2224264) - `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` (TaskRunner pid=2224264) This matches the **side chain** of the product. (TaskRunner pid=2224264) - The `O=C1c2ccccc2C(=O)N1` part suggests a **diketone ring** (like a benzene ring with two ketone groups), which may be a **starting material** for forming the nitrogen ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This **looks like a good candidate** as it contains the same side chain and a structure that could be used to form the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option B: (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This includes a Cl group on the ring and a nitrogen, but it **does not match the side chain** of the product. It also has a different structure and is not consistent with the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option C: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **trifluoromethyl group** and a **Cl group**, but the **side chain is different**. It has a **hydroxyl group** that is not in the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) #### Option D: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **CF₃ group** and a **benzene ring**, but the **side chain is different**. It does not match the structure of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Step 4: Final determination (TaskRunner pid=2224264) Only **Option A** includes the **same side chain** as the product and includes a **ring structure** that could be used in the synthesis of the nitrogen-containing ring in the product. It is the only one that matches the **structural features** of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2224264) step:60 - global_seqlen/min:28478 - global_seqlen/max:38447 - global_seqlen/minmax_diff:9969 - global_seqlen/balanced_min:32368 - global_seqlen/balanced_max:32372 - global_seqlen/mean:32370.0 - actor/entropy:0.8034039735794067 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4281436800956726 - training/rollout_probs_diff_mean:0.004418870899826288 - training/rollout_probs_diff_std:0.008324537426233292 - training/rollout_actor_probs_pearson_corr:0.9996195435523987 - rollout_corr/rollout_is_mean:1.0001076459884644 - rollout_corr/rollout_is_ratio_fraction_high:1.047778732754523e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.238893663772615e-06 - rollout_corr/rollout_is_max:2.1436619758605957 - rollout_corr/rollout_is_min:0.026059480383992195 - rollout_corr/rollout_is_std:0.0351596400141716 - rollout_corr/rollout_is_eff_sample_size:0.9987655638020635 - rollout_corr/rollout_is_seq_mean:1.0000665187835693 - rollout_corr/rollout_is_seq_std:0.0014459878439083695 - rollout_corr/rollout_is_seq_max:1.005103349685669 - rollout_corr/rollout_is_seq_min:0.9960857629776001 - rollout_corr/rollout_is_seq_max_deviation:0.005103349685668945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2919784113764763) - rollout_corr/training_log_ppl:np.float64(0.806903613905888) - rollout_corr/kl:np.float64(0.0006409502570816628) - rollout_corr/k3_kl:np.float64(0.0006509173431368254) - rollout_corr/rollout_ppl:np.float64(2.290479706134647) - rollout_corr/rollout_log_ppl:np.float64(0.8062626618193462) - rollout_corr/log_ppl_diff:np.float64(0.0006409520865418017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012293330510146916) - rollout_corr/log_ppl_diff_max:np.float64(0.005213141441345215) - rollout_corr/log_ppl_diff_min:np.float64(-0.004137933254241943) - rollout_corr/ppl_ratio:np.float64(1.0006422218866646) - rollout_corr/chi2_token:np.float64(0.0013303321320563555) - rollout_corr/chi2_seq:np.float64(2.3223254224285483) - actor/pg_loss:np.float64(-0.0001239327248185873) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011024115792679368) - actor/ppo_kl:np.float64(6.963595266995526e-05) - actor/pg_clipfrac_lower:np.float64(2.784212483675219e-06) - actor/grad_norm:np.float64(0.24239107966423035) - perf/mfu/actor:np.float64(0.12592590460533137) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.14717817306519) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6589285714285714) - val-aux/sciknoweval/reward/std@16:np.float64(0.26147699146732906) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7634047619047619) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19852720557117806) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5552285714285714) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2397040092225637) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6592000000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.26073502970265483) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8421285714285713) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13617544486794253) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46132857142857137) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.20013008279957237) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6862619047619049) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20619876382539531) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8954952380952381) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08197180786429441) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3817190476190476) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.1582460634427849) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6993809523809524) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14750004177458542) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9245333333333334) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03779423823528531) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.31384761904761904) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.11509261056785722) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7040333333333333) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.108925053237339) - val-aux/sciknoweval/score/mean@16:np.float64(0.6589285714285714) - val-aux/sciknoweval/score/std@16:np.float64(0.26147699146732906) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7634047619047619) - val-aux/sciknoweval/score/best@2/std:np.float64(0.19852720557117806) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5552285714285714) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2397040092225637) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6592000000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.26073502970265483) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8421285714285713) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13617544486794253) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.46132857142857137) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.20013008279957237) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6862619047619049) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20619876382539531) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8954952380952381) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08197180786429441) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3817190476190476) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.1582460634427849) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6993809523809524) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14750004177458542) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9245333333333334) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03779423823528531) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.31384761904761904) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.11509261056785722) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7040333333333333) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.108925053237339) - val-core/sciknoweval/acc/mean@16:np.float64(0.6589285714285714) - val-aux/sciknoweval/acc/std@16:np.float64(0.26147699146732906) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7634047619047619) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.19852720557117806) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5552285714285714) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2397040092225637) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6592000000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.26073502970265483) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8421285714285713) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13617544486794253) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.46132857142857137) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.20013008279957237) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6862619047619049) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20619876382539531) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8954952380952381) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08197180786429441) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3817190476190476) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.1582460634427849) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6993809523809524) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14750004177458542) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9245333333333334) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03779423823528531) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.31384761904761904) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.11509261056785722) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7040333333333333) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.108925053237339) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.986904761904762) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.036673747734753556) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9979476190476191) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.012620842930515878) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9751857142857142) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04786735705451152) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9864428571428573) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.037432914347073416) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998619047619047) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0017967329312416017) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9577666666666667) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05639443888290546) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9949285714285714) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.022714201842546834) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9332380952380953) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05880143064108464) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9987619047619049) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009403419069561486) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9058904761904761) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.050590370282386964) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9997761904761904) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002838983070955884) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:1 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001474093645811081 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001474093645811081 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:745.625 - response_length/max:1629.0 - response_length/min:179.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:745.625 - response_length_non_aborted/max:1629.0 - response_length_non_aborted/min:179.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.9375 - prompt_length/max:451.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00029304809868335724 - timing_s/agent_loop/generate_sequences/min:np.float64(2.268099818378687) - timing_s/agent_loop/generate_sequences/max:np.float64(14.966608583927155) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.466599339502864) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.966608583927155) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:1629 - timing_s/gen:21.801297603175044 - timing_s/reward:0.08670702017843723 - timing_s/old_log_prob:3.37482875213027 - timing_s/adv:0.1377221718430519 - timing_s/update_actor:13.325021024793386 - timing_s/step:38.84823939576745 - timing_s/testing:189.37862921506166 - timing_s/save_checkpoint:12.842970063909888 - timing_s/stop_profile:0.00024105235934257507 - timing_per_token_ms/adv:0.0005318279728261195 - timing_per_token_ms/gen:0.11421467730079131 - timing_per_token_ms/update_actor:0.05145590448252003 - perf/total_num_tokens:258960 - perf/time_per_step:38.84823939576745 - perf/throughput:833.2423940820015 (TaskRunner pid=2224264) Training Progress: 60%|██████ | 60/100 [59:23<1:08:47, 103.19s/it] (TaskRunner pid=2224264) step:61 - global_seqlen/min:26866 - global_seqlen/max:38557 - global_seqlen/minmax_diff:11691 - global_seqlen/balanced_min:32591 - global_seqlen/balanced_max:32615 - global_seqlen/mean:32607.625 - actor/entropy:0.7484701871871948 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.32022327184677124 - training/rollout_probs_diff_mean:0.004327286966145039 - training/rollout_probs_diff_std:0.008308346383273602 - training/rollout_actor_probs_pearson_corr:0.9996346235275269 - rollout_corr/rollout_is_mean:1.0000005960464478 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5481393347727135e-05 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:0.3159535825252533 - rollout_corr/rollout_is_std:0.03432586044073105 - rollout_corr/rollout_is_eff_sample_size:0.9988231220557214 - rollout_corr/rollout_is_seq_mean:1.0000786781311035 - rollout_corr/rollout_is_seq_std:0.001447982620447874 - rollout_corr/rollout_is_seq_max:1.0040830373764038 - rollout_corr/rollout_is_seq_min:0.9964959025382996 - rollout_corr/rollout_is_seq_max_deviation:0.004083037376403809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.167487114202231) - rollout_corr/training_log_ppl:np.float64(0.7570779783418402) - rollout_corr/kl:np.float64(0.0007175489161515003) - rollout_corr/k3_kl:np.float64(0.0006208742135527245) - rollout_corr/rollout_ppl:np.float64(2.1659105462022126) - rollout_corr/rollout_log_ppl:np.float64(0.7563604272436351) - rollout_corr/log_ppl_diff:np.float64(0.0007175510982051492) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012418738333508372) - rollout_corr/log_ppl_diff_max:np.float64(0.007266104221343994) - rollout_corr/log_ppl_diff_min:np.float64(-0.003547847270965576) - rollout_corr/ppl_ratio:np.float64(1.0007189356256276) - rollout_corr/chi2_token:np.float64(0.0010508422274142504) - rollout_corr/chi2_seq:np.float64(1.2128563108853996) - actor/pg_loss:np.float64(7.9984194599092e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012339052045717835) - actor/ppo_kl:np.float64(0.0001892128098228696) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21570049226284027) - perf/mfu/actor:np.float64(0.1265814073644025) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.56413173675537) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024512207601219416 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024512207601219416 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:756.95703125 - response_length/max:1627.0 - response_length/min:211.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:756.95703125 - response_length_non_aborted/max:1627.0 - response_length_non_aborted/min:211.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.03125 - prompt_length/max:462.0 - prompt_length/min:177.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018923170864582062 - timing_s/agent_loop/generate_sequences/min:np.float64(2.607853963971138) - timing_s/agent_loop/generate_sequences/max:np.float64(14.494692988693714) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.566406984020432) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.494692988693714) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:312 - timing_s/agent_loop/slowest/response_length:1627 - timing_s/gen:20.789081823080778 - timing_s/reward:0.07305939309298992 - timing_s/old_log_prob:3.3369578402489424 - timing_s/adv:0.11469490267336369 - timing_s/update_actor:13.50451728142798 - timing_s/step:37.91396317072213 - timing_s/stop_profile:0.00010998733341693878 - timing_per_token_ms/adv:0.00043967822968310206 - timing_per_token_ms/gen:0.10728132181731324 - timing_per_token_ms/update_actor:0.05176901599483242 - perf/total_num_tokens:260861 - perf/time_per_step:37.91396317072213 - perf/throughput:860.0426405747057 (TaskRunner pid=2224264) Training Progress: 61%|██████ | 61/100 [1:00:01<54:21, 83.62s/it] (TaskRunner pid=2224264) step:62 - global_seqlen/min:27396 - global_seqlen/max:35660 - global_seqlen/minmax_diff:8264 - global_seqlen/balanced_min:31880 - global_seqlen/balanced_max:31902 - global_seqlen/mean:31893.125 - actor/entropy:0.8587515950202942 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2571102976799011 - training/rollout_probs_diff_mean:0.004673279356211424 - training/rollout_probs_diff_std:0.008435208350419998 - training/rollout_actor_probs_pearson_corr:0.9996413588523865 - rollout_corr/rollout_is_mean:0.9998844861984253 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9304524660110474 - rollout_corr/rollout_is_min:0.5585559010505676 - rollout_corr/rollout_is_std:0.03600391373038292 - rollout_corr/rollout_is_eff_sample_size:0.9987051585549779 - rollout_corr/rollout_is_seq_mean:0.9999316930770874 - rollout_corr/rollout_is_seq_std:0.0016182069666683674 - rollout_corr/rollout_is_seq_max:1.0045205354690552 - rollout_corr/rollout_is_seq_min:0.9909459948539734 - rollout_corr/rollout_is_seq_max_deviation:0.009054005146026611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.4564432627521455) - rollout_corr/training_log_ppl:np.float64(0.8728480438003317) - rollout_corr/kl:np.float64(0.000761049760463095) - rollout_corr/k3_kl:np.float64(0.000696839270858618) - rollout_corr/rollout_ppl:np.float64(2.454547442961484) - rollout_corr/rollout_log_ppl:np.float64(0.8720869926037267) - rollout_corr/log_ppl_diff:np.float64(0.0007610511966049671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013679289259016514) - rollout_corr/log_ppl_diff_max:np.float64(0.009840637445449829) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033975839614868164) - rollout_corr/ppl_ratio:np.float64(1.0007626954466105) - rollout_corr/chi2_token:np.float64(0.001287585822865367) - rollout_corr/chi2_seq:np.float64(1.209890850353986) - actor/pg_loss:np.float64(-3.718864172697067e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001223060548909416) - actor/ppo_kl:np.float64(2.9713928825003677e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29783353582024574) - perf/mfu/actor:np.float64(0.12234924411478575) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.59233093261719) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006781507283449173 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006781507283449173 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:697.41015625 - response_length/max:1847.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:697.41015625 - response_length_non_aborted/max:1847.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.25 - prompt_length/max:662.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017420388758182526 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4971247743815184) - timing_s/agent_loop/generate_sequences/max:np.float64(15.592459442093968) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.807667781009513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.592459442093968) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:171 - timing_s/agent_loop/slowest/response_length:1847 - timing_s/gen:22.337838161736727 - timing_s/reward:0.07594001106917858 - timing_s/old_log_prob:3.2577473018318415 - timing_s/adv:0.1236040536314249 - timing_s/update_actor:13.773704160004854 - timing_s/step:39.65648138895631 - timing_s/stop_profile:0.0001120399683713913 - timing_per_token_ms/adv:0.00048444630947666974 - timing_per_token_ms/gen:0.12511601607362466 - timing_per_token_ms/update_actor:0.05398382943034296 - perf/total_num_tokens:255145 - perf/time_per_step:39.65648138895631 - perf/throughput:804.2348661039232 (TaskRunner pid=2224264) Training Progress: 62%|██████▏ | 62/100 [1:00:41<44:37, 70.45s/it] (TaskRunner pid=2224264) step:63 - global_seqlen/min:26919 - global_seqlen/max:40692 - global_seqlen/minmax_diff:13773 - global_seqlen/balanced_min:33436 - global_seqlen/balanced_max:33474 - global_seqlen/mean:33459.75 - actor/entropy:0.7692099809646606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975038170814514 - training/rollout_probs_diff_mean:0.0043107387609779835 - training/rollout_probs_diff_std:0.008506524376571178 - training/rollout_actor_probs_pearson_corr:0.9996923804283142 - rollout_corr/rollout_is_mean:0.999955415725708 - rollout_corr/rollout_is_ratio_fraction_high:5.143874204804888e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.143874204804888e-06 - rollout_corr/rollout_is_max:2.5493264198303223 - rollout_corr/rollout_is_min:0.3577776551246643 - rollout_corr/rollout_is_std:0.03481551632285118 - rollout_corr/rollout_is_eff_sample_size:0.9987891095585573 - rollout_corr/rollout_is_seq_mean:1.000030755996704 - rollout_corr/rollout_is_seq_std:0.0014692960539832711 - rollout_corr/rollout_is_seq_max:1.0053269863128662 - rollout_corr/rollout_is_seq_min:0.9961836934089661 - rollout_corr/rollout_is_seq_max_deviation:0.005326986312866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.23876611655578) - rollout_corr/training_log_ppl:np.float64(0.7817316737491637) - rollout_corr/kl:np.float64(0.0004539784086303733) - rollout_corr/k3_kl:np.float64(0.0006596663926075053) - rollout_corr/rollout_ppl:np.float64(2.2376751606352627) - rollout_corr/rollout_log_ppl:np.float64(0.781277688452974) - rollout_corr/log_ppl_diff:np.float64(0.000453985296189785) - rollout_corr/log_ppl_abs_diff:np.float64(0.001180113758891821) - rollout_corr/log_ppl_diff_max:np.float64(0.005360126495361328) - rollout_corr/log_ppl_diff_min:np.float64(-0.004343867301940918) - rollout_corr/ppl_ratio:np.float64(1.0004551277961582) - rollout_corr/chi2_token:np.float64(0.001740993233397603) - rollout_corr/chi2_seq:np.float64(1.6487001434434205) - actor/pg_loss:np.float64(-0.00019339460413902998) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018608193454383581) - actor/ppo_kl:np.float64(-0.00013630927301377938) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25826945155858994) - perf/mfu/actor:np.float64(0.12769393570296111) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.44427490234375) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002252373844385147 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002252373844385147 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:759.3984375 - response_length/max:1951.0 - response_length/min:194.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:759.3984375 - response_length_non_aborted/max:1951.0 - response_length_non_aborted/min:194.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.21875 - prompt_length/max:532.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010393001139163971 - timing_s/agent_loop/generate_sequences/min:np.float64(2.39320345595479) - timing_s/agent_loop/generate_sequences/max:np.float64(15.86801179125905) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.712352399430529) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.86801179125905) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:1951 - timing_s/gen:23.052989764139056 - timing_s/reward:0.07871297188103199 - timing_s/old_log_prob:3.3043668009340763 - timing_s/adv:0.11869384348392487 - timing_s/update_actor:13.733131846413016 - timing_s/step:40.37277623265982 - timing_s/stop_profile:0.00011059269309043884 - timing_per_token_ms/adv:0.0004434202417977005 - timing_per_token_ms/gen:0.1185816783645518 - timing_per_token_ms/update_actor:0.05130467145754607 - perf/total_num_tokens:267678 - perf/time_per_step:40.37277623265982 - perf/throughput:828.7701050623444 (TaskRunner pid=2224264) Training Progress: 63%|██████▎ | 63/100 [1:01:21<37:53, 61.44s/it] (TaskRunner pid=2224264) step:64 - global_seqlen/min:26306 - global_seqlen/max:34933 - global_seqlen/minmax_diff:8627 - global_seqlen/balanced_min:30904 - global_seqlen/balanced_max:30933 - global_seqlen/mean:30923.0 - actor/entropy:0.7835981845855713 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3175749182701111 - training/rollout_probs_diff_mean:0.004405513871461153 - training/rollout_probs_diff_std:0.008360867388546467 - training/rollout_actor_probs_pearson_corr:0.9996961355209351 - rollout_corr/rollout_is_mean:0.9999273419380188 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1598237506404985e-05 - rollout_corr/rollout_is_max:1.7908501625061035 - rollout_corr/rollout_is_min:0.2933122217655182 - rollout_corr/rollout_is_std:0.03481380268931389 - rollout_corr/rollout_is_eff_sample_size:0.9987892284793369 - rollout_corr/rollout_is_seq_mean:0.9998888373374939 - rollout_corr/rollout_is_seq_std:0.0014235756825655699 - rollout_corr/rollout_is_seq_max:1.0034641027450562 - rollout_corr/rollout_is_seq_min:0.9946786165237427 - rollout_corr/rollout_is_seq_max_deviation:0.005321383476257324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.228744763880968) - rollout_corr/training_log_ppl:np.float64(0.7778443787246943) - rollout_corr/kl:np.float64(0.000828439747124321) - rollout_corr/k3_kl:np.float64(0.0006999520201134146) - rollout_corr/rollout_ppl:np.float64(2.2268494172021747) - rollout_corr/rollout_log_ppl:np.float64(0.7770159327192232) - rollout_corr/log_ppl_diff:np.float64(0.0008284460054710507) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013873394345864654) - rollout_corr/log_ppl_diff_max:np.float64(0.007387757301330566) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033279061317443848) - rollout_corr/ppl_ratio:np.float64(1.0008300342597067) - rollout_corr/chi2_token:np.float64(0.0011283555068075657) - rollout_corr/chi2_seq:np.float64(1.5243452661670744) - actor/pg_loss:np.float64(-7.703946903347969e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002916122386977804) - actor/ppo_kl:np.float64(8.220684872384254e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.241146732121706) - perf/mfu/actor:np.float64(0.12080489767203606) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.29822540283203) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005282997153699398 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005282997153699398 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:673.59375 - response_length/max:1796.0 - response_length/min:176.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:673.59375 - response_length_non_aborted/max:1796.0 - response_length_non_aborted/min:176.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.75 - prompt_length/max:495.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012784451246261597 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1317031513899565) - timing_s/agent_loop/generate_sequences/max:np.float64(14.859888169914484) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.550355381361442) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.859888169914484) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1796 - timing_s/gen:21.599143993109465 - timing_s/reward:0.07637070119380951 - timing_s/old_log_prob:3.1958085894584656 - timing_s/adv:0.11453193984925747 - timing_s/update_actor:13.29977210238576 - timing_s/step:38.375933611765504 - timing_s/stop_profile:0.00011328049004077911 - timing_per_token_ms/adv:0.0004629723015605596 - timing_per_token_ms/gen:0.12525599624860512 - timing_per_token_ms/update_actor:0.053761650318475566 - perf/total_num_tokens:247384 - perf/time_per_step:38.375933611765504 - perf/throughput:805.7914711036361 (TaskRunner pid=2224264) Training Progress: 64%|██████▍ | 64/100 [1:01:59<32:43, 54.53s/it] (TaskRunner pid=2224264) step:65 - global_seqlen/min:24032 - global_seqlen/max:34111 - global_seqlen/minmax_diff:10079 - global_seqlen/balanced_min:29951 - global_seqlen/balanced_max:30559 - global_seqlen/mean:30032.625 - actor/entropy:0.7714234590530396 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28636491298675537 - training/rollout_probs_diff_mean:0.004574572201818228 - training/rollout_probs_diff_std:0.008538489229977131 - training/rollout_actor_probs_pearson_corr:0.9996505379676819 - rollout_corr/rollout_is_mean:0.9998939633369446 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6290990114212036 - rollout_corr/rollout_is_min:0.5353134870529175 - rollout_corr/rollout_is_std:0.03545675054192543 - rollout_corr/rollout_is_eff_sample_size:0.9987440406221719 - rollout_corr/rollout_is_seq_mean:0.9999128580093384 - rollout_corr/rollout_is_seq_std:0.0015794334467500448 - rollout_corr/rollout_is_seq_max:1.0056021213531494 - rollout_corr/rollout_is_seq_min:0.9935817122459412 - rollout_corr/rollout_is_seq_max_deviation:0.006418287754058838 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.22273992234841) - rollout_corr/training_log_ppl:np.float64(0.7840376188978553) - rollout_corr/kl:np.float64(0.0006947023282570797) - rollout_corr/k3_kl:np.float64(0.0007119409705182989) - rollout_corr/rollout_ppl:np.float64(2.221208442468196) - rollout_corr/rollout_log_ppl:np.float64(0.7833429137244821) - rollout_corr/log_ppl_diff:np.float64(0.0006947051733732224) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013338688295334578) - rollout_corr/log_ppl_diff_max:np.float64(0.0072593092918396) - rollout_corr/log_ppl_diff_min:np.float64(-0.004377126693725586) - rollout_corr/ppl_ratio:np.float64(1.0006962057668716) - rollout_corr/chi2_token:np.float64(0.0014604993630200624) - rollout_corr/chi2_seq:np.float64(0.6011870829388499) - actor/pg_loss:np.float64(0.0001900325296446681) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003155937018846089) - actor/ppo_kl:np.float64(-3.965757715462814e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28370653092861176) - perf/mfu/actor:np.float64(0.11707659877392018) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.31705093383789) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018750634044408798 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018750634044408798 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:654.92578125 - response_length/max:2451.0 - response_length/min:231.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:654.92578125 - response_length_non_aborted/max:2451.0 - response_length_non_aborted/min:231.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:818.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011583603918552399 - timing_s/agent_loop/generate_sequences/min:np.float64(2.9298688657581806) - timing_s/agent_loop/generate_sequences/max:np.float64(18.389138467609882) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.613169322678004) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.389138467609882) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:2451 - timing_s/gen:25.793642738834023 - timing_s/reward:0.07493023946881294 - timing_s/old_log_prob:3.18870216794312 - timing_s/adv:0.1171203050762415 - timing_s/update_actor:13.542755868285894 - timing_s/step:42.80827572196722 - timing_s/stop_profile:0.00011235475540161133 - timing_per_token_ms/adv:0.00048747114627942733 - timing_per_token_ms/gen:0.15384402299183486 - timing_per_token_ms/update_actor:0.0563668505012711 - perf/total_num_tokens:240261 - perf/time_per_step:42.80827572196722 - perf/throughput:701.5611933322662 (TaskRunner pid=2224264) Training Progress: 65%|██████▌ | 65/100 [1:02:42<29:46, 51.03s/it] (TaskRunner pid=2224264) step:66 - global_seqlen/min:28697 - global_seqlen/max:35089 - global_seqlen/minmax_diff:6392 - global_seqlen/balanced_min:30450 - global_seqlen/balanced_max:30469 - global_seqlen/mean:30460.5 - actor/entropy:0.7473804354667664 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424738645553589 - training/rollout_probs_diff_mean:0.004387002903968096 - training/rollout_probs_diff_std:0.008388126268982887 - training/rollout_actor_probs_pearson_corr:0.9996775388717651 - rollout_corr/rollout_is_mean:0.9999525547027588 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1807769624283537e-05 - rollout_corr/rollout_is_max:1.7755447626113892 - rollout_corr/rollout_is_min:0.3809736371040344 - rollout_corr/rollout_is_std:0.0348651260137558 - rollout_corr/rollout_is_eff_sample_size:0.9987857797882222 - rollout_corr/rollout_is_seq_mean:0.999984622001648 - rollout_corr/rollout_is_seq_std:0.001510056434199214 - rollout_corr/rollout_is_seq_max:1.0043302774429321 - rollout_corr/rollout_is_seq_min:0.9965217709541321 - rollout_corr/rollout_is_seq_max_deviation:0.004330277442932129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1832842687144876) - rollout_corr/training_log_ppl:np.float64(0.7604161555645987) - rollout_corr/kl:np.float64(0.0006697535459316839) - rollout_corr/k3_kl:np.float64(0.0006770667089313065) - rollout_corr/rollout_ppl:np.float64(2.18177503393963) - rollout_corr/rollout_log_ppl:np.float64(0.7597463995916769) - rollout_corr/log_ppl_diff:np.float64(0.0006697559729218483) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012636978644877672) - rollout_corr/log_ppl_diff_max:np.float64(0.005221843719482422) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034908056259155273) - rollout_corr/ppl_ratio:np.float64(1.0006710432935506) - rollout_corr/chi2_token:np.float64(0.001379573019221425) - rollout_corr/chi2_seq:np.float64(0.7963614012114704) - actor/pg_loss:np.float64(-9.32483235374093e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021860310857846343) - actor/ppo_kl:np.float64(3.254798675556003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2768869362771511) - perf/mfu/actor:np.float64(0.11916675434662388) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.34953784942627) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009460975416004658 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009460975416004658 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:661.640625 - response_length/max:1563.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:661.640625 - response_length_non_aborted/max:1563.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.25 - prompt_length/max:617.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000106053426861763 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3329896610230207) - timing_s/agent_loop/generate_sequences/max:np.float64(13.444917742162943) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.728698585859092) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.444917742162943) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:1563 - timing_s/gen:20.854996770620346 - timing_s/reward:0.0732195358723402 - timing_s/old_log_prob:3.1821087524294853 - timing_s/adv:0.1175671610981226 - timing_s/update_actor:13.4094133451581 - timing_s/step:37.7182090356946 - timing_s/stop_profile:0.00011660158634185791 - timing_per_token_ms/adv:0.0004824574493939799 - timing_per_token_ms/gen:0.12312549752403086 - timing_per_token_ms/update_actor:0.05502787768240057 - perf/total_num_tokens:243684 - perf/time_per_step:37.7182090356946 - perf/throughput:807.5807621505498 (TaskRunner pid=2224264) Training Progress: 66%|██████▌ | 66/100 [1:03:20<26:39, 47.05s/it] (TaskRunner pid=2224264) step:67 - global_seqlen/min:21976 - global_seqlen/max:31921 - global_seqlen/minmax_diff:9945 - global_seqlen/balanced_min:28720 - global_seqlen/balanced_max:28745 - global_seqlen/mean:28734.25 - actor/entropy:0.7726788520812988 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3997594714164734 - training/rollout_probs_diff_mean:0.004599802196025848 - training/rollout_probs_diff_std:0.008748593740165234 - training/rollout_actor_probs_pearson_corr:0.9996384978294373 - rollout_corr/rollout_is_mean:1.0000046491622925 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9334631360834464e-05 - rollout_corr/rollout_is_max:1.7132740020751953 - rollout_corr/rollout_is_min:0.35777419805526733 - rollout_corr/rollout_is_std:0.03550714626908302 - rollout_corr/rollout_is_eff_sample_size:0.9987409489708702 - rollout_corr/rollout_is_seq_mean:1.0000040531158447 - rollout_corr/rollout_is_seq_std:0.0016247251769527793 - rollout_corr/rollout_is_seq_max:1.005348563194275 - rollout_corr/rollout_is_seq_min:0.9943971037864685 - rollout_corr/rollout_is_seq_max_deviation:0.005602896213531494 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.254453372210264) - rollout_corr/training_log_ppl:np.float64(0.7888535652309656) - rollout_corr/kl:np.float64(0.0006227922752195525) - rollout_corr/k3_kl:np.float64(0.0006862304701940047) - rollout_corr/rollout_ppl:np.float64(2.2529089520685375) - rollout_corr/rollout_log_ppl:np.float64(0.7882307717809454) - rollout_corr/log_ppl_diff:np.float64(0.0006227934500202537) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014135913224890828) - rollout_corr/log_ppl_diff_max:np.float64(0.007272124290466309) - rollout_corr/log_ppl_diff_min:np.float64(-0.004362225532531738) - rollout_corr/ppl_ratio:np.float64(1.000624400563538) - rollout_corr/chi2_token:np.float64(0.001499387202784419) - rollout_corr/chi2_seq:np.float64(1.2466026986949146) - actor/pg_loss:np.float64(-2.0926236175000668e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024252940147562185) - actor/ppo_kl:np.float64(-2.8959387931237757e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29370710253715515) - perf/mfu/actor:np.float64(0.11386412359551698) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.32535934448242) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009934777393937111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009934777393937111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:606.1015625 - response_length/max:1545.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:606.1015625 - response_length_non_aborted/max:1545.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.84375 - prompt_length/max:576.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00022127665579319 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8514165561646223) - timing_s/agent_loop/generate_sequences/max:np.float64(13.32896644063294) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.998300799677963) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.32896644063294) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:193 - timing_s/agent_loop/slowest/response_length:1545 - timing_s/gen:20.499601708725095 - timing_s/reward:0.07158185541629791 - timing_s/old_log_prob:3.1310275867581367 - timing_s/adv:0.12111156806349754 - timing_s/update_actor:13.336501782760024 - timing_s/step:37.24595209956169 - timing_s/stop_profile:0.00011993572115898132 - timing_per_token_ms/adv:0.0005268606630741082 - timing_per_token_ms/gen:0.1321174108913593 - timing_per_token_ms/update_actor:0.058016573352184346 - perf/total_num_tokens:229874 - perf/time_per_step:37.24595209956169 - perf/throughput:771.4730965445812 (TaskRunner pid=2224264) Training Progress: 67%|██████▋ | 67/100 [1:03:57<24:16, 44.12s/it] (TaskRunner pid=2224264) step:68 - global_seqlen/min:23879 - global_seqlen/max:35240 - global_seqlen/minmax_diff:11361 - global_seqlen/balanced_min:29992 - global_seqlen/balanced_max:30029 - global_seqlen/mean:30017.25 - actor/entropy:0.7595304250717163 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7168671488761902 - training/rollout_probs_diff_mean:0.004404507111757994 - training/rollout_probs_diff_std:0.008633471094071865 - training/rollout_actor_probs_pearson_corr:0.9996846318244934 - rollout_corr/rollout_is_mean:0.999897301197052 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8815619114320725e-05 - rollout_corr/rollout_is_max:1.5993539094924927 - rollout_corr/rollout_is_min:0.019239548593759537 - rollout_corr/rollout_is_std:0.03493344411253929 - rollout_corr/rollout_is_eff_sample_size:0.9987809040931451 - rollout_corr/rollout_is_seq_mean:0.9998229742050171 - rollout_corr/rollout_is_seq_std:0.0016775608528405428 - rollout_corr/rollout_is_seq_max:1.005442500114441 - rollout_corr/rollout_is_seq_min:0.9936239123344421 - rollout_corr/rollout_is_seq_max_deviation:0.006376087665557861 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.246089845895767) - rollout_corr/training_log_ppl:np.float64(0.7848675699206069) - rollout_corr/kl:np.float64(0.0009016538949424557) - rollout_corr/k3_kl:np.float64(0.0007493318355500378) - rollout_corr/rollout_ppl:np.float64(2.243979589547962) - rollout_corr/rollout_log_ppl:np.float64(0.7839659146266058) - rollout_corr/log_ppl_diff:np.float64(0.0009016552940011024) - rollout_corr/log_ppl_abs_diff:np.float64(0.001556948758661747) - rollout_corr/log_ppl_diff_max:np.float64(0.012223243713378906) - rollout_corr/log_ppl_diff_min:np.float64(-0.004822075366973877) - rollout_corr/ppl_ratio:np.float64(1.0009038699790835) - rollout_corr/chi2_token:np.float64(0.0011472513433545828) - rollout_corr/chi2_seq:np.float64(2.487598191248253) - actor/pg_loss:np.float64(-4.085572436451912e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027236666869612236) - actor/ppo_kl:np.float64(5.479786341200388e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2867882438004017) - perf/mfu/actor:np.float64(0.11808746991936632) - perf/max_memory_allocated_gb:np.float64(131.9590311050415) - perf/max_memory_reserved_gb:np.float64(144.658203125) - perf/cpu_memory_used_gb:np.float64(105.29296493530273) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.41796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006193474982865155 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006193474982865155 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:622.8203125 - response_length/max:1786.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:622.8203125 - response_length_non_aborted/max:1786.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.21875 - prompt_length/max:704.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014729611575603485 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0367041416466236) - timing_s/agent_loop/generate_sequences/max:np.float64(14.751183833926916) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.172361494318466) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.751183833926916) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:1786 - timing_s/gen:22.098993657156825 - timing_s/reward:0.10232115164399147 - timing_s/old_log_prob:3.2531425300985575 - timing_s/adv:0.1175235714763403 - timing_s/update_actor:13.300975374877453 - timing_s/step:38.95930672250688 - timing_s/stop_profile:0.00012533552944660187 - timing_per_token_ms/adv:0.0004894001427360113 - timing_per_token_ms/gen:0.13860208512911795 - timing_per_token_ms/update_actor:0.05538888212143623 - perf/total_num_tokens:240138 - perf/time_per_step:38.95930672250688 - perf/throughput:770.4770060155862 (TaskRunner pid=2224264) Training Progress: 68%|██████▊ | 68/100 [1:04:36<22:42, 42.59s/it] (TaskRunner pid=2224264) step:69 - global_seqlen/min:24189 - global_seqlen/max:45366 - global_seqlen/minmax_diff:21177 - global_seqlen/balanced_min:33420 - global_seqlen/balanced_max:39649 - global_seqlen/mean:34239.875 - actor/entropy:0.7071983814239502 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38365769386291504 - training/rollout_probs_diff_mean:0.0040749297477304935 - training/rollout_probs_diff_std:0.00811900943517685 - training/rollout_actor_probs_pearson_corr:0.9996503591537476 - rollout_corr/rollout_is_mean:1.000025749206543 - rollout_corr/rollout_is_ratio_fraction_high:5.1385613915044814e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.1385613915044814e-06 - rollout_corr/rollout_is_max:2.4045958518981934 - rollout_corr/rollout_is_min:0.4110872149467468 - rollout_corr/rollout_is_std:0.03386257588863373 - rollout_corr/rollout_is_eff_sample_size:0.9988547581358254 - rollout_corr/rollout_is_seq_mean:1.0000370740890503 - rollout_corr/rollout_is_seq_std:0.0013557960046455264 - rollout_corr/rollout_is_seq_max:1.003797173500061 - rollout_corr/rollout_is_seq_min:0.9964458346366882 - rollout_corr/rollout_is_seq_max_deviation:0.003797173500061035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.177241276949644) - rollout_corr/training_log_ppl:np.float64(0.7569285913486965) - rollout_corr/kl:np.float64(0.0004947985197105709) - rollout_corr/k3_kl:np.float64(0.0006430674019384242) - rollout_corr/rollout_ppl:np.float64(2.1761399055831134) - rollout_corr/rollout_log_ppl:np.float64(0.7564337924704887) - rollout_corr/log_ppl_diff:np.float64(0.0004947988782078028) - rollout_corr/log_ppl_abs_diff:np.float64(0.001128945965319872) - rollout_corr/log_ppl_diff_max:np.float64(0.004031240940093994) - rollout_corr/log_ppl_diff_min:np.float64(-0.004697918891906738) - rollout_corr/ppl_ratio:np.float64(1.00049584871158) - rollout_corr/chi2_token:np.float64(0.001590353436768055) - rollout_corr/chi2_seq:np.float64(0.8412239120807499) - actor/pg_loss:np.float64(-7.094477768987417e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019783686673235934) - actor/ppo_kl:np.float64(-9.313821129719457e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25627975165843964) - perf/mfu/actor:np.float64(0.12214039553244277) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.25387191772461) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010957982391119003 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010957982391119003 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:760.18359375 - response_length/max:8192.0 - response_length/min:169.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:760.18359375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:309.8125 - prompt_length/max:597.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012904033064842224 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1227447520941496) - timing_s/agent_loop/generate_sequences/max:np.float64(60.78352793678641) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.487850829478703) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(60.78352793678641) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:597 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:67.3937315158546 - timing_s/reward:0.07918412238359451 - timing_s/old_log_prob:3.5191430747509003 - timing_s/adv:0.11993566527962685 - timing_s/update_actor:14.636007921770215 - timing_s/step:85.83626089058816 - timing_s/stop_profile:0.00012475252151489258 - timing_per_token_ms/adv:0.00043785084378822515 - timing_per_token_ms/gen:0.3463068210077469 - timing_per_token_ms/update_actor:0.0534318828623433 - perf/total_num_tokens:273919 - perf/time_per_step:85.83626089058816 - perf/throughput:398.8975596647216 (TaskRunner pid=2224264) Training Progress: 69%|██████▉ | 69/100 [1:06:02<28:43, 55.58s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a product: (TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a molecular structure with the following key features: (TaskRunner pid=2224264) - A nitrogen atom at the end of a chain (Nc1...n1) (TaskRunner pid=2224264) - A benzene ring with a substituent: `Cc2ccc(Cl)cc2C(F)(F)F` (i.e., a substituted benzene ring with a chloro group and a trifluoromethyl group) (TaskRunner pid=2224264) - The overall structure appears to be a **pyrrole-like ring** (Nc1ccn...n1) with a substituent group on the nitrogen. (TaskRunner pid=2224264) (TaskRunner pid=2224264) We are to identify which of the four options is the **only correct reactant** that would be used in the synthesis of this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Let’s analyze each option: (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - This contains the core structure: `c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` — which matches the product. (TaskRunner pid=2224264) - It also has a ketone group `O=C1` and another ketone group `C(=O)N1` — which may be a part of a synthesis pathway (e.g., a diketone or acylated structure). (TaskRunner pid=2224264) - This seems like a plausible reactant that could be used to form the product via a reaction like a **condensation** or **cyclization**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - This contains a `C(Cl)` group, not the `C(F)(F)F` group we see in the product. (TaskRunner pid=2224264) - The rest of the structure is not matching the product's ring system or substituents. (TaskRunner pid=2224264) - Therefore, this is **not a correct reactant**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a `C(F)(F)F` group (trifluoromethyl), which matches the product. (TaskRunner pid=2224264) - It also has a `Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1` — which includes a benzene ring with a chloro and trifluoromethyl group. (TaskRunner pid=2224264) - However, this structure is more of a **complex aromatic ring system**, not a direct match for the pyrrole-like structure in the product. (TaskRunner pid=2224264) - It may be a reactant in a more complex multi-step synthesis, but it is **not the only correct one**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains the `C(F)(F)F` group and a benzene ring. (TaskRunner pid=2224264) - However, it does **not** match the nitrogen-containing ring structure of the product. (TaskRunner pid=2224264) - The presence of a double bond in the side chain (CC=C) is not matching the pyrrole-like ring in the product. (TaskRunner pid=2224264) - So this is **not the correct reactant**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Conclusion: (TaskRunner pid=2224264) Only **Option A** contains the **exact ring structure** and **substituents** needed to form the product. It includes the pyrrole-like ring (`c1ccn(...n1`), the chloro-substituted benzene ring with a trifluoromethyl group, and has functional groups that are consistent with a synthetic pathway leading to the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2224264) step:70 - global_seqlen/min:22032 - global_seqlen/max:35660 - global_seqlen/minmax_diff:13628 - global_seqlen/balanced_min:30368 - global_seqlen/balanced_max:30409 - global_seqlen/mean:30397.25 - actor/entropy:0.8035950064659119 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22990918159484863 - training/rollout_probs_diff_mean:0.004655633121728897 - training/rollout_probs_diff_std:0.008662798441946507 - training/rollout_actor_probs_pearson_corr:0.9996584057807922 - rollout_corr/rollout_is_mean:1.0000817775726318 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.9939820857835e-06 - rollout_corr/rollout_is_max:1.8103430271148682 - rollout_corr/rollout_is_min:0.4562520384788513 - rollout_corr/rollout_is_std:0.035914406180381775 - rollout_corr/rollout_is_eff_sample_size:0.9987120548475574 - rollout_corr/rollout_is_seq_mean:1.0000861883163452 - rollout_corr/rollout_is_seq_std:0.0016128499992191792 - rollout_corr/rollout_is_seq_max:1.00638747215271 - rollout_corr/rollout_is_seq_min:0.9938380718231201 - rollout_corr/rollout_is_seq_max_deviation:0.006387472152709961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.279810177627951) - rollout_corr/training_log_ppl:np.float64(0.8013737860601395) - rollout_corr/kl:np.float64(0.0007208169924197705) - rollout_corr/k3_kl:np.float64(0.0007489672824476656) - rollout_corr/rollout_ppl:np.float64(2.278083552606404) - rollout_corr/rollout_log_ppl:np.float64(0.8006529699778184) - rollout_corr/log_ppl_diff:np.float64(0.0007208160823211074) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014672434190288186) - rollout_corr/log_ppl_diff_max:np.float64(0.008777737617492676) - rollout_corr/log_ppl_diff_min:np.float64(-0.005332350730895996) - rollout_corr/ppl_ratio:np.float64(1.000722888391465) - rollout_corr/chi2_token:np.float64(0.0015351453330367804) - rollout_corr/chi2_seq:np.float64(2.378258903743699) - actor/pg_loss:np.float64(-0.000118100899271667) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018792178138937743) - actor/ppo_kl:np.float64(0.00014457460659045296) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2343890555202961) - perf/mfu/actor:np.float64(0.11821289785724132) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.17000198364258) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6714285714285714) - val-aux/sciknoweval/reward/std@16:np.float64(0.2317466189895333) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7653619047619048) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1782119464541067) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5797619047619048) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2136060282647753) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6726619047619047) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.23202999913590713) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8335190476190476) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11938512568929552) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49611428571428573) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1773846797198801) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6981047619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.184153742497439) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8789999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0730800489542387) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.42429047619047616) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.13809200842124467) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7115333333333334) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13558200983257696) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9054095238095239) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03841748334380248) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3662142857142857) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.09495676645373348) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7192904761904763) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1010536286928432) - val-aux/sciknoweval/score/mean@16:np.float64(0.6714285714285714) - val-aux/sciknoweval/score/std@16:np.float64(0.2317466189895333) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7653619047619048) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1782119464541067) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5797619047619048) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2136060282647753) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6726619047619047) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.23202999913590713) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8335190476190476) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11938512568929552) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49611428571428573) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1773846797198801) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6981047619047619) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.184153742497439) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8789999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0730800489542387) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.42429047619047616) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.13809200842124467) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7115333333333334) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13558200983257696) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9054095238095239) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03841748334380248) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3662142857142857) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.09495676645373348) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7192904761904763) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1010536286928432) - val-core/sciknoweval/acc/mean@16:np.float64(0.6714285714285714) - val-aux/sciknoweval/acc/std@16:np.float64(0.2317466189895333) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7653619047619048) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1782119464541067) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5797619047619048) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2136060282647753) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6726619047619047) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.23202999913590713) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8335190476190476) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11938512568929552) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49611428571428573) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1773846797198801) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6981047619047619) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.184153742497439) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8789999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0730800489542387) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.42429047619047616) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.13809200842124467) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7115333333333334) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13558200983257696) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9054095238095239) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03841748334380248) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3662142857142857) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.09495676645373348) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7192904761904763) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1010536286928432) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9851190476190477) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.044118643209065715) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9977809523809523) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.015483180068832298) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9722142857142857) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.057096894053364385) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9851714285714287) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04372250124311388) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998476190476191) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.001196522485392713) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9513904761904762) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0690690442763583) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9942000000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.027176785822535302) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.920295238095238) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07418068438997008) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9985380952380953) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.012138625400942288) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8851476190476191) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06448749000951351) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998714285714286) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002292661410582752) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01194525696337223 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01194525696337223 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:651.6953125 - response_length/max:1845.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:651.6953125 - response_length_non_aborted/max:1845.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.21875 - prompt_length/max:701.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011080317199230194 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8444180563092232) - timing_s/agent_loop/generate_sequences/max:np.float64(14.495014188811183) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.144595855919761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.495014188811183) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:1676 - timing_s/gen:21.431634962558746 - timing_s/reward:0.07384154014289379 - timing_s/old_log_prob:3.262227326631546 - timing_s/adv:0.11903642117977142 - timing_s/update_actor:13.540440015494823 - timing_s/step:38.51284538023174 - timing_s/testing:189.72432128712535 - timing_s/save_checkpoint:13.012803422287107 - timing_s/stop_profile:0.00014807283878326416 - timing_per_token_ms/adv:0.0004895032493883963 - timing_per_token_ms/gen:0.12846083509691517 - timing_per_token_ms/update_actor:0.05568118832910388 - perf/total_num_tokens:243178 - perf/time_per_step:38.51284538023174 - perf/throughput:789.2756221954612 (TaskRunner pid=2224264) Training Progress: 70%|███████ | 70/100 [1:10:04<55:39, 111.31s/it] (TaskRunner pid=2224264) step:71 - global_seqlen/min:26528 - global_seqlen/max:38219 - global_seqlen/minmax_diff:11691 - global_seqlen/balanced_min:31790 - global_seqlen/balanced_max:32177 - global_seqlen/mean:31863.5 - actor/entropy:0.7618645429611206 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21302002668380737 - training/rollout_probs_diff_mean:0.004489706829190254 - training/rollout_probs_diff_std:0.008394083939492702 - training/rollout_actor_probs_pearson_corr:0.9996015429496765 - rollout_corr/rollout_is_mean:0.9997878670692444 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.690140925347805e-05 - rollout_corr/rollout_is_max:1.6874182224273682 - rollout_corr/rollout_is_min:0.4357602596282959 - rollout_corr/rollout_is_std:0.034910403192043304 - rollout_corr/rollout_is_eff_sample_size:0.9987822122017861 - rollout_corr/rollout_is_seq_mean:0.9997093677520752 - rollout_corr/rollout_is_seq_std:0.0016728744376450777 - rollout_corr/rollout_is_seq_max:1.005326747894287 - rollout_corr/rollout_is_seq_min:0.9935998916625977 - rollout_corr/rollout_is_seq_max_deviation:0.006400108337402344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.216639978811145) - rollout_corr/training_log_ppl:np.float64(0.7745629880810156) - rollout_corr/kl:np.float64(0.000979817594189747) - rollout_corr/k3_kl:np.float64(0.0007290477265087247) - rollout_corr/rollout_ppl:np.float64(2.2143593542277813) - rollout_corr/rollout_log_ppl:np.float64(0.7735831652535126) - rollout_corr/log_ppl_diff:np.float64(0.0009798228275030851) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014419290237128735) - rollout_corr/log_ppl_diff_max:np.float64(0.008339285850524902) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046994686126708984) - rollout_corr/ppl_ratio:np.float64(1.0009817637037486) - rollout_corr/chi2_token:np.float64(0.0009503676556050777) - rollout_corr/chi2_seq:np.float64(0.568886311724782) - actor/pg_loss:np.float64(5.502370186150074e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019256132691225503) - actor/ppo_kl:np.float64(4.591261260600277e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20790256932377815) - perf/mfu/actor:np.float64(0.12276829314252474) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(109.04192352294922) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034556337632238865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034556337632238865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:693.359375 - response_length/max:2588.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:693.359375 - response_length_non_aborted/max:2588.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.375 - prompt_length/max:666.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.34839928150177e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.169468689709902) - timing_s/agent_loop/generate_sequences/max:np.float64(19.67856295593083) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.767557479441166) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.67856295593083) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:2588 - timing_s/gen:26.15543474815786 - timing_s/reward:0.07458089478313923 - timing_s/old_log_prob:3.3945595882833004 - timing_s/adv:0.11478821560740471 - timing_s/update_actor:13.66449068300426 - timing_s/step:43.516275111585855 - timing_s/stop_profile:3.4091994166374207e-05 - timing_per_token_ms/adv:0.0004503123307522899 - timing_per_token_ms/gen:0.14735456196145272 - timing_per_token_ms/update_actor:0.05360557802424506 - perf/total_num_tokens:254908 - perf/time_per_step:43.516275111585855 - perf/throughput:732.220299607321 (TaskRunner pid=2224264) Training Progress: 71%|███████ | 71/100 [1:10:47<43:58, 90.98s/it] (TaskRunner pid=2224264) step:72 - global_seqlen/min:26255 - global_seqlen/max:35299 - global_seqlen/minmax_diff:9044 - global_seqlen/balanced_min:30687 - global_seqlen/balanced_max:30704 - global_seqlen/mean:30699.25 - actor/entropy:0.7660148739814758 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2309356927871704 - training/rollout_probs_diff_mean:0.0045791687443852425 - training/rollout_probs_diff_std:0.008554509840905666 - training/rollout_actor_probs_pearson_corr:0.999636709690094 - rollout_corr/rollout_is_mean:0.9999493360519409 - rollout_corr/rollout_is_ratio_fraction_high:5.8084829106519464e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0826847553253174 - rollout_corr/rollout_is_min:0.5040711760520935 - rollout_corr/rollout_is_std:0.03556250035762787 - rollout_corr/rollout_is_eff_sample_size:0.9987369060711254 - rollout_corr/rollout_is_seq_mean:0.9999403953552246 - rollout_corr/rollout_is_seq_std:0.0016658001113682985 - rollout_corr/rollout_is_seq_max:1.006662368774414 - rollout_corr/rollout_is_seq_min:0.99498051404953 - rollout_corr/rollout_is_seq_max_deviation:0.0066623687744140625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.172296098433435) - rollout_corr/training_log_ppl:np.float64(0.7561264133546501) - rollout_corr/kl:np.float64(0.0008874626383423845) - rollout_corr/k3_kl:np.float64(0.0007379499492117247) - rollout_corr/rollout_ppl:np.float64(2.170314473565668) - rollout_corr/rollout_log_ppl:np.float64(0.7552389547927305) - rollout_corr/log_ppl_diff:np.float64(0.0008874585619196296) - rollout_corr/log_ppl_abs_diff:np.float64(0.001500150072388351) - rollout_corr/log_ppl_diff_max:np.float64(0.00498625636100769) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046173930168151855) - rollout_corr/ppl_ratio:np.float64(1.0008892305195332) - rollout_corr/chi2_token:np.float64(0.0011937415692955256) - rollout_corr/chi2_seq:np.float64(0.66025594715029) - actor/pg_loss:np.float64(3.53293726220727e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002114741264449549) - actor/ppo_kl:np.float64(0.00018003375489444862) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2194691300392151) - perf/mfu/actor:np.float64(0.1192681330197729) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.72951126098633) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007243903819471598 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007243903819471598 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:672.5078125 - response_length/max:1734.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:672.5078125 - response_length_non_aborted/max:1734.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.84375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6632100343704224e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.31886431761086) - timing_s/agent_loop/generate_sequences/max:np.float64(14.253411900252104) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.534527560528659) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.253411900252104) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:244 - timing_s/agent_loop/slowest/response_length:1734 - timing_s/gen:21.547888031229377 - timing_s/reward:0.07392375729978085 - timing_s/old_log_prob:3.2580875027924776 - timing_s/adv:0.11826586164534092 - timing_s/update_actor:13.605389777570963 - timing_s/step:38.68964970111847 - timing_s/stop_profile:0.00017290934920310974 - timing_per_token_ms/adv:0.00048155028887245175 - timing_per_token_ms/gen:0.12516053502648306 - timing_per_token_ms/update_actor:0.055397891550978295 - perf/total_num_tokens:245594 - perf/time_per_step:38.68964970111847 - perf/throughput:793.4744883232304 (TaskRunner pid=2224264) Training Progress: 72%|███████▏ | 72/100 [1:11:26<35:08, 75.31s/it] (TaskRunner pid=2224264) step:73 - global_seqlen/min:32171 - global_seqlen/max:38705 - global_seqlen/minmax_diff:6534 - global_seqlen/balanced_min:33645 - global_seqlen/balanced_max:39741 - global_seqlen/mean:34425.125 - actor/entropy:0.763630211353302 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30194756388664246 - training/rollout_probs_diff_mean:0.004307533614337444 - training/rollout_probs_diff_std:0.008280006237328053 - training/rollout_actor_probs_pearson_corr:0.999650776386261 - rollout_corr/rollout_is_mean:0.9999016523361206 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.910024017590331e-06 - rollout_corr/rollout_is_max:1.6241782903671265 - rollout_corr/rollout_is_min:0.4921497702598572 - rollout_corr/rollout_is_std:0.034490425139665604 - rollout_corr/rollout_is_eff_sample_size:0.9988115860890442 - rollout_corr/rollout_is_seq_mean:0.9998833537101746 - rollout_corr/rollout_is_seq_std:0.0013762600719928741 - rollout_corr/rollout_is_seq_max:1.0045596361160278 - rollout_corr/rollout_is_seq_min:0.9957677125930786 - rollout_corr/rollout_is_seq_max_deviation:0.004559636116027832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2688772692345083) - rollout_corr/training_log_ppl:np.float64(0.8037596073554596) - rollout_corr/kl:np.float64(0.0007620832186034932) - rollout_corr/k3_kl:np.float64(0.0006744360291293106) - rollout_corr/rollout_ppl:np.float64(2.267069778870791) - rollout_corr/rollout_log_ppl:np.float64(0.8029975256649777) - rollout_corr/log_ppl_diff:np.float64(0.0007620816904818639) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012150325201218948) - rollout_corr/log_ppl_diff_max:np.float64(0.006102800369262695) - rollout_corr/log_ppl_diff_min:np.float64(-0.003519415855407715) - rollout_corr/ppl_ratio:np.float64(1.0007633492350578) - rollout_corr/chi2_token:np.float64(0.0011786054819822311) - rollout_corr/chi2_seq:np.float64(0.6752500396687537) - actor/pg_loss:np.float64(7.905624806880951e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014205676188794314) - actor/ppo_kl:np.float64(1.4047067626421494e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2183140367269516) - perf/mfu/actor:np.float64(0.12278116839994453) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.55410766601562) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005568580701947212 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005568580701947212 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:795.56640625 - response_length/max:8192.0 - response_length/min:236.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:795.56640625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:236.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:280.21875 - prompt_length/max:557.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014955177903175354 - timing_s/agent_loop/generate_sequences/min:np.float64(2.818276045843959) - timing_s/agent_loop/generate_sequences/max:np.float64(61.54518798738718) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.143103411246557) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(61.54518798738718) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:68.7704401165247 - timing_s/reward:0.0806842241436243 - timing_s/old_log_prob:3.6475528068840504 - timing_s/adv:0.12522803619503975 - timing_s/update_actor:14.82589746080339 - timing_s/step:87.5357283577323 - timing_s/stop_profile:0.00011074915528297424 - timing_per_token_ms/adv:0.0004547116248490011 - timing_per_token_ms/gen:0.3376644986449547 - timing_per_token_ms/update_actor:0.053833854854569846 - perf/total_num_tokens:275401 - perf/time_per_step:87.5357283577323 - perf/throughput:393.26941862315726 (TaskRunner pid=2224264) Training Progress: 73%|███████▎ | 73/100 [1:12:54<35:32, 78.99s/it] (TaskRunner pid=2224264) step:74 - global_seqlen/min:29986 - global_seqlen/max:43985 - global_seqlen/minmax_diff:13999 - global_seqlen/balanced_min:33665 - global_seqlen/balanced_max:33695 - global_seqlen/mean:33681.375 - actor/entropy:0.761839747428894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19873207807540894 - training/rollout_probs_diff_mean:0.004497237037867308 - training/rollout_probs_diff_std:0.008453729562461376 - training/rollout_actor_probs_pearson_corr:0.999616265296936 - rollout_corr/rollout_is_mean:0.9999936819076538 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6013860702514648 - rollout_corr/rollout_is_min:0.606749951839447 - rollout_corr/rollout_is_std:0.03470577299594879 - rollout_corr/rollout_is_eff_sample_size:0.9987970773134115 - rollout_corr/rollout_is_seq_mean:0.9999818801879883 - rollout_corr/rollout_is_seq_std:0.0015479137655347586 - rollout_corr/rollout_is_seq_max:1.0074827671051025 - rollout_corr/rollout_is_seq_min:0.9962047338485718 - rollout_corr/rollout_is_seq_max_deviation:0.007482767105102539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2138938223943114) - rollout_corr/training_log_ppl:np.float64(0.7749610797036439) - rollout_corr/kl:np.float64(0.0006632767860637601) - rollout_corr/k3_kl:np.float64(0.0006568010118712664) - rollout_corr/rollout_ppl:np.float64(2.212330491282046) - rollout_corr/rollout_log_ppl:np.float64(0.7742978031164967) - rollout_corr/log_ppl_diff:np.float64(0.0006632765871472657) - rollout_corr/log_ppl_abs_diff:np.float64(0.001370829122606665) - rollout_corr/log_ppl_diff_max:np.float64(0.004554629325866699) - rollout_corr/log_ppl_diff_min:np.float64(-0.006599485874176025) - rollout_corr/ppl_ratio:np.float64(1.0006648083217442) - rollout_corr/chi2_token:np.float64(0.0013132649473845959) - rollout_corr/chi2_seq:np.float64(1.9479922475293279) - actor/pg_loss:np.float64(-5.3111580200493336e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014415476721296727) - actor/ppo_kl:np.float64(1.6327075574196215e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20973877236247063) - perf/mfu/actor:np.float64(0.12778629328477945) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.42087173461914) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005659057758748531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005659057758748531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:779.13671875 - response_length/max:2249.0 - response_length/min:207.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:779.13671875 - response_length_non_aborted/max:2249.0 - response_length_non_aborted/min:207.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.40625 - prompt_length/max:665.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010413303971290588 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6374386623501778) - timing_s/agent_loop/generate_sequences/max:np.float64(18.268491968512535) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.814450513171323) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.268491968512535) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:2249 - timing_s/gen:25.160439206287265 - timing_s/reward:0.07928537577390671 - timing_s/old_log_prob:3.3276401832699776 - timing_s/adv:0.12078331597149372 - timing_s/update_actor:13.96530625037849 - timing_s/step:42.73991168104112 - timing_s/stop_profile:0.00011014938354492188 - timing_per_token_ms/adv:0.0004482570707531006 - timing_per_token_ms/gen:0.12614341396621492 - timing_per_token_ms/update_actor:0.05182874159078456 - perf/total_num_tokens:269451 - perf/time_per_step:42.73991168104112 - perf/throughput:788.0543893341883 (TaskRunner pid=2224264) Training Progress: 74%|███████▍ | 74/100 [1:13:36<29:31, 68.13s/it] (TaskRunner pid=2224264) step:75 - global_seqlen/min:24487 - global_seqlen/max:40274 - global_seqlen/minmax_diff:15787 - global_seqlen/balanced_min:34105 - global_seqlen/balanced_max:34135 - global_seqlen/mean:34124.625 - actor/entropy:0.8144033551216125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917940020561218 - training/rollout_probs_diff_mean:0.00457530515268445 - training/rollout_probs_diff_std:0.008413956500589848 - training/rollout_actor_probs_pearson_corr:0.9996695518493652 - rollout_corr/rollout_is_mean:1.0000370740890503 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.187879196455469e-06 - rollout_corr/rollout_is_max:1.7824959754943848 - rollout_corr/rollout_is_min:0.4731596112251282 - rollout_corr/rollout_is_std:0.035698000341653824 - rollout_corr/rollout_is_eff_sample_size:0.9987273934949555 - rollout_corr/rollout_is_seq_mean:1.0000509023666382 - rollout_corr/rollout_is_seq_std:0.0015275044133886695 - rollout_corr/rollout_is_seq_max:1.004511833190918 - rollout_corr/rollout_is_seq_min:0.9952264428138733 - rollout_corr/rollout_is_seq_max_deviation:0.004773557186126709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2711348473094404) - rollout_corr/training_log_ppl:np.float64(0.803941803984344) - rollout_corr/kl:np.float64(0.0005384247337580916) - rollout_corr/k3_kl:np.float64(0.000659077487966897) - rollout_corr/rollout_ppl:np.float64(2.2699187486432493) - rollout_corr/rollout_log_ppl:np.float64(0.803403376834467) - rollout_corr/log_ppl_diff:np.float64(0.0005384271498769522) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012698217760771513) - rollout_corr/log_ppl_diff_max:np.float64(0.0053713321685791016) - rollout_corr/log_ppl_diff_min:np.float64(-0.003410249948501587) - rollout_corr/ppl_ratio:np.float64(1.0005397242493927) - rollout_corr/chi2_token:np.float64(0.0015677101910114288) - rollout_corr/chi2_seq:np.float64(1.5271832724101841) - actor/pg_loss:np.float64(-4.3374369852244854e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.082515417598188e-05) - actor/ppo_kl:np.float64(-5.413583712154946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17228768952190876) - perf/mfu/actor:np.float64(0.13115778980210233) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.54808044433594) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018546667415648699 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018546667415648699 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:752.95703125 - response_length/max:1483.0 - response_length/min:199.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:752.95703125 - response_length_non_aborted/max:1483.0 - response_length_non_aborted/min:199.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.4375 - prompt_length/max:608.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011150166392326355 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5929522551596165) - timing_s/agent_loop/generate_sequences/max:np.float64(14.319374568760395) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.897164289628563) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.319374568760395) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:1483 - timing_s/gen:21.023408722132444 - timing_s/reward:0.08007885701954365 - timing_s/old_log_prob:3.324468068778515 - timing_s/adv:0.12420106492936611 - timing_s/update_actor:13.739917827770114 - timing_s/step:38.37722675316036 - timing_s/stop_profile:0.00010982714593410492 - timing_per_token_ms/adv:0.0004549539552792379 - timing_per_token_ms/gen:0.109066901446549 - timing_per_token_ms/update_actor:0.050329922408561684 - perf/total_num_tokens:272997 - perf/time_per_step:38.37722675316036 - perf/throughput:889.189446113113 (TaskRunner pid=2224264) Training Progress: 75%|███████▌ | 75/100 [1:14:15<24:40, 59.22s/it] (TaskRunner pid=2224264) step:76 - global_seqlen/min:31915 - global_seqlen/max:42520 - global_seqlen/minmax_diff:10605 - global_seqlen/balanced_min:37284 - global_seqlen/balanced_max:37334 - global_seqlen/mean:37307.375 - actor/entropy:0.746766984462738 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48574504256248474 - training/rollout_probs_diff_mean:0.004121627192944288 - training/rollout_probs_diff_std:0.008096432313323021 - training/rollout_actor_probs_pearson_corr:0.999716579914093 - rollout_corr/rollout_is_mean:1.0002018213272095 - rollout_corr/rollout_is_ratio_fraction_high:4.438073119672481e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.438073119672481e-06 - rollout_corr/rollout_is_max:2.379199504852295 - rollout_corr/rollout_is_min:0.4145638346672058 - rollout_corr/rollout_is_std:0.03402938321232796 - rollout_corr/rollout_is_eff_sample_size:0.9988438161069871 - rollout_corr/rollout_is_seq_mean:1.000188946723938 - rollout_corr/rollout_is_seq_std:0.0011295655276626348 - rollout_corr/rollout_is_seq_max:1.0031288862228394 - rollout_corr/rollout_is_seq_min:0.9967604875564575 - rollout_corr/rollout_is_seq_max_deviation:0.0032395124435424805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.173908702097833) - rollout_corr/training_log_ppl:np.float64(0.7561200592317618) - rollout_corr/kl:np.float64(0.0004855001487325261) - rollout_corr/k3_kl:np.float64(0.0005892427249136745) - rollout_corr/rollout_ppl:np.float64(2.1727346065454185) - rollout_corr/rollout_log_ppl:np.float64(0.755634555243887) - rollout_corr/log_ppl_diff:np.float64(0.00048550398787483573) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009925910853780806) - rollout_corr/log_ppl_diff_max:np.float64(0.0035272836685180664) - rollout_corr/log_ppl_diff_min:np.float64(-0.002400696277618408) - rollout_corr/ppl_ratio:np.float64(1.0004863012582064) - rollout_corr/chi2_token:np.float64(0.0013905672822147608) - rollout_corr/chi2_seq:np.float64(0.9606744900811464) - actor/pg_loss:np.float64(-2.3183762095868587e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.338781410406227e-05) - actor/ppo_kl:np.float64(8.314622859639798e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18520821258425713) - perf/mfu/actor:np.float64(0.13935883033564134) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.53511428833008) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00035671013756655157 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00035671013756655157 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:880.16796875 - response_length/max:2522.0 - response_length/min:204.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:880.16796875 - response_length_non_aborted/max:2522.0 - response_length_non_aborted/min:204.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:285.6875 - prompt_length/max:526.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001479722559452057 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7096176706254482) - timing_s/agent_loop/generate_sequences/max:np.float64(20.600152047351003) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.33266173733864) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.600152047351003) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:196 - timing_s/agent_loop/slowest/response_length:2522 - timing_s/gen:27.267525793984532 - timing_s/reward:0.08974926918745041 - timing_s/old_log_prob:3.3883168008178473 - timing_s/adv:0.11706031486392021 - timing_s/update_actor:13.990974662825465 - timing_s/step:44.949168203398585 - timing_s/stop_profile:0.00010977871716022491 - timing_per_token_ms/adv:0.0003922157310180635 - timing_per_token_ms/gen:0.12101527937221027 - timing_per_token_ms/update_actor:0.0468773756624041 - perf/total_num_tokens:298459 - perf/time_per_step:44.949168203398585 - perf/throughput:829.990331104263 (TaskRunner pid=2224264) Training Progress: 76%|███████▌ | 76/100 [1:15:00<21:59, 54.96s/it] (TaskRunner pid=2224264) step:77 - global_seqlen/min:27604 - global_seqlen/max:39041 - global_seqlen/minmax_diff:11437 - global_seqlen/balanced_min:34288 - global_seqlen/balanced_max:34332 - global_seqlen/mean:34317.375 - actor/entropy:0.8058213591575623 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28673720359802246 - training/rollout_probs_diff_mean:0.004518347326666117 - training/rollout_probs_diff_std:0.008365931920707226 - training/rollout_actor_probs_pearson_corr:0.999616265296936 - rollout_corr/rollout_is_mean:0.9999557733535767 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6204079389572144 - rollout_corr/rollout_is_min:0.6416282057762146 - rollout_corr/rollout_is_std:0.03523077070713043 - rollout_corr/rollout_is_eff_sample_size:0.9987602126485932 - rollout_corr/rollout_is_seq_mean:0.9999356269836426 - rollout_corr/rollout_is_seq_std:0.0013528199633583426 - rollout_corr/rollout_is_seq_max:1.0050373077392578 - rollout_corr/rollout_is_seq_min:0.9947447180747986 - rollout_corr/rollout_is_seq_max_deviation:0.005255281925201416 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.28120540920645) - rollout_corr/training_log_ppl:np.float64(0.8050968096358702) - rollout_corr/kl:np.float64(0.0006265352302827765) - rollout_corr/k3_kl:np.float64(0.0006796217078317568) - rollout_corr/rollout_ppl:np.float64(2.279738489538431) - rollout_corr/rollout_log_ppl:np.float64(0.8044702711631544) - rollout_corr/log_ppl_diff:np.float64(0.0006265384727157652) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010923692607320845) - rollout_corr/log_ppl_diff_max:np.float64(0.005904078483581543) - rollout_corr/log_ppl_diff_min:np.float64(-0.0028671622276306152) - rollout_corr/ppl_ratio:np.float64(1.0006276031490415) - rollout_corr/chi2_token:np.float64(0.0014646041672676802) - rollout_corr/chi2_seq:np.float64(0.7338430834934115) - actor/pg_loss:np.float64(0.00012833683285862207) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017851591564976843) - actor/ppo_kl:np.float64(-6.952557651018765e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2424900233745575) - perf/mfu/actor:np.float64(0.13009512944939702) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.56500244140625) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007992738857865334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007992738857865334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:755.32421875 - response_length/max:2071.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:755.32421875 - response_length_non_aborted/max:2071.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.09375 - prompt_length/max:716.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013962015509605408 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1678621973842382) - timing_s/agent_loop/generate_sequences/max:np.float64(17.2276831548661) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.9278243403023225) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.2276831548661) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:2071 - timing_s/gen:24.56353418342769 - timing_s/reward:0.08003837987780571 - timing_s/old_log_prob:3.337373360991478 - timing_s/adv:0.12115020304918289 - timing_s/update_actor:13.919541602954268 - timing_s/step:42.11578721739352 - timing_s/stop_profile:0.000111408531665802 - timing_per_token_ms/adv:0.00044128594862363047 - timing_per_token_ms/gen:0.12703326998147366 - timing_per_token_ms/update_actor:0.05070150908597419 - perf/total_num_tokens:274539 - perf/time_per_step:42.11578721739352 - perf/throughput:814.8339914165767 (TaskRunner pid=2224264) Training Progress: 77%|███████▋ | 77/100 [1:15:42<19:35, 51.12s/it] (TaskRunner pid=2224264) step:78 - global_seqlen/min:28938 - global_seqlen/max:37974 - global_seqlen/minmax_diff:9036 - global_seqlen/balanced_min:34229 - global_seqlen/balanced_max:34251 - global_seqlen/mean:34245.0 - actor/entropy:0.7469825148582458 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48943376541137695 - training/rollout_probs_diff_mean:0.004324706271290779 - training/rollout_probs_diff_std:0.008407223038375378 - training/rollout_actor_probs_pearson_corr:0.9996141195297241 - rollout_corr/rollout_is_mean:1.0000590085983276 - rollout_corr/rollout_is_ratio_fraction_high:4.9409068196837325e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.881813639367465e-06 - rollout_corr/rollout_is_max:4.681254863739014 - rollout_corr/rollout_is_min:0.4222910404205322 - rollout_corr/rollout_is_std:0.03437097743153572 - rollout_corr/rollout_is_eff_sample_size:0.9988201488429446 - rollout_corr/rollout_is_seq_mean:1.00004243850708 - rollout_corr/rollout_is_seq_std:0.001362150302156806 - rollout_corr/rollout_is_seq_max:1.0039869546890259 - rollout_corr/rollout_is_seq_min:0.9943282008171082 - rollout_corr/rollout_is_seq_max_deviation:0.005671799182891846 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1253380407579243) - rollout_corr/training_log_ppl:np.float64(0.7371736962813884) - rollout_corr/kl:np.float64(0.0005827662733111083) - rollout_corr/k3_kl:np.float64(0.0006460418212554941) - rollout_corr/rollout_ppl:np.float64(2.1241166265681386) - rollout_corr/rollout_log_ppl:np.float64(0.7365909329382703) - rollout_corr/log_ppl_diff:np.float64(0.0005827633431181312) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012038786662742496) - rollout_corr/log_ppl_diff_max:np.float64(0.006325364112854004) - rollout_corr/log_ppl_diff_min:np.float64(-0.003412604331970215) - rollout_corr/ppl_ratio:np.float64(1.0005839837249368) - rollout_corr/chi2_token:np.float64(0.0014792552683502436) - rollout_corr/chi2_seq:np.float64(2.1139756981283426) - actor/pg_loss:np.float64(0.0001295657129958272) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001085545256955811) - actor/ppo_kl:np.float64(1.3940017076219391e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20791660994291306) - perf/mfu/actor:np.float64(0.1296498704894837) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.49341583251953) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018497519195079803 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0018497519195079803 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:790.59375 - response_length/max:1840.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:790.59375 - response_length_non_aborted/max:1840.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5625 - prompt_length/max:449.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010417215526103973 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2816403061151505) - timing_s/agent_loop/generate_sequences/max:np.float64(16.65826540067792) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.189224266308884) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.65826540067792) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:1840 - timing_s/gen:23.61698125116527 - timing_s/reward:0.08289889059960842 - timing_s/old_log_prob:3.274001955986023 - timing_s/adv:0.12328475899994373 - timing_s/update_actor:13.82346528582275 - timing_s/step:41.00753286853433 - timing_s/stop_profile:0.00011822767555713654 - timing_per_token_ms/adv:0.000450010070813052 - timing_per_token_ms/gen:0.11668930220149645 - timing_per_token_ms/update_actor:0.050457969359843584 - perf/total_num_tokens:273960 - perf/time_per_step:41.00753286853433 - perf/throughput:835.0904725184452 (TaskRunner pid=2224264) Training Progress: 78%|███████▊ | 78/100 [1:16:23<17:38, 48.10s/it] (TaskRunner pid=2224264) step:79 - global_seqlen/min:31052 - global_seqlen/max:40489 - global_seqlen/minmax_diff:9437 - global_seqlen/balanced_min:35267 - global_seqlen/balanced_max:35291 - global_seqlen/mean:35282.375 - actor/entropy:0.7419886589050293 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3267946243286133 - training/rollout_probs_diff_mean:0.004350726492702961 - training/rollout_probs_diff_std:0.008377747610211372 - training/rollout_actor_probs_pearson_corr:0.9996976256370544 - rollout_corr/rollout_is_mean:1.0000663995742798 - rollout_corr/rollout_is_ratio_fraction_high:4.6575100896006916e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.315020179201383e-06 - rollout_corr/rollout_is_max:2.486323833465576 - rollout_corr/rollout_is_min:0.3386184871196747 - rollout_corr/rollout_is_std:0.034230221062898636 - rollout_corr/rollout_is_eff_sample_size:0.9988299010470403 - rollout_corr/rollout_is_seq_mean:1.0000535249710083 - rollout_corr/rollout_is_seq_std:0.001242220401763916 - rollout_corr/rollout_is_seq_max:1.0033190250396729 - rollout_corr/rollout_is_seq_min:0.9950276017189026 - rollout_corr/rollout_is_seq_max_deviation:0.004972398281097412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1713191107846797) - rollout_corr/training_log_ppl:np.float64(0.7553618648089468) - rollout_corr/kl:np.float64(0.0005726922157927028) - rollout_corr/k3_kl:np.float64(0.0006373734220233018) - rollout_corr/rollout_ppl:np.float64(2.170036384370178) - rollout_corr/rollout_log_ppl:np.float64(0.754789168946445) - rollout_corr/log_ppl_diff:np.float64(0.0005726958625018597) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010784545447677374) - rollout_corr/log_ppl_diff_max:np.float64(0.004006564617156982) - rollout_corr/log_ppl_diff_min:np.float64(-0.00302046537399292) - rollout_corr/ppl_ratio:np.float64(1.0005736567545682) - rollout_corr/chi2_token:np.float64(0.001405506394803524) - rollout_corr/chi2_seq:np.float64(1.6988238336052746) - actor/pg_loss:np.float64(6.845139432698488e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017011942372846534) - actor/ppo_kl:np.float64(1.6240256268673647e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24053851515054703) - perf/mfu/actor:np.float64(0.13349716649984694) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.54805374145508) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011310762725770473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011310762725770473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:838.69921875 - response_length/max:2007.0 - response_length/min:283.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:838.69921875 - response_length_non_aborted/max:2007.0 - response_length_non_aborted/min:283.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:766.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010603666305541992 - timing_s/agent_loop/generate_sequences/min:np.float64(3.655642168596387) - timing_s/agent_loop/generate_sequences/max:np.float64(17.38879804685712) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.424919297925953) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.38879804685712) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:295 - timing_s/agent_loop/slowest/response_length:2007 - timing_s/gen:24.167077971622348 - timing_s/reward:0.08212696015834808 - timing_s/old_log_prob:3.3000302650034428 - timing_s/adv:0.11685498803853989 - timing_s/update_actor:13.943561296910048 - timing_s/step:41.69688728451729 - timing_s/stop_profile:0.00011464767158031464 - timing_per_token_ms/adv:0.00041399915693933546 - timing_per_token_ms/gen:0.11255840737201092 - timing_per_token_ms/update_actor:0.04939988201230093 - perf/total_num_tokens:282259 - perf/time_per_step:41.69688728451729 - perf/throughput:846.1632821474639 (TaskRunner pid=2224264) Training Progress: 79%|███████▉ | 79/100 [1:17:05<16:10, 46.19s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a target product: (TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) This is a nitrogen-containing compound with a ring structure. Let's analyze it: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - The formula appears to represent a **pyrrole ring** (a 5-membered ring with one nitrogen atom). (TaskRunner pid=2224264) - The nitrogen is at the first position (Nc1...), and there's a substituent at the third carbon: **Cc2ccc(Cl)cc2C(F)(F)F**, which implies a **benzene ring** with a **chlorine atom** and a **trifluoromethyl group (C(F)(F)F)** on the same carbon. (TaskRunner pid=2224264) - So the full structure is: a **pyrrole ring** with a **benzene ring** substituted with **Cl** and **CF₃** at the same position, attached to the third carbon of the pyrrole. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Now we examine the options to find which one is the **correct reactant** that would lead to this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option A: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) - This has a **pyrrole ring** (from the N1 and the 5-membered ring). (TaskRunner pid=2224264) - The ring is substituted with a **benzene ring** (c2ccccc2) with a **Cl** and **CF₃** at the same position. (TaskRunner pid=2224264) - The structure matches the **target product** exactly. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option B: (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) - This contains a **Cl** on the ring, but the ring structure is different (not a pyrrole, and the Cl is not in the correct position). (TaskRunner pid=2224264) - The structure is not matching the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option C: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) - Contains a **benzene ring** with Cl and CF₃, but it's attached to an **oxygen** (Oc2...), not directly to the pyrrole. (TaskRunner pid=2224264) - The structure is different from the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option D: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) - This has a **trifluoromethyl group** (C(F)(F)F), but it is attached to a **methylene group** (CC=C) and not directly to the ring. (TaskRunner pid=2224264) - The structure is not matching the target. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Conclusion: (TaskRunner pid=2224264) Only **Option A** has the correct **pyrrole ring** with the correct **benzene substituent** (Cl and CF₃) in the correct position, matching the structure of the given product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2224264) step:80 - global_seqlen/min:28862 - global_seqlen/max:47477 - global_seqlen/minmax_diff:18615 - global_seqlen/balanced_min:36267 - global_seqlen/balanced_max:36835 - global_seqlen/mean:36375.375 - actor/entropy:0.7752931714057922 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967353940010071 - training/rollout_probs_diff_mean:0.004443757236003876 - training/rollout_probs_diff_std:0.008464205078780651 - training/rollout_actor_probs_pearson_corr:0.9995999336242676 - rollout_corr/rollout_is_mean:1.0000269412994385 - rollout_corr/rollout_is_ratio_fraction_high:9.492214303463697e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.492214303463697e-06 - rollout_corr/rollout_is_max:2.5489871501922607 - rollout_corr/rollout_is_min:0.2599738538265228 - rollout_corr/rollout_is_std:0.03524937480688095 - rollout_corr/rollout_is_eff_sample_size:0.99875914242478 - rollout_corr/rollout_is_seq_mean:0.9999639391899109 - rollout_corr/rollout_is_seq_std:0.0014692033873870969 - rollout_corr/rollout_is_seq_max:1.0044381618499756 - rollout_corr/rollout_is_seq_min:0.9949309825897217 - rollout_corr/rollout_is_seq_max_deviation:0.00506901741027832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.2062102765776217) - rollout_corr/training_log_ppl:np.float64(0.7724762546131387) - rollout_corr/kl:np.float64(0.0008447698922706692) - rollout_corr/k3_kl:np.float64(0.0006712018573580281) - rollout_corr/rollout_ppl:np.float64(2.204339082352817) - rollout_corr/rollout_log_ppl:np.float64(0.7716314849676564) - rollout_corr/log_ppl_diff:np.float64(0.0008447696454823017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013305179309099913) - rollout_corr/log_ppl_diff_max:np.float64(0.005142807960510254) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035692453384399414) - rollout_corr/ppl_ratio:np.float64(1.0008462956175208) - rollout_corr/chi2_token:np.float64(0.0010051389690488577) - rollout_corr/chi2_seq:np.float64(3.9995412554126233) - actor/pg_loss:np.float64(6.170920096337795e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.559897284816543e-05) - actor/ppo_kl:np.float64(0.00016689712089679176) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20688721537590027) - perf/mfu/actor:np.float64(0.1360888320089338) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(108.50061416625977) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6782738095238096) - val-aux/sciknoweval/reward/std@16:np.float64(0.2194428960706272) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7662047619047618) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1707809803907824) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5911047619047619) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.19703442080325248) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6786285714285715) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2187494224175815) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8316666666666666) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1211692975832172) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.513647619047619) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16233574997526817) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7021999999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17154812598360286) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8789666666666667) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08054198286562807) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.44790952380952387) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12433382532071413) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7142047619047619) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12568969808131827) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9095333333333334) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0468538841472672) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3950142857142857) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.08487101067852891) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7184190476190476) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09186432979311497) - val-aux/sciknoweval/score/mean@16:np.float64(0.6782738095238096) - val-aux/sciknoweval/score/std@16:np.float64(0.2194428960706272) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7662047619047618) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1707809803907824) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5911047619047619) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.19703442080325248) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6786285714285715) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2187494224175815) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8316666666666666) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1211692975832172) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.513647619047619) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.16233574997526817) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7021999999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17154812598360286) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8789666666666667) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08054198286562807) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.44790952380952387) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.12433382532071413) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7142047619047619) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12568969808131827) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9095333333333334) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0468538841472672) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3950142857142857) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.08487101067852891) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7184190476190476) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09186432979311497) - val-core/sciknoweval/acc/mean@16:np.float64(0.6782738095238096) - val-aux/sciknoweval/acc/std@16:np.float64(0.2194428960706272) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7662047619047618) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1707809803907824) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5911047619047619) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.19703442080325248) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6786285714285715) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2187494224175815) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8316666666666666) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1211692975832172) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.513647619047619) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.16233574997526817) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7021999999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17154812598360286) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8789666666666667) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08054198286562807) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.44790952380952387) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.12433382532071413) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7142047619047619) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12568969808131827) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9095333333333334) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0468538841472672) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3950142857142857) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.08487101067852891) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7184190476190476) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09186432979311497) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9797619047619047) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0529405396018874) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9968523809523809) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.02007926101605294) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9632904761904761) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06706255313069127) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9803428571428572) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.052420251171557214) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998619047619047) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.002377576095584018) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9359809523809526) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07827091020757276) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9914190476190478) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.034868422157064094) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8997619047619049) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07810157852659988) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9974428571428571) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.017502498474107602) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.863804761904762) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06109163301171088) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9996809523809523) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004826434845870424) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007080005016177893 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007080005016177893 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:823.04296875 - response_length/max:3037.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:823.04296875 - response_length_non_aborted/max:3037.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.6875 - prompt_length/max:950.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014653801918029785 - timing_s/agent_loop/generate_sequences/min:np.float64(2.194973863661289) - timing_s/agent_loop/generate_sequences/max:np.float64(22.72264883853495) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.372557892595069) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.72264883853495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:3037 - timing_s/gen:30.101325387135148 - timing_s/reward:0.08319770731031895 - timing_s/old_log_prob:3.4729851465672255 - timing_s/adv:0.14867497608065605 - timing_s/update_actor:14.160021048039198 - timing_s/step:48.05162427574396 - timing_s/testing:207.67929325439036 - timing_s/save_checkpoint:12.710731318220496 - timing_s/stop_profile:0.00015132129192352295 - timing_per_token_ms/adv:0.0005109053036589178 - timing_per_token_ms/gen:0.14286411130159682 - timing_per_token_ms/update_actor:0.04865936450153159 - perf/total_num_tokens:291003 - perf/time_per_step:48.05162427574396 - perf/throughput:757.0061480390366 (TaskRunner pid=2224264) Training Progress: 80%|████████ | 80/100 [1:21:33<37:37, 112.89s/it] (TaskRunner pid=2224264) step:81 - global_seqlen/min:23435 - global_seqlen/max:40414 - global_seqlen/minmax_diff:16979 - global_seqlen/balanced_min:31718 - global_seqlen/balanced_max:31758 - global_seqlen/mean:31742.875 - actor/entropy:0.728011965751648 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25490427017211914 - training/rollout_probs_diff_mean:0.00432547414675355 - training/rollout_probs_diff_std:0.00843879021704197 - training/rollout_actor_probs_pearson_corr:0.9996644258499146 - rollout_corr/rollout_is_mean:0.9999246001243591 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.7127294995589182e-05 - rollout_corr/rollout_is_max:1.6446915864944458 - rollout_corr/rollout_is_min:0.3955734968185425 - rollout_corr/rollout_is_std:0.03461290895938873 - rollout_corr/rollout_is_eff_sample_size:0.9988031424060505 - rollout_corr/rollout_is_seq_mean:0.9999666810035706 - rollout_corr/rollout_is_seq_std:0.001528046908788383 - rollout_corr/rollout_is_seq_max:1.0054874420166016 - rollout_corr/rollout_is_seq_min:0.9947968125343323 - rollout_corr/rollout_is_seq_max_deviation:0.0054874420166015625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.084522245451808) - rollout_corr/training_log_ppl:np.float64(0.7124685312155634) - rollout_corr/kl:np.float64(0.0007158686560133631) - rollout_corr/k3_kl:np.float64(0.000656772480510881) - rollout_corr/rollout_ppl:np.float64(2.083047821652144) - rollout_corr/rollout_log_ppl:np.float64(0.7117526578949764) - rollout_corr/log_ppl_diff:np.float64(0.0007158733205869794) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012598995817825198) - rollout_corr/log_ppl_diff_max:np.float64(0.005965232849121094) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034725069999694824) - rollout_corr/ppl_ratio:np.float64(1.000717243179679) - rollout_corr/chi2_token:np.float64(0.0012002624571323395) - rollout_corr/chi2_seq:np.float64(0.9090896914713085) - actor/pg_loss:np.float64(-3.0948780477046967e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011802908329627826) - actor/ppo_kl:np.float64(6.148918862525221e-05) - actor/pg_clipfrac_lower:np.float64(2.239822151750559e-06) - actor/grad_norm:np.float64(0.2090231031179428) - perf/mfu/actor:np.float64(0.12132654821351155) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(109.81432342529297) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020438572391867638 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020438572391867638 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:684.21484375 - response_length/max:1807.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:684.21484375 - response_length_non_aborted/max:1807.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.75 - prompt_length/max:735.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001064930111169815 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4930436480790377) - timing_s/agent_loop/generate_sequences/max:np.float64(14.782667718827724) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.092499443388078) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.782667718827724) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:1744 - timing_s/gen:21.074421510100365 - timing_s/reward:0.071615781635046 - timing_s/old_log_prob:3.5168352890759706 - timing_s/adv:0.12568366713821888 - timing_s/update_actor:13.775931548327208 - timing_s/step:38.66653628088534 - timing_s/stop_profile:0.00012343190610408783 - timing_per_token_ms/adv:0.0004949286538247516 - timing_per_token_ms/gen:0.12031595013730591 - timing_per_token_ms/update_actor:0.05424812476944514 - perf/total_num_tokens:253943 - perf/time_per_step:38.66653628088534 - perf/throughput:820.9391906585637 (TaskRunner pid=2224264) Training Progress: 81%|████████ | 81/100 [1:22:12<28:42, 90.64s/it] (TaskRunner pid=2224264) step:82 - global_seqlen/min:30553 - global_seqlen/max:40093 - global_seqlen/minmax_diff:9540 - global_seqlen/balanced_min:34588 - global_seqlen/balanced_max:34791 - global_seqlen/mean:34631.0 - actor/entropy:0.7073292136192322 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48784658312797546 - training/rollout_probs_diff_mean:0.004207909107208252 - training/rollout_probs_diff_std:0.008302947506308556 - training/rollout_actor_probs_pearson_corr:0.9996032118797302 - rollout_corr/rollout_is_mean:0.9999532103538513 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4358463886310346e-05 - rollout_corr/rollout_is_max:1.861779808998108 - rollout_corr/rollout_is_min:0.3777696490287781 - rollout_corr/rollout_is_std:0.033776216208934784 - rollout_corr/rollout_is_eff_sample_size:0.998860348177861 - rollout_corr/rollout_is_seq_mean:0.9999899864196777 - rollout_corr/rollout_is_seq_std:0.0011922791600227356 - rollout_corr/rollout_is_seq_max:1.004441738128662 - rollout_corr/rollout_is_seq_min:0.9973618984222412 - rollout_corr/rollout_is_seq_max_deviation:0.004441738128662109 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0682692220434546) - rollout_corr/training_log_ppl:np.float64(0.7130721984431148) - rollout_corr/kl:np.float64(0.0005954480367265091) - rollout_corr/k3_kl:np.float64(0.0006088201273541927) - rollout_corr/rollout_ppl:np.float64(2.067028284072876) - rollout_corr/rollout_log_ppl:np.float64(0.712476747808978) - rollout_corr/log_ppl_diff:np.float64(0.000595450634136796) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011131800711154938) - rollout_corr/log_ppl_diff_max:np.float64(0.0041391849517822266) - rollout_corr/log_ppl_diff_min:np.float64(-0.00474703311920166) - rollout_corr/ppl_ratio:np.float64(1.0005964431911707) - rollout_corr/chi2_token:np.float64(0.0012475282419472933) - rollout_corr/chi2_seq:np.float64(2.9451562664471567) - actor/pg_loss:np.float64(0.00018601585179567337) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019039797803088732) - actor/ppo_kl:np.float64(-5.924717907390686e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25703465938568115) - perf/mfu/actor:np.float64(0.1307738033068337) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(109.3072280883789) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006862149457447231 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006862149457447231 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:816.15625 - response_length/max:2571.0 - response_length/min:362.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:816.15625 - response_length_non_aborted/max:2571.0 - response_length_non_aborted/min:362.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.0625 - prompt_length/max:408.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.565187454223633e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(4.610755139961839) - timing_s/agent_loop/generate_sequences/max:np.float64(20.74920883215964) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.369914709837758) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.74920883215964) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:2571 - timing_s/gen:27.471973603591323 - timing_s/reward:0.07828585989773273 - timing_s/old_log_prob:3.2765041179955006 - timing_s/adv:0.11844107136130333 - timing_s/update_actor:13.957106951624155 - timing_s/step:44.991200095042586 - timing_s/stop_profile:0.00011249631643295288 - timing_per_token_ms/adv:0.0004275110138362426 - timing_per_token_ms/gen:0.13148511316188366 - timing_per_token_ms/update_actor:0.05037793794441452 - perf/total_num_tokens:277048 - perf/time_per_step:44.991200095042586 - perf/throughput:769.7283007975567 (TaskRunner pid=2224264) Training Progress: 82%|████████▏ | 82/100 [1:22:57<23:05, 76.96s/it] (TaskRunner pid=2224264) step:83 - global_seqlen/min:24830 - global_seqlen/max:36777 - global_seqlen/minmax_diff:11947 - global_seqlen/balanced_min:31744 - global_seqlen/balanced_max:31974 - global_seqlen/mean:31783.0 - actor/entropy:0.7531368136405945 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977966308594 - training/rollout_probs_diff_mean:0.0044290670193731785 - training/rollout_probs_diff_std:0.008538827300071716 - training/rollout_actor_probs_pearson_corr:0.999602735042572 - rollout_corr/rollout_is_mean:1.0000147819519043 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6260687415488064e-06 - rollout_corr/rollout_is_max:1.5903000831604004 - rollout_corr/rollout_is_min:0.4172116219997406 - rollout_corr/rollout_is_std:0.03477611765265465 - rollout_corr/rollout_is_eff_sample_size:0.9987920825865458 - rollout_corr/rollout_is_seq_mean:0.9999656677246094 - rollout_corr/rollout_is_seq_std:0.0014977967366576195 - rollout_corr/rollout_is_seq_max:1.0056312084197998 - rollout_corr/rollout_is_seq_min:0.9933651685714722 - rollout_corr/rollout_is_seq_max_deviation:0.006634831428527832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.162785124965012) - rollout_corr/training_log_ppl:np.float64(0.7481051302747801) - rollout_corr/kl:np.float64(0.0006437221610795518) - rollout_corr/k3_kl:np.float64(0.0006854262796878174) - rollout_corr/rollout_ppl:np.float64(2.161293275654316) - rollout_corr/rollout_log_ppl:np.float64(0.7474614095408469) - rollout_corr/log_ppl_diff:np.float64(0.0006437207339331508) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012794457143172622) - rollout_corr/log_ppl_diff_max:np.float64(0.0074920654296875) - rollout_corr/log_ppl_diff_min:np.float64(-0.004778653383255005) - rollout_corr/ppl_ratio:np.float64(1.000645110849291) - rollout_corr/chi2_token:np.float64(0.0014334383886307478) - rollout_corr/chi2_seq:np.float64(0.4927413531113416) - actor/pg_loss:np.float64(-0.00020378001499921083) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024108612069539959) - actor/ppo_kl:np.float64(-1.7973163554674265e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23665034770965576) - perf/mfu/actor:np.float64(0.12241472211925314) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.91914367675781) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008029807358980179 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008029807358980179 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:694.3125 - response_length/max:2176.0 - response_length/min:179.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:694.3125 - response_length_non_aborted/max:2176.0 - response_length_non_aborted/min:179.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.90625 - prompt_length/max:563.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010167807340621948 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2234518099576235) - timing_s/agent_loop/generate_sequences/max:np.float64(17.420780858024955) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.779617345870065) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.420780858024955) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:2176 - timing_s/gen:23.951165637001395 - timing_s/reward:0.07695263624191284 - timing_s/old_log_prob:3.1909585148096085 - timing_s/adv:0.11885049380362034 - timing_s/update_actor:13.669648755341768 - timing_s/step:41.09482858143747 - timing_s/stop_profile:0.00010753236711025238 - timing_per_token_ms/adv:0.00046742949770168146 - timing_per_token_ms/gen:0.13475090938091522 - timing_per_token_ms/update_actor:0.05376163654839761 - perf/total_num_tokens:254264 - perf/time_per_step:41.09482858143747 - perf/throughput:773.4063164910336 (TaskRunner pid=2224264) Training Progress: 83%|████████▎ | 83/100 [1:23:38<18:45, 66.22s/it] (TaskRunner pid=2224264) step:84 - global_seqlen/min:20654 - global_seqlen/max:34303 - global_seqlen/minmax_diff:13649 - global_seqlen/balanced_min:29574 - global_seqlen/balanced_max:29701 - global_seqlen/mean:29614.0 - actor/entropy:0.6809946894645691 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43606680631637573 - training/rollout_probs_diff_mean:0.004484572447836399 - training/rollout_probs_diff_std:0.008784463629126549 - training/rollout_actor_probs_pearson_corr:0.999607264995575 - rollout_corr/rollout_is_mean:0.9998617172241211 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1636025192274246e-05 - rollout_corr/rollout_is_max:1.5042952299118042 - rollout_corr/rollout_is_min:0.4855743646621704 - rollout_corr/rollout_is_std:0.0344783253967762 - rollout_corr/rollout_is_eff_sample_size:0.9988122996462617 - rollout_corr/rollout_is_seq_mean:0.9998700618743896 - rollout_corr/rollout_is_seq_std:0.0014390589203685522 - rollout_corr/rollout_is_seq_max:1.0044506788253784 - rollout_corr/rollout_is_seq_min:0.9937525391578674 - rollout_corr/rollout_is_seq_max_deviation:0.006247460842132568 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9907695450820029) - rollout_corr/training_log_ppl:np.float64(0.670324468635954) - rollout_corr/kl:np.float64(0.0006556954003826831) - rollout_corr/k3_kl:np.float64(0.000665820000790518) - rollout_corr/rollout_ppl:np.float64(1.9894441780634224) - rollout_corr/rollout_log_ppl:np.float64(0.6696687689982355) - rollout_corr/log_ppl_diff:np.float64(0.0006556996377184987) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012462410377338529) - rollout_corr/log_ppl_diff_max:np.float64(0.007571697235107422) - rollout_corr/log_ppl_diff_min:np.float64(-0.004120081663131714) - rollout_corr/ppl_ratio:np.float64(1.000657090684399) - rollout_corr/chi2_token:np.float64(0.0013479453045874834) - rollout_corr/chi2_seq:np.float64(0.807735110167414) - actor/pg_loss:np.float64(-0.00014557677786797285) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001528950670035556) - actor/ppo_kl:np.float64(-9.21530613155197e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18504177778959274) - perf/mfu/actor:np.float64(0.1154202224364208) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.7387924194336) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01133712474256754 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01133712474256754 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:671.40625 - response_length/max:2226.0 - response_length/min:177.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:671.40625 - response_length_non_aborted/max:2226.0 - response_length_non_aborted/min:177.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.03125 - prompt_length/max:466.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017830543220043182 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6220184583216906) - timing_s/agent_loop/generate_sequences/max:np.float64(16.999597314745188) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.399542242870666) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.999597314745188) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:177 - timing_s/agent_loop/slowest/response_length:2226 - timing_s/gen:24.433550149202347 - timing_s/reward:0.07261811941862106 - timing_s/old_log_prob:3.231289431452751 - timing_s/adv:0.12676293589174747 - timing_s/update_actor:13.54585151001811 - timing_s/step:41.49533977732062 - timing_s/stop_profile:0.00012038461863994598 - timing_per_token_ms/adv:0.0005350633817271707 - timing_per_token_ms/gen:0.14215470182221518 - timing_per_token_ms/update_actor:0.05717672177862713 - perf/total_num_tokens:236912 - perf/time_per_step:41.49533977732062 - perf/throughput:713.6705027340348 (TaskRunner pid=2224264) Training Progress: 84%|████████▍ | 84/100 [1:24:20<15:41, 58.82s/it] (TaskRunner pid=2224264) step:85 - global_seqlen/min:27367 - global_seqlen/max:37161 - global_seqlen/minmax_diff:9794 - global_seqlen/balanced_min:32947 - global_seqlen/balanced_max:32956 - global_seqlen/mean:32953.5 - actor/entropy:0.735228419303894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948740243911743 - training/rollout_probs_diff_mean:0.004468359053134918 - training/rollout_probs_diff_std:0.008735636249184608 - training/rollout_actor_probs_pearson_corr:0.9996408224105835 - rollout_corr/rollout_is_mean:0.9999686479568481 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0702284271246754e-05 - rollout_corr/rollout_is_max:1.7922857999801636 - rollout_corr/rollout_is_min:0.23469160497188568 - rollout_corr/rollout_is_std:0.03531694784760475 - rollout_corr/rollout_is_eff_sample_size:0.9987541480773114 - rollout_corr/rollout_is_seq_mean:1.0000033378601074 - rollout_corr/rollout_is_seq_std:0.0015298437792807817 - rollout_corr/rollout_is_seq_max:1.0047695636749268 - rollout_corr/rollout_is_seq_min:0.9963509440422058 - rollout_corr/rollout_is_seq_max_deviation:0.004769563674926758 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1323386910371482) - rollout_corr/training_log_ppl:np.float64(0.7435957858106121) - rollout_corr/kl:np.float64(0.0005658202565279602) - rollout_corr/k3_kl:np.float64(0.0006766391076098444) - rollout_corr/rollout_ppl:np.float64(2.1311398232355714) - rollout_corr/rollout_log_ppl:np.float64(0.7430299640400335) - rollout_corr/log_ppl_diff:np.float64(0.0005658217705786228) - rollout_corr/log_ppl_abs_diff:np.float64(0.00123508065007627) - rollout_corr/log_ppl_diff_max:np.float64(0.004446566104888916) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032745003700256348) - rollout_corr/ppl_ratio:np.float64(1.000567066250369) - rollout_corr/chi2_token:np.float64(0.0015738557558506727) - rollout_corr/chi2_seq:np.float64(1.2473218482919037) - actor/pg_loss:np.float64(-0.00010735273826867342) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.374866226607992e-05) - actor/ppo_kl:np.float64(-8.093407468590108e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18677664920687675) - perf/mfu/actor:np.float64(0.1273840230570504) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.69750595092773) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00654712226241827 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00654712226241827 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:729.984375 - response_length/max:1647.0 - response_length/min:212.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:729.984375 - response_length_non_aborted/max:1647.0 - response_length_non_aborted/min:212.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.8125 - prompt_length/max:706.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014764443039894104 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8548108004033566) - timing_s/agent_loop/generate_sequences/max:np.float64(14.612083107233047) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.586479216020962) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.612083107233047) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:205 - timing_s/agent_loop/slowest/response_length:1647 - timing_s/gen:21.41356766782701 - timing_s/reward:0.08048001676797867 - timing_s/old_log_prob:3.2531582564115524 - timing_s/adv:0.11875008791685104 - timing_s/update_actor:13.537027467042208 - timing_s/step:38.487668914720416 - timing_s/stop_profile:0.00011523440480232239 - timing_per_token_ms/adv:0.00045044565796065303 - timing_per_token_ms/gen:0.11458703989718856 - timing_per_token_ms/update_actor:0.051348974566594624 - perf/total_num_tokens:263628 - perf/time_per_step:38.487668914720416 - perf/throughput:856.2092984383432 (TaskRunner pid=2224264) Training Progress: 85%|████████▌ | 85/100 [1:24:58<13:11, 52.74s/it] (TaskRunner pid=2224264) step:86 - global_seqlen/min:24044 - global_seqlen/max:35362 - global_seqlen/minmax_diff:11318 - global_seqlen/balanced_min:30989 - global_seqlen/balanced_max:31001 - global_seqlen/mean:30998.375 - actor/entropy:0.6628086566925049 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4241427183151245 - training/rollout_probs_diff_mean:0.004253617953509092 - training/rollout_probs_diff_std:0.008793003857135773 - training/rollout_actor_probs_pearson_corr:0.9995636940002441 - rollout_corr/rollout_is_mean:1.00003182888031 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.650387720379513e-06 - rollout_corr/rollout_is_max:1.8955367803573608 - rollout_corr/rollout_is_min:0.47840172052383423 - rollout_corr/rollout_is_std:0.03403814136981964 - rollout_corr/rollout_is_eff_sample_size:0.998842864637548 - rollout_corr/rollout_is_seq_mean:0.999988317489624 - rollout_corr/rollout_is_seq_std:0.001431034179404378 - rollout_corr/rollout_is_seq_max:1.0039067268371582 - rollout_corr/rollout_is_seq_min:0.9941695332527161 - rollout_corr/rollout_is_seq_max_deviation:0.0058304667472839355 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.980902411043644) - rollout_corr/training_log_ppl:np.float64(0.6618329326156527) - rollout_corr/kl:np.float64(0.0007042251129403265) - rollout_corr/k3_kl:np.float64(0.0006341270048437764) - rollout_corr/rollout_ppl:np.float64(1.979446857701987) - rollout_corr/rollout_log_ppl:np.float64(0.6611287064151838) - rollout_corr/log_ppl_diff:np.float64(0.0007042262004688382) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012508378131315112) - rollout_corr/log_ppl_diff_max:np.float64(0.006446421146392822) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046353936195373535) - rollout_corr/ppl_ratio:np.float64(1.0007055627647787) - rollout_corr/chi2_token:np.float64(0.0011301424819976091) - rollout_corr/chi2_seq:np.float64(0.7622500413563102) - actor/pg_loss:np.float64(0.00021916942205280066) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026571475882519735) - actor/ppo_kl:np.float64(8.79584136299627e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2480062022805214) - perf/mfu/actor:np.float64(0.12020480007027705) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.7301025390625) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002775753091555089 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002775753091555089 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:691.32421875 - response_length/max:1574.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:691.32421875 - response_length_non_aborted/max:1574.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.375 - prompt_length/max:503.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017972663044929504 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9018322117626667) - timing_s/agent_loop/generate_sequences/max:np.float64(14.014412458986044) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.677346075157402) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.014412458986044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:1521 - timing_s/gen:20.71387554332614 - timing_s/reward:0.0748683363199234 - timing_s/old_log_prob:3.207970952615142 - timing_s/adv:0.11068613082170486 - timing_s/update_actor:13.604356173425913 - timing_s/step:37.79681725241244 - timing_s/stop_profile:2.8118491172790527e-05 - timing_per_token_ms/adv:0.00044633844040899265 - timing_per_token_ms/gen:0.11704143171407987 - timing_per_token_ms/update_actor:0.05485915057412652 - perf/total_num_tokens:247987 - perf/time_per_step:37.79681725241244 - perf/throughput:820.1318855232839 (TaskRunner pid=2224264) Training Progress: 86%|████████▌ | 86/100 [1:25:36<11:15, 48.26s/it] (TaskRunner pid=2224264) step:87 - global_seqlen/min:27376 - global_seqlen/max:34559 - global_seqlen/minmax_diff:7183 - global_seqlen/balanced_min:30988 - global_seqlen/balanced_max:31014 - global_seqlen/mean:31004.375 - actor/entropy:0.6844121217727661 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45646268129348755 - training/rollout_probs_diff_mean:0.004233994521200657 - training/rollout_probs_diff_std:0.008540390990674496 - training/rollout_actor_probs_pearson_corr:0.9996417760848999 - rollout_corr/rollout_is_mean:1.000025749206543 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1601264304772485e-05 - rollout_corr/rollout_is_max:1.5985411405563354 - rollout_corr/rollout_is_min:0.412740021944046 - rollout_corr/rollout_is_std:0.033871378749608994 - rollout_corr/rollout_is_eff_sample_size:0.998854282390456 - rollout_corr/rollout_is_seq_mean:1.0000381469726562 - rollout_corr/rollout_is_seq_std:0.0015245388494804502 - rollout_corr/rollout_is_seq_max:1.0053566694259644 - rollout_corr/rollout_is_seq_min:0.9960179328918457 - rollout_corr/rollout_is_seq_max_deviation:0.0053566694259643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0139922401867807) - rollout_corr/training_log_ppl:np.float64(0.6768635550397448) - rollout_corr/kl:np.float64(0.0006538237156501481) - rollout_corr/k3_kl:np.float64(0.0006354092881792894) - rollout_corr/rollout_ppl:np.float64(2.0125835882499814) - rollout_corr/rollout_log_ppl:np.float64(0.6762097296887077) - rollout_corr/log_ppl_diff:np.float64(0.0006538253510370851) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013526437105610967) - rollout_corr/log_ppl_diff_max:np.float64(0.00598597526550293) - rollout_corr/log_ppl_diff_min:np.float64(-0.005899667739868164) - rollout_corr/ppl_ratio:np.float64(1.0006554168649018) - rollout_corr/chi2_token:np.float64(0.0012414318043738604) - rollout_corr/chi2_seq:np.float64(0.7429793914780021) - actor/pg_loss:np.float64(-8.32361401990056e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030786197271481797) - actor/ppo_kl:np.float64(9.98133720919725e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2665778249502182) - perf/mfu/actor:np.float64(0.12025084389863877) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.70088958740234) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005423591006547213 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005423591006547213 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:673.41796875 - response_length/max:1719.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:673.41796875 - response_length_non_aborted/max:1719.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.46875 - prompt_length/max:845.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.614036202430725e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9507676176726818) - timing_s/agent_loop/generate_sequences/max:np.float64(14.615875907242298) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.613317727045796) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.615875907242298) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:1719 - timing_s/gen:21.28308396600187 - timing_s/reward:0.0755326934158802 - timing_s/old_log_prob:3.1504952367395163 - timing_s/adv:0.11150607839226723 - timing_s/update_actor:13.641991836950183 - timing_s/step:38.34868148341775 - timing_s/stop_profile:3.3913180232048035e-05 - timing_per_token_ms/adv:0.00044955783817714125 - timing_per_token_ms/gen:0.12345534363526708 - timing_per_token_ms/update_actor:0.055000269465801936 - perf/total_num_tokens:248035 - perf/time_per_step:38.34868148341775 - perf/throughput:808.4860756791995 (TaskRunner pid=2224264) Training Progress: 87%|████████▋ | 87/100 [1:26:14<09:48, 45.29s/it] (TaskRunner pid=2224264) step:88 - global_seqlen/min:25663 - global_seqlen/max:37476 - global_seqlen/minmax_diff:11813 - global_seqlen/balanced_min:30723 - global_seqlen/balanced_max:30761 - global_seqlen/mean:30750.875 - actor/entropy:0.6911125183105469 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.729590892791748 - training/rollout_probs_diff_mean:0.004324982408434153 - training/rollout_probs_diff_std:0.0088273286819458 - training/rollout_actor_probs_pearson_corr:0.9995751976966858 - rollout_corr/rollout_is_mean:0.9999687671661377 - rollout_corr/rollout_is_ratio_fraction_high:5.643627446261235e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.128725489252247e-05 - rollout_corr/rollout_is_max:3.966233491897583 - rollout_corr/rollout_is_min:0.06101220101118088 - rollout_corr/rollout_is_std:0.03435536473989487 - rollout_corr/rollout_is_eff_sample_size:0.9988211002691073 - rollout_corr/rollout_is_seq_mean:0.9998989105224609 - rollout_corr/rollout_is_seq_std:0.0017780093476176262 - rollout_corr/rollout_is_seq_max:1.012873888015747 - rollout_corr/rollout_is_seq_min:0.9941136837005615 - rollout_corr/rollout_is_seq_max_deviation:0.01287388801574707 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0185752864927053) - rollout_corr/training_log_ppl:np.float64(0.6845818213769235) - rollout_corr/kl:np.float64(0.0008537062007487606) - rollout_corr/k3_kl:np.float64(0.0007372877449256521) - rollout_corr/rollout_ppl:np.float64(2.0169089869596064) - rollout_corr/rollout_log_ppl:np.float64(0.6837281162734143) - rollout_corr/log_ppl_diff:np.float64(0.0008537051035091281) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014192980015650392) - rollout_corr/log_ppl_diff_max:np.float64(0.015641063451766968) - rollout_corr/log_ppl_diff_min:np.float64(-0.0030567049980163574) - rollout_corr/ppl_ratio:np.float64(1.0008557450491935) - rollout_corr/chi2_token:np.float64(0.0012827848549932241) - rollout_corr/chi2_seq:np.float64(1.2393478201702237) - actor/pg_loss:np.float64(-2.0556850358843803e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026034799202534487) - actor/ppo_kl:np.float64(7.98297634219125e-05) - actor/pg_clipfrac_lower:np.float64(1.187310044770129e-05) - actor/grad_norm:np.float64(0.25147897377610207) - perf/mfu/actor:np.float64(0.11886404252645519) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.61107635498047) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010137365898117423 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010137365898117423 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:692.15234375 - response_length/max:1935.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:692.15234375 - response_length_non_aborted/max:1935.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.8125 - prompt_length/max:482.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.903677523136139e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8212964609265327) - timing_s/agent_loop/generate_sequences/max:np.float64(16.11359540745616) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.880805795379274) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.11359540745616) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:371 - timing_s/agent_loop/slowest/response_length:1935 - timing_s/gen:22.885224971920252 - timing_s/reward:0.0695581566542387 - timing_s/old_log_prob:3.2812732346355915 - timing_s/adv:0.15897942706942558 - timing_s/update_actor:13.396265247836709 - timing_s/step:39.86858329549432 - timing_s/stop_profile:0.00011085905134677887 - timing_per_token_ms/adv:0.0006462394446882633 - timing_per_token_ms/gen:0.1291556849496885 - timing_per_token_ms/update_actor:0.05445481326887734 - perf/total_num_tokens:246007 - perf/time_per_step:39.86858329549432 - perf/throughput:771.3059371105182 (TaskRunner pid=2224264) Training Progress: 88%|████████▊ | 88/100 [1:26:54<08:44, 43.68s/it] (TaskRunner pid=2224264) step:89 - global_seqlen/min:30256 - global_seqlen/max:40801 - global_seqlen/minmax_diff:10545 - global_seqlen/balanced_min:34912 - global_seqlen/balanced_max:34937 - global_seqlen/mean:34929.25 - actor/entropy:0.7014224529266357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5826672315597534 - training/rollout_probs_diff_mean:0.004312381613999605 - training/rollout_probs_diff_std:0.008647686801850796 - training/rollout_actor_probs_pearson_corr:0.9996581673622131 - rollout_corr/rollout_is_mean:1.0000931024551392 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.952648199221585e-05 - rollout_corr/rollout_is_max:1.9627834558486938 - rollout_corr/rollout_is_min:0.3244985342025757 - rollout_corr/rollout_is_std:0.03435710072517395 - rollout_corr/rollout_is_eff_sample_size:0.9988213381259312 - rollout_corr/rollout_is_seq_mean:1.000161051750183 - rollout_corr/rollout_is_seq_std:0.0014398518251255155 - rollout_corr/rollout_is_seq_max:1.0050846338272095 - rollout_corr/rollout_is_seq_min:0.9958447813987732 - rollout_corr/rollout_is_seq_max_deviation:0.005084633827209473 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.095038112718612) - rollout_corr/training_log_ppl:np.float64(0.7223394099273719) - rollout_corr/kl:np.float64(0.0004870060852217861) - rollout_corr/k3_kl:np.float64(0.0006450895384659816) - rollout_corr/rollout_ppl:np.float64(2.094029687810689) - rollout_corr/rollout_log_ppl:np.float64(0.7218523986521177) - rollout_corr/log_ppl_diff:np.float64(0.00048701127525418997) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011224675690755248) - rollout_corr/log_ppl_diff_max:np.float64(0.004818141460418701) - rollout_corr/log_ppl_diff_min:np.float64(-0.003881394863128662) - rollout_corr/ppl_ratio:np.float64(1.0004880586639047) - rollout_corr/chi2_token:np.float64(0.0016097698826342821) - rollout_corr/chi2_seq:np.float64(1.3918255360331386) - actor/pg_loss:np.float64(7.724866736680269e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001636164154206199) - actor/ppo_kl:np.float64(3.664902180844365e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23648764193058014) - perf/mfu/actor:np.float64(0.1338068099945285) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.57841873168945) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012478643329814076 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0012478643329814076 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:800.1953125 - response_length/max:1701.0 - response_length/min:290.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:800.1953125 - response_length_non_aborted/max:1701.0 - response_length_non_aborted/min:290.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.34375 - prompt_length/max:720.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.920712232589722e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.7672608997672796) - timing_s/agent_loop/generate_sequences/max:np.float64(15.327744403854012) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.27638334871881) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.327744403854012) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:223 - timing_s/agent_loop/slowest/response_length:1701 - timing_s/gen:21.85275011509657 - timing_s/reward:0.08627365715801716 - timing_s/old_log_prob:3.273784726858139 - timing_s/adv:0.1331963650882244 - timing_s/update_actor:13.806754915043712 - timing_s/step:39.24401411600411 - timing_s/stop_profile:0.00011419318616390228 - timing_per_token_ms/adv:0.0004766648478289128 - timing_per_token_ms/gen:0.10667683727164544 - timing_per_token_ms/update_actor:0.04940971719634587 - perf/total_num_tokens:279434 - perf/time_per_step:39.24401411600411 - perf/throughput:890.0529363981524 (TaskRunner pid=2224264) Training Progress: 89%|████████▉ | 89/100 [1:27:34<07:45, 42.36s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a product molecule: (TaskRunner pid=2224264) **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This is a nitrogen-containing heterocyclic compound with the following structural features: (TaskRunner pid=2224264) - A ring with the formula Nc1ccn(...)n1, suggesting a pyrrole-like ring (a five-membered ring with a nitrogen). (TaskRunner pid=2224264) - A substituent on the nitrogen: **Cc2ccc(Cl)cc2C(F)(F)F**, which is a benzene ring with a chlorine and a trifluoromethyl group attached to the same carbon. (TaskRunner pid=2224264) (TaskRunner pid=2224264) We are to determine which of the four options is the **only correct reactant** used in the synthesis of this product. Let's analyze the options one by one. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option A: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This molecule has a **benzene ring** (c2ccccc2), a **carbonyl group** (O=C), and a **trifluoromethyl group** (C(F)(F)F) attached to a carbon that is also connected to a **chlorine-substituted benzene ring**. Additionally, it contains a **pyrrole-like ring** with a nitrogen (N1) and a substituent **Cc2ccc(Cl)cc2C(F)(F)F**, which is **identical to the substituent in the product**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) This matches the **product** almost exactly, and it appears to be a **precursor** or **reactant** that could form the product via a reaction such as a **condensation or cyclization**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option B: (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This molecule has a **benzene ring** and a **trifluoromethyl group is missing**, and the nitrogen is attached to a **chlorine** (C(Cl)N1), not a **trifluoromethyl** group. It also has a **different structure** (e.g., a different substituent on the nitrogen), and the ring structure doesn't match the product's pyrrole-like ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Not a match.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option C: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **benzene ring**, a **trifluoromethyl group** (C(F)(F)F), and a **chlorine** (on a benzene ring). However, the **substituent on the nitrogen is different** (it has an **ether oxygen** and a different aromatic ring), and the **structure of the ring is different** from the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Not a match.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Option D: (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) This has a **trifluoromethyl group** and a **benzene ring**, but the **substituent on the nitrogen is not the same** as in the product (it has an ethyl group, not the same benzene ring with a chlorine). The **ring structure is also different**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Not a match.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Final Analysis: (TaskRunner pid=2224264) Only **Option A** has: (TaskRunner pid=2224264) - A **pyrrole-like ring** with a nitrogen. (TaskRunner pid=2224264) - A **trifluoromethyl group** (C(F)(F)F). (TaskRunner pid=2224264) - A **chlorine-substituted benzene ring** (Cc2ccc(Cl)cc2C(F)(F)F). (TaskRunner pid=2224264) - The **same substituent as in the product**. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Therefore, **Option A** is the only molecule that is a **valid reactant** for the synthesis of the given product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2224264) step:90 - global_seqlen/min:26503 - global_seqlen/max:35349 - global_seqlen/minmax_diff:8846 - global_seqlen/balanced_min:32189 - global_seqlen/balanced_max:32202 - global_seqlen/mean:32197.5 - actor/entropy:0.6767091751098633 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24459987878799438 - training/rollout_probs_diff_mean:0.004204816184937954 - training/rollout_probs_diff_std:0.008432798087596893 - training/rollout_actor_probs_pearson_corr:0.999668538570404 - rollout_corr/rollout_is_mean:1.000195026397705 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.363540822145296e-06 - rollout_corr/rollout_is_max:1.6443707942962646 - rollout_corr/rollout_is_min:0.3406277298927307 - rollout_corr/rollout_is_std:0.033881932497024536 - rollout_corr/rollout_is_eff_sample_size:0.9988539255817266 - rollout_corr/rollout_is_seq_mean:1.0002124309539795 - rollout_corr/rollout_is_seq_std:0.0013589308364316821 - rollout_corr/rollout_is_seq_max:1.0058317184448242 - rollout_corr/rollout_is_seq_min:0.9959684610366821 - rollout_corr/rollout_is_seq_max_deviation:0.005831718444824219 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.031145083718002) - rollout_corr/training_log_ppl:np.float64(0.6887705723638646) - rollout_corr/kl:np.float64(0.0006024212293984088) - rollout_corr/k3_kl:np.float64(0.0006607583559343766) - rollout_corr/rollout_ppl:np.float64(2.0299026197753847) - rollout_corr/rollout_log_ppl:np.float64(0.6881681511295028) - rollout_corr/log_ppl_diff:np.float64(0.0006024212343618274) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012092863908037543) - rollout_corr/log_ppl_diff_max:np.float64(0.008667349815368652) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036773681640625) - rollout_corr/ppl_ratio:np.float64(1.000603698194027) - rollout_corr/chi2_token:np.float64(0.0014522264245897532) - rollout_corr/chi2_seq:np.float64(0.6241840105503798) - actor/pg_loss:np.float64(-5.9257843531668186e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015337120976255392) - actor/ppo_kl:np.float64(0.00021126134707571964) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18526915088295937) - perf/mfu/actor:np.float64(0.11914403018889852) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.8291244506836) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6815476190476191) - val-aux/sciknoweval/reward/std@16:np.float64(0.2156221846747797) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7688476190476189) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18144705579138373) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5926619047619047) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1824273460834292) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6801857142857143) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2154753727607372) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8427380952380952) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13509620467575745) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5238857142857143) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13671434851677694) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7026523809523809) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1762779201364988) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.895695238095238) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08638656437772123) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47133333333333327) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09762932212980469) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7126714285714286) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13634398316845442) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9269000000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.048177977724428236) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.42947619047619046) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06802419225042791) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7179285714285714) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10690355682237213) - val-aux/sciknoweval/score/mean@16:np.float64(0.6815476190476191) - val-aux/sciknoweval/score/std@16:np.float64(0.2156221846747797) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7688476190476189) - val-aux/sciknoweval/score/best@2/std:np.float64(0.18144705579138373) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5926619047619047) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1824273460834292) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6801857142857143) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2154753727607372) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8427380952380952) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13509620467575745) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5238857142857143) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13671434851677694) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7026523809523809) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1762779201364988) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.895695238095238) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08638656437772123) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.47133333333333327) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09762932212980469) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7126714285714286) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13634398316845442) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9269000000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.048177977724428236) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.42947619047619046) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06802419225042791) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7179285714285714) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10690355682237213) - val-core/sciknoweval/acc/mean@16:np.float64(0.6815476190476191) - val-aux/sciknoweval/acc/std@16:np.float64(0.2156221846747797) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7688476190476189) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.18144705579138373) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5926619047619047) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1824273460834292) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6801857142857143) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2154753727607372) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8427380952380952) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13509620467575745) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5238857142857143) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13671434851677694) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7026523809523809) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1762779201364988) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.895695238095238) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08638656437772123) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.47133333333333327) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09762932212980469) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7126714285714286) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13634398316845442) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9269000000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.048177977724428236) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.42947619047619046) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06802419225042791) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7179285714285714) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10690355682237213) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9767857142857143) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0671951594177618) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9968238095238093) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.022641622754649007) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9569095238095239) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.08670860027074634) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9766666666666668) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.06717160756445564) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998523809523809) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0023879182972022734) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9241857142857143) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.10421760940450413) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9911857142857143) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.04162781171103064) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8778714285714286) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.11040129964532797) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9977285714285715) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.020137431304512083) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8244380952380953) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.09488164505205152) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9997952380952381) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00450120576193258) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:1 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026113740168511868 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026113740168511868 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:728.296875 - response_length/max:1524.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:728.296875 - response_length_non_aborted/max:1524.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.875 - prompt_length/max:621.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011400692164897919 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4655274618417025) - timing_s/agent_loop/generate_sequences/max:np.float64(14.063222439959645) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.401196123711998) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.063222439959645) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:371 - timing_s/agent_loop/slowest/response_length:1524 - timing_s/gen:20.9882550816983 - timing_s/reward:0.07745247334241867 - timing_s/old_log_prob:3.1980783604085445 - timing_s/adv:0.11822893470525742 - timing_s/update_actor:14.292423715814948 - timing_s/step:38.767167622223496 - timing_s/testing:206.41960541717708 - timing_s/save_checkpoint:12.91796376183629 - timing_s/stop_profile:0.00014068372547626495 - timing_per_token_ms/adv:0.00045899889240335977 - timing_per_token_ms/gen:0.11257136234847084 - timing_per_token_ms/update_actor:0.05548731934084536 - perf/total_num_tokens:257580 - perf/time_per_step:38.767167622223496 - perf/throughput:830.5352692710675 (TaskRunner pid=2224264) Training Progress: 90%|█████████ | 90/100 [1:31:52<17:51, 107.11s/it] (TaskRunner pid=2224264) step:91 - global_seqlen/min:27956 - global_seqlen/max:38739 - global_seqlen/minmax_diff:10783 - global_seqlen/balanced_min:31665 - global_seqlen/balanced_max:31695 - global_seqlen/mean:31683.5 - actor/entropy:0.7154297232627869 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9744127988815308 - training/rollout_probs_diff_mean:0.004367751069366932 - training/rollout_probs_diff_std:0.00891902670264244 - training/rollout_actor_probs_pearson_corr:0.9995865821838379 - rollout_corr/rollout_is_mean:0.9999821186065674 - rollout_corr/rollout_is_ratio_fraction_high:5.548773515329231e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.1097547030658461e-05 - rollout_corr/rollout_is_max:2.104722261428833 - rollout_corr/rollout_is_min:0.00197325786575675 - rollout_corr/rollout_is_std:0.03477783128619194 - rollout_corr/rollout_is_eff_sample_size:0.9987919636650865 - rollout_corr/rollout_is_seq_mean:0.9999819993972778 - rollout_corr/rollout_is_seq_std:0.0012956903083249927 - rollout_corr/rollout_is_seq_max:1.00333833694458 - rollout_corr/rollout_is_seq_min:0.9953454732894897 - rollout_corr/rollout_is_seq_max_deviation:0.004654526710510254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.104408088605851) - rollout_corr/training_log_ppl:np.float64(0.7243244127603248) - rollout_corr/kl:np.float64(0.0006612393046498966) - rollout_corr/k3_kl:np.float64(0.0006583722558843874) - rollout_corr/rollout_ppl:np.float64(2.102920087054372) - rollout_corr/rollout_log_ppl:np.float64(0.7236631693085656) - rollout_corr/log_ppl_diff:np.float64(0.0006612434517592192) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012168984394520521) - rollout_corr/log_ppl_diff_max:np.float64(0.00693821907043457) - rollout_corr/log_ppl_diff_min:np.float64(-0.0027844905853271484) - rollout_corr/ppl_ratio:np.float64(1.0006624751258641) - rollout_corr/chi2_token:np.float64(0.001317617017775774) - rollout_corr/chi2_seq:np.float64(1.2634925008751452) - actor/pg_loss:np.float64(0.0001394905848428607) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015355198502220446) - actor/ppo_kl:np.float64(-4.81430525312021e-06) - actor/pg_clipfrac_lower:np.float64(4.3841191654792055e-06) - actor/grad_norm:np.float64(0.22441749274730682) - perf/mfu/actor:np.float64(0.1235221491000339) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(107.04840850830078) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006852735532447696 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006852735532447696 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:703.984375 - response_length/max:1848.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:703.984375 - response_length_non_aborted/max:1848.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.125 - prompt_length/max:1009.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.789582014083862e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.103301614522934) - timing_s/agent_loop/generate_sequences/max:np.float64(15.148089375346899) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.013559678431193) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.148089375346899) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:191 - timing_s/agent_loop/slowest/response_length:1848 - timing_s/gen:21.60629509203136 - timing_s/reward:0.07231813855469227 - timing_s/old_log_prob:3.4730070438236 - timing_s/adv:0.13708950020372868 - timing_s/update_actor:13.167173700407147 - timing_s/step:38.57017913646996 - timing_s/stop_profile:0.00017870403826236725 - timing_per_token_ms/adv:0.0005408552566940548 - timing_per_token_ms/gen:0.11988844241500034 - timing_per_token_ms/update_actor:0.051948071158517635 - perf/total_num_tokens:253468 - perf/time_per_step:38.57017913646996 - perf/throughput:821.4506831274145 (TaskRunner pid=2224264) Training Progress: 91%|█████████ | 91/100 [1:32:30<12:59, 86.56s/it] (TaskRunner pid=2224264) step:92 - global_seqlen/min:28907 - global_seqlen/max:37134 - global_seqlen/minmax_diff:8227 - global_seqlen/balanced_min:32432 - global_seqlen/balanced_max:32477 - global_seqlen/mean:32465.625 - actor/entropy:0.7178817987442017 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2975451946258545 - training/rollout_probs_diff_mean:0.004235805943608284 - training/rollout_probs_diff_std:0.008502726443111897 - training/rollout_actor_probs_pearson_corr:0.9996208548545837 - rollout_corr/rollout_is_mean:0.9999630451202393 - rollout_corr/rollout_is_ratio_fraction_high:5.372733539843466e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1497116088867188 - rollout_corr/rollout_is_min:0.550010621547699 - rollout_corr/rollout_is_std:0.03401536867022514 - rollout_corr/rollout_is_eff_sample_size:0.9988440539746302 - rollout_corr/rollout_is_seq_mean:0.9999423623085022 - rollout_corr/rollout_is_seq_std:0.0013055680319666862 - rollout_corr/rollout_is_seq_max:1.004227876663208 - rollout_corr/rollout_is_seq_min:0.9956101775169373 - rollout_corr/rollout_is_seq_max_deviation:0.004389822483062744 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.064156681764871) - rollout_corr/training_log_ppl:np.float64(0.7071324991993606) - rollout_corr/kl:np.float64(0.0008166503134958703) - rollout_corr/k3_kl:np.float64(0.0006099401913388647) - rollout_corr/rollout_ppl:np.float64(2.0623925644904375) - rollout_corr/rollout_log_ppl:np.float64(0.7063158513628878) - rollout_corr/log_ppl_diff:np.float64(0.0008166478364728391) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012665537069551647) - rollout_corr/log_ppl_diff_max:np.float64(0.007030129432678223) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036208629608154297) - rollout_corr/ppl_ratio:np.float64(1.0008179140277207) - rollout_corr/chi2_token:np.float64(0.0008041651453822851) - rollout_corr/chi2_seq:np.float64(0.6329832167830318) - actor/pg_loss:np.float64(6.987596862018108e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012492035489231057) - actor/ppo_kl:np.float64(0.0001699282564437965) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19231200218200684) - perf/mfu/actor:np.float64(0.12487351766630518) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.44847011566162) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004429147113114595 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004429147113114595 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:727.05078125 - response_length/max:1984.0 - response_length/min:182.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:727.05078125 - response_length_non_aborted/max:1984.0 - response_length_non_aborted/min:182.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:287.5 - prompt_length/max:436.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019785761833190918 - timing_s/agent_loop/generate_sequences/min:np.float64(2.329558404162526) - timing_s/agent_loop/generate_sequences/max:np.float64(16.53738614730537) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.280053526119445) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.53738614730537) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:1984 - timing_s/gen:23.22112664207816 - timing_s/reward:0.07667875476181507 - timing_s/old_log_prob:3.3302839752286673 - timing_s/adv:0.12147887982428074 - timing_s/update_actor:13.518468294292688 - timing_s/step:40.354144191369414 - timing_s/stop_profile:0.00012041814625263214 - timing_per_token_ms/adv:0.00046772116594197993 - timing_per_token_ms/gen:0.12476092218712242 - timing_per_token_ms/update_actor:0.0520491608212251 - perf/total_num_tokens:259725 - perf/time_per_step:40.354144191369414 - perf/throughput:804.5177428627878 (TaskRunner pid=2224264) Training Progress: 92%|█████████▏| 92/100 [1:33:11<09:41, 72.72s/it] (TaskRunner pid=2224264) step:93 - global_seqlen/min:23013 - global_seqlen/max:34946 - global_seqlen/minmax_diff:11933 - global_seqlen/balanced_min:27908 - global_seqlen/balanced_max:27913 - global_seqlen/mean:27910.875 - actor/entropy:0.7500928044319153 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1930849552154541 - training/rollout_probs_diff_mean:0.004663003142923117 - training/rollout_probs_diff_std:0.008796103298664093 - training/rollout_actor_probs_pearson_corr:0.9996201992034912 - rollout_corr/rollout_is_mean:0.999956488609314 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6296987533569336 - rollout_corr/rollout_is_min:0.6042289137840271 - rollout_corr/rollout_is_std:0.035632822662591934 - rollout_corr/rollout_is_eff_sample_size:0.9987316741318075 - rollout_corr/rollout_is_seq_mean:0.9998897910118103 - rollout_corr/rollout_is_seq_std:0.0015105620259419084 - rollout_corr/rollout_is_seq_max:1.0058465003967285 - rollout_corr/rollout_is_seq_min:0.9952189922332764 - rollout_corr/rollout_is_seq_max_deviation:0.005846500396728516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1263672169297934) - rollout_corr/training_log_ppl:np.float64(0.7347684952546842) - rollout_corr/kl:np.float64(0.0007506872085336624) - rollout_corr/k3_kl:np.float64(0.0006735208150985272) - rollout_corr/rollout_ppl:np.float64(2.124785815831274) - rollout_corr/rollout_log_ppl:np.float64(0.7340178099111654) - rollout_corr/log_ppl_diff:np.float64(0.0007506853435188532) - rollout_corr/log_ppl_abs_diff:np.float64(0.001359031768515706) - rollout_corr/log_ppl_diff_max:np.float64(0.005597174167633057) - rollout_corr/log_ppl_diff_min:np.float64(-0.003313422203063965) - rollout_corr/ppl_ratio:np.float64(1.000752320047468) - rollout_corr/chi2_token:np.float64(0.0011984859593212605) - rollout_corr/chi2_seq:np.float64(0.6925807157531381) - actor/pg_loss:np.float64(-4.779419396072626e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.594229848313262e-05) - actor/ppo_kl:np.float64(-7.275951552143667e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18249434977769852) - perf/mfu/actor:np.float64(0.11041731672624022) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.24386596679688) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.8515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007838345132768154 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007838345132768154 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:605.74609375 - response_length/max:1369.0 - response_length/min:190.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:605.74609375 - response_length_non_aborted/max:1369.0 - response_length_non_aborted/min:190.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:709.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018016062676906586 - timing_s/agent_loop/generate_sequences/min:np.float64(2.355575904250145) - timing_s/agent_loop/generate_sequences/max:np.float64(11.529610631987453) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.815270844308543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.529610631987453) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:1369 - timing_s/gen:18.318352431058884 - timing_s/reward:0.07145974412560463 - timing_s/old_log_prob:3.1092032585293055 - timing_s/adv:0.12323503755033016 - timing_s/update_actor:13.111440777778625 - timing_s/step:34.81759689189494 - timing_s/stop_profile:2.6732683181762695e-05 - timing_per_token_ms/adv:0.0005519131769889432 - timing_per_token_ms/gen:0.11812880829464492 - timing_per_token_ms/update_actor:0.05872012601619721 - perf/total_num_tokens:223287 - perf/time_per_step:34.81759689189494 - perf/throughput:801.6312868076565 (TaskRunner pid=2224264) Training Progress: 93%|█████████▎| 93/100 [1:33:46<07:09, 61.36s/it] (TaskRunner pid=2224264) step:94 - global_seqlen/min:24960 - global_seqlen/max:37338 - global_seqlen/minmax_diff:12378 - global_seqlen/balanced_min:30359 - global_seqlen/balanced_max:30372 - global_seqlen/mean:30366.25 - actor/entropy:0.7063392400741577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3128166198730469 - training/rollout_probs_diff_mean:0.004224883858114481 - training/rollout_probs_diff_std:0.008449923247098923 - training/rollout_actor_probs_pearson_corr:0.9996349811553955 - rollout_corr/rollout_is_mean:0.9998794198036194 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6113572100002784e-06 - rollout_corr/rollout_is_max:1.5920956134796143 - rollout_corr/rollout_is_min:0.19120460748672485 - rollout_corr/rollout_is_std:0.03371969982981682 - rollout_corr/rollout_is_eff_sample_size:0.99886415419973 - rollout_corr/rollout_is_seq_mean:0.9998659491539001 - rollout_corr/rollout_is_seq_std:0.0013268847251310945 - rollout_corr/rollout_is_seq_max:1.0026715993881226 - rollout_corr/rollout_is_seq_min:0.9956828355789185 - rollout_corr/rollout_is_seq_max_deviation:0.004317164421081543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.097409463953227) - rollout_corr/training_log_ppl:np.float64(0.7194324630545452) - rollout_corr/kl:np.float64(0.0006619715687250505) - rollout_corr/k3_kl:np.float64(0.0006161111765550231) - rollout_corr/rollout_ppl:np.float64(2.0959334070794284) - rollout_corr/rollout_log_ppl:np.float64(0.7187704919488169) - rollout_corr/log_ppl_diff:np.float64(0.0006619711057282984) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012567761004902422) - rollout_corr/log_ppl_diff_max:np.float64(0.004305779933929443) - rollout_corr/log_ppl_diff_min:np.float64(-0.004627227783203125) - rollout_corr/ppl_ratio:np.float64(1.0006632735021412) - rollout_corr/chi2_token:np.float64(0.00113762472756207) - rollout_corr/chi2_seq:np.float64(2.2418907159008086) - actor/pg_loss:np.float64(9.933649562299252e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.07081788580399e-05) - actor/ppo_kl:np.float64(-6.808040846806307e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1753209587186575) - perf/mfu/actor:np.float64(0.11873190434052154) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.19572830200195) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006172493100166321 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006172493100166321 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:696.1328125 - response_length/max:1462.0 - response_length/min:227.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:696.1328125 - response_length_non_aborted/max:1462.0 - response_length_non_aborted/min:227.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.8125 - prompt_length/max:384.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.6177377700805664e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.743150135502219) - timing_s/agent_loop/generate_sequences/max:np.float64(13.47325548902154) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.079402086157643) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.47325548902154) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:1462 - timing_s/gen:20.203036999329925 - timing_s/reward:0.07774028927087784 - timing_s/old_log_prob:3.2072019651532173 - timing_s/adv:0.12383914738893509 - timing_s/update_actor:13.169512040913105 - timing_s/step:36.86546222306788 - timing_s/stop_profile:0.00011588446795940399 - timing_per_token_ms/adv:0.0005097729691225254 - timing_per_token_ms/gen:0.11336646091313576 - timing_per_token_ms/update_actor:0.054211139179653006 - perf/total_num_tokens:242930 - perf/time_per_step:36.86546222306788 - perf/throughput:823.7045779124636 (TaskRunner pid=2224264) Training Progress: 94%|█████████▍| 94/100 [1:34:23<05:24, 54.02s/it] (TaskRunner pid=2224264) step:95 - global_seqlen/min:20976 - global_seqlen/max:34422 - global_seqlen/minmax_diff:13446 - global_seqlen/balanced_min:29028 - global_seqlen/balanced_max:29035 - global_seqlen/mean:29031.75 - actor/entropy:0.712871789932251 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2599616050720215 - training/rollout_probs_diff_mean:0.004302844405174255 - training/rollout_probs_diff_std:0.008476506918668747 - training/rollout_actor_probs_pearson_corr:0.9996457099914551 - rollout_corr/rollout_is_mean:1.0001041889190674 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.065090474294266e-06 - rollout_corr/rollout_is_max:1.6687859296798706 - rollout_corr/rollout_is_min:0.4844895303249359 - rollout_corr/rollout_is_std:0.03429459035396576 - rollout_corr/rollout_is_eff_sample_size:0.9988256195681333 - rollout_corr/rollout_is_seq_mean:1.0001366138458252 - rollout_corr/rollout_is_seq_std:0.0015432714717462659 - rollout_corr/rollout_is_seq_max:1.005467414855957 - rollout_corr/rollout_is_seq_min:0.9945797324180603 - rollout_corr/rollout_is_seq_max_deviation:0.005467414855957031 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0434681265614927) - rollout_corr/training_log_ppl:np.float64(0.6912423392059281) - rollout_corr/kl:np.float64(0.0005482688078317643) - rollout_corr/k3_kl:np.float64(0.0006431571834468741) - rollout_corr/rollout_ppl:np.float64(2.0423742262646556) - rollout_corr/rollout_log_ppl:np.float64(0.6906940678600222) - rollout_corr/log_ppl_diff:np.float64(0.0005482713459059596) - rollout_corr/log_ppl_abs_diff:np.float64(0.001258363830856979) - rollout_corr/log_ppl_diff_max:np.float64(0.005878806114196777) - rollout_corr/log_ppl_diff_min:np.float64(-0.005048781633377075) - rollout_corr/ppl_ratio:np.float64(1.000549679622054) - rollout_corr/chi2_token:np.float64(0.0014886993449181318) - rollout_corr/chi2_seq:np.float64(3.0285099220927805) - actor/pg_loss:np.float64(-0.000160741969011724) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010068776555272052) - actor/ppo_kl:np.float64(7.83663859635908e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23990549892187119) - perf/mfu/actor:np.float64(0.11424140158128934) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.15832138061523) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014393975958228111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014393975958228111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:644.0546875 - response_length/max:1596.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:644.0546875 - response_length_non_aborted/max:1596.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.1875 - prompt_length/max:646.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.495019912719727e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.937621222808957) - timing_s/agent_loop/generate_sequences/max:np.float64(13.714711025357246) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.356954876297095) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.714711025357246) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:218 - timing_s/agent_loop/slowest/response_length:1596 - timing_s/gen:20.76876393146813 - timing_s/reward:0.07113218307495117 - timing_s/old_log_prob:3.2435854095965624 - timing_s/adv:0.1196875274181366 - timing_s/update_actor:13.214123142883182 - timing_s/step:37.502956656739116 - timing_s/stop_profile:0.00011238828301429749 - timing_per_token_ms/adv:0.0005153303168864115 - timing_per_token_ms/gen:0.12596443389335224 - timing_per_token_ms/update_actor:0.056895136974532974 - perf/total_num_tokens:232254 - perf/time_per_step:37.502956656739116 - perf/throughput:774.1189652251891 (TaskRunner pid=2224264) Training Progress: 95%|█████████▌| 95/100 [1:35:00<04:05, 49.08s/it] (TaskRunner pid=2224264) step:96 - global_seqlen/min:27198 - global_seqlen/max:35708 - global_seqlen/minmax_diff:8510 - global_seqlen/balanced_min:32177 - global_seqlen/balanced_max:32213 - global_seqlen/mean:32201.875 - actor/entropy:0.7203529477119446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6764640808105469 - training/rollout_probs_diff_mean:0.004217594396322966 - training/rollout_probs_diff_std:0.008475503884255886 - training/rollout_actor_probs_pearson_corr:0.999649703502655 - rollout_corr/rollout_is_mean:1.0000306367874146 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.28119744558353e-06 - rollout_corr/rollout_is_max:1.7523001432418823 - rollout_corr/rollout_is_min:0.0011872303439304233 - rollout_corr/rollout_is_std:0.03390303626656532 - rollout_corr/rollout_is_eff_sample_size:0.9988520226061409 - rollout_corr/rollout_is_seq_mean:0.9999527931213379 - rollout_corr/rollout_is_seq_std:0.0014494654024019837 - rollout_corr/rollout_is_seq_max:1.0033957958221436 - rollout_corr/rollout_is_seq_min:0.9943961501121521 - rollout_corr/rollout_is_seq_max_deviation:0.0056038498878479 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0765154217369854) - rollout_corr/training_log_ppl:np.float64(0.7100228805211373) - rollout_corr/kl:np.float64(0.0007996692028076069) - rollout_corr/k3_kl:np.float64(0.0006997747711352531) - rollout_corr/rollout_ppl:np.float64(2.074850105214864) - rollout_corr/rollout_log_ppl:np.float64(0.7092232063878328) - rollout_corr/log_ppl_diff:np.float64(0.0007996741333045065) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013392329565249383) - rollout_corr/log_ppl_diff_max:np.float64(0.006226241588592529) - rollout_corr/log_ppl_diff_min:np.float64(-0.002697765827178955) - rollout_corr/ppl_ratio:np.float64(1.0008012598846108) - rollout_corr/chi2_token:np.float64(0.0011850390583276749) - rollout_corr/chi2_seq:np.float64(0.8988333453889936) - actor/pg_loss:np.float64(-6.152375135570765e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024159732652151433) - actor/ppo_kl:np.float64(0.00012709150854451137) - actor/pg_clipfrac_lower:np.float64(4.822531082027126e-06) - actor/grad_norm:np.float64(0.21260051056742668) - perf/mfu/actor:np.float64(0.12350445771061563) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.11673736572266) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01730252243578434 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01730252243578434 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:739.65234375 - response_length/max:1942.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:739.65234375 - response_length_non_aborted/max:1942.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.65625 - prompt_length/max:456.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010816939175128937 - timing_s/agent_loop/generate_sequences/min:np.float64(1.915278973057866) - timing_s/agent_loop/generate_sequences/max:np.float64(16.20021486841142) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.129882725828793) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.20021486841142) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:1903 - timing_s/gen:23.026053931564093 - timing_s/reward:0.07937229797244072 - timing_s/old_log_prob:3.2644342444837093 - timing_s/adv:0.1240159124135971 - timing_s/update_actor:13.31886836886406 - timing_s/step:39.908353347331285 - timing_s/stop_profile:0.00011824257671833038 - timing_per_token_ms/adv:0.00048140019957532403 - timing_per_token_ms/gen:0.12160513507488259 - timing_per_token_ms/update_actor:0.05170067103570856 - perf/total_num_tokens:257615 - perf/time_per_step:39.908353347331285 - perf/throughput:806.8956070359484 (TaskRunner pid=2224264) Training Progress: 96%|█████████▌| 96/100 [1:35:40<03:05, 46.34s/it] (TaskRunner pid=2224264) step:97 - global_seqlen/min:31313 - global_seqlen/max:43694 - global_seqlen/minmax_diff:12381 - global_seqlen/balanced_min:35934 - global_seqlen/balanced_max:35960 - global_seqlen/mean:35950.125 - actor/entropy:0.7323668003082275 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915082573890686 - training/rollout_probs_diff_mean:0.004103551618754864 - training/rollout_probs_diff_std:0.008156946860253811 - training/rollout_actor_probs_pearson_corr:0.9996232986450195 - rollout_corr/rollout_is_mean:0.9999871850013733 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.7118023758230265e-06 - rollout_corr/rollout_is_max:1.6210092306137085 - rollout_corr/rollout_is_min:0.4609464108943939 - rollout_corr/rollout_is_std:0.03369848057627678 - rollout_corr/rollout_is_eff_sample_size:0.9988657004043994 - rollout_corr/rollout_is_seq_mean:1.000003695487976 - rollout_corr/rollout_is_seq_std:0.0014436065685003996 - rollout_corr/rollout_is_seq_max:1.0053772926330566 - rollout_corr/rollout_is_seq_min:0.9946680665016174 - rollout_corr/rollout_is_seq_max_deviation:0.005377292633056641 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1260625598952174) - rollout_corr/training_log_ppl:np.float64(0.7336573095526546) - rollout_corr/kl:np.float64(0.0006526818591758854) - rollout_corr/k3_kl:np.float64(0.0006329746193500796) - rollout_corr/rollout_ppl:np.float64(2.124627975281328) - rollout_corr/rollout_log_ppl:np.float64(0.7330046314746141) - rollout_corr/log_ppl_diff:np.float64(0.0006526780780404806) - rollout_corr/log_ppl_abs_diff:np.float64(0.001235815929248929) - rollout_corr/log_ppl_diff_max:np.float64(0.00612562894821167) - rollout_corr/log_ppl_diff_min:np.float64(-0.00426250696182251) - rollout_corr/ppl_ratio:np.float64(1.0006539451424032) - rollout_corr/chi2_token:np.float64(0.0012377453967928886) - rollout_corr/chi2_seq:np.float64(0.6915090307593346) - actor/pg_loss:np.float64(-3.366696182638407e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016491950168529002) - actor/ppo_kl:np.float64(6.479799488445792e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20394239574670792) - perf/mfu/actor:np.float64(0.135031184027903) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(106.04956817626953) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012476264499127865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012476264499127865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:829.03515625 - response_length/max:2245.0 - response_length/min:215.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:829.03515625 - response_length_non_aborted/max:2245.0 - response_length_non_aborted/min:215.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.40625 - prompt_length/max:900.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010967254638671875 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4484062399715185) - timing_s/agent_loop/generate_sequences/max:np.float64(18.13408470340073) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.178321859493735) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.13408470340073) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:2245 - timing_s/gen:24.973692329600453 - timing_s/reward:0.08095383085310459 - timing_s/old_log_prob:3.3533995244652033 - timing_s/adv:0.12184758298099041 - timing_s/update_actor:13.665139958262444 - timing_s/step:42.28041913919151 - timing_s/stop_profile:4.5822933316230774e-05 - timing_per_token_ms/adv:0.0004236688432272155 - timing_per_token_ms/gen:0.11767110830832365 - timing_per_token_ms/update_actor:0.047514229638500716 - perf/total_num_tokens:287601 - perf/time_per_step:42.28041913919151 - perf/throughput:850.2783494564818 (TaskRunner pid=2224264) Training Progress: 97%|█████████▋| 97/100 [1:36:22<02:15, 45.13s/it] (TaskRunner pid=2224264) step:98 - global_seqlen/min:25644 - global_seqlen/max:42510 - global_seqlen/minmax_diff:16866 - global_seqlen/balanced_min:34661 - global_seqlen/balanced_max:34705 - global_seqlen/mean:34687.75 - actor/entropy:0.7501354813575745 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19508248567581177 - training/rollout_probs_diff_mean:0.004144626669585705 - training/rollout_probs_diff_std:0.008178996853530407 - training/rollout_actor_probs_pearson_corr:0.9996928572654724 - rollout_corr/rollout_is_mean:1.0000689029693604 - rollout_corr/rollout_is_ratio_fraction_high:4.974382136424538e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2251296043395996 - rollout_corr/rollout_is_min:0.6142621636390686 - rollout_corr/rollout_is_std:0.03403989225625992 - rollout_corr/rollout_is_eff_sample_size:0.9988425078369757 - rollout_corr/rollout_is_seq_mean:1.000027060508728 - rollout_corr/rollout_is_seq_std:0.001423987210728228 - rollout_corr/rollout_is_seq_max:1.0042808055877686 - rollout_corr/rollout_is_seq_min:0.9920931458473206 - rollout_corr/rollout_is_seq_max_deviation:0.007906854152679443 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1749815698713064) - rollout_corr/training_log_ppl:np.float64(0.7591774334432557) - rollout_corr/kl:np.float64(0.0006937429601876488) - rollout_corr/k3_kl:np.float64(0.0006346612378251848) - rollout_corr/rollout_ppl:np.float64(2.17338493373245) - rollout_corr/rollout_log_ppl:np.float64(0.7584836900932714) - rollout_corr/log_ppl_diff:np.float64(0.0006937433499842882) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011937296949326992) - rollout_corr/log_ppl_diff_max:np.float64(0.004764825105667114) - rollout_corr/log_ppl_diff_min:np.float64(-0.003519713878631592) - rollout_corr/ppl_ratio:np.float64(1.000694976421073) - rollout_corr/chi2_token:np.float64(0.0011504706926643848) - rollout_corr/chi2_seq:np.float64(0.7677441604901105) - actor/pg_loss:np.float64(-0.00023154309019446373) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014778738523091306) - actor/ppo_kl:np.float64(0.00011894125558598034) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22566131502389908) - perf/mfu/actor:np.float64(0.13134675089113737) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.8719253540039) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0074124508537352085 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0074124508537352085 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:785.2734375 - response_length/max:2121.0 - response_length/min:178.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:785.2734375 - response_length_non_aborted/max:2121.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:631.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.526918590068817e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.268034750595689) - timing_s/agent_loop/generate_sequences/max:np.float64(17.475637735798955) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.994100491654535) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.475637735798955) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:2121 - timing_s/gen:24.36921063065529 - timing_s/reward:0.08015711978077888 - timing_s/old_log_prob:3.372231272980571 - timing_s/adv:0.11694180965423584 - timing_s/update_actor:13.581646379083395 - timing_s/step:41.60994468256831 - timing_s/stop_profile:0.00010863691568374634 - timing_per_token_ms/adv:0.0004214088895007454 - timing_per_token_ms/gen:0.12122176108369541 - timing_per_token_ms/update_actor:0.04894251709567281 - perf/total_num_tokens:277502 - perf/time_per_step:41.60994468256831 - perf/throughput:833.6408583242305 (TaskRunner pid=2224264) Training Progress: 98%|█████████▊| 98/100 [1:37:04<01:28, 44.09s/it] (TaskRunner pid=2224264) step:99 - global_seqlen/min:31172 - global_seqlen/max:42762 - global_seqlen/minmax_diff:11590 - global_seqlen/balanced_min:36579 - global_seqlen/balanced_max:36620 - global_seqlen/mean:36605.125 - actor/entropy:0.7388753890991211 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583175301551819 - training/rollout_probs_diff_mean:0.004119738936424255 - training/rollout_probs_diff_std:0.008252698928117752 - training/rollout_actor_probs_pearson_corr:0.9996559023857117 - rollout_corr/rollout_is_mean:1.0000630617141724 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.395273082191125e-06 - rollout_corr/rollout_is_max:1.7297682762145996 - rollout_corr/rollout_is_min:0.46209102869033813 - rollout_corr/rollout_is_std:0.03351224586367607 - rollout_corr/rollout_is_eff_sample_size:0.998878308098044 - rollout_corr/rollout_is_seq_mean:1.0000331401824951 - rollout_corr/rollout_is_seq_std:0.0012539547169581056 - rollout_corr/rollout_is_seq_max:1.0045143365859985 - rollout_corr/rollout_is_seq_min:0.9951211214065552 - rollout_corr/rollout_is_seq_max_deviation:0.004878878593444824 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.149727716576308) - rollout_corr/training_log_ppl:np.float64(0.7441099616116844) - rollout_corr/kl:np.float64(0.0006301436051412423) - rollout_corr/k3_kl:np.float64(0.0006160447873639896) - rollout_corr/rollout_ppl:np.float64(2.1482933391816914) - rollout_corr/rollout_log_ppl:np.float64(0.7434798121103086) - rollout_corr/log_ppl_diff:np.float64(0.000630149501375854) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011455799685791135) - rollout_corr/log_ppl_diff_max:np.float64(0.0055887699127197266) - rollout_corr/log_ppl_diff_min:np.float64(-0.00408819317817688) - rollout_corr/ppl_ratio:np.float64(1.0006313105113804) - rollout_corr/chi2_token:np.float64(0.0012098602019250393) - rollout_corr/chi2_seq:np.float64(0.841775347944349) - actor/pg_loss:np.float64(-8.723500650376081e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012781737518707814) - actor/ppo_kl:np.float64(8.237922273224285e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19401156529784203) - perf/mfu/actor:np.float64(0.13594204140089314) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.93074417114258) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006903177127242088 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006903177127242088 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:831.53515625 - response_length/max:1998.0 - response_length/min:192.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:831.53515625 - response_length_non_aborted/max:1998.0 - response_length_non_aborted/min:192.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.375 - prompt_length/max:836.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.847611904144287e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.493832629173994) - timing_s/agent_loop/generate_sequences/max:np.float64(17.390929045155644) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.606087130981905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.390929045155644) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:493 - timing_s/agent_loop/slowest/response_length:1964 - timing_s/gen:24.117084549739957 - timing_s/reward:0.08298422396183014 - timing_s/old_log_prob:3.351647572591901 - timing_s/adv:0.12600908987224102 - timing_s/update_actor:14.029597079381347 - timing_s/step:41.793392254039645 - timing_s/stop_profile:0.00011212378740310669 - timing_per_token_ms/adv:0.0004302986599289069 - timing_per_token_ms/gen:0.11329329952478688 - timing_per_token_ms/update_actor:0.04790858206119138 - perf/total_num_tokens:292841 - perf/time_per_step:41.793392254039645 - perf/throughput:875.859149635355 (TaskRunner pid=2224264) Training Progress: 99%|█████████▉| 99/100 [1:37:46<00:43, 43.41s/it] (TaskRunner pid=2224264) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2224264) validation generation end (TaskRunner pid=2224264) [prompt] system (TaskRunner pid=2224264) (TaskRunner pid=2224264) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) ... (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) user (TaskRunner pid=2224264) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2224264) (TaskRunner pid=2224264) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2224264) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2224264) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2224264) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2224264) Please reason step by step. (TaskRunner pid=2224264) assistant (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) [response] (TaskRunner pid=2224264) We are given a target product: (TaskRunner pid=2224264) **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"** (TaskRunner pid=2224264) This is a chemical formula representing a molecule. Let's break it down: (TaskRunner pid=2224264) (TaskRunner pid=2224264) - The main structure is: **Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) - This can be interpreted as: a ring with a nitrogen (N) at position 1, and a substituent at position 1 (the "C..." part). (TaskRunner pid=2224264) - The substituent is: **Cc2ccc(Cl)cc2C(F)(F)F** (TaskRunner pid=2224264) This is a benzene ring (c2ccc...), with a chlorine (Cl) on one carbon, and a trifluoromethyl group (C(F)(F)F) on another carbon. The entire group is attached to a carbon (C) which is attached to the nitrogen (N) in the ring. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Now, we are to identify which of the four options is the **only correct reactant** that would be used in the synthesis of this product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) Let’s analyze each option: (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option A:** (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - This is a complex molecule. (TaskRunner pid=2224264) - It contains a ring (C1) with two double bonds (c2ccccc2), and a carbonyl group (O=C). (TaskRunner pid=2224264) - Then, **C(=O)N1** suggests a carbonyl group attached to a nitrogen (N1). (TaskRunner pid=2224264) - Then, **c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1** matches the target product's ring and substituent. (TaskRunner pid=2224264) - So, this option contains the full structure of the target product, possibly as a derivative or a starting point for synthesis. (TaskRunner pid=2224264) ✅ **Possibly a correct reactant.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option B:** (TaskRunner pid=2224264) **O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a ring with Cl attached. (TaskRunner pid=2224264) - The rest of the structure does not match the substituent **Cc2ccc(Cl)cc2C(F)(F)F**. (TaskRunner pid=2224264) - **Does not match the target structure.** (TaskRunner pid=2224264) ❌ **Not correct.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option C:** (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - This has a ring with a substituent that includes a **C(F)(F)F** group. (TaskRunner pid=2224264) - However, the **Oc2ccc(Cl)c...** is different from **Cc2ccc(Cl)cc2C(F)(F)F**. (TaskRunner pid=2224264) - The structure is not matching the target. (TaskRunner pid=2224264) ❌ **Not correct.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) **Option D:** (TaskRunner pid=2224264) **O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F** (TaskRunner pid=2224264) (TaskRunner pid=2224264) - Contains a **C(F)(F)F** group. (TaskRunner pid=2224264) - However, the **CC=C(Cc1ccccc1)** is not matching the **Cc2ccc(Cl)cc2C(F)(F)F** part. (TaskRunner pid=2224264) - No Cl is present in the substituent. (TaskRunner pid=2224264) ❌ **Not correct.** (TaskRunner pid=2224264) (TaskRunner pid=2224264) --- (TaskRunner pid=2224264) (TaskRunner pid=2224264) ### Final Evaluation: (TaskRunner pid=2224264) Only **Option A** contains the exact substituent **Cc2ccc(Cl)cc2C(F)(F)F**, which is part of the target molecule, and the rest of the structure is consistent with the ring and functional group of the product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) So, **Option A** is the only correct reactant that could be used in the synthesis of the given product. (TaskRunner pid=2224264) (TaskRunner pid=2224264) (TaskRunner pid=2224264) A (TaskRunner pid=2224264) (TaskRunner pid=2224264) [ground_truth] A (TaskRunner pid=2224264) [score] 1.0 (TaskRunner pid=2224264) [acc] 1.0 (TaskRunner pid=2224264) [pred] A (TaskRunner pid=2224264) [incorrect_format] 1 (TaskRunner pid=2224264) [feedback] (TaskRunner pid=2224264) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2224264) step:100 - global_seqlen/min:32181 - global_seqlen/max:40888 - global_seqlen/minmax_diff:8707 - global_seqlen/balanced_min:36112 - global_seqlen/balanced_max:36131 - global_seqlen/mean:36125.5 - actor/entropy:0.7533097267150879 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2046983242034912 - training/rollout_probs_diff_mean:0.0041552940383553505 - training/rollout_probs_diff_std:0.00816031452268362 - training/rollout_actor_probs_pearson_corr:0.9996163845062256 - rollout_corr/rollout_is_mean:0.999994695186615 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.690695732278982e-06 - rollout_corr/rollout_is_max:1.4722437858581543 - rollout_corr/rollout_is_min:0.08076818287372589 - rollout_corr/rollout_is_std:0.03385905548930168 - rollout_corr/rollout_is_eff_sample_size:0.9988548770722385 - rollout_corr/rollout_is_seq_mean:0.9999972581863403 - rollout_corr/rollout_is_seq_std:0.0013772835955023766 - rollout_corr/rollout_is_seq_max:1.0046179294586182 - rollout_corr/rollout_is_seq_min:0.9932000041007996 - rollout_corr/rollout_is_seq_max_deviation:0.0067999958992004395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1957929017953575) - rollout_corr/training_log_ppl:np.float64(0.7637257019523531) - rollout_corr/kl:np.float64(0.0006403183531915602) - rollout_corr/k3_kl:np.float64(0.0006129170444637566) - rollout_corr/rollout_ppl:np.float64(2.194364957511425) - rollout_corr/rollout_log_ppl:np.float64(0.7630853808368556) - rollout_corr/log_ppl_diff:np.float64(0.0006403211154974997) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010970584698952734) - rollout_corr/log_ppl_diff_max:np.float64(0.007486581802368164) - rollout_corr/log_ppl_diff_min:np.float64(-0.0030992627143859863) - rollout_corr/ppl_ratio:np.float64(1.000641372287646) - rollout_corr/chi2_token:np.float64(0.0011783752124756575) - rollout_corr/chi2_seq:np.float64(1.0062491311691701) - actor/pg_loss:np.float64(1.9038328900933266e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016279506189675885) - actor/ppo_kl:np.float64(4.644049759150448e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20593710616230965) - perf/mfu/actor:np.float64(0.13613751711674105) - perf/max_memory_allocated_gb:np.float64(132.26526165008545) - perf/max_memory_reserved_gb:np.float64(144.970703125) - perf/cpu_memory_used_gb:np.float64(105.93215942382812) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6901785714285714) - val-aux/sciknoweval/reward/std@16:np.float64(0.21497342883717385) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7771380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17605374457438094) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6025857142857143) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18719434569533389) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6898333333333334) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21523251642703473) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8483380952380953) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.128790863116223) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5304333333333333) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14423738321883778) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7146857142857143) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17575431812173817) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8984857142857143) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08310585797508012) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47553809523809526) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10546043434417916) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7290142857142858) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13381993498072356) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9295428571428572) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.046578487700934236) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4326952380952381) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07212858795272402) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7341666666666667) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10114866635456188) - val-aux/sciknoweval/score/mean@16:np.float64(0.6901785714285714) - val-aux/sciknoweval/score/std@16:np.float64(0.21497342883717385) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7771380952380953) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17605374457438094) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6025857142857143) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.18719434569533389) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6898333333333334) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21523251642703473) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8483380952380953) - val-aux/sciknoweval/score/best@4/std:np.float64(0.128790863116223) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5304333333333333) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14423738321883778) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7146857142857143) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17575431812173817) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8984857142857143) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08310585797508012) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.47553809523809526) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10546043434417916) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7290142857142858) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13381993498072356) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9295428571428572) - val-aux/sciknoweval/score/best@16/std:np.float64(0.046578487700934236) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4326952380952381) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07212858795272402) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7341666666666667) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10114866635456188) - val-core/sciknoweval/acc/mean@16:np.float64(0.6901785714285714) - val-aux/sciknoweval/acc/std@16:np.float64(0.21497342883717385) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7771380952380953) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17605374457438094) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6025857142857143) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.18719434569533389) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6898333333333334) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21523251642703473) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8483380952380953) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.128790863116223) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5304333333333333) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14423738321883778) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7146857142857143) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17575431812173817) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8984857142857143) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08310585797508012) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.47553809523809526) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10546043434417916) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7290142857142858) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13381993498072356) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9295428571428572) - val-core/sciknoweval/acc/best@16/std:np.float64(0.046578487700934236) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4326952380952381) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07212858795272402) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7341666666666667) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10114866635456188) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9645833333333333) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.08812713073175192) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9939428571428572) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.03432708478331659) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9356666666666668) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.11103403939640785) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9648952380952381) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.08777577522757704) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9996333333333334) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.004692755954297982) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.890790476190476) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12713323596686327) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9825095238095238) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.06141628384478388) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8332238095238094) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.12414222258147589) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9934761904761904) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.03401010083743055) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7765571428571428) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0965096914106855) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9985285714285715) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.011901839678247213) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:1 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0025611198507249355 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0025611198507249355 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:832.765625 - response_length/max:1661.0 - response_length/min:260.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:832.765625 - response_length_non_aborted/max:1661.0 - response_length_non_aborted/min:260.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.15625 - prompt_length/max:625.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019051693379878998 - timing_s/agent_loop/generate_sequences/min:np.float64(3.3696037754416466) - timing_s/agent_loop/generate_sequences/max:np.float64(15.491487443447113) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.964753172069322) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.491487443447113) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:255 - timing_s/agent_loop/slowest/response_length:1661 - timing_s/gen:22.220765927806497 - timing_s/reward:0.08236124739050865 - timing_s/old_log_prob:3.2945325896143913 - timing_s/adv:0.11797017231583595 - timing_s/update_actor:13.748360374942422 - timing_s/step:39.5498822350055 - timing_s/testing:247.1615258976817 - timing_s/save_checkpoint:12.889757573604584 - timing_s/stop_profile:6.954558193683624e-05 - timing_per_token_ms/adv:0.00040819563852346666 - timing_per_token_ms/gen:0.1042308475514874 - timing_per_token_ms/update_actor:0.047571522798793175 - perf/total_num_tokens:289004 - perf/time_per_step:39.5498822350055 - perf/throughput:913.4161205674948 (TaskRunner pid=2224264) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2224264) "np.float64(0.6901785714285714), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2224264) "np.float64(0.21497342883717385), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2224264) "np.float64(0.7771380952380953), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2224264) "np.float64(0.17605374457438094), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2224264) "np.float64(0.6025857142857143), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2224264) "np.float64(0.18719434569533389), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2224264) "np.float64(0.6898333333333334), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2224264) "np.float64(0.21523251642703473), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2224264) "np.float64(0.8483380952380953), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2224264) "np.float64(0.128790863116223), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2224264) "np.float64(0.5304333333333333), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2224264) "np.float64(0.14423738321883778), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2224264) "np.float64(0.7146857142857143), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2224264) "np.float64(0.17575431812173817), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2224264) "np.float64(0.8984857142857143), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2224264) "np.float64(0.08310585797508012), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2224264) "np.float64(0.47553809523809526), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2224264) "np.float64(0.10546043434417916), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2224264) "np.float64(0.7290142857142858), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2224264) "np.float64(0.13381993498072356), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2224264) "np.float64(0.9295428571428572), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2224264) 'np.float64(0.046578487700934236), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.4326952380952381), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.07212858795272402), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7341666666666667), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.10114866635456188), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/mean@16': np.float64(0.6901785714285714), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/std@16': np.float64(0.21497342883717385), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.7771380952380953), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.17605374457438094), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.6025857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.18719434569533389), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.6898333333333334), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.21523251642703473), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.8483380952380953), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.128790863116223), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.5304333333333333), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.14423738321883778), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.7146857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.17575431812173817), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8984857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.08310585797508012), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.47553809523809526), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.10546043434417916), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.7290142857142858), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.13381993498072356), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.9295428571428572), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.046578487700934236), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.4326952380952381), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.07212858795272402), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7341666666666667), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.10114866635456188), " (TaskRunner pid=2224264) "'val-core/sciknoweval/acc/mean@16': np.float64(0.6901785714285714), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/std@16': np.float64(0.21497342883717385), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.7771380952380953), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.17605374457438094), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.6025857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.18719434569533389), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.6898333333333334), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.21523251642703473), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.8483380952380953), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.128790863116223), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.5304333333333333), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.14423738321883778), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.7146857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.17575431812173817), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8984857142857143), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.08310585797508012), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.47553809523809526), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.10546043434417916), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.7290142857142858), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.13381993498072356), " (TaskRunner pid=2224264) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.9295428571428572), " (TaskRunner pid=2224264) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.046578487700934236), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.4326952380952381), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.07212858795272402), " (TaskRunner pid=2224264) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7341666666666667), " (TaskRunner pid=2224264) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.10114866635456188), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/mean@16': " (TaskRunner pid=2224264) 'np.float64(0.9645833333333333), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2224264) 'np.float64(0.08812713073175192), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2224264) 'np.float64(0.9939428571428572), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2224264) 'np.float64(0.03432708478331659), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2224264) 'np.float64(0.9356666666666668), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2224264) 'np.float64(0.11103403939640785), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': " (TaskRunner pid=2224264) 'np.float64(0.9648952380952381), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2224264) 'np.float64(0.08777577522757704), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@4/mean': " (TaskRunner pid=2224264) 'np.float64(0.9996333333333334), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2224264) 'np.float64(0.004692755954297982), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2224264) 'np.float64(0.890790476190476), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2224264) 'np.float64(0.12713323596686327), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2224264) 'np.float64(0.9825095238095238), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2224264) 'np.float64(0.06141628384478388), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2224264) 'np.float64(0.8332238095238094), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2224264) 'np.float64(0.12414222258147589), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2224264) 'np.float64(0.9934761904761904), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2224264) 'np.float64(0.03401010083743055), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2224264) 'np.float64(0.7765571428571428), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2224264) 'np.float64(0.0965096914106855), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2224264) 'np.float64(0.9985285714285715), ' (TaskRunner pid=2224264) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2224264) "np.float64(0.011901839678247213), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2224264) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2224264) 'np.float64(2.0)}') (TaskRunner pid=2224264) Training Progress: 100%|██████████| 100/100 [1:42:46<00:00, 120.29s/it] Training Progress: 100%|██████████| 100/100 [1:42:46<00:00, 61.66s/it] (TaskRunner pid=2224264) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2224264) wandb: uploading output.log (TaskRunner pid=2224264) wandb: uploading history steps 98-99, summary, console lines 1082-1211 (TaskRunner pid=2224264) wandb: (TaskRunner pid=2224264) wandb: Run history: (TaskRunner pid=2224264) wandb: actor/entropy ▁▁▂▃▄▄▅▅▅▅▅▅▅▆▆▆▆▆▆▄▆▅▇▇▆▆█▆▆▆▆▆▆▇▆▅▅▆▅▆ (TaskRunner pid=2224264) wandb: actor/grad_norm █▇▇█▅▅▆▇▅▆▅▄▅▆▄▄▅▄▃▄▃▄▃▂▃▅▄▃▂▂▂▃▁▃▁▂▁▃▂▂ (TaskRunner pid=2224264) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2224264) wandb: actor/lr ▁▃▆█████████████████████████████████████ (TaskRunner pid=2224264) wandb: actor/pg_clipfrac ▇▃██▆▆▆▄▃▄▃▅▅▄▅▆▆▄▅▄▃▅▄▇▃▇▆▃▃▁▄▃▄▆▃▆▄▃▂▃ (TaskRunner pid=2224264) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▅▁▁▁▁▁▁▁▁▁▅▁▁▁▁█▁▁▁▁ (TaskRunner pid=2224264) wandb: actor/pg_loss ▂▂▅▆▂▃▃█▂▄▃▁▄▆▅▄▇▆▄▅▃▆▄▃▄▂▅▄▄▄▅▅▅▆▃▄▅▄▄▄ (TaskRunner pid=2224264) wandb: actor/ppo_kl ▆▆▁▄▄▄▆▇▅▄▇▅▃▆▇▅▄▄█▆▃▃▄▅▅▄▇▅▁▄▅▆▇▄▄▃▂▇▄▅ (TaskRunner pid=2224264) wandb: critic/advantages/max ███████████████████████████████▁████████ (TaskRunner pid=2224264) wandb: critic/advantages/mean ▃▄▄▆▄▇▆▄▆▄▃▂▁▃▄▁▄▄▅▅▄▂▂▅▄▃▄█▆▇▇▆▇▃▅▄▃▁▇▆ (TaskRunner pid=2224264) wandb: +204 ... (TaskRunner pid=2224264) wandb: (TaskRunner pid=2224264) wandb: Run summary: (TaskRunner pid=2224264) wandb: actor/entropy 0.75331 (TaskRunner pid=2224264) wandb: actor/grad_norm 0.20594 (TaskRunner pid=2224264) wandb: actor/kl_loss 0 (TaskRunner pid=2224264) wandb: actor/lr 0.0 (TaskRunner pid=2224264) wandb: actor/pg_clipfrac 0.00016 (TaskRunner pid=2224264) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2224264) wandb: actor/pg_loss 2e-05 (TaskRunner pid=2224264) wandb: actor/ppo_kl 5e-05 (TaskRunner pid=2224264) wandb: critic/advantages/max 0.875 (TaskRunner pid=2224264) wandb: critic/advantages/mean 0.00256 (TaskRunner pid=2224264) wandb: +204 ... (TaskRunner pid=2224264) wandb: (TaskRunner pid=2224264) wandb: 🚀 View run qwen3gen-chemistry-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/ffovp19e (TaskRunner pid=2224264) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2224264) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2224264) wandb: Find logs at: ./wandb/run-20260702_235814-ffovp19e/logs (TaskRunner pid=2224264) Exception ignored in atexit callback: .teardown_atexit at 0x7797755e3880> (TaskRunner pid=2224264) Traceback (most recent call last): (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2224264) conn.teardown(hooks.exit_code) (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2224264) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2224264) return future.result() (TaskRunner pid=2224264) ^^^^^^^^^^^^^^^ (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2224264) return self.__get_result() (TaskRunner pid=2224264) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2224264) raise self._exception (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2224264) return await fn() (TaskRunner pid=2224264) ^^^^^^^^^^ (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2224264) await self._client.close() (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2224264) await self._writer.wait_closed() (TaskRunner pid=2224264) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2224264) await self._protocol._get_close_waiter(self) (TaskRunner pid=2224264) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-03 01:41:07] Finished Qwen/Qwen3-8B chemistry grpo. [2026-07-03 01:41:07] Starting Qwen/Qwen3-8B chemistry rlsd_tr. Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: chemistry Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet Ray tmp: /tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B 2026-07-03 01:41:09,351 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/chemistry Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 01:41:19,207 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2255397) TaskRunner hostname: mole-workspace-rw, PID: 2255397 (TaskRunner pid=2255397) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2255397) 'calculate_entropy': False, (TaskRunner pid=2255397) 'calculate_sum_pi_squared': False, (TaskRunner pid=2255397) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2255397) 'async_save': False, (TaskRunner pid=2255397) 'load_contents': ['model', (TaskRunner pid=2255397) 'optimizer', (TaskRunner pid=2255397) 'extra'], (TaskRunner pid=2255397) 'save_contents': ['model', (TaskRunner pid=2255397) 'optimizer', (TaskRunner pid=2255397) 'extra']}, (TaskRunner pid=2255397) 'clip_ratio': 0.2, (TaskRunner pid=2255397) 'clip_ratio_c': 3.0, (TaskRunner pid=2255397) 'clip_ratio_high': 0.28, (TaskRunner pid=2255397) 'clip_ratio_low': 0.2, (TaskRunner pid=2255397) 'data_loader_seed': 42, (TaskRunner pid=2255397) 'entropy_checkpointing': False, (TaskRunner pid=2255397) 'entropy_coeff': 0, (TaskRunner pid=2255397) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2255397) 'freeze_vision_tower': False, (TaskRunner pid=2255397) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2255397) 'dtype': 'bfloat16', (TaskRunner pid=2255397) 'entropy_checkpointing': False, (TaskRunner pid=2255397) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2255397) 'forward_only': False, (TaskRunner pid=2255397) 'forward_prefetch': False, (TaskRunner pid=2255397) 'fsdp_size': -1, (TaskRunner pid=2255397) 'full_determinism': False, (TaskRunner pid=2255397) 'model_dtype': 'fp32', (TaskRunner pid=2255397) 'offload_policy': False, (TaskRunner pid=2255397) 'optimizer_offload': False, (TaskRunner pid=2255397) 'param_offload': False, (TaskRunner pid=2255397) 'reshard_after_forward': True, (TaskRunner pid=2255397) 'seed': 42, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_orig_params': False, (TaskRunner pid=2255397) 'use_torch_compile': True, (TaskRunner pid=2255397) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2255397) 'grad_clip': 1.0, (TaskRunner pid=2255397) 'kl_loss_coef': 0.001, (TaskRunner pid=2255397) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2255397) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2255397) 'loss_scale_factor': None, (TaskRunner pid=2255397) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2255397) 'betas': [0.9, 0.999], (TaskRunner pid=2255397) 'clip_grad': 1.0, (TaskRunner pid=2255397) 'lr': 1e-06, (TaskRunner pid=2255397) 'lr_scheduler_type': 'constant', (TaskRunner pid=2255397) 'lr_warmup_steps': 10, (TaskRunner pid=2255397) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2255397) 'min_lr_ratio': 0.0, (TaskRunner pid=2255397) 'num_cycles': 0.5, (TaskRunner pid=2255397) 'optimizer': 'AdamW', (TaskRunner pid=2255397) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2255397) 'override_optimizer_config': None, (TaskRunner pid=2255397) 'total_training_steps': -1, (TaskRunner pid=2255397) 'warmup_style': None, (TaskRunner pid=2255397) 'weight_decay': 0.01}, (TaskRunner pid=2255397) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2255397) 'clip_cov_lb': 1.0, (TaskRunner pid=2255397) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2255397) 'clip_cov_ub': 5.0, (TaskRunner pid=2255397) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2255397) 'loss_mode': 'rlsd', (TaskRunner pid=2255397) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2255397) 'ppo_epochs': 1, (TaskRunner pid=2255397) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2255397) 'ppo_micro_batch_size': None, (TaskRunner pid=2255397) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2255397) 'ppo_mini_batch_size': 8, (TaskRunner pid=2255397) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'all_ranks': False, (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'ranks': [], (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'tool': None, (TaskRunner pid=2255397) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2255397) 'analysis': True, (TaskRunner pid=2255397) 'contents': [], (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'level': 'level0'}, (TaskRunner pid=2255397) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'discrete': False}, (TaskRunner pid=2255397) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2255397) 'step_end': None, (TaskRunner pid=2255397) 'step_start': 0}, (TaskRunner pid=2255397) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2255397) 'rollout_n': 8, (TaskRunner pid=2255397) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2255397) 'mode': 'disabled', (TaskRunner pid=2255397) 'record_file': None, (TaskRunner pid=2255397) 'replay_file': None}, (TaskRunner pid=2255397) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2255397) 'alpha': 0.5, (TaskRunner pid=2255397) 'distillation_add_tail': True, (TaskRunner pid=2255397) 'distillation_topk': 100, (TaskRunner pid=2255397) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2255397) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2255397) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'loss_weight': 0.0, (TaskRunner pid=2255397) 'mask': 'all'}, (TaskRunner pid=2255397) 'feedback_template': '\n' (TaskRunner pid=2255397) 'The ' (TaskRunner pid=2255397) 'following ' (TaskRunner pid=2255397) 'is ' (TaskRunner pid=2255397) 'feedback ' (TaskRunner pid=2255397) 'from ' (TaskRunner pid=2255397) 'your ' (TaskRunner pid=2255397) 'unsuccessful ' (TaskRunner pid=2255397) 'earlier ' (TaskRunner pid=2255397) 'attempt:\n' (TaskRunner pid=2255397) '\n' (TaskRunner pid=2255397) '{feedback_raw}', (TaskRunner pid=2255397) 'full_logit_distillation': True, (TaskRunner pid=2255397) 'include_environment_feedback': True, (TaskRunner pid=2255397) 'is_clip': 2, (TaskRunner pid=2255397) 'max_reprompt_len': 10240, (TaskRunner pid=2255397) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2255397) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2255397) '\n' (TaskRunner pid=2255397) 'Correctly ' (TaskRunner pid=2255397) 'solve ' (TaskRunner pid=2255397) 'the ' (TaskRunner pid=2255397) 'original ' (TaskRunner pid=2255397) 'question.', (TaskRunner pid=2255397) 'reprompt_truncation': 'right', (TaskRunner pid=2255397) 'solution_template': '\n' (TaskRunner pid=2255397) 'Correct ' (TaskRunner pid=2255397) 'solution:\n' (TaskRunner pid=2255397) '\n' (TaskRunner pid=2255397) '{successful_previous_attempt}', (TaskRunner pid=2255397) 'srpo_dynamic_weighting': False, (TaskRunner pid=2255397) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2255397) 'success_reward_threshold': 0.5, (TaskRunner pid=2255397) 'teacher_regularization': 'trust-region', (TaskRunner pid=2255397) 'teacher_update_rate': 0.1, (TaskRunner pid=2255397) 'token_reweight_decay_steps': 0, (TaskRunner pid=2255397) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2255397) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2255397) 'shuffle': False, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2255397) 'tau_neg': 1.05, (TaskRunner pid=2255397) 'tau_pos': 1.0, (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_dynamic_bsz': False, (TaskRunner pid=2255397) 'use_fused_kernels': False, (TaskRunner pid=2255397) 'use_kl_loss': False, (TaskRunner pid=2255397) 'use_prefix_grouper': False, (TaskRunner pid=2255397) 'use_remove_padding': True, (TaskRunner pid=2255397) 'use_torch_compile': True}, (TaskRunner pid=2255397) 'hybrid_engine': True, (TaskRunner pid=2255397) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2255397) 'custom_chat_template': None, (TaskRunner pid=2255397) 'enable_activation_offload': False, (TaskRunner pid=2255397) 'enable_gradient_checkpointing': True, (TaskRunner pid=2255397) 'exclude_modules': None, (TaskRunner pid=2255397) 'external_lib': None, (TaskRunner pid=2255397) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2255397) 'hf_config_path': None, (TaskRunner pid=2255397) 'lora_adapter_path': None, (TaskRunner pid=2255397) 'lora_alpha': 16, (TaskRunner pid=2255397) 'lora_rank': 0, (TaskRunner pid=2255397) 'override_config': {}, (TaskRunner pid=2255397) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2255397) 'target_modules': 'all-linear', (TaskRunner pid=2255397) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2255397) 'num_shards': 4}, (TaskRunner pid=2255397) 'tokenizer_path': None, (TaskRunner pid=2255397) 'trust_remote_code': True, (TaskRunner pid=2255397) 'use_fused_kernels': False, (TaskRunner pid=2255397) 'use_liger': False, (TaskRunner pid=2255397) 'use_remove_padding': True, (TaskRunner pid=2255397) 'use_shm': False}, (TaskRunner pid=2255397) 'nccl_timeout': 600, (TaskRunner pid=2255397) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2255397) 'entropy_checkpointing': False, (TaskRunner pid=2255397) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2255397) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2255397) 'dtype': 'bfloat16', (TaskRunner pid=2255397) 'entropy_checkpointing': False, (TaskRunner pid=2255397) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2255397) 'forward_only': True, (TaskRunner pid=2255397) 'forward_prefetch': False, (TaskRunner pid=2255397) 'fsdp_size': -1, (TaskRunner pid=2255397) 'full_determinism': False, (TaskRunner pid=2255397) 'model_dtype': 'fp32', (TaskRunner pid=2255397) 'offload_policy': False, (TaskRunner pid=2255397) 'optimizer_offload': False, (TaskRunner pid=2255397) 'param_offload': False, (TaskRunner pid=2255397) 'reshard_after_forward': True, (TaskRunner pid=2255397) 'seed': 42, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_orig_params': False, (TaskRunner pid=2255397) 'use_torch_compile': True, (TaskRunner pid=2255397) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2255397) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2255397) 'log_prob_micro_batch_size': None, (TaskRunner pid=2255397) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2255397) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2255397) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'all_ranks': False, (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'ranks': [], (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'tool': None, (TaskRunner pid=2255397) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2255397) 'analysis': True, (TaskRunner pid=2255397) 'contents': [], (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'level': 'level0'}, (TaskRunner pid=2255397) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'discrete': False}, (TaskRunner pid=2255397) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2255397) 'step_end': None, (TaskRunner pid=2255397) 'step_start': 0}, (TaskRunner pid=2255397) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2255397) 'rollout_n': 8, (TaskRunner pid=2255397) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2255397) 'mode': 'disabled', (TaskRunner pid=2255397) 'record_file': None, (TaskRunner pid=2255397) 'replay_file': None}, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_torch_compile': True}, (TaskRunner pid=2255397) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2255397) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2255397) 'agent_loop_config_path': None, (TaskRunner pid=2255397) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2255397) 'name': None, (TaskRunner pid=2255397) 'path': None}, (TaskRunner pid=2255397) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2255397) 'num_workers': 8}, (TaskRunner pid=2255397) 'calculate_log_probs': True, (TaskRunner pid=2255397) 'cudagraph_capture_sizes': None, (TaskRunner pid=2255397) 'data_parallel_size': 1, (TaskRunner pid=2255397) 'disable_log_stats': True, (TaskRunner pid=2255397) 'do_sample': True, (TaskRunner pid=2255397) 'dtype': 'bfloat16', (TaskRunner pid=2255397) 'enable_chunked_prefill': True, (TaskRunner pid=2255397) 'enable_prefix_caching': True, (TaskRunner pid=2255397) 'enable_rollout_routing_replay': False, (TaskRunner pid=2255397) 'enforce_eager': False, (TaskRunner pid=2255397) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2255397) 'expert_parallel_size': 1, (TaskRunner pid=2255397) 'free_cache_engine': True, (TaskRunner pid=2255397) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2255397) 'ignore_eos': False, (TaskRunner pid=2255397) 'layered_summon': False, (TaskRunner pid=2255397) 'load_format': 'dummy', (TaskRunner pid=2255397) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2255397) 'log_prob_micro_batch_size': None, (TaskRunner pid=2255397) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2255397) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2255397) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2255397) 'max_model_len': 10240, (TaskRunner pid=2255397) 'max_num_batched_tokens': 10240, (TaskRunner pid=2255397) 'max_num_seqs': 1024, (TaskRunner pid=2255397) 'mode': 'async', (TaskRunner pid=2255397) 'multi_stage_wake_up': False, (TaskRunner pid=2255397) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'format': 'hermes', (TaskRunner pid=2255397) 'interaction_config_path': None, (TaskRunner pid=2255397) 'max_assistant_turns': None, (TaskRunner pid=2255397) 'max_parallel_calls': 1, (TaskRunner pid=2255397) 'max_tool_response_length': 256, (TaskRunner pid=2255397) 'max_user_turns': None, (TaskRunner pid=2255397) 'num_repeat_rollouts': None, (TaskRunner pid=2255397) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2255397) 'tool_config_path': None, (TaskRunner pid=2255397) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2255397) 'use_inference_chat_template': False}, (TaskRunner pid=2255397) 'n': 8, (TaskRunner pid=2255397) 'name': 'vllm', (TaskRunner pid=2255397) 'over_sample_rate': 0, (TaskRunner pid=2255397) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2255397) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'all_ranks': False, (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'ranks': [], (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'tool': None, (TaskRunner pid=2255397) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2255397) 'analysis': True, (TaskRunner pid=2255397) 'contents': [], (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'level': 'level0'}, (TaskRunner pid=2255397) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'discrete': False}, (TaskRunner pid=2255397) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2255397) 'step_end': None, (TaskRunner pid=2255397) 'step_start': 0}, (TaskRunner pid=2255397) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2255397) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2255397) 'port': 9090, (TaskRunner pid=2255397) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2255397) 'prompt_length': 2048, (TaskRunner pid=2255397) 'quantization': None, (TaskRunner pid=2255397) 'quantization_config_file': None, (TaskRunner pid=2255397) 'response_length': 8192, (TaskRunner pid=2255397) 'scheduling_policy': 'fcfs', (TaskRunner pid=2255397) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2255397) 'skip_rollout': False, (TaskRunner pid=2255397) 'skip_tokenizer_init': True, (TaskRunner pid=2255397) 'temperature': 1.0, (TaskRunner pid=2255397) 'tensor_model_parallel_size': 2, (TaskRunner pid=2255397) 'top_k': -1, (TaskRunner pid=2255397) 'top_p': 1.0, (TaskRunner pid=2255397) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2255397) 'backend': None, (TaskRunner pid=2255397) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2255397) 'token2text': False}, (TaskRunner pid=2255397) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2255397) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2255397) 'do_sample': True, (TaskRunner pid=2255397) 'n': 16, (TaskRunner pid=2255397) 'temperature': 0.6, (TaskRunner pid=2255397) 'top_k': -1, (TaskRunner pid=2255397) 'top_p': 0.95}}}, (TaskRunner pid=2255397) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2255397) 'adv_estimator': 'grpo', (TaskRunner pid=2255397) 'gamma': 1.0, (TaskRunner pid=2255397) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2255397) 'horizon': 10000, (TaskRunner pid=2255397) 'kl_coef': 0.001, (TaskRunner pid=2255397) 'target_kl': 0.1, (TaskRunner pid=2255397) 'type': 'fixed'}, (TaskRunner pid=2255397) 'kl_penalty': 'kl', (TaskRunner pid=2255397) 'lam': 1.0, (TaskRunner pid=2255397) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2255397) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2255397) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2255397) 'loss_type': 'ppo_clip', (TaskRunner pid=2255397) 'rollout_is': 'token', (TaskRunner pid=2255397) 'rollout_is_batch_normalize': False, (TaskRunner pid=2255397) 'rollout_is_threshold': 2.0, (TaskRunner pid=2255397) 'rollout_rs': None, (TaskRunner pid=2255397) 'rollout_rs_threshold': None}, (TaskRunner pid=2255397) 'use_kl_in_reward': False, (TaskRunner pid=2255397) 'use_pf_ppo': False}, (TaskRunner pid=2255397) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2255397) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2255397) 'async_save': False, (TaskRunner pid=2255397) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2255397) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2255397) 'cliprange_value': 0.5, (TaskRunner pid=2255397) 'data_loader_seed': 42, (TaskRunner pid=2255397) 'enable': None, (TaskRunner pid=2255397) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2255397) 'forward_micro_batch_size': None, (TaskRunner pid=2255397) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2255397) 'grad_clip': 1.0, (TaskRunner pid=2255397) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2255397) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2255397) 'enable_activation_offload': False, (TaskRunner pid=2255397) 'enable_gradient_checkpointing': True, (TaskRunner pid=2255397) 'external_lib': None, (TaskRunner pid=2255397) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2255397) 'dtype': 'bfloat16', (TaskRunner pid=2255397) 'entropy_checkpointing': False, (TaskRunner pid=2255397) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2255397) 'forward_only': False, (TaskRunner pid=2255397) 'forward_prefetch': False, (TaskRunner pid=2255397) 'fsdp_size': -1, (TaskRunner pid=2255397) 'full_determinism': False, (TaskRunner pid=2255397) 'model_dtype': 'fp32', (TaskRunner pid=2255397) 'offload_policy': False, (TaskRunner pid=2255397) 'optimizer_offload': False, (TaskRunner pid=2255397) 'param_offload': False, (TaskRunner pid=2255397) 'reshard_after_forward': True, (TaskRunner pid=2255397) 'seed': 42, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_orig_params': False, (TaskRunner pid=2255397) 'use_torch_compile': True, (TaskRunner pid=2255397) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2255397) 'lora_alpha': 16, (TaskRunner pid=2255397) 'lora_rank': 0, (TaskRunner pid=2255397) 'override_config': {}, (TaskRunner pid=2255397) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2255397) 'target_modules': 'all-linear', (TaskRunner pid=2255397) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2255397) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2255397) 'trust_remote_code': True, (TaskRunner pid=2255397) 'use_remove_padding': False, (TaskRunner pid=2255397) 'use_shm': False}, (TaskRunner pid=2255397) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2255397) 'betas': [0.9, 0.999], (TaskRunner pid=2255397) 'clip_grad': 1.0, (TaskRunner pid=2255397) 'lr': 1e-05, (TaskRunner pid=2255397) 'lr_scheduler_type': 'constant', (TaskRunner pid=2255397) 'lr_warmup_steps': -1, (TaskRunner pid=2255397) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2255397) 'min_lr_ratio': 0.0, (TaskRunner pid=2255397) 'num_cycles': 0.5, (TaskRunner pid=2255397) 'optimizer': 'AdamW', (TaskRunner pid=2255397) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2255397) 'override_optimizer_config': None, (TaskRunner pid=2255397) 'total_training_steps': -1, (TaskRunner pid=2255397) 'warmup_style': None, (TaskRunner pid=2255397) 'weight_decay': 0.01}, (TaskRunner pid=2255397) 'ppo_epochs': 1, (TaskRunner pid=2255397) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2255397) 'ppo_micro_batch_size': None, (TaskRunner pid=2255397) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2255397) 'ppo_mini_batch_size': 8, (TaskRunner pid=2255397) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'all_ranks': False, (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'ranks': [], (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'tool': None, (TaskRunner pid=2255397) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2255397) 'analysis': True, (TaskRunner pid=2255397) 'contents': [], (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'level': 'level0'}, (TaskRunner pid=2255397) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'discrete': False}, (TaskRunner pid=2255397) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2255397) 'step_end': None, (TaskRunner pid=2255397) 'step_start': 0}, (TaskRunner pid=2255397) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2255397) 'rollout_n': 8, (TaskRunner pid=2255397) 'shuffle': False, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_dynamic_bsz': False}, (TaskRunner pid=2255397) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2255397) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2255397) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2255397) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2255397) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2255397) 'dataloader_num_workers': 8, (TaskRunner pid=2255397) 'filter_overlong_prompts': True, (TaskRunner pid=2255397) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2255397) 'image_key': 'images', (TaskRunner pid=2255397) 'image_patch_size': 14, (TaskRunner pid=2255397) 'max_prompt_length': 2048, (TaskRunner pid=2255397) 'max_response_length': 8192, (TaskRunner pid=2255397) 'prompt_key': 'prompt', (TaskRunner pid=2255397) 'return_full_prompt': False, (TaskRunner pid=2255397) 'return_multi_modal_inputs': True, (TaskRunner pid=2255397) 'return_raw_chat': True, (TaskRunner pid=2255397) 'return_raw_input_ids': False, (TaskRunner pid=2255397) 'reward_fn_key': 'data_source', (TaskRunner pid=2255397) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2255397) 'seed': None, (TaskRunner pid=2255397) 'shuffle': True, (TaskRunner pid=2255397) 'tokenizer': None, (TaskRunner pid=2255397) 'tool_config_path': None, (TaskRunner pid=2255397) 'train_batch_size': 32, (TaskRunner pid=2255397) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'], (TaskRunner pid=2255397) 'train_max_samples': 3200, (TaskRunner pid=2255397) 'truncation': 'error', (TaskRunner pid=2255397) 'trust_remote_code': True, (TaskRunner pid=2255397) 'use_shm': False, (TaskRunner pid=2255397) 'val_batch_size': None, (TaskRunner pid=2255397) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'], (TaskRunner pid=2255397) 'val_max_samples': -1, (TaskRunner pid=2255397) 'validation_shuffle': False, (TaskRunner pid=2255397) 'video_key': 'videos'}, (TaskRunner pid=2255397) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2255397) 'cuda-memory-usage': 'true', (TaskRunner pid=2255397) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2255397) 'capture-range-end': None, (TaskRunner pid=2255397) 'cuda-graph-trace': 'graph', (TaskRunner pid=2255397) 'cuda-memory-usage': 'true', (TaskRunner pid=2255397) 'kill': 'none', (TaskRunner pid=2255397) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2255397) 'torch_memory': {'context': 'all', (TaskRunner pid=2255397) 'kw_args': {}, (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'stacks': 'all', (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2255397) 'profile_continuous_steps': False, (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'steps': None, (TaskRunner pid=2255397) 'tool': None}, (TaskRunner pid=2255397) 'max_model_len': 10240, (TaskRunner pid=2255397) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_8B', (TaskRunner pid=2255397) 'include_dashboard': False, (TaskRunner pid=2255397) 'num_cpus': None, (TaskRunner pid=2255397) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2255397) 'TASK': 'datasets/sciknoweval/chemistry', (TaskRunner pid=2255397) 'USER': 'root'}}}, (TaskRunner pid=2255397) 'timeline_json_file': None}, (TaskRunner pid=2255397) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2255397) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2255397) 'name': 'custom_reward_manager', (TaskRunner pid=2255397) 'path': None}, (TaskRunner pid=2255397) 'name': 'naive', (TaskRunner pid=2255397) 'source': 'register'}, (TaskRunner pid=2255397) 'reward_model': {'enable': False, (TaskRunner pid=2255397) 'enable_resource_pool': False, (TaskRunner pid=2255397) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2255397) 'launch_reward_fn_async': False, (TaskRunner pid=2255397) 'max_length': None, (TaskRunner pid=2255397) 'micro_batch_size': None, (TaskRunner pid=2255397) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2255397) 'model': {'external_lib': None, (TaskRunner pid=2255397) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2255397) 'forward_prefetch': False, (TaskRunner pid=2255397) 'fsdp_size': -1, (TaskRunner pid=2255397) 'param_offload': False, (TaskRunner pid=2255397) 'reshard_after_forward': True, (TaskRunner pid=2255397) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2255397) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2255397) 'override_config': {}, (TaskRunner pid=2255397) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2255397) 'trust_remote_code': False, (TaskRunner pid=2255397) 'use_fused_kernels': False, (TaskRunner pid=2255397) 'use_remove_padding': False, (TaskRunner pid=2255397) 'use_shm': False}, (TaskRunner pid=2255397) 'n_gpus_per_node': 8, (TaskRunner pid=2255397) 'nnodes': 0, (TaskRunner pid=2255397) 'num_workers': 1, (TaskRunner pid=2255397) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2255397) 'all_ranks': False, (TaskRunner pid=2255397) 'enable': False, (TaskRunner pid=2255397) 'ranks': [], (TaskRunner pid=2255397) 'save_path': 'outputs/profile', (TaskRunner pid=2255397) 'tool': None, (TaskRunner pid=2255397) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2255397) 'analysis': True, (TaskRunner pid=2255397) 'contents': [], (TaskRunner pid=2255397) 'discrete': False, (TaskRunner pid=2255397) 'level': 'level0'}, (TaskRunner pid=2255397) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2255397) 'discrete': False}, (TaskRunner pid=2255397) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2255397) 'step_end': None, (TaskRunner pid=2255397) 'step_start': 0}, (TaskRunner pid=2255397) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2255397) 'stack_depth': 32, (TaskRunner pid=2255397) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2255397) 'reward_loop_class_name': None, (TaskRunner pid=2255397) 'reward_loop_module_path': None, (TaskRunner pid=2255397) 'reward_loop_source': 'register', (TaskRunner pid=2255397) 'reward_manager': 'naive', (TaskRunner pid=2255397) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2255397) 'cudagraph_capture_sizes': None, (TaskRunner pid=2255397) 'data_parallel_size': 1, (TaskRunner pid=2255397) 'disable_log_stats': True, (TaskRunner pid=2255397) 'dtype': 'bfloat16', (TaskRunner pid=2255397) 'enable_chunked_prefill': True, (TaskRunner pid=2255397) 'enable_prefix_caching': True, (TaskRunner pid=2255397) 'enforce_eager': True, (TaskRunner pid=2255397) 'engine_kwargs': {}, (TaskRunner pid=2255397) 'expert_parallel_size': 1, (TaskRunner pid=2255397) 'free_cache_engine': True, (TaskRunner pid=2255397) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2255397) 'limit_images': None, (TaskRunner pid=2255397) 'load_format': 'auto', (TaskRunner pid=2255397) 'max_model_len': None, (TaskRunner pid=2255397) 'max_num_batched_tokens': 8192, (TaskRunner pid=2255397) 'max_num_seqs': 1024, (TaskRunner pid=2255397) 'name': '???', (TaskRunner pid=2255397) 'prompt_length': 2048, (TaskRunner pid=2255397) 'response_length': 2048, (TaskRunner pid=2255397) 'skip_tokenizer_init': False, (TaskRunner pid=2255397) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2255397) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2255397) 'memory_limit_mb': 1024, (TaskRunner pid=2255397) 'url': None}, (TaskRunner pid=2255397) 'strategy': 'fsdp', (TaskRunner pid=2255397) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2255397) 'use_dynamic_bsz': False, (TaskRunner pid=2255397) 'use_reward_loop': False}, (TaskRunner pid=2255397) 'trainer': {'balance_batch': True, (TaskRunner pid=2255397) 'critic_warmup': 0, (TaskRunner pid=2255397) 'default_hdfs_dir': None, (TaskRunner pid=2255397) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2255397) 'del_local_ckpt_after_load': False, (TaskRunner pid=2255397) 'device': 'cuda', (TaskRunner pid=2255397) 'esi_redundant_time': 0, (TaskRunner pid=2255397) 'experiment_name': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2255397) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2255397) 'log_val_generations': 0, (TaskRunner pid=2255397) 'logger': ['console', 'wandb'], (TaskRunner pid=2255397) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2255397) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2255397) 'n_gpus_per_node': 8, (TaskRunner pid=2255397) 'nnodes': 1, (TaskRunner pid=2255397) 'project_name': 'SDPO-root', (TaskRunner pid=2255397) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2255397) 'resume_from_path': None, (TaskRunner pid=2255397) 'resume_mode': 'auto', (TaskRunner pid=2255397) 'rollout_data_dir': None, (TaskRunner pid=2255397) 'save_freq': 10, (TaskRunner pid=2255397) 'test_freq': 10, (TaskRunner pid=2255397) 'total_epochs': 30, (TaskRunner pid=2255397) 'total_training_steps': 100, (TaskRunner pid=2255397) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2255397) 'val_before_train': False, (TaskRunner pid=2255397) 'val_only': False, (TaskRunner pid=2255397) 'validation_data_dir': None}, (TaskRunner pid=2255397) 'transfer_queue': {'enable': False}, (TaskRunner pid=2255397) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry', (TaskRunner pid=2255397) 'dir': '/users/root/SDPO', (TaskRunner pid=2255397) 'log_dir': '/users/root/output', (TaskRunner pid=2255397) 'task': 'datasets/sciknoweval/chemistry'}} (TaskRunner pid=2255397) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2255397) use_critic=need_critic(config), (TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2255397) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2255397) Using dataset class: RLHFDataset (TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2255397) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2255397) dataset len: 1890 (TaskRunner pid=2255397) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2255397) WARNING:2026-07-03 01:41:28,094:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2255397) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/1890 [00:00 (TaskRunner pid=2255397) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2255397) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2255397) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2255397) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2255397) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2255397) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 276.01 examples/s] (TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2255397) self.use_critic = need_critic(self.config) (WorkerDict pid=2255793) [W703 01:41:37.908542315 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:41933 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2255788) [Gloo] Rank 1 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2255789) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2255789) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2255792) [W703 01:41:38.533011673 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:41933 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2255787) Model config after override: Qwen3Config { (WorkerDict pid=2255787) "architectures": [ (WorkerDict pid=2255787) "Qwen3ForCausalLM" (WorkerDict pid=2255787) ], (WorkerDict pid=2255787) "attention_bias": false, (WorkerDict pid=2255787) "attention_dropout": 0.0, (WorkerDict pid=2255787) "dtype": "bfloat16", (WorkerDict pid=2255787) "eos_token_id": 151645, (WorkerDict pid=2255787) "head_dim": 128, (WorkerDict pid=2255787) "hidden_act": "silu", (WorkerDict pid=2255787) "hidden_size": 4096, (WorkerDict pid=2255787) "initializer_range": 0.02, (WorkerDict pid=2255787) "intermediate_size": 12288, (WorkerDict pid=2255787) "layer_types": [ (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention" (WorkerDict pid=2255787) ], (WorkerDict pid=2255787) "max_position_embeddings": 40960, (WorkerDict pid=2255787) "max_window_layers": 36, (WorkerDict pid=2255787) "model_type": "qwen3", (WorkerDict pid=2255787) "num_attention_heads": 32, (WorkerDict pid=2255787) "num_hidden_layers": 36, (WorkerDict pid=2255787) "num_key_value_heads": 8, (WorkerDict pid=2255787) "pad_token_id": 151643, (WorkerDict pid=2255787) "rms_norm_eps": 1e-06, (WorkerDict pid=2255787) "rope_scaling": null, (WorkerDict pid=2255787) "rope_theta": 1000000, (WorkerDict pid=2255787) "sliding_window": null, (WorkerDict pid=2255787) "tie_word_embeddings": false, (WorkerDict pid=2255787) "transformers_version": "4.57.1", (WorkerDict pid=2255787) "use_cache": true, (WorkerDict pid=2255787) "use_sliding_window": false, (WorkerDict pid=2255787) "vocab_size": 151936 (WorkerDict pid=2255787) } (WorkerDict pid=2255787) (WorkerDict pid=2255794) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2255794) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2255794) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2255789) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2255787) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2255788) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2255788) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2255790) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.90s/it] [repeated 15x across cluster] (WorkerDict pid=2255790) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.20s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.60s/it] [repeated 7x across cluster] (WorkerDict pid=2255787) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2255790) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2255790) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2255787) Actor use_remove_padding=True (WorkerDict pid=2255787) Actor use_fused_kernels=False (WorkerDict pid=2255787) Actor use_prefix_grouper=False (WorkerDict pid=2255794) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2255794) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2255788) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2255787) reference model: Qwen/Qwen3-8B (WorkerDict pid=2255787) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2255787) warnings.warn( (WorkerDict pid=2255787) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2255787) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2255787) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2255787) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2255787) Model config after override: Qwen3Config { (WorkerDict pid=2255787) "architectures": [ (WorkerDict pid=2255787) "Qwen3ForCausalLM" (WorkerDict pid=2255787) ], (WorkerDict pid=2255787) "attention_bias": false, (WorkerDict pid=2255787) "attention_dropout": 0.0, (WorkerDict pid=2255787) "dtype": "bfloat16", (WorkerDict pid=2255787) "eos_token_id": 151645, (WorkerDict pid=2255787) "head_dim": 128, (WorkerDict pid=2255787) "hidden_act": "silu", (WorkerDict pid=2255787) "hidden_size": 4096, (WorkerDict pid=2255787) "initializer_range": 0.02, (WorkerDict pid=2255787) "intermediate_size": 12288, (WorkerDict pid=2255787) "layer_types": [ (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention", (WorkerDict pid=2255787) "full_attention" (WorkerDict pid=2255787) ], (WorkerDict pid=2255787) "max_position_embeddings": 40960, (WorkerDict pid=2255787) "max_window_layers": 36, (WorkerDict pid=2255787) "model_type": "qwen3", (WorkerDict pid=2255787) "num_attention_heads": 32, (WorkerDict pid=2255787) "num_hidden_layers": 36, (WorkerDict pid=2255787) "num_key_value_heads": 8, (WorkerDict pid=2255787) "pad_token_id": 151643, (WorkerDict pid=2255787) "rms_norm_eps": 1e-06, (WorkerDict pid=2255787) "rope_scaling": null, (WorkerDict pid=2255787) "rope_theta": 1000000, (WorkerDict pid=2255787) "sliding_window": null, (WorkerDict pid=2255787) "tie_word_embeddings": false, (WorkerDict pid=2255787) "transformers_version": "4.57.1", (WorkerDict pid=2255787) "use_cache": true, (WorkerDict pid=2255787) "use_sliding_window": false, (WorkerDict pid=2255787) "vocab_size": 151936 (WorkerDict pid=2255787) } (WorkerDict pid=2255787) (WorkerDict pid=2255792) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (WorkerDict pid=2255787) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2255787) Ref use_remove_padding=True (WorkerDict pid=2255787) Ref use_fused_kernels=False (WorkerDict pid=2255787) Ref use_prefix_grouper=False (WorkerDict pid=2255792) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2255792) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2255397) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2255397) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2255792) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.93s/it] [repeated 15x across cluster] (WorkerDict pid=2255792) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.22s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.62s/it] [repeated 7x across cluster] (vLLMHttpServer pid=2256706) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2256706) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2256707) ['serve', (vLLMHttpServer pid=2256707) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2256707) '--dtype', (vLLMHttpServer pid=2256707) 'bfloat16', (vLLMHttpServer pid=2256707) '--load_format', (vLLMHttpServer pid=2256707) 'dummy', (vLLMHttpServer pid=2256707) '--trust_remote_code', (vLLMHttpServer pid=2256707) '--max_model_len', (vLLMHttpServer pid=2256707) '40960', (vLLMHttpServer pid=2256707) '--max_num_seqs', (vLLMHttpServer pid=2256707) '1024', (vLLMHttpServer pid=2256707) '--enable_chunked_prefill', (vLLMHttpServer pid=2256707) '--max_num_batched_tokens', (vLLMHttpServer pid=2256707) '10240', (vLLMHttpServer pid=2256707) '--enable_prefix_caching', (vLLMHttpServer pid=2256707) '--enable_sleep_mode', (vLLMHttpServer pid=2256707) '--logprobs_mode', (vLLMHttpServer pid=2256707) 'processed_logprobs', (vLLMHttpServer pid=2256707) '--disable_custom_all_reduce', (vLLMHttpServer pid=2256707) '--gpu_memory_utilization', (vLLMHttpServer pid=2256707) '0.8', (vLLMHttpServer pid=2256707) '--disable_log_stats', (vLLMHttpServer pid=2256707) '--tensor_parallel_size', (vLLMHttpServer pid=2256707) '2', (vLLMHttpServer pid=2256707) '--seed', (vLLMHttpServer pid=2256707) '0', (vLLMHttpServer pid=2256707) '--override_generation_config', (vLLMHttpServer pid=2256707) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2256707) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2256707) '--hf_overrides', (vLLMHttpServer pid=2256707) '{}', (vLLMHttpServer pid=2256707) '--scheduling_policy', (vLLMHttpServer pid=2256707) 'fcfs'] (vLLMHttpServer pid=2256707) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2256706) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2256706) WARNING 07-03 01:42:55 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2255788) WARNING 07-03 01:43:03 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2256709) WARNING 07-03 01:42:55 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2255789) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2255793) 2026-07-03 01:43:17,091 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (vLLMHttpServer pid=2256709) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2256709) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2256709) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2256709) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2255793) 2026-07-03 01:43:17,106 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2255787) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to examine the structure of the product and match it with the structure of the reactant provided in the options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product has the following key features: (TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or imidazole-like structure). (TaskRunner pid=2255397) - A substituent "Cc2ccc(Cl)cc2C(F)(F)F", which is a benzene ring substituted with a chlorine atom and a trifluoromethyl group. (TaskRunner pid=2255397) - The overall structure suggests that the reactant must have a similar core structure, with the necessary functional groups for the formation of the final product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Let's analyze the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A**: (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" (TaskRunner pid=2255397) This structure contains a nitrogen-containing ring with the correct substituent "Cc2ccc(Cl)cc2C(F)(F)F". It also has a carbonyl group and a nitrogen atom, which could be part of a condensation or coupling reaction. This matches the structure of the product closely. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B**: (TaskRunner pid=2255397) "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" (TaskRunner pid=2255397) This option introduces a chlorine atom directly on the nitrogen, which is not present in the product. Also, the structure seems to have a different arrangement of substituents, making it unlikely to be the correct reactant. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C**: (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" (TaskRunner pid=2255397) This structure has a hydroxyl group attached to the benzene ring, which is not present in the product. The presence of this hydroxyl group suggests a different synthetic pathway and thus is not the correct reactant. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D**: (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" (TaskRunner pid=2255397) This option has a different arrangement of the substituents and introduces a double bond between the carbon atoms, which is not part of the product structure. It also lacks the necessary nitrogen-containing ring. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Conclusion**: (TaskRunner pid=2255397) Only **Option A** contains the correct substituent "Cc2ccc(Cl)cc2C(F)(F)F" and the necessary nitrogen-containing ring structure. It is the only option that matches the structure of the product directly. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2255397) step:10 - global_seqlen/min:18995 - global_seqlen/max:25222 - global_seqlen/minmax_diff:6227 - global_seqlen/balanced_min:22810 - global_seqlen/balanced_max:22819 - global_seqlen/mean:22815.375 - actor/entropy:0.5086405277252197 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48631101846694946 - training/rollout_probs_diff_mean:0.004814056679606438 - training/rollout_probs_diff_std:0.010112768970429897 - training/rollout_actor_probs_pearson_corr:0.9993922114372253 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.37109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:1.0000672340393066 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.7611369912629016e-05 - rollout_corr/rollout_is_max:1.758644938468933 - rollout_corr/rollout_is_min:0.47177526354789734 - rollout_corr/rollout_is_std:0.0358346551656723 - rollout_corr/rollout_is_eff_sample_size:0.9987177621962007 - rollout_corr/rollout_is_seq_mean:1.0001494884490967 - rollout_corr/rollout_is_seq_std:0.001915071508847177 - rollout_corr/rollout_is_seq_max:1.0051805973052979 - rollout_corr/rollout_is_seq_min:0.9944745898246765 - rollout_corr/rollout_is_seq_max_deviation:0.005525410175323486 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7283513699658215) - rollout_corr/training_log_ppl:np.float64(0.5342370558064431) - rollout_corr/kl:np.float64(0.00048592705383754264) - rollout_corr/k3_kl:np.float64(0.0007897751748942028) - rollout_corr/rollout_ppl:np.float64(1.7275344631634653) - rollout_corr/rollout_log_ppl:np.float64(0.5337511280667968) - rollout_corr/log_ppl_diff:np.float64(0.00048592773964628577) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017402138910256326) - rollout_corr/log_ppl_diff_max:np.float64(0.008036613464355469) - rollout_corr/log_ppl_diff_min:np.float64(-0.007182866334915161) - rollout_corr/ppl_ratio:np.float64(1.0004885760135949) - rollout_corr/chi2_token:np.float64(0.002185102319344878) - rollout_corr/chi2_seq:np.float64(1.4425214361399412) - actor/pg_loss:np.float64(0.008299664361402392) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006863867054107686) - actor/ppo_kl:np.float64(-5.119195938618759e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37109375) - self_distillation/token_reweight_w_mean:np.float64(42394.31746227876) - self_distillation/token_reweight_w_std:np.float64(702394.3469068523) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012782558333129) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1625702439341694) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.3786138668656349) - perf/mfu/actor:np.float64(0.054599702355521036) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.04931259155273) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4217261904761905) - val-aux/sciknoweval/reward/std@16:np.float64(0.28826355015906874) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5430761904761905) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.26128001800402223) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3013761904761905) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2277461781455412) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4231666666666667) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2873520999211171) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6505000000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.2040037803005508) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.21330476190476189) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15085658746075342) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.43689523809523806) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.24031934791752282) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7316095238095237) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.13822877715965945) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.15793333333333331) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0882011251130811) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.444252380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.19013145421249977) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7838952380952381) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07974930858404744) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.12557619047619048) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04538785689784369) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.4482333333333334) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.14374275063423717) - val-aux/sciknoweval/score/mean@16:np.float64(0.4217261904761905) - val-aux/sciknoweval/score/std@16:np.float64(0.28826355015906874) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5430761904761905) - val-aux/sciknoweval/score/best@2/std:np.float64(0.26128001800402223) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.3013761904761905) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2277461781455412) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4231666666666667) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2873520999211171) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6505000000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.2040037803005508) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.21330476190476189) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15085658746075342) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.43689523809523806) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.24031934791752282) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7316095238095237) - val-aux/sciknoweval/score/best@8/std:np.float64(0.13822877715965945) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.15793333333333331) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0882011251130811) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.444252380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.19013145421249977) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7838952380952381) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07974930858404744) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.12557619047619048) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04538785689784369) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.4482333333333334) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.14374275063423717) - val-core/sciknoweval/acc/mean@16:np.float64(0.4217261904761905) - val-aux/sciknoweval/acc/std@16:np.float64(0.28826355015906874) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5430761904761905) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.26128001800402223) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.3013761904761905) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2277461781455412) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4231666666666667) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2873520999211171) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6505000000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.2040037803005508) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.21330476190476189) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15085658746075342) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.43689523809523806) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.24031934791752282) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7316095238095237) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.13822877715965945) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.15793333333333331) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0882011251130811) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.444252380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.19013145421249977) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7838952380952381) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07974930858404744) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.12557619047619048) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04538785689784369) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.4482333333333334) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.14374275063423717) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9946428571428572) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.018556639291927544) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995666666666666) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.005201629481674079) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9896238095238097) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.024846621608394712) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9946904761904761) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.018505164380603217) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0001505093393264677) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9811428571428572) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0310421950289628) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9980904761904762) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.011107450564340775) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9676523809523809) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.034830287112445846) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9995761904761905) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.004892275828419172) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9506190476190476) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0319196096808831) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999571428571428) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0009723855004127487) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003302131954114884 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0003302131954114884 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:443.60546875 - response_length/max:1164.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:443.60546875 - response_length_non_aborted/max:1164.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.375 - prompt_length/max:701.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001530386507511139 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8319511320441961) - timing_s/agent_loop/generate_sequences/max:np.float64(9.405133683234453) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.998352278737002) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.405133683234453) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:1164 - timing_s/gen:16.217690873891115 - timing_s/reward:0.0643529836088419 - timing_s/old_log_prob:3.16649679094553 - timing_s/adv:0.5727238319814205 - timing_s/update_actor:21.792541556060314 - timing_s/step:41.90203979611397 - timing_s/testing:145.202635621652 - timing_s/save_checkpoint:13.197123931720853 - timing_s/stop_profile:0.00014385022222995758 - timing_per_token_ms/update_actor:0.11939613942385516 - timing_per_token_ms/gen:0.14280787645528134 - timing_per_token_ms/adv:0.003137817327029583 - perf/total_num_tokens:182523 - perf/time_per_step:41.90203979611397 - perf/throughput:544.4931824563805 (TaskRunner pid=2255397) Training Progress: 10%|█ | 10/100 [10:23<2:18:30, 92.33s/it] (TaskRunner pid=2255397) step:11 - global_seqlen/min:18238 - global_seqlen/max:25942 - global_seqlen/minmax_diff:7704 - global_seqlen/balanced_min:22325 - global_seqlen/balanced_max:22730 - global_seqlen/mean:22380.25 - actor/entropy:0.5434318780899048 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38079291582107544 - training/rollout_probs_diff_mean:0.005151212681084871 - training/rollout_probs_diff_std:0.010387294925749302 - training/rollout_actor_probs_pearson_corr:0.9993837475776672 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0001193284988403 - rollout_corr/rollout_is_ratio_fraction_high:9.758572559803724e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.194515228271484 - rollout_corr/rollout_is_min:0.5521866679191589 - rollout_corr/rollout_is_std:0.03714950755238533 - rollout_corr/rollout_is_eff_sample_size:0.9986220538290654 - rollout_corr/rollout_is_seq_mean:1.0001423358917236 - rollout_corr/rollout_is_seq_std:0.0020691656973212957 - rollout_corr/rollout_is_seq_max:1.0071853399276733 - rollout_corr/rollout_is_seq_min:0.993625283241272 - rollout_corr/rollout_is_seq_max_deviation:0.00718533992767334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7692446084693074) - rollout_corr/training_log_ppl:np.float64(0.5590797059703618) - rollout_corr/kl:np.float64(0.0006710044406990079) - rollout_corr/k3_kl:np.float64(0.0007906234051233696) - rollout_corr/rollout_ppl:np.float64(1.7680208389647305) - rollout_corr/rollout_log_ppl:np.float64(0.5584087027818896) - rollout_corr/log_ppl_diff:np.float64(0.0006710031884722412) - rollout_corr/log_ppl_abs_diff:np.float64(0.001697185740340501) - rollout_corr/log_ppl_diff_max:np.float64(0.006523817777633667) - rollout_corr/log_ppl_diff_min:np.float64(-0.005903065204620361) - rollout_corr/ppl_ratio:np.float64(1.0006734766066074) - rollout_corr/chi2_token:np.float64(0.0018631014972925186) - rollout_corr/chi2_seq:np.float64(1.0500562572851777) - actor/pg_loss:np.float64(0.010553930653259158) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004439412439296575) - actor/ppo_kl:np.float64(6.877072929745154e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(85283.25226895092) - self_distillation/token_reweight_w_std:np.float64(1114820.7389926258) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022825964260846) - self_distillation/token_reweight_w_clip_frac:np.float64(0.21137651632307097) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4761762246489525) - perf/mfu/actor:np.float64(0.05262875043012334) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.41241455078125) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003749731695279479 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003749731695279479 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:400.2890625 - response_length/max:1690.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:400.2890625 - response_length_non_aborted/max:1690.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.09375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012122094631195068 - timing_s/agent_loop/generate_sequences/min:np.float64(2.067027226090431) - timing_s/agent_loop/generate_sequences/max:np.float64(12.363675134256482) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.645478486927459) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.363675134256482) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:1690 - timing_s/gen:18.65842711366713 - timing_s/reward:0.055716294795274734 - timing_s/old_log_prob:3.1651459094136953 - timing_s/adv:0.6307941991835833 - timing_s/update_actor:22.146489353850484 - timing_s/step:44.69863370805979 - timing_s/stop_profile:0.00010724924504756927 - timing_per_token_ms/update_actor:0.1236943809488862 - timing_per_token_ms/gen:0.18207962130557145 - timing_per_token_ms/adv:0.0035231632755642994 - perf/total_num_tokens:179042 - perf/time_per_step:44.69863370805979 - perf/throughput:500.69203784107003 (TaskRunner pid=2255397) Training Progress: 11%|█ | 11/100 [11:08<1:55:21, 77.77s/it] (TaskRunner pid=2255397) step:12 - global_seqlen/min:19921 - global_seqlen/max:27357 - global_seqlen/minmax_diff:7436 - global_seqlen/balanced_min:23213 - global_seqlen/balanced_max:23297 - global_seqlen/mean:23233.875 - actor/entropy:0.5037309527397156 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16815555095672607 - training/rollout_probs_diff_mean:0.004780740477144718 - training/rollout_probs_diff_std:0.009941385127604008 - training/rollout_actor_probs_pearson_corr:0.9994201064109802 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.999710738658905 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.6984993635560386e-05 - rollout_corr/rollout_is_max:1.6297073364257812 - rollout_corr/rollout_is_min:0.3698853552341461 - rollout_corr/rollout_is_std:0.035432368516922 - rollout_corr/rollout_is_eff_sample_size:0.9987455864550007 - rollout_corr/rollout_is_seq_mean:0.9997212886810303 - rollout_corr/rollout_is_seq_std:0.0021238892804831266 - rollout_corr/rollout_is_seq_max:1.0065120458602905 - rollout_corr/rollout_is_seq_min:0.9934496879577637 - rollout_corr/rollout_is_seq_max_deviation:0.006550312042236328 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7103436021134257) - rollout_corr/training_log_ppl:np.float64(0.5244237306469586) - rollout_corr/kl:np.float64(0.0010640324547948055) - rollout_corr/k3_kl:np.float64(0.0007240620033144296) - rollout_corr/rollout_ppl:np.float64(1.7084603938274086) - rollout_corr/rollout_log_ppl:np.float64(0.5233596969628707) - rollout_corr/log_ppl_diff:np.float64(0.0010640336840879172) - rollout_corr/log_ppl_abs_diff:np.float64(0.001916373468702659) - rollout_corr/log_ppl_diff_max:np.float64(0.007551968097686768) - rollout_corr/log_ppl_diff_min:np.float64(-0.006095826625823975) - rollout_corr/ppl_ratio:np.float64(1.001067083561793) - rollout_corr/chi2_token:np.float64(0.0007682361174374819) - rollout_corr/chi2_seq:np.float64(0.6293782293796539) - actor/pg_loss:np.float64(0.007974700187332928) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048827441059984267) - actor/ppo_kl:np.float64(0.00010889819034787251) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(5987.44374671299) - self_distillation/token_reweight_w_std:np.float64(110518.26587392716) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0042279572226107) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17869807308306918) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.39314158260822296) - perf/mfu/actor:np.float64(0.05490032892199064) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.42404556274414) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0070593878626823425 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0070593878626823425 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:459.96484375 - response_length/max:1546.0 - response_length/min:137.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:459.96484375 - response_length_non_aborted/max:1546.0 - response_length_non_aborted/min:137.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.09375 - prompt_length/max:665.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016945600509643555 - timing_s/agent_loop/generate_sequences/min:np.float64(1.736042970791459) - timing_s/agent_loop/generate_sequences/max:np.float64(12.078167820349336) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.221707455886644) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.078167820349336) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:1546 - timing_s/gen:18.70925861224532 - timing_s/reward:0.06456440314650536 - timing_s/old_log_prob:3.0842963717877865 - timing_s/adv:0.5920961294323206 - timing_s/update_actor:22.019973430782557 - timing_s/step:44.561775021255016 - timing_s/stop_profile:0.0001148320734500885 - timing_per_token_ms/update_actor:0.11846911799464444 - timing_per_token_ms/gen:0.15888832037303566 - timing_per_token_ms/adv:0.0031855218373620445 - perf/total_num_tokens:185871 - perf/time_per_step:44.561775021255016 - perf/throughput:521.3857614270961 (TaskRunner pid=2255397) Training Progress: 12%|█▏ | 12/100 [11:52<1:39:16, 67.68s/it] (TaskRunner pid=2255397) step:13 - global_seqlen/min:18814 - global_seqlen/max:23875 - global_seqlen/minmax_diff:5061 - global_seqlen/balanced_min:21556 - global_seqlen/balanced_max:21562 - global_seqlen/mean:21560.5 - actor/entropy:0.540065348148346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2304113507270813 - training/rollout_probs_diff_mean:0.005129465367645025 - training/rollout_probs_diff_std:0.010314878076314926 - training/rollout_actor_probs_pearson_corr:0.9993942975997925 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:0.9999523758888245 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.84678445092868e-06 - rollout_corr/rollout_is_max:1.643043875694275 - rollout_corr/rollout_is_min:0.3293769061565399 - rollout_corr/rollout_is_std:0.03673160448670387 - rollout_corr/rollout_is_eff_sample_size:0.9986523694012616 - rollout_corr/rollout_is_seq_mean:0.9999930262565613 - rollout_corr/rollout_is_seq_std:0.0019651646725833416 - rollout_corr/rollout_is_seq_max:1.0069950819015503 - rollout_corr/rollout_is_seq_min:0.9950668215751648 - rollout_corr/rollout_is_seq_max_deviation:0.006995081901550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.75340425549075) - rollout_corr/training_log_ppl:np.float64(0.5529010602040216) - rollout_corr/kl:np.float64(0.0008567688251606143) - rollout_corr/k3_kl:np.float64(0.0008690143870353495) - rollout_corr/rollout_ppl:np.float64(1.7518776911310852) - rollout_corr/rollout_log_ppl:np.float64(0.5520442884298973) - rollout_corr/log_ppl_diff:np.float64(0.0008567717741243541) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018868366605602205) - rollout_corr/log_ppl_diff_max:np.float64(0.00990372896194458) - rollout_corr/log_ppl_diff_min:np.float64(-0.005962073802947998) - rollout_corr/ppl_ratio:np.float64(1.0008597834967077) - rollout_corr/chi2_token:np.float64(0.0017287565860897303) - rollout_corr/chi2_seq:np.float64(0.7434897748753428) - actor/pg_loss:np.float64(0.009388480219058692) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001301017089645029) - actor/ppo_kl:np.float64(0.0001421315267471357) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(1917.8817092673853) - self_distillation/token_reweight_w_std:np.float64(43886.33348266815) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0036602555774152) - self_distillation/token_reweight_w_clip_frac:np.float64(0.2082537584064994) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.43746187537908554) - perf/mfu/actor:np.float64(0.051655857866616195) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.21465253829956) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003404525574296713 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003404525574296713 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:396.703125 - response_length/max:972.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:396.703125 - response_length_non_aborted/max:972.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:612.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013491325080394745 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8798387572169304) - timing_s/agent_loop/generate_sequences/max:np.float64(8.628671053797007) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.614353348057193) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.628671053797007) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:972 - timing_s/gen:15.247638115659356 - timing_s/reward:0.06261933967471123 - timing_s/old_log_prob:3.0381316114217043 - timing_s/adv:0.5441188924014568 - timing_s/update_actor:21.71716057509184 - timing_s/step:40.69797484949231 - timing_s/stop_profile:0.00012857653200626373 - timing_per_token_ms/update_actor:0.12590826149145334 - timing_per_token_ms/gen:0.15014019965003894 - timing_per_token_ms/adv:0.0031546050207639947 - perf/total_num_tokens:172484 - perf/time_per_step:40.69797484949231 - perf/throughput:529.7683749556143 (TaskRunner pid=2255397) Training Progress: 13%|█▎ | 13/100 [12:33<1:26:18, 59.52s/it] (TaskRunner pid=2255397) step:14 - global_seqlen/min:17920 - global_seqlen/max:21796 - global_seqlen/minmax_diff:3876 - global_seqlen/balanced_min:20032 - global_seqlen/balanced_max:20152 - global_seqlen/mean:20054.25 - actor/entropy:0.5020557641983032 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18802747130393982 - training/rollout_probs_diff_mean:0.005197192542254925 - training/rollout_probs_diff_std:0.01069364883005619 - training/rollout_actor_probs_pearson_corr:0.9992953538894653 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.40625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.000118374824524 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6423780918121338 - rollout_corr/rollout_is_min:0.6023645997047424 - rollout_corr/rollout_is_std:0.03736548125743866 - rollout_corr/rollout_is_eff_sample_size:0.9986062429080049 - rollout_corr/rollout_is_seq_mean:1.0001424551010132 - rollout_corr/rollout_is_seq_std:0.002069722395390272 - rollout_corr/rollout_is_seq_max:1.0075104236602783 - rollout_corr/rollout_is_seq_min:0.9939577579498291 - rollout_corr/rollout_is_seq_max_deviation:0.00751042366027832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7107822382822633) - rollout_corr/training_log_ppl:np.float64(0.5240909514250234) - rollout_corr/kl:np.float64(0.0007750868314850834) - rollout_corr/k3_kl:np.float64(0.0011061159723340097) - rollout_corr/rollout_ppl:np.float64(1.7093983073718846) - rollout_corr/rollout_log_ppl:np.float64(0.5233158656046726) - rollout_corr/log_ppl_diff:np.float64(0.000775085820350796) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019063884974457324) - rollout_corr/log_ppl_diff_max:np.float64(0.009017229080200195) - rollout_corr/log_ppl_diff_min:np.float64(-0.007934510707855225) - rollout_corr/ppl_ratio:np.float64(1.0007783074397594) - rollout_corr/chi2_token:np.float64(0.0027928389608860016) - rollout_corr/chi2_seq:np.float64(0.6327834515832365) - actor/pg_loss:np.float64(0.009141277987509966) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0023054060643517005) - actor/ppo_kl:np.float64(0.00017033189165172047) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40625) - self_distillation/token_reweight_w_mean:np.float64(51790.3643477282) - self_distillation/token_reweight_w_std:np.float64(710581.8369147222) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0032089522574097) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1833768331562169) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.5500108152627945) - perf/mfu/actor:np.float64(0.04745480566246875) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.28086853027344) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:1.6309666534652933e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:1.6309666534652933e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:359.2578125 - response_length/max:1234.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:359.2578125 - response_length_non_aborted/max:1234.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.4375 - prompt_length/max:666.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013267621397972107 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5782812405377626) - timing_s/agent_loop/generate_sequences/max:np.float64(9.458859227597713) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9894911859955755) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.458859227597713) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:1234 - timing_s/gen:16.56269684806466 - timing_s/reward:0.05258593708276749 - timing_s/old_log_prob:2.994526704773307 - timing_s/adv:0.5849139746278524 - timing_s/update_actor:21.962995065376163 - timing_s/step:42.20455348305404 - timing_s/stop_profile:0.0001186169683933258 - timing_per_token_ms/update_actor:0.13689738500178367 - timing_per_token_ms/gen:0.18008803792611353 - timing_per_token_ms/adv:0.003645823046410689 - perf/total_num_tokens:160434 - perf/time_per_step:42.20455348305404 - perf/throughput:475.1679225335763 (TaskRunner pid=2255397) Training Progress: 14%|█▍ | 14/100 [13:15<1:17:50, 54.31s/it] (TaskRunner pid=2255397) step:15 - global_seqlen/min:17292 - global_seqlen/max:23462 - global_seqlen/minmax_diff:6170 - global_seqlen/balanced_min:20153 - global_seqlen/balanced_max:20223 - global_seqlen/mean:20173.0 - actor/entropy:0.4989277422428131 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23447632789611816 - training/rollout_probs_diff_mean:0.005335042718797922 - training/rollout_probs_diff_std:0.011061806231737137 - training/rollout_actor_probs_pearson_corr:0.99927818775177 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.46875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9997453093528748 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5897420644760132 - rollout_corr/rollout_is_min:0.5000474452972412 - rollout_corr/rollout_is_std:0.03860742598772049 - rollout_corr/rollout_is_eff_sample_size:0.9985110313307147 - rollout_corr/rollout_is_seq_mean:0.9997358918190002 - rollout_corr/rollout_is_seq_std:0.0024659933988004923 - rollout_corr/rollout_is_seq_max:1.0084636211395264 - rollout_corr/rollout_is_seq_min:0.9930576682090759 - rollout_corr/rollout_is_seq_max_deviation:0.008463621139526367 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7213433790020645) - rollout_corr/training_log_ppl:np.float64(0.530678162467666) - rollout_corr/kl:np.float64(0.0010814671836847367) - rollout_corr/k3_kl:np.float64(0.000888961919315534) - rollout_corr/rollout_ppl:np.float64(1.7193831540644169) - rollout_corr/rollout_log_ppl:np.float64(0.5295966898556799) - rollout_corr/log_ppl_diff:np.float64(0.0010814726119861007) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019996468909084797) - rollout_corr/log_ppl_diff_max:np.float64(0.012681245803833008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0078064799308776855) - rollout_corr/ppl_ratio:np.float64(1.0010852627456188) - rollout_corr/chi2_token:np.float64(0.0014137444086372852) - rollout_corr/chi2_seq:np.float64(0.5360429051797837) - actor/pg_loss:np.float64(0.007955617504194379) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006377134768627002) - actor/ppo_kl:np.float64(1.769121835337728e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46875) - self_distillation/token_reweight_w_mean:np.float64(8754.255326425191) - self_distillation/token_reweight_w_std:np.float64(117017.50059508975) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001879719318822) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1550164521613624) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.4198375418782234) - perf/mfu/actor:np.float64(0.04879610399473919) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.12841796875) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00847491342574358 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00847491342574358 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:343.5 - response_length/max:1036.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.5 - response_length_non_aborted/max:1036.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.90625 - prompt_length/max:494.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012453272938728333 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3633983470499516) - timing_s/agent_loop/generate_sequences/max:np.float64(8.056044036522508) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.874184662854532) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.056044036522508) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:1036 - timing_s/gen:14.75450462475419 - timing_s/reward:0.052955711260437965 - timing_s/old_log_prob:3.0404567383229733 - timing_s/adv:0.516296835616231 - timing_s/update_actor:21.39274295605719 - timing_s/step:39.795078329741955 - timing_s/stop_profile:3.5120174288749695e-05 - timing_per_token_ms/update_actor:0.13255801663149502 - timing_per_token_ms/gen:0.16778685208281238 - timing_per_token_ms/adv:0.003199182295743264 - perf/total_num_tokens:161384 - perf/time_per_step:39.795078329741955 - perf/throughput:506.9219824835261 (TaskRunner pid=2255397) Training Progress: 15%|█▌ | 15/100 [13:55<1:10:45, 49.94s/it] (TaskRunner pid=2255397) step:16 - global_seqlen/min:17365 - global_seqlen/max:24430 - global_seqlen/minmax_diff:7065 - global_seqlen/balanced_min:20809 - global_seqlen/balanced_max:20815 - global_seqlen/mean:20814.0 - actor/entropy:0.4525703191757202 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44003361463546753 - training/rollout_probs_diff_mean:0.0051152328960597515 - training/rollout_probs_diff_std:0.011277093552052975 - training/rollout_actor_probs_pearson_corr:0.9991934299468994 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:1.000049352645874 - rollout_corr/rollout_is_ratio_fraction_high:1.0565463526290841e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.339278479572386e-05 - rollout_corr/rollout_is_max:2.1190927028656006 - rollout_corr/rollout_is_min:0.16761082410812378 - rollout_corr/rollout_is_std:0.03827483579516411 - rollout_corr/rollout_is_eff_sample_size:0.9985372988785958 - rollout_corr/rollout_is_seq_mean:1.0000265836715698 - rollout_corr/rollout_is_seq_std:0.0022894139401614666 - rollout_corr/rollout_is_seq_max:1.0063250064849854 - rollout_corr/rollout_is_seq_min:0.992612361907959 - rollout_corr/rollout_is_seq_max_deviation:0.007387638092041016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6002246476709843) - rollout_corr/training_log_ppl:np.float64(0.46000494839972816) - rollout_corr/kl:np.float64(0.0006955834787873982) - rollout_corr/k3_kl:np.float64(0.0009704830327450509) - rollout_corr/rollout_ppl:np.float64(1.5991095416247845) - rollout_corr/rollout_log_ppl:np.float64(0.45930936612421647) - rollout_corr/log_ppl_diff:np.float64(0.0006955822755116969) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021134040143806487) - rollout_corr/log_ppl_diff_max:np.float64(0.007766246795654297) - rollout_corr/log_ppl_diff_min:np.float64(-0.005490928888320923) - rollout_corr/ppl_ratio:np.float64(1.0006991014815867) - rollout_corr/chi2_token:np.float64(0.0025813004467636347) - rollout_corr/chi2_seq:np.float64(3.1832106430083513) - actor/pg_loss:np.float64(0.009350149193778634) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011533425140441977) - actor/ppo_kl:np.float64(-8.012323441874969e-05) - actor/pg_clipfrac_lower:np.float64(1.187310044770129e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(35795.58769591758) - self_distillation/token_reweight_w_std:np.float64(482687.72877264104) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0030966859776527) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18662144461995922) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.49581966549158096) - perf/mfu/actor:np.float64(0.05050208271241423) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.1108741760254) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01058791484683752 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01058791484683752 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.71875 - response_length/max:1008.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.71875 - response_length_non_aborted/max:1008.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.71875 - prompt_length/max:859.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.693439066410065e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3586726989597082) - timing_s/agent_loop/generate_sequences/max:np.float64(7.759539771825075) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9693014876902453) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.759539771825075) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:1008 - timing_s/gen:14.533444579690695 - timing_s/reward:0.06137754209339619 - timing_s/old_log_prob:3.1735431142151356 - timing_s/adv:0.5521691609174013 - timing_s/update_actor:21.5248114336282 - timing_s/step:39.9321037363261 - timing_s/stop_profile:0.00011518038809299469 - timing_per_token_ms/update_actor:0.1292688300760798 - timing_per_token_ms/gen:0.15355257987163695 - timing_per_token_ms/adv:0.0033160922991580266 - perf/total_num_tokens:166512 - perf/time_per_step:39.9321037363261 - perf/throughput:521.2347472959601 (TaskRunner pid=2255397) Training Progress: 16%|█▌ | 16/100 [14:35<1:05:43, 46.94s/it] (TaskRunner pid=2255397) step:17 - global_seqlen/min:22601 - global_seqlen/max:25800 - global_seqlen/minmax_diff:3199 - global_seqlen/balanced_min:23884 - global_seqlen/balanced_max:24138 - global_seqlen/mean:23919.75 - actor/entropy:0.5084906220436096 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23104846477508545 - training/rollout_probs_diff_mean:0.004854131024330854 - training/rollout_probs_diff_std:0.010198368690907955 - training/rollout_actor_probs_pearson_corr:0.9993425011634827 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:1.0001150369644165 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.8287475288379937e-05 - rollout_corr/rollout_is_max:1.7099158763885498 - rollout_corr/rollout_is_min:0.4197293221950531 - rollout_corr/rollout_is_std:0.036216843873262405 - rollout_corr/rollout_is_eff_sample_size:0.9986904150765131 - rollout_corr/rollout_is_seq_mean:1.0001122951507568 - rollout_corr/rollout_is_seq_std:0.0019326574401929975 - rollout_corr/rollout_is_seq_max:1.0060887336730957 - rollout_corr/rollout_is_seq_min:0.9933558106422424 - rollout_corr/rollout_is_seq_max_deviation:0.006644189357757568 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7412785273045301) - rollout_corr/training_log_ppl:np.float64(0.5394562017754652) - rollout_corr/kl:np.float64(0.00046429189457519726) - rollout_corr/k3_kl:np.float64(0.0007774811874128318) - rollout_corr/rollout_ppl:np.float64(1.7404227014631033) - rollout_corr/rollout_log_ppl:np.float64(0.538991907320451) - rollout_corr/log_ppl_diff:np.float64(0.0004642944550141692) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017179182032123208) - rollout_corr/log_ppl_diff_max:np.float64(0.007203280925750732) - rollout_corr/log_ppl_diff_min:np.float64(-0.006280422210693359) - rollout_corr/ppl_ratio:np.float64(1.0004668079782277) - rollout_corr/chi2_token:np.float64(0.002199894981458783) - rollout_corr/chi2_seq:np.float64(2.1964850607328117) - actor/pg_loss:np.float64(0.006313061923719943) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00043986083028357825) - actor/ppo_kl:np.float64(-0.00013203885243839864) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(1803.160887064878) - self_distillation/token_reweight_w_std:np.float64(34584.773948686314) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014728792011738) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12900599383283406) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.38481806963682175) - perf/mfu/actor:np.float64(0.05734825677685727) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.12204360961914) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005190751980990171 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005190751980990171 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:414.2734375 - response_length/max:1415.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:414.2734375 - response_length_non_aborted/max:1415.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:333.21875 - prompt_length/max:617.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010232068598270416 - timing_s/agent_loop/generate_sequences/min:np.float64(1.907282380387187) - timing_s/agent_loop/generate_sequences/max:np.float64(11.15155959688127) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.6968326987480395) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.15155959688127) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1415 - timing_s/gen:17.835143700242043 - timing_s/reward:0.06522902101278305 - timing_s/old_log_prob:3.041203547269106 - timing_s/adv:0.5475675780326128 - timing_s/update_actor:21.574721600860357 - timing_s/step:43.15138044394553 - timing_s/stop_profile:4.641711711883545e-05 - timing_per_token_ms/update_actor:0.11274533388131333 - timing_per_token_ms/gen:0.16817040093011149 - timing_per_token_ms/adv:0.002861482551200435 - perf/total_num_tokens:191358 - perf/time_per_step:43.15138044394553 - perf/throughput:554.3217796026761 (TaskRunner pid=2255397) Training Progress: 17%|█▋ | 17/100 [15:18<1:03:22, 45.81s/it] (TaskRunner pid=2255397) step:18 - global_seqlen/min:17607 - global_seqlen/max:24855 - global_seqlen/minmax_diff:7248 - global_seqlen/balanced_min:20730 - global_seqlen/balanced_max:20735 - global_seqlen/mean:20733.125 - actor/entropy:0.4945189952850342 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30840808153152466 - training/rollout_probs_diff_mean:0.005118089262396097 - training/rollout_probs_diff_std:0.010652771219611168 - training/rollout_actor_probs_pearson_corr:0.9993004202842712 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9998429417610168 - rollout_corr/rollout_is_ratio_fraction_high:2.0853969544987194e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.032266139984131 - rollout_corr/rollout_is_min:0.5130106210708618 - rollout_corr/rollout_is_std:0.03713827580213547 - rollout_corr/rollout_is_eff_sample_size:0.9986221727100675 - rollout_corr/rollout_is_seq_mean:0.9997930526733398 - rollout_corr/rollout_is_seq_std:0.0022385723423212767 - rollout_corr/rollout_is_seq_max:1.007550835609436 - rollout_corr/rollout_is_seq_min:0.9926072359085083 - rollout_corr/rollout_is_seq_max_deviation:0.007550835609436035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6632476961240172) - rollout_corr/training_log_ppl:np.float64(0.49761353904614225) - rollout_corr/kl:np.float64(0.0008254678116763614) - rollout_corr/k3_kl:np.float64(0.0008354976650934987) - rollout_corr/rollout_ppl:np.float64(1.6618767492473125) - rollout_corr/rollout_log_ppl:np.float64(0.49678807152668014) - rollout_corr/log_ppl_diff:np.float64(0.0008254675194621086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019702696008607745) - rollout_corr/log_ppl_diff_max:np.float64(0.008582055568695068) - rollout_corr/log_ppl_diff_min:np.float64(-0.006475538015365601) - rollout_corr/ppl_ratio:np.float64(1.0008287394884974) - rollout_corr/chi2_token:np.float64(0.0016966825351119041) - rollout_corr/chi2_seq:np.float64(0.7993498002178967) - actor/pg_loss:np.float64(0.006876209634356201) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009630748777453846) - actor/ppo_kl:np.float64(-0.00010461421089758005) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(6797.09068774269) - self_distillation/token_reweight_w_std:np.float64(117938.16381233273) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013849884271622) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17171310502453707) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.4298411011695862) - perf/mfu/actor:np.float64(0.05005069497056926) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.3214340209961) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005354256834834814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005354256834834814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.62890625 - response_length/max:907.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.62890625 - response_length_non_aborted/max:907.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.28125 - prompt_length/max:540.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001042727380990982 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7595505379140377) - timing_s/agent_loop/generate_sequences/max:np.float64(7.9033639673143625) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.420700289687375) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.9033639673143625) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:245 - timing_s/agent_loop/slowest/response_length:907 - timing_s/gen:14.91362101584673 - timing_s/reward:0.05069681629538536 - timing_s/old_log_prob:3.109211662784219 - timing_s/adv:0.5708821713924408 - timing_s/update_actor:21.388794276863337 - timing_s/step:40.0685121640563 - timing_s/stop_profile:0.00011024437844753265 - timing_per_token_ms/update_actor:0.12895302973420153 - timing_per_token_ms/gen:0.15550410318384578 - timing_per_token_ms/adv:0.0034418483187679183 - perf/total_num_tokens:165865 - perf/time_per_step:40.0685121640563 - perf/throughput:517.4418484796842 (TaskRunner pid=2255397) Training Progress: 18%|█▊ | 18/100 [15:58<1:00:16, 44.11s/it] (TaskRunner pid=2255397) step:19 - global_seqlen/min:16539 - global_seqlen/max:26420 - global_seqlen/minmax_diff:9881 - global_seqlen/balanced_min:21542 - global_seqlen/balanced_max:21548 - global_seqlen/mean:21545.625 - actor/entropy:0.48952940106391907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21504220366477966 - training/rollout_probs_diff_mean:0.0051950011402368546 - training/rollout_probs_diff_std:0.010906816460192204 - training/rollout_actor_probs_pearson_corr:0.9992679953575134 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0002551078796387 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.993241310119629 - rollout_corr/rollout_is_min:0.508678138256073 - rollout_corr/rollout_is_std:0.03832463547587395 - rollout_corr/rollout_is_eff_sample_size:0.9985340896470543 - rollout_corr/rollout_is_seq_mean:1.000322699546814 - rollout_corr/rollout_is_seq_std:0.0022897543385624886 - rollout_corr/rollout_is_seq_max:1.007610559463501 - rollout_corr/rollout_is_seq_min:0.9937204718589783 - rollout_corr/rollout_is_seq_max_deviation:0.0076105594635009766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6905396389774978) - rollout_corr/training_log_ppl:np.float64(0.5122466792818159) - rollout_corr/kl:np.float64(0.0005300049446929078) - rollout_corr/k3_kl:np.float64(0.0008489201253496503) - rollout_corr/rollout_ppl:np.float64(1.6896098447032273) - rollout_corr/rollout_log_ppl:np.float64(0.511716675653588) - rollout_corr/log_ppl_diff:np.float64(0.0005300036282278597) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018085126648657024) - rollout_corr/log_ppl_diff_max:np.float64(0.011858522891998291) - rollout_corr/log_ppl_diff_min:np.float64(-0.007662832736968994) - rollout_corr/ppl_ratio:np.float64(1.0005329437553883) - rollout_corr/chi2_token:np.float64(0.00233305711299181) - rollout_corr/chi2_seq:np.float64(0.9715974144637585) - actor/pg_loss:np.float64(0.006131203263066709) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005582695239354507) - actor/ppo_kl:np.float64(7.291653415109067e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(19979.935181184206) - self_distillation/token_reweight_w_std:np.float64(310836.0626574742) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012645116075873) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1186036104336381) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.3470497876405716) - perf/mfu/actor:np.float64(0.051614476906136436) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.35358047485352) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004195125307887793 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004195125307887793 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:371.17578125 - response_length/max:894.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:371.17578125 - response_length_non_aborted/max:894.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.125 - prompt_length/max:836.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014657527208328247 - timing_s/agent_loop/generate_sequences/min:np.float64(1.919897235929966) - timing_s/agent_loop/generate_sequences/max:np.float64(7.8232190404087305) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.320013291013311) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.8232190404087305) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:894 - timing_s/gen:14.59238414466381 - timing_s/reward:0.06268099136650562 - timing_s/old_log_prob:2.9667536336928606 - timing_s/adv:0.558667978271842 - timing_s/update_actor:21.68359724804759 - timing_s/step:39.951241597533226 - timing_s/stop_profile:0.00011624395847320557 - timing_per_token_ms/update_actor:0.12580046557043245 - timing_per_token_ms/gen:0.153570096554065 - timing_per_token_ms/adv:0.0032411915311800076 - perf/total_num_tokens:172365 - perf/time_per_step:39.951241597533226 - perf/throughput:539.2980077327641 (TaskRunner pid=2255397) Training Progress: 19%|█▉ | 19/100 [16:38<57:52, 42.87s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we must identify the structure of the product and match it with the most appropriate reactant that would lead to its formation through a plausible synthetic pathway. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product has the following key structural features: (TaskRunner pid=2255397) - A **nitrogen-containing ring** (likely a pyrrole or imidazole-like structure). (TaskRunner pid=2255397) - A **chlorinated aromatic ring** (e.g., **Cl-phenyl**). (TaskRunner pid=2255397) - A **trifluoromethyl group** (C(F)(F)F). (TaskRunner pid=2255397) - A **nitrogen atom** bonded to a carbon chain that includes the **Cl-phenyl** and **trifluoromethyl** groups. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let’s examine the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` (TaskRunner pid=2255397) This structure contains a **nitrogen ring** with the same **Cl-phenyl** and **trifluoromethyl** substituents as the product. The **carbonyl groups** and **nitrogen ring** suggest a possible **amide** or **nitrile** structure that could be a **precursor** to the product. It is the **most structurally similar** to the product and would likely be the **correct reactant** in a synthesis involving ring formation or substitution. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1` (TaskRunner pid=2255397) This structure contains **two chlorines** and a **nitrogen ring**, but it lacks the **trifluoromethyl group** and the **correct arrangement of substituents** as in the product. It is **not a good match**. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1` (TaskRunner pid=2255397) This structure includes a **trifluoromethyl group** and **Cl-phenyl**, but the **nitrogen ring** is **not directly bonded to the Cl-phenyl** or **trifluoromethyl**, and the **structure is more complex**. It might be a **byproduct** or **intermediate**, but not the **direct reactant**. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F` (TaskRunner pid=2255397) This structure contains a **trifluoromethyl group**, but it **lacks the nitrogen ring** and the **Cl-phenyl** substituents. It is **structurally different** from the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Conclusion**: (TaskRunner pid=2255397) Only **Option A** contains the **correct substituents** (Cl-phenyl, trifluoromethyl, and a nitrogen ring), and the **structure is most consistent** with the product. It is the **only plausible reactant** that would lead to the synthesis of the given product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2255397) step:20 - global_seqlen/min:18853 - global_seqlen/max:23232 - global_seqlen/minmax_diff:4379 - global_seqlen/balanced_min:20703 - global_seqlen/balanced_max:20714 - global_seqlen/mean:20709.5 - actor/entropy:0.4852030277252197 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3635062277317047 - training/rollout_probs_diff_mean:0.0052037229761481285 - training/rollout_probs_diff_std:0.010991442948579788 - training/rollout_actor_probs_pearson_corr:0.9992546439170837 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0001426935195923 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1487816411536187e-05 - rollout_corr/rollout_is_max:1.6168099641799927 - rollout_corr/rollout_is_min:0.4058627188205719 - rollout_corr/rollout_is_std:0.03768158331513405 - rollout_corr/rollout_is_eff_sample_size:0.9985827057725277 - rollout_corr/rollout_is_seq_mean:1.0002155303955078 - rollout_corr/rollout_is_seq_std:0.002177149523049593 - rollout_corr/rollout_is_seq_max:1.0087448358535767 - rollout_corr/rollout_is_seq_min:0.9949045181274414 - rollout_corr/rollout_is_seq_max_deviation:0.00874483585357666 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.65907796099782) - rollout_corr/training_log_ppl:np.float64(0.49622454700875096) - rollout_corr/kl:np.float64(0.0007081711968908166) - rollout_corr/k3_kl:np.float64(0.000836590931498904) - rollout_corr/rollout_ppl:np.float64(1.6578862462192774) - rollout_corr/rollout_log_ppl:np.float64(0.49551637563854456) - rollout_corr/log_ppl_diff:np.float64(0.0007081713702064008) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018356915388721973) - rollout_corr/log_ppl_diff_max:np.float64(0.007012069225311279) - rollout_corr/log_ppl_diff_min:np.float64(-0.006520330905914307) - rollout_corr/ppl_ratio:np.float64(1.0007108724676073) - rollout_corr/chi2_token:np.float64(0.001943168230354786) - rollout_corr/chi2_seq:np.float64(0.7969548560213298) - actor/pg_loss:np.float64(0.006117190117947757) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006000254938953731) - actor/ppo_kl:np.float64(0.0001683622528645401) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(23049.971880125348) - self_distillation/token_reweight_w_std:np.float64(317665.6890155331) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982667530421168) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15381890590651892) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3626445010304451) - perf/mfu/actor:np.float64(0.05021794625693002) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.33641052246094) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5196428571428572) - val-aux/sciknoweval/reward/std@16:np.float64(0.2570318204515784) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6278380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.21727863221866373) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.41209523809523807) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2203367172556475) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5193428571428571) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2582634523002606) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.713442857142857) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15828990121163083) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.3252571428571428) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15921732171540784) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5385380952380953) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2095760136213612) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7737142857142857) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1018065135469096) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.2657142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10472175061646756) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5479142857142857) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15900279342137813) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8120428571428572) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05885508241318426) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.22566190476190473) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06395190241306174) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5533238095238096) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11781703809110204) - val-aux/sciknoweval/score/mean@16:np.float64(0.5196428571428572) - val-aux/sciknoweval/score/std@16:np.float64(0.2570318204515784) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6278380952380953) - val-aux/sciknoweval/score/best@2/std:np.float64(0.21727863221866373) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.41209523809523807) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2203367172556475) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5193428571428571) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2582634523002606) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.713442857142857) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15828990121163083) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.3252571428571428) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15921732171540784) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5385380952380953) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2095760136213612) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7737142857142857) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1018065135469096) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.2657142857142857) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10472175061646756) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5479142857142857) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15900279342137813) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8120428571428572) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05885508241318426) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.22566190476190473) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06395190241306174) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5533238095238096) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11781703809110204) - val-core/sciknoweval/acc/mean@16:np.float64(0.5196428571428572) - val-aux/sciknoweval/acc/std@16:np.float64(0.2570318204515784) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6278380952380953) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.21727863221866373) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.41209523809523807) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2203367172556475) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5193428571428571) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2582634523002606) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.713442857142857) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15828990121163083) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.3252571428571428) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15921732171540784) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5385380952380953) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2095760136213612) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7737142857142857) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1018065135469096) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.2657142857142857) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10472175061646756) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5479142857142857) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15900279342137813) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8120428571428572) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05885508241318426) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.22566190476190473) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06395190241306174) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5533238095238096) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11781703809110204) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9979166666666667) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.005738829557423338) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9996761904761906) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0021556297891449894) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9960809523809524) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.007408678245820718) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9977904761904762) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0058674039295688075) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9931095238095238) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.008738507306529836) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9991428571428571) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0038031567601600064) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9892238095238096) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.008745323711299243) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997809523809524) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0019148844274678545) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9851285714285715) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.006912686317569252) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999857142857144) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00036325512888540755) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004073284100741148 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004073284100741148 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:363.578125 - response_length/max:1073.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:363.578125 - response_length_non_aborted/max:1073.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:392.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001299697905778885 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8756434936076403) - timing_s/agent_loop/generate_sequences/max:np.float64(8.891039799898863) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.303868516799412) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.891039799898863) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1073 - timing_s/gen:15.654097167775035 - timing_s/reward:0.05749098025262356 - timing_s/old_log_prob:3.016568712890148 - timing_s/adv:0.5642522405833006 - timing_s/update_actor:21.53763866983354 - timing_s/step:40.86700990051031 - timing_s/testing:131.59580136090517 - timing_s/save_checkpoint:13.167504344135523 - timing_s/stop_profile:0.00014436431229114532 - timing_per_token_ms/update_actor:0.12999854336073746 - timing_per_token_ms/gen:0.16818618298782753 - timing_per_token_ms/adv:0.003405757264681068 - perf/total_num_tokens:165676 - perf/time_per_step:40.86700990051031 - perf/throughput:506.7534926195175 (TaskRunner pid=2255397) Training Progress: 20%|██ | 20/100 [19:44<1:54:20, 85.76s/it] (TaskRunner pid=2255397) step:21 - global_seqlen/min:17376 - global_seqlen/max:26208 - global_seqlen/minmax_diff:8832 - global_seqlen/balanced_min:22075 - global_seqlen/balanced_max:22088 - global_seqlen/mean:22085.25 - actor/entropy:0.4656538665294647 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102373421192169 - training/rollout_probs_diff_mean:0.004885641857981682 - training/rollout_probs_diff_std:0.010877853259444237 - training/rollout_actor_probs_pearson_corr:0.9992596507072449 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.84765625 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84765625 - rollout_corr/rollout_is_mean:0.9998842477798462 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.879465500882361e-05 - rollout_corr/rollout_is_max:1.9200012683868408 - rollout_corr/rollout_is_min:0.25445759296417236 - rollout_corr/rollout_is_std:0.03666176274418831 - rollout_corr/rollout_is_eff_sample_size:0.9986574816202841 - rollout_corr/rollout_is_seq_mean:0.9998617172241211 - rollout_corr/rollout_is_seq_std:0.0022321075666695833 - rollout_corr/rollout_is_seq_max:1.0074141025543213 - rollout_corr/rollout_is_seq_min:0.9920339584350586 - rollout_corr/rollout_is_seq_max_deviation:0.007966041564941406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6367876464501023) - rollout_corr/training_log_ppl:np.float64(0.47835954203037545) - rollout_corr/kl:np.float64(0.0009155909243130367) - rollout_corr/k3_kl:np.float64(0.0008391680780164279) - rollout_corr/rollout_ppl:np.float64(1.6352251395583153) - rollout_corr/rollout_log_ppl:np.float64(0.4774439476314001) - rollout_corr/log_ppl_diff:np.float64(0.0009155943989753723) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018019898561760783) - rollout_corr/log_ppl_diff_max:np.float64(0.008913397789001465) - rollout_corr/log_ppl_diff_min:np.float64(-0.006456807255744934) - rollout_corr/ppl_ratio:np.float64(1.0009184170048684) - rollout_corr/chi2_token:np.float64(0.001522636041045189) - rollout_corr/chi2_seq:np.float64(0.3302574837580323) - actor/pg_loss:np.float64(0.00977602880448103) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008731294096833153) - actor/ppo_kl:np.float64(2.3401729779659597e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(19834.05608875933) - self_distillation/token_reweight_w_std:np.float64(292799.855937861) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0034907914232463) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1998413290712051) - self_distillation/empty_target_batch:np.float64(0.15234375) - actor/grad_norm:np.float64(0.506427951157093) - perf/mfu/actor:np.float64(0.05293222627939937) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.2050437927246) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00826286617666483 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00826286617666483 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.9765625 - response_length/max:1132.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.9765625 - response_length_non_aborted/max:1132.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.1875 - prompt_length/max:669.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.725825369358063e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8232946544885635) - timing_s/agent_loop/generate_sequences/max:np.float64(9.148927487432957) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.626729334348056) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.148927487432957) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:234 - timing_s/agent_loop/slowest/response_length:1132 - timing_s/gen:15.357284164056182 - timing_s/reward:0.06359587237238884 - timing_s/old_log_prob:3.120852567255497 - timing_s/adv:0.6262220405042171 - timing_s/update_actor:21.799812253564596 - timing_s/step:41.07217570580542 - timing_s/stop_profile:7.509812712669373e-05 - timing_per_token_ms/update_actor:0.12338445485994383 - timing_per_token_ms/gen:0.14740257005793658 - timing_per_token_ms/adv:0.0035443454370236763 - perf/total_num_tokens:176682 - perf/time_per_step:41.07217570580542 - perf/throughput:537.7180444053837 (TaskRunner pid=2255397) Training Progress: 21%|██ | 21/100 [20:25<1:35:16, 72.35s/it] (TaskRunner pid=2255397) step:22 - global_seqlen/min:15780 - global_seqlen/max:20714 - global_seqlen/minmax_diff:4934 - global_seqlen/balanced_min:18793 - global_seqlen/balanced_max:18799 - global_seqlen/mean:18797.25 - actor/entropy:0.4407426416873932 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23966830968856812 - training/rollout_probs_diff_mean:0.0051012844778597355 - training/rollout_probs_diff_std:0.011158647015690804 - training/rollout_actor_probs_pearson_corr:0.9991984963417053 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.4375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.9999550580978394 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.586863398551941 - rollout_corr/rollout_is_min:0.6212320327758789 - rollout_corr/rollout_is_std:0.03741012141108513 - rollout_corr/rollout_is_eff_sample_size:0.9986023199750391 - rollout_corr/rollout_is_seq_mean:0.9999154806137085 - rollout_corr/rollout_is_seq_std:0.0023052047472447157 - rollout_corr/rollout_is_seq_max:1.0075262784957886 - rollout_corr/rollout_is_seq_min:0.9921414256095886 - rollout_corr/rollout_is_seq_max_deviation:0.007858574390411377 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6121524260379374) - rollout_corr/training_log_ppl:np.float64(0.46336599756614305) - rollout_corr/kl:np.float64(0.0012106716192334943) - rollout_corr/k3_kl:np.float64(0.001044533465915265) - rollout_corr/rollout_ppl:np.float64(1.6101201642304659) - rollout_corr/rollout_log_ppl:np.float64(0.4621553277829662) - rollout_corr/log_ppl_diff:np.float64(0.0012106697831768543) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022202231630217284) - rollout_corr/log_ppl_diff_max:np.float64(0.011315524578094482) - rollout_corr/log_ppl_diff_min:np.float64(-0.007631242275238037) - rollout_corr/ppl_ratio:np.float64(1.0012149058748037) - rollout_corr/chi2_token:np.float64(0.001704103546217084) - rollout_corr/chi2_seq:np.float64(0.4757801443338394) - actor/pg_loss:np.float64(0.006979008438065648) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011709280256582133) - actor/ppo_kl:np.float64(0.00036738665134805615) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4375) - self_distillation/token_reweight_w_mean:np.float64(53386.58389568445) - self_distillation/token_reweight_w_std:np.float64(762098.1771182105) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011509482283145) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1306488252885174) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.4595351442694664) - perf/mfu/actor:np.float64(0.0457439323763552) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.23744583129883) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024807543959468603 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024807543959468603 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:331.8515625 - response_length/max:891.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.8515625 - response_length_non_aborted/max:891.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:592.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.7911500334739685e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6936665810644627) - timing_s/agent_loop/generate_sequences/max:np.float64(7.335243610665202) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.780559276121494) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.335243610665202) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:891 - timing_s/gen:14.717649631202221 - timing_s/reward:0.05497334711253643 - timing_s/old_log_prob:2.9507923033088446 - timing_s/adv:0.4809166081249714 - timing_s/update_actor:21.40921910852194 - timing_s/step:39.70510539226234 - timing_s/stop_profile:0.00011640414595603943 - timing_per_token_ms/update_actor:0.1423693566114853 - timing_per_token_ms/gen:0.1732425739953648 - timing_per_token_ms/adv:0.003198051630723719 - perf/total_num_tokens:150378 - perf/time_per_step:39.70510539226234 - perf/throughput:473.42148608584665 (TaskRunner pid=2255397) Training Progress: 22%|██▏ | 22/100 [21:05<1:21:20, 62.57s/it] (TaskRunner pid=2255397) step:23 - global_seqlen/min:15655 - global_seqlen/max:22837 - global_seqlen/minmax_diff:7182 - global_seqlen/balanced_min:19794 - global_seqlen/balanced_max:19804 - global_seqlen/mean:19802.0 - actor/entropy:0.43892350792884827 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22363799810409546 - training/rollout_probs_diff_mean:0.005129741504788399 - training/rollout_probs_diff_std:0.011249431408941746 - training/rollout_actor_probs_pearson_corr:0.9991777539253235 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.9999939203262329 - rollout_corr/rollout_is_ratio_fraction_high:1.1452129911049269e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1670188903808594 - rollout_corr/rollout_is_min:0.5688050389289856 - rollout_corr/rollout_is_std:0.03843645006418228 - rollout_corr/rollout_is_eff_sample_size:0.9985248186495871 - rollout_corr/rollout_is_seq_mean:0.9999755024909973 - rollout_corr/rollout_is_seq_std:0.0022403132170438766 - rollout_corr/rollout_is_seq_max:1.008310079574585 - rollout_corr/rollout_is_seq_min:0.9935483336448669 - rollout_corr/rollout_is_seq_max_deviation:0.008310079574584961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.585592514835298) - rollout_corr/training_log_ppl:np.float64(0.45206161157693714) - rollout_corr/kl:np.float64(0.0010271025659691624) - rollout_corr/k3_kl:np.float64(0.0009754648472721783) - rollout_corr/rollout_ppl:np.float64(1.583907154854387) - rollout_corr/rollout_log_ppl:np.float64(0.45103450742317364) - rollout_corr/log_ppl_diff:np.float64(0.0010271041537635028) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021158340969122946) - rollout_corr/log_ppl_diff_max:np.float64(0.008697479963302612) - rollout_corr/log_ppl_diff_min:np.float64(-0.007354050874710083) - rollout_corr/ppl_ratio:np.float64(1.0010308369528502) - rollout_corr/chi2_token:np.float64(0.0018195987213402987) - rollout_corr/chi2_seq:np.float64(1.1621388338971883) - actor/pg_loss:np.float64(0.007446382311172783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000959160078309651) - actor/ppo_kl:np.float64(0.00021888582077522756) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(6609.447738413466) - self_distillation/token_reweight_w_std:np.float64(122546.46291852725) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9963779090903699) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17003195008146577) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.43443264812231064) - perf/mfu/actor:np.float64(0.04804480134893495) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.1064910888672) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033225491642951965 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033225491642951965 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:341.09375 - response_length/max:846.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:341.09375 - response_length_non_aborted/max:846.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.71875 - prompt_length/max:766.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010677054524421692 - timing_s/agent_loop/generate_sequences/min:np.float64(1.892102275043726) - timing_s/agent_loop/generate_sequences/max:np.float64(7.085802031680942) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9653804742338252) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.085802031680942) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:224 - timing_s/agent_loop/slowest/response_length:846 - timing_s/gen:13.813347652554512 - timing_s/reward:0.060129137709736824 - timing_s/old_log_prob:2.9367834143340588 - timing_s/adv:0.5313698258250952 - timing_s/update_actor:21.404147198423743 - timing_s/step:38.83369122073054 - timing_s/stop_profile:0.00010784715414047241 - timing_per_token_ms/update_actor:0.1351135440765058 - timing_per_token_ms/gen:0.15819225438106405 - timing_per_token_ms/adv:0.0033542686712522423 - perf/total_num_tokens:158416 - perf/time_per_step:38.83369122073054 - perf/throughput:509.91804738430636 (TaskRunner pid=2255397) Training Progress: 23%|██▎ | 23/100 [21:44<1:11:10, 55.46s/it] (TaskRunner pid=2255397) step:24 - global_seqlen/min:18708 - global_seqlen/max:22807 - global_seqlen/minmax_diff:4099 - global_seqlen/balanced_min:20352 - global_seqlen/balanced_max:20355 - global_seqlen/mean:20353.75 - actor/entropy:0.48541581630706787 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16695591807365417 - training/rollout_probs_diff_mean:0.005245619919151068 - training/rollout_probs_diff_std:0.01105139683932066 - training/rollout_actor_probs_pearson_corr:0.9992536306381226 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:0.9997487664222717 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6379640102386475 - rollout_corr/rollout_is_min:0.55048668384552 - rollout_corr/rollout_is_std:0.038326967507600784 - rollout_corr/rollout_is_eff_sample_size:0.998532425609191 - rollout_corr/rollout_is_seq_mean:0.9996849894523621 - rollout_corr/rollout_is_seq_std:0.0021940250881016254 - rollout_corr/rollout_is_seq_max:1.0061367750167847 - rollout_corr/rollout_is_seq_min:0.9898834228515625 - rollout_corr/rollout_is_seq_max_deviation:0.0101165771484375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6475044381804764) - rollout_corr/training_log_ppl:np.float64(0.4905120711773634) - rollout_corr/kl:np.float64(0.0011423102059779922) - rollout_corr/k3_kl:np.float64(0.0009413222521743592) - rollout_corr/rollout_ppl:np.float64(1.6456106547266245) - rollout_corr/rollout_log_ppl:np.float64(0.48936975869582966) - rollout_corr/log_ppl_diff:np.float64(0.001142312481533736) - rollout_corr/log_ppl_abs_diff:np.float64(0.002205758646596223) - rollout_corr/log_ppl_diff_max:np.float64(0.011655747890472412) - rollout_corr/log_ppl_diff_min:np.float64(-0.006167978048324585) - rollout_corr/ppl_ratio:np.float64(1.0011465374846011) - rollout_corr/chi2_token:np.float64(0.001514944015070796) - rollout_corr/chi2_seq:np.float64(0.9692259337753057) - actor/pg_loss:np.float64(0.006828168756328523) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008421741476922762) - actor/ppo_kl:np.float64(5.722692924337025e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(16169.235121513018) - self_distillation/token_reweight_w_std:np.float64(204586.64157430374) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000086387153715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14809982472797856) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.38066764548420906) - perf/mfu/actor:np.float64(0.04935736118807353) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.85000133514404) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-8.236870780820027e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-8.236870780820027e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:355.6796875 - response_length/max:641.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:355.6796875 - response_length_non_aborted/max:641.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.375 - prompt_length/max:848.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011152960360050201 - timing_s/agent_loop/generate_sequences/min:np.float64(2.046153662726283) - timing_s/agent_loop/generate_sequences/max:np.float64(6.436841234564781) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.186256590895937) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.436841234564781) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:234 - timing_s/agent_loop/slowest/response_length:641 - timing_s/gen:13.05754235945642 - timing_s/reward:0.05888623557984829 - timing_s/old_log_prob:2.9728885516524315 - timing_s/adv:0.5305459387600422 - timing_s/update_actor:21.35632175952196 - timing_s/step:38.063012946397066 - timing_s/stop_profile:0.00010835938155651093 - timing_per_token_ms/update_actor:0.13115716857779255 - timing_per_token_ms/gen:0.14340437937329958 - timing_per_token_ms/adv:0.0032582812673342884 - perf/total_num_tokens:162830 - perf/time_per_step:38.063012946397066 - perf/throughput:534.7382780407726 (TaskRunner pid=2255397) Training Progress: 24%|██▍ | 24/100 [22:22<1:03:39, 50.25s/it] (TaskRunner pid=2255397) step:25 - global_seqlen/min:16381 - global_seqlen/max:25188 - global_seqlen/minmax_diff:8807 - global_seqlen/balanced_min:21088 - global_seqlen/balanced_max:21097 - global_seqlen/mean:21093.125 - actor/entropy:0.4773712158203125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26223766803741455 - training/rollout_probs_diff_mean:0.005165540147572756 - training/rollout_probs_diff_std:0.01117149367928505 - training/rollout_actor_probs_pearson_corr:0.9992371797561646 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9999747276306152 - rollout_corr/rollout_is_ratio_fraction_high:1.0959985047520604e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.094183921813965 - rollout_corr/rollout_is_min:0.5062851309776306 - rollout_corr/rollout_is_std:0.038087502121925354 - rollout_corr/rollout_is_eff_sample_size:0.9985513246511216 - rollout_corr/rollout_is_seq_mean:1.0000536441802979 - rollout_corr/rollout_is_seq_std:0.0022864702623337507 - rollout_corr/rollout_is_seq_max:1.0067065954208374 - rollout_corr/rollout_is_seq_min:0.9937045574188232 - rollout_corr/rollout_is_seq_max_deviation:0.006706595420837402 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.669403042178601) - rollout_corr/training_log_ppl:np.float64(0.4993552331288811) - rollout_corr/kl:np.float64(0.0006479631488460758) - rollout_corr/k3_kl:np.float64(0.0008453664596572708) - rollout_corr/rollout_ppl:np.float64(1.668324178084731) - rollout_corr/rollout_log_ppl:np.float64(0.4987072692310903) - rollout_corr/log_ppl_diff:np.float64(0.0006479638977907598) - rollout_corr/log_ppl_abs_diff:np.float64(0.001830984663683921) - rollout_corr/log_ppl_diff_max:np.float64(0.008455544710159302) - rollout_corr/log_ppl_diff_min:np.float64(-0.007198035717010498) - rollout_corr/ppl_ratio:np.float64(1.0006508629303426) - rollout_corr/chi2_token:np.float64(0.002105830702930689) - rollout_corr/chi2_seq:np.float64(0.6466955123469234) - actor/pg_loss:np.float64(0.007964683580212295) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000689070585394802) - actor/ppo_kl:np.float64(-6.814886182837654e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(26990.30881792493) - self_distillation/token_reweight_w_std:np.float64(421488.5579388592) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002008835086599) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15447656423202716) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4783755913376808) - perf/mfu/actor:np.float64(0.0494590320538676) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.90256881713867) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02120209112763405 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02120209112763405 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:356.41015625 - response_length/max:956.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:356.41015625 - response_length_non_aborted/max:956.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.75 - prompt_length/max:646.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000149555504322052 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6570597048848867) - timing_s/agent_loop/generate_sequences/max:np.float64(7.838871484622359) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9465300647134427) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.838871484622359) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:956 - timing_s/gen:14.78430269099772 - timing_s/reward:0.06230742111802101 - timing_s/old_log_prob:3.0198799911886454 - timing_s/adv:0.5167359504848719 - timing_s/update_actor:22.208827931433916 - timing_s/step:40.68130003660917 - timing_s/stop_profile:0.0001026783138513565 - timing_per_token_ms/update_actor:0.13161176883127745 - timing_per_token_ms/gen:0.16203573712473252 - timing_per_token_ms/adv:0.003062229698568087 - perf/total_num_tokens:168745 - perf/time_per_step:40.68130003660917 - perf/throughput:518.496827314227 (TaskRunner pid=2255397) Training Progress: 25%|██▌ | 25/100 [23:03<59:14, 47.40s/it] (TaskRunner pid=2255397) step:26 - global_seqlen/min:18171 - global_seqlen/max:23062 - global_seqlen/minmax_diff:4891 - global_seqlen/balanced_min:20385 - global_seqlen/balanced_max:20394 - global_seqlen/mean:20391.25 - actor/entropy:0.5068786144256592 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17874199151992798 - training/rollout_probs_diff_mean:0.005472649820148945 - training/rollout_probs_diff_std:0.011403916403651237 - training/rollout_actor_probs_pearson_corr:0.9991965889930725 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.6328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0000735521316528 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.292263525305316e-05 - rollout_corr/rollout_is_max:1.8139365911483765 - rollout_corr/rollout_is_min:0.43659546971321106 - rollout_corr/rollout_is_std:0.039478376507759094 - rollout_corr/rollout_is_eff_sample_size:0.9984440018610561 - rollout_corr/rollout_is_seq_mean:1.0000500679016113 - rollout_corr/rollout_is_seq_std:0.0023560679983347654 - rollout_corr/rollout_is_seq_max:1.0076462030410767 - rollout_corr/rollout_is_seq_min:0.9908225536346436 - rollout_corr/rollout_is_seq_max_deviation:0.009177446365356445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6881171250715852) - rollout_corr/training_log_ppl:np.float64(0.510973722091876) - rollout_corr/kl:np.float64(0.0007455622756609159) - rollout_corr/k3_kl:np.float64(0.0008864333188967066) - rollout_corr/rollout_ppl:np.float64(1.6867859680205584) - rollout_corr/rollout_log_ppl:np.float64(0.5102281617000699) - rollout_corr/log_ppl_diff:np.float64(0.000745560391806066) - rollout_corr/log_ppl_abs_diff:np.float64(0.002050930866971612) - rollout_corr/log_ppl_diff_max:np.float64(0.00910806655883789) - rollout_corr/log_ppl_diff_min:np.float64(-0.006127357482910156) - rollout_corr/ppl_ratio:np.float64(1.0007490692660213) - rollout_corr/chi2_token:np.float64(0.0020302317570894957) - rollout_corr/chi2_seq:np.float64(1.0315312510356307) - actor/pg_loss:np.float64(0.005001060431823134) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048566531631877297) - actor/ppo_kl:np.float64(-2.1595971896104516e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_w_mean:np.float64(13624.06982974615) - self_distillation/token_reweight_w_std:np.float64(206433.80913565995) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992130598984659) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09108548518270254) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.3169581815600395) - perf/mfu/actor:np.float64(0.04962560549786612) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.88824081420898) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027206302620470524 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027206302620470524 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:340.8203125 - response_length/max:592.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:340.8203125 - response_length_non_aborted/max:592.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.40625 - prompt_length/max:709.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016283243894577026 - timing_s/agent_loop/generate_sequences/min:np.float64(2.038805991411209) - timing_s/agent_loop/generate_sequences/max:np.float64(5.906553823500872) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.0381240394635824) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.906553823500872) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:331 - timing_s/agent_loop/slowest/response_length:592 - timing_s/gen:12.749464539811015 - timing_s/reward:0.061460068449378014 - timing_s/old_log_prob:3.0010161008685827 - timing_s/adv:0.5166078116744757 - timing_s/update_actor:21.353253232315183 - timing_s/step:37.770273169502616 - timing_s/stop_profile:0.00010509975254535675 - timing_per_token_ms/update_actor:0.13089715706684965 - timing_per_token_ms/gen:0.1461256680780632 - timing_per_token_ms/adv:0.0031668473712650994 - perf/total_num_tokens:163130 - perf/time_per_step:37.770273169502616 - perf/throughput:539.8756294001282 (TaskRunner pid=2255397) Training Progress: 26%|██▌ | 26/100 [23:40<54:55, 44.53s/it] (TaskRunner pid=2255397) step:27 - global_seqlen/min:17480 - global_seqlen/max:23456 - global_seqlen/minmax_diff:5976 - global_seqlen/balanced_min:20067 - global_seqlen/balanced_max:20076 - global_seqlen/mean:20073.875 - actor/entropy:0.4675213694572449 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105892539024353 - training/rollout_probs_diff_mean:0.005088143516331911 - training/rollout_probs_diff_std:0.010994603857398033 - training/rollout_actor_probs_pearson_corr:0.9992254972457886 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.51171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.9997977018356323 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6167669296264648 - rollout_corr/rollout_is_min:0.5378821492195129 - rollout_corr/rollout_is_std:0.03790003061294556 - rollout_corr/rollout_is_eff_sample_size:0.9985651130827546 - rollout_corr/rollout_is_seq_mean:0.9997107982635498 - rollout_corr/rollout_is_seq_std:0.002149628708139062 - rollout_corr/rollout_is_seq_max:1.007951021194458 - rollout_corr/rollout_is_seq_min:0.9923518896102905 - rollout_corr/rollout_is_seq_max_deviation:0.007951021194458008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6248419084586203) - rollout_corr/training_log_ppl:np.float64(0.47598731669131666) - rollout_corr/kl:np.float64(0.0010493483772009427) - rollout_corr/k3_kl:np.float64(0.0008248598063573809) - rollout_corr/rollout_ppl:np.float64(1.623152432963252) - rollout_corr/rollout_log_ppl:np.float64(0.47493796655908227) - rollout_corr/log_ppl_diff:np.float64(0.0010493501322343946) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018501168815419078) - rollout_corr/log_ppl_diff_max:np.float64(0.007712453603744507) - rollout_corr/log_ppl_diff_min:np.float64(-0.004121840000152588) - rollout_corr/ppl_ratio:np.float64(1.0010521123185754) - rollout_corr/chi2_token:np.float64(0.0011714110150933266) - rollout_corr/chi2_seq:np.float64(0.32961429841816425) - actor/pg_loss:np.float64(0.008322581415995955) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008014981349333539) - actor/ppo_kl:np.float64(4.5028834745330926e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51171875) - self_distillation/token_reweight_w_mean:np.float64(30577.34729726822) - self_distillation/token_reweight_w_std:np.float64(521484.4462188269) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020933467894793) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16031731336261146) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.42759493738412857) - perf/mfu/actor:np.float64(0.048367516590069666) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.80517578125) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014940870460122824 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014940870460122824 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.62109375 - response_length/max:943.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.62109375 - response_length_non_aborted/max:943.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.6875 - prompt_length/max:380.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013620592653751373 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9484890308231115) - timing_s/agent_loop/generate_sequences/max:np.float64(7.838706968352199) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.320610681475955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.838706968352199) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:943 - timing_s/gen:14.69153231009841 - timing_s/reward:0.06106424517929554 - timing_s/old_log_prob:3.0161088667809963 - timing_s/adv:0.5223662443459034 - timing_s/update_actor:21.658704286441207 - timing_s/step:40.0367049369961 - timing_s/stop_profile:0.00010894984006881714 - timing_per_token_ms/update_actor:0.13486873041727873 - timing_per_token_ms/gen:0.15526386090166672 - timing_per_token_ms/adv:0.003252774092856408 - perf/total_num_tokens:160591 - perf/time_per_step:40.0367049369961 - perf/throughput:501.38679073588406 (TaskRunner pid=2255397) Training Progress: 27%|██▋ | 27/100 [24:21<52:33, 43.20s/it] (TaskRunner pid=2255397) step:28 - global_seqlen/min:18201 - global_seqlen/max:23560 - global_seqlen/minmax_diff:5359 - global_seqlen/balanced_min:20410 - global_seqlen/balanced_max:20497 - global_seqlen/mean:20425.125 - actor/entropy:0.4737909138202667 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294958233833313 - training/rollout_probs_diff_mean:0.0052576144225895405 - training/rollout_probs_diff_std:0.011254720389842987 - training/rollout_actor_probs_pearson_corr:0.999194860458374 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0001050233840942 - rollout_corr/rollout_is_ratio_fraction_high:1.0676346391846891e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.8465442657470703 - rollout_corr/rollout_is_min:0.6126700639724731 - rollout_corr/rollout_is_std:0.03862826153635979 - rollout_corr/rollout_is_eff_sample_size:0.9985103182038877 - rollout_corr/rollout_is_seq_mean:1.0001041889190674 - rollout_corr/rollout_is_seq_std:0.002421688986942172 - rollout_corr/rollout_is_seq_max:1.00888991355896 - rollout_corr/rollout_is_seq_min:0.9913090467453003 - rollout_corr/rollout_is_seq_max_deviation:0.008889913558959961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6370238508097827) - rollout_corr/training_log_ppl:np.float64(0.4820473712461535) - rollout_corr/kl:np.float64(0.0006596640439546775) - rollout_corr/k3_kl:np.float64(0.000909588395757055) - rollout_corr/rollout_ppl:np.float64(1.6358301592990756) - rollout_corr/rollout_log_ppl:np.float64(0.48138770982041024) - rollout_corr/log_ppl_diff:np.float64(0.000659661425743252) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018744217813946307) - rollout_corr/log_ppl_diff_max:np.float64(0.007977485656738281) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074624717235565186) - rollout_corr/ppl_ratio:np.float64(1.0006625556852669) - rollout_corr/chi2_token:np.float64(0.0023598361294716597) - rollout_corr/chi2_seq:np.float64(0.9705478034447879) - actor/pg_loss:np.float64(0.006433643866330385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006121892383816885) - actor/ppo_kl:np.float64(-2.0443672603676077e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(22090.04282637965) - self_distillation/token_reweight_w_std:np.float64(363460.48439942) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007038170006126) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12272839152137749) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.4047693833708763) - perf/mfu/actor:np.float64(0.0494901405269842) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.81372833251953) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004289222415536642 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004289222415536642 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:365.87890625 - response_length/max:1022.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:365.87890625 - response_length_non_aborted/max:1022.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.40625 - prompt_length/max:443.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001571252942085266 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0796666648238897) - timing_s/agent_loop/generate_sequences/max:np.float64(8.435203347355127) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.275154207913147) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.435203347355127) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:330 - timing_s/agent_loop/slowest/response_length:1022 - timing_s/gen:15.208884691819549 - timing_s/reward:0.060688696801662445 - timing_s/old_log_prob:3.0850410480052233 - timing_s/adv:0.5393318682909012 - timing_s/update_actor:21.212657935917377 - timing_s/step:40.19403674826026 - timing_s/stop_profile:0.00011958740651607513 - timing_per_token_ms/update_actor:0.1298196335145891 - timing_per_token_ms/gen:0.1623753236728719 - timing_per_token_ms/adv:0.003300664428558584 - perf/total_num_tokens:163401 - perf/time_per_step:40.19403674826026 - perf/throughput:508.1630672710193 (TaskRunner pid=2255397) Training Progress: 28%|██▊ | 28/100 [25:01<50:46, 42.31s/it] (TaskRunner pid=2255397) step:29 - global_seqlen/min:17728 - global_seqlen/max:24689 - global_seqlen/minmax_diff:6961 - global_seqlen/balanced_min:21875 - global_seqlen/balanced_max:21882 - global_seqlen/mean:21878.875 - actor/entropy:0.45051109790802 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25539880990982056 - training/rollout_probs_diff_mean:0.004729867912828922 - training/rollout_probs_diff_std:0.010498874820768833 - training/rollout_actor_probs_pearson_corr:0.9993234872817993 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.9999346733093262 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9196990251657553e-05 - rollout_corr/rollout_is_max:1.6618187427520752 - rollout_corr/rollout_is_min:0.48319488763809204 - rollout_corr/rollout_is_std:0.03575139120221138 - rollout_corr/rollout_is_eff_sample_size:0.998723350704955 - rollout_corr/rollout_is_seq_mean:0.9999228119850159 - rollout_corr/rollout_is_seq_std:0.00180347659625113 - rollout_corr/rollout_is_seq_max:1.0053354501724243 - rollout_corr/rollout_is_seq_min:0.9929618835449219 - rollout_corr/rollout_is_seq_max_deviation:0.007038116455078125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5977162146009505) - rollout_corr/training_log_ppl:np.float64(0.4593473710701801) - rollout_corr/kl:np.float64(0.0006978295002575408) - rollout_corr/k3_kl:np.float64(0.0007481996697720206) - rollout_corr/rollout_ppl:np.float64(1.5965832527726889) - rollout_corr/rollout_log_ppl:np.float64(0.45864954427815974) - rollout_corr/log_ppl_diff:np.float64(0.0006978267920203507) - rollout_corr/log_ppl_abs_diff:np.float64(0.001622283656615764) - rollout_corr/log_ppl_diff_max:np.float64(0.006626904010772705) - rollout_corr/log_ppl_diff_min:np.float64(-0.00486445426940918) - rollout_corr/ppl_ratio:np.float64(1.0007000237237662) - rollout_corr/chi2_token:np.float64(0.001576458802446723) - rollout_corr/chi2_seq:np.float64(0.6520844814367592) - actor/pg_loss:np.float64(0.0060147366020828485) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005360728382584057) - actor/ppo_kl:np.float64(-6.641209190716424e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(26070.862267469056) - self_distillation/token_reweight_w_std:np.float64(478879.7805042105) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025979080237448) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1094071113911923) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.3129398711025715) - perf/mfu/actor:np.float64(0.052564897191348076) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.9274024963379) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01068192534148693 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01068192534148693 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.96484375 - response_length/max:856.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.96484375 - response_length_non_aborted/max:856.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.75 - prompt_length/max:492.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011670216917991638 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0912411771714687) - timing_s/agent_loop/generate_sequences/max:np.float64(7.972104227170348) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.765111505068489) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.972104227170348) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:856 - timing_s/gen:14.771529672667384 - timing_s/reward:0.06256661377847195 - timing_s/old_log_prob:3.17423870973289 - timing_s/adv:0.5342923663556576 - timing_s/update_actor:21.539450408890843 - timing_s/step:40.168734323233366 - timing_s/stop_profile:0.00010610558092594147 - timing_per_token_ms/update_actor:0.12306077442790617 - timing_per_token_ms/gen:0.1417844530553678 - timing_per_token_ms/adv:0.003052558497384221 - perf/total_num_tokens:175031 - perf/time_per_step:40.168734323233366 - perf/throughput:544.6742440013945 (TaskRunner pid=2255397) Training Progress: 29%|██▉ | 29/100 [25:41<49:19, 41.68s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and identify the key functional groups and substituents that must be present in the reactant. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product has the following key features: (TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or similar heterocycle). (TaskRunner pid=2255397) - A benzene ring with a chlorine substituent. (TaskRunner pid=2255397) - A trifluoromethyl group (C(F)(F)F) attached to the benzene ring. (TaskRunner pid=2255397) - A substituent group (likely a nitrogen-containing chain) that connects to the ring. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let's examine each option: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A:** (TaskRunner pid=2255397) O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) This option contains the trifluoromethyl group (C(F)(F)F) and the chlorinated benzene ring. It also contains a nitrogen-containing chain and a ring structure similar to the product. This option closely matches the structure of the product and is a strong candidate. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B:** (TaskRunner pid=2255397) O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) This option has a chlorine substituent on the ring, but it lacks the trifluoromethyl group and the correct nitrogen-containing chain. It also has a different ring structure than the product. Therefore, it is not the correct reactant. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C:** (TaskRunner pid=2255397) O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) This option contains the trifluoromethyl group and the chlorinated benzene ring. However, the structure is more complex and includes an oxygen-containing side chain that is not present in the product. It introduces an unnecessary oxygen group, making it incompatible with the product structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D:** (TaskRunner pid=2255397) O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) This option contains the trifluoromethyl group and a benzene ring. However, it lacks the correct nitrogen-containing chain and the chlorinated benzene ring. The structure is not aligned with the product, so it is not the correct reactant. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Conclusion:** (TaskRunner pid=2255397) Only **Option A** contains all the necessary functional groups and substituents present in the product, making it the only correct reactant for the synthesis of the given product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2255397) step:30 - global_seqlen/min:18031 - global_seqlen/max:24561 - global_seqlen/minmax_diff:6530 - global_seqlen/balanced_min:20880 - global_seqlen/balanced_max:20888 - global_seqlen/mean:20884.625 - actor/entropy:0.4658919870853424 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3171748220920563 - training/rollout_probs_diff_mean:0.005262547638267279 - training/rollout_probs_diff_std:0.01141402218490839 - training/rollout_actor_probs_pearson_corr:0.9991620779037476 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.44921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:0.9999656677246094 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.412619844311848e-05 - rollout_corr/rollout_is_max:1.9128998517990112 - rollout_corr/rollout_is_min:0.2724512219429016 - rollout_corr/rollout_is_std:0.03873920068144798 - rollout_corr/rollout_is_eff_sample_size:0.9985014042044956 - rollout_corr/rollout_is_seq_mean:1.0000505447387695 - rollout_corr/rollout_is_seq_std:0.002504864940419793 - rollout_corr/rollout_is_seq_max:1.0067648887634277 - rollout_corr/rollout_is_seq_min:0.9914875626564026 - rollout_corr/rollout_is_seq_max_deviation:0.008512437343597412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6606428353115916) - rollout_corr/training_log_ppl:np.float64(0.49546970275696367) - rollout_corr/kl:np.float64(0.0007864780202773147) - rollout_corr/k3_kl:np.float64(0.000911510776063551) - rollout_corr/rollout_ppl:np.float64(1.659348614513874) - rollout_corr/rollout_log_ppl:np.float64(0.4946832267160062) - rollout_corr/log_ppl_diff:np.float64(0.0007864760409574956) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019896051089745015) - rollout_corr/log_ppl_diff_max:np.float64(0.009873658418655396) - rollout_corr/log_ppl_diff_min:np.float64(-0.007493495941162109) - rollout_corr/ppl_ratio:np.float64(1.0007900123018771) - rollout_corr/chi2_token:np.float64(0.002100849524140358) - rollout_corr/chi2_seq:np.float64(0.6980828407686204) - actor/pg_loss:np.float64(0.0044937689090147614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004526598731899867) - actor/ppo_kl:np.float64(2.1002829232230624e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44921875) - self_distillation/token_reweight_w_mean:np.float64(16658.25212058681) - self_distillation/token_reweight_w_std:np.float64(262596.2304097906) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002752567641437) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09863085992401466) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.3747030645608902) - perf/mfu/actor:np.float64(0.050935440136856504) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.77979278564453) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.606845238095238) - val-aux/sciknoweval/reward/std@16:np.float64(0.2138942013523591) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6934285714285715) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.16904326809342193) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.521195238095238) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1926339687939083) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6076809523809524) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21426439691535565) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7574380952380952) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11875313696867805) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4448476190476191) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15604155437402734) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6224000000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16648090775602947) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8030238095238095) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07678824143757296) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.38081904761904767) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11615050215883854) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6278476190476191) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.11644569510751888) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8329428571428572) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04605969898285256) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3325333333333333) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07403573137391646) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6297142857142858) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.079464901369361) - val-aux/sciknoweval/score/mean@16:np.float64(0.606845238095238) - val-aux/sciknoweval/score/std@16:np.float64(0.2138942013523591) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6934285714285715) - val-aux/sciknoweval/score/best@2/std:np.float64(0.16904326809342193) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.521195238095238) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1926339687939083) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6076809523809524) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21426439691535565) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7574380952380952) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11875313696867805) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4448476190476191) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15604155437402734) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6224000000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.16648090775602947) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8030238095238095) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07678824143757296) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.38081904761904767) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11615050215883854) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6278476190476191) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.11644569510751888) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8329428571428572) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04605969898285256) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3325333333333333) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07403573137391646) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6297142857142858) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.079464901369361) - val-core/sciknoweval/acc/mean@16:np.float64(0.606845238095238) - val-aux/sciknoweval/acc/std@16:np.float64(0.2138942013523591) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6934285714285715) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.16904326809342193) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.521195238095238) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1926339687939083) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6076809523809524) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21426439691535565) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7574380952380952) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11875313696867805) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4448476190476191) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15604155437402734) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6224000000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.16648090775602947) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8030238095238095) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07678824143757296) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.38081904761904767) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11615050215883854) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6278476190476191) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.11644569510751888) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8329428571428572) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04605969898285256) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3325333333333333) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07403573137391646) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6297142857142858) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.079464901369361) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019352398812770844 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0019352398812770844 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:343.39453125 - response_length/max:941.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.39453125 - response_length_non_aborted/max:941.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.25 - prompt_length/max:516.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015097111463546753 - timing_s/agent_loop/generate_sequences/min:np.float64(1.869465732946992) - timing_s/agent_loop/generate_sequences/max:np.float64(7.8817349802702665) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.971169372940494) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.8817349802702665) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:235 - timing_s/agent_loop/slowest/response_length:941 - timing_s/gen:15.309523962438107 - timing_s/reward:0.06142178550362587 - timing_s/old_log_prob:3.0037817489355803 - timing_s/adv:0.5514516998082399 - timing_s/update_actor:21.344387443736196 - timing_s/step:40.35734134726226 - timing_s/testing:78.26906821876764 - timing_s/save_checkpoint:13.220651166513562 - timing_s/stop_profile:0.00014671683311462402 - timing_per_token_ms/update_actor:0.12775179973147827 - timing_per_token_ms/gen:0.17415195216005308 - timing_per_token_ms/adv:0.0033005841606459294 - perf/total_num_tokens:167077 - perf/time_per_step:40.35734134726226 - perf/throughput:517.4925875392621 (TaskRunner pid=2255397) Training Progress: 30%|███ | 30/100 [27:53<1:20:12, 68.76s/it] (TaskRunner pid=2255397) step:31 - global_seqlen/min:19249 - global_seqlen/max:25029 - global_seqlen/minmax_diff:5780 - global_seqlen/balanced_min:21607 - global_seqlen/balanced_max:21619 - global_seqlen/mean:21615.375 - actor/entropy:0.4569137990474701 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.303112655878067 - training/rollout_probs_diff_mean:0.004967319313436747 - training/rollout_probs_diff_std:0.010901066474616528 - training/rollout_actor_probs_pearson_corr:0.9992502331733704 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000017762184143 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.816590428352356 - rollout_corr/rollout_is_min:0.5354147553443909 - rollout_corr/rollout_is_std:0.03712382912635803 - rollout_corr/rollout_is_eff_sample_size:0.9986238370470686 - rollout_corr/rollout_is_seq_mean:1.0000417232513428 - rollout_corr/rollout_is_seq_std:0.002106655156239867 - rollout_corr/rollout_is_seq_max:1.0081474781036377 - rollout_corr/rollout_is_seq_min:0.9948439002037048 - rollout_corr/rollout_is_seq_max_deviation:0.008147478103637695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6235803924500942) - rollout_corr/training_log_ppl:np.float64(0.4749105938244611) - rollout_corr/kl:np.float64(0.0008353461136003659) - rollout_corr/k3_kl:np.float64(0.00080985213742224) - rollout_corr/rollout_ppl:np.float64(1.6222249972634017) - rollout_corr/rollout_log_ppl:np.float64(0.4740752446523402) - rollout_corr/log_ppl_diff:np.float64(0.0008353491721209139) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017292899719905108) - rollout_corr/log_ppl_diff_max:np.float64(0.006912916898727417) - rollout_corr/log_ppl_diff_min:np.float64(-0.0048788487911224365) - rollout_corr/ppl_ratio:np.float64(1.0008378000929952) - rollout_corr/chi2_token:np.float64(0.0016067922115325928) - rollout_corr/chi2_seq:np.float64(0.6984529327601194) - actor/pg_loss:np.float64(0.005961017799563706) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028992991656195954) - actor/ppo_kl:np.float64(0.0001396356336407223) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(9105.852561026346) - self_distillation/token_reweight_w_std:np.float64(160757.80174179428) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988879463635385) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1225835490913596) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.33824220299720764) - perf/mfu/actor:np.float64(0.05203885094538403) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.26508331298828) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001895678462460637 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001895678462460637 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:395.63671875 - response_length/max:1088.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:395.63671875 - response_length_non_aborted/max:1088.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.84375 - prompt_length/max:635.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.687268614768982e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6187728364020586) - timing_s/agent_loop/generate_sequences/max:np.float64(9.020029896870255) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.584768897177128) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.020029896870255) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:194 - timing_s/agent_loop/slowest/response_length:1088 - timing_s/gen:15.492790538817644 - timing_s/reward:0.05757281742990017 - timing_s/old_log_prob:3.137273196130991 - timing_s/adv:0.5795937422662973 - timing_s/update_actor:21.805233089253306 - timing_s/step:41.175627982243896 - timing_s/stop_profile:0.00012617185711860657 - timing_per_token_ms/update_actor:0.12609793427857086 - timing_per_token_ms/gen:0.15296535982166448 - timing_per_token_ms/adv:0.00335174466245842 - perf/total_num_tokens:172923 - perf/time_per_step:41.175627982243896 - perf/throughput:524.9555637456499 (TaskRunner pid=2255397) Training Progress: 31%|███ | 31/100 [28:34<1:09:34, 60.50s/it] (TaskRunner pid=2255397) step:32 - global_seqlen/min:17172 - global_seqlen/max:25152 - global_seqlen/minmax_diff:7980 - global_seqlen/balanced_min:20878 - global_seqlen/balanced_max:20886 - global_seqlen/mean:20883.0 - actor/entropy:0.4830181300640106 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3013932406902313 - training/rollout_probs_diff_mean:0.00495893694460392 - training/rollout_probs_diff_std:0.010702594183385372 - training/rollout_actor_probs_pearson_corr:0.9992987513542175 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.55078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001959800720215 - rollout_corr/rollout_is_ratio_fraction_high:1.0022450624092016e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0067350962781347e-05 - rollout_corr/rollout_is_max:2.244924545288086 - rollout_corr/rollout_is_min:0.39147886633872986 - rollout_corr/rollout_is_std:0.037390995770692825 - rollout_corr/rollout_is_eff_sample_size:0.998604459752836 - rollout_corr/rollout_is_seq_mean:1.0002524852752686 - rollout_corr/rollout_is_seq_std:0.0021919296123087406 - rollout_corr/rollout_is_seq_max:1.0080853700637817 - rollout_corr/rollout_is_seq_min:0.9943554401397705 - rollout_corr/rollout_is_seq_max_deviation:0.008085370063781738 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6529409568756819) - rollout_corr/training_log_ppl:np.float64(0.4933903770870529) - rollout_corr/kl:np.float64(0.0006465979992356097) - rollout_corr/k3_kl:np.float64(0.0008415120931317688) - rollout_corr/rollout_ppl:np.float64(1.651906920131296) - rollout_corr/rollout_log_ppl:np.float64(0.49274377603433095) - rollout_corr/log_ppl_diff:np.float64(0.0006466010527219623) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017735535220708698) - rollout_corr/log_ppl_diff_max:np.float64(0.007363229990005493) - rollout_corr/log_ppl_diff_min:np.float64(-0.010225236415863037) - rollout_corr/ppl_ratio:np.float64(1.000649468274787) - rollout_corr/chi2_token:np.float64(0.0021671864669770002) - rollout_corr/chi2_seq:np.float64(1.2125774417072535) - actor/pg_loss:np.float64(0.007539922837167978) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005248926790955011) - actor/ppo_kl:np.float64(0.00015378757747441796) - actor/pg_clipfrac_lower:np.float64(1.2849506674683653e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55078125) - self_distillation/token_reweight_w_mean:np.float64(28962.190404822817) - self_distillation/token_reweight_w_std:np.float64(594865.4367179495) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994088341481984) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15152001177193597) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.4039623737335205) - perf/mfu/actor:np.float64(0.050354520500870484) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.79799795150757) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005901970434933901 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.005901970434933901 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:389.75 - response_length/max:1026.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:389.75 - response_length_non_aborted/max:1026.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.84375 - prompt_length/max:404.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001097116619348526 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7604525182396173) - timing_s/agent_loop/generate_sequences/max:np.float64(8.474037919193506) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.519642143648525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.474037919193506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:1026 - timing_s/gen:15.50579597428441 - timing_s/reward:0.05374091491103172 - timing_s/old_log_prob:3.0371324717998505 - timing_s/adv:0.5603868458420038 - timing_s/update_actor:21.465026825666428 - timing_s/step:40.65800975076854 - timing_s/stop_profile:0.0001172088086605072 - timing_per_token_ms/update_actor:0.12848385544262336 - timing_per_token_ms/gen:0.1554060693381616 - timing_per_token_ms/adv:0.0033543243657640416 - perf/total_num_tokens:167064 - perf/time_per_step:40.65800975076854 - perf/throughput:513.6257315105114 (TaskRunner pid=2255397) Training Progress: 32%|███▏ | 32/100 [29:15<1:01:50, 54.56s/it] (TaskRunner pid=2255397) step:33 - global_seqlen/min:16273 - global_seqlen/max:24765 - global_seqlen/minmax_diff:8492 - global_seqlen/balanced_min:20358 - global_seqlen/balanced_max:20368 - global_seqlen/mean:20365.75 - actor/entropy:0.5308820605278015 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2607361674308777 - training/rollout_probs_diff_mean:0.005311037413775921 - training/rollout_probs_diff_std:0.010893904604017735 - training/rollout_actor_probs_pearson_corr:0.9993165731430054 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:2.2798258214606903e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1399129107303452e-05 - rollout_corr/rollout_is_max:2.4295754432678223 - rollout_corr/rollout_is_min:0.4068605899810791 - rollout_corr/rollout_is_std:0.038521647453308105 - rollout_corr/rollout_is_eff_sample_size:0.9985184003675729 - rollout_corr/rollout_is_seq_mean:1.0000193119049072 - rollout_corr/rollout_is_seq_std:0.0023478458169847727 - rollout_corr/rollout_is_seq_max:1.0072718858718872 - rollout_corr/rollout_is_seq_min:0.9926888346672058 - rollout_corr/rollout_is_seq_max_deviation:0.0073111653327941895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7451868052594364) - rollout_corr/training_log_ppl:np.float64(0.5472010709927417) - rollout_corr/kl:np.float64(0.000750623744014689) - rollout_corr/k3_kl:np.float64(0.0009170059764755933) - rollout_corr/rollout_ppl:np.float64(1.7438560076989233) - rollout_corr/rollout_log_ppl:np.float64(0.5464504445553757) - rollout_corr/log_ppl_diff:np.float64(0.0007506264373660088) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020804237574338913) - rollout_corr/log_ppl_diff_max:np.float64(0.011185050010681152) - rollout_corr/log_ppl_diff_min:np.float64(-0.008495688438415527) - rollout_corr/ppl_ratio:np.float64(1.0007542204111814) - rollout_corr/chi2_token:np.float64(0.0021686023101210594) - rollout_corr/chi2_seq:np.float64(1.0599001871887594) - actor/pg_loss:np.float64(0.0046818595146760345) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000626342507530353) - actor/ppo_kl:np.float64(-7.711236621688045e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(406.9383929811884) - self_distillation/token_reweight_w_std:np.float64(6888.8986393570085) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006252727471292) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10766920002060942) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.35059650987386703) - perf/mfu/actor:np.float64(0.04906215016751242) - perf/max_memory_allocated_gb:np.float64(130.98335647583008) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.48283004760742) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038529057055711746 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038529057055711746 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:342.6796875 - response_length/max:820.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:342.6796875 - response_length_non_aborted/max:820.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.75 - prompt_length/max:474.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011102668941020966 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7211053427308798) - timing_s/agent_loop/generate_sequences/max:np.float64(6.794723700731993) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.86705601822905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.794723700731993) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:378 - timing_s/agent_loop/slowest/response_length:820 - timing_s/gen:13.564623160287738 - timing_s/reward:0.06091204658150673 - timing_s/old_log_prob:2.9387052599340677 - timing_s/adv:0.5545471422374249 - timing_s/update_actor:21.675176206976175 - timing_s/step:38.87970031797886 - timing_s/stop_profile:3.417767584323883e-05 - timing_per_token_ms/update_actor:0.1330369382847193 - timing_per_token_ms/gen:0.15462489068563182 - timing_per_token_ms/adv:0.0034036749336350544 - perf/total_num_tokens:162926 - perf/time_per_step:38.87970031797886 - perf/throughput:523.8144798812251 (TaskRunner pid=2255397) Training Progress: 33%|███▎ | 33/100 [29:54<55:40, 49.86s/it] (TaskRunner pid=2255397) step:34 - global_seqlen/min:19407 - global_seqlen/max:28954 - global_seqlen/minmax_diff:9547 - global_seqlen/balanced_min:20764 - global_seqlen/balanced_max:28104 - global_seqlen/mean:21689.625 - actor/entropy:0.4711659848690033 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26104798913002014 - training/rollout_probs_diff_mean:0.004729302600026131 - training/rollout_probs_diff_std:0.010431492701172829 - training/rollout_actor_probs_pearson_corr:0.9993188977241516 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.9998348355293274 - rollout_corr/rollout_is_ratio_fraction_high:1.0172008842346258e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.372540235519409 - rollout_corr/rollout_is_min:0.5585512518882751 - rollout_corr/rollout_is_std:0.03565623611211777 - rollout_corr/rollout_is_eff_sample_size:0.9987297716220098 - rollout_corr/rollout_is_seq_mean:0.9998084902763367 - rollout_corr/rollout_is_seq_std:0.0021642311476171017 - rollout_corr/rollout_is_seq_max:1.0065914392471313 - rollout_corr/rollout_is_seq_min:0.9937243461608887 - rollout_corr/rollout_is_seq_max_deviation:0.006591439247131348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.732406364288181) - rollout_corr/training_log_ppl:np.float64(0.53354305683024) - rollout_corr/kl:np.float64(0.0010742717632254706) - rollout_corr/k3_kl:np.float64(0.0007844316083662761) - rollout_corr/rollout_ppl:np.float64(1.7304212134331465) - rollout_corr/rollout_log_ppl:np.float64(0.5324687818174425) - rollout_corr/log_ppl_diff:np.float64(0.0010742750127974432) - rollout_corr/log_ppl_abs_diff:np.float64(0.002004527148528723) - rollout_corr/log_ppl_diff_max:np.float64(0.010212719440460205) - rollout_corr/log_ppl_diff_min:np.float64(-0.0042626261711120605) - rollout_corr/ppl_ratio:np.float64(1.0010776109993458) - rollout_corr/chi2_token:np.float64(0.000989705789834261) - rollout_corr/chi2_seq:np.float64(0.5540260763373226) - actor/pg_loss:np.float64(0.005981583381071687) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004929028409605962) - actor/ppo_kl:np.float64(0.00015171869483765477) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(11402.142899290891) - self_distillation/token_reweight_w_std:np.float64(223975.93910462921) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007735511753708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13933006773004308) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.3575190082192421) - perf/mfu/actor:np.float64(0.04880687442998337) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.40957641601562) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.05439735949039459 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.05439735949039459 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:384.01953125 - response_length/max:8192.0 - response_length/min:160.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:384.01953125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:293.78125 - prompt_length/max:621.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010099261999130249 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5427038352936506) - timing_s/agent_loop/generate_sequences/max:np.float64(57.38408918492496) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.160430314703262) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(57.38408918492496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:438 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:64.30811233632267 - timing_s/reward:0.06254505552351475 - timing_s/old_log_prob:3.2206109445542097 - timing_s/adv:0.5401895362883806 - timing_s/update_actor:23.241615699604154 - timing_s/step:91.45890662260354 - timing_s/stop_profile:2.9904767870903015e-05 - timing_per_token_ms/update_actor:0.1339443149639756 - timing_per_token_ms/gen:0.6541426760146342 - timing_per_token_ms/adv:0.003113179321267545 - perf/total_num_tokens:173517 - perf/time_per_step:91.45890662260354 - perf/throughput:237.15158863094845 (TaskRunner pid=2255397) Training Progress: 34%|███▍ | 34/100 [31:25<1:08:35, 62.35s/it] (TaskRunner pid=2255397) step:35 - global_seqlen/min:19463 - global_seqlen/max:21625 - global_seqlen/minmax_diff:2162 - global_seqlen/balanced_min:20560 - global_seqlen/balanced_max:20574 - global_seqlen/mean:20569.125 - actor/entropy:0.4701376259326935 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27064740657806396 - training/rollout_probs_diff_mean:0.00488562835380435 - training/rollout_probs_diff_std:0.010647308081388474 - training/rollout_actor_probs_pearson_corr:0.999289870262146 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:0.9999282956123352 - rollout_corr/rollout_is_ratio_fraction_high:1.0713864867284428e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1437816619873047 - rollout_corr/rollout_is_min:0.5562102794647217 - rollout_corr/rollout_is_std:0.036710504442453384 - rollout_corr/rollout_is_eff_sample_size:0.9986539149503298 - rollout_corr/rollout_is_seq_mean:0.9999589323997498 - rollout_corr/rollout_is_seq_std:0.0022408063523471355 - rollout_corr/rollout_is_seq_max:1.0093075037002563 - rollout_corr/rollout_is_seq_min:0.9939238429069519 - rollout_corr/rollout_is_seq_max_deviation:0.009307503700256348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.643273327499628) - rollout_corr/training_log_ppl:np.float64(0.48507256753509864) - rollout_corr/kl:np.float64(0.0008028178892658389) - rollout_corr/k3_kl:np.float64(0.0007829796426221947) - rollout_corr/rollout_ppl:np.float64(1.641896395944059) - rollout_corr/rollout_log_ppl:np.float64(0.4842697436106391) - rollout_corr/log_ppl_diff:np.float64(0.000802823924459517) - rollout_corr/log_ppl_abs_diff:np.float64(0.002036678488366306) - rollout_corr/log_ppl_diff_max:np.float64(0.008617639541625977) - rollout_corr/log_ppl_diff_min:np.float64(-0.010091245174407959) - rollout_corr/ppl_ratio:np.float64(1.0008062841370702) - rollout_corr/chi2_token:np.float64(0.0015425898600369692) - rollout_corr/chi2_seq:np.float64(1.5650525961536914) - actor/pg_loss:np.float64(0.0024570953100919724) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004893736586382147) - actor/ppo_kl:np.float64(4.371891989851662e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(28164.23297266336) - self_distillation/token_reweight_w_std:np.float64(438345.5305433142) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0024336164351553) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09937885942053981) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2858409211039543) - perf/mfu/actor:np.float64(0.04990645011261699) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.3169288635254) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004351168405264616 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004351168405264616 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:364.59765625 - response_length/max:846.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:364.59765625 - response_length_non_aborted/max:846.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.1875 - prompt_length/max:514.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.146092295646667e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5682779010385275) - timing_s/agent_loop/generate_sequences/max:np.float64(7.532088892534375) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.356410696163948) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.532088892534375) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:216 - timing_s/agent_loop/slowest/response_length:846 - timing_s/gen:14.197907466441393 - timing_s/reward:0.05897785350680351 - timing_s/old_log_prob:3.0003139451146126 - timing_s/adv:0.5386089980602264 - timing_s/update_actor:21.315383506938815 - timing_s/step:39.194119876250625 - timing_s/stop_profile:0.00012629106640815735 - timing_per_token_ms/update_actor:0.12953506473257137 - timing_per_token_ms/gen:0.15211446121518146 - timing_per_token_ms/adv:0.003273164257474652 - perf/total_num_tokens:164553 - perf/time_per_step:39.194119876250625 - perf/throughput:524.8012983821 (TaskRunner pid=2255397) Training Progress: 35%|███▌ | 35/100 [32:05<1:00:02, 55.42s/it] (TaskRunner pid=2255397) step:36 - global_seqlen/min:15946 - global_seqlen/max:23368 - global_seqlen/minmax_diff:7422 - global_seqlen/balanced_min:21144 - global_seqlen/balanced_max:21148 - global_seqlen/mean:21146.25 - actor/entropy:0.4903087913990021 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462920188903809 - training/rollout_probs_diff_mean:0.005082453601062298 - training/rollout_probs_diff_std:0.010714489035308361 - training/rollout_actor_probs_pearson_corr:0.9993000030517578 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0000641345977783 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6886519193649292 - rollout_corr/rollout_is_min:0.6301243901252747 - rollout_corr/rollout_is_std:0.037293627858161926 - rollout_corr/rollout_is_eff_sample_size:0.9986113546547832 - rollout_corr/rollout_is_seq_mean:1.000097632408142 - rollout_corr/rollout_is_seq_std:0.002074406947940588 - rollout_corr/rollout_is_seq_max:1.0062254667282104 - rollout_corr/rollout_is_seq_min:0.9938981533050537 - rollout_corr/rollout_is_seq_max_deviation:0.006225466728210449 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6528221243061125) - rollout_corr/training_log_ppl:np.float64(0.4915998738142662) - rollout_corr/kl:np.float64(0.0008901074607035042) - rollout_corr/k3_kl:np.float64(0.000780510345350649) - rollout_corr/rollout_ppl:np.float64(1.6513647134415805) - rollout_corr/rollout_log_ppl:np.float64(0.49070976028451696) - rollout_corr/log_ppl_diff:np.float64(0.0008901135297492146) - rollout_corr/log_ppl_abs_diff:np.float64(0.001717470120638609) - rollout_corr/log_ppl_diff_max:np.float64(0.006238460540771484) - rollout_corr/log_ppl_diff_min:np.float64(-0.005200505256652832) - rollout_corr/ppl_ratio:np.float64(1.0008924398571253) - rollout_corr/chi2_token:np.float64(0.0013305631000548601) - rollout_corr/chi2_seq:np.float64(0.19096036697737873) - actor/pg_loss:np.float64(0.0061755290953442454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007742183429400029) - actor/ppo_kl:np.float64(0.0002702638142206837) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(5297.740719528403) - self_distillation/token_reweight_w_std:np.float64(105322.54200576013) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011047541629523) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1423137153033167) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.3517342582345009) - perf/mfu/actor:np.float64(0.051460493101783154) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.31007385253906) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027716271579265594 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027716271579265594 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.8203125 - response_length/max:718.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.8203125 - response_length_non_aborted/max:718.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.0 - prompt_length/max:704.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014107488095760345 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2529861833900213) - timing_s/agent_loop/generate_sequences/max:np.float64(7.170260231941938) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.674387803606805) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.170260231941938) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:682 - timing_s/gen:14.726460101082921 - timing_s/reward:0.061883002519607544 - timing_s/old_log_prob:2.9912548549473286 - timing_s/adv:0.5396729540079832 - timing_s/update_actor:21.34972732514143 - timing_s/step:39.758028803393245 - timing_s/stop_profile:0.0001045670360326767 - timing_per_token_ms/update_actor:0.12620279792600006 - timing_per_token_ms/gen:0.15026693435932859 - timing_per_token_ms/adv:0.0031901220902523096 - perf/total_num_tokens:169170 - perf/time_per_step:39.758028803393245 - perf/throughput:531.8737029084103 (TaskRunner pid=2255397) Training Progress: 36%|███▌ | 36/100 [32:44<54:07, 50.74s/it] (TaskRunner pid=2255397) step:37 - global_seqlen/min:18794 - global_seqlen/max:24933 - global_seqlen/minmax_diff:6139 - global_seqlen/balanced_min:22097 - global_seqlen/balanced_max:22101 - global_seqlen/mean:22100.0 - actor/entropy:0.4772471785545349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8962392210960388 - training/rollout_probs_diff_mean:0.004988708533346653 - training/rollout_probs_diff_std:0.011156635358929634 - training/rollout_actor_probs_pearson_corr:0.9992755651473999 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999436140060425 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.1014806154416874e-05 - rollout_corr/rollout_is_max:1.5039196014404297 - rollout_corr/rollout_is_min:0.0692535936832428 - rollout_corr/rollout_is_std:0.03678025305271149 - rollout_corr/rollout_is_eff_sample_size:0.9986488027678234 - rollout_corr/rollout_is_seq_mean:0.9999807476997375 - rollout_corr/rollout_is_seq_std:0.002165940823033452 - rollout_corr/rollout_is_seq_max:1.0075201988220215 - rollout_corr/rollout_is_seq_min:0.9936956763267517 - rollout_corr/rollout_is_seq_max_deviation:0.007520198822021484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6701486743986607) - rollout_corr/training_log_ppl:np.float64(0.5012833332875744) - rollout_corr/kl:np.float64(0.0008904281232275935) - rollout_corr/k3_kl:np.float64(0.0009389091193838794) - rollout_corr/rollout_ppl:np.float64(1.6685718595981598) - rollout_corr/rollout_log_ppl:np.float64(0.5003929063677788) - rollout_corr/log_ppl_diff:np.float64(0.0008904269197955728) - rollout_corr/log_ppl_abs_diff:np.float64(0.001979760592803359) - rollout_corr/log_ppl_diff_max:np.float64(0.02197730541229248) - rollout_corr/log_ppl_diff_min:np.float64(-0.00723642110824585) - rollout_corr/ppl_ratio:np.float64(1.0008947688620538) - rollout_corr/chi2_token:np.float64(0.0017904054839164019) - rollout_corr/chi2_seq:np.float64(0.7441312682349235) - actor/pg_loss:np.float64(0.0051974470261484385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006028023444741848) - actor/ppo_kl:np.float64(0.00011284644474807148) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(17486.880942083197) - self_distillation/token_reweight_w_std:np.float64(248698.93032729038) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008525024168193) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10978465108200908) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.33661027252674103) - perf/mfu/actor:np.float64(0.053537470413758295) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.20091247558594) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00370756140910089 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00370756140910089 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:377.84375 - response_length/max:858.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:377.84375 - response_length_non_aborted/max:858.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.78125 - prompt_length/max:1009.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010385364294052124 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9452894497662783) - timing_s/agent_loop/generate_sequences/max:np.float64(7.737249631434679) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.423385596266598) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.737249631434679) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:858 - timing_s/gen:14.69729588367045 - timing_s/reward:0.05707615241408348 - timing_s/old_log_prob:3.1259428039193153 - timing_s/adv:0.5428317990154028 - timing_s/update_actor:21.357854707166553 - timing_s/step:39.88826394826174 - timing_s/stop_profile:0.00012905150651931763 - timing_per_token_ms/update_actor:0.1208023456287701 - timing_per_token_ms/gen:0.1519445856801593 - timing_per_token_ms/adv:0.00307031560529074 - perf/total_num_tokens:176800 - perf/time_per_step:39.88826394826174 - perf/throughput:554.0476774989622 (TaskRunner pid=2255397) Training Progress: 37%|███▋ | 37/100 [33:24<49:52, 47.50s/it] (TaskRunner pid=2255397) step:38 - global_seqlen/min:17401 - global_seqlen/max:23183 - global_seqlen/minmax_diff:5782 - global_seqlen/balanced_min:20614 - global_seqlen/balanced_max:20619 - global_seqlen/mean:20617.125 - actor/entropy:0.4591502249240875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21379658579826355 - training/rollout_probs_diff_mean:0.005224332213401794 - training/rollout_probs_diff_std:0.011259016580879688 - training/rollout_actor_probs_pearson_corr:0.9992035627365112 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001171827316284 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0498313460848294e-05 - rollout_corr/rollout_is_max:1.9823302030563354 - rollout_corr/rollout_is_min:0.3327290415763855 - rollout_corr/rollout_is_std:0.03824835270643234 - rollout_corr/rollout_is_eff_sample_size:0.9985396761004448 - rollout_corr/rollout_is_seq_mean:1.0001174211502075 - rollout_corr/rollout_is_seq_std:0.002267003059387207 - rollout_corr/rollout_is_seq_max:1.007875919342041 - rollout_corr/rollout_is_seq_min:0.9941402077674866 - rollout_corr/rollout_is_seq_max_deviation:0.007875919342041016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.597476608119905) - rollout_corr/training_log_ppl:np.float64(0.4596842197352089) - rollout_corr/kl:np.float64(0.0007120158846873892) - rollout_corr/k3_kl:np.float64(0.0008242313452342387) - rollout_corr/rollout_ppl:np.float64(1.5963355526328087) - rollout_corr/rollout_log_ppl:np.float64(0.45897220366168767) - rollout_corr/log_ppl_diff:np.float64(0.0007120160735212266) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018952106474898756) - rollout_corr/log_ppl_diff_max:np.float64(0.006997823715209961) - rollout_corr/log_ppl_diff_min:np.float64(-0.006651520729064941) - rollout_corr/ppl_ratio:np.float64(1.0007150201126933) - rollout_corr/chi2_token:np.float64(0.001860827673226595) - rollout_corr/chi2_seq:np.float64(0.6607308345846832) - actor/pg_loss:np.float64(0.003047003992833197) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027509686378834886) - actor/ppo_kl:np.float64(6.150551095629453e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(24324.67800112092) - self_distillation/token_reweight_w_std:np.float64(319731.90878068085) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001283828169107) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07227780664106831) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.25013007409870625) - perf/mfu/actor:np.float64(0.05036454480977212) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.1811408996582) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011094711953774095 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011094711953774095 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:381.12890625 - response_length/max:764.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.12890625 - response_length_non_aborted/max:764.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.15625 - prompt_length/max:500.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010602176189422607 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6879636067897081) - timing_s/agent_loop/generate_sequences/max:np.float64(6.8860987443476915) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.44739578160079) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.8860987443476915) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:335 - timing_s/agent_loop/slowest/response_length:743 - timing_s/gen:14.023487392812967 - timing_s/reward:0.054636452347040176 - timing_s/old_log_prob:2.9677660800516605 - timing_s/adv:0.5456443149596453 - timing_s/update_actor:21.413069240748882 - timing_s/step:39.11168333515525 - timing_s/stop_profile:0.00010557286441326141 - timing_per_token_ms/update_actor:0.1298257470473507 - timing_per_token_ms/gen:0.14372892407232796 - timing_per_token_ms/adv:0.003308198372467338 - perf/total_num_tokens:164937 - perf/time_per_step:39.11168333515525 - perf/throughput:527.1346882037278 (TaskRunner pid=2255397) Training Progress: 38%|███▊ | 38/100 [34:04<46:30, 45.01s/it] (TaskRunner pid=2255397) step:39 - global_seqlen/min:16098 - global_seqlen/max:24192 - global_seqlen/minmax_diff:8094 - global_seqlen/balanced_min:21469 - global_seqlen/balanced_max:21484 - global_seqlen/mean:21479.5 - actor/entropy:0.4425225853919983 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17422929406166077 - training/rollout_probs_diff_mean:0.004969022236764431 - training/rollout_probs_diff_std:0.010954358614981174 - training/rollout_actor_probs_pearson_corr:0.9992186427116394 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0003507137298584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6815071105957031 - rollout_corr/rollout_is_min:0.5040706396102905 - rollout_corr/rollout_is_std:0.03746902570128441 - rollout_corr/rollout_is_eff_sample_size:0.9985991103255378 - rollout_corr/rollout_is_seq_mean:1.0004005432128906 - rollout_corr/rollout_is_seq_std:0.0023794230073690414 - rollout_corr/rollout_is_seq_max:1.012694001197815 - rollout_corr/rollout_is_seq_min:0.9931012392044067 - rollout_corr/rollout_is_seq_max_deviation:0.012694001197814941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6068592290394008) - rollout_corr/training_log_ppl:np.float64(0.4637760063924361) - rollout_corr/kl:np.float64(0.0005125066249966892) - rollout_corr/k3_kl:np.float64(0.0008192024238837803) - rollout_corr/rollout_ppl:np.float64(1.6059974883683026) - rollout_corr/rollout_log_ppl:np.float64(0.46326349792070687) - rollout_corr/log_ppl_diff:np.float64(0.0005125084717292339) - rollout_corr/log_ppl_abs_diff:np.float64(0.001737583806971088) - rollout_corr/log_ppl_diff_max:np.float64(0.006271004676818848) - rollout_corr/log_ppl_diff_min:np.float64(-0.01115313172340393) - rollout_corr/ppl_ratio:np.float64(1.0005151890218258) - rollout_corr/chi2_token:np.float64(0.0022532844450324774) - rollout_corr/chi2_seq:np.float64(1.0010677098762244) - actor/pg_loss:np.float64(0.004201453877612948) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00037899658491369337) - actor/ppo_kl:np.float64(0.000158508868715046) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(21191.4891625538) - self_distillation/token_reweight_w_std:np.float64(305819.6095345903) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013274641241878) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08719093064428307) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.2567609027028084) - perf/mfu/actor:np.float64(0.05189741811293595) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.25640106201172) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00024856385425664485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00024856385425664485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:388.953125 - response_length/max:1077.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:388.953125 - response_length_non_aborted/max:1077.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:452.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001035984605550766 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4913760963827372) - timing_s/agent_loop/generate_sequences/max:np.float64(8.839037172496319) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.348892316258571) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.839037172496319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1077 - timing_s/gen:15.795821461826563 - timing_s/reward:0.061745528131723404 - timing_s/old_log_prob:3.0005341172218323 - timing_s/adv:0.5373612400144339 - timing_s/update_actor:21.613999467343092 - timing_s/step:41.096161579713225 - timing_s/stop_profile:2.9422342777252197e-05 - timing_per_token_ms/update_actor:0.12578271996172566 - timing_per_token_ms/gen:0.15863718175618208 - timing_per_token_ms/adv:0.0031271749808796405 - perf/total_num_tokens:171836 - perf/time_per_step:41.096161579713225 - perf/throughput:522.6643845639145 (TaskRunner pid=2255397) Training Progress: 39%|███▉ | 39/100 [34:45<44:34, 43.84s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we must examine the structure of the product and compare it to the structures of the given options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product contains a nitrogen-containing ring with a substituent group that includes a chlorinated benzene ring and a trifluoromethyl group. Specifically, the product has a structure where a nitrogen atom is part of a ring that is connected to a carbon chain leading to a chlorinated benzene ring with a trifluoromethyl group. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let's analyze the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) - **Option A**: Contains a structure with a nitrogen ring and a chlorinated benzene ring, along with a trifluoromethyl group. It closely resembles the structure of the target product, especially in terms of the nitrogen-containing ring and the substituent groups. (TaskRunner pid=2255397) (TaskRunner pid=2255397) - **Option B**: Contains a chlorine atom directly attached to the nitrogen ring, which is not present in the target product. This makes it an incorrect match. (TaskRunner pid=2255397) (TaskRunner pid=2255397) - **Option C**: Features a hydroxyl group and a different substituent pattern, which does not match the target structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) - **Option D**: Includes a double bond and a fluorinated carbon chain, but not the specific chlorinated benzene ring and nitrogen ring arrangement of the target product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Only **Option A** contains the correct structural elements that match the target product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2255397) step:40 - global_seqlen/min:18616 - global_seqlen/max:26413 - global_seqlen/minmax_diff:7797 - global_seqlen/balanced_min:22648 - global_seqlen/balanced_max:22652 - global_seqlen/mean:22651.25 - actor/entropy:0.46682530641555786 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102363586425781 - training/rollout_probs_diff_mean:0.005161741282790899 - training/rollout_probs_diff_std:0.011252960190176964 - training/rollout_actor_probs_pearson_corr:0.9991944432258606 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:1.0001730918884277 - rollout_corr/rollout_is_ratio_fraction_high:9.88513511401834e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.977027022803668e-05 - rollout_corr/rollout_is_max:2.2787070274353027 - rollout_corr/rollout_is_min:0.2646176815032959 - rollout_corr/rollout_is_std:0.03861745819449425 - rollout_corr/rollout_is_eff_sample_size:0.9985113878945102 - rollout_corr/rollout_is_seq_mean:1.0001778602600098 - rollout_corr/rollout_is_seq_std:0.0021594883874058723 - rollout_corr/rollout_is_seq_max:1.0066732168197632 - rollout_corr/rollout_is_seq_min:0.9935801029205322 - rollout_corr/rollout_is_seq_max_deviation:0.006673216819763184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6297845151275396) - rollout_corr/training_log_ppl:np.float64(0.4774156858911738) - rollout_corr/kl:np.float64(0.0007412694044468537) - rollout_corr/k3_kl:np.float64(0.0009004596647628205) - rollout_corr/rollout_ppl:np.float64(1.628512756433338) - rollout_corr/rollout_log_ppl:np.float64(0.47667441627709195) - rollout_corr/log_ppl_diff:np.float64(0.0007412696140818298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018867639009840786) - rollout_corr/log_ppl_diff_max:np.float64(0.00960838794708252) - rollout_corr/log_ppl_diff_min:np.float64(-0.005440711975097656) - rollout_corr/ppl_ratio:np.float64(1.0007440785411745) - rollout_corr/chi2_token:np.float64(0.0020869714207947254) - rollout_corr/chi2_seq:np.float64(0.9351563951931894) - actor/pg_loss:np.float64(0.003354526706971228) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000359561878212844) - actor/ppo_kl:np.float64(0.00013469158743095733) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(9349.600620643701) - self_distillation/token_reweight_w_std:np.float64(148328.15592557297) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9980465415865183) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08430611382937059) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.28926024958491325) - perf/mfu/actor:np.float64(0.054041057160877155) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.16180038452148) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6455357142857143) - val-aux/sciknoweval/reward/std@16:np.float64(0.19709042049186706) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7234142857142857) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15652759126874422) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5656380952380953) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17688873171951364) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6442142857142857) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19706989234623676) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7823238095238095) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11006778616480029) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4971333333333333) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14506188764023242) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.657395238095238) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.15434237064022824) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8243714285714285) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07581488544967824) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.43891904761904754) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10956011972160636) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6651571428571428) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10880726297391681) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8535380952380953) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04777075262896029) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3930904761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0728366598950776) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6670523809523811) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07675932385019639) - val-aux/sciknoweval/score/mean@16:np.float64(0.6455357142857143) - val-aux/sciknoweval/score/std@16:np.float64(0.19709042049186706) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7234142857142857) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15652759126874422) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5656380952380953) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17688873171951364) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6442142857142857) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.19706989234623676) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7823238095238095) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11006778616480029) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4971333333333333) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14506188764023242) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.657395238095238) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.15434237064022824) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8243714285714285) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07581488544967824) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.43891904761904754) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10956011972160636) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6651571428571428) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10880726297391681) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8535380952380953) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04777075262896029) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3930904761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0728366598950776) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6670523809523811) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07675932385019639) - val-core/sciknoweval/acc/mean@16:np.float64(0.6455357142857143) - val-aux/sciknoweval/acc/std@16:np.float64(0.19709042049186706) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7234142857142857) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15652759126874422) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5656380952380953) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17688873171951364) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6442142857142857) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.19706989234623676) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7823238095238095) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11006778616480029) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4971333333333333) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14506188764023242) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.657395238095238) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.15434237064022824) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8243714285714285) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07581488544967824) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.43891904761904754) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10956011972160636) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6651571428571428) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10880726297391681) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8535380952380953) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04777075262896029) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3930904761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0728366598950776) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6670523809523811) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07675932385019639) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9970238095238095) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.008247511677748322) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.999504761904762) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0031118665720636365) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9944428571428572) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.010692005762772086) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.996947619047619) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.00845281646011912) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999857142857144) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0002604287412166727) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9903238095238096) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.012632535914270398) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9988238095238094) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.005212831716796075) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9847809523809524) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.01316242814531538) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997285714285713) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.002375569550118018) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9783666666666666) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.010799831142838577) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0004863840055089889) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015742076793685555 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015742076793685555 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:395.1640625 - response_length/max:913.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:395.1640625 - response_length_non_aborted/max:913.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.6875 - prompt_length/max:501.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6823952794075012e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.958938129246235) - timing_s/agent_loop/generate_sequences/max:np.float64(8.071589579805732) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.574361211241921) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.071589579805732) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:913 - timing_s/gen:15.109674572944641 - timing_s/reward:0.06512491777539253 - timing_s/old_log_prob:3.012589380145073 - timing_s/adv:0.5300268922001123 - timing_s/update_actor:21.85186207294464 - timing_s/step:40.66220090165734 - timing_s/testing:138.401902012527 - timing_s/save_checkpoint:13.304814949631691 - timing_s/stop_profile:3.7167221307754517e-05 - timing_per_token_ms/update_actor:0.12058861030265791 - timing_per_token_ms/gen:0.1493611689462905 - timing_per_token_ms/adv:0.0029249318039849476 - perf/total_num_tokens:181210 - perf/time_per_step:40.66220090165734 - perf/throughput:557.0591236510458 (TaskRunner pid=2255397) Training Progress: 40%|████ | 40/100 [37:57<1:28:24, 88.41s/it] (TaskRunner pid=2255397) step:41 - global_seqlen/min:18283 - global_seqlen/max:24700 - global_seqlen/minmax_diff:6417 - global_seqlen/balanced_min:21101 - global_seqlen/balanced_max:21109 - global_seqlen/mean:21105.5 - actor/entropy:0.42351460456848145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9826107025146484 - training/rollout_probs_diff_mean:0.005102348513901234 - training/rollout_probs_diff_std:0.01174638606607914 - training/rollout_actor_probs_pearson_corr:0.9991093873977661 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.999843180179596 - rollout_corr/rollout_is_ratio_fraction_high:9.873228009382728e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.873228009382728e-06 - rollout_corr/rollout_is_max:2.636786937713623 - rollout_corr/rollout_is_min:0.013885053806006908 - rollout_corr/rollout_is_std:0.03809376060962677 - rollout_corr/rollout_is_eff_sample_size:0.9985506114667393 - rollout_corr/rollout_is_seq_mean:0.9998304843902588 - rollout_corr/rollout_is_seq_std:0.001992590492591262 - rollout_corr/rollout_is_seq_max:1.0068778991699219 - rollout_corr/rollout_is_seq_min:0.9926929473876953 - rollout_corr/rollout_is_seq_max_deviation:0.0073070526123046875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5631140493787825) - rollout_corr/training_log_ppl:np.float64(0.436920084990561) - rollout_corr/kl:np.float64(0.0011501897124975846) - rollout_corr/k3_kl:np.float64(0.0009579621572584074) - rollout_corr/rollout_ppl:np.float64(1.5612433003261685) - rollout_corr/rollout_log_ppl:np.float64(0.4357698948588222) - rollout_corr/log_ppl_diff:np.float64(0.001150190131738782) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019631539471447468) - rollout_corr/log_ppl_diff_max:np.float64(0.010622143745422363) - rollout_corr/log_ppl_diff_min:np.float64(-0.00589299201965332) - rollout_corr/ppl_ratio:np.float64(1.001153603894636) - rollout_corr/chi2_token:np.float64(0.0014891347382217646) - rollout_corr/chi2_seq:np.float64(0.3713759642560035) - actor/pg_loss:np.float64(0.007653480512090027) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008902258696252829) - actor/ppo_kl:np.float64(0.0001822680885723571) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(1181.0103185500484) - self_distillation/token_reweight_w_std:np.float64(19672.96590518701) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9963315855711699) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1375392735353671) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.34661057591438293) - perf/mfu/actor:np.float64(0.05125395817020568) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.22666931152344) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004591051023453474 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004591051023453474 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:395.640625 - response_length/max:930.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:395.640625 - response_length_non_aborted/max:930.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.90625 - prompt_length/max:631.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7123838663101196e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.571682434529066) - timing_s/agent_loop/generate_sequences/max:np.float64(8.058521373197436) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.402195893351745) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.058521373197436) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:354 - timing_s/agent_loop/slowest/response_length:930 - timing_s/gen:14.408957185223699 - timing_s/reward:0.055850500240921974 - timing_s/old_log_prob:3.1053357888013124 - timing_s/adv:0.5568513758480549 - timing_s/update_actor:21.503483591601253 - timing_s/step:39.73492740653455 - timing_s/stop_profile:3.6053359508514404e-05 - timing_per_token_ms/update_actor:0.12735710828694685 - timing_per_token_ms/gen:0.1422629160106601 - timing_per_token_ms/adv:0.003298022884130054 - perf/total_num_tokens:168844 - perf/time_per_step:39.73492740653455 - perf/throughput:531.157381617089 (TaskRunner pid=2255397) Training Progress: 41%|████ | 41/100 [38:37<1:12:34, 73.81s/it] (TaskRunner pid=2255397) step:42 - global_seqlen/min:15988 - global_seqlen/max:22821 - global_seqlen/minmax_diff:6833 - global_seqlen/balanced_min:19797 - global_seqlen/balanced_max:19801 - global_seqlen/mean:19798.875 - actor/entropy:0.39622214436531067 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6605160236358643 - training/rollout_probs_diff_mean:0.005419079214334488 - training/rollout_probs_diff_std:0.012383614666759968 - training/rollout_actor_probs_pearson_corr:0.998907744884491 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.9609375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9609375 - rollout_corr/rollout_is_mean:0.9999876022338867 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1576137442025356e-05 - rollout_corr/rollout_is_max:1.7245110273361206 - rollout_corr/rollout_is_min:0.2930092215538025 - rollout_corr/rollout_is_std:0.03913719579577446 - rollout_corr/rollout_is_eff_sample_size:0.9984703847272242 - rollout_corr/rollout_is_seq_mean:0.9999865293502808 - rollout_corr/rollout_is_seq_std:0.002267190022394061 - rollout_corr/rollout_is_seq_max:1.0080516338348389 - rollout_corr/rollout_is_seq_min:0.9936971664428711 - rollout_corr/rollout_is_seq_max_deviation:0.008051633834838867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5080496412701905) - rollout_corr/training_log_ppl:np.float64(0.4042311883531511) - rollout_corr/kl:np.float64(0.0007036857559299392) - rollout_corr/k3_kl:np.float64(0.0009411578781737262) - rollout_corr/rollout_ppl:np.float64(1.5069640525616705) - rollout_corr/rollout_log_ppl:np.float64(0.40352750167949125) - rollout_corr/log_ppl_diff:np.float64(0.0007036866736598313) - rollout_corr/log_ppl_abs_diff:np.float64(0.001983315742108971) - rollout_corr/log_ppl_diff_max:np.float64(0.0076701343059539795) - rollout_corr/log_ppl_diff_min:np.float64(-0.006415665149688721) - rollout_corr/ppl_ratio:np.float64(1.0007067790720612) - rollout_corr/chi2_token:np.float64(0.002371277194470167) - rollout_corr/chi2_seq:np.float64(0.9433175600133836) - actor/pg_loss:np.float64(0.010519832605496049) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009025154786286294) - actor/ppo_kl:np.float64(-0.00013643282175568316) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(12954.285722455475) - self_distillation/token_reweight_w_std:np.float64(227959.35449937586) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025726119056344) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14800434719654731) - self_distillation/empty_target_batch:np.float64(0.0390625) - actor/grad_norm:np.float64(0.45579010248184204) - perf/mfu/actor:np.float64(0.04795526011784892) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.25069046020508) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013215383514761925 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013215383514761925 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:362.08984375 - response_length/max:864.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:362.08984375 - response_length_non_aborted/max:864.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:256.625 - prompt_length/max:557.0 - prompt_length/min:175.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.4523348808288574e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8027989882975817) - timing_s/agent_loop/generate_sequences/max:np.float64(7.439005896449089) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.104638532247918) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.439005896449089) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:199 - timing_s/agent_loop/slowest/response_length:864 - timing_s/gen:14.218583673238754 - timing_s/reward:0.05999644473195076 - timing_s/old_log_prob:2.961333120241761 - timing_s/adv:0.5369714200496674 - timing_s/update_actor:21.455332016572356 - timing_s/step:39.31888073682785 - timing_s/stop_profile:0.00011654756963253021 - timing_per_token_ms/update_actor:0.1354580248661373 - timing_per_token_ms/gen:0.15339105316617677 - timing_per_token_ms/adv:0.00339016370911016 - perf/total_num_tokens:158391 - perf/time_per_step:39.31888073682785 - perf/throughput:503.54625129131597 (TaskRunner pid=2255397) Training Progress: 42%|████▏ | 42/100 [39:16<1:01:21, 63.48s/it] (TaskRunner pid=2255397) step:43 - global_seqlen/min:18065 - global_seqlen/max:23096 - global_seqlen/minmax_diff:5031 - global_seqlen/balanced_min:20756 - global_seqlen/balanced_max:20762 - global_seqlen/mean:20760.875 - actor/entropy:0.42775002121925354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424738645553589 - training/rollout_probs_diff_mean:0.005239918828010559 - training/rollout_probs_diff_std:0.01176416501402855 - training/rollout_actor_probs_pearson_corr:0.999097466468811 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.79296875 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.79296875 - rollout_corr/rollout_is_mean:0.9999403357505798 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1294968317088205e-05 - rollout_corr/rollout_is_max:1.6902360916137695 - rollout_corr/rollout_is_min:0.39213547110557556 - rollout_corr/rollout_is_std:0.038980014622211456 - rollout_corr/rollout_is_eff_sample_size:0.9984827447477223 - rollout_corr/rollout_is_seq_mean:0.9998909831047058 - rollout_corr/rollout_is_seq_std:0.002513354877009988 - rollout_corr/rollout_is_seq_max:1.0098495483398438 - rollout_corr/rollout_is_seq_min:0.9915899038314819 - rollout_corr/rollout_is_seq_max_deviation:0.00984954833984375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5651745153591037) - rollout_corr/training_log_ppl:np.float64(0.4374735167075414) - rollout_corr/kl:np.float64(0.0007840867953743391) - rollout_corr/k3_kl:np.float64(0.0008964727625198066) - rollout_corr/rollout_ppl:np.float64(1.5639647683128715) - rollout_corr/rollout_log_ppl:np.float64(0.4366894279664848) - rollout_corr/log_ppl_diff:np.float64(0.0007840887410566211) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020498010562732816) - rollout_corr/log_ppl_diff_max:np.float64(0.009105801582336426) - rollout_corr/log_ppl_diff_min:np.float64(-0.00906282663345337) - rollout_corr/ppl_ratio:np.float64(1.0007877822499722) - rollout_corr/chi2_token:np.float64(0.00206923414953053) - rollout_corr/chi2_seq:np.float64(1.3589389722328633) - actor/pg_loss:np.float64(0.004318759543821216) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004535638540801301) - actor/ppo_kl:np.float64(-0.00012510620860073374) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.79296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(39451.46745206439) - self_distillation/token_reweight_w_std:np.float64(461544.07824653096) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995274562388659) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09244409290840849) - self_distillation/empty_target_batch:np.float64(0.20703125) - actor/grad_norm:np.float64(0.32581663131713867) - perf/mfu/actor:np.float64(0.05044233796581919) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.0129623413086) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011210255324840546 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011210255324840546 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:345.83984375 - response_length/max:801.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:345.83984375 - response_length_non_aborted/max:801.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.9375 - prompt_length/max:813.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001546517014503479 - timing_s/agent_loop/generate_sequences/min:np.float64(1.693193232640624) - timing_s/agent_loop/generate_sequences/max:np.float64(6.869511658325791) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.046683865577506) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.869511658325791) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:801 - timing_s/gen:14.301691135391593 - timing_s/reward:0.06055835448205471 - timing_s/old_log_prob:2.9482695516198874 - timing_s/adv:0.5146379265934229 - timing_s/update_actor:21.42802003212273 - timing_s/step:39.34179303236306 - timing_s/stop_profile:0.00010741874575614929 - timing_per_token_ms/update_actor:0.12901684076491676 - timing_per_token_ms/gen:0.16153714503181332 - timing_per_token_ms/adv:0.0030986045060325184 - perf/total_num_tokens:166087 - perf/time_per_step:39.34179303236306 - perf/throughput:527.7053585972008 (TaskRunner pid=2255397) Training Progress: 43%|████▎ | 43/100 [39:56<53:26, 56.25s/it] (TaskRunner pid=2255397) step:44 - global_seqlen/min:19213 - global_seqlen/max:22622 - global_seqlen/minmax_diff:3409 - global_seqlen/balanced_min:20879 - global_seqlen/balanced_max:20890 - global_seqlen/mean:20886.75 - actor/entropy:0.43929049372673035 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22547024488449097 - training/rollout_probs_diff_mean:0.0053957682102918625 - training/rollout_probs_diff_std:0.011915630660951138 - training/rollout_actor_probs_pearson_corr:0.9990841150283813 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.000036358833313 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9659661054611206 - rollout_corr/rollout_is_min:0.5844541192054749 - rollout_corr/rollout_is_std:0.039155468344688416 - rollout_corr/rollout_is_eff_sample_size:0.9984691962798411 - rollout_corr/rollout_is_seq_mean:1.0000959634780884 - rollout_corr/rollout_is_seq_std:0.0024589113891124725 - rollout_corr/rollout_is_seq_max:1.0087720155715942 - rollout_corr/rollout_is_seq_min:0.9921355843544006 - rollout_corr/rollout_is_seq_max_deviation:0.008772015571594238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6047232481651008) - rollout_corr/training_log_ppl:np.float64(0.46260377258295193) - rollout_corr/kl:np.float64(0.0009372506496678756) - rollout_corr/k3_kl:np.float64(0.000976888309367041) - rollout_corr/rollout_ppl:np.float64(1.6032404340803623) - rollout_corr/rollout_log_ppl:np.float64(0.4616665196372196) - rollout_corr/log_ppl_diff:np.float64(0.0009372529457323253) - rollout_corr/log_ppl_abs_diff:np.float64(0.002075349271763116) - rollout_corr/log_ppl_diff_max:np.float64(0.013183057308197021) - rollout_corr/log_ppl_diff_min:np.float64(-0.006263077259063721) - rollout_corr/ppl_ratio:np.float64(1.000940943369642) - rollout_corr/chi2_token:np.float64(0.0020958164241164923) - rollout_corr/chi2_seq:np.float64(0.34359696647152305) - actor/pg_loss:np.float64(0.005748242023400962) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005829084911965765) - actor/ppo_kl:np.float64(0.0002053698056716513) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(14803.191471951548) - self_distillation/token_reweight_w_std:np.float64(246306.37441055127) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9985546870157123) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10855769729823805) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.36420872062444687) - perf/mfu/actor:np.float64(0.05008791245123715) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.9407615661621) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00279244058765471 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00279244058765471 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:341.1484375 - response_length/max:1060.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:341.1484375 - response_length_non_aborted/max:1060.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:311.5625 - prompt_length/max:701.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014213845133781433 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7569324560463428) - timing_s/agent_loop/generate_sequences/max:np.float64(8.538507629185915) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.876661294561927) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.538507629185915) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:1060 - timing_s/gen:16.138035461306572 - timing_s/reward:0.06066634878516197 - timing_s/old_log_prob:3.082423932850361 - timing_s/adv:0.5534890294075012 - timing_s/update_actor:21.732121851295233 - timing_s/step:41.653873067349195 - timing_s/stop_profile:7.493607699871063e-05 - timing_per_token_ms/update_actor:0.13005925916726652 - timing_per_token_ms/gen:0.18478525501301407 - timing_per_token_ms/adv:0.00331244107752224 - perf/total_num_tokens:167094 - perf/time_per_step:41.653873067349195 - perf/throughput:501.4359640993934 (TaskRunner pid=2255397) Training Progress: 44%|████▍ | 44/100 [40:37<48:25, 51.89s/it] (TaskRunner pid=2255397) step:45 - global_seqlen/min:18835 - global_seqlen/max:26317 - global_seqlen/minmax_diff:7482 - global_seqlen/balanced_min:21440 - global_seqlen/balanced_max:21451 - global_seqlen/mean:21447.875 - actor/entropy:0.4101407527923584 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28218671679496765 - training/rollout_probs_diff_mean:0.004841715097427368 - training/rollout_probs_diff_std:0.01116796862334013 - training/rollout_actor_probs_pearson_corr:0.9992083311080933 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:0.9999707937240601 - rollout_corr/rollout_is_ratio_fraction_high:1.0986233974108472e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0986233974108472e-05 - rollout_corr/rollout_is_max:2.0019893646240234 - rollout_corr/rollout_is_min:0.25255540013313293 - rollout_corr/rollout_is_std:0.036969371140003204 - rollout_corr/rollout_is_eff_sample_size:0.998635012024894 - rollout_corr/rollout_is_seq_mean:1.0000436305999756 - rollout_corr/rollout_is_seq_std:0.002479569287970662 - rollout_corr/rollout_is_seq_max:1.0116991996765137 - rollout_corr/rollout_is_seq_min:0.9924298524856567 - rollout_corr/rollout_is_seq_max_deviation:0.011699199676513672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5578296538442373) - rollout_corr/training_log_ppl:np.float64(0.432097623241134) - rollout_corr/kl:np.float64(0.0007768278248150295) - rollout_corr/k3_kl:np.float64(0.0008250089271086836) - rollout_corr/rollout_ppl:np.float64(1.5565427541732788) - rollout_corr/rollout_log_ppl:np.float64(0.4313207929953933) - rollout_corr/log_ppl_diff:np.float64(0.0007768302457407117) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019471754785627127) - rollout_corr/log_ppl_diff_max:np.float64(0.011698126792907715) - rollout_corr/log_ppl_diff_min:np.float64(-0.010318875312805176) - rollout_corr/ppl_ratio:np.float64(1.0007803542539477) - rollout_corr/chi2_token:np.float64(0.0017588012851774693) - rollout_corr/chi2_seq:np.float64(0.6141417354810983) - actor/pg_loss:np.float64(0.0026712483959272504) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039884855823402177) - actor/ppo_kl:np.float64(6.892802709934642e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(38907.24203802785) - self_distillation/token_reweight_w_std:np.float64(568287.8935087718) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006591160781682) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08028327141073532) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.28873957321047783) - perf/mfu/actor:np.float64(0.05157141247510403) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.14898681640625) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005405227188020945 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005405227188020945 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:355.55859375 - response_length/max:782.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:355.55859375 - response_length_non_aborted/max:782.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.6875 - prompt_length/max:845.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.7983601689338684e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.374290145933628) - timing_s/agent_loop/generate_sequences/max:np.float64(7.021162023767829) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9449067628447665) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.021162023767829) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:782 - timing_s/gen:13.897942123934627 - timing_s/reward:0.06329947151243687 - timing_s/old_log_prob:3.025046695023775 - timing_s/adv:0.56037213280797 - timing_s/update_actor:21.67712887376547 - timing_s/step:39.31348724476993 - timing_s/stop_profile:0.00011695176362991333 - timing_per_token_ms/update_actor:0.1263361106506208 - timing_per_token_ms/gen:0.15268604774545583 - timing_per_token_ms/adv:0.0032658954139277788 - perf/total_num_tokens:171583 - perf/time_per_step:39.31348724476993 - perf/throughput:545.5602263534461 (TaskRunner pid=2255397) Training Progress: 45%|████▌ | 45/100 [41:17<44:07, 48.13s/it] (TaskRunner pid=2255397) step:46 - global_seqlen/min:17315 - global_seqlen/max:21947 - global_seqlen/minmax_diff:4632 - global_seqlen/balanced_min:19767 - global_seqlen/balanced_max:19776 - global_seqlen/mean:19773.25 - actor/entropy:0.4300471842288971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5463374257087708 - training/rollout_probs_diff_mean:0.0050852419808506966 - training/rollout_probs_diff_std:0.011606025509536266 - training/rollout_actor_probs_pearson_corr:0.9991208910942078 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8203125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8203125 - rollout_corr/rollout_is_mean:0.9999197125434875 - rollout_corr/rollout_is_ratio_fraction_high:1.1397570233384613e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2795140466769226e-05 - rollout_corr/rollout_is_max:2.4014859199523926 - rollout_corr/rollout_is_min:0.26859816908836365 - rollout_corr/rollout_is_std:0.03753577917814255 - rollout_corr/rollout_is_eff_sample_size:0.9985928099624436 - rollout_corr/rollout_is_seq_mean:1.0000156164169312 - rollout_corr/rollout_is_seq_std:0.002315930090844631 - rollout_corr/rollout_is_seq_max:1.0097026824951172 - rollout_corr/rollout_is_seq_min:0.9926007986068726 - rollout_corr/rollout_is_seq_max_deviation:0.009702682495117188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5882981927134097) - rollout_corr/training_log_ppl:np.float64(0.44962009048322216) - rollout_corr/kl:np.float64(0.0011486389046666545) - rollout_corr/k3_kl:np.float64(0.0009466409361493788) - rollout_corr/rollout_ppl:np.float64(1.5864842752926052) - rollout_corr/rollout_log_ppl:np.float64(0.4484714485006407) - rollout_corr/log_ppl_diff:np.float64(0.0011486419825814664) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022174433688633144) - rollout_corr/log_ppl_diff_max:np.float64(0.015058279037475586) - rollout_corr/log_ppl_diff_min:np.float64(-0.00858527421951294) - rollout_corr/ppl_ratio:np.float64(1.0011532127391547) - rollout_corr/chi2_token:np.float64(0.0015598325990140438) - rollout_corr/chi2_seq:np.float64(0.6058120087254792) - actor/pg_loss:np.float64(0.005358678288757801) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006527249729515461) - actor/ppo_kl:np.float64(0.0004008172310321356) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(13853.629367629066) - self_distillation/token_reweight_w_std:np.float64(223926.18253548152) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000649391906336) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12465728094684891) - self_distillation/empty_target_batch:np.float64(0.1796875) - actor/grad_norm:np.float64(0.37411294877529144) - perf/mfu/actor:np.float64(0.04759140321566382) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.96075439453125) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004386639688163996 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004386639688163996 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:342.7265625 - response_length/max:874.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:342.7265625 - response_length_non_aborted/max:874.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.1875 - prompt_length/max:595.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012195669114589691 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6501337420195341) - timing_s/agent_loop/generate_sequences/max:np.float64(7.341919541358948) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8824371568043716) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.341919541358948) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:177 - timing_s/agent_loop/slowest/response_length:874 - timing_s/gen:14.224360808730125 - timing_s/reward:0.07079944014549255 - timing_s/old_log_prob:2.932241167873144 - timing_s/adv:0.537244874984026 - timing_s/update_actor:21.595252191647887 - timing_s/step:39.45111795514822 - timing_s/stop_profile:0.00012188218533992767 - timing_per_token_ms/update_actor:0.13651810015834454 - timing_per_token_ms/gen:0.16212314856424953 - timing_per_token_ms/adv:0.0033962858595831865 - perf/total_num_tokens:158186 - perf/time_per_step:39.45111795514822 - perf/throughput:501.2088636494436 (TaskRunner pid=2255397) Training Progress: 46%|████▌ | 46/100 [41:56<40:59, 45.54s/it] (TaskRunner pid=2255397) step:47 - global_seqlen/min:16338 - global_seqlen/max:23481 - global_seqlen/minmax_diff:7143 - global_seqlen/balanced_min:20960 - global_seqlen/balanced_max:20969 - global_seqlen/mean:20966.0 - actor/entropy:0.4562165141105652 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135512113571167 - training/rollout_probs_diff_mean:0.004941638559103012 - training/rollout_probs_diff_std:0.01093968003988266 - training/rollout_actor_probs_pearson_corr:0.9992673993110657 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.0000667572021484 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9514927864074707 - rollout_corr/rollout_is_min:0.5686209797859192 - rollout_corr/rollout_is_std:0.03698226809501648 - rollout_corr/rollout_is_eff_sample_size:0.998634417604882 - rollout_corr/rollout_is_seq_mean:1.0001049041748047 - rollout_corr/rollout_is_seq_std:0.0021275379694998264 - rollout_corr/rollout_is_seq_max:1.0070573091506958 - rollout_corr/rollout_is_seq_min:0.9932472705841064 - rollout_corr/rollout_is_seq_max_deviation:0.007057309150695801 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6164640178903937) - rollout_corr/training_log_ppl:np.float64(0.46599303945549764) - rollout_corr/kl:np.float64(0.0011402188001117253) - rollout_corr/k3_kl:np.float64(0.0008741923521711215) - rollout_corr/rollout_ppl:np.float64(1.6145951971411705) - rollout_corr/rollout_log_ppl:np.float64(0.46485281962668523) - rollout_corr/log_ppl_diff:np.float64(0.0011402198288124055) - rollout_corr/log_ppl_abs_diff:np.float64(0.001965065923286602) - rollout_corr/log_ppl_diff_max:np.float64(0.008194833993911743) - rollout_corr/log_ppl_diff_min:np.float64(-0.005479037761688232) - rollout_corr/ppl_ratio:np.float64(1.0011436399072409) - rollout_corr/chi2_token:np.float64(0.0012568566016852856) - rollout_corr/chi2_seq:np.float64(0.40265263244509697) - actor/pg_loss:np.float64(0.006021094392053783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005619370990643802) - actor/ppo_kl:np.float64(0.0005166811963022155) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(12667.028565393994) - self_distillation/token_reweight_w_std:np.float64(223391.28332038334) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007588975131512) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12597810916486196) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.35431264340877533) - perf/mfu/actor:np.float64(0.050087304585172576) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.9956817626953) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005755655933171511 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005755655933171511 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.90625 - response_length/max:784.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.90625 - response_length_non_aborted/max:784.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.28125 - prompt_length/max:950.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.334273636341095e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7290426678955555) - timing_s/agent_loop/generate_sequences/max:np.float64(7.266604153439403) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.170146645323257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.266604153439403) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:784 - timing_s/gen:14.346795970574021 - timing_s/reward:0.06195123679935932 - timing_s/old_log_prob:2.8864920753985643 - timing_s/adv:0.5422863606363535 - timing_s/update_actor:21.55750539712608 - timing_s/step:39.47956304065883 - timing_s/stop_profile:2.954714000225067e-05 - timing_per_token_ms/update_actor:0.12852657515218735 - timing_per_token_ms/gen:0.1548527326070074 - timing_per_token_ms/adv:0.0032331295945599633 - perf/total_num_tokens:167728 - perf/time_per_step:39.47956304065883 - perf/throughput:531.0595757710828 (TaskRunner pid=2255397) Training Progress: 47%|████▋ | 47/100 [42:36<38:37, 43.73s/it] (TaskRunner pid=2255397) step:48 - global_seqlen/min:16138 - global_seqlen/max:20539 - global_seqlen/minmax_diff:4401 - global_seqlen/balanced_min:19360 - global_seqlen/balanced_max:19367 - global_seqlen/mean:19365.625 - actor/entropy:0.45748016238212585 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.423759400844574 - training/rollout_probs_diff_mean:0.00517674908041954 - training/rollout_probs_diff_std:0.011389809660613537 - training/rollout_actor_probs_pearson_corr:0.9992020130157471 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000624656677246 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.4522043279139325e-05 - rollout_corr/rollout_is_max:1.9825339317321777 - rollout_corr/rollout_is_min:0.29720374941825867 - rollout_corr/rollout_is_std:0.038425594568252563 - rollout_corr/rollout_is_eff_sample_size:0.9985258883712781 - rollout_corr/rollout_is_seq_mean:1.0000547170639038 - rollout_corr/rollout_is_seq_std:0.0022232853807508945 - rollout_corr/rollout_is_seq_max:1.0069233179092407 - rollout_corr/rollout_is_seq_min:0.9934024810791016 - rollout_corr/rollout_is_seq_max_deviation:0.006923317909240723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6189399659633636) - rollout_corr/training_log_ppl:np.float64(0.47103061771485955) - rollout_corr/kl:np.float64(0.0008533685957203829) - rollout_corr/k3_kl:np.float64(0.0008472814761262271) - rollout_corr/rollout_ppl:np.float64(1.6175275961868465) - rollout_corr/rollout_log_ppl:np.float64(0.47017724718898535) - rollout_corr/log_ppl_diff:np.float64(0.0008533705258741975) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018438339466229081) - rollout_corr/log_ppl_diff_max:np.float64(0.009492874145507812) - rollout_corr/log_ppl_diff_min:np.float64(-0.004758328199386597) - rollout_corr/ppl_ratio:np.float64(1.000856165541336) - rollout_corr/chi2_token:np.float64(0.0016905255615711212) - rollout_corr/chi2_seq:np.float64(0.5319139340426773) - actor/pg_loss:np.float64(0.004102788749150932) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002375331782786816) - actor/ppo_kl:np.float64(0.00012016280814464153) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(1381.5336034777574) - self_distillation/token_reweight_w_std:np.float64(20333.09212728584) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9975523005705327) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11302358307875693) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.2960100993514061) - perf/mfu/actor:np.float64(0.04606089601890587) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.1104965209961) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013794433325529099 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013794433325529099 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:339.45703125 - response_length/max:832.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.45703125 - response_length_non_aborted/max:832.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.71875 - prompt_length/max:563.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.479366958141327e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9788021426647902) - timing_s/agent_loop/generate_sequences/max:np.float64(7.109138231724501) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9673785015329486) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.109138231724501) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:832 - timing_s/gen:13.836422380059958 - timing_s/reward:0.06020788662135601 - timing_s/old_log_prob:2.906719511374831 - timing_s/adv:0.5168796796351671 - timing_s/update_actor:21.836700728163123 - timing_s/step:39.24484575167298 - timing_s/stop_profile:0.00012063421308994293 - timing_per_token_ms/update_actor:0.1409501418632443 - timing_per_token_ms/gen:0.1592205196725004 - timing_per_token_ms/adv:0.0033363219598848935 - perf/total_num_tokens:154925 - perf/time_per_step:39.24484575167298 - perf/throughput:493.4565196800259 (TaskRunner pid=2255397) Training Progress: 48%|████▊ | 48/100 [43:15<36:44, 42.40s/it] (TaskRunner pid=2255397) step:49 - global_seqlen/min:19946 - global_seqlen/max:22692 - global_seqlen/minmax_diff:2746 - global_seqlen/balanced_min:21730 - global_seqlen/balanced_max:21733 - global_seqlen/mean:21731.75 - actor/entropy:0.4979252517223358 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43991324305534363 - training/rollout_probs_diff_mean:0.004984842147678137 - training/rollout_probs_diff_std:0.01066806074231863 - training/rollout_actor_probs_pearson_corr:0.9993179440498352 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9998939037322998 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.758192391018383e-06 - rollout_corr/rollout_is_max:1.9085274934768677 - rollout_corr/rollout_is_min:0.12017349898815155 - rollout_corr/rollout_is_std:0.03726325184106827 - rollout_corr/rollout_is_eff_sample_size:0.9986130189557251 - rollout_corr/rollout_is_seq_mean:0.9998945593833923 - rollout_corr/rollout_is_seq_std:0.001821837155148387 - rollout_corr/rollout_is_seq_max:1.005088448524475 - rollout_corr/rollout_is_seq_min:0.9940054416656494 - rollout_corr/rollout_is_seq_max_deviation:0.005994558334350586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6754766842350364) - rollout_corr/training_log_ppl:np.float64(0.5048167807399295) - rollout_corr/kl:np.float64(0.0009508746554258707) - rollout_corr/k3_kl:np.float64(0.0007735003869697721) - rollout_corr/rollout_ppl:np.float64(1.6738910162821412) - rollout_corr/rollout_log_ppl:np.float64(0.503865905397106) - rollout_corr/log_ppl_diff:np.float64(0.000950875342823565) - rollout_corr/log_ppl_abs_diff:np.float64(0.001617251429706812) - rollout_corr/log_ppl_diff_max:np.float64(0.008522868156433105) - rollout_corr/log_ppl_diff_min:np.float64(-0.00493285059928894) - rollout_corr/ppl_ratio:np.float64(1.0009531062096357) - rollout_corr/chi2_token:np.float64(0.0011934449430555105) - rollout_corr/chi2_seq:np.float64(0.1550909043289721) - actor/pg_loss:np.float64(0.005771090276539326) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002795115901790268) - actor/ppo_kl:np.float64(0.00010383528175317291) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(4803.391529331682) - self_distillation/token_reweight_w_std:np.float64(100370.09618178767) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001389476004988) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10803735570516437) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.3376929610967636) - perf/mfu/actor:np.float64(0.05224311765341433) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.75490951538086) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0023309881798923016 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0023309881798923016 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:400.3046875 - response_length/max:826.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:400.3046875 - response_length_non_aborted/max:826.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.8125 - prompt_length/max:418.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013683363795280457 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6328133698552847) - timing_s/agent_loop/generate_sequences/max:np.float64(7.4781682174652815) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.834140434708388) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.4781682174652815) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:191 - timing_s/agent_loop/slowest/response_length:826 - timing_s/gen:14.471364917233586 - timing_s/reward:0.060647664591670036 - timing_s/old_log_prob:2.9566055797040462 - timing_s/adv:0.5396076701581478 - timing_s/update_actor:21.56921719573438 - timing_s/step:39.68868784233928 - timing_s/stop_profile:2.8738752007484436e-05 - timing_per_token_ms/update_actor:0.12406511898336754 - timing_per_token_ms/gen:0.14121435739606147 - timing_per_token_ms/adv:0.0031037978427769725 - perf/total_num_tokens:173854 - perf/time_per_step:39.68868784233928 - perf/throughput:547.5552652767952 (TaskRunner pid=2255397) Training Progress: 49%|████▉ | 49/100 [43:55<35:21, 41.59s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to examine the structure of the product and compare it with the structures of the given options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product has a nitrogen-containing ring with a substituent group: "Cc2ccc(Cl)cc2C(F)(F)F". This suggests that the molecule contains a chlorinated aromatic ring attached to a trifluoromethyl group, which is connected to a nitrogen-containing ring. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let's evaluate the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) - **Option A**: Contains a structure with a nitrogen ring and a chlorinated aromatic ring connected to a trifluoromethyl group. The structure is very similar to the product, with the correct substituents and connectivity. (TaskRunner pid=2255397) - **Option B**: Contains a chlorinated nitrogen ring but lacks the trifluoromethyl group and the correct aromatic ring structure. (TaskRunner pid=2255397) - **Option C**: Contains a hydroxyl group and a different substituent, not matching the product. (TaskRunner pid=2255397) - **Option D**: Contains a different ring structure and lacks the trifluoromethyl group and the correct aromatic ring. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Only **Option A** matches the structure and substituents of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2255397) step:50 - global_seqlen/min:19045 - global_seqlen/max:30156 - global_seqlen/minmax_diff:11111 - global_seqlen/balanced_min:21698 - global_seqlen/balanced_max:21699 - global_seqlen/mean:21698.5 - actor/entropy:0.39573362469673157 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491804838180542 - training/rollout_probs_diff_mean:0.004589053802192211 - training/rollout_probs_diff_std:0.01072707585990429 - training/rollout_actor_probs_pearson_corr:0.9992213845252991 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000442266464233 - rollout_corr/rollout_is_ratio_fraction_high:9.806613888940774e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.004254102706909 - rollout_corr/rollout_is_min:0.6065316796302795 - rollout_corr/rollout_is_std:0.03603039309382439 - rollout_corr/rollout_is_eff_sample_size:0.9987034939469002 - rollout_corr/rollout_is_seq_mean:1.000090479850769 - rollout_corr/rollout_is_seq_std:0.0021530245430767536 - rollout_corr/rollout_is_seq_max:1.0064139366149902 - rollout_corr/rollout_is_seq_min:0.991096556186676 - rollout_corr/rollout_is_seq_max_deviation:0.008903443813323975 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.524656287394464) - rollout_corr/training_log_ppl:np.float64(0.4108033778902609) - rollout_corr/kl:np.float64(0.0007370334822254243) - rollout_corr/k3_kl:np.float64(0.0008599899813646061) - rollout_corr/rollout_ppl:np.float64(1.5234609101898968) - rollout_corr/rollout_log_ppl:np.float64(0.41006634262157604) - rollout_corr/log_ppl_diff:np.float64(0.0007370352686848491) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018268449639435858) - rollout_corr/log_ppl_diff_max:np.float64(0.00987979769706726) - rollout_corr/log_ppl_diff_min:np.float64(-0.00624692440032959) - rollout_corr/ppl_ratio:np.float64(1.000740017509088) - rollout_corr/chi2_token:np.float64(0.0019624873530119658) - rollout_corr/chi2_seq:np.float64(0.6737326441798359) - actor/pg_loss:np.float64(0.00585430464707315) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007581320141980541) - actor/ppo_kl:np.float64(0.00011956099752552518) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(4577.494679181604) - self_distillation/token_reweight_w_std:np.float64(75540.59034832189) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9987221618648618) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11705390218412504) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3779703937470913) - perf/mfu/actor:np.float64(0.0522593230068327) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(195.6965789794922) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.66875) - val-aux/sciknoweval/reward/std@16:np.float64(0.17551919057486814) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7372904761904763) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13660825245257055) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6003428571428571) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15818906699634022) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6687047619047619) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17558003827147262) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7877285714285714) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09825969433764362) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5388809523809523) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13247872360110166) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6810333333333333) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13288868774672) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8258047619047618) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07020796516216546) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4855857142857143) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10698787291527515) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6865809523809525) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09029984972979069) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8547666666666667) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04666779709351366) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4396285714285714) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07582314271837051) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6883333333333334) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06437108221973292) - val-aux/sciknoweval/score/mean@16:np.float64(0.66875) - val-aux/sciknoweval/score/std@16:np.float64(0.17551919057486814) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7372904761904763) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13660825245257055) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6003428571428571) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15818906699634022) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6687047619047619) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17558003827147262) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7877285714285714) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09825969433764362) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5388809523809523) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13247872360110166) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6810333333333333) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13288868774672) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8258047619047618) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07020796516216546) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4855857142857143) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10698787291527515) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6865809523809525) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09029984972979069) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8547666666666667) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04666779709351366) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4396285714285714) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07582314271837051) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6883333333333334) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06437108221973292) - val-core/sciknoweval/acc/mean@16:np.float64(0.66875) - val-aux/sciknoweval/acc/std@16:np.float64(0.17551919057486814) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7372904761904763) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13660825245257055) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6003428571428571) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15818906699634022) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6687047619047619) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17558003827147262) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7877285714285714) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09825969433764362) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5388809523809523) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13247872360110166) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6810333333333333) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13288868774672) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8258047619047618) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07020796516216546) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4855857142857143) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10698787291527515) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6865809523809525) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09029984972979069) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8547666666666667) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04666779709351366) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4396285714285714) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07582314271837051) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6883333333333334) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06437108221973292) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9985119047619048) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.005763368074713418) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9998666666666667) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.001758076041122951) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9971714285714286) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.007695791728287569) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9986190476190475) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.00554804164831311) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9947428571428573) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.00986918416859868) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9995904761904763) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.003032413337024054) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9906380952380953) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.011629673038340291) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999619047619047) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0006991297950679286) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9847428571428571) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.011415170996457474) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00015050933932646775) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015592515701428056 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0015592515701428056 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.328125 - response_length/max:872.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:398.328125 - response_length_non_aborted/max:872.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.75 - prompt_length/max:900.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.404837429523468e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0187409687787294) - timing_s/agent_loop/generate_sequences/max:np.float64(7.631963180378079) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.57371993160632) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.631963180378079) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:872 - timing_s/gen:14.39490077085793 - timing_s/reward:0.06266830302774906 - timing_s/old_log_prob:3.103602586314082 - timing_s/adv:0.5463018678128719 - timing_s/update_actor:21.542298316955566 - timing_s/step:39.73873019963503 - timing_s/testing:133.93844243697822 - timing_s/save_checkpoint:13.662320464849472 - timing_s/stop_profile:0.0001660715788602829 - timing_per_token_ms/update_actor:0.12410015851876607 - timing_per_token_ms/gen:0.14116522938510503 - timing_per_token_ms/adv:0.003147117702910754 - perf/total_num_tokens:173588 - perf/time_per_step:39.73873019963503 - perf/throughput:546.0290223415162 (TaskRunner pid=2255397) Training Progress: 50%|█████ | 50/100 [47:02<1:11:07, 85.35s/it] (TaskRunner pid=2255397) step:51 - global_seqlen/min:19988 - global_seqlen/max:24236 - global_seqlen/minmax_diff:4248 - global_seqlen/balanced_min:21993 - global_seqlen/balanced_max:22000 - global_seqlen/mean:21997.625 - actor/entropy:0.47643905878067017 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292527914047241 - training/rollout_probs_diff_mean:0.004974652547389269 - training/rollout_probs_diff_std:0.010949385352432728 - training/rollout_actor_probs_pearson_corr:0.9992679953575134 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000044584274292 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0419596947031096e-05 - rollout_corr/rollout_is_max:1.6338132619857788 - rollout_corr/rollout_is_min:0.19046327471733093 - rollout_corr/rollout_is_std:0.03751512989401817 - rollout_corr/rollout_is_eff_sample_size:0.998594711950472 - rollout_corr/rollout_is_seq_mean:0.9999472498893738 - rollout_corr/rollout_is_seq_std:0.002286979230120778 - rollout_corr/rollout_is_seq_max:1.0055614709854126 - rollout_corr/rollout_is_seq_min:0.9931483864784241 - rollout_corr/rollout_is_seq_max_deviation:0.006851613521575928 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6695182812400162) - rollout_corr/training_log_ppl:np.float64(0.49964700039708987) - rollout_corr/kl:np.float64(0.0010526214147859037) - rollout_corr/k3_kl:np.float64(0.0008514240415706809) - rollout_corr/rollout_ppl:np.float64(1.6677322825416923) - rollout_corr/rollout_log_ppl:np.float64(0.49859437631675974) - rollout_corr/log_ppl_diff:np.float64(0.0010526240803301334) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019851572578772902) - rollout_corr/log_ppl_diff_max:np.float64(0.008082211017608643) - rollout_corr/log_ppl_diff_min:np.float64(-0.008058786392211914) - rollout_corr/ppl_ratio:np.float64(1.0010559679940343) - rollout_corr/chi2_token:np.float64(0.0013342886231839657) - rollout_corr/chi2_seq:np.float64(1.1409833203069866) - actor/pg_loss:np.float64(0.005735161481425166) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006951400860089052) - actor/ppo_kl:np.float64(0.00024209431599508946) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(7385.579081017524) - self_distillation/token_reweight_w_std:np.float64(136267.17127367936) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020049593877047) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11803711034008302) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3889169320464134) - perf/mfu/actor:np.float64(0.053154140926217126) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.41073989868164) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006512248073704541 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006512248073704541 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.89453125 - response_length/max:928.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.89453125 - response_length_non_aborted/max:928.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.53125 - prompt_length/max:608.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013144873082637787 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7525868192315102) - timing_s/agent_loop/generate_sequences/max:np.float64(7.869243094697595) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.208094254077878) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.869243094697595) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:928 - timing_s/gen:14.724404215812683 - timing_s/reward:0.059697989374399185 - timing_s/old_log_prob:3.061056010425091 - timing_s/adv:0.5633108653128147 - timing_s/update_actor:21.545322271063924 - timing_s/step:40.06024991162121 - timing_s/stop_profile:0.00010966137051582336 - timing_per_token_ms/update_actor:0.12242982066850355 - timing_per_token_ms/gen:0.15342236062030656 - timing_per_token_ms/adv:0.00320097547640265 - perf/total_num_tokens:175981 - perf/time_per_step:40.06024991162121 - perf/throughput:549.1135239677732 (TaskRunner pid=2255397) Training Progress: 51%|█████ | 51/100 [47:42<58:37, 71.78s/it] (TaskRunner pid=2255397) step:52 - global_seqlen/min:16418 - global_seqlen/max:23760 - global_seqlen/minmax_diff:7342 - global_seqlen/balanced_min:20391 - global_seqlen/balanced_max:20400 - global_seqlen/mean:20398.0 - actor/entropy:0.4327685832977295 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4141288995742798 - training/rollout_probs_diff_mean:0.004852033220231533 - training/rollout_probs_diff_std:0.011078495532274246 - training/rollout_actor_probs_pearson_corr:0.9992164969444275 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.999881386756897 - rollout_corr/rollout_is_ratio_fraction_high:1.0429704161651898e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.107421636581421 - rollout_corr/rollout_is_min:0.5139061212539673 - rollout_corr/rollout_is_std:0.03718959912657738 - rollout_corr/rollout_is_eff_sample_size:0.9986184874121644 - rollout_corr/rollout_is_seq_mean:0.9998266696929932 - rollout_corr/rollout_is_seq_std:0.0023117384407669306 - rollout_corr/rollout_is_seq_max:1.010014533996582 - rollout_corr/rollout_is_seq_min:0.9893986582756042 - rollout_corr/rollout_is_seq_max_deviation:0.010601341724395752 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5747977024875581) - rollout_corr/training_log_ppl:np.float64(0.44207731005735695) - rollout_corr/kl:np.float64(0.0007742779266513011) - rollout_corr/k3_kl:np.float64(0.0008074032141394127) - rollout_corr/rollout_ppl:np.float64(1.5735519244335592) - rollout_corr/rollout_log_ppl:np.float64(0.44130302700796165) - rollout_corr/log_ppl_diff:np.float64(0.0007742830493953079) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017560763808432966) - rollout_corr/log_ppl_diff_max:np.float64(0.011528313159942627) - rollout_corr/log_ppl_diff_min:np.float64(-0.008214741945266724) - rollout_corr/ppl_ratio:np.float64(1.0007772489916533) - rollout_corr/chi2_token:np.float64(0.0016987936105579138) - rollout_corr/chi2_seq:np.float64(0.6159643346909434) - actor/pg_loss:np.float64(0.004326079739257693) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003785648150369525) - actor/ppo_kl:np.float64(-0.00013583274305517534) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(23087.692809279542) - self_distillation/token_reweight_w_std:np.float64(411971.67904203804) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013474761508405) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10722107632318512) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.31624583154916763) - perf/mfu/actor:np.float64(0.04994189212775807) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.2727928161621) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00683667091652751 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00683667091652751 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.53125 - response_length/max:748.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.53125 - response_length_non_aborted/max:748.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.90625 - prompt_length/max:453.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.608851253986359e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5058961473405361) - timing_s/agent_loop/generate_sequences/max:np.float64(6.993606846779585) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.290860692432034) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.993606846779585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:748 - timing_s/gen:13.7207720130682 - timing_s/reward:0.060612039640545845 - timing_s/old_log_prob:2.9474493078887463 - timing_s/adv:0.5020784568041563 - timing_s/update_actor:21.135599818080664 - timing_s/step:38.45533044077456 - timing_s/stop_profile:0.0001184120774269104 - timing_per_token_ms/update_actor:0.1295200498705796 - timing_per_token_ms/gen:0.14310358795440342 - timing_per_token_ms/adv:0.0030767627757877997 - perf/total_num_tokens:163184 - perf/time_per_step:38.45533044077456 - perf/throughput:530.4336165155352 (TaskRunner pid=2255397) Training Progress: 52%|█████▏ | 52/100 [48:21<49:26, 61.79s/it] (TaskRunner pid=2255397) step:53 - global_seqlen/min:19241 - global_seqlen/max:23446 - global_seqlen/minmax_diff:4205 - global_seqlen/balanced_min:21257 - global_seqlen/balanced_max:21274 - global_seqlen/mean:21261.625 - actor/entropy:0.4501931071281433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294929027557373 - training/rollout_probs_diff_mean:0.0048794192261993885 - training/rollout_probs_diff_std:0.010888302698731422 - training/rollout_actor_probs_pearson_corr:0.9992465376853943 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9998956918716431 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1179934265092015e-05 - rollout_corr/rollout_is_max:1.6338140964508057 - rollout_corr/rollout_is_min:0.37676042318344116 - rollout_corr/rollout_is_std:0.036510247737169266 - rollout_corr/rollout_is_eff_sample_size:0.9986683006749928 - rollout_corr/rollout_is_seq_mean:0.9998424053192139 - rollout_corr/rollout_is_seq_std:0.0020514947827905416 - rollout_corr/rollout_is_seq_max:1.0057796239852905 - rollout_corr/rollout_is_seq_min:0.9921916127204895 - rollout_corr/rollout_is_seq_max_deviation:0.007808387279510498 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6163378930650651) - rollout_corr/training_log_ppl:np.float64(0.4686294782150071) - rollout_corr/kl:np.float64(0.000762243592463463) - rollout_corr/k3_kl:np.float64(0.000810159646334796) - rollout_corr/rollout_ppl:np.float64(1.6150716617703438) - rollout_corr/rollout_log_ppl:np.float64(0.46786723379045725) - rollout_corr/log_ppl_diff:np.float64(0.0007622444245498627) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019291590724606067) - rollout_corr/log_ppl_diff_max:np.float64(0.008153051137924194) - rollout_corr/log_ppl_diff_min:np.float64(-0.005449235439300537) - rollout_corr/ppl_ratio:np.float64(1.000765364151448) - rollout_corr/chi2_token:np.float64(0.001715675462037325) - rollout_corr/chi2_seq:np.float64(0.6910448384005576) - actor/pg_loss:np.float64(0.0023863104870542884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002726298084780865) - actor/ppo_kl:np.float64(-0.00011610987491650349) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(1507.4308137190528) - self_distillation/token_reweight_w_std:np.float64(33027.9989340055) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995625410228968) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06903543486259878) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.21907289698719978) - perf/mfu/actor:np.float64(0.051939957351732395) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.24185180664062) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003284213598817587 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003284213598817587 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:368.86328125 - response_length/max:1113.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:368.86328125 - response_length_non_aborted/max:1113.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.5625 - prompt_length/max:532.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.354894816875458e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2522868532687426) - timing_s/agent_loop/generate_sequences/max:np.float64(8.869224239140749) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.229014496719174) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.869224239140749) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:227 - timing_s/agent_loop/slowest/response_length:1113 - timing_s/gen:15.673873921856284 - timing_s/reward:0.061943380162119865 - timing_s/old_log_prob:2.946569699794054 - timing_s/adv:0.5094821080565453 - timing_s/update_actor:21.332264576107264 - timing_s/step:40.6122511010617 - timing_s/stop_profile:0.00011801905930042267 - timing_per_token_ms/update_actor:0.1254152997248991 - timing_per_token_ms/gen:0.16598580861659326 - timing_per_token_ms/adv:0.0029953149633232716 - perf/total_num_tokens:170093 - perf/time_per_step:40.6122511010617 - perf/throughput:523.5273697853249 (TaskRunner pid=2255397) Training Progress: 53%|█████▎ | 53/100 [49:01<43:26, 55.45s/it] (TaskRunner pid=2255397) step:54 - global_seqlen/min:16771 - global_seqlen/max:23434 - global_seqlen/minmax_diff:6663 - global_seqlen/balanced_min:20121 - global_seqlen/balanced_max:20134 - global_seqlen/mean:20129.375 - actor/entropy:0.39255428314208984 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2144627422094345 - training/rollout_probs_diff_mean:0.004672019276767969 - training/rollout_probs_diff_std:0.011115854606032372 - training/rollout_actor_probs_pearson_corr:0.9991840720176697 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999427795410156 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.969622254371643 - rollout_corr/rollout_is_min:0.5503029227256775 - rollout_corr/rollout_is_std:0.03688544034957886 - rollout_corr/rollout_is_eff_sample_size:0.998641194034968 - rollout_corr/rollout_is_seq_mean:0.9999832510948181 - rollout_corr/rollout_is_seq_std:0.002377620432525873 - rollout_corr/rollout_is_seq_max:1.0109039545059204 - rollout_corr/rollout_is_seq_min:0.9921522736549377 - rollout_corr/rollout_is_seq_max_deviation:0.01090395450592041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5304169207811356) - rollout_corr/training_log_ppl:np.float64(0.4121860411542002) - rollout_corr/kl:np.float64(0.0008754907883350427) - rollout_corr/k3_kl:np.float64(0.0008680712330715323) - rollout_corr/rollout_ppl:np.float64(1.5290372776798904) - rollout_corr/rollout_log_ppl:np.float64(0.411310548952315) - rollout_corr/log_ppl_diff:np.float64(0.0008754922018852085) - rollout_corr/log_ppl_abs_diff:np.float64(0.002070396934868768) - rollout_corr/log_ppl_diff_max:np.float64(0.016439110040664673) - rollout_corr/log_ppl_diff_min:np.float64(-0.00867973268032074) - rollout_corr/ppl_ratio:np.float64(1.0008794867899269) - rollout_corr/chi2_token:np.float64(0.0017218261491507292) - rollout_corr/chi2_seq:np.float64(0.5381895105820149) - actor/pg_loss:np.float64(0.006413624971173704) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005543413315081125) - actor/ppo_kl:np.float64(9.862682850325655e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(21560.225262402324) - self_distillation/token_reweight_w_std:np.float64(340430.1742267248) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004913297016174) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10845586232608184) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.3543579652905464) - perf/mfu/actor:np.float64(0.04868076222396018) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.25546646118164) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011497573927044868 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011497573927044868 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:358.26171875 - response_length/max:1148.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:358.26171875 - response_length_non_aborted/max:1148.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.78125 - prompt_length/max:597.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010633282363414764 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4096278883516788) - timing_s/agent_loop/generate_sequences/max:np.float64(9.100342744961381) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.021014133017161) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.100342744961381) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:1148 - timing_s/gen:15.916911805048585 - timing_s/reward:0.059211257845163345 - timing_s/old_log_prob:3.0376540310680866 - timing_s/adv:0.5192954838275909 - timing_s/update_actor:21.510440975427628 - timing_s/step:41.131219344213605 - timing_s/stop_profile:3.115646541118622e-05 - timing_per_token_ms/update_actor:0.13357618514874176 - timing_per_token_ms/gen:0.17354753099327902 - timing_per_token_ms/adv:0.003224736758019008 - perf/total_num_tokens:161035 - perf/time_per_step:41.131219344213605 - perf/throughput:489.3940739160661 (TaskRunner pid=2255397) Training Progress: 54%|█████▍ | 54/100 [49:42<39:13, 51.16s/it] (TaskRunner pid=2255397) step:55 - global_seqlen/min:19138 - global_seqlen/max:21905 - global_seqlen/minmax_diff:2767 - global_seqlen/balanced_min:20379 - global_seqlen/balanced_max:20392 - global_seqlen/mean:20387.375 - actor/entropy:0.46607786417007446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44003328680992126 - training/rollout_probs_diff_mean:0.005262596998363733 - training/rollout_probs_diff_std:0.011458019725978374 - training/rollout_actor_probs_pearson_corr:0.9992002844810486 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000497102737427 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.523318446241319e-05 - rollout_corr/rollout_is_max:1.6244064569473267 - rollout_corr/rollout_is_min:0.396142840385437 - rollout_corr/rollout_is_std:0.03780318796634674 - rollout_corr/rollout_is_eff_sample_size:0.99857307726417 - rollout_corr/rollout_is_seq_mean:0.9999815225601196 - rollout_corr/rollout_is_seq_std:0.002393631497398019 - rollout_corr/rollout_is_seq_max:1.0126123428344727 - rollout_corr/rollout_is_seq_min:0.9941840171813965 - rollout_corr/rollout_is_seq_max_deviation:0.012612342834472656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.630201779305935) - rollout_corr/training_log_ppl:np.float64(0.47879417467629537) - rollout_corr/kl:np.float64(0.0010309481362873463) - rollout_corr/k3_kl:np.float64(0.0008061554526079817) - rollout_corr/rollout_ppl:np.float64(1.6284840498119593) - rollout_corr/rollout_log_ppl:np.float64(0.4777632247423753) - rollout_corr/log_ppl_diff:np.float64(0.0010309499339200556) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020579289994202554) - rollout_corr/log_ppl_diff_max:np.float64(0.007941722869873047) - rollout_corr/log_ppl_diff_min:np.float64(-0.0057179927825927734) - rollout_corr/ppl_ratio:np.float64(1.0010342469904572) - rollout_corr/chi2_token:np.float64(0.00116553227417171) - rollout_corr/chi2_seq:np.float64(0.674807739444077) - actor/pg_loss:np.float64(0.004637886886484921) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022305224865704076) - actor/ppo_kl:np.float64(0.0002447084712073888) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(5661.525033625308) - self_distillation/token_reweight_w_std:np.float64(108029.05970195204) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004851918201894) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08942344659590162) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.34092599898576736) - perf/mfu/actor:np.float64(0.04916186108901947) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.24573516845703) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027775494381785393 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027775494381785393 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:332.60546875 - response_length/max:929.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:332.60546875 - response_length_non_aborted/max:929.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.5 - prompt_length/max:625.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5229528546333313e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2314282320439816) - timing_s/agent_loop/generate_sequences/max:np.float64(7.571410246193409) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8092300324497046) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.571410246193409) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:337 - timing_s/agent_loop/slowest/response_length:895 - timing_s/gen:14.387957438826561 - timing_s/reward:0.05958708934485912 - timing_s/old_log_prob:2.9241322185844183 - timing_s/adv:0.5914540458470583 - timing_s/update_actor:21.573941450566053 - timing_s/step:39.624077497050166 - timing_s/stop_profile:0.0001155603677034378 - timing_per_token_ms/update_actor:0.13227513013915507 - timing_per_token_ms/gen:0.16897785522480605 - timing_per_token_ms/adv:0.003626349921502022 - perf/total_num_tokens:163099 - perf/time_per_step:39.624077497050166 - perf/throughput:514.5198648856305 (TaskRunner pid=2255397) Training Progress: 55%|█████▌ | 55/100 [50:22<35:47, 47.71s/it] (TaskRunner pid=2255397) step:56 - global_seqlen/min:16165 - global_seqlen/max:23611 - global_seqlen/minmax_diff:7446 - global_seqlen/balanced_min:19603 - global_seqlen/balanced_max:19611 - global_seqlen/mean:19608.875 - actor/entropy:0.42714759707450867 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2066904902458191 - training/rollout_probs_diff_mean:0.004840976092964411 - training/rollout_probs_diff_std:0.011059464886784554 - training/rollout_actor_probs_pearson_corr:0.9992184638977051 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.0001850128173828 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7041244506835938 - rollout_corr/rollout_is_min:0.5909921526908875 - rollout_corr/rollout_is_std:0.03720882534980774 - rollout_corr/rollout_is_eff_sample_size:0.9986178930118242 - rollout_corr/rollout_is_seq_mean:1.0002026557922363 - rollout_corr/rollout_is_seq_std:0.0023563196882605553 - rollout_corr/rollout_is_seq_max:1.0108153820037842 - rollout_corr/rollout_is_seq_min:0.9901830554008484 - rollout_corr/rollout_is_seq_max_deviation:0.01081538200378418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5864489898085594) - rollout_corr/training_log_ppl:np.float64(0.4489033415738959) - rollout_corr/kl:np.float64(0.0005826077157036202) - rollout_corr/k3_kl:np.float64(0.0008374885150033151) - rollout_corr/rollout_ppl:np.float64(1.5854855361394584) - rollout_corr/rollout_log_ppl:np.float64(0.4483207310840953) - rollout_corr/log_ppl_diff:np.float64(0.0005826104898005724) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019630271708592772) - rollout_corr/log_ppl_diff_max:np.float64(0.011552989482879639) - rollout_corr/log_ppl_diff_min:np.float64(-0.00930735468864441) - rollout_corr/ppl_ratio:np.float64(1.0005861730314791) - rollout_corr/chi2_token:np.float64(0.0022083548828959465) - rollout_corr/chi2_seq:np.float64(1.1513643299695104) - actor/pg_loss:np.float64(0.0053109838627278805) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004679971348195977) - actor/ppo_kl:np.float64(4.2931282393965375e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(133.22875495348126) - self_distillation/token_reweight_w_std:np.float64(2217.535897645983) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020651405211538) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08932297839783132) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.3138376846909523) - perf/mfu/actor:np.float64(0.04696252700818672) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.17254257202148) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004433032590895891 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.004433032590895891 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:347.62109375 - response_length/max:909.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:347.62109375 - response_length_non_aborted/max:909.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.15625 - prompt_length/max:458.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.741447865962982e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4314610064029694) - timing_s/agent_loop/generate_sequences/max:np.float64(7.775194456800818) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5458272197138285) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.775194456800818) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:230 - timing_s/agent_loop/slowest/response_length:909 - timing_s/gen:14.528320418670774 - timing_s/reward:0.0595672782510519 - timing_s/old_log_prob:2.9559899885207415 - timing_s/adv:0.5271639805287123 - timing_s/update_actor:21.64877647906542 - timing_s/step:39.80573454126716 - timing_s/stop_profile:2.855248749256134e-05 - timing_per_token_ms/update_actor:0.13800368760998158 - timing_per_token_ms/gen:0.16325606430617448 - timing_per_token_ms/adv:0.00336049352989853 - perf/total_num_tokens:156871 - perf/time_per_step:39.80573454126716 - perf/throughput:492.61432368924653 (TaskRunner pid=2255397) Training Progress: 56%|█████▌ | 56/100 [51:02<33:15, 45.35s/it] (TaskRunner pid=2255397) step:57 - global_seqlen/min:17187 - global_seqlen/max:22734 - global_seqlen/minmax_diff:5547 - global_seqlen/balanced_min:19426 - global_seqlen/balanced_max:19436 - global_seqlen/mean:19433.875 - actor/entropy:0.419066846370697 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45647868514060974 - training/rollout_probs_diff_mean:0.005257493816316128 - training/rollout_probs_diff_std:0.012042389251291752 - training/rollout_actor_probs_pearson_corr:0.9990561008453369 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9998766779899597 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.874779187957756e-05 - rollout_corr/rollout_is_max:1.9517626762390137 - rollout_corr/rollout_is_min:0.412737101316452 - rollout_corr/rollout_is_std:0.039478376507759094 - rollout_corr/rollout_is_eff_sample_size:0.9984436453453855 - rollout_corr/rollout_is_seq_mean:0.9998682141304016 - rollout_corr/rollout_is_seq_std:0.002758133690804243 - rollout_corr/rollout_is_seq_max:1.010321855545044 - rollout_corr/rollout_is_seq_min:0.9932553768157959 - rollout_corr/rollout_is_seq_max_deviation:0.010321855545043945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5485623660497367) - rollout_corr/training_log_ppl:np.float64(0.4282529861666262) - rollout_corr/kl:np.float64(0.0008480568455153659) - rollout_corr/k3_kl:np.float64(0.0008880625406391118) - rollout_corr/rollout_ppl:np.float64(1.5472203940153122) - rollout_corr/rollout_log_ppl:np.float64(0.42740493212477304) - rollout_corr/log_ppl_diff:np.float64(0.0008480540418531746) - rollout_corr/log_ppl_abs_diff:np.float64(0.002164474077289924) - rollout_corr/log_ppl_diff_max:np.float64(0.007681757211685181) - rollout_corr/log_ppl_diff_min:np.float64(-0.008211076259613037) - rollout_corr/ppl_ratio:np.float64(1.0008519866969436) - rollout_corr/chi2_token:np.float64(0.0018828820902854204) - rollout_corr/chi2_seq:np.float64(1.0425616323482245) - actor/pg_loss:np.float64(0.002906394307501614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029778121393064794) - actor/ppo_kl:np.float64(-0.00014163734993388744) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(3090.494025749387) - self_distillation/token_reweight_w_std:np.float64(77747.68742029642) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994963144417852) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06993347342358902) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.22349200583994389) - perf/mfu/actor:np.float64(0.04662792309159809) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.20368194580078) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013542745728045702 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013542745728045702 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:320.52734375 - response_length/max:1046.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:320.52734375 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.78125 - prompt_length/max:660.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.3910775780677795e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5050442274659872) - timing_s/agent_loop/generate_sequences/max:np.float64(8.29741439782083) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6069312687322963) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.29741439782083) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:15.101491201668978 - timing_s/reward:0.05853153392672539 - timing_s/old_log_prob:2.997370980679989 - timing_s/adv:0.5432093199342489 - timing_s/update_actor:21.456135919317603 - timing_s/step:40.24460173584521 - timing_s/stop_profile:0.00010164454579353333 - timing_per_token_ms/update_actor:0.13800731917410708 - timing_per_token_ms/gen:0.18404108465869207 - timing_per_token_ms/adv:0.003493959130218812 - perf/total_num_tokens:155471 - perf/time_per_step:40.24460173584521 - perf/throughput:482.89395749419396 (TaskRunner pid=2255397) Training Progress: 57%|█████▋ | 57/100 [51:42<31:24, 43.83s/it] (TaskRunner pid=2255397) step:58 - global_seqlen/min:17591 - global_seqlen/max:22208 - global_seqlen/minmax_diff:4617 - global_seqlen/balanced_min:19936 - global_seqlen/balanced_max:19980 - global_seqlen/mean:19944.0 - actor/entropy:0.39805784821510315 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5443886518478394 - training/rollout_probs_diff_mean:0.0047958018258214 - training/rollout_probs_diff_std:0.011325978673994541 - training/rollout_actor_probs_pearson_corr:0.9991068243980408 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999314546585083 - rollout_corr/rollout_is_ratio_fraction_high:1.1276500117674004e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1276500117674004e-05 - rollout_corr/rollout_is_max:2.245145320892334 - rollout_corr/rollout_is_min:0.3792571425437927 - rollout_corr/rollout_is_std:0.037710048258304596 - rollout_corr/rollout_is_eff_sample_size:0.9985797339908784 - rollout_corr/rollout_is_seq_mean:0.9998851418495178 - rollout_corr/rollout_is_seq_std:0.002318696118891239 - rollout_corr/rollout_is_seq_max:1.007555365562439 - rollout_corr/rollout_is_seq_min:0.9894842505455017 - rollout_corr/rollout_is_seq_max_deviation:0.010515749454498291 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5350726060569286) - rollout_corr/training_log_ppl:np.float64(0.4149510355782695) - rollout_corr/kl:np.float64(0.0008157491595808253) - rollout_corr/k3_kl:np.float64(0.0008106242991630097) - rollout_corr/rollout_ppl:np.float64(1.533764130435884) - rollout_corr/rollout_log_ppl:np.float64(0.41413528152043) - rollout_corr/log_ppl_diff:np.float64(0.0008157540578395128) - rollout_corr/log_ppl_abs_diff:np.float64(0.00186522479634732) - rollout_corr/log_ppl_diff_max:np.float64(0.012087315320968628) - rollout_corr/log_ppl_diff_min:np.float64(-0.005733311176300049) - rollout_corr/ppl_ratio:np.float64(1.0008189915679395) - rollout_corr/chi2_token:np.float64(0.0016144579276442528) - rollout_corr/chi2_seq:np.float64(0.5739311659708619) - actor/pg_loss:np.float64(0.0032648365013301373) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039790487971913535) - actor/ppo_kl:np.float64(-6.927822899172043e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(20.40596836898476) - self_distillation/token_reweight_w_std:np.float64(398.28047909680754) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010924299713224) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060014044371200725) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.22483210638165474) - perf/mfu/actor:np.float64(0.048109296920814684) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.0834503173828) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00522806728258729 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00522806728258729 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:346.40625 - response_length/max:1083.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:346.40625 - response_length_non_aborted/max:1083.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:598.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012578070163726807 - timing_s/agent_loop/generate_sequences/min:np.float64(1.711790930479765) - timing_s/agent_loop/generate_sequences/max:np.float64(8.412942063063383) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.874052923943964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.412942063063383) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:312 - timing_s/agent_loop/slowest/response_length:1083 - timing_s/gen:14.847420612350106 - timing_s/reward:0.05969010666012764 - timing_s/old_log_prob:2.983126612380147 - timing_s/adv:0.5337398238480091 - timing_s/update_actor:21.578375332057476 - timing_s/step:40.09062473848462 - timing_s/stop_profile:0.00010734237730503082 - timing_per_token_ms/update_actor:0.1352435277029274 - timing_per_token_ms/gen:0.16742693518662727 - timing_per_token_ms/adv:0.003345240572653487 - perf/total_num_tokens:159552 - perf/time_per_step:40.09062473848462 - perf/throughput:497.4729161767076 (TaskRunner pid=2255397) Training Progress: 58%|█████▊ | 58/100 [52:22<29:54, 42.72s/it] (TaskRunner pid=2255397) step:59 - global_seqlen/min:16516 - global_seqlen/max:23158 - global_seqlen/minmax_diff:6642 - global_seqlen/balanced_min:19285 - global_seqlen/balanced_max:19288 - global_seqlen/mean:19286.5 - actor/entropy:0.39392659068107605 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105931282043457 - training/rollout_probs_diff_mean:0.004885942675173283 - training/rollout_probs_diff_std:0.01138182170689106 - training/rollout_actor_probs_pearson_corr:0.9991214275360107 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65234375 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65234375 - rollout_corr/rollout_is_mean:0.9998358488082886 - rollout_corr/rollout_is_ratio_fraction_high:1.1690437531797215e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1690437531797215e-05 - rollout_corr/rollout_is_max:3.892078161239624 - rollout_corr/rollout_is_min:0.4614630937576294 - rollout_corr/rollout_is_std:0.03779766708612442 - rollout_corr/rollout_is_eff_sample_size:0.9985728395255719 - rollout_corr/rollout_is_seq_mean:0.999951183795929 - rollout_corr/rollout_is_seq_std:0.002800993388518691 - rollout_corr/rollout_is_seq_max:1.0143204927444458 - rollout_corr/rollout_is_seq_min:0.9928278923034668 - rollout_corr/rollout_is_seq_max_deviation:0.0143204927444458 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5359033830463886) - rollout_corr/training_log_ppl:np.float64(0.4180586190195754) - rollout_corr/kl:np.float64(0.0011108192077600165) - rollout_corr/k3_kl:np.float64(0.0008730394356462057) - rollout_corr/rollout_ppl:np.float64(1.5341403023339808) - rollout_corr/rollout_log_ppl:np.float64(0.41694779935642146) - rollout_corr/log_ppl_diff:np.float64(0.0011108196631539613) - rollout_corr/log_ppl_abs_diff:np.float64(0.002291953394887969) - rollout_corr/log_ppl_diff_max:np.float64(0.008277744054794312) - rollout_corr/log_ppl_diff_min:np.float64(-0.007644861936569214) - rollout_corr/ppl_ratio:np.float64(1.0011151337530464) - rollout_corr/chi2_token:np.float64(0.0013092008884996176) - rollout_corr/chi2_seq:np.float64(0.4192327775526792) - actor/pg_loss:np.float64(0.0025457896990701556) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003396774759494292) - actor/ppo_kl:np.float64(0.0002558892238049282) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(9165.766619424336) - self_distillation/token_reweight_w_std:np.float64(181822.19756039116) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998559767846018) - self_distillation/token_reweight_w_clip_frac:np.float64(0.057435350317973644) - self_distillation/empty_target_batch:np.float64(0.34765625) - actor/grad_norm:np.float64(0.21796750277280807) - perf/mfu/actor:np.float64(0.0472734036832577) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(198.940327167511) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026581131387501955 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026581131387501955 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.140625 - response_length/max:859.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.140625 - response_length_non_aborted/max:859.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:615.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001378711313009262 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6116437073796988) - timing_s/agent_loop/generate_sequences/max:np.float64(7.250096688047051) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7876410359021975) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.250096688047051) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:217 - timing_s/agent_loop/slowest/response_length:844 - timing_s/gen:14.891584478318691 - timing_s/reward:0.05919893458485603 - timing_s/old_log_prob:2.9019656274467707 - timing_s/adv:0.4909003674983978 - timing_s/update_actor:21.302226703613997 - timing_s/step:39.73218045942485 - timing_s/stop_profile:0.00011939182877540588 - timing_per_token_ms/update_actor:0.1380643630493739 - timing_per_token_ms/gen:0.17408913348513785 - timing_per_token_ms/adv:0.0031816320191480948 - perf/total_num_tokens:154292 - perf/time_per_step:39.73218045942485 - perf/throughput:485.41257431606823 (TaskRunner pid=2255397) Training Progress: 59%|█████▉ | 59/100 [53:02<28:35, 41.85s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the structures of the given options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product contains a nitrogen-containing ring with a substituent group: a chlorine-substituted benzene ring connected to a trifluoromethyl group (C(F)(F)F). This suggests that the correct reactant must contain a similar structure, specifically a nitrogen-containing ring with the same substituents. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains a nitrogen-containing ring with the exact substituents: a chlorine-substituted benzene ring and a trifluoromethyl group. This matches the structure of the product exactly. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Option B: Contains a nitrogen but not the correct substituents. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Option C: Contains a hydroxyl group and a different substituent, not matching the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Option D: Contains a different structure with a carbon chain and a fluorinated group, not matching the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Therefore, the only correct reactant is Option A. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2255397) step:60 - global_seqlen/min:15969 - global_seqlen/max:20753 - global_seqlen/minmax_diff:4784 - global_seqlen/balanced_min:19004 - global_seqlen/balanced_max:19008 - global_seqlen/mean:19006.625 - actor/entropy:0.40371620655059814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8045270442962646 - training/rollout_probs_diff_mean:0.005058089271187782 - training/rollout_probs_diff_std:0.01212889701128006 - training/rollout_actor_probs_pearson_corr:0.9990420341491699 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.9997034668922424 - rollout_corr/rollout_is_ratio_fraction_high:1.1908589840459172e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.763435936183669e-05 - rollout_corr/rollout_is_max:2.487139940261841 - rollout_corr/rollout_is_min:0.0640438050031662 - rollout_corr/rollout_is_std:0.03801152855157852 - rollout_corr/rollout_is_eff_sample_size:0.9985564358356739 - rollout_corr/rollout_is_seq_mean:0.9996296167373657 - rollout_corr/rollout_is_seq_std:0.0025690472684800625 - rollout_corr/rollout_is_seq_max:1.0076467990875244 - rollout_corr/rollout_is_seq_min:0.9901750683784485 - rollout_corr/rollout_is_seq_max_deviation:0.009824931621551514 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.534271144773811) - rollout_corr/training_log_ppl:np.float64(0.41600528117851354) - rollout_corr/kl:np.float64(0.001141798264093552) - rollout_corr/k3_kl:np.float64(0.000918725037195145) - rollout_corr/rollout_ppl:np.float64(1.5324818124063313) - rollout_corr/rollout_log_ppl:np.float64(0.4148634799348656) - rollout_corr/log_ppl_diff:np.float64(0.001141801243647933) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023173787049017847) - rollout_corr/log_ppl_diff_max:np.float64(0.009830474853515625) - rollout_corr/log_ppl_diff_min:np.float64(-0.006625443696975708) - rollout_corr/ppl_ratio:np.float64(1.0011462338734418) - rollout_corr/chi2_token:np.float64(0.0013863001950085163) - rollout_corr/chi2_seq:np.float64(0.6778646670281887) - actor/pg_loss:np.float64(0.005434742895886302) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011875206650984182) - actor/ppo_kl:np.float64(-4.705816970229648e-05) - actor/pg_clipfrac_lower:np.float64(2.3182492441264912e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(18888.49168556533) - self_distillation/token_reweight_w_std:np.float64(311593.95773308317) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0030431225895882) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10766072731348686) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.35879671573638916) - perf/mfu/actor:np.float64(0.045344773814958786) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.29048919677734) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6875) - val-aux/sciknoweval/reward/std@16:np.float64(0.16029546205916068) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7511857142857143) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13365319867771744) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6229333333333333) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13860952641542026) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6874666666666667) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16050258419670824) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8023714285714285) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10001117146576141) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5698666666666666) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10827654197635113) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.694952380952381) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12671450210804772) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8414666666666667) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07147207072829928) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5278238095238095) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07866088591113561) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7011380952380954) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09304988511758287) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8708) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0457160969007615) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.49634285714285714) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05141404847998308) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7056047619047618) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06605128200501377) - val-aux/sciknoweval/score/mean@16:np.float64(0.6875) - val-aux/sciknoweval/score/std@16:np.float64(0.16029546205916068) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7511857142857143) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13365319867771744) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6229333333333333) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13860952641542026) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6874666666666667) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16050258419670824) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8023714285714285) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10001117146576141) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5698666666666666) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10827654197635113) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.694952380952381) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12671450210804772) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8414666666666667) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07147207072829928) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5278238095238095) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07866088591113561) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7011380952380954) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09304988511758287) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8708) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0457160969007615) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.49634285714285714) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05141404847998308) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7056047619047618) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06605128200501377) - val-core/sciknoweval/acc/mean@16:np.float64(0.6875) - val-aux/sciknoweval/acc/std@16:np.float64(0.16029546205916068) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7511857142857143) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13365319867771744) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6229333333333333) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13860952641542026) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6874666666666667) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16050258419670824) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8023714285714285) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10001117146576141) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5698666666666666) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10827654197635113) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.694952380952381) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12671450210804772) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8414666666666667) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07147207072829928) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5278238095238095) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07866088591113561) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7011380952380954) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09304988511758287) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8708) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0457160969007615) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.49634285714285714) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05141404847998308) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7056047619047618) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06605128200501377) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9982142857142857) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.006185546430642514) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9998571428571429) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0017362747371820737) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9967380952380952) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.008056772590456738) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9983857142857143) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.005874140160911555) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.00015050933932646775) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9938904761904761) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01020614644674161) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9993380952380951) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0038135892390515993) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9893857142857142) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.011570601968058545) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9998333333333334) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00189074176643801) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9839095238095237) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.01075169136568833) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999857142857144) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00036325512888540755) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:1 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029056959319859743 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0029056959319859743 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:328.01953125 - response_length/max:944.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:328.01953125 - response_length_non_aborted/max:944.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.9375 - prompt_length/max:451.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00029645487666130066 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6444534286856651) - timing_s/agent_loop/generate_sequences/max:np.float64(7.757375167682767) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.735986279869394) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.757375167682767) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:187 - timing_s/agent_loop/slowest/response_length:944 - timing_s/gen:14.574015535414219 - timing_s/reward:0.05933819338679314 - timing_s/old_log_prob:3.192529769614339 - timing_s/adv:0.5636057835072279 - timing_s/update_actor:21.57127202488482 - timing_s/step:40.06707218103111 - timing_s/testing:128.52822875231504 - timing_s/save_checkpoint:13.383056795224547 - timing_s/stop_profile:0.00014889799058437347 - timing_per_token_ms/update_actor:0.14186679660963494 - timing_per_token_ms/gen:0.17355597079316232 - timing_per_token_ms/adv:0.0037066403392713586 - perf/total_num_tokens:152053 - perf/time_per_step:40.06707218103111 - perf/throughput:474.3701989035844 (TaskRunner pid=2255397) Training Progress: 60%|██████ | 60/100 [56:05<56:00, 84.02s/it] (TaskRunner pid=2255397) step:61 - global_seqlen/min:15598 - global_seqlen/max:23498 - global_seqlen/minmax_diff:7900 - global_seqlen/balanced_min:19661 - global_seqlen/balanced_max:19666 - global_seqlen/mean:19664.0 - actor/entropy:0.375679075717926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102344214916229 - training/rollout_probs_diff_mean:0.00489032082259655 - training/rollout_probs_diff_std:0.011647011153399944 - training/rollout_actor_probs_pearson_corr:0.9990482330322266 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.00008225440979 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1082543096563313e-05 - rollout_corr/rollout_is_max:1.639464259147644 - rollout_corr/rollout_is_min:0.3959808051586151 - rollout_corr/rollout_is_std:0.03841473534703255 - rollout_corr/rollout_is_eff_sample_size:0.9985268392369275 - rollout_corr/rollout_is_seq_mean:1.000138759613037 - rollout_corr/rollout_is_seq_std:0.002528289332985878 - rollout_corr/rollout_is_seq_max:1.007035493850708 - rollout_corr/rollout_is_seq_min:0.993572473526001 - rollout_corr/rollout_is_seq_max_deviation:0.007035493850708008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4903184147551656) - rollout_corr/training_log_ppl:np.float64(0.3903155774460174) - rollout_corr/kl:np.float64(0.0006912411112880434) - rollout_corr/k3_kl:np.float64(0.0009254292584728319) - rollout_corr/rollout_ppl:np.float64(1.4893131870776415) - rollout_corr/rollout_log_ppl:np.float64(0.38962433370761573) - rollout_corr/log_ppl_diff:np.float64(0.0006912437384016812) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020783296204172075) - rollout_corr/log_ppl_diff_max:np.float64(0.008693814277648926) - rollout_corr/log_ppl_diff_min:np.float64(-0.006119638681411743) - rollout_corr/ppl_ratio:np.float64(1.0006948560476303) - rollout_corr/chi2_token:np.float64(0.002362349070608616) - rollout_corr/chi2_seq:np.float64(0.9650198624003679) - actor/pg_loss:np.float64(0.005624711047858) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006064838689781027) - actor/ppo_kl:np.float64(1.730111737785478e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(5193.730784527026) - self_distillation/token_reweight_w_std:np.float64(104429.90018021135) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993646144866943) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08162092015845701) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.354709729552269) - perf/mfu/actor:np.float64(0.04738440538771457) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.02718353271484) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004777961410582066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004777961410582066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.46875 - response_length/max:838.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.46875 - response_length_non_aborted/max:838.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.03125 - prompt_length/max:462.0 - prompt_length/min:177.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.24973326921463e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6630962695926428) - timing_s/agent_loop/generate_sequences/max:np.float64(7.389603292569518) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.984407690099033) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.389603292569518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:830 - timing_s/gen:13.70044944435358 - timing_s/reward:0.06281237117946148 - timing_s/old_log_prob:3.0475483629852533 - timing_s/adv:0.5286510195583105 - timing_s/update_actor:21.522440258413553 - timing_s/step:38.972098929807544 - timing_s/stop_profile:0.0001547839492559433 - timing_per_token_ms/update_actor:0.13681372214715695 - timing_per_token_ms/gen:0.15183581705330237 - timing_per_token_ms/adv:0.0033605257040677794 - perf/total_num_tokens:157312 - perf/time_per_step:38.972098929807544 - perf/throughput:504.5661008768538 (TaskRunner pid=2255397) Training Progress: 61%|██████ | 61/100 [56:44<45:50, 70.52s/it] (TaskRunner pid=2255397) step:62 - global_seqlen/min:17175 - global_seqlen/max:22516 - global_seqlen/minmax_diff:5341 - global_seqlen/balanced_min:19524 - global_seqlen/balanced_max:19535 - global_seqlen/mean:19531.875 - actor/entropy:0.4294959306716919 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20105724036693573 - training/rollout_probs_diff_mean:0.005539817735552788 - training/rollout_probs_diff_std:0.012213723734021187 - training/rollout_actor_probs_pearson_corr:0.9990189075469971 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000041723251343 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9759001731872559 - rollout_corr/rollout_is_min:0.5684848427772522 - rollout_corr/rollout_is_std:0.04096618667244911 - rollout_corr/rollout_is_eff_sample_size:0.9983247021636537 - rollout_corr/rollout_is_seq_mean:0.9999959468841553 - rollout_corr/rollout_is_seq_std:0.0028718190733343363 - rollout_corr/rollout_is_seq_max:1.008870244026184 - rollout_corr/rollout_is_seq_min:0.987141489982605 - rollout_corr/rollout_is_seq_max_deviation:0.01285851001739502 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.584152389317751) - rollout_corr/training_log_ppl:np.float64(0.4482707906863652) - rollout_corr/kl:np.float64(0.001287838817653153) - rollout_corr/k3_kl:np.float64(0.0013142074097913792) - rollout_corr/rollout_ppl:np.float64(1.5821097679436207) - rollout_corr/rollout_log_ppl:np.float64(0.4469829489826225) - rollout_corr/log_ppl_diff:np.float64(0.0012878417037427425) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026649965438991785) - rollout_corr/log_ppl_diff_max:np.float64(0.014283061027526855) - rollout_corr/log_ppl_diff_min:np.float64(-0.010016441345214844) - rollout_corr/ppl_ratio:np.float64(1.0012943353503942) - rollout_corr/chi2_token:np.float64(0.002664147410541773) - rollout_corr/chi2_seq:np.float64(0.6478703722823411) - actor/pg_loss:np.float64(0.005159451276995242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001237628416674852) - actor/ppo_kl:np.float64(0.0003923675134416271) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(9608.619164069882) - self_distillation/token_reweight_w_std:np.float64(147490.825859931) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000067490618676) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10762355217593722) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.41205330193042755) - perf/mfu/actor:np.float64(0.04771627973118822) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.9746551513672) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012106545269489288 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012106545269489288 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:311.12109375 - response_length/max:1015.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:311.12109375 - response_length_non_aborted/max:1015.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.25 - prompt_length/max:662.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013557448983192444 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3771552108228207) - timing_s/agent_loop/generate_sequences/max:np.float64(7.987416280433536) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6242045445760596) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.987416280433536) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:1015 - timing_s/gen:14.992265343666077 - timing_s/reward:0.06058591604232788 - timing_s/old_log_prob:2.887193411588669 - timing_s/adv:0.5111278612166643 - timing_s/update_actor:21.240903863683343 - timing_s/step:39.78005450963974 - timing_s/stop_profile:0.00012206099927425385 - timing_per_token_ms/update_actor:0.13593743472966205 - timing_per_token_ms/gen:0.18823389887461017 - timing_per_token_ms/adv:0.0032711136361502947 - perf/total_num_tokens:156255 - perf/time_per_step:39.78005450963974 - perf/throughput:490.99668768093113 (TaskRunner pid=2255397) Training Progress: 62%|██████▏ | 62/100 [57:23<38:49, 61.31s/it] (TaskRunner pid=2255397) step:63 - global_seqlen/min:15986 - global_seqlen/max:20551 - global_seqlen/minmax_diff:4565 - global_seqlen/balanced_min:18758 - global_seqlen/balanced_max:18761 - global_seqlen/mean:18759.625 - actor/entropy:0.35515502095222473 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24133962392807007 - training/rollout_probs_diff_mean:0.004971614107489586 - training/rollout_probs_diff_std:0.012004957534372807 - training/rollout_actor_probs_pearson_corr:0.9989368915557861 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.53125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001026391983032 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3019985999562778e-05 - rollout_corr/rollout_is_max:1.8590831756591797 - rollout_corr/rollout_is_min:0.4984112083911896 - rollout_corr/rollout_is_std:0.03878994286060333 - rollout_corr/rollout_is_eff_sample_size:0.998497719797901 - rollout_corr/rollout_is_seq_mean:1.000045895576477 - rollout_corr/rollout_is_seq_std:0.002819458954036236 - rollout_corr/rollout_is_seq_max:1.0079857110977173 - rollout_corr/rollout_is_seq_min:0.9885396957397461 - rollout_corr/rollout_is_seq_max_deviation:0.011460304260253906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4885337855666876) - rollout_corr/training_log_ppl:np.float64(0.38630677238688804) - rollout_corr/kl:np.float64(0.0008763675811707117) - rollout_corr/k3_kl:np.float64(0.0011061657141908654) - rollout_corr/rollout_ppl:np.float64(1.4872424700297415) - rollout_corr/rollout_log_ppl:np.float64(0.3854304051492363) - rollout_corr/log_ppl_diff:np.float64(0.0008763672376517206) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025414098927285522) - rollout_corr/log_ppl_diff_max:np.float64(0.016067981719970703) - rollout_corr/log_ppl_diff_min:np.float64(-0.010530859231948853) - rollout_corr/ppl_ratio:np.float64(1.0008826348930597) - rollout_corr/chi2_token:np.float64(0.002648768713697791) - rollout_corr/chi2_seq:np.float64(1.621614676201716) - actor/pg_loss:np.float64(0.004201351199299097) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006837096566414402) - actor/ppo_kl:np.float64(9.11460165120559e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53125) - self_distillation/token_reweight_w_mean:np.float64(328.2442679554224) - self_distillation/token_reweight_w_std:np.float64(5757.5124940559035) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0031547287944704) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0877059873600956) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.3450646325945854) - perf/mfu/actor:np.float64(0.04625351205637495) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.83617782592773) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001894407905638218 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001894407905638218 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:300.01953125 - response_length/max:723.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:300.01953125 - response_length_non_aborted/max:723.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.21875 - prompt_length/max:532.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013262778520584106 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4198575951159) - timing_s/agent_loop/generate_sequences/max:np.float64(6.652159977704287) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4228165260647074) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.652159977704287) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:723 - timing_s/gen:13.547540098428726 - timing_s/reward:0.06083318777382374 - timing_s/old_log_prob:2.9801845103502274 - timing_s/adv:0.5016210991889238 - timing_s/update_actor:21.087883727625012 - timing_s/step:38.26442988216877 - timing_s/stop_profile:0.00010959990322589874 - timing_per_token_ms/update_actor:0.1405137611201251 - timing_per_token_ms/gen:0.17638877805388617 - timing_per_token_ms/adv:0.003342424883152807 - perf/total_num_tokens:150077 - perf/time_per_step:38.26442988216877 - perf/throughput:490.26275989916127 (TaskRunner pid=2255397) Training Progress: 63%|██████▎ | 63/100 [58:02<33:33, 54.41s/it] (TaskRunner pid=2255397) step:64 - global_seqlen/min:15100 - global_seqlen/max:20599 - global_seqlen/minmax_diff:5499 - global_seqlen/balanced_min:18082 - global_seqlen/balanced_max:18090 - global_seqlen/mean:18088.125 - actor/entropy:0.3758540451526642 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24836724996566772 - training/rollout_probs_diff_mean:0.005264882929623127 - training/rollout_probs_diff_std:0.012344393879175186 - training/rollout_actor_probs_pearson_corr:0.9989309310913086 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000416040420532 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.8669313905993477e-05 - rollout_corr/rollout_is_max:1.78631591796875 - rollout_corr/rollout_is_min:0.3966720402240753 - rollout_corr/rollout_is_std:0.04027950391173363 - rollout_corr/rollout_is_eff_sample_size:0.9983804272587137 - rollout_corr/rollout_is_seq_mean:1.0000438690185547 - rollout_corr/rollout_is_seq_std:0.0027673090808093548 - rollout_corr/rollout_is_seq_max:1.0073356628417969 - rollout_corr/rollout_is_seq_min:0.9913079738616943 - rollout_corr/rollout_is_seq_max_deviation:0.008692026138305664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.488935824483633) - rollout_corr/training_log_ppl:np.float64(0.3868299189489335) - rollout_corr/kl:np.float64(0.0007672079092806428) - rollout_corr/k3_kl:np.float64(0.0009736951585637144) - rollout_corr/rollout_ppl:np.float64(1.4878136315383017) - rollout_corr/rollout_log_ppl:np.float64(0.38606271165190265) - rollout_corr/log_ppl_diff:np.float64(0.0007672072970308363) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023564034490846097) - rollout_corr/log_ppl_diff_max:np.float64(0.011275023221969604) - rollout_corr/log_ppl_diff_min:np.float64(-0.008923828601837158) - rollout_corr/ppl_ratio:np.float64(1.000771946972236) - rollout_corr/chi2_token:np.float64(0.002396456664428115) - rollout_corr/chi2_seq:np.float64(0.5857332549057901) - actor/pg_loss:np.float64(0.0037766702007502317) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045624542644873145) - actor/ppo_kl:np.float64(-8.097473682511236e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(34646.944526499836) - self_distillation/token_reweight_w_std:np.float64(444681.0361109813) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998654320370406) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06305540914763696) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.2998654544353485) - perf/mfu/actor:np.float64(0.04428300784041782) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(199.82562017440796) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009668726474046707 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009668726474046707 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.50390625 - response_length/max:914.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.50390625 - response_length_non_aborted/max:914.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.75 - prompt_length/max:495.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010181218385696411 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3763788174837828) - timing_s/agent_loop/generate_sequences/max:np.float64(7.2169532254338264) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.232598840339051) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.2169532254338264) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:879 - timing_s/gen:14.560809908434749 - timing_s/reward:0.057523222640156746 - timing_s/old_log_prob:2.9381075855344534 - timing_s/adv:0.5264706443995237 - timing_s/update_actor:21.378882113844156 - timing_s/step:39.54835497029126 - timing_s/stop_profile:0.00011551380157470703 - timing_per_token_ms/update_actor:0.14774114311077127 - timing_per_token_ms/gen:0.2087242142233447 - timing_per_token_ms/adv:0.0036382339545939927 - perf/total_num_tokens:144705 - perf/time_per_step:39.54835497029126 - perf/throughput:457.3673168855647 (TaskRunner pid=2255397) Training Progress: 64%|██████▍ | 64/100 [58:41<29:58, 49.97s/it] (TaskRunner pid=2255397) step:65 - global_seqlen/min:15051 - global_seqlen/max:21236 - global_seqlen/minmax_diff:6185 - global_seqlen/balanced_min:18004 - global_seqlen/balanced_max:18008 - global_seqlen/mean:18006.625 - actor/entropy:0.3955051898956299 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1947164535522461 - training/rollout_probs_diff_mean:0.0055443341843783855 - training/rollout_probs_diff_std:0.012429744005203247 - training/rollout_actor_probs_pearson_corr:0.9989457726478577 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9999876618385315 - rollout_corr/rollout_is_ratio_fraction_high:1.3995213521411642e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.031123399734497 - rollout_corr/rollout_is_min:0.504069447517395 - rollout_corr/rollout_is_std:0.041325442492961884 - rollout_corr/rollout_is_eff_sample_size:0.9982950004968502 - rollout_corr/rollout_is_seq_mean:0.9999707341194153 - rollout_corr/rollout_is_seq_std:0.002948605688288808 - rollout_corr/rollout_is_seq_max:1.009615182876587 - rollout_corr/rollout_is_seq_min:0.9913778305053711 - rollout_corr/rollout_is_seq_max_deviation:0.009615182876586914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.525312386918813) - rollout_corr/training_log_ppl:np.float64(0.4125790528487414) - rollout_corr/kl:np.float64(0.00100548441882653) - rollout_corr/k3_kl:np.float64(0.001056205478960237) - rollout_corr/rollout_ppl:np.float64(1.5237079304642975) - rollout_corr/rollout_log_ppl:np.float64(0.4115735680970829) - rollout_corr/log_ppl_diff:np.float64(0.0010054847516585141) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024253172159660608) - rollout_corr/log_ppl_diff_max:np.float64(0.012961804866790771) - rollout_corr/log_ppl_diff_min:np.float64(-0.007540702819824219) - rollout_corr/ppl_ratio:np.float64(1.0010107702109963) - rollout_corr/chi2_token:np.float64(0.0022052584681659937) - rollout_corr/chi2_seq:np.float64(0.9272577015217394) - actor/pg_loss:np.float64(0.003355647437274456) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000539568421572767) - actor/ppo_kl:np.float64(7.455754089491506e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(178.37819157890044) - self_distillation/token_reweight_w_std:np.float64(3052.0747698339983) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989271517843008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07668937157723121) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.31669436022639275) - perf/mfu/actor:np.float64(0.04449943785630652) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.69052124023438) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013841266743838787 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013841266743838787 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:279.11328125 - response_length/max:754.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:279.11328125 - response_length_non_aborted/max:754.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:818.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.972788393497467e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2479094993323088) - timing_s/agent_loop/generate_sequences/max:np.float64(6.392431171610951) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2372494999872288) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.392431171610951) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:228 - timing_s/agent_loop/slowest/response_length:754 - timing_s/gen:13.095486616715789 - timing_s/reward:0.0594562403857708 - timing_s/old_log_prob:2.941354649141431 - timing_s/adv:0.5120643582195044 - timing_s/update_actor:20.915571719408035 - timing_s/step:37.61257514543831 - timing_s/stop_profile:0.00016906112432479858 - timing_per_token_ms/update_actor:0.14519358652307163 - timing_per_token_ms/gen:0.1832741328805759 - timing_per_token_ms/adv:0.003554694162700564 - perf/total_num_tokens:144053 - perf/time_per_step:37.61257514543831 - perf/throughput:478.7394888643741 (TaskRunner pid=2255397) Training Progress: 65%|██████▌ | 65/100 [59:19<26:59, 46.27s/it] (TaskRunner pid=2255397) step:66 - global_seqlen/min:16554 - global_seqlen/max:19900 - global_seqlen/minmax_diff:3346 - global_seqlen/balanced_min:18653 - global_seqlen/balanced_max:18704 - global_seqlen/mean:18662.625 - actor/entropy:0.3692823350429535 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24191009998321533 - training/rollout_probs_diff_mean:0.005031523294746876 - training/rollout_probs_diff_std:0.011805592104792595 - training/rollout_actor_probs_pearson_corr:0.998999834060669 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999994039535522 - rollout_corr/rollout_is_ratio_fraction_high:1.3333866263565142e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2511892318725586 - rollout_corr/rollout_is_min:0.5517903566360474 - rollout_corr/rollout_is_std:0.03924669697880745 - rollout_corr/rollout_is_eff_sample_size:0.998462065654954 - rollout_corr/rollout_is_seq_mean:1.0000253915786743 - rollout_corr/rollout_is_seq_std:0.002698667347431183 - rollout_corr/rollout_is_seq_max:1.0073356628417969 - rollout_corr/rollout_is_seq_min:0.992901086807251 - rollout_corr/rollout_is_seq_max_deviation:0.007335662841796875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5085271298885345) - rollout_corr/training_log_ppl:np.float64(0.4001465483015636) - rollout_corr/kl:np.float64(0.0009092299301867168) - rollout_corr/k3_kl:np.float64(0.001055825624348472) - rollout_corr/rollout_ppl:np.float64(1.5071750050410628) - rollout_corr/rollout_log_ppl:np.float64(0.3992373197543202) - rollout_corr/log_ppl_diff:np.float64(0.0009092285472434014) - rollout_corr/log_ppl_abs_diff:np.float64(0.002248107542982325) - rollout_corr/log_ppl_diff_max:np.float64(0.008349627256393433) - rollout_corr/log_ppl_diff_min:np.float64(-0.008058488368988037) - rollout_corr/ppl_ratio:np.float64(1.0009135268628597) - rollout_corr/chi2_token:np.float64(0.0025282949209213257) - rollout_corr/chi2_seq:np.float64(0.6103038967121392) - actor/pg_loss:np.float64(0.002379831741563976) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033230165354325436) - actor/ppo_kl:np.float64(8.482311253033004e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(14.794726703781635) - self_distillation/token_reweight_w_std:np.float64(315.56329661473865) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989614612422884) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04611051009851508) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.24986116960644722) - perf/mfu/actor:np.float64(0.044863735441974) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.69231033325195) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023150925990194082 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023150925990194082 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:292.95703125 - response_length/max:1040.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:292.95703125 - response_length_non_aborted/max:1040.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.25 - prompt_length/max:617.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014759600162506104 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7828350365161896) - timing_s/agent_loop/generate_sequences/max:np.float64(8.071565691381693) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.388918655909947) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.071565691381693) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:1040 - timing_s/gen:15.357409680262208 - timing_s/reward:0.0539607759565115 - timing_s/old_log_prob:2.9237958528101444 - timing_s/adv:0.5357848312705755 - timing_s/update_actor:21.778983931988478 - timing_s/step:40.74699264019728 - timing_s/stop_profile:0.00011424347758293152 - timing_per_token_ms/update_actor:0.14587299436700676 - timing_per_token_ms/gen:0.2047736533496301 - timing_per_token_ms/adv:0.0035886218529720196 - perf/total_num_tokens:149301 - perf/time_per_step:40.74699264019728 - perf/throughput:458.01232902742254 (TaskRunner pid=2255397) Training Progress: 66%|██████▌ | 66/100 [1:00:00<25:17, 44.63s/it] (TaskRunner pid=2255397) step:67 - global_seqlen/min:14936 - global_seqlen/max:21018 - global_seqlen/minmax_diff:6082 - global_seqlen/balanced_min:18548 - global_seqlen/balanced_max:18548 - global_seqlen/mean:18548.0 - actor/entropy:0.4010680615901947 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18490827083587646 - training/rollout_probs_diff_mean:0.005276753567159176 - training/rollout_probs_diff_std:0.011970635503530502 - training/rollout_actor_probs_pearson_corr:0.9990053772926331 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0001569986343384 - rollout_corr/rollout_is_ratio_fraction_high:1.357367818854982e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.357367818854982e-05 - rollout_corr/rollout_is_max:2.239859104156494 - rollout_corr/rollout_is_min:0.4357939660549164 - rollout_corr/rollout_is_std:0.03935437649488449 - rollout_corr/rollout_is_eff_sample_size:0.9984541032441931 - rollout_corr/rollout_is_seq_mean:1.0001627206802368 - rollout_corr/rollout_is_seq_std:0.002580940490588546 - rollout_corr/rollout_is_seq_max:1.0100871324539185 - rollout_corr/rollout_is_seq_min:0.9937511682510376 - rollout_corr/rollout_is_seq_max_deviation:0.010087132453918457 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5274818786419928) - rollout_corr/training_log_ppl:np.float64(0.41380137618398294) - rollout_corr/kl:np.float64(0.0010146739747582956) - rollout_corr/k3_kl:np.float64(0.0009835226737777703) - rollout_corr/rollout_ppl:np.float64(1.5259358463808894) - rollout_corr/rollout_log_ppl:np.float64(0.41278669840539806) - rollout_corr/log_ppl_diff:np.float64(0.0010146777785848826) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021784571872558445) - rollout_corr/log_ppl_diff_max:np.float64(0.014455139636993408) - rollout_corr/log_ppl_diff_min:np.float64(-0.0060721635818481445) - rollout_corr/ppl_ratio:np.float64(1.0010192152112722) - rollout_corr/chi2_token:np.float64(0.0019758078269660473) - rollout_corr/chi2_seq:np.float64(0.4387206360697746) - actor/pg_loss:np.float64(0.0029250025982037187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039720970880807727) - actor/ppo_kl:np.float64(0.0003566578742226767) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(547.347474707989) - self_distillation/token_reweight_w_std:np.float64(8326.106724163285) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982913499698043) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06155556399608031) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2857639491558075) - perf/mfu/actor:np.float64(0.04548089224569089) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.682861328125) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016373249236494303 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016373249236494303 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:287.78125 - response_length/max:673.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.78125 - response_length_non_aborted/max:673.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.84375 - prompt_length/max:576.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013326294720172882 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3205897863954306) - timing_s/agent_loop/generate_sequences/max:np.float64(5.870759850367904) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2840450326475548) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.870759850367904) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:228 - timing_s/agent_loop/slowest/response_length:673 - timing_s/gen:12.662429416552186 - timing_s/reward:0.05886896140873432 - timing_s/old_log_prob:2.852442506700754 - timing_s/adv:0.49736564978957176 - timing_s/update_actor:21.174225766211748 - timing_s/step:37.33046619966626 - timing_s/stop_profile:2.7060508728027344e-05 - timing_per_token_ms/update_actor:0.14269884735693705 - timing_per_token_ms/gen:0.1718757386327531 - timing_per_token_ms/adv:0.00335188194003108 - perf/total_num_tokens:148384 - perf/time_per_step:37.33046619966626 - perf/throughput:496.8595865048645 (TaskRunner pid=2255397) Training Progress: 67%|██████▋ | 67/100 [1:00:37<23:20, 42.45s/it] (TaskRunner pid=2255397) step:68 - global_seqlen/min:15394 - global_seqlen/max:21848 - global_seqlen/minmax_diff:6454 - global_seqlen/balanced_min:19111 - global_seqlen/balanced_max:19113 - global_seqlen/mean:19112.0 - actor/entropy:0.454579621553421 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294895052909851 - training/rollout_probs_diff_mean:0.005558677017688751 - training/rollout_probs_diff_std:0.011987541802227497 - training/rollout_actor_probs_pearson_corr:0.999093234539032 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.46875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:0.999990701675415 - rollout_corr/rollout_is_ratio_fraction_high:2.770083119685296e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.329737901687622 - rollout_corr/rollout_is_min:0.5801910758018494 - rollout_corr/rollout_is_std:0.04091522842645645 - rollout_corr/rollout_is_eff_sample_size:0.9983287417268669 - rollout_corr/rollout_is_seq_mean:0.9999210834503174 - rollout_corr/rollout_is_seq_std:0.0029238073620945215 - rollout_corr/rollout_is_seq_max:1.010511040687561 - rollout_corr/rollout_is_seq_min:0.9896543025970459 - rollout_corr/rollout_is_seq_max_deviation:0.010511040687561035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6329940822906792) - rollout_corr/training_log_ppl:np.float64(0.4769620407023467) - rollout_corr/kl:np.float64(0.0010583482470849503) - rollout_corr/k3_kl:np.float64(0.0009939358607766735) - rollout_corr/rollout_ppl:np.float64(1.631263299845159) - rollout_corr/rollout_log_ppl:np.float64(0.47590369096724316) - rollout_corr/log_ppl_diff:np.float64(0.0010583497351035476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024261822691187263) - rollout_corr/log_ppl_diff_max:np.float64(0.012225866317749023) - rollout_corr/log_ppl_diff_min:np.float64(-0.008922696113586426) - rollout_corr/ppl_ratio:np.float64(1.0010634786449373) - rollout_corr/chi2_token:np.float64(0.0018958586733788252) - rollout_corr/chi2_seq:np.float64(0.7360206483863294) - actor/pg_loss:np.float64(0.0039513889933004975) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004996858078811783) - actor/ppo_kl:np.float64(9.023317768352968e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46875) - self_distillation/token_reweight_w_mean:np.float64(16211.263574679615) - self_distillation/token_reweight_w_std:np.float64(247760.0435439156) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013059759512544) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09038829631754197) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.35276055335998535) - perf/mfu/actor:np.float64(0.04664085001944063) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.4664192199707) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022922437638044357 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022922437638044357 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:282.03125 - response_length/max:777.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:282.03125 - response_length_non_aborted/max:777.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.21875 - prompt_length/max:704.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.814541459083557e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1940552610903978) - timing_s/agent_loop/generate_sequences/max:np.float64(6.472350174561143) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2505520836712094) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.472350174561143) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:777 - timing_s/gen:13.590019196271896 - timing_s/reward:0.05382108315825462 - timing_s/old_log_prob:3.070949576795101 - timing_s/adv:0.5018815603107214 - timing_s/update_actor:21.207215555012226 - timing_s/step:38.45727911219001 - timing_s/stop_profile:2.8835609555244446e-05 - timing_per_token_ms/update_actor:0.13870353413439349 - timing_per_token_ms/gen:0.1882274126907465 - timing_per_token_ms/adv:0.003282502879805367 - perf/total_num_tokens:152896 - perf/time_per_step:38.45727911219001 - perf/throughput:496.9670356617083 (TaskRunner pid=2255397) Training Progress: 68%|██████▊ | 68/100 [1:01:16<22:00, 41.26s/it] (TaskRunner pid=2255397) step:69 - global_seqlen/min:17581 - global_seqlen/max:21958 - global_seqlen/minmax_diff:4377 - global_seqlen/balanced_min:20257 - global_seqlen/balanced_max:20260 - global_seqlen/mean:20258.25 - actor/entropy:0.46467649936676025 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135446548461914 - training/rollout_probs_diff_mean:0.00523096090182662 - training/rollout_probs_diff_std:0.011185083538293839 - training/rollout_actor_probs_pearson_corr:0.999234676361084 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0000218152999878 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6324684619903564 - rollout_corr/rollout_is_min:0.5706769227981567 - rollout_corr/rollout_is_std:0.03825926035642624 - rollout_corr/rollout_is_eff_sample_size:0.998538606349212 - rollout_corr/rollout_is_seq_mean:1.0000791549682617 - rollout_corr/rollout_is_seq_std:0.002675000810995698 - rollout_corr/rollout_is_seq_max:1.0090395212173462 - rollout_corr/rollout_is_seq_min:0.989310622215271 - rollout_corr/rollout_is_seq_max_deviation:0.010689377784729004 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.646835464052856) - rollout_corr/training_log_ppl:np.float64(0.4856712990440428) - rollout_corr/kl:np.float64(0.0008948077715515979) - rollout_corr/k3_kl:np.float64(0.0010960273730233894) - rollout_corr/rollout_ppl:np.float64(1.6454020515084267) - rollout_corr/rollout_log_ppl:np.float64(0.48477648955304176) - rollout_corr/log_ppl_diff:np.float64(0.0008948094910010695) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023694130359217525) - rollout_corr/log_ppl_diff_max:np.float64(0.012593626976013184) - rollout_corr/log_ppl_diff_min:np.float64(-0.010994911193847656) - rollout_corr/ppl_ratio:np.float64(1.0008999034762383) - rollout_corr/chi2_token:np.float64(0.0027323926333338022) - rollout_corr/chi2_seq:np.float64(1.6403908263891935) - actor/pg_loss:np.float64(0.004149097250774503) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006782121126889251) - actor/ppo_kl:np.float64(0.0001744897412141455) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(112.03770920354873) - self_distillation/token_reweight_w_std:np.float64(1537.8677468675305) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990846202708781) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10416805295972154) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.3852573037147522) - perf/mfu/actor:np.float64(0.04884632323523585) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.50916624069214) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001025630161166191 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001025630161166191 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:323.2578125 - response_length/max:740.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:323.2578125 - response_length_non_aborted/max:740.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.8125 - prompt_length/max:597.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5704503059387207e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7049627471715212) - timing_s/agent_loop/generate_sequences/max:np.float64(6.463043302297592) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.702042857927154) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.463043302297592) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:225 - timing_s/agent_loop/slowest/response_length:740 - timing_s/gen:13.723623352125287 - timing_s/reward:0.06722629256546497 - timing_s/old_log_prob:3.223280254751444 - timing_s/adv:0.5192670281976461 - timing_s/update_actor:21.539511624723673 - timing_s/step:39.17174446769059 - timing_s/stop_profile:0.00010973773896694183 - timing_per_token_ms/update_actor:0.13290580149274786 - timing_per_token_ms/gen:0.16583637470243476 - timing_per_token_ms/adv:0.003204046673562907 - perf/total_num_tokens:162066 - perf/time_per_step:39.17174446769059 - perf/throughput:517.1648665458158 (TaskRunner pid=2255397) Training Progress: 69%|██████▉ | 69/100 [1:01:55<21:00, 40.65s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we must analyze the structure and functional groups present in both the product and the options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product contains: (TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or similar heterocycle). (TaskRunner pid=2255397) - A chlorinated aromatic ring (–Cl group). (TaskRunner pid=2255397) - A trifluoromethyl group (–C(F)(F)F). (TaskRunner pid=2255397) - A nitrogen atom bonded to a carbon chain that includes the chlorinated aromatic ring. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let's examine the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A**: Contains a chlorinated aromatic ring, a trifluoromethyl group, and a nitrogen-containing ring with a nitrogen atom bonded to a carbon chain that includes the chlorinated aromatic ring. It closely matches the structure of the product and is a strong candidate. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B**: Contains a chlorinated aromatic ring but lacks the trifluoromethyl group and the correct nitrogen-containing ring structure. It is not a match. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C**: Contains a chlorinated aromatic ring and a trifluoromethyl group, but the nitrogen-containing ring is not correctly structured. It does not match the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D**: Contains a trifluoromethyl group and a carbon chain with a chlorinated aromatic ring, but the nitrogen-containing ring is not correctly structured. It also does not match the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Thus, **Option A** is the only one that contains all the necessary functional groups and structural features required to synthesize the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2255397) step:70 - global_seqlen/min:14508 - global_seqlen/max:21779 - global_seqlen/minmax_diff:7271 - global_seqlen/balanced_min:19295 - global_seqlen/balanced_max:19376 - global_seqlen/mean:19309.25 - actor/entropy:0.48265349864959717 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931375205516815 - training/rollout_probs_diff_mean:0.005450577940791845 - training/rollout_probs_diff_std:0.011540403589606285 - training/rollout_actor_probs_pearson_corr:0.9992024302482605 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:1.0000245571136475 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6136518716812134 - rollout_corr/rollout_is_min:0.5989248752593994 - rollout_corr/rollout_is_std:0.03922086954116821 - rollout_corr/rollout_is_eff_sample_size:0.9984642048317262 - rollout_corr/rollout_is_seq_mean:1.0000163316726685 - rollout_corr/rollout_is_seq_std:0.002710202941671014 - rollout_corr/rollout_is_seq_max:1.0095508098602295 - rollout_corr/rollout_is_seq_min:0.9921322464942932 - rollout_corr/rollout_is_seq_max_deviation:0.009550809860229492 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6673680869862437) - rollout_corr/training_log_ppl:np.float64(0.4960062170866877) - rollout_corr/kl:np.float64(0.0012155248211556113) - rollout_corr/k3_kl:np.float64(0.0012827583032049006) - rollout_corr/rollout_ppl:np.float64(1.6654691863805056) - rollout_corr/rollout_log_ppl:np.float64(0.4947906881570816) - rollout_corr/log_ppl_diff:np.float64(0.00121552892960608) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026022420497611165) - rollout_corr/log_ppl_diff_max:np.float64(0.016035795211791992) - rollout_corr/log_ppl_diff_min:np.float64(-0.00719606876373291) - rollout_corr/ppl_ratio:np.float64(1.0012216775212437) - rollout_corr/chi2_token:np.float64(0.002924366621300578) - rollout_corr/chi2_seq:np.float64(1.3708638157695532) - actor/pg_loss:np.float64(0.0026991143822669983) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006329814200398687) - actor/ppo_kl:np.float64(0.0004067609955740181) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(1099.6124461798463) - self_distillation/token_reweight_w_std:np.float64(17668.276791150623) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005380015354604) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08962540709762834) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.30823376774787903) - perf/mfu/actor:np.float64(0.04660907978159935) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.58329391479492) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6967261904761904) - val-aux/sciknoweval/reward/std@16:np.float64(0.15836902316671747) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7580857142857141) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12446919093953822) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6347571428571429) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1415950973799327) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6969619047619046) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1573615579638144) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8046666666666668) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09049082267851498) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5808857142857142) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11888321084630435) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7069904761904763) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1207083032051214) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8401380952380952) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0621350178422806) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.533304761904762) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09702903481875863) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7099809523809524) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08266567659967641) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8649238095238094) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.037984835978200744) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4909904761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07165352375405533) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7088047619047618) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.057017736779803374) - val-aux/sciknoweval/score/mean@16:np.float64(0.6967261904761904) - val-aux/sciknoweval/score/std@16:np.float64(0.15836902316671747) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7580857142857141) - val-aux/sciknoweval/score/best@2/std:np.float64(0.12446919093953822) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6347571428571429) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1415950973799327) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6969619047619046) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1573615579638144) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8046666666666668) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09049082267851498) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5808857142857142) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11888321084630435) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7069904761904763) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1207083032051214) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8401380952380952) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0621350178422806) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.533304761904762) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09702903481875863) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7099809523809524) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08266567659967641) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8649238095238094) - val-aux/sciknoweval/score/best@16/std:np.float64(0.037984835978200744) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4909904761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07165352375405533) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7088047619047618) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.057017736779803374) - val-core/sciknoweval/acc/mean@16:np.float64(0.6967261904761904) - val-aux/sciknoweval/acc/std@16:np.float64(0.15836902316671747) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7580857142857141) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.12446919093953822) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6347571428571429) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1415950973799327) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6969619047619046) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1573615579638144) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8046666666666668) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09049082267851498) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5808857142857142) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11888321084630435) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7069904761904763) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1207083032051214) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8401380952380952) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0621350178422806) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.533304761904762) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09702903481875863) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7099809523809524) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08266567659967641) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8649238095238094) - val-core/sciknoweval/acc/best@16/std:np.float64(0.037984835978200744) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4909904761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07165352375405533) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7088047619047618) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.057017736779803374) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999809523809524) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.000300566358820181) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9993666666666667) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0016170225430592812) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9996809523809523) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0011905809477718342) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9988428571428571) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.002042357581276089) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9998476190476191) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0008380952380952382) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980571428571428) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023403005795668832) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999428571428571) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0005185012414787124) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9969666666666667) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.002289832599412746) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011535261757671833 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011535261757671833 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:305.1953125 - response_length/max:1167.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.1953125 - response_length_non_aborted/max:1167.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.21875 - prompt_length/max:701.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000107618048787117 - timing_s/agent_loop/generate_sequences/min:np.float64(1.561738733202219) - timing_s/agent_loop/generate_sequences/max:np.float64(8.901570096611977) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.487381806007761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.901570096611977) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:191 - timing_s/agent_loop/slowest/response_length:1167 - timing_s/gen:15.990489525720477 - timing_s/reward:0.05796127952635288 - timing_s/old_log_prob:2.9395081773400307 - timing_s/adv:0.5521378461271524 - timing_s/update_actor:21.566690308973193 - timing_s/step:41.193913504481316 - timing_s/testing:129.7496616318822 - timing_s/save_checkpoint:13.531663544476032 - timing_s/stop_profile:0.00015689432621002197 - timing_per_token_ms/update_actor:0.13961372340311765 - timing_per_token_ms/gen:0.20466516735851117 - timing_per_token_ms/adv:0.0035743092438025328 - perf/total_num_tokens:154474 - perf/time_per_step:41.193913504481316 - perf/throughput:468.7403637408577 (TaskRunner pid=2255397) Training Progress: 70%|███████ | 70/100 [1:04:59<41:54, 83.82s/it] (TaskRunner pid=2255397) step:71 - global_seqlen/min:19243 - global_seqlen/max:22825 - global_seqlen/minmax_diff:3582 - global_seqlen/balanced_min:21239 - global_seqlen/balanced_max:21249 - global_seqlen/mean:21244.625 - actor/entropy:0.48243123292922974 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1782420575618744 - training/rollout_probs_diff_mean:0.004945038352161646 - training/rollout_probs_diff_std:0.010490105487406254 - training/rollout_actor_probs_pearson_corr:0.9993298649787903 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.0001386404037476 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.646081805229187 - rollout_corr/rollout_is_min:0.6092609167098999 - rollout_corr/rollout_is_std:0.036472681909799576 - rollout_corr/rollout_is_eff_sample_size:0.998671986340567 - rollout_corr/rollout_is_seq_mean:1.00009286403656 - rollout_corr/rollout_is_seq_std:0.0023478141520172358 - rollout_corr/rollout_is_seq_max:1.0064475536346436 - rollout_corr/rollout_is_seq_min:0.9918783903121948 - rollout_corr/rollout_is_seq_max_deviation:0.008121609687805176 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6814591218717396) - rollout_corr/training_log_ppl:np.float64(0.5044128161389381) - rollout_corr/kl:np.float64(0.0009273746180333831) - rollout_corr/k3_kl:np.float64(0.0009558818163668548) - rollout_corr/rollout_ppl:np.float64(1.679951566271484) - rollout_corr/rollout_log_ppl:np.float64(0.5034854415280279) - rollout_corr/log_ppl_diff:np.float64(0.0009273746109101921) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021467784827109426) - rollout_corr/log_ppl_diff_max:np.float64(0.009796887636184692) - rollout_corr/log_ppl_diff_min:np.float64(-0.007009267807006836) - rollout_corr/ppl_ratio:np.float64(1.0009312871843576) - rollout_corr/chi2_token:np.float64(0.002058735117316246) - rollout_corr/chi2_seq:np.float64(0.9684193118009716) - actor/pg_loss:np.float64(0.002505662152543664) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036506839978756034) - actor/ppo_kl:np.float64(0.0002983230811750559) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(1927.8832940307911) - self_distillation/token_reweight_w_std:np.float64(37339.10144847393) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009577083401382) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08062945469282568) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.5023428723216057) - perf/mfu/actor:np.float64(0.051698067264255104) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(201.09897232055664) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013479345943778753 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013479345943778753 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.51953125 - response_length/max:967.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.51953125 - response_length_non_aborted/max:967.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.375 - prompt_length/max:666.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.134435534477234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8741174843162298) - timing_s/agent_loop/generate_sequences/max:np.float64(8.054633025079966) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.1346120286834775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.054633025079966) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:967 - timing_s/gen:14.367894859984517 - timing_s/reward:0.06300288252532482 - timing_s/old_log_prob:3.210338668897748 - timing_s/adv:0.5408081188797951 - timing_s/update_actor:21.329918820410967 - timing_s/step:39.616243589669466 - timing_s/stop_profile:8.219294250011444e-05 - timing_per_token_ms/update_actor:0.12550185529522742 - timing_per_token_ms/gen:0.15524635447151797 - timing_per_token_ms/adv:0.0031820290948875013 - perf/total_num_tokens:169957 - perf/time_per_step:39.616243589669466 - perf/throughput:536.260459725663 (TaskRunner pid=2255397) Training Progress: 71%|███████ | 71/100 [1:05:39<34:06, 70.57s/it] (TaskRunner pid=2255397) step:72 - global_seqlen/min:16380 - global_seqlen/max:21611 - global_seqlen/minmax_diff:5231 - global_seqlen/balanced_min:19858 - global_seqlen/balanced_max:19861 - global_seqlen/mean:19860.0 - actor/entropy:0.5429559946060181 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15169164538383484 - training/rollout_probs_diff_mean:0.005359958857297897 - training/rollout_probs_diff_std:0.010805261321365833 - training/rollout_actor_probs_pearson_corr:0.9993027448654175 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000475645065308 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.644222617149353 - rollout_corr/rollout_is_min:0.6008924245834351 - rollout_corr/rollout_is_std:0.03811253234744072 - rollout_corr/rollout_is_eff_sample_size:0.9985496605558143 - rollout_corr/rollout_is_seq_mean:0.9999592304229736 - rollout_corr/rollout_is_seq_std:0.0024577102158218622 - rollout_corr/rollout_is_seq_max:1.0064175128936768 - rollout_corr/rollout_is_seq_min:0.9917982220649719 - rollout_corr/rollout_is_seq_max_deviation:0.008201777935028076 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7471187515184283) - rollout_corr/training_log_ppl:np.float64(0.5459236325696111) - rollout_corr/kl:np.float64(0.0010623894131924772) - rollout_corr/k3_kl:np.float64(0.0008694290827975237) - rollout_corr/rollout_ppl:np.float64(1.745131317526102) - rollout_corr/rollout_log_ppl:np.float64(0.5448612435720861) - rollout_corr/log_ppl_diff:np.float64(0.0010623889975249767) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021515846019610763) - rollout_corr/log_ppl_diff_max:np.float64(0.010035932064056396) - rollout_corr/log_ppl_diff_min:np.float64(-0.00523415207862854) - rollout_corr/ppl_ratio:np.float64(1.0010662653949112) - rollout_corr/chi2_token:np.float64(0.001382074085995555) - rollout_corr/chi2_seq:np.float64(0.6654008245095611) - actor/pg_loss:np.float64(0.005195157486014068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003912408333235362) - actor/ppo_kl:np.float64(0.0002666896097824889) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(7515.255449632183) - self_distillation/token_reweight_w_std:np.float64(120734.30576296372) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992036235053092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1153781593311578) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.3598603457212448) - perf/mfu/actor:np.float64(0.048463804802359345) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.67909240722656) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006963299238123 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006963299238123 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:333.78125 - response_length/max:799.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:333.78125 - response_length_non_aborted/max:799.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.84375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005877390503883362 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6762614008039236) - timing_s/agent_loop/generate_sequences/max:np.float64(6.705963581800461) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7834387093244004) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.705963581800461) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:767 - timing_s/gen:14.802263040095568 - timing_s/reward:0.06146353296935558 - timing_s/old_log_prob:3.034574717283249 - timing_s/adv:0.524946503341198 - timing_s/update_actor:21.136499028652906 - timing_s/step:39.64624011516571 - timing_s/stop_profile:2.850964665412903e-05 - timing_per_token_ms/update_actor:0.13303435944519704 - timing_per_token_ms/gen:0.17323124052166894 - timing_per_token_ms/adv:0.003304043953557389 - perf/total_num_tokens:158880 - perf/time_per_step:39.64624011516571 - perf/throughput:500.9302254718232 (TaskRunner pid=2255397) Training Progress: 72%|███████▏ | 72/100 [1:06:19<28:36, 61.30s/it] (TaskRunner pid=2255397) step:73 - global_seqlen/min:19647 - global_seqlen/max:22925 - global_seqlen/minmax_diff:3278 - global_seqlen/balanced_min:21192 - global_seqlen/balanced_max:21202 - global_seqlen/mean:21199.0 - actor/entropy:0.5374614596366882 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.301607221364975 - training/rollout_probs_diff_mean:0.004932303912937641 - training/rollout_probs_diff_std:0.010089674964547157 - training/rollout_actor_probs_pearson_corr:0.9993661642074585 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999767541885376 - rollout_corr/rollout_is_ratio_fraction_high:1.021909702103585e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1998536586761475 - rollout_corr/rollout_is_min:0.6301840543746948 - rollout_corr/rollout_is_std:0.03611796349287033 - rollout_corr/rollout_is_eff_sample_size:0.9986969544688866 - rollout_corr/rollout_is_seq_mean:1.0000014305114746 - rollout_corr/rollout_is_seq_std:0.0019163991091772914 - rollout_corr/rollout_is_seq_max:1.0062246322631836 - rollout_corr/rollout_is_seq_min:0.9927696585655212 - rollout_corr/rollout_is_seq_max_deviation:0.00723034143447876 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7496894244104624) - rollout_corr/training_log_ppl:np.float64(0.5468992951209657) - rollout_corr/kl:np.float64(0.0007367991016540287) - rollout_corr/k3_kl:np.float64(0.0007361613289731395) - rollout_corr/rollout_ppl:np.float64(1.7483539679087698) - rollout_corr/rollout_log_ppl:np.float64(0.5461624904419295) - rollout_corr/log_ppl_diff:np.float64(0.0007368046790361404) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016733732772991061) - rollout_corr/log_ppl_diff_max:np.float64(0.009091198444366455) - rollout_corr/log_ppl_diff_min:np.float64(-0.005505651235580444) - rollout_corr/ppl_ratio:np.float64(1.0007393239066005) - rollout_corr/chi2_token:np.float64(0.0014872262254357338) - rollout_corr/chi2_seq:np.float64(0.43628761847503483) - actor/pg_loss:np.float64(0.0029139972757548094) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018539705752118607) - actor/ppo_kl:np.float64(6.380892117796577e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(1624.953690484399) - self_distillation/token_reweight_w_std:np.float64(32560.52424334915) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991668974980712) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06964316591620445) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.21447288990020752) - perf/mfu/actor:np.float64(0.050516435918862526) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.0999493598938) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007218515034765005 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007218515034765005 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.25 - response_length/max:996.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.25 - response_length_non_aborted/max:996.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.21875 - prompt_length/max:557.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8507784008979797e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6397324707359076) - timing_s/agent_loop/generate_sequences/max:np.float64(8.402389820665121) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.477701659416198) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.402389820665121) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:216 - timing_s/agent_loop/slowest/response_length:996 - timing_s/gen:15.686312183737755 - timing_s/reward:0.06267971731722355 - timing_s/old_log_prob:2.8734097313135862 - timing_s/adv:0.5590537320822477 - timing_s/update_actor:21.74830207787454 - timing_s/step:41.018087424337864 - timing_s/stop_profile:0.00012700818479061127 - timing_per_token_ms/update_actor:0.12823896220266606 - timing_per_token_ms/gen:0.16029995282596626 - timing_per_token_ms/adv:0.003296462876092314 - perf/total_num_tokens:169592 - perf/time_per_step:41.018087424337864 - perf/throughput:516.8207815418933 (TaskRunner pid=2255397) Training Progress: 73%|███████▎ | 73/100 [1:07:00<24:51, 55.23s/it] (TaskRunner pid=2255397) step:74 - global_seqlen/min:18287 - global_seqlen/max:25091 - global_seqlen/minmax_diff:6804 - global_seqlen/balanced_min:20904 - global_seqlen/balanced_max:20912 - global_seqlen/mean:20910.0 - actor/entropy:0.5127137303352356 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27330654859542847 - training/rollout_probs_diff_mean:0.004899736028164625 - training/rollout_probs_diff_std:0.010343815200030804 - training/rollout_actor_probs_pearson_corr:0.9993671774864197 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000099539756775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0278759873472154e-05 - rollout_corr/rollout_is_max:1.7854251861572266 - rollout_corr/rollout_is_min:0.48449382185935974 - rollout_corr/rollout_is_std:0.0365249402821064 - rollout_corr/rollout_is_eff_sample_size:0.9986679439991228 - rollout_corr/rollout_is_seq_mean:1.000190258026123 - rollout_corr/rollout_is_seq_std:0.0022744936868548393 - rollout_corr/rollout_is_seq_max:1.0073591470718384 - rollout_corr/rollout_is_seq_min:0.9917874336242676 - rollout_corr/rollout_is_seq_max_deviation:0.008212566375732422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7097132932394743) - rollout_corr/training_log_ppl:np.float64(0.5229829574818723) - rollout_corr/kl:np.float64(0.0006870645115757412) - rollout_corr/k3_kl:np.float64(0.0007633993515128168) - rollout_corr/rollout_ppl:np.float64(1.7085823002271354) - rollout_corr/rollout_log_ppl:np.float64(0.5222958918311633) - rollout_corr/log_ppl_diff:np.float64(0.0006870656507089734) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019122406374663115) - rollout_corr/log_ppl_diff_max:np.float64(0.008660197257995605) - rollout_corr/log_ppl_diff_min:np.float64(-0.005268260836601257) - rollout_corr/ppl_ratio:np.float64(1.000690019223839) - rollout_corr/chi2_token:np.float64(0.0016905386000871658) - rollout_corr/chi2_seq:np.float64(1.4013629674445838) - actor/pg_loss:np.float64(0.0029980618273839355) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017913388501256122) - actor/ppo_kl:np.float64(0.00016305825718454514) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(15.748328928370029) - self_distillation/token_reweight_w_std:np.float64(289.3728471599752) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004254090599716) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06948782474501058) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2460213266313076) - perf/mfu/actor:np.float64(0.0504502030203242) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.804443359375) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005830626469105482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005830626469105482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:380.03125 - response_length/max:953.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:380.03125 - response_length_non_aborted/max:953.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.40625 - prompt_length/max:665.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.984895586967468e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.763165319338441) - timing_s/agent_loop/generate_sequences/max:np.float64(8.103548426181078) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.310445136674389) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.103548426181078) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:953 - timing_s/gen:14.785902259871364 - timing_s/reward:0.07314063049852848 - timing_s/old_log_prob:2.9278498981148005 - timing_s/adv:0.5283288117498159 - timing_s/update_actor:21.43048808723688 - timing_s/step:39.835200319066644 - timing_s/stop_profile:0.00012897513806819916 - timing_per_token_ms/update_actor:0.1281114782833386 - timing_per_token_ms/gen:0.15198074027497085 - timing_per_token_ms/adv:0.0031583501419764222 - perf/total_num_tokens:167280 - perf/time_per_step:39.835200319066644 - perf/throughput:524.9126358727428 (TaskRunner pid=2255397) Training Progress: 74%|███████▍ | 74/100 [1:07:40<21:56, 50.63s/it] (TaskRunner pid=2255397) step:75 - global_seqlen/min:16481 - global_seqlen/max:26987 - global_seqlen/minmax_diff:10506 - global_seqlen/balanced_min:22439 - global_seqlen/balanced_max:22447 - global_seqlen/mean:22444.625 - actor/entropy:0.5642927289009094 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2400461733341217 - training/rollout_probs_diff_mean:0.0051166643388569355 - training/rollout_probs_diff_std:0.010140416212379932 - training/rollout_actor_probs_pearson_corr:0.9994109869003296 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999061822891235 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0068769370263908e-05 - rollout_corr/rollout_is_max:1.5713963508605957 - rollout_corr/rollout_is_min:0.4978005588054657 - rollout_corr/rollout_is_std:0.03701609745621681 - rollout_corr/rollout_is_eff_sample_size:0.9986314455154365 - rollout_corr/rollout_is_seq_mean:0.999901294708252 - rollout_corr/rollout_is_seq_std:0.0019736075773835182 - rollout_corr/rollout_is_seq_max:1.0060852766036987 - rollout_corr/rollout_is_seq_min:0.9931412935256958 - rollout_corr/rollout_is_seq_max_deviation:0.006858706474304199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.791621611919254) - rollout_corr/training_log_ppl:np.float64(0.570267406059429) - rollout_corr/kl:np.float64(0.0008821988329259511) - rollout_corr/k3_kl:np.float64(0.0008163573830870519) - rollout_corr/rollout_ppl:np.float64(1.7901061396114528) - rollout_corr/rollout_log_ppl:np.float64(0.56938520638505) - rollout_corr/log_ppl_diff:np.float64(0.0008821996743790805) - rollout_corr/log_ppl_abs_diff:np.float64(0.00185499427607283) - rollout_corr/log_ppl_diff_max:np.float64(0.00786745548248291) - rollout_corr/log_ppl_diff_min:np.float64(-0.005721062421798706) - rollout_corr/ppl_ratio:np.float64(1.000884982291609) - rollout_corr/chi2_token:np.float64(0.001510836649686098) - rollout_corr/chi2_seq:np.float64(0.9038489798549563) - actor/pg_loss:np.float64(0.002076247939839959) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023045666375764995) - actor/ppo_kl:np.float64(5.9166708297198056e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(6473.88584861136) - self_distillation/token_reweight_w_std:np.float64(110608.01085398125) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989379325415939) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0619521141925361) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.19757572747766972) - perf/mfu/actor:np.float64(0.053976922098505964) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.7879409790039) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026480865199118853 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026480865199118853 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:387.95703125 - response_length/max:918.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:387.95703125 - response_length_non_aborted/max:918.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.4375 - prompt_length/max:608.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013186223804950714 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1773367393761873) - timing_s/agent_loop/generate_sequences/max:np.float64(8.078486654907465) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.527875997897354) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.078486654907465) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:918 - timing_s/gen:14.925510486587882 - timing_s/reward:0.05783644877374172 - timing_s/old_log_prob:2.9777278639376163 - timing_s/adv:0.5886598229408264 - timing_s/update_actor:21.493691984564066 - timing_s/step:40.15656405314803 - timing_s/stop_profile:7.43325799703598e-05 - timing_per_token_ms/update_actor:0.11970400477042982 - timing_per_token_ms/gen:0.1502815277000703 - timing_per_token_ms/adv:0.003278400858450667 - perf/total_num_tokens:179557 - perf/time_per_step:40.15656405314803 - perf/throughput:558.9279244681911 (TaskRunner pid=2255397) Training Progress: 75%|███████▌ | 75/100 [1:08:20<19:47, 47.50s/it] (TaskRunner pid=2255397) step:76 - global_seqlen/min:20385 - global_seqlen/max:23735 - global_seqlen/minmax_diff:3350 - global_seqlen/balanced_min:22458 - global_seqlen/balanced_max:22461 - global_seqlen/mean:22459.875 - actor/entropy:0.5038673281669617 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917871475219727 - training/rollout_probs_diff_mean:0.004544694907963276 - training/rollout_probs_diff_std:0.009781488217413425 - training/rollout_actor_probs_pearson_corr:0.9994357228279114 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.999882698059082 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.531444787979126 - rollout_corr/rollout_is_min:0.6298530101776123 - rollout_corr/rollout_is_std:0.03470920771360397 - rollout_corr/rollout_is_eff_sample_size:0.9987963637779437 - rollout_corr/rollout_is_seq_mean:0.9998737573623657 - rollout_corr/rollout_is_seq_std:0.001968828495591879 - rollout_corr/rollout_is_seq_max:1.0092756748199463 - rollout_corr/rollout_is_seq_min:0.9928333163261414 - rollout_corr/rollout_is_seq_max_deviation:0.009275674819946289 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7229888113215566) - rollout_corr/training_log_ppl:np.float64(0.5273766612517647) - rollout_corr/kl:np.float64(0.0006758117173214506) - rollout_corr/k3_kl:np.float64(0.0006499772979395857) - rollout_corr/rollout_ppl:np.float64(1.72180867055431) - rollout_corr/rollout_log_ppl:np.float64(0.526700847200118) - rollout_corr/log_ppl_diff:np.float64(0.0006758140516467392) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016295489040203393) - rollout_corr/log_ppl_diff_max:np.float64(0.007842868566513062) - rollout_corr/log_ppl_diff_min:np.float64(-0.005845367908477783) - rollout_corr/ppl_ratio:np.float64(1.0006780109833926) - rollout_corr/chi2_token:np.float64(0.0012538495939224958) - rollout_corr/chi2_seq:np.float64(0.6266895108856261) - actor/pg_loss:np.float64(0.0024852192727848887) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022367794190358836) - actor/ppo_kl:np.float64(-8.031681899822729e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(11.586508150445297) - self_distillation/token_reweight_w_std:np.float64(205.42800806532614) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991163620725274) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07528238961822353) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.21515963971614838) - perf/mfu/actor:np.float64(0.05377171741192728) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.86333084106445) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009749584714882076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009749584714882076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:416.18359375 - response_length/max:835.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:416.18359375 - response_length_non_aborted/max:835.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:285.6875 - prompt_length/max:526.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015104003250598907 - timing_s/agent_loop/generate_sequences/min:np.float64(1.69934743642807) - timing_s/agent_loop/generate_sequences/max:np.float64(7.779057312756777) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.804315655819664) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.779057312756777) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:835 - timing_s/gen:14.658727264031768 - timing_s/reward:0.06544599309563637 - timing_s/old_log_prob:3.1654940266162157 - timing_s/adv:0.5449143052101135 - timing_s/update_actor:21.7036225348711 - timing_s/step:40.22667368501425 - timing_s/stop_profile:0.00010753795504570007 - timing_per_token_ms/update_actor:0.1207910915291776 - timing_per_token_ms/gen:0.13758508080335421 - timing_per_token_ms/adv:0.003032710028495893 - perf/total_num_tokens:179679 - perf/time_per_step:40.22667368501425 - perf/throughput:558.3328906552629 (TaskRunner pid=2255397) Training Progress: 76%|███████▌ | 76/100 [1:09:00<18:08, 45.33s/it] (TaskRunner pid=2255397) step:77 - global_seqlen/min:19528 - global_seqlen/max:26165 - global_seqlen/minmax_diff:6637 - global_seqlen/balanced_min:22384 - global_seqlen/balanced_max:22391 - global_seqlen/mean:22389.375 - actor/entropy:0.5504021048545837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2723144292831421 - training/rollout_probs_diff_mean:0.005151774734258652 - training/rollout_probs_diff_std:0.01037499401718378 - training/rollout_actor_probs_pearson_corr:0.9993544816970825 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:0.9999526739120483 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6774260997772217 - rollout_corr/rollout_is_min:0.6362783908843994 - rollout_corr/rollout_is_std:0.037128645926713943 - rollout_corr/rollout_is_eff_sample_size:0.9986231237591031 - rollout_corr/rollout_is_seq_mean:0.9999860525131226 - rollout_corr/rollout_is_seq_std:0.002109500812366605 - rollout_corr/rollout_is_seq_max:1.0075759887695312 - rollout_corr/rollout_is_seq_min:0.993508517742157 - rollout_corr/rollout_is_seq_max_deviation:0.00757598876953125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7856992213055491) - rollout_corr/training_log_ppl:np.float64(0.5679072652128525) - rollout_corr/kl:np.float64(0.0009176925997458341) - rollout_corr/k3_kl:np.float64(0.0007757482836723284) - rollout_corr/rollout_ppl:np.float64(1.784037644509226) - rollout_corr/rollout_log_ppl:np.float64(0.5669895681203343) - rollout_corr/log_ppl_diff:np.float64(0.0009176970925182104) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019323080778121948) - rollout_corr/log_ppl_diff_max:np.float64(0.007754087448120117) - rollout_corr/log_ppl_diff_min:np.float64(-0.00577545166015625) - rollout_corr/ppl_ratio:np.float64(1.00092074717395) - rollout_corr/chi2_token:np.float64(0.00127337034791708) - rollout_corr/chi2_seq:np.float64(0.6404874103609473) - actor/pg_loss:np.float64(0.0034807646879926324) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002253228935842344) - actor/ppo_kl:np.float64(0.00018203446227005315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(130.1160973869264) - self_distillation/token_reweight_w_std:np.float64(2325.1732160223182) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002648527733982) - self_distillation/token_reweight_w_clip_frac:np.float64(0.091672543407185) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.2861635349690914) - perf/mfu/actor:np.float64(0.053185240362257795) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.9845733642578) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007428093231283128 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007428093231283128 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.57421875 - response_length/max:872.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.57421875 - response_length_non_aborted/max:872.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.09375 - prompt_length/max:716.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.42423939704895e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.215482749044895) - timing_s/agent_loop/generate_sequences/max:np.float64(7.765327328816056) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.492655532936624) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.765327328816056) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:368 - timing_s/agent_loop/slowest/response_length:857 - timing_s/gen:14.60610330849886 - timing_s/reward:0.06456474959850311 - timing_s/old_log_prob:2.9403065145015717 - timing_s/adv:0.5702805798500776 - timing_s/update_actor:21.948025058954954 - timing_s/step:40.21948102675378 - timing_s/stop_profile:0.00011724233627319336 - timing_per_token_ms/update_actor:0.12253594092596909 - timing_per_token_ms/gen:0.14913469923624767 - timing_per_token_ms/adv:0.003183879517907923 - perf/total_num_tokens:179115 - perf/time_per_step:40.21948102675378 - perf/throughput:556.6798583280254 (TaskRunner pid=2255397) Training Progress: 77%|███████▋ | 77/100 [1:09:40<16:47, 43.82s/it] (TaskRunner pid=2255397) step:78 - global_seqlen/min:20134 - global_seqlen/max:31492 - global_seqlen/minmax_diff:11358 - global_seqlen/balanced_min:22482 - global_seqlen/balanced_max:29618 - global_seqlen/mean:23385.375 - actor/entropy:0.48590248823165894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40464895963668823 - training/rollout_probs_diff_mean:0.004600349348038435 - training/rollout_probs_diff_std:0.009926710277795792 - training/rollout_actor_probs_pearson_corr:0.9993968605995178 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9998959302902222 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.656884347146843e-06 - rollout_corr/rollout_is_max:1.8092979192733765 - rollout_corr/rollout_is_min:0.48850229382514954 - rollout_corr/rollout_is_std:0.03525106608867645 - rollout_corr/rollout_is_eff_sample_size:0.9987587856840187 - rollout_corr/rollout_is_seq_mean:0.9998641014099121 - rollout_corr/rollout_is_seq_std:0.0020814789459109306 - rollout_corr/rollout_is_seq_max:1.0097109079360962 - rollout_corr/rollout_is_seq_min:0.9942805171012878 - rollout_corr/rollout_is_seq_max_deviation:0.009710907936096191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7207122039981186) - rollout_corr/training_log_ppl:np.float64(0.5288956113836321) - rollout_corr/kl:np.float64(0.0008265634318824766) - rollout_corr/k3_kl:np.float64(0.0007540007430009155) - rollout_corr/rollout_ppl:np.float64(1.7192450258880854) - rollout_corr/rollout_log_ppl:np.float64(0.5280690497056639) - rollout_corr/log_ppl_diff:np.float64(0.0008265616779681295) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017743823991622776) - rollout_corr/log_ppl_diff_max:np.float64(0.009331107139587402) - rollout_corr/log_ppl_diff_min:np.float64(-0.004447638988494873) - rollout_corr/ppl_ratio:np.float64(1.0008291078265756) - rollout_corr/chi2_token:np.float64(0.0013703966978937387) - rollout_corr/chi2_seq:np.float64(0.8413228269200772) - actor/pg_loss:np.float64(0.004617663682438433) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030698799764650175) - actor/ppo_kl:np.float64(-3.2745020313029727e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(38.89375674771145) - self_distillation/token_reweight_w_std:np.float64(700.807301894878) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001390076475218) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09478579316055402) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.28953084722161293) - perf/mfu/actor:np.float64(0.0530806510376341) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.00171661376953) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.025150412693619728 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.025150412693619728 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:451.23046875 - response_length/max:8192.0 - response_length/min:163.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:451.23046875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:279.5625 - prompt_length/max:449.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.3435564041137695e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.152110392227769) - timing_s/agent_loop/generate_sequences/max:np.float64(59.49123938381672) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.095185146856238) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.49123938381672) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:398 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:66.28803966008127 - timing_s/reward:0.0661248117685318 - timing_s/old_log_prob:3.3414545375853777 - timing_s/adv:0.5907729119062424 - timing_s/update_actor:23.06786256469786 - timing_s/step:93.44836739264429 - timing_s/stop_profile:0.00011118501424789429 - timing_per_token_ms/update_actor:0.12330282582970052 - timing_per_token_ms/gen:0.5738478955986778 - timing_per_token_ms/adv:0.003157811837025504 - perf/total_num_tokens:187083 - perf/time_per_step:93.44836739264429 - perf/throughput:250.24915525534115 (TaskRunner pid=2255397) Training Progress: 78%|███████▊ | 78/100 [1:11:14<21:31, 58.72s/it] (TaskRunner pid=2255397) step:79 - global_seqlen/min:20379 - global_seqlen/max:23902 - global_seqlen/minmax_diff:3523 - global_seqlen/balanced_min:22483 - global_seqlen/balanced_max:22828 - global_seqlen/mean:22531.875 - actor/entropy:0.47975319623947144 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.999380886554718 - training/rollout_probs_diff_mean:0.004774122033268213 - training/rollout_probs_diff_std:0.010841231793165207 - training/rollout_actor_probs_pearson_corr:0.9993086457252502 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999167919158936 - rollout_corr/rollout_is_ratio_fraction_high:8.872878424881492e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.7745756849762984e-05 - rollout_corr/rollout_is_max:2.256157159805298 - rollout_corr/rollout_is_min:1.8565188497632334e-07 - rollout_corr/rollout_is_std:0.035090066492557526 - rollout_corr/rollout_is_eff_sample_size:0.9987699636823175 - rollout_corr/rollout_is_seq_mean:0.9999032020568848 - rollout_corr/rollout_is_seq_std:0.002013471210375428 - rollout_corr/rollout_is_seq_max:1.0075592994689941 - rollout_corr/rollout_is_seq_min:0.9934999942779541 - rollout_corr/rollout_is_seq_max_deviation:0.007559299468994141 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6734817675314844) - rollout_corr/training_log_ppl:np.float64(0.5028578019700944) - rollout_corr/kl:np.float64(0.0007867071553491201) - rollout_corr/k3_kl:np.float64(0.0008091878738127889) - rollout_corr/rollout_ppl:np.float64(1.6721355388872325) - rollout_corr/rollout_log_ppl:np.float64(0.5020710933895316) - rollout_corr/log_ppl_diff:np.float64(0.0007867085805628449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017737957823555917) - rollout_corr/log_ppl_diff_max:np.float64(0.022169530391693115) - rollout_corr/log_ppl_diff_min:np.float64(-0.006566405296325684) - rollout_corr/ppl_ratio:np.float64(1.000790162710473) - rollout_corr/chi2_token:np.float64(0.0015391185879707336) - rollout_corr/chi2_seq:np.float64(1.4496739441528916) - actor/pg_loss:np.float64(0.005306927720084786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039829218349041184) - actor/ppo_kl:np.float64(-4.736486153222863e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(76.99012057809159) - self_distillation/token_reweight_w_std:np.float64(1541.8618622039212) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995152873452753) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11978079983964562) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2994599714875221) - perf/mfu/actor:np.float64(0.052974969811297) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.06073379516602) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009753511287271976 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009753511287271976 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.24609375 - response_length/max:1635.0 - response_length/min:188.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.24609375 - response_length_non_aborted/max:1635.0 - response_length_non_aborted/min:188.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:766.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010463036596775055 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4277297761291265) - timing_s/agent_loop/generate_sequences/max:np.float64(12.42059437930584) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.095716175761481) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.42059437930584) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:234 - timing_s/agent_loop/slowest/response_length:1635 - timing_s/gen:19.97506350465119 - timing_s/reward:0.06424454413354397 - timing_s/old_log_prob:2.966035883873701 - timing_s/adv:0.6315709538757801 - timing_s/update_actor:21.955910548567772 - timing_s/step:45.67770639620721 - timing_s/stop_profile:3.654882311820984e-05 - timing_per_token_ms/update_actor:0.12180472413285497 - timing_per_token_ms/gen:0.17723630697187465 - timing_per_token_ms/adv:0.003503763856069347 - perf/total_num_tokens:180255 - perf/time_per_step:45.67770639620721 - perf/throughput:493.2794743361042 (TaskRunner pid=2255397) Training Progress: 79%|███████▉ | 79/100 [1:12:00<19:11, 54.82s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we must analyze the structure and functional groups of the target molecule and compare it to the structures of the given options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The target molecule contains: (TaskRunner pid=2255397) - A nitrogen-containing ring (likely an azole or similar). (TaskRunner pid=2255397) - A chlorinated aromatic ring (–Cl). (TaskRunner pid=2255397) - A trifluoromethyl group (–C(F)(F)F). (TaskRunner pid=2255397) - A nitrogen-containing side chain that is connected to a carbon chain with a chlorinated aromatic group and a trifluoromethyl group. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, we analyze the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A:** (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" (TaskRunner pid=2255397) - Contains a chlorinated aromatic ring (–Cl). (TaskRunner pid=2255397) - Contains a trifluoromethyl group (–C(F)(F)F). (TaskRunner pid=2255397) - Contains a nitrogen-containing ring with a side chain that connects to the chlorinated aromatic ring and the trifluoromethyl group. (TaskRunner pid=2255397) - Matches the structure of the target molecule closely. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B:** (TaskRunner pid=2255397) "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" (TaskRunner pid=2255397) - Contains a chlorinated ring, but the side chain and functional groups do not match the target molecule. (TaskRunner pid=2255397) - The nitrogen-containing ring is different and does not include the trifluoromethyl group. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C:** (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" (TaskRunner pid=2255397) - Contains a chlorinated aromatic ring and a trifluoromethyl group. (TaskRunner pid=2255397) - However, the functional group arrangement and the nitrogen-containing ring do not match the target molecule's structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D:** (TaskRunner pid=2255397) "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" (TaskRunner pid=2255397) - Contains a trifluoromethyl group and a chlorinated ring. (TaskRunner pid=2255397) - However, the nitrogen-containing ring and the side chain are not structured correctly to match the target molecule. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Conclusion:** (TaskRunner pid=2255397) Only **Option A** contains all the necessary functional groups and structural elements that match the target molecule "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2255397) step:80 - global_seqlen/min:21379 - global_seqlen/max:27736 - global_seqlen/minmax_diff:6357 - global_seqlen/balanced_min:24041 - global_seqlen/balanced_max:24046 - global_seqlen/mean:24044.0 - actor/entropy:0.5321111679077148 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22554826736450195 - training/rollout_probs_diff_mean:0.005127668380737305 - training/rollout_probs_diff_std:0.010481650941073895 - training/rollout_actor_probs_pearson_corr:0.999333381652832 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999201893806458 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.784020185470581 - rollout_corr/rollout_is_min:0.5773587822914124 - rollout_corr/rollout_is_std:0.03691612929105759 - rollout_corr/rollout_is_eff_sample_size:0.9986386974447631 - rollout_corr/rollout_is_seq_mean:0.9999967813491821 - rollout_corr/rollout_is_seq_std:0.002032036194577813 - rollout_corr/rollout_is_seq_max:1.0090333223342896 - rollout_corr/rollout_is_seq_min:0.99375981092453 - rollout_corr/rollout_is_seq_max_deviation:0.00903332233428955 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7447218438610435) - rollout_corr/training_log_ppl:np.float64(0.5451616903301328) - rollout_corr/kl:np.float64(0.0007389033336671069) - rollout_corr/k3_kl:np.float64(0.0007813431365661927) - rollout_corr/rollout_ppl:np.float64(1.743419658858329) - rollout_corr/rollout_log_ppl:np.float64(0.5444227863335982) - rollout_corr/log_ppl_diff:np.float64(0.0007389039965346456) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016596049536019564) - rollout_corr/log_ppl_diff_max:np.float64(0.005907297134399414) - rollout_corr/log_ppl_diff_min:np.float64(-0.0042874813079833984) - rollout_corr/ppl_ratio:np.float64(1.0007411232218146) - rollout_corr/chi2_token:np.float64(0.001654008636251092) - rollout_corr/chi2_seq:np.float64(0.4709114315919578) - actor/pg_loss:np.float64(0.004708616062998772) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004071622186074819) - actor/ppo_kl:np.float64(2.530917598608795e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(13418.10702541261) - self_distillation/token_reweight_w_std:np.float64(249965.62142065133) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001714508980513) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10478754635551013) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.277141023427248) - perf/mfu/actor:np.float64(0.05686537883473178) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.12682485580444) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7098214285714286) - val-aux/sciknoweval/reward/std@16:np.float64(0.14355997006676555) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7664952380952381) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10898051269342317) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6531333333333332) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13319323391175897) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7104857142857143) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14460069804004294) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8059285714285714) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07362869726753447) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6003619047619048) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11450096751541655) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7241619047619048) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10970231087023842) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8335476190476191) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05060468808975912) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5525523809523809) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0905650570350032) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7320380952380953) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07469910045616918) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8541285714285713) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.032361125070489466) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5139714285714285) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0607653237770218) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7368714285714285) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0520517104444629) - val-aux/sciknoweval/score/mean@16:np.float64(0.7098214285714286) - val-aux/sciknoweval/score/std@16:np.float64(0.14355997006676555) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7664952380952381) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10898051269342317) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6531333333333332) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13319323391175897) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7104857142857143) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14460069804004294) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8059285714285714) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07362869726753447) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6003619047619048) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11450096751541655) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7241619047619048) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10970231087023842) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8335476190476191) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05060468808975912) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5525523809523809) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0905650570350032) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7320380952380953) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07469910045616918) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8541285714285713) - val-aux/sciknoweval/score/best@16/std:np.float64(0.032361125070489466) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5139714285714285) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0607653237770218) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7368714285714285) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0520517104444629) - val-core/sciknoweval/acc/mean@16:np.float64(0.7098214285714286) - val-aux/sciknoweval/acc/std@16:np.float64(0.14355997006676555) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7664952380952381) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10898051269342317) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6531333333333332) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13319323391175897) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7104857142857143) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14460069804004294) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8059285714285714) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07362869726753447) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6003619047619048) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11450096751541655) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7241619047619048) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10970231087023842) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8335476190476191) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05060468808975912) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5525523809523809) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0905650570350032) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7320380952380953) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07469910045616918) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8541285714285713) - val-core/sciknoweval/acc/best@16/std:np.float64(0.032361125070489466) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5139714285714285) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0607653237770218) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7368714285714285) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0520517104444629) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9991071428571429) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0034580208448280513) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999428571428571) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0008968697662193748) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9982333333333334) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.004700687579230329) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9991285714285715) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0034158694143255103) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9967) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.006019070521218273) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9996714285714285) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.002136984020463583) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9942619047619048) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.007002323475158229) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999476190476191) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0008225042333482549) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9909714285714285) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.006886708259921708) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0001505093393264677) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:1 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000642581726424396 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000642581726424396 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:437.6875 - response_length/max:1165.0 - response_length/min:201.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.6875 - response_length_non_aborted/max:1165.0 - response_length_non_aborted/min:201.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.6875 - prompt_length/max:950.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.976747393608093e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5183244552463293) - timing_s/agent_loop/generate_sequences/max:np.float64(9.816529558971524) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.047968282407965) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.816529558971524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:1165 - timing_s/gen:16.67618520371616 - timing_s/reward:0.06666623428463936 - timing_s/old_log_prob:3.0512595120817423 - timing_s/adv:0.615724191069603 - timing_s/update_actor:21.976544301956892 - timing_s/step:42.47468729875982 - timing_s/testing:132.552817100659 - timing_s/save_checkpoint:13.726791623979807 - timing_s/stop_profile:0.00016298145055770874 - timing_per_token_ms/update_actor:0.11425170677693443 - timing_per_token_ms/gen:0.14883072615054405 - timing_per_token_ms/adv:0.003201028276646996 - perf/total_num_tokens:192352 - perf/time_per_step:42.47468729875982 - perf/throughput:566.0783287438596 (TaskRunner pid=2255397) Training Progress: 80%|████████ | 80/100 [1:15:09<31:40, 95.02s/it] (TaskRunner pid=2255397) step:81 - global_seqlen/min:19063 - global_seqlen/max:27921 - global_seqlen/minmax_diff:8858 - global_seqlen/balanced_min:22607 - global_seqlen/balanced_max:22618 - global_seqlen/mean:22615.0 - actor/entropy:0.4837610721588135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462881445884705 - training/rollout_probs_diff_mean:0.004895538557320833 - training/rollout_probs_diff_std:0.010500817559659481 - training/rollout_actor_probs_pearson_corr:0.9993140697479248 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999274611473083 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.790866897674277e-06 - rollout_corr/rollout_is_max:1.9132018089294434 - rollout_corr/rollout_is_min:0.4658448398113251 - rollout_corr/rollout_is_std:0.036682888865470886 - rollout_corr/rollout_is_eff_sample_size:0.998655936060176 - rollout_corr/rollout_is_seq_mean:0.9999848008155823 - rollout_corr/rollout_is_seq_std:0.002200948540121317 - rollout_corr/rollout_is_seq_max:1.0080451965332031 - rollout_corr/rollout_is_seq_min:0.9924934506416321 - rollout_corr/rollout_is_seq_max_deviation:0.008045196533203125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6707927286624908) - rollout_corr/training_log_ppl:np.float64(0.496672905923333) - rollout_corr/kl:np.float64(0.0007028676274636325) - rollout_corr/k3_kl:np.float64(0.0008774436023486487) - rollout_corr/rollout_ppl:np.float64(1.6695120050571859) - rollout_corr/rollout_log_ppl:np.float64(0.4959700393374078) - rollout_corr/log_ppl_diff:np.float64(0.0007028665859252214) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018403253052383661) - rollout_corr/log_ppl_diff_max:np.float64(0.008645951747894287) - rollout_corr/log_ppl_diff_min:np.float64(-0.005466446280479431) - rollout_corr/ppl_ratio:np.float64(1.0007056852336973) - rollout_corr/chi2_token:np.float64(0.002068758476525545) - rollout_corr/chi2_seq:np.float64(0.5465720517095178) - actor/pg_loss:np.float64(0.0029664566973224282) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035411453472988796) - actor/ppo_kl:np.float64(-4.833459730235745e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(46.35069849877618) - self_distillation/token_reweight_w_std:np.float64(809.1466785562807) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000212090089917) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07078566148993559) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.2487758994102478) - perf/mfu/actor:np.float64(0.0539087676600922) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.76457595825195) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008355281315743923 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008355281315743923 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.96875 - response_length/max:1130.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:398.96875 - response_length_non_aborted/max:1130.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.75 - prompt_length/max:735.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.358565926551819e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.753862602636218) - timing_s/agent_loop/generate_sequences/max:np.float64(8.9995809327811) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.538052094459999) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.9995809327811) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:1130 - timing_s/gen:15.53846220485866 - timing_s/reward:0.06122434884309769 - timing_s/old_log_prob:3.0600726436823606 - timing_s/adv:0.5756961815059185 - timing_s/update_actor:21.892810126766562 - timing_s/step:41.233498902991414 - timing_s/stop_profile:0.00010543875396251678 - timing_per_token_ms/update_actor:0.12100823638495778 - timing_per_token_ms/gen:0.15213501806276591 - timing_per_token_ms/adv:0.0031820483169683754 - perf/total_num_tokens:180920 - perf/time_per_step:41.233498902991414 - perf/throughput:548.461823557722 (TaskRunner pid=2255397) Training Progress: 81%|████████ | 81/100 [1:15:50<24:59, 78.90s/it] (TaskRunner pid=2255397) step:82 - global_seqlen/min:20637 - global_seqlen/max:24388 - global_seqlen/minmax_diff:3751 - global_seqlen/balanced_min:22213 - global_seqlen/balanced_max:22222 - global_seqlen/mean:22220.125 - actor/entropy:0.44993990659713745 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135524034500122 - training/rollout_probs_diff_mean:0.0047300467267632484 - training/rollout_probs_diff_std:0.010585080832242966 - training/rollout_actor_probs_pearson_corr:0.9992892146110535 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999090433120728 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8240019926452078e-05 - rollout_corr/rollout_is_max:1.7895359992980957 - rollout_corr/rollout_is_min:0.000389088352676481 - rollout_corr/rollout_is_std:0.03580636531114578 - rollout_corr/rollout_is_eff_sample_size:0.9987193079476842 - rollout_corr/rollout_is_seq_mean:0.9998470544815063 - rollout_corr/rollout_is_seq_std:0.0019274608930572867 - rollout_corr/rollout_is_seq_max:1.0058091878890991 - rollout_corr/rollout_is_seq_min:0.9948349595069885 - rollout_corr/rollout_is_seq_max_deviation:0.005809187889099121 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6126991645433009) - rollout_corr/training_log_ppl:np.float64(0.4650081274448894) - rollout_corr/kl:np.float64(0.0009749815301454134) - rollout_corr/k3_kl:np.float64(0.0008126581537908351) - rollout_corr/rollout_ppl:np.float64(1.6110614440403879) - rollout_corr/rollout_log_ppl:np.float64(0.46403314505005255) - rollout_corr/log_ppl_diff:np.float64(0.000974982394836843) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017385364044457674) - rollout_corr/log_ppl_diff_max:np.float64(0.01138317584991455) - rollout_corr/log_ppl_diff_min:np.float64(-0.004374325275421143) - rollout_corr/ppl_ratio:np.float64(1.0009776803199202) - rollout_corr/chi2_token:np.float64(0.0012213955633342266) - rollout_corr/chi2_seq:np.float64(0.3682185474317521) - actor/pg_loss:np.float64(0.0047809198731556535) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044549755830303184) - actor/ppo_kl:np.float64(9.73689185599369e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(8763.06257886719) - self_distillation/token_reweight_w_std:np.float64(183308.58723631455) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998282515443861) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10694929157034494) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.27595510706305504) - perf/mfu/actor:np.float64(0.053128164780379857) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.69163131713867) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036035440862178802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036035440862178802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:428.31640625 - response_length/max:948.0 - response_length/min:190.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:428.31640625 - response_length_non_aborted/max:948.0 - response_length_non_aborted/min:190.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.0625 - prompt_length/max:408.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010885484516620636 - timing_s/agent_loop/generate_sequences/min:np.float64(2.409843420609832) - timing_s/agent_loop/generate_sequences/max:np.float64(8.630525501444936) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.044243809614272) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.630525501444936) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:336 - timing_s/agent_loop/slowest/response_length:948 - timing_s/gen:15.308033773675561 - timing_s/reward:0.06256439350545406 - timing_s/old_log_prob:2.968469714745879 - timing_s/adv:0.5528020858764648 - timing_s/update_actor:21.71131898276508 - timing_s/step:40.68941228836775 - timing_s/stop_profile:3.694556653499603e-05 - timing_per_token_ms/update_actor:0.12213769602311575 - timing_per_token_ms/gen:0.13960942437847643 - timing_per_token_ms/adv:0.0031098052209228393 - perf/total_num_tokens:177761 - perf/time_per_step:40.68941228836775 - perf/throughput:546.0910775148323 (TaskRunner pid=2255397) Training Progress: 82%|████████▏ | 82/100 [1:16:31<20:13, 67.44s/it] (TaskRunner pid=2255397) step:83 - global_seqlen/min:17994 - global_seqlen/max:23375 - global_seqlen/minmax_diff:5381 - global_seqlen/balanced_min:21160 - global_seqlen/balanced_max:21170 - global_seqlen/mean:21165.25 - actor/entropy:0.4580913484096527 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1788254976272583 - training/rollout_probs_diff_mean:0.005244065076112747 - training/rollout_probs_diff_std:0.011341297999024391 - training/rollout_actor_probs_pearson_corr:0.9992040395736694 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7578125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7578125 - rollout_corr/rollout_is_mean:1.0000450611114502 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6948878765106201 - rollout_corr/rollout_is_min:0.5044571757316589 - rollout_corr/rollout_is_std:0.03826860338449478 - rollout_corr/rollout_is_eff_sample_size:0.9985378931829969 - rollout_corr/rollout_is_seq_mean:0.9999917149543762 - rollout_corr/rollout_is_seq_std:0.0024205187801271677 - rollout_corr/rollout_is_seq_max:1.007947325706482 - rollout_corr/rollout_is_seq_min:0.9913325309753418 - rollout_corr/rollout_is_seq_max_deviation:0.008667469024658203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6229573977179825) - rollout_corr/training_log_ppl:np.float64(0.4732308021048084) - rollout_corr/kl:np.float64(0.0009288024647275961) - rollout_corr/k3_kl:np.float64(0.0008689218968811474) - rollout_corr/rollout_ppl:np.float64(1.6214703912846744) - rollout_corr/rollout_log_ppl:np.float64(0.47230199666228145) - rollout_corr/log_ppl_diff:np.float64(0.0009288054425269365) - rollout_corr/log_ppl_abs_diff:np.float64(0.002043650601990521) - rollout_corr/log_ppl_diff_max:np.float64(0.01018601655960083) - rollout_corr/log_ppl_diff_min:np.float64(-0.0065596699714660645) - rollout_corr/ppl_ratio:np.float64(1.0009324771817774) - rollout_corr/chi2_token:np.float64(0.0016192824114114046) - rollout_corr/chi2_seq:np.float64(0.7770539077464491) - actor/pg_loss:np.float64(0.005605395417660475) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005950954305262712) - actor/ppo_kl:np.float64(0.00014756420982209306) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(43651.46375838155) - self_distillation/token_reweight_w_std:np.float64(663337.3462570815) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001920053968206) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12145704132854007) - self_distillation/empty_target_batch:np.float64(0.2421875) - actor/grad_norm:np.float64(0.37171755731105804) - perf/mfu/actor:np.float64(0.051307628909952094) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.40789794921875) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006963750929571688 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006963750929571688 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:362.5078125 - response_length/max:903.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:362.5078125 - response_length_non_aborted/max:903.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.90625 - prompt_length/max:563.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010122545063495636 - timing_s/agent_loop/generate_sequences/min:np.float64(1.972249450162053) - timing_s/agent_loop/generate_sequences/max:np.float64(8.24346206523478) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.390775169689732) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.24346206523478) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:312 - timing_s/agent_loop/slowest/response_length:903 - timing_s/gen:14.806242102757096 - timing_s/reward:0.06265864707529545 - timing_s/old_log_prob:3.0254632886499166 - timing_s/adv:0.5114385951310396 - timing_s/update_actor:21.41834749095142 - timing_s/step:39.9120793864131 - timing_s/stop_profile:0.00010847300291061401 - timing_per_token_ms/update_actor:0.12649477026583325 - timing_per_token_ms/gen:0.15954658415505157 - timing_per_token_ms/adv:0.0030205088241990977 - perf/total_num_tokens:169322 - perf/time_per_step:39.9120793864131 - perf/throughput:530.2968506122257 (TaskRunner pid=2255397) Training Progress: 83%|████████▎ | 83/100 [1:17:10<16:46, 59.20s/it] (TaskRunner pid=2255397) step:84 - global_seqlen/min:16326 - global_seqlen/max:23634 - global_seqlen/minmax_diff:7308 - global_seqlen/balanced_min:19956 - global_seqlen/balanced_max:19966 - global_seqlen/mean:19962.0 - actor/entropy:0.4355461001396179 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105639219284058 - training/rollout_probs_diff_mean:0.0052682701498270035 - training/rollout_probs_diff_std:0.011572654359042645 - training/rollout_actor_probs_pearson_corr:0.9991452097892761 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999979138374329 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7088184356689453 - rollout_corr/rollout_is_min:0.6109715700149536 - rollout_corr/rollout_is_std:0.038198452442884445 - rollout_corr/rollout_is_eff_sample_size:0.9985430042300492 - rollout_corr/rollout_is_seq_mean:0.999947190284729 - rollout_corr/rollout_is_seq_std:0.0022522625513374805 - rollout_corr/rollout_is_seq_max:1.0068590641021729 - rollout_corr/rollout_is_seq_min:0.9928377866744995 - rollout_corr/rollout_is_seq_max_deviation:0.007162213325500488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5662654708139598) - rollout_corr/training_log_ppl:np.float64(0.4385948544368148) - rollout_corr/kl:np.float64(0.0010095660052041877) - rollout_corr/k3_kl:np.float64(0.0008661396312845682) - rollout_corr/rollout_ppl:np.float64(1.5646833986975253) - rollout_corr/rollout_log_ppl:np.float64(0.4375852865341585) - rollout_corr/log_ppl_diff:np.float64(0.001009567902656272) - rollout_corr/log_ppl_abs_diff:np.float64(0.002128819382051006) - rollout_corr/log_ppl_diff_max:np.float64(0.010426610708236694) - rollout_corr/log_ppl_diff_min:np.float64(-0.0056244730949401855) - rollout_corr/ppl_ratio:np.float64(1.0010133925825357) - rollout_corr/chi2_token:np.float64(0.0014718861784785986) - rollout_corr/chi2_seq:np.float64(0.8280399544164538) - actor/pg_loss:np.float64(0.005005670711398125) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003053363407161669) - actor/ppo_kl:np.float64(0.00018772196818872544) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(1562.93544895784) - self_distillation/token_reweight_w_std:np.float64(29233.69878470525) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9984272990841419) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09582373814191669) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.27201734483242035) - perf/mfu/actor:np.float64(0.04843784064306516) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.41874313354492) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011461590183898807 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011461590183898807 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.78125 - response_length/max:898.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.78125 - response_length_non_aborted/max:898.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.03125 - prompt_length/max:466.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014925189316272736 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8480602335184813) - timing_s/agent_loop/generate_sequences/max:np.float64(7.672212071716785) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.319992882228689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.672212071716785) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:898 - timing_s/gen:14.744418485090137 - timing_s/reward:0.05119308829307556 - timing_s/old_log_prob:3.006163891404867 - timing_s/adv:0.5809233784675598 - timing_s/update_actor:21.276869893074036 - timing_s/step:39.69830561801791 - timing_s/stop_profile:2.6756897568702698e-05 - timing_per_token_ms/update_actor:0.13323358063491908 - timing_per_token_ms/gen:0.15575528696326096 - timing_per_token_ms/adv:0.0036376827125761436 - perf/total_num_tokens:159696 - perf/time_per_step:39.69830561801791 - perf/throughput:502.8426198356392 (TaskRunner pid=2255397) Training Progress: 84%|████████▍ | 84/100 [1:17:50<14:13, 53.36s/it] (TaskRunner pid=2255397) step:85 - global_seqlen/min:18150 - global_seqlen/max:26353 - global_seqlen/minmax_diff:8203 - global_seqlen/balanced_min:22089 - global_seqlen/balanced_max:22097 - global_seqlen/mean:22095.125 - actor/entropy:0.5015257596969604 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22547149658203125 - training/rollout_probs_diff_mean:0.005411901045590639 - training/rollout_probs_diff_std:0.011227205395698547 - training/rollout_actor_probs_pearson_corr:0.9992418885231018 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0000245571136475 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9998200514237396e-05 - rollout_corr/rollout_is_max:1.8690942525863647 - rollout_corr/rollout_is_min:0.3950064182281494 - rollout_corr/rollout_is_std:0.038822196424007416 - rollout_corr/rollout_is_eff_sample_size:0.9984952239250111 - rollout_corr/rollout_is_seq_mean:1.0000362396240234 - rollout_corr/rollout_is_seq_std:0.0021999897435307503 - rollout_corr/rollout_is_seq_max:1.0113379955291748 - rollout_corr/rollout_is_seq_min:0.9936926364898682 - rollout_corr/rollout_is_seq_max_deviation:0.011337995529174805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6718262596987188) - rollout_corr/training_log_ppl:np.float64(0.5036372784525156) - rollout_corr/kl:np.float64(0.0008671445817096668) - rollout_corr/k3_kl:np.float64(0.0008546028941509576) - rollout_corr/rollout_ppl:np.float64(1.670389695558697) - rollout_corr/rollout_log_ppl:np.float64(0.5027701300568879) - rollout_corr/log_ppl_diff:np.float64(0.000867148395627737) - rollout_corr/log_ppl_abs_diff:np.float64(0.00184359191916883) - rollout_corr/log_ppl_diff_max:np.float64(0.007470607757568359) - rollout_corr/log_ppl_diff_min:np.float64(-0.005036354064941406) - rollout_corr/ppl_ratio:np.float64(1.000870036194101) - rollout_corr/chi2_token:np.float64(0.001706940121948719) - rollout_corr/chi2_seq:np.float64(0.803199248155579) - actor/pg_loss:np.float64(0.0017579543637111783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.799221642315388e-05) - actor/ppo_kl:np.float64(0.00011437771347289072) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(531.5800662350375) - self_distillation/token_reweight_w_std:np.float64(6761.905646110186) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.998828696552664) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02915754570858553) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.16114498674869537) - perf/mfu/actor:np.float64(0.052797853671240076) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.15685653686523) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009824115550145507 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009824115550145507 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:390.66015625 - response_length/max:984.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:390.66015625 - response_length_non_aborted/max:984.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.8125 - prompt_length/max:706.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7002766728401184e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1650554779917) - timing_s/agent_loop/generate_sequences/max:np.float64(8.331863490864635) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.615080809744541) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.331863490864635) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:235 - timing_s/agent_loop/slowest/response_length:984 - timing_s/gen:15.139516476541758 - timing_s/reward:0.0630094688385725 - timing_s/old_log_prob:2.9453230183571577 - timing_s/adv:0.5465950388461351 - timing_s/update_actor:21.74274236895144 - timing_s/step:40.526393212378025 - timing_s/stop_profile:0.0001103375107049942 - timing_per_token_ms/update_actor:0.12300644581639299 - timing_per_token_ms/gen:0.15138154042677918 - timing_per_token_ms/adv:0.0030922830197053375 - perf/total_num_tokens:176761 - perf/time_per_step:40.526393212378025 - perf/throughput:545.2033415411727 (TaskRunner pid=2255397) Training Progress: 85%|████████▌ | 85/100 [1:18:31<12:22, 49.52s/it] (TaskRunner pid=2255397) step:86 - global_seqlen/min:19172 - global_seqlen/max:23707 - global_seqlen/minmax_diff:4535 - global_seqlen/balanced_min:21178 - global_seqlen/balanced_max:21185 - global_seqlen/mean:21182.625 - actor/entropy:0.4417911767959595 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772989869117737 - training/rollout_probs_diff_mean:0.004951337352395058 - training/rollout_probs_diff_std:0.011015241034328938 - training/rollout_actor_probs_pearson_corr:0.999230682849884 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0001946687698364 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0157130418519955e-05 - rollout_corr/rollout_is_max:1.913230538368225 - rollout_corr/rollout_is_min:0.48568272590637207 - rollout_corr/rollout_is_std:0.03722003847360611 - rollout_corr/rollout_is_eff_sample_size:0.9986170608525367 - rollout_corr/rollout_is_seq_mean:1.000307321548462 - rollout_corr/rollout_is_seq_std:0.0024071792140603065 - rollout_corr/rollout_is_seq_max:1.0088176727294922 - rollout_corr/rollout_is_seq_min:0.9944450259208679 - rollout_corr/rollout_is_seq_max_deviation:0.008817672729492188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.608684377744794) - rollout_corr/training_log_ppl:np.float64(0.4590825633786153) - rollout_corr/kl:np.float64(0.0005738776003347823) - rollout_corr/k3_kl:np.float64(0.0008272058389025005) - rollout_corr/rollout_ppl:np.float64(1.6077406979165971) - rollout_corr/rollout_log_ppl:np.float64(0.45850868528941646) - rollout_corr/log_ppl_diff:np.float64(0.0005738780891988426) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020339481707196683) - rollout_corr/log_ppl_diff_max:np.float64(0.008264422416687012) - rollout_corr/log_ppl_diff_min:np.float64(-0.009043440222740173) - rollout_corr/ppl_ratio:np.float64(1.00057721324265) - rollout_corr/chi2_token:np.float64(0.0023612575605511665) - rollout_corr/chi2_seq:np.float64(1.4658026506658643) - actor/pg_loss:np.float64(0.00446285936050117) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004066334154231299) - actor/ppo_kl:np.float64(0.00014794760728875644) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(132.06133578903973) - self_distillation/token_reweight_w_std:np.float64(2865.135968480725) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988646688871086) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1047386541031301) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.297544926404953) - perf/mfu/actor:np.float64(0.05125659080467514) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.21656799316406) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011623566970229149 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011623566970229149 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:384.58203125 - response_length/max:958.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:384.58203125 - response_length_non_aborted/max:958.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.375 - prompt_length/max:503.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012109242379665375 - timing_s/agent_loop/generate_sequences/min:np.float64(2.020683040842414) - timing_s/agent_loop/generate_sequences/max:np.float64(8.49462386034429) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.405667071034259) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.49462386034429) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:958 - timing_s/gen:15.48012137785554 - timing_s/reward:0.06186640448868275 - timing_s/old_log_prob:2.9315725062042475 - timing_s/adv:0.5595468077808619 - timing_s/update_actor:21.52425908483565 - timing_s/step:40.646166034042835 - timing_s/stop_profile:0.00010807625949382782 - timing_per_token_ms/update_actor:0.12701600418288367 - timing_per_token_ms/gen:0.15723361784664297 - timing_per_token_ms/adv:0.0033019208418507023 - perf/total_num_tokens:169461 - perf/time_per_step:40.646166034042835 - perf/throughput:521.1469387360835 (TaskRunner pid=2255397) Training Progress: 86%|████████▌ | 86/100 [1:19:11<10:56, 46.87s/it] (TaskRunner pid=2255397) step:87 - global_seqlen/min:19881 - global_seqlen/max:22805 - global_seqlen/minmax_diff:2924 - global_seqlen/balanced_min:21579 - global_seqlen/balanced_max:21581 - global_seqlen/mean:21580.5 - actor/entropy:0.4654737114906311 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3751887083053589 - training/rollout_probs_diff_mean:0.0050046672113239765 - training/rollout_probs_diff_std:0.010888464748859406 - training/rollout_actor_probs_pearson_corr:0.9992407560348511 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0000450611114502 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0308853234164417e-05 - rollout_corr/rollout_is_max:1.854941487312317 - rollout_corr/rollout_is_min:0.3242693543434143 - rollout_corr/rollout_is_std:0.03734952583909035 - rollout_corr/rollout_is_eff_sample_size:0.998607075049263 - rollout_corr/rollout_is_seq_mean:1.0000066757202148 - rollout_corr/rollout_is_seq_std:0.00233814911916852 - rollout_corr/rollout_is_seq_max:1.0092549324035645 - rollout_corr/rollout_is_seq_min:0.9934799671173096 - rollout_corr/rollout_is_seq_max_deviation:0.009254932403564453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6343936491757631) - rollout_corr/training_log_ppl:np.float64(0.47929042315809056) - rollout_corr/kl:np.float64(0.0010372451878630784) - rollout_corr/k3_kl:np.float64(0.00085206899655077) - rollout_corr/rollout_ppl:np.float64(1.6326408791355789) - rollout_corr/rollout_log_ppl:np.float64(0.4782531792880036) - rollout_corr/log_ppl_diff:np.float64(0.001037243870086968) - rollout_corr/log_ppl_abs_diff:np.float64(0.002030018251389265) - rollout_corr/log_ppl_diff_max:np.float64(0.011031702160835266) - rollout_corr/log_ppl_diff_min:np.float64(-0.006544649600982666) - rollout_corr/ppl_ratio:np.float64(1.0010407408699393) - rollout_corr/chi2_token:np.float64(0.0013177578803151846) - rollout_corr/chi2_seq:np.float64(0.7552365991286933) - actor/pg_loss:np.float64(0.006197207723744214) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000917987231787265) - actor/ppo_kl:np.float64(0.00031247709554804715) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(8798.193575591547) - self_distillation/token_reweight_w_std:np.float64(137248.53065139498) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011895855423063) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1116473093861714) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.3543427586555481) - perf/mfu/actor:np.float64(0.052530135237403436) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.04389572143555) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004306523595005274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004306523595005274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:378.921875 - response_length/max:813.0 - response_length/min:167.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:378.921875 - response_length_non_aborted/max:813.0 - response_length_non_aborted/min:167.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.46875 - prompt_length/max:845.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001322329044342041 - timing_s/agent_loop/generate_sequences/min:np.float64(2.250560261309147) - timing_s/agent_loop/generate_sequences/max:np.float64(7.587926117703319) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.51870351490652) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.587926117703319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:800 - timing_s/gen:14.453906053677201 - timing_s/reward:0.06278987228870392 - timing_s/old_log_prob:3.1354457326233387 - timing_s/adv:0.5562533307820559 - timing_s/update_actor:21.309262923896313 - timing_s/step:39.60505427233875 - timing_s/stop_profile:0.00011653825640678406 - timing_per_token_ms/update_actor:0.12342892266106156 - timing_per_token_ms/gen:0.14900319629785577 - timing_per_token_ms/adv:0.0032219673477332307 - perf/total_num_tokens:172644 - perf/time_per_step:39.60505427233875 - perf/throughput:544.8925748619012 (TaskRunner pid=2255397) Training Progress: 87%|████████▋ | 87/100 [1:19:51<09:41, 44.71s/it] (TaskRunner pid=2255397) step:88 - global_seqlen/min:18137 - global_seqlen/max:23978 - global_seqlen/minmax_diff:5841 - global_seqlen/balanced_min:21618 - global_seqlen/balanced_max:21626 - global_seqlen/mean:21623.75 - actor/entropy:0.4558359980583191 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135497212409973 - training/rollout_probs_diff_mean:0.005039794836193323 - training/rollout_probs_diff_std:0.01103699766099453 - training/rollout_actor_probs_pearson_corr:0.9992441534996033 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.84765625 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84765625 - rollout_corr/rollout_is_mean:0.9999773502349854 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.879797284549568e-05 - rollout_corr/rollout_is_max:1.6429718732833862 - rollout_corr/rollout_is_min:0.34945860505104065 - rollout_corr/rollout_is_std:0.03734952583909035 - rollout_corr/rollout_is_eff_sample_size:0.9986068372944764 - rollout_corr/rollout_is_seq_mean:0.9999490976333618 - rollout_corr/rollout_is_seq_std:0.00212614587508142 - rollout_corr/rollout_is_seq_max:1.007338047027588 - rollout_corr/rollout_is_seq_min:0.9925395846366882 - rollout_corr/rollout_is_seq_max_deviation:0.007460415363311768 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6032785079441965) - rollout_corr/training_log_ppl:np.float64(0.46080948115559295) - rollout_corr/kl:np.float64(0.0011248015301781322) - rollout_corr/k3_kl:np.float64(0.0008873714502897201) - rollout_corr/rollout_ppl:np.float64(1.601451831869781) - rollout_corr/rollout_log_ppl:np.float64(0.4596846750937402) - rollout_corr/log_ppl_diff:np.float64(0.0011248060618527234) - rollout_corr/log_ppl_abs_diff:np.float64(0.002022003580350429) - rollout_corr/log_ppl_diff_max:np.float64(0.012501776218414307) - rollout_corr/log_ppl_diff_min:np.float64(-0.007802844047546387) - rollout_corr/ppl_ratio:np.float64(1.0011287301313132) - rollout_corr/chi2_token:np.float64(0.0012870903592556715) - rollout_corr/chi2_seq:np.float64(0.6421570053789765) - actor/pg_loss:np.float64(0.004205821780487895) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004930502600473119) - actor/ppo_kl:np.float64(0.00033077471513121015) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(21107.031607169192) - self_distillation/token_reweight_w_std:np.float64(309542.6647549386) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0036525861360133) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08997728512622416) - self_distillation/empty_target_batch:np.float64(0.15234375) - actor/grad_norm:np.float64(0.29456714540719986) - perf/mfu/actor:np.float64(0.05240681242673272) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.06198120117188) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007283487357199192 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007283487357199192 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.9296875 - response_length/max:943.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.9296875 - response_length_non_aborted/max:943.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.8125 - prompt_length/max:482.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010753795504570007 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9089515656232834) - timing_s/agent_loop/generate_sequences/max:np.float64(8.329806737601757) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.679670039906341) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.329806737601757) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:329 - timing_s/agent_loop/slowest/response_length:943 - timing_s/gen:15.13785439543426 - timing_s/reward:0.06380515545606613 - timing_s/old_log_prob:3.0357294902205467 - timing_s/adv:0.5423688981682062 - timing_s/update_actor:21.611202096566558 - timing_s/step:40.480039244517684 - timing_s/stop_profile:0.00011552684009075165 - timing_per_token_ms/update_actor:0.12492746457348146 - timing_per_token_ms/gen:0.14531317214884962 - timing_per_token_ms/adv:0.0031352615652246152 - perf/total_num_tokens:172990 - perf/time_per_step:40.480039244517684 - perf/throughput:534.1830295514983 (TaskRunner pid=2255397) Training Progress: 88%|████████▊ | 88/100 [1:20:32<08:41, 43.45s/it] (TaskRunner pid=2255397) step:89 - global_seqlen/min:20192 - global_seqlen/max:26260 - global_seqlen/minmax_diff:6068 - global_seqlen/balanced_min:23178 - global_seqlen/balanced_max:23186 - global_seqlen/mean:23183.75 - actor/entropy:0.4386472702026367 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7019415497779846 - training/rollout_probs_diff_mean:0.0047370451502501965 - training/rollout_probs_diff_std:0.010689225047826767 - training/rollout_actor_probs_pearson_corr:0.999261736869812 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9999846816062927 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8036542314803228e-05 - rollout_corr/rollout_is_max:1.5554907321929932 - rollout_corr/rollout_is_min:0.03433068096637726 - rollout_corr/rollout_is_std:0.0357397198677063 - rollout_corr/rollout_is_eff_sample_size:0.9987241830413971 - rollout_corr/rollout_is_seq_mean:0.9999765157699585 - rollout_corr/rollout_is_seq_std:0.0020081165712326765 - rollout_corr/rollout_is_seq_max:1.0105236768722534 - rollout_corr/rollout_is_seq_min:0.9940011501312256 - rollout_corr/rollout_is_seq_max_deviation:0.010523676872253418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6203870358876884) - rollout_corr/training_log_ppl:np.float64(0.47031728448928334) - rollout_corr/kl:np.float64(0.0010279387461658018) - rollout_corr/k3_kl:np.float64(0.0008464122184363987) - rollout_corr/rollout_ppl:np.float64(1.6186228357255459) - rollout_corr/rollout_log_ppl:np.float64(0.4692893385363277) - rollout_corr/log_ppl_diff:np.float64(0.0010279459529556334) - rollout_corr/log_ppl_abs_diff:np.float64(0.00189788214629516) - rollout_corr/log_ppl_diff_max:np.float64(0.0077957212924957275) - rollout_corr/log_ppl_diff_min:np.float64(-0.004509687423706055) - rollout_corr/ppl_ratio:np.float64(1.0010310297366232) - rollout_corr/chi2_token:np.float64(0.0013277053367346525) - rollout_corr/chi2_seq:np.float64(0.5938672970514745) - actor/pg_loss:np.float64(0.0077381625305861235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008514890976130118) - actor/ppo_kl:np.float64(0.0002891868470378256) - actor/pg_clipfrac_lower:np.float64(8.172070920409169e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(47.27642107871361) - self_distillation/token_reweight_w_std:np.float64(801.2714838348329) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000362384133041) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13113477459410205) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.3602307364344597) - perf/mfu/actor:np.float64(0.05579510012189607) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.08685684204102) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0041562956757843494 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0041562956757843494 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:433.1484375 - response_length/max:1235.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:433.1484375 - response_length_non_aborted/max:1235.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.34375 - prompt_length/max:720.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.681098163127899e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.020791845396161) - timing_s/agent_loop/generate_sequences/max:np.float64(9.594862563535571) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.899216158031777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.594862563535571) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:223 - timing_s/agent_loop/slowest/response_length:1053 - timing_s/gen:16.61686352454126 - timing_s/reward:0.055103445425629616 - timing_s/old_log_prob:2.937300179153681 - timing_s/adv:0.5702405758202076 - timing_s/update_actor:21.570211308076978 - timing_s/step:41.78375179320574 - timing_s/stop_profile:2.8584152460098267e-05 - timing_per_token_ms/update_actor:0.11630027124643866 - timing_per_token_ms/gen:0.1498553787181543 - timing_per_token_ms/adv:0.003074570420122972 - perf/total_num_tokens:185470 - perf/time_per_step:41.78375179320574 - perf/throughput:554.8508452457781 (TaskRunner pid=2255397) Training Progress: 89%|████████▉ | 89/100 [1:21:13<07:52, 42.96s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure of the product and compare it with the options provided. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product has the following key features: (TaskRunner pid=2255397) - A nitrogen-containing ring (likely a pyrrole or similar structure). (TaskRunner pid=2255397) - A **chlorine-substituted benzene ring** (from the **Cl** in **"ccc(Cl)"**). (TaskRunner pid=2255397) - A **trifluoromethyl group** (from **"C(F)(F)F"**). (TaskRunner pid=2255397) - A **nitrogen-containing bridge** connecting the ring to the rest of the molecule. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let’s evaluate each option: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1` (TaskRunner pid=2255397) - Contains a **chlorine-substituted benzene ring**. (TaskRunner pid=2255397) - Contains a **trifluoromethyl group**. (TaskRunner pid=2255397) - Contains a **nitrogen-containing bridge** with the correct connectivity. (TaskRunner pid=2255397) - This matches the structure of the product and appears to be a direct precursor. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1` (TaskRunner pid=2255397) - Contains **chlorine atoms**, but the **trifluoromethyl group** is missing. (TaskRunner pid=2255397) - The **nitrogen-containing bridge** is not properly connected to the **trifluoromethyl** group. (TaskRunner pid=2255397) - This does not match the product structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1` (TaskRunner pid=2255397) - Contains a **chlorine-substituted benzene ring** and a **trifluoromethyl group**. (TaskRunner pid=2255397) - However, the **nitrogen-containing bridge** is not properly positioned and is connected via an **oxygen**, which is not part of the product. (TaskRunner pid=2255397) - This is not a correct match. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D**: (TaskRunner pid=2255397) `O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F` (TaskRunner pid=2255397) - Contains a **trifluoromethyl group**. (TaskRunner pid=2255397) - However, it **lacks the chlorine-substituted benzene ring** and the **correct nitrogen-containing bridge**. (TaskRunner pid=2255397) - This also does not match the product structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Conclusion**: (TaskRunner pid=2255397) Only **Option A** contains all the necessary functional groups and the correct connectivity to serve as the only correct reactant in the synthesis of the given product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2255397) step:90 - global_seqlen/min:18865 - global_seqlen/max:24932 - global_seqlen/minmax_diff:6067 - global_seqlen/balanced_min:22815 - global_seqlen/balanced_max:22831 - global_seqlen/mean:22826.625 - actor/entropy:0.40077415108680725 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5775288939476013 - training/rollout_probs_diff_mean:0.0045609320513904095 - training/rollout_probs_diff_std:0.010716044344007969 - training/rollout_actor_probs_pearson_corr:0.9992610216140747 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:0.9999971985816956 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.970459930424113e-06 - rollout_corr/rollout_is_max:1.8681867122650146 - rollout_corr/rollout_is_min:0.2344389706850052 - rollout_corr/rollout_is_std:0.03548699617385864 - rollout_corr/rollout_is_eff_sample_size:0.9987421380652595 - rollout_corr/rollout_is_seq_mean:1.0000590085983276 - rollout_corr/rollout_is_seq_std:0.0020531753543764353 - rollout_corr/rollout_is_seq_max:1.0109810829162598 - rollout_corr/rollout_is_seq_min:0.9949836730957031 - rollout_corr/rollout_is_seq_max_deviation:0.010981082916259766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5583982807584107) - rollout_corr/training_log_ppl:np.float64(0.43078241116018035) - rollout_corr/kl:np.float64(0.0008912494752149058) - rollout_corr/k3_kl:np.float64(0.000791456644748223) - rollout_corr/rollout_ppl:np.float64(1.5569715378805995) - rollout_corr/rollout_log_ppl:np.float64(0.42989116214448586) - rollout_corr/log_ppl_diff:np.float64(0.0008912490156944841) - rollout_corr/log_ppl_abs_diff:np.float64(0.001720420696074143) - rollout_corr/log_ppl_diff_max:np.float64(0.01338726282119751) - rollout_corr/log_ppl_diff_min:np.float64(-0.00902646780014038) - rollout_corr/ppl_ratio:np.float64(1.0008942331187427) - rollout_corr/chi2_token:np.float64(0.0014041787944734097) - rollout_corr/chi2_seq:np.float64(0.752974865725264) - actor/pg_loss:np.float64(0.004781530937179923) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005209917633237637) - actor/ppo_kl:np.float64(0.0002642208678480529) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(619.1751045542769) - self_distillation/token_reweight_w_std:np.float64(7680.31531262747) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999175232835114) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1011948958330322) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.260063748806715) - perf/mfu/actor:np.float64(0.05435728539450956) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(196.9617691040039) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7092261904761905) - val-aux/sciknoweval/reward/std@16:np.float64(0.1437505073441559) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7665238095238094) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11153257282472556) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6511952380952379) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13026803458061142) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7090476190476193) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14272451357066823) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8092428571428572) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07643847293889101) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5996714285714285) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10642069734217646) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7207095238095238) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11336797001466493) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8373142857142858) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.051272065112436704) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5569380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07817843871980924) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7271952380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08253191872957805) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8582761904761905) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03254491463909445) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.525004761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05193613577764248) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7298285714285715) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05986238009450025) - val-aux/sciknoweval/score/mean@16:np.float64(0.7092261904761905) - val-aux/sciknoweval/score/std@16:np.float64(0.1437505073441559) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7665238095238094) - val-aux/sciknoweval/score/best@2/std:np.float64(0.11153257282472556) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6511952380952379) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13026803458061142) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7090476190476193) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14272451357066823) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8092428571428572) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07643847293889101) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5996714285714285) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10642069734217646) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7207095238095238) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.11336797001466493) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8373142857142858) - val-aux/sciknoweval/score/best@8/std:np.float64(0.051272065112436704) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5569380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07817843871980924) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7271952380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08253191872957805) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8582761904761905) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03254491463909445) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.525004761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05193613577764248) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7298285714285715) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05986238009450025) - val-core/sciknoweval/acc/mean@16:np.float64(0.7092261904761905) - val-aux/sciknoweval/acc/std@16:np.float64(0.1437505073441559) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7665238095238094) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.11153257282472556) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6511952380952379) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13026803458061142) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7090476190476193) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14272451357066823) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8092428571428572) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07643847293889101) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5996714285714285) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10642069734217646) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7207095238095238) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.11336797001466493) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8373142857142858) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.051272065112436704) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5569380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07817843871980924) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7271952380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08253191872957805) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8582761904761905) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03254491463909445) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.525004761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05193613577764248) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7298285714285715) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05986238009450025) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9967261904761905) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.011218419246342346) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9996714285714285) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0035108104210424323) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9937666666666667) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.014896548825679245) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9966666666666667) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.011281400445573764) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0001505093393264677) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9886285714285715) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.018614743576868955) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9989571428571429) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.00637191435060849) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9802904761904762) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.02088499966775818) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997285714285713) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0030510338258627335) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9701857142857143) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.018986995237275702) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0003010186786529354) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0026496497448533773 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0026496497448533773 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:435.45703125 - response_length/max:1197.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:435.45703125 - response_length_non_aborted/max:1197.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.875 - prompt_length/max:621.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.609938383102417e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6878780890256166) - timing_s/agent_loop/generate_sequences/max:np.float64(10.072803135961294) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.094866805477068) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.072803135961294) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:1197 - timing_s/gen:16.879685267806053 - timing_s/reward:0.06467907875776291 - timing_s/old_log_prob:3.10450641810894 - timing_s/adv:0.582694835960865 - timing_s/update_actor:21.818151073530316 - timing_s/step:42.53507168777287 - timing_s/testing:140.61721943691373 - timing_s/save_checkpoint:13.685923740267754 - timing_s/stop_profile:0.00018855556845664978 - timing_per_token_ms/update_actor:0.11947753486077287 - timing_per_token_ms/gen:0.15141854613782263 - timing_per_token_ms/adv:0.0031908726977863846 - perf/total_num_tokens:182613 - perf/time_per_step:42.53507168777287 - perf/throughput:536.6542030905226 (TaskRunner pid=2255397) Training Progress: 90%|█████████ | 90/100 [1:24:30<14:51, 89.15s/it] (TaskRunner pid=2255397) step:91 - global_seqlen/min:20110 - global_seqlen/max:28143 - global_seqlen/minmax_diff:8033 - global_seqlen/balanced_min:22954 - global_seqlen/balanced_max:22958 - global_seqlen/mean:22955.625 - actor/entropy:0.4345111846923828 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.318805456161499 - training/rollout_probs_diff_mean:0.00483555207028985 - training/rollout_probs_diff_std:0.010917664505541325 - training/rollout_actor_probs_pearson_corr:0.9992064833641052 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000874996185303 - rollout_corr/rollout_is_ratio_fraction_high:9.058217074198183e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8116434148396365e-05 - rollout_corr/rollout_is_max:2.0618367195129395 - rollout_corr/rollout_is_min:0.27237245440483093 - rollout_corr/rollout_is_std:0.037339948117733 - rollout_corr/rollout_is_eff_sample_size:0.9986079071919081 - rollout_corr/rollout_is_seq_mean:1.0000126361846924 - rollout_corr/rollout_is_seq_std:0.0018744589760899544 - rollout_corr/rollout_is_seq_max:1.0062435865402222 - rollout_corr/rollout_is_seq_min:0.9926995038986206 - rollout_corr/rollout_is_seq_max_deviation:0.0073004961013793945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5809713881462812) - rollout_corr/training_log_ppl:np.float64(0.4482057260465808) - rollout_corr/kl:np.float64(0.0007782714664905654) - rollout_corr/k3_kl:np.float64(0.0007742295367734187) - rollout_corr/rollout_ppl:np.float64(1.5796857569366693) - rollout_corr/rollout_log_ppl:np.float64(0.4474274521926418) - rollout_corr/log_ppl_diff:np.float64(0.0007782738539390266) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016364274197258055) - rollout_corr/log_ppl_diff_max:np.float64(0.007638216018676758) - rollout_corr/log_ppl_diff_min:np.float64(-0.005470573902130127) - rollout_corr/ppl_ratio:np.float64(1.0007805426139385) - rollout_corr/chi2_token:np.float64(0.001570558873936534) - rollout_corr/chi2_seq:np.float64(0.9185826345346868) - actor/pg_loss:np.float64(0.004598198342137039) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000316593827392353) - actor/ppo_kl:np.float64(5.939174411295767e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(7563.319094766863) - self_distillation/token_reweight_w_std:np.float64(150023.32301567192) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999886890873313) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08767912839539349) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.2757617086172104) - perf/mfu/actor:np.float64(0.054246744251757004) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(201.1460475921631) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008684565545991063 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008684565545991063 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:431.23828125 - response_length/max:1001.0 - response_length/min:187.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:431.23828125 - response_length_non_aborted/max:1001.0 - response_length_non_aborted/min:187.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.125 - prompt_length/max:1009.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010878220200538635 - timing_s/agent_loop/generate_sequences/min:np.float64(2.29626976698637) - timing_s/agent_loop/generate_sequences/max:np.float64(8.867659989744425) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.9120760589357815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.867659989744425) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:212 - timing_s/agent_loop/slowest/response_length:1001 - timing_s/gen:15.32518370822072 - timing_s/reward:0.061427073553204536 - timing_s/old_log_prob:3.1340453140437603 - timing_s/adv:0.5956883151084185 - timing_s/update_actor:22.10231014713645 - timing_s/step:41.29857017099857 - timing_s/stop_profile:0.00013570114970207214 - timing_per_token_ms/update_actor:0.12035345447540881 - timing_per_token_ms/gen:0.1388188420719831 - timing_per_token_ms/adv:0.003243694710492627 - perf/total_num_tokens:183645 - perf/time_per_step:41.29857017099857 - perf/throughput:555.8455148677353 (TaskRunner pid=2255397) Training Progress: 91%|█████████ | 91/100 [1:25:12<11:13, 74.81s/it] (TaskRunner pid=2255397) step:92 - global_seqlen/min:20808 - global_seqlen/max:26065 - global_seqlen/minmax_diff:5257 - global_seqlen/balanced_min:23344 - global_seqlen/balanced_max:23359 - global_seqlen/mean:23353.875 - actor/entropy:0.43259748816490173 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583375573158264 - training/rollout_probs_diff_mean:0.004716021008789539 - training/rollout_probs_diff_std:0.010604353621602058 - training/rollout_actor_probs_pearson_corr:0.9992642402648926 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0000042915344238 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.831503691908438e-06 - rollout_corr/rollout_is_max:1.6297050714492798 - rollout_corr/rollout_is_min:0.3157358169555664 - rollout_corr/rollout_is_std:0.03597741574048996 - rollout_corr/rollout_is_eff_sample_size:0.9987072987735173 - rollout_corr/rollout_is_seq_mean:0.9999913573265076 - rollout_corr/rollout_is_seq_std:0.0019688971806317568 - rollout_corr/rollout_is_seq_max:1.0057662725448608 - rollout_corr/rollout_is_seq_min:0.9920455813407898 - rollout_corr/rollout_is_seq_max_deviation:0.007954418659210205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5944940163753927) - rollout_corr/training_log_ppl:np.float64(0.4538156447233632) - rollout_corr/kl:np.float64(0.0008374510896800302) - rollout_corr/k3_kl:np.float64(0.000751384576176406) - rollout_corr/rollout_ppl:np.float64(1.5930784032680094) - rollout_corr/rollout_log_ppl:np.float64(0.45297819224651903) - rollout_corr/log_ppl_diff:np.float64(0.0008374524768441916) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017089889151975513) - rollout_corr/log_ppl_diff_max:np.float64(0.008830234408378601) - rollout_corr/log_ppl_diff_min:np.float64(-0.005124390125274658) - rollout_corr/ppl_ratio:np.float64(1.0008400096558034) - rollout_corr/chi2_token:np.float64(0.0013158204965293407) - rollout_corr/chi2_seq:np.float64(0.8615647666156292) - actor/pg_loss:np.float64(0.00415497412905097) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000329681670109494) - actor/ppo_kl:np.float64(0.000143634846812013) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(2081.352001107065) - self_distillation/token_reweight_w_std:np.float64(46560.20298913785) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010203949641436) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09130151267163455) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.25993651896715164) - perf/mfu/actor:np.float64(0.05560204307595709) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.7599754333496) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011590245179831982 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011590245179831982 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:442.30859375 - response_length/max:1140.0 - response_length/min:182.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:442.30859375 - response_length_non_aborted/max:1140.0 - response_length_non_aborted/min:182.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:287.5 - prompt_length/max:436.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015668943524360657 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3020109608769417) - timing_s/agent_loop/generate_sequences/max:np.float64(9.68418238312006) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.095685060987307) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.68418238312006) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:1140 - timing_s/gen:16.275774881243706 - timing_s/reward:0.06520042568445206 - timing_s/old_log_prob:3.0083443708717823 - timing_s/adv:0.5682667158544064 - timing_s/update_actor:22.008645860478282 - timing_s/step:42.016618859022856 - timing_s/stop_profile:0.00013844482600688934 - timing_per_token_ms/update_actor:0.11779975411188873 - timing_per_token_ms/gen:0.1437395667374103 - timing_per_token_ms/adv:0.0030416082762197193 - perf/total_num_tokens:186831 - perf/time_per_step:42.016618859022856 - perf/throughput:555.8247101785744 (TaskRunner pid=2255397) Training Progress: 92%|█████████▏| 92/100 [1:25:54<08:39, 64.99s/it] (TaskRunner pid=2255397) step:93 - global_seqlen/min:20104 - global_seqlen/max:26536 - global_seqlen/minmax_diff:6432 - global_seqlen/balanced_min:22165 - global_seqlen/balanced_max:22172 - global_seqlen/mean:22170.375 - actor/entropy:0.4773474633693695 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463870882987976 - training/rollout_probs_diff_mean:0.005192984826862812 - training/rollout_probs_diff_std:0.01093792449682951 - training/rollout_actor_probs_pearson_corr:0.9992780685424805 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.9921875 - self_distillation/correct_sample_fraction:0.83203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:1.0000450611114502 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.661463737487793 - rollout_corr/rollout_is_min:0.516041100025177 - rollout_corr/rollout_is_std:0.037531014531850815 - rollout_corr/rollout_is_eff_sample_size:0.9985935232071051 - rollout_corr/rollout_is_seq_mean:1.0000596046447754 - rollout_corr/rollout_is_seq_std:0.0020964141003787518 - rollout_corr/rollout_is_seq_max:1.0067566633224487 - rollout_corr/rollout_is_seq_min:0.9933486580848694 - rollout_corr/rollout_is_seq_max_deviation:0.0067566633224487305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6435860991477966) - rollout_corr/training_log_ppl:np.float64(0.48362978905788623) - rollout_corr/kl:np.float64(0.000619826348660979) - rollout_corr/k3_kl:np.float64(0.0007837708870965798) - rollout_corr/rollout_ppl:np.float64(1.6425714306533337) - rollout_corr/rollout_log_ppl:np.float64(0.48300995968747884) - rollout_corr/log_ppl_diff:np.float64(0.0006198293704073876) - rollout_corr/log_ppl_abs_diff:np.float64(0.001762611762387678) - rollout_corr/log_ppl_diff_max:np.float64(0.009479671716690063) - rollout_corr/log_ppl_diff_min:np.float64(-0.005926311016082764) - rollout_corr/ppl_ratio:np.float64(1.0006224988028407) - rollout_corr/chi2_token:np.float64(0.0019295907113701105) - rollout_corr/chi2_seq:np.float64(1.5755573513451964) - actor/pg_loss:np.float64(0.004148329375311732) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002517454886401538) - actor/ppo_kl:np.float64(-5.375289406117645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_w_mean:np.float64(17544.475337664597) - self_distillation/token_reweight_w_std:np.float64(333058.533660894) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001345220953226) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08209344942588359) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.2505064271390438) - perf/mfu/actor:np.float64(0.05288764926923402) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(200.60185623168945) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.83203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00024851804482750595 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00024851804482750595 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:426.35546875 - response_length/max:917.0 - response_length/min:192.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:426.35546875 - response_length_non_aborted/max:917.0 - response_length_non_aborted/min:192.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:709.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014604441821575165 - timing_s/agent_loop/generate_sequences/min:np.float64(2.234377281740308) - timing_s/agent_loop/generate_sequences/max:np.float64(8.298982378095388) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.870763707032893) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.298982378095388) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:193 - timing_s/agent_loop/slowest/response_length:917 - timing_s/gen:15.365983800962567 - timing_s/reward:0.06459944881498814 - timing_s/old_log_prob:2.9253345616161823 - timing_s/adv:0.5848103743046522 - timing_s/update_actor:22.019390346482396 - timing_s/step:41.04795786552131 - timing_s/stop_profile:0.00011224299669265747 - timing_per_token_ms/update_actor:0.1241487251934304 - timing_per_token_ms/gen:0.1407824658576284 - timing_per_token_ms/adv:0.0032972512547975183 - perf/total_num_tokens:177363 - perf/time_per_step:41.04795786552131 - perf/throughput:540.1090858803052 (TaskRunner pid=2255397) Training Progress: 93%|█████████▎| 93/100 [1:26:35<06:44, 57.82s/it] (TaskRunner pid=2255397) step:94 - global_seqlen/min:20450 - global_seqlen/max:26778 - global_seqlen/minmax_diff:6328 - global_seqlen/balanced_min:23452 - global_seqlen/balanced_max:23461 - global_seqlen/mean:23457.625 - actor/entropy:0.45124438405036926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47588256001472473 - training/rollout_probs_diff_mean:0.004685731139034033 - training/rollout_probs_diff_std:0.010418391786515713 - training/rollout_actor_probs_pearson_corr:0.999316930770874 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9999880194664001 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4401948394370265e-05 - rollout_corr/rollout_is_max:1.8073612451553345 - rollout_corr/rollout_is_min:0.04583345353603363 - rollout_corr/rollout_is_std:0.03505437821149826 - rollout_corr/rollout_is_eff_sample_size:0.9987726987625546 - rollout_corr/rollout_is_seq_mean:0.9999796152114868 - rollout_corr/rollout_is_seq_std:0.0017362802755087614 - rollout_corr/rollout_is_seq_max:1.0060876607894897 - rollout_corr/rollout_is_seq_min:0.9951460361480713 - rollout_corr/rollout_is_seq_max_deviation:0.006087660789489746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.600627651438117) - rollout_corr/training_log_ppl:np.float64(0.4611963724019006) - rollout_corr/kl:np.float64(0.0007944866175790821) - rollout_corr/k3_kl:np.float64(0.000699135587751698) - rollout_corr/rollout_ppl:np.float64(1.599346871022135) - rollout_corr/rollout_log_ppl:np.float64(0.46040188503684476) - rollout_corr/log_ppl_diff:np.float64(0.0007944873650558293) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015184025396592915) - rollout_corr/log_ppl_diff_max:np.float64(0.007160961627960205) - rollout_corr/log_ppl_diff_min:np.float64(-0.004008620977401733) - rollout_corr/ppl_ratio:np.float64(1.0007963897660375) - rollout_corr/chi2_token:np.float64(0.0012088804505765438) - rollout_corr/chi2_seq:np.float64(0.3547051209025085) - actor/pg_loss:np.float64(0.0030129868537187576) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016773132483649533) - actor/ppo_kl:np.float64(0.00010856679642934353) - actor/pg_clipfrac_lower:np.float64(7.659314178454224e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(4465.609730498167) - self_distillation/token_reweight_w_std:np.float64(81083.046648157) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998130949679762) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04856777528766543) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.18460461869835854) - perf/mfu/actor:np.float64(0.05546533287041886) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.40380096435547) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005895104259252548 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.005895104259252548 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:480.23828125 - response_length/max:1044.0 - response_length/min:226.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.23828125 - response_length_non_aborted/max:1044.0 - response_length_non_aborted/min:226.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.8125 - prompt_length/max:384.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012932159006595612 - timing_s/agent_loop/generate_sequences/min:np.float64(2.888417324051261) - timing_s/agent_loop/generate_sequences/max:np.float64(9.281017616391182) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.676639520912431) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.281017616391182) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:997 - timing_s/gen:15.90216557495296 - timing_s/reward:0.06385539285838604 - timing_s/old_log_prob:2.9568253997713327 - timing_s/adv:0.5866717882454395 - timing_s/update_actor:21.823786733672023 - timing_s/step:41.424420485273004 - timing_s/stop_profile:0.00012981146574020386 - timing_per_token_ms/update_actor:0.11629367174677756 - timing_per_token_ms/gen:0.12934794393207277 - timing_per_token_ms/adv:0.0031262318129256454 - perf/total_num_tokens:187661 - perf/time_per_step:41.424420485273004 - perf/throughput:566.2752725373559 (TaskRunner pid=2255397) Training Progress: 94%|█████████▍| 94/100 [1:27:16<05:17, 52.92s/it] (TaskRunner pid=2255397) step:95 - global_seqlen/min:18176 - global_seqlen/max:26822 - global_seqlen/minmax_diff:8646 - global_seqlen/balanced_min:23320 - global_seqlen/balanced_max:23330 - global_seqlen/mean:23326.75 - actor/entropy:0.42938148975372314 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102376401424408 - training/rollout_probs_diff_mean:0.0046544442884624004 - training/rollout_probs_diff_std:0.010648963041603565 - training/rollout_actor_probs_pearson_corr:0.9992654323577881 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000568628311157 - rollout_corr/rollout_is_ratio_fraction_high:1.6773175957496278e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.386587978748139e-06 - rollout_corr/rollout_is_max:2.396075487136841 - rollout_corr/rollout_is_min:0.3959798514842987 - rollout_corr/rollout_is_std:0.035107050091028214 - rollout_corr/rollout_is_eff_sample_size:0.9987691312695665 - rollout_corr/rollout_is_seq_mean:1.0000056028366089 - rollout_corr/rollout_is_seq_std:0.0018295958871021867 - rollout_corr/rollout_is_seq_max:1.0062776803970337 - rollout_corr/rollout_is_seq_min:0.9954047203063965 - rollout_corr/rollout_is_seq_max_deviation:0.006277680397033691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5648417291231453) - rollout_corr/training_log_ppl:np.float64(0.4370654333033599) - rollout_corr/kl:np.float64(0.0006150017093133897) - rollout_corr/k3_kl:np.float64(0.0007087535628897967) - rollout_corr/rollout_ppl:np.float64(1.5638857381418347) - rollout_corr/rollout_log_ppl:np.float64(0.43645042926073074) - rollout_corr/log_ppl_diff:np.float64(0.0006150040426291525) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015464308089576662) - rollout_corr/log_ppl_diff_max:np.float64(0.005993485450744629) - rollout_corr/log_ppl_diff_min:np.float64(-0.004650503396987915) - rollout_corr/ppl_ratio:np.float64(1.000617001671344) - rollout_corr/chi2_token:np.float64(0.0016058182809501886) - rollout_corr/chi2_seq:np.float64(0.5998094449751079) - actor/pg_loss:np.float64(0.005259391153231263) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024053280617408745) - actor/ppo_kl:np.float64(-4.200763549278008e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(142.92776992870495) - self_distillation/token_reweight_w_std:np.float64(3127.1786301988177) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005555916577578) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0809866230993066) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.24522369727492332) - perf/mfu/actor:np.float64(0.05459364258201367) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.3312644958496) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017624415457248688 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.017624415457248688 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:465.7734375 - response_length/max:1081.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:465.7734375 - response_length_non_aborted/max:1081.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.1875 - prompt_length/max:646.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012592412531375885 - timing_s/agent_loop/generate_sequences/min:np.float64(1.609277019277215) - timing_s/agent_loop/generate_sequences/max:np.float64(9.628803690895438) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.290896388272813) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.628803690895438) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1081 - timing_s/gen:16.404801338911057 - timing_s/reward:0.06480440311133862 - timing_s/old_log_prob:3.627877188846469 - timing_s/adv:0.5782617889344692 - timing_s/update_actor:21.97222731821239 - timing_s/step:42.739498157054186 - timing_s/stop_profile:0.00011696107685565948 - timing_per_token_ms/update_actor:0.1177415805792298 - timing_per_token_ms/gen:0.13758031281060615 - timing_per_token_ms/adv:0.0030987052897128256 - perf/total_num_tokens:186614 - perf/time_per_step:42.739498157054186 - perf/throughput:545.7890477394363 (TaskRunner pid=2255397) Training Progress: 95%|█████████▌| 95/100 [1:27:59<04:09, 49.88s/it] (TaskRunner pid=2255397) step:96 - global_seqlen/min:20301 - global_seqlen/max:28288 - global_seqlen/minmax_diff:7987 - global_seqlen/balanced_min:23030 - global_seqlen/balanced_max:23033 - global_seqlen/mean:23031.375 - actor/entropy:0.44385668635368347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772989869117737 - training/rollout_probs_diff_mean:0.004754771012812853 - training/rollout_probs_diff_std:0.010701586492359638 - training/rollout_actor_probs_pearson_corr:0.99928879737854 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.000027060508728 - rollout_corr/rollout_is_ratio_fraction_high:8.62165597936837e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.084261417388916 - rollout_corr/rollout_is_min:0.5124618411064148 - rollout_corr/rollout_is_std:0.036241523921489716 - rollout_corr/rollout_is_eff_sample_size:0.9986881560335953 - rollout_corr/rollout_is_seq_mean:1.0000017881393433 - rollout_corr/rollout_is_seq_std:0.0018420576816424727 - rollout_corr/rollout_is_seq_max:1.0063093900680542 - rollout_corr/rollout_is_seq_min:0.9944624304771423 - rollout_corr/rollout_is_seq_max_deviation:0.006309390068054199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.598726301919669) - rollout_corr/training_log_ppl:np.float64(0.45704233669675887) - rollout_corr/kl:np.float64(0.0006188384301958338) - rollout_corr/k3_kl:np.float64(0.0007431540022935224) - rollout_corr/rollout_ppl:np.float64(1.5976339941844344) - rollout_corr/rollout_log_ppl:np.float64(0.45642349659465253) - rollout_corr/log_ppl_diff:np.float64(0.0006188401021063328) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015413255896419287) - rollout_corr/log_ppl_diff_max:np.float64(0.0064958930015563965) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055953264236450195) - rollout_corr/ppl_ratio:np.float64(1.0006208247505128) - rollout_corr/chi2_token:np.float64(0.0018268234562128782) - rollout_corr/chi2_seq:np.float64(1.2716746137011796) - actor/pg_loss:np.float64(0.004606917384080589) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021080635951875593) - actor/ppo_kl:np.float64(-7.161125205357166e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(99.39291401347145) - self_distillation/token_reweight_w_std:np.float64(2908.174926977954) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999677997548133) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08357357300701551) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.23777736723423004) - perf/mfu/actor:np.float64(0.05523982050510799) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.2464942932129) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003513324772939086 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003513324772939086 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.07421875 - response_length/max:1135.0 - response_length/min:170.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.07421875 - response_length_non_aborted/max:1135.0 - response_length_non_aborted/min:170.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.65625 - prompt_length/max:456.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.842030704021454e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1754575334489346) - timing_s/agent_loop/generate_sequences/max:np.float64(9.51023275591433) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.9925034317566315) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.51023275591433) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:196 - timing_s/agent_loop/slowest/response_length:1135 - timing_s/gen:16.42838684655726 - timing_s/reward:0.07112030126154423 - timing_s/old_log_prob:3.2094656210392714 - timing_s/adv:0.5936893373727798 - timing_s/update_actor:21.62316256016493 - timing_s/step:42.020951598882675 - timing_s/stop_profile:0.00010878965258598328 - timing_per_token_ms/update_actor:0.11735709743862953 - timing_per_token_ms/gen:0.14163989797612886 - timing_per_token_ms/adv:0.0032221770159878635 - perf/total_num_tokens:184251 - perf/time_per_step:42.020951598882675 - perf/throughput:548.0926567263269 (TaskRunner pid=2255397) Training Progress: 96%|█████████▌| 96/100 [1:28:41<03:10, 47.53s/it] (TaskRunner pid=2255397) step:97 - global_seqlen/min:21184 - global_seqlen/max:28019 - global_seqlen/minmax_diff:6835 - global_seqlen/balanced_min:24523 - global_seqlen/balanced_max:24538 - global_seqlen/mean:24532.875 - actor/entropy:0.43073078989982605 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5830099582672119 - training/rollout_probs_diff_mean:0.004653396084904671 - training/rollout_probs_diff_std:0.010732254013419151 - training/rollout_actor_probs_pearson_corr:0.9992837905883789 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:0.999879002571106 - rollout_corr/rollout_is_ratio_fraction_high:8.271640581369866e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.654328116273973e-05 - rollout_corr/rollout_is_max:2.069552421569824 - rollout_corr/rollout_is_min:0.06639717519283295 - rollout_corr/rollout_is_std:0.03590112552046776 - rollout_corr/rollout_is_eff_sample_size:0.9987124115549366 - rollout_corr/rollout_is_seq_mean:0.9998928308486938 - rollout_corr/rollout_is_seq_std:0.001641205046325922 - rollout_corr/rollout_is_seq_max:1.008506178855896 - rollout_corr/rollout_is_seq_min:0.9953399300575256 - rollout_corr/rollout_is_seq_max_deviation:0.008506178855895996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5909629054367542) - rollout_corr/training_log_ppl:np.float64(0.45360937030636705) - rollout_corr/kl:np.float64(0.0007517823871872054) - rollout_corr/k3_kl:np.float64(0.0007286852571155578) - rollout_corr/rollout_ppl:np.float64(1.5898262090049684) - rollout_corr/rollout_log_ppl:np.float64(0.45285758542013355) - rollout_corr/log_ppl_diff:np.float64(0.0007517848862335086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015345667488873005) - rollout_corr/log_ppl_diff_max:np.float64(0.00910806655883789) - rollout_corr/log_ppl_diff_min:np.float64(-0.007346630096435547) - rollout_corr/ppl_ratio:np.float64(1.0007539454381913) - rollout_corr/chi2_token:np.float64(0.00142996059730649) - rollout_corr/chi2_seq:np.float64(0.9373617491219193) - actor/pg_loss:np.float64(0.0030622705817222595) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003035150989489921) - actor/ppo_kl:np.float64(-6.880035599365897e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(482.64709699410014) - self_distillation/token_reweight_w_std:np.float64(7838.689288595924) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0027755422051996) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06985412852372974) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.2353973463177681) - perf/mfu/actor:np.float64(0.058007253459953285) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.27971267700195) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036426237784326077 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036426237784326077 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:472.24609375 - response_length/max:1302.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:472.24609375 - response_length_non_aborted/max:1302.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.40625 - prompt_length/max:900.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013850070536136627 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9791246633976698) - timing_s/agent_loop/generate_sequences/max:np.float64(10.536064883694053) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.344745161499304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.536064883694053) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:1302 - timing_s/gen:17.246258018538356 - timing_s/reward:0.08357716910541058 - timing_s/old_log_prob:2.9908031839877367 - timing_s/adv:0.5831174645572901 - timing_s/update_actor:22.14829995855689 - timing_s/step:43.14054214023054 - timing_s/stop_profile:0.00011339783668518066 - timing_per_token_ms/update_actor:0.11285010398575834 - timing_per_token_ms/gen:0.14265484940269121 - timing_per_token_ms/adv:0.0029711023705807516 - perf/total_num_tokens:196263 - perf/time_per_step:43.14054214023054 - perf/throughput:568.6733124552454 (TaskRunner pid=2255397) Training Progress: 97%|█████████▋| 97/100 [1:29:24<02:18, 46.23s/it] (TaskRunner pid=2255397) step:98 - global_seqlen/min:21081 - global_seqlen/max:28015 - global_seqlen/minmax_diff:6934 - global_seqlen/balanced_min:24041 - global_seqlen/balanced_max:24240 - global_seqlen/mean:24067.875 - actor/entropy:0.4481050968170166 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3950059413909912 - training/rollout_probs_diff_mean:0.0048210457898676395 - training/rollout_probs_diff_std:0.010794337838888168 - training/rollout_actor_probs_pearson_corr:0.9992629289627075 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.000078797340393 - rollout_corr/rollout_is_ratio_fraction_high:3.446166738285683e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.7230833691428415e-05 - rollout_corr/rollout_is_max:4.591516971588135 - rollout_corr/rollout_is_min:0.39532703161239624 - rollout_corr/rollout_is_std:0.03653962165117264 - rollout_corr/rollout_is_eff_sample_size:0.9986668739730418 - rollout_corr/rollout_is_seq_mean:1.000028133392334 - rollout_corr/rollout_is_seq_std:0.0019362398888915777 - rollout_corr/rollout_is_seq_max:1.0084097385406494 - rollout_corr/rollout_is_seq_min:0.9942269325256348 - rollout_corr/rollout_is_seq_max_deviation:0.008409738540649414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.616344683803618) - rollout_corr/training_log_ppl:np.float64(0.4696293716551736) - rollout_corr/kl:np.float64(0.0008239057563663721) - rollout_corr/k3_kl:np.float64(0.000780692749458467) - rollout_corr/rollout_ppl:np.float64(1.6150271990336478) - rollout_corr/rollout_log_ppl:np.float64(0.46880546608008444) - rollout_corr/log_ppl_diff:np.float64(0.0008239055750891566) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015701879747211933) - rollout_corr/log_ppl_diff_max:np.float64(0.007200688123703003) - rollout_corr/log_ppl_diff_min:np.float64(-0.004303097724914551) - rollout_corr/ppl_ratio:np.float64(1.0008261122275144) - rollout_corr/chi2_token:np.float64(0.001507394015789032) - rollout_corr/chi2_seq:np.float64(0.24129036953672767) - actor/pg_loss:np.float64(0.003733934834599495) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000350107799476973) - actor/ppo_kl:np.float64(0.00011479198082042785) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(2963.129227875499) - self_distillation/token_reweight_w_std:np.float64(42459.57496579323) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0034989113919437) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09955895229359157) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.2265518605709076) - perf/mfu/actor:np.float64(0.057423452397621935) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.2193260192871) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008424799889326096 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008424799889326096 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.40234375 - response_length/max:1533.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.40234375 - response_length_non_aborted/max:1533.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:631.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013044476509094238 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0533475428819656) - timing_s/agent_loop/generate_sequences/max:np.float64(10.802223604172468) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.998830246309808) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.802223604172468) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:188 - timing_s/agent_loop/slowest/response_length:1533 - timing_s/gen:17.642000133171678 - timing_s/reward:0.06738298386335373 - timing_s/old_log_prob:3.014816017821431 - timing_s/adv:0.5671375151723623 - timing_s/update_actor:21.815314499661326 - timing_s/step:43.19491817057133 - timing_s/stop_profile:0.00011375732719898224 - timing_per_token_ms/update_actor:0.11330100029427882 - timing_per_token_ms/gen:0.15199317773751994 - timing_per_token_ms/adv:0.002945510951695789 - perf/total_num_tokens:192543 - perf/time_per_step:43.19491817057133 - perf/throughput:557.1922813919678 (TaskRunner pid=2255397) Training Progress: 98%|█████████▊| 98/100 [1:30:08<01:30, 45.33s/it] (TaskRunner pid=2255397) step:99 - global_seqlen/min:19692 - global_seqlen/max:25383 - global_seqlen/minmax_diff:5691 - global_seqlen/balanced_min:23698 - global_seqlen/balanced_max:23716 - global_seqlen/mean:23712.375 - actor/entropy:0.4742501676082611 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4091079831123352 - training/rollout_probs_diff_mean:0.004931129980832338 - training/rollout_probs_diff_std:0.010619550943374634 - training/rollout_actor_probs_pearson_corr:0.9992989301681519 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:0.9998912215232849 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.113195119425654e-06 - rollout_corr/rollout_is_max:1.7866209745407104 - rollout_corr/rollout_is_min:0.43442291021347046 - rollout_corr/rollout_is_std:0.03628097102046013 - rollout_corr/rollout_is_eff_sample_size:0.9986851836242739 - rollout_corr/rollout_is_seq_mean:0.9998824596405029 - rollout_corr/rollout_is_seq_std:0.00225405883975327 - rollout_corr/rollout_is_seq_max:1.0066941976547241 - rollout_corr/rollout_is_seq_min:0.9925881028175354 - rollout_corr/rollout_is_seq_max_deviation:0.0074118971824646 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6540217595174909) - rollout_corr/training_log_ppl:np.float64(0.49091246334137395) - rollout_corr/kl:np.float64(0.0009071645049343324) - rollout_corr/k3_kl:np.float64(0.0007826964022683569) - rollout_corr/rollout_ppl:np.float64(1.6525658327154815) - rollout_corr/rollout_log_ppl:np.float64(0.4900052981101908) - rollout_corr/log_ppl_diff:np.float64(0.0009071652311831713) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018514589173719287) - rollout_corr/log_ppl_diff_max:np.float64(0.008251607418060303) - rollout_corr/log_ppl_diff_min:np.float64(-0.005766570568084717) - rollout_corr/ppl_ratio:np.float64(1.0009100551251322) - rollout_corr/chi2_token:np.float64(0.001357766566798091) - rollout_corr/chi2_seq:np.float64(1.0302318546455353) - actor/pg_loss:np.float64(0.004117259988561273) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030947469667808036) - actor/ppo_kl:np.float64(9.188319657127408e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(190.36794193601236) - self_distillation/token_reweight_w_std:np.float64(4398.700665255601) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010327426716685) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09465871116844937) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.2390700727701187) - perf/mfu/actor:np.float64(0.055725058096870825) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.29999542236328) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006170772016048431 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006170772016048431 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:428.63671875 - response_length/max:1227.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:428.63671875 - response_length_non_aborted/max:1227.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.375 - prompt_length/max:836.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001377556473016739 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2593938652426004) - timing_s/agent_loop/generate_sequences/max:np.float64(10.128664135932922) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.985631817675312) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.128664135932922) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:317 - timing_s/agent_loop/slowest/response_length:1227 - timing_s/gen:17.193039959296584 - timing_s/reward:0.06556063517928123 - timing_s/old_log_prob:2.973387535661459 - timing_s/adv:0.5816671065986156 - timing_s/update_actor:21.934669038280845 - timing_s/step:42.834324315190315 - timing_s/stop_profile:0.00016217492520809174 - timing_per_token_ms/update_actor:0.11562880689028854 - timing_per_token_ms/gen:0.15668352570646932 - timing_per_token_ms/adv:0.003066263431007099 - perf/total_num_tokens:189699 - perf/time_per_step:42.834324315190315 - perf/throughput:553.5834959252735 (TaskRunner pid=2255397) Training Progress: 99%|█████████▉| 99/100 [1:30:51<00:44, 44.60s/it] (TaskRunner pid=2255397) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2255397) validation generation end (TaskRunner pid=2255397) [prompt] system (TaskRunner pid=2255397) (TaskRunner pid=2255397) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) ... (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) user (TaskRunner pid=2255397) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2255397) (TaskRunner pid=2255397) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2255397) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2255397) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2255397) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2255397) Please reason step by step. (TaskRunner pid=2255397) assistant (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) [response] (TaskRunner pid=2255397) To determine the correct reactant for the synthesis of the product **"Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1"**, we need to analyze the structure of the product and compare it with the structures of the given options. (TaskRunner pid=2255397) (TaskRunner pid=2255397) The product contains: (TaskRunner pid=2255397) - A **nitrogen-containing ring** (likely an imine or azole ring), (TaskRunner pid=2255397) - A **chlorine atom** attached to a benzene ring, (TaskRunner pid=2255397) - A **trifluoromethyl group** (C(F)(F)F), (TaskRunner pid=2255397) - A **nitrogen atom** connected to a carbon chain containing a **chlorinated benzene ring** and a **fluorinated group**. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Now, let’s examine the options: (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option A:** (TaskRunner pid=2255397) - Contains a **nitrogen-containing ring** (likely a pyrrole or similar), (TaskRunner pid=2255397) - A **chlorinated benzene ring** (with Cl), (TaskRunner pid=2255397) - A **trifluoromethyl group** (C(F)(F)F), (TaskRunner pid=2255397) - A **nitrogen atom** connected to a carbon chain that includes the **chlorinated benzene** and **fluorinated group**. (TaskRunner pid=2255397) (TaskRunner pid=2255397) This matches the structure of the target product very closely, including all the key functional groups and their positions. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option B:** (TaskRunner pid=2255397) - Contains a **chlorine** and **nitrogen**, but the **fluorinated group** is not present in the correct position, and the **nitrogen ring** is not properly structured. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option C:** (TaskRunner pid=2255397) - Contains a **fluorinated group**, but the **chlorine** is attached to an **oxygen**, not directly to the benzene ring. The **fluorinated group** is in a different position than in the product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) **Option D:** (TaskRunner pid=2255397) - Contains a **fluorinated group** and a **chlorinated benzene ring**, but the **nitrogen-containing ring** is not present in the correct structure. (TaskRunner pid=2255397) (TaskRunner pid=2255397) Only **Option A** contains all the required functional groups in the correct positions to serve as the **only correct reactant** for the synthesis of the given product. (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) (TaskRunner pid=2255397) A (TaskRunner pid=2255397) (TaskRunner pid=2255397) [ground_truth] A (TaskRunner pid=2255397) [score] 1.0 (TaskRunner pid=2255397) [acc] 1.0 (TaskRunner pid=2255397) [pred] A (TaskRunner pid=2255397) [incorrect_format] 1 (TaskRunner pid=2255397) [feedback] (TaskRunner pid=2255397) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2255397) step:100 - global_seqlen/min:23741 - global_seqlen/max:31057 - global_seqlen/minmax_diff:7316 - global_seqlen/balanced_min:24739 - global_seqlen/balanced_max:31757 - global_seqlen/mean:25617.625 - actor/entropy:0.4436495304107666 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24008721113204956 - training/rollout_probs_diff_mean:0.004401609301567078 - training/rollout_probs_diff_std:0.009758861735463142 - training/rollout_actor_probs_pearson_corr:0.9993872046470642 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.91796875 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.91796875 - rollout_corr/rollout_is_mean:1.0000593662261963 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7518070936203003 - rollout_corr/rollout_is_min:0.5589723587036133 - rollout_corr/rollout_is_std:0.03378680348396301 - rollout_corr/rollout_is_eff_sample_size:0.9988599913647978 - rollout_corr/rollout_is_seq_mean:1.0000724792480469 - rollout_corr/rollout_is_seq_std:0.0018252491718158126 - rollout_corr/rollout_is_seq_max:1.0057400465011597 - rollout_corr/rollout_is_seq_min:0.9935486912727356 - rollout_corr/rollout_is_seq_max_deviation:0.006451308727264404 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6480820523574948) - rollout_corr/training_log_ppl:np.float64(0.4857633382116546) - rollout_corr/kl:np.float64(0.0007351505751955756) - rollout_corr/k3_kl:np.float64(0.0006855184932135217) - rollout_corr/rollout_ppl:np.float64(1.6468097120523453) - rollout_corr/rollout_log_ppl:np.float64(0.48502818830229444) - rollout_corr/log_ppl_diff:np.float64(0.0007351499093601888) - rollout_corr/log_ppl_abs_diff:np.float64(0.001427958528893214) - rollout_corr/log_ppl_diff_max:np.float64(0.007762610912322998) - rollout_corr/log_ppl_diff_min:np.float64(-0.004005610942840576) - rollout_corr/ppl_ratio:np.float64(1.0007370123639703) - rollout_corr/chi2_token:np.float64(0.0012815699446946383) - rollout_corr/chi2_seq:np.float64(0.31913869245909154) - actor/pg_loss:np.float64(0.003367550321854651) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003017752374034899) - actor/ppo_kl:np.float64(0.00018170115079385596) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.91796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.91796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(1570.6249493323267) - self_distillation/token_reweight_w_std:np.float64(40484.064735737746) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001023317920044) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0961508322507143) - self_distillation/empty_target_batch:np.float64(0.08203125) - actor/grad_norm:np.float64(0.2478880025446415) - perf/mfu/actor:np.float64(0.05716674172282474) - perf/max_memory_allocated_gb:np.float64(134.57307815551758) - perf/max_memory_reserved_gb:np.float64(149.07421875) - perf/cpu_memory_used_gb:np.float64(197.12733459472656) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7065476190476191) - val-aux/sciknoweval/reward/std@16:np.float64(0.14963967195739616) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7672380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12522806043858783) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6460952380952382) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.12793485078945213) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7068333333333333) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14978365629442147) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8158333333333333) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09352166658220648) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5964190476190476) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09674390853009297) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7151904761904762) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1197973344827459) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8520904761904763) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06467821428517558) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5596142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06871965760347537) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7202857142857142) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08852864730736695) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8776952380952381) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0403808452308748) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5314904761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04527152807209723) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7233190476190475) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06487826170016521) - val-aux/sciknoweval/score/mean@16:np.float64(0.7065476190476191) - val-aux/sciknoweval/score/std@16:np.float64(0.14963967195739616) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7672380952380953) - val-aux/sciknoweval/score/best@2/std:np.float64(0.12522806043858783) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6460952380952382) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.12793485078945213) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7068333333333333) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14978365629442147) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8158333333333333) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09352166658220648) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5964190476190476) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09674390853009297) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7151904761904762) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1197973344827459) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8520904761904763) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06467821428517558) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5596142857142857) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06871965760347537) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7202857142857142) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08852864730736695) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8776952380952381) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0403808452308748) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5314904761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04527152807209723) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7233190476190475) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06487826170016521) - val-core/sciknoweval/acc/mean@16:np.float64(0.7065476190476191) - val-aux/sciknoweval/acc/std@16:np.float64(0.14963967195739616) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7672380952380953) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.12522806043858783) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6460952380952382) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.12793485078945213) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7068333333333333) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14978365629442147) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8158333333333333) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09352166658220648) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5964190476190476) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09674390853009297) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7151904761904762) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1197973344827459) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8520904761904763) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06467821428517558) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5596142857142857) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06871965760347537) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7202857142857142) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08852864730736695) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8776952380952381) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0403808452308748) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5314904761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04527152807209723) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7233190476190475) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06487826170016521) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9982142857142857) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.006916041689656103) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.999904761904762) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0016180983939162673) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9966190476190476) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.009226171602153282) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9982380952380953) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.006862312067166463) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9934619047619048) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.012001753208129275) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9993142857142857) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0043365672892029085) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.988647619047619) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.01397823789672266) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9998380952380953) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0019314678388328032) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9817619047619048) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.013705261025670075) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006015850374731164) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026495642960071564 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026495642960071564 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:504.39453125 - response_length/max:8192.0 - response_length/min:195.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:504.39453125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:296.15625 - prompt_length/max:625.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013913773000240326 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3870061337947845) - timing_s/agent_loop/generate_sequences/max:np.float64(60.303343798965216) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.73472642654815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(60.303343798965216) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:326 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:67.0998271573335 - timing_s/reward:0.0683623980730772 - timing_s/old_log_prob:3.3458864334970713 - timing_s/adv:0.5801629964262247 - timing_s/update_actor:23.488009363412857 - timing_s/step:94.66876287944615 - timing_s/testing:143.36135754734278 - timing_s/save_checkpoint:13.737380957230926 - timing_s/stop_profile:0.00018646568059921265 - timing_per_token_ms/update_actor:0.11460864035704353 - timing_per_token_ms/gen:0.5196501619154579 - timing_per_token_ms/adv:0.002830878137738299 - perf/total_num_tokens:204941 - perf/time_per_step:94.66876287944615 - perf/throughput:270.6027228075453 (TaskRunner pid=2255397) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2255397) "np.float64(0.7065476190476191), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2255397) "np.float64(0.14963967195739616), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7672380952380953), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2255397) "np.float64(0.12522806043858783), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2255397) "np.float64(0.6460952380952382), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2255397) "np.float64(0.12793485078945213), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7068333333333333), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2255397) "np.float64(0.14978365629442147), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2255397) "np.float64(0.8158333333333333), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2255397) "np.float64(0.09352166658220648), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2255397) "np.float64(0.5964190476190476), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2255397) "np.float64(0.09674390853009297), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2255397) "np.float64(0.7151904761904762), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2255397) "np.float64(0.1197973344827459), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2255397) "np.float64(0.8520904761904763), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2255397) "np.float64(0.06467821428517558), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2255397) "np.float64(0.5596142857142857), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2255397) "np.float64(0.06871965760347537), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2255397) "np.float64(0.7202857142857142), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2255397) "np.float64(0.08852864730736695), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2255397) "np.float64(0.8776952380952381), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2255397) "np.float64(0.0403808452308748), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2255397) "np.float64(0.5314904761904762), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2255397) "np.float64(0.04527152807209723), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2255397) "np.float64(0.7233190476190475), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2255397) "np.float64(0.06487826170016521), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2255397) "np.float64(0.7065476190476191), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2255397) "np.float64(0.14963967195739616), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7672380952380953), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2255397) "np.float64(0.12522806043858783), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2255397) "np.float64(0.6460952380952382), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2255397) "np.float64(0.12793485078945213), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7068333333333333), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2255397) "np.float64(0.14978365629442147), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2255397) "np.float64(0.8158333333333333), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2255397) "np.float64(0.09352166658220648), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2255397) "np.float64(0.5964190476190476), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2255397) "np.float64(0.09674390853009297), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2255397) "np.float64(0.7151904761904762), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2255397) "np.float64(0.1197973344827459), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2255397) "np.float64(0.8520904761904763), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2255397) "np.float64(0.06467821428517558), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2255397) "np.float64(0.5596142857142857), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2255397) "np.float64(0.06871965760347537), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2255397) "np.float64(0.7202857142857142), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2255397) "np.float64(0.08852864730736695), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2255397) "np.float64(0.8776952380952381), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2255397) "np.float64(0.0403808452308748), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2255397) "np.float64(0.5314904761904762), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2255397) "np.float64(0.04527152807209723), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2255397) "np.float64(0.7233190476190475), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2255397) "np.float64(0.06487826170016521), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2255397) "np.float64(0.7065476190476191), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2255397) "np.float64(0.14963967195739616), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7672380952380953), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2255397) "np.float64(0.12522806043858783), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2255397) "np.float64(0.6460952380952382), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2255397) "np.float64(0.12793485078945213), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2255397) "np.float64(0.7068333333333333), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2255397) "np.float64(0.14978365629442147), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2255397) "np.float64(0.8158333333333333), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2255397) "np.float64(0.09352166658220648), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2255397) "np.float64(0.5964190476190476), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2255397) "np.float64(0.09674390853009297), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2255397) "np.float64(0.7151904761904762), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2255397) "np.float64(0.1197973344827459), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2255397) "np.float64(0.8520904761904763), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2255397) "np.float64(0.06467821428517558), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2255397) "np.float64(0.5596142857142857), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2255397) "np.float64(0.06871965760347537), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2255397) "np.float64(0.7202857142857142), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2255397) "np.float64(0.08852864730736695), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2255397) "np.float64(0.8776952380952381), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2255397) "np.float64(0.0403808452308748), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2255397) "np.float64(0.5314904761904762), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2255397) "np.float64(0.04527152807209723), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2255397) "np.float64(0.7233190476190475), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2255397) 'np.float64(0.06487826170016521), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/mean@16': " (TaskRunner pid=2255397) 'np.float64(0.9982142857142857), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2255397) 'np.float64(0.006916041689656103), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2255397) 'np.float64(0.999904761904762), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2255397) 'np.float64(0.0016180983939162673), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2255397) 'np.float64(0.9966190476190476), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2255397) 'np.float64(0.009226171602153282), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': " (TaskRunner pid=2255397) 'np.float64(0.9982380952380953), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2255397) 'np.float64(0.006862312067166463), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2255397) 'np.float64(0.9934619047619048), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2255397) 'np.float64(0.012001753208129275), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2255397) 'np.float64(0.9993142857142857), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2255397) 'np.float64(0.0043365672892029085), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2255397) 'np.float64(0.988647619047619), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2255397) 'np.float64(0.01397823789672266), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2255397) 'np.float64(0.9998380952380953), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2255397) 'np.float64(0.0019314678388328032), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2255397) 'np.float64(0.9817619047619048), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2255397) 'np.float64(0.013705261025670075), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2255397) 'np.float64(0.9999714285714286), ' (TaskRunner pid=2255397) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2255397) "np.float64(0.0006015850374731164), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2255397) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2255397) 'np.float64(2.0)}') (TaskRunner pid=2255397) Training Progress: 100%|██████████| 100/100 [1:35:02<00:00, 106.78s/it] Training Progress: 100%|██████████| 100/100 [1:35:02<00:00, 57.03s/it] (TaskRunner pid=2255397) Traceback (most recent call last): (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/workers/default_worker.py", line 322, in (TaskRunner pid=2255397) worker.main_loop() (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py", line 1031, in main_loop (TaskRunner pid=2255397) self.core_worker.run_task_loop() (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/signal_handling.py", line 73, in handler (TaskRunner pid=2255397) _error_if_any_worker_fails() (TaskRunner pid=2255397) RuntimeError: DataLoader worker (pid 2270092) is killed by signal: Killed. (TaskRunner pid=2255397) Traceback (most recent call last): (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/workers/default_worker.py", line 322, in (TaskRunner pid=2255397) worker.main_loop() (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py", line 1031, in main_loop (TaskRunner pid=2255397) self.core_worker.run_task_loop() (TaskRunner pid=2255397) Exception ignored in: (TaskRunner pid=2255397) Traceback (most recent call last): (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/verl/utils/tracking.py", line 169, in __del__ (TaskRunner pid=2255397) self.logger["wandb"].finish(exit_code=0) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 4097, in finish (TaskRunner pid=2255397) wandb.run.finish(exit_code=exit_code, quiet=quiet) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2255397) return func(self, *args, **kwargs) (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 445, in wrapper (TaskRunner pid=2255397) return func(self, *args, **kwargs) (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2275, in finish (TaskRunner pid=2255397) return self._finish(exit_code) (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2255397) return func(self, *args, **kwargs) (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2288, in _finish (TaskRunner pid=2255397) with telemetry.context(run=self) as tel: (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/telemetry.py", line 42, in __exit__ (TaskRunner pid=2255397) self._run._telemetry_callback(self._obj) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 752, in _telemetry_callback (TaskRunner pid=2255397) self._telemetry_flush() (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 765, in _telemetry_flush (TaskRunner pid=2255397) self._backend.interface._publish_telemetry(self._telemetry_obj) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_shared.py", line 103, in _publish_telemetry (TaskRunner pid=2255397) self._publish(rec) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_sock.py", line 46, in _publish (TaskRunner pid=2255397) self._asyncer.run(lambda: self._client.publish(request)) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2255397) return future.result() (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2255397) return self.__get_result() (TaskRunner pid=2255397) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2255397) raise self._exception (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2255397) return await fn() (TaskRunner pid=2255397) ^^^^^^^^^^ (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 38, in publish (TaskRunner pid=2255397) await self._send_server_request(request) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 59, in _send_server_request (TaskRunner pid=2255397) self._writer.write(header) (TaskRunner pid=2255397) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 346, in write (TaskRunner pid=2255397) self._transport.write(data) (TaskRunner pid=2255397) File "uvloop/handles/stream.pyx", line 675, in uvloop.loop.UVStream.write (TaskRunner pid=2255397) File "uvloop/handles/handle.pyx", line 159, in uvloop.loop.UVHandle._ensure_alive (TaskRunner pid=2255397) RuntimeError: unable to perform operation on ; the handler is closed main_ppo exit code: 0 [2026-07-03 03:18:43] Finished Qwen/Qwen3-8B chemistry rlsd_tr. [2026-07-03 03:18:43] Starting Qwen/Qwen3-8B physics grpo. Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_grpo_Qwen3_8B 2026-07-03 03:18:44,953 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_grpo_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 03:18:54,758 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2281424) TaskRunner hostname: mole-workspace-rw, PID: 2281424 (TaskRunner pid=2281424) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2281424) 'calculate_entropy': False, (TaskRunner pid=2281424) 'calculate_sum_pi_squared': False, (TaskRunner pid=2281424) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2281424) 'async_save': False, (TaskRunner pid=2281424) 'load_contents': ['model', (TaskRunner pid=2281424) 'optimizer', (TaskRunner pid=2281424) 'extra'], (TaskRunner pid=2281424) 'save_contents': ['model', (TaskRunner pid=2281424) 'optimizer', (TaskRunner pid=2281424) 'extra']}, (TaskRunner pid=2281424) 'clip_ratio': 0.2, (TaskRunner pid=2281424) 'clip_ratio_c': 3.0, (TaskRunner pid=2281424) 'clip_ratio_high': 0.28, (TaskRunner pid=2281424) 'clip_ratio_low': 0.2, (TaskRunner pid=2281424) 'data_loader_seed': 42, (TaskRunner pid=2281424) 'entropy_checkpointing': False, (TaskRunner pid=2281424) 'entropy_coeff': 0, (TaskRunner pid=2281424) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2281424) 'freeze_vision_tower': False, (TaskRunner pid=2281424) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2281424) 'dtype': 'bfloat16', (TaskRunner pid=2281424) 'entropy_checkpointing': False, (TaskRunner pid=2281424) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2281424) 'forward_only': False, (TaskRunner pid=2281424) 'forward_prefetch': False, (TaskRunner pid=2281424) 'fsdp_size': -1, (TaskRunner pid=2281424) 'full_determinism': False, (TaskRunner pid=2281424) 'model_dtype': 'fp32', (TaskRunner pid=2281424) 'offload_policy': False, (TaskRunner pid=2281424) 'optimizer_offload': False, (TaskRunner pid=2281424) 'param_offload': False, (TaskRunner pid=2281424) 'reshard_after_forward': True, (TaskRunner pid=2281424) 'seed': 42, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_orig_params': False, (TaskRunner pid=2281424) 'use_torch_compile': True, (TaskRunner pid=2281424) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2281424) 'grad_clip': 1.0, (TaskRunner pid=2281424) 'kl_loss_coef': 0.0, (TaskRunner pid=2281424) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2281424) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2281424) 'loss_scale_factor': None, (TaskRunner pid=2281424) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2281424) 'betas': [0.9, 0.999], (TaskRunner pid=2281424) 'clip_grad': 1.0, (TaskRunner pid=2281424) 'lr': 1e-06, (TaskRunner pid=2281424) 'lr_scheduler_type': 'constant', (TaskRunner pid=2281424) 'lr_warmup_steps': 10, (TaskRunner pid=2281424) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2281424) 'min_lr_ratio': 0.0, (TaskRunner pid=2281424) 'num_cycles': 0.5, (TaskRunner pid=2281424) 'optimizer': 'AdamW', (TaskRunner pid=2281424) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2281424) 'override_optimizer_config': None, (TaskRunner pid=2281424) 'total_training_steps': -1, (TaskRunner pid=2281424) 'warmup_style': None, (TaskRunner pid=2281424) 'weight_decay': 0.01}, (TaskRunner pid=2281424) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2281424) 'clip_cov_lb': 1.0, (TaskRunner pid=2281424) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2281424) 'clip_cov_ub': 5.0, (TaskRunner pid=2281424) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2281424) 'loss_mode': 'vanilla', (TaskRunner pid=2281424) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2281424) 'ppo_epochs': 1, (TaskRunner pid=2281424) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2281424) 'ppo_micro_batch_size': None, (TaskRunner pid=2281424) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2281424) 'ppo_mini_batch_size': 8, (TaskRunner pid=2281424) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'all_ranks': False, (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'ranks': [], (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'tool': None, (TaskRunner pid=2281424) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2281424) 'analysis': True, (TaskRunner pid=2281424) 'contents': [], (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'level': 'level0'}, (TaskRunner pid=2281424) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'discrete': False}, (TaskRunner pid=2281424) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2281424) 'step_end': None, (TaskRunner pid=2281424) 'step_start': 0}, (TaskRunner pid=2281424) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2281424) 'rollout_n': 8, (TaskRunner pid=2281424) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2281424) 'mode': 'disabled', (TaskRunner pid=2281424) 'record_file': None, (TaskRunner pid=2281424) 'replay_file': None}, (TaskRunner pid=2281424) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2281424) 'alpha': 0.5, (TaskRunner pid=2281424) 'distillation_add_tail': True, (TaskRunner pid=2281424) 'distillation_topk': 100, (TaskRunner pid=2281424) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2281424) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2281424) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'loss_weight': 0.0, (TaskRunner pid=2281424) 'mask': 'all'}, (TaskRunner pid=2281424) 'feedback_template': '\n' (TaskRunner pid=2281424) 'The ' (TaskRunner pid=2281424) 'following ' (TaskRunner pid=2281424) 'is ' (TaskRunner pid=2281424) 'feedback ' (TaskRunner pid=2281424) 'from ' (TaskRunner pid=2281424) 'your ' (TaskRunner pid=2281424) 'unsuccessful ' (TaskRunner pid=2281424) 'earlier ' (TaskRunner pid=2281424) 'attempt:\n' (TaskRunner pid=2281424) '\n' (TaskRunner pid=2281424) '{feedback_raw}', (TaskRunner pid=2281424) 'full_logit_distillation': True, (TaskRunner pid=2281424) 'include_environment_feedback': True, (TaskRunner pid=2281424) 'is_clip': 2, (TaskRunner pid=2281424) 'max_reprompt_len': 22528, (TaskRunner pid=2281424) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2281424) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2281424) '\n' (TaskRunner pid=2281424) 'Correctly ' (TaskRunner pid=2281424) 'solve ' (TaskRunner pid=2281424) 'the ' (TaskRunner pid=2281424) 'original ' (TaskRunner pid=2281424) 'question.', (TaskRunner pid=2281424) 'reprompt_truncation': 'right', (TaskRunner pid=2281424) 'solution_template': '\n' (TaskRunner pid=2281424) 'Correct ' (TaskRunner pid=2281424) 'solution:\n' (TaskRunner pid=2281424) '\n' (TaskRunner pid=2281424) '{successful_previous_attempt}', (TaskRunner pid=2281424) 'srpo_dynamic_weighting': False, (TaskRunner pid=2281424) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2281424) 'success_reward_threshold': 0.5, (TaskRunner pid=2281424) 'teacher_regularization': 'ema', (TaskRunner pid=2281424) 'teacher_update_rate': 0.0, (TaskRunner pid=2281424) 'token_reweight_decay_steps': None, (TaskRunner pid=2281424) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2281424) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2281424) 'shuffle': False, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2281424) 'tau_neg': 1.05, (TaskRunner pid=2281424) 'tau_pos': 1.0, (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_dynamic_bsz': False, (TaskRunner pid=2281424) 'use_fused_kernels': False, (TaskRunner pid=2281424) 'use_kl_loss': False, (TaskRunner pid=2281424) 'use_prefix_grouper': False, (TaskRunner pid=2281424) 'use_remove_padding': True, (TaskRunner pid=2281424) 'use_torch_compile': True}, (TaskRunner pid=2281424) 'hybrid_engine': True, (TaskRunner pid=2281424) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2281424) 'custom_chat_template': None, (TaskRunner pid=2281424) 'enable_activation_offload': False, (TaskRunner pid=2281424) 'enable_gradient_checkpointing': True, (TaskRunner pid=2281424) 'exclude_modules': None, (TaskRunner pid=2281424) 'external_lib': None, (TaskRunner pid=2281424) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2281424) 'hf_config_path': None, (TaskRunner pid=2281424) 'lora_adapter_path': None, (TaskRunner pid=2281424) 'lora_alpha': 16, (TaskRunner pid=2281424) 'lora_rank': 0, (TaskRunner pid=2281424) 'override_config': {}, (TaskRunner pid=2281424) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2281424) 'target_modules': 'all-linear', (TaskRunner pid=2281424) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2281424) 'num_shards': 4}, (TaskRunner pid=2281424) 'tokenizer_path': None, (TaskRunner pid=2281424) 'trust_remote_code': True, (TaskRunner pid=2281424) 'use_fused_kernels': False, (TaskRunner pid=2281424) 'use_liger': False, (TaskRunner pid=2281424) 'use_remove_padding': True, (TaskRunner pid=2281424) 'use_shm': False}, (TaskRunner pid=2281424) 'nccl_timeout': 600, (TaskRunner pid=2281424) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2281424) 'entropy_checkpointing': False, (TaskRunner pid=2281424) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2281424) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2281424) 'dtype': 'bfloat16', (TaskRunner pid=2281424) 'entropy_checkpointing': False, (TaskRunner pid=2281424) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2281424) 'forward_only': True, (TaskRunner pid=2281424) 'forward_prefetch': False, (TaskRunner pid=2281424) 'fsdp_size': -1, (TaskRunner pid=2281424) 'full_determinism': False, (TaskRunner pid=2281424) 'model_dtype': 'fp32', (TaskRunner pid=2281424) 'offload_policy': False, (TaskRunner pid=2281424) 'optimizer_offload': False, (TaskRunner pid=2281424) 'param_offload': False, (TaskRunner pid=2281424) 'reshard_after_forward': True, (TaskRunner pid=2281424) 'seed': 42, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_orig_params': False, (TaskRunner pid=2281424) 'use_torch_compile': True, (TaskRunner pid=2281424) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2281424) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2281424) 'log_prob_micro_batch_size': None, (TaskRunner pid=2281424) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2281424) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2281424) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'all_ranks': False, (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'ranks': [], (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'tool': None, (TaskRunner pid=2281424) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2281424) 'analysis': True, (TaskRunner pid=2281424) 'contents': [], (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'level': 'level0'}, (TaskRunner pid=2281424) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'discrete': False}, (TaskRunner pid=2281424) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2281424) 'step_end': None, (TaskRunner pid=2281424) 'step_start': 0}, (TaskRunner pid=2281424) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2281424) 'rollout_n': 8, (TaskRunner pid=2281424) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2281424) 'mode': 'disabled', (TaskRunner pid=2281424) 'record_file': None, (TaskRunner pid=2281424) 'replay_file': None}, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_torch_compile': True}, (TaskRunner pid=2281424) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2281424) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2281424) 'agent_loop_config_path': None, (TaskRunner pid=2281424) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2281424) 'name': None, (TaskRunner pid=2281424) 'path': None}, (TaskRunner pid=2281424) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2281424) 'num_workers': 8}, (TaskRunner pid=2281424) 'calculate_log_probs': True, (TaskRunner pid=2281424) 'cudagraph_capture_sizes': None, (TaskRunner pid=2281424) 'data_parallel_size': 1, (TaskRunner pid=2281424) 'disable_log_stats': True, (TaskRunner pid=2281424) 'do_sample': True, (TaskRunner pid=2281424) 'dtype': 'bfloat16', (TaskRunner pid=2281424) 'enable_chunked_prefill': True, (TaskRunner pid=2281424) 'enable_prefix_caching': True, (TaskRunner pid=2281424) 'enable_rollout_routing_replay': False, (TaskRunner pid=2281424) 'enforce_eager': False, (TaskRunner pid=2281424) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2281424) 'expert_parallel_size': 1, (TaskRunner pid=2281424) 'free_cache_engine': True, (TaskRunner pid=2281424) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2281424) 'ignore_eos': False, (TaskRunner pid=2281424) 'layered_summon': False, (TaskRunner pid=2281424) 'load_format': 'dummy', (TaskRunner pid=2281424) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2281424) 'log_prob_micro_batch_size': None, (TaskRunner pid=2281424) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2281424) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2281424) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2281424) 'max_model_len': 10240, (TaskRunner pid=2281424) 'max_num_batched_tokens': 10240, (TaskRunner pid=2281424) 'max_num_seqs': 1024, (TaskRunner pid=2281424) 'mode': 'async', (TaskRunner pid=2281424) 'multi_stage_wake_up': False, (TaskRunner pid=2281424) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'format': 'hermes', (TaskRunner pid=2281424) 'interaction_config_path': None, (TaskRunner pid=2281424) 'max_assistant_turns': None, (TaskRunner pid=2281424) 'max_parallel_calls': 1, (TaskRunner pid=2281424) 'max_tool_response_length': 256, (TaskRunner pid=2281424) 'max_user_turns': None, (TaskRunner pid=2281424) 'num_repeat_rollouts': None, (TaskRunner pid=2281424) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2281424) 'tool_config_path': None, (TaskRunner pid=2281424) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2281424) 'use_inference_chat_template': False}, (TaskRunner pid=2281424) 'n': 8, (TaskRunner pid=2281424) 'name': 'vllm', (TaskRunner pid=2281424) 'over_sample_rate': 0, (TaskRunner pid=2281424) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2281424) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'all_ranks': False, (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'ranks': [], (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'tool': None, (TaskRunner pid=2281424) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2281424) 'analysis': True, (TaskRunner pid=2281424) 'contents': [], (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'level': 'level0'}, (TaskRunner pid=2281424) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'discrete': False}, (TaskRunner pid=2281424) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2281424) 'step_end': None, (TaskRunner pid=2281424) 'step_start': 0}, (TaskRunner pid=2281424) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2281424) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2281424) 'port': 9090, (TaskRunner pid=2281424) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2281424) 'prompt_length': 2048, (TaskRunner pid=2281424) 'quantization': None, (TaskRunner pid=2281424) 'quantization_config_file': None, (TaskRunner pid=2281424) 'response_length': 8192, (TaskRunner pid=2281424) 'scheduling_policy': 'fcfs', (TaskRunner pid=2281424) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2281424) 'skip_rollout': False, (TaskRunner pid=2281424) 'skip_tokenizer_init': True, (TaskRunner pid=2281424) 'temperature': 1.0, (TaskRunner pid=2281424) 'tensor_model_parallel_size': 2, (TaskRunner pid=2281424) 'top_k': -1, (TaskRunner pid=2281424) 'top_p': 1.0, (TaskRunner pid=2281424) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2281424) 'backend': None, (TaskRunner pid=2281424) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2281424) 'token2text': False}, (TaskRunner pid=2281424) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2281424) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2281424) 'do_sample': True, (TaskRunner pid=2281424) 'n': 16, (TaskRunner pid=2281424) 'temperature': 0.6, (TaskRunner pid=2281424) 'top_k': -1, (TaskRunner pid=2281424) 'top_p': 0.95}}}, (TaskRunner pid=2281424) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2281424) 'adv_estimator': 'grpo', (TaskRunner pid=2281424) 'gamma': 1.0, (TaskRunner pid=2281424) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2281424) 'horizon': 10000, (TaskRunner pid=2281424) 'kl_coef': 0.001, (TaskRunner pid=2281424) 'target_kl': 0.1, (TaskRunner pid=2281424) 'type': 'fixed'}, (TaskRunner pid=2281424) 'kl_penalty': 'kl', (TaskRunner pid=2281424) 'lam': 1.0, (TaskRunner pid=2281424) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2281424) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2281424) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2281424) 'loss_type': 'ppo_clip', (TaskRunner pid=2281424) 'rollout_is': 'token', (TaskRunner pid=2281424) 'rollout_is_batch_normalize': False, (TaskRunner pid=2281424) 'rollout_is_threshold': 2.0, (TaskRunner pid=2281424) 'rollout_rs': None, (TaskRunner pid=2281424) 'rollout_rs_threshold': None}, (TaskRunner pid=2281424) 'use_kl_in_reward': False, (TaskRunner pid=2281424) 'use_pf_ppo': False}, (TaskRunner pid=2281424) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2281424) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2281424) 'async_save': False, (TaskRunner pid=2281424) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2281424) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2281424) 'cliprange_value': 0.5, (TaskRunner pid=2281424) 'data_loader_seed': 42, (TaskRunner pid=2281424) 'enable': None, (TaskRunner pid=2281424) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2281424) 'forward_micro_batch_size': None, (TaskRunner pid=2281424) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2281424) 'grad_clip': 1.0, (TaskRunner pid=2281424) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2281424) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2281424) 'enable_activation_offload': False, (TaskRunner pid=2281424) 'enable_gradient_checkpointing': True, (TaskRunner pid=2281424) 'external_lib': None, (TaskRunner pid=2281424) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2281424) 'dtype': 'bfloat16', (TaskRunner pid=2281424) 'entropy_checkpointing': False, (TaskRunner pid=2281424) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2281424) 'forward_only': False, (TaskRunner pid=2281424) 'forward_prefetch': False, (TaskRunner pid=2281424) 'fsdp_size': -1, (TaskRunner pid=2281424) 'full_determinism': False, (TaskRunner pid=2281424) 'model_dtype': 'fp32', (TaskRunner pid=2281424) 'offload_policy': False, (TaskRunner pid=2281424) 'optimizer_offload': False, (TaskRunner pid=2281424) 'param_offload': False, (TaskRunner pid=2281424) 'reshard_after_forward': True, (TaskRunner pid=2281424) 'seed': 42, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_orig_params': False, (TaskRunner pid=2281424) 'use_torch_compile': True, (TaskRunner pid=2281424) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2281424) 'lora_alpha': 16, (TaskRunner pid=2281424) 'lora_rank': 0, (TaskRunner pid=2281424) 'override_config': {}, (TaskRunner pid=2281424) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2281424) 'target_modules': 'all-linear', (TaskRunner pid=2281424) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2281424) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2281424) 'trust_remote_code': True, (TaskRunner pid=2281424) 'use_remove_padding': False, (TaskRunner pid=2281424) 'use_shm': False}, (TaskRunner pid=2281424) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2281424) 'betas': [0.9, 0.999], (TaskRunner pid=2281424) 'clip_grad': 1.0, (TaskRunner pid=2281424) 'lr': 1e-05, (TaskRunner pid=2281424) 'lr_scheduler_type': 'constant', (TaskRunner pid=2281424) 'lr_warmup_steps': -1, (TaskRunner pid=2281424) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2281424) 'min_lr_ratio': 0.0, (TaskRunner pid=2281424) 'num_cycles': 0.5, (TaskRunner pid=2281424) 'optimizer': 'AdamW', (TaskRunner pid=2281424) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2281424) 'override_optimizer_config': None, (TaskRunner pid=2281424) 'total_training_steps': -1, (TaskRunner pid=2281424) 'warmup_style': None, (TaskRunner pid=2281424) 'weight_decay': 0.01}, (TaskRunner pid=2281424) 'ppo_epochs': 1, (TaskRunner pid=2281424) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2281424) 'ppo_micro_batch_size': None, (TaskRunner pid=2281424) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2281424) 'ppo_mini_batch_size': 8, (TaskRunner pid=2281424) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'all_ranks': False, (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'ranks': [], (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'tool': None, (TaskRunner pid=2281424) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2281424) 'analysis': True, (TaskRunner pid=2281424) 'contents': [], (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'level': 'level0'}, (TaskRunner pid=2281424) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'discrete': False}, (TaskRunner pid=2281424) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2281424) 'step_end': None, (TaskRunner pid=2281424) 'step_start': 0}, (TaskRunner pid=2281424) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2281424) 'rollout_n': 8, (TaskRunner pid=2281424) 'shuffle': False, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_dynamic_bsz': False}, (TaskRunner pid=2281424) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2281424) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2281424) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2281424) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2281424) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2281424) 'dataloader_num_workers': 8, (TaskRunner pid=2281424) 'filter_overlong_prompts': True, (TaskRunner pid=2281424) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2281424) 'image_key': 'images', (TaskRunner pid=2281424) 'image_patch_size': 14, (TaskRunner pid=2281424) 'max_prompt_length': 2048, (TaskRunner pid=2281424) 'max_response_length': 8192, (TaskRunner pid=2281424) 'prompt_key': 'prompt', (TaskRunner pid=2281424) 'return_full_prompt': False, (TaskRunner pid=2281424) 'return_multi_modal_inputs': True, (TaskRunner pid=2281424) 'return_raw_chat': True, (TaskRunner pid=2281424) 'return_raw_input_ids': False, (TaskRunner pid=2281424) 'reward_fn_key': 'data_source', (TaskRunner pid=2281424) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2281424) 'seed': None, (TaskRunner pid=2281424) 'shuffle': True, (TaskRunner pid=2281424) 'tokenizer': None, (TaskRunner pid=2281424) 'tool_config_path': None, (TaskRunner pid=2281424) 'train_batch_size': 32, (TaskRunner pid=2281424) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2281424) 'train_max_samples': 3200, (TaskRunner pid=2281424) 'truncation': 'error', (TaskRunner pid=2281424) 'trust_remote_code': True, (TaskRunner pid=2281424) 'use_shm': False, (TaskRunner pid=2281424) 'val_batch_size': None, (TaskRunner pid=2281424) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2281424) 'val_max_samples': -1, (TaskRunner pid=2281424) 'validation_shuffle': False, (TaskRunner pid=2281424) 'video_key': 'videos'}, (TaskRunner pid=2281424) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2281424) 'cuda-memory-usage': 'true', (TaskRunner pid=2281424) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2281424) 'capture-range-end': None, (TaskRunner pid=2281424) 'cuda-graph-trace': 'graph', (TaskRunner pid=2281424) 'cuda-memory-usage': 'true', (TaskRunner pid=2281424) 'kill': 'none', (TaskRunner pid=2281424) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2281424) 'torch_memory': {'context': 'all', (TaskRunner pid=2281424) 'kw_args': {}, (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'stacks': 'all', (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2281424) 'profile_continuous_steps': False, (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'steps': None, (TaskRunner pid=2281424) 'tool': None}, (TaskRunner pid=2281424) 'max_model_len': 10240, (TaskRunner pid=2281424) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_8B', (TaskRunner pid=2281424) 'include_dashboard': False, (TaskRunner pid=2281424) 'num_cpus': None, (TaskRunner pid=2281424) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2281424) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2281424) 'USER': 'root'}}}, (TaskRunner pid=2281424) 'timeline_json_file': None}, (TaskRunner pid=2281424) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2281424) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2281424) 'name': 'custom_reward_manager', (TaskRunner pid=2281424) 'path': None}, (TaskRunner pid=2281424) 'name': 'naive', (TaskRunner pid=2281424) 'source': 'register'}, (TaskRunner pid=2281424) 'reward_model': {'enable': False, (TaskRunner pid=2281424) 'enable_resource_pool': False, (TaskRunner pid=2281424) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2281424) 'launch_reward_fn_async': False, (TaskRunner pid=2281424) 'max_length': None, (TaskRunner pid=2281424) 'micro_batch_size': None, (TaskRunner pid=2281424) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2281424) 'model': {'external_lib': None, (TaskRunner pid=2281424) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2281424) 'forward_prefetch': False, (TaskRunner pid=2281424) 'fsdp_size': -1, (TaskRunner pid=2281424) 'param_offload': False, (TaskRunner pid=2281424) 'reshard_after_forward': True, (TaskRunner pid=2281424) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2281424) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2281424) 'override_config': {}, (TaskRunner pid=2281424) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2281424) 'trust_remote_code': False, (TaskRunner pid=2281424) 'use_fused_kernels': False, (TaskRunner pid=2281424) 'use_remove_padding': False, (TaskRunner pid=2281424) 'use_shm': False}, (TaskRunner pid=2281424) 'n_gpus_per_node': 8, (TaskRunner pid=2281424) 'nnodes': 0, (TaskRunner pid=2281424) 'num_workers': 1, (TaskRunner pid=2281424) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2281424) 'all_ranks': False, (TaskRunner pid=2281424) 'enable': False, (TaskRunner pid=2281424) 'ranks': [], (TaskRunner pid=2281424) 'save_path': 'outputs/profile', (TaskRunner pid=2281424) 'tool': None, (TaskRunner pid=2281424) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2281424) 'analysis': True, (TaskRunner pid=2281424) 'contents': [], (TaskRunner pid=2281424) 'discrete': False, (TaskRunner pid=2281424) 'level': 'level0'}, (TaskRunner pid=2281424) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2281424) 'discrete': False}, (TaskRunner pid=2281424) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2281424) 'step_end': None, (TaskRunner pid=2281424) 'step_start': 0}, (TaskRunner pid=2281424) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2281424) 'stack_depth': 32, (TaskRunner pid=2281424) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2281424) 'reward_loop_class_name': None, (TaskRunner pid=2281424) 'reward_loop_module_path': None, (TaskRunner pid=2281424) 'reward_loop_source': 'register', (TaskRunner pid=2281424) 'reward_manager': 'naive', (TaskRunner pid=2281424) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2281424) 'cudagraph_capture_sizes': None, (TaskRunner pid=2281424) 'data_parallel_size': 1, (TaskRunner pid=2281424) 'disable_log_stats': True, (TaskRunner pid=2281424) 'dtype': 'bfloat16', (TaskRunner pid=2281424) 'enable_chunked_prefill': True, (TaskRunner pid=2281424) 'enable_prefix_caching': True, (TaskRunner pid=2281424) 'enforce_eager': True, (TaskRunner pid=2281424) 'engine_kwargs': {}, (TaskRunner pid=2281424) 'expert_parallel_size': 1, (TaskRunner pid=2281424) 'free_cache_engine': True, (TaskRunner pid=2281424) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2281424) 'limit_images': None, (TaskRunner pid=2281424) 'load_format': 'auto', (TaskRunner pid=2281424) 'max_model_len': None, (TaskRunner pid=2281424) 'max_num_batched_tokens': 8192, (TaskRunner pid=2281424) 'max_num_seqs': 1024, (TaskRunner pid=2281424) 'name': '???', (TaskRunner pid=2281424) 'prompt_length': 2048, (TaskRunner pid=2281424) 'response_length': 2048, (TaskRunner pid=2281424) 'skip_tokenizer_init': False, (TaskRunner pid=2281424) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2281424) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2281424) 'memory_limit_mb': 1024, (TaskRunner pid=2281424) 'url': None}, (TaskRunner pid=2281424) 'strategy': 'fsdp', (TaskRunner pid=2281424) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2281424) 'use_dynamic_bsz': False, (TaskRunner pid=2281424) 'use_reward_loop': False}, (TaskRunner pid=2281424) 'trainer': {'balance_batch': True, (TaskRunner pid=2281424) 'critic_warmup': 0, (TaskRunner pid=2281424) 'default_hdfs_dir': None, (TaskRunner pid=2281424) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2281424) 'del_local_ckpt_after_load': False, (TaskRunner pid=2281424) 'device': 'cuda', (TaskRunner pid=2281424) 'esi_redundant_time': 0, (TaskRunner pid=2281424) 'experiment_name': 'qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2281424) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2281424) 'log_val_generations': 0, (TaskRunner pid=2281424) 'logger': ['console', 'wandb'], (TaskRunner pid=2281424) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2281424) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2281424) 'n_gpus_per_node': 8, (TaskRunner pid=2281424) 'nnodes': 1, (TaskRunner pid=2281424) 'project_name': 'SDPO-root', (TaskRunner pid=2281424) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2281424) 'resume_from_path': None, (TaskRunner pid=2281424) 'resume_mode': 'auto', (TaskRunner pid=2281424) 'rollout_data_dir': None, (TaskRunner pid=2281424) 'save_freq': 10, (TaskRunner pid=2281424) 'test_freq': 10, (TaskRunner pid=2281424) 'total_epochs': 30, (TaskRunner pid=2281424) 'total_training_steps': 100, (TaskRunner pid=2281424) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2281424) 'val_before_train': False, (TaskRunner pid=2281424) 'val_only': False, (TaskRunner pid=2281424) 'validation_data_dir': None}, (TaskRunner pid=2281424) 'transfer_queue': {'enable': False}, (TaskRunner pid=2281424) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2281424) 'dir': '/users/root/SDPO', (TaskRunner pid=2281424) 'log_dir': '/users/root/output', (TaskRunner pid=2281424) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2281424) use_critic=need_critic(config), (TaskRunner pid=2281424) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2281424) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2281424) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2281424) Using dataset class: RLHFDataset (TaskRunner pid=2281424) dataset len: 720 (TaskRunner pid=2281424) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2281424) WARNING:2026-07-03 03:19:03,686:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2281424) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2281424) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2281424) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2281424) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2281424) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2281424) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2281424) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 122.84 examples/s] (TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2281424) self.use_critic = need_critic(self.config) (WorkerDict pid=2281814) [W703 03:19:12.382603555 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53237 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2281810) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2281810) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2281810) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2281817) [W703 03:19:13.439209755 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53237 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2281810) Model config after override: Qwen3Config { (WorkerDict pid=2281810) "architectures": [ (WorkerDict pid=2281810) "Qwen3ForCausalLM" (WorkerDict pid=2281810) ], (WorkerDict pid=2281810) "attention_bias": false, (WorkerDict pid=2281810) "attention_dropout": 0.0, (WorkerDict pid=2281810) "dtype": "bfloat16", (WorkerDict pid=2281810) "eos_token_id": 151645, (WorkerDict pid=2281810) "head_dim": 128, (WorkerDict pid=2281810) "hidden_act": "silu", (WorkerDict pid=2281810) "hidden_size": 4096, (WorkerDict pid=2281810) "initializer_range": 0.02, (WorkerDict pid=2281810) "intermediate_size": 12288, (WorkerDict pid=2281810) "layer_types": [ (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention", (WorkerDict pid=2281810) "full_attention" (WorkerDict pid=2281810) ], (WorkerDict pid=2281810) "max_position_embeddings": 40960, (WorkerDict pid=2281810) "max_window_layers": 36, (WorkerDict pid=2281810) "model_type": "qwen3", (WorkerDict pid=2281810) "num_attention_heads": 32, (WorkerDict pid=2281810) "num_hidden_layers": 36, (WorkerDict pid=2281810) "num_key_value_heads": 8, (WorkerDict pid=2281810) "pad_token_id": 151643, (WorkerDict pid=2281810) "rms_norm_eps": 1e-06, (WorkerDict pid=2281810) "rope_scaling": null, (WorkerDict pid=2281810) "rope_theta": 1000000, (WorkerDict pid=2281810) "sliding_window": null, (WorkerDict pid=2281810) "tie_word_embeddings": false, (WorkerDict pid=2281810) "transformers_version": "4.57.1", (WorkerDict pid=2281810) "use_cache": true, (WorkerDict pid=2281810) "use_sliding_window": false, (WorkerDict pid=2281810) "vocab_size": 151936 (WorkerDict pid=2281810) } (WorkerDict pid=2281810) (WorkerDict pid=2281810) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2281810) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2281812) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2281810) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2281811) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2281811) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2281817) Loading checkpoint shards: 80%|████████ | 4/5 [00:11<00:02, 2.89s/it] [repeated 15x across cluster] (WorkerDict pid=2281817) Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00, 2.20s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00, 2.58s/it] [repeated 7x across cluster] (WorkerDict pid=2281810) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2281810) Actor use_remove_padding=True (WorkerDict pid=2281810) Actor use_fused_kernels=False (WorkerDict pid=2281810) Actor use_prefix_grouper=False (WorkerDict pid=2281817) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2281817) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2281814) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2281814) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2281811) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2281810) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2281810) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2281816) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2281816) warnings.warn( (WorkerDict pid=2281810) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2281810) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2281424) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2281424) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2281817) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2281817) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2282697) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2282697) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2282700) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2282700) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2282697) ['serve', (vLLMHttpServer pid=2282697) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2282697) '--dtype', (vLLMHttpServer pid=2282697) 'bfloat16', (vLLMHttpServer pid=2282697) '--load_format', (vLLMHttpServer pid=2282697) 'dummy', (vLLMHttpServer pid=2282697) '--trust_remote_code', (vLLMHttpServer pid=2282697) '--max_model_len', (vLLMHttpServer pid=2282697) '40960', (vLLMHttpServer pid=2282697) '--max_num_seqs', (vLLMHttpServer pid=2282697) '1024', (vLLMHttpServer pid=2282697) '--enable_chunked_prefill', (vLLMHttpServer pid=2282697) '--max_num_batched_tokens', (vLLMHttpServer pid=2282697) '10240', (vLLMHttpServer pid=2282697) '--enable_prefix_caching', (vLLMHttpServer pid=2282697) '--enable_sleep_mode', (vLLMHttpServer pid=2282697) '--logprobs_mode', (vLLMHttpServer pid=2282697) 'processed_logprobs', (vLLMHttpServer pid=2282697) '--disable_custom_all_reduce', (vLLMHttpServer pid=2282697) '--gpu_memory_utilization', (vLLMHttpServer pid=2282697) '0.8', (vLLMHttpServer pid=2282697) '--disable_log_stats', (vLLMHttpServer pid=2282697) '--tensor_parallel_size', (vLLMHttpServer pid=2282697) '2', (vLLMHttpServer pid=2282697) '--seed', (vLLMHttpServer pid=2282697) '0', (vLLMHttpServer pid=2282697) '--override_generation_config', (vLLMHttpServer pid=2282697) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2282697) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2282697) '--hf_overrides', (vLLMHttpServer pid=2282697) '{}', (vLLMHttpServer pid=2282697) '--scheduling_policy', (vLLMHttpServer pid=2282697) 'fcfs'] (WorkerDict pid=2281817) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2282697) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2282697) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2282697) WARNING 07-03 03:20:04 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2281811) WARNING 07-03 03:20:12 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2282699) WARNING 07-03 03:20:05 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2281812) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2281810) 2026-07-03 03:20:26,670 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (vLLMHttpServer pid=2282699) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2282699) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2282699) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2282699) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2281814) 2026-07-03 03:20:26,669 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2281810) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] (TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $, the charge. Rearranging the formula: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Substitute the known values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0 \, \mathrm{N/C}) \times (0.50 \, \mathrm{m})^2}{8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2}} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest option is **B: 56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2281424) step:10 - global_seqlen/min:18370 - global_seqlen/max:31673 - global_seqlen/minmax_diff:13303 - global_seqlen/balanced_min:26421 - global_seqlen/balanced_max:26444 - global_seqlen/mean:26434.75 - actor/entropy:0.32802483439445496 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45646193623542786 - training/rollout_probs_diff_mean:0.0033578479196876287 - training/rollout_probs_diff_std:0.00903936568647623 - training/rollout_actor_probs_pearson_corr:0.9993472099304199 - rollout_corr/rollout_is_mean:0.9999197721481323 - rollout_corr/rollout_is_ratio_fraction_high:7.013114554865751e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.4026229109731503e-05 - rollout_corr/rollout_is_max:3.0499253273010254 - rollout_corr/rollout_is_min:0.36710941791534424 - rollout_corr/rollout_is_std:0.029133660718798637 - rollout_corr/rollout_is_eff_sample_size:0.999151830651037 - rollout_corr/rollout_is_seq_mean:0.9999980330467224 - rollout_corr/rollout_is_seq_std:0.0017985748127102852 - rollout_corr/rollout_is_seq_max:1.008771300315857 - rollout_corr/rollout_is_seq_min:0.9932235479354858 - rollout_corr/rollout_is_seq_max_deviation:0.008771300315856934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.377936966251582) - rollout_corr/training_log_ppl:np.float64(0.3002843804570148) - rollout_corr/kl:np.float64(0.0005245888745832872) - rollout_corr/k3_kl:np.float64(0.000498560940002335) - rollout_corr/rollout_ppl:np.float64(1.377146857790649) - rollout_corr/rollout_log_ppl:np.float64(0.29975979137816466) - rollout_corr/log_ppl_diff:np.float64(0.0005245890788501129) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013371909590205178) - rollout_corr/log_ppl_diff_max:np.float64(0.0069696009159088135) - rollout_corr/log_ppl_diff_min:np.float64(-0.004811465740203857) - rollout_corr/ppl_ratio:np.float64(1.0005262338090688) - rollout_corr/chi2_token:np.float64(0.0009716309141367674) - rollout_corr/chi2_seq:np.float64(1.0234597465023398) - actor/pg_loss:np.float64(2.4606240913271904e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024767978288764425) - actor/ppo_kl:np.float64(4.621054888964693e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18785973638296127) - perf/mfu/actor:np.float64(0.10560325767310388) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(99.76485443115234) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.58359375) - val-aux/sciknoweval/reward/std@16:np.float64(0.21906251159087287) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.67415) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19020325207669364) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.4956125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17791978642388112) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5849125000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2188371622304282) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7478) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14805144079456356) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.425825) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12986404819208913) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6028874999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17818578143360303) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8051999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10766830979919102) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3753875) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08819187467072624) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6146375) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13509611267304375) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8500624999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07012187539545785) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3417125) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05316338904072637) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6237) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09735119649509659) - val-aux/sciknoweval/score/mean@16:np.float64(0.58359375) - val-aux/sciknoweval/score/std@16:np.float64(0.21906251159087287) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.67415) - val-aux/sciknoweval/score/best@2/std:np.float64(0.19020325207669364) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.4956125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17791978642388112) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5849125000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2188371622304282) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7478) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14805144079456356) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.425825) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12986404819208913) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6028874999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17818578143360303) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8051999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10766830979919102) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3753875) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08819187467072624) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6146375) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13509611267304375) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8500624999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07012187539545785) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3417125) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05316338904072637) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6237) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09735119649509659) - val-core/sciknoweval/acc/mean@16:np.float64(0.58359375) - val-aux/sciknoweval/acc/std@16:np.float64(0.21906251159087287) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.67415) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.19020325207669364) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.4956125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17791978642388112) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5849125000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2188371622304282) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7478) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14805144079456356) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.425825) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12986404819208913) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6028874999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17818578143360303) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8051999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10766830979919102) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3753875) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08819187467072624) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6146375) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13509611267304375) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8500624999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07012187539545785) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3417125) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05316338904072637) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6237) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09735119649509659) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0432113421230519) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99315) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.016507906298959766) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9690875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.053154032584486456) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9808874999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.042999991812243614) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9968875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.005405422624550276) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9512750000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.06454074729263456) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9880125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.028221887660256677) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9994) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0026720778431774776) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9238625) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07328509472283329) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.992225) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.014819305487201267) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.887475) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06968272194151441) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9943125) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.007892775584975463) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01252454612404108 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01252454612404108 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:556.9921875 - response_length/max:1781.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:556.9921875 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014546141028404236 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1660982128232718) - timing_s/agent_loop/generate_sequences/max:np.float64(13.751114128157496) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.805240769754164) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.751114128157496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1781 - timing_s/gen:20.531365724280477 - timing_s/reward:0.06193559989333153 - timing_s/old_log_prob:3.251886686310172 - timing_s/adv:0.15388510562479496 - timing_s/update_actor:13.160292625427246 - timing_s/step:37.241492161527276 - timing_s/testing:116.28815667517483 - timing_s/save_checkpoint:12.811152569949627 - timing_s/stop_profile:0.00015922822058200836 - timing_per_token_ms/gen:0.1439888191617959 - timing_per_token_ms/update_actor:0.06223007889911596 - timing_per_token_ms/adv:0.0007276648427959171 - perf/total_num_tokens:211478 - perf/time_per_step:37.241492161527276 - perf/throughput:709.8198397997786 (TaskRunner pid=2281424) Training Progress: 10%|█ | 10/100 [09:39<2:02:00, 81.34s/it] (TaskRunner pid=2281424) step:11 - global_seqlen/min:20995 - global_seqlen/max:35423 - global_seqlen/minmax_diff:14428 - global_seqlen/balanced_min:27571 - global_seqlen/balanced_max:27590 - global_seqlen/mean:27580.5 - actor/entropy:0.24689078330993652 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6190903186798096 - training/rollout_probs_diff_mean:0.0029323738999664783 - training/rollout_probs_diff_std:0.008981931023299694 - training/rollout_actor_probs_pearson_corr:0.999229907989502 - rollout_corr/rollout_is_mean:0.9999557137489319 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.325087163830176e-05 - rollout_corr/rollout_is_max:1.9537171125411987 - rollout_corr/rollout_is_min:0.08846962451934814 - rollout_corr/rollout_is_std:0.0269706342369318 - rollout_corr/rollout_is_eff_sample_size:0.9992731136632474 - rollout_corr/rollout_is_seq_mean:1.0000543594360352 - rollout_corr/rollout_is_seq_std:0.0017303781351074576 - rollout_corr/rollout_is_seq_max:1.0074354410171509 - rollout_corr/rollout_is_seq_min:0.9947044253349304 - rollout_corr/rollout_is_seq_max_deviation:0.007435441017150879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2710583903826773) - rollout_corr/training_log_ppl:np.float64(0.22995059203822166) - rollout_corr/kl:np.float64(0.0004979196115613149) - rollout_corr/k3_kl:np.float64(0.00043510435352800414) - rollout_corr/rollout_ppl:np.float64(1.270424968097359) - rollout_corr/rollout_log_ppl:np.float64(0.22945267148315907) - rollout_corr/log_ppl_diff:np.float64(0.000497920555062592) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013768301578238606) - rollout_corr/log_ppl_diff_max:np.float64(0.006137266755104065) - rollout_corr/log_ppl_diff_min:np.float64(-0.007355779409408569) - rollout_corr/ppl_ratio:np.float64(1.000499751418829) - rollout_corr/chi2_token:np.float64(0.0007434843573719263) - rollout_corr/chi2_seq:np.float64(0.47075980668887496) - actor/pg_loss:np.float64(4.087574779987335e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001813313299408037) - actor/ppo_kl:np.float64(0.00013543530537063475) - actor/pg_clipfrac_lower:np.float64(4.278477717889473e-06) - actor/grad_norm:np.float64(0.17897598445415497) - perf/mfu/actor:np.float64(0.10733546751479987) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(104.49744415283203) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004445641301572323 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.004445641301572323 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:587.390625 - response_length/max:1781.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:587.390625 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.45193862915039e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9449637047946453) - timing_s/agent_loop/generate_sequences/max:np.float64(13.539711721241474) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.947522593545727) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.539711721241474) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1671 - timing_s/gen:20.11647711135447 - timing_s/reward:0.06534852646291256 - timing_s/old_log_prob:3.4682928640395403 - timing_s/adv:0.13045809976756573 - timing_s/update_actor:13.693839404731989 - timing_s/step:37.579134887084365 - timing_s/stop_profile:0.00012080185115337372 - timing_per_token_ms/gen:0.13377807777614495 - timing_per_token_ms/update_actor:0.06206304909597356 - timing_per_token_ms/adv:0.0005912605816046016 - perf/total_num_tokens:220644 - perf/time_per_step:37.579134887084365 - perf/throughput:733.9312116383815 (TaskRunner pid=2281424) Training Progress: 11%|█ | 11/100 [10:17<1:40:48, 67.96s/it] (TaskRunner pid=2281424) step:12 - global_seqlen/min:18284 - global_seqlen/max:36197 - global_seqlen/minmax_diff:17913 - global_seqlen/balanced_min:27117 - global_seqlen/balanced_max:27742 - global_seqlen/mean:27219.25 - actor/entropy:0.2918388545513153 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463563919067383 - training/rollout_probs_diff_mean:0.00314357434399426 - training/rollout_probs_diff_std:0.008670532144606113 - training/rollout_actor_probs_pearson_corr:0.9993590712547302 - rollout_corr/rollout_is_mean:0.9999263882637024 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.328426969848806e-05 - rollout_corr/rollout_is_max:1.6952860355377197 - rollout_corr/rollout_is_min:0.2599896490573883 - rollout_corr/rollout_is_std:0.02788124419748783 - rollout_corr/rollout_is_eff_sample_size:0.9992230019996114 - rollout_corr/rollout_is_seq_mean:0.9999516010284424 - rollout_corr/rollout_is_seq_std:0.0013419243041425943 - rollout_corr/rollout_is_seq_max:1.0067940950393677 - rollout_corr/rollout_is_seq_min:0.9959049224853516 - rollout_corr/rollout_is_seq_max_deviation:0.006794095039367676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.284353747498244) - rollout_corr/training_log_ppl:np.float64(0.24118711793562397) - rollout_corr/kl:np.float64(0.0004640274163274505) - rollout_corr/k3_kl:np.float64(0.000438987553025072) - rollout_corr/rollout_ppl:np.float64(1.2837417596019804) - rollout_corr/rollout_log_ppl:np.float64(0.2407230889366474) - rollout_corr/log_ppl_diff:np.float64(0.00046402899897657335) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012675680045504123) - rollout_corr/log_ppl_diff_max:np.float64(0.00536249577999115) - rollout_corr/log_ppl_diff_min:np.float64(-0.005736485123634338) - rollout_corr/ppl_ratio:np.float64(1.0004653770010918) - rollout_corr/chi2_token:np.float64(0.0008103374857455492) - rollout_corr/chi2_seq:np.float64(0.7635552103165537) - actor/pg_loss:np.float64(6.2944600358605385e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022421280800699606) - actor/ppo_kl:np.float64(8.907836768656807e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1681164726614952) - perf/mfu/actor:np.float64(0.10657234495140916) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.30431365966797) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02310134656727314 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02310134656727314 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:588.1015625 - response_length/max:3570.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:588.1015625 - response_length_non_aborted/max:3570.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.758848369121552e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9172126762568951) - timing_s/agent_loop/generate_sequences/max:np.float64(22.92699559032917) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.20948198466067) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.92699559032917) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:3570 - timing_s/gen:29.500037664547563 - timing_s/reward:0.06999753788113594 - timing_s/old_log_prob:3.321774547919631 - timing_s/adv:0.16080131568014622 - timing_s/update_actor:13.495587630197406 - timing_s/step:46.61714624799788 - timing_s/stop_profile:0.00011683814227581024 - timing_per_token_ms/gen:0.19594323408576034 - timing_per_token_ms/update_actor:0.06197630183692334 - timing_per_token_ms/adv:0.0007384540154492969 - perf/total_num_tokens:217754 - perf/time_per_step:46.61714624799788 - perf/throughput:583.8892379897454 (TaskRunner pid=2281424) Training Progress: 12%|█▏ | 12/100 [11:03<1:30:10, 61.48s/it] (TaskRunner pid=2281424) step:13 - global_seqlen/min:24964 - global_seqlen/max:33451 - global_seqlen/minmax_diff:8487 - global_seqlen/balanced_min:28949 - global_seqlen/balanced_max:28972 - global_seqlen/mean:28959.375 - actor/entropy:0.31463027000427246 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24114426970481873 - training/rollout_probs_diff_mean:0.00320067978464067 - training/rollout_probs_diff_std:0.008584943599998951 - training/rollout_actor_probs_pearson_corr:0.9994125962257385 - rollout_corr/rollout_is_mean:1.0001263618469238 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.238653440959752e-06 - rollout_corr/rollout_is_max:1.859130859375 - rollout_corr/rollout_is_min:0.4844902455806732 - rollout_corr/rollout_is_std:0.02811329998075962 - rollout_corr/rollout_is_eff_sample_size:0.9992105046242893 - rollout_corr/rollout_is_seq_mean:1.0000969171524048 - rollout_corr/rollout_is_seq_std:0.0011731568956747651 - rollout_corr/rollout_is_seq_max:1.0035706758499146 - rollout_corr/rollout_is_seq_min:0.9957525134086609 - rollout_corr/rollout_is_seq_max_deviation:0.004247486591339111 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3780980156734586) - rollout_corr/training_log_ppl:np.float64(0.2991240085102618) - rollout_corr/kl:np.float64(0.0003781683996528784) - rollout_corr/k3_kl:np.float64(0.0004593968702977236) - rollout_corr/rollout_ppl:np.float64(1.3775346302427351) - rollout_corr/rollout_log_ppl:np.float64(0.2987458378338488) - rollout_corr/log_ppl_diff:np.float64(0.0003781706764129922) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010912520374404266) - rollout_corr/log_ppl_diff_max:np.float64(0.004893004894256592) - rollout_corr/log_ppl_diff_min:np.float64(-0.003899678587913513) - rollout_corr/ppl_ratio:np.float64(1.0003791959024966) - rollout_corr/chi2_token:np.float64(0.001073798630386591) - rollout_corr/chi2_seq:np.float64(0.506177173461765) - actor/pg_loss:np.float64(3.237277269363403e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004119513098430616) - actor/ppo_kl:np.float64(7.283486395426308e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21577078476548195) - perf/mfu/actor:np.float64(0.11467366616050297) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.31635284423828) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002386284904787317 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0002386284904787317 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:626.13671875 - response_length/max:1861.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:626.13671875 - response_length_non_aborted/max:1861.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013312511146068573 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9897179938852787) - timing_s/agent_loop/generate_sequences/max:np.float64(15.492640441283584) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.851898552646162) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.492640441283584) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:1861 - timing_s/gen:22.31190705113113 - timing_s/reward:0.0678411666303873 - timing_s/old_log_prob:3.3336705192923546 - timing_s/adv:0.18029628694057465 - timing_s/update_actor:13.361042624339461 - timing_s/step:39.349795043468475 - timing_s/stop_profile:0.00011570192873477936 - timing_per_token_ms/gen:0.1391962558791893 - timing_per_token_ms/update_actor:0.05767149077086203 - timing_per_token_ms/adv:0.0007782293598384575 - perf/total_num_tokens:231675 - perf/time_per_step:39.349795043468475 - perf/throughput:735.9472893825621 (TaskRunner pid=2281424) Training Progress: 13%|█▎ | 13/100 [11:43<1:19:27, 54.80s/it] (TaskRunner pid=2281424) step:14 - global_seqlen/min:18482 - global_seqlen/max:39950 - global_seqlen/minmax_diff:21468 - global_seqlen/balanced_min:27989 - global_seqlen/balanced_max:28043 - global_seqlen/mean:28025.875 - actor/entropy:0.27694693207740784 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9996616840362549 - training/rollout_probs_diff_mean:0.0032118367962539196 - training/rollout_probs_diff_std:0.009403628297150135 - training/rollout_actor_probs_pearson_corr:0.9992220997810364 - rollout_corr/rollout_is_mean:0.9999675154685974 - rollout_corr/rollout_is_ratio_fraction_high:2.5921677661244757e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.2402098440798e-05 - rollout_corr/rollout_is_max:2.0761139392852783 - rollout_corr/rollout_is_min:0.000335350981913507 - rollout_corr/rollout_is_std:0.028911858797073364 - rollout_corr/rollout_is_eff_sample_size:0.9991645645790151 - rollout_corr/rollout_is_seq_mean:0.9999456405639648 - rollout_corr/rollout_is_seq_std:0.001373259350657463 - rollout_corr/rollout_is_seq_max:1.0052764415740967 - rollout_corr/rollout_is_seq_min:0.995773434638977 - rollout_corr/rollout_is_seq_max_deviation:0.00527644157409668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3075349098071456) - rollout_corr/training_log_ppl:np.float64(0.2546500996249961) - rollout_corr/kl:np.float64(0.0006805974277988014) - rollout_corr/k3_kl:np.float64(0.0006370248192979489) - rollout_corr/rollout_ppl:np.float64(1.3065974819473922) - rollout_corr/rollout_log_ppl:np.float64(0.2539695030573057) - rollout_corr/log_ppl_diff:np.float64(0.0006805965676903725) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013074282323941588) - rollout_corr/log_ppl_diff_max:np.float64(0.01114201545715332) - rollout_corr/log_ppl_diff_min:np.float64(-0.004195064306259155) - rollout_corr/ppl_ratio:np.float64(1.000682536046952) - rollout_corr/chi2_token:np.float64(0.0010629112366586924) - rollout_corr/chi2_seq:np.float64(0.7319330512546003) - actor/pg_loss:np.float64(-7.280893623828888e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004904413003714581) - actor/ppo_kl:np.float64(0.00013513596405800854) - actor/pg_clipfrac_lower:np.float64(5.243288796918932e-06) - actor/grad_norm:np.float64(0.22375838086009026) - perf/mfu/actor:np.float64(0.11024404131767285) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.50100326538086) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004044591914862394 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004044591914862394 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:602.77734375 - response_length/max:2091.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:602.77734375 - response_length_non_aborted/max:2091.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.801425039768219e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1079022623598576) - timing_s/agent_loop/generate_sequences/max:np.float64(15.903715757653117) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.53672656758863) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.903715757653117) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:249 - timing_s/agent_loop/slowest/response_length:2091 - timing_s/gen:22.835159683600068 - timing_s/reward:0.07095035165548325 - timing_s/old_log_prob:3.195645259693265 - timing_s/adv:0.15754373371601105 - timing_s/update_actor:13.296314042061567 - timing_s/step:39.66534601524472 - timing_s/stop_profile:0.00011271052062511444 - timing_per_token_ms/gen:0.14798141210672 - timing_per_token_ms/update_actor:0.0593037418192187 - timing_per_token_ms/adv:0.0007026708966089866 - perf/total_num_tokens:224207 - perf/time_per_step:39.66534601524472 - perf/throughput:706.5581878254312 (TaskRunner pid=2281424) Training Progress: 14%|█▍ | 14/100 [12:23<1:12:00, 50.24s/it] (TaskRunner pid=2281424) step:15 - global_seqlen/min:18409 - global_seqlen/max:34972 - global_seqlen/minmax_diff:16563 - global_seqlen/balanced_min:27584 - global_seqlen/balanced_max:27657 - global_seqlen/mean:27632.0 - actor/entropy:0.23792505264282227 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2081579566001892 - training/rollout_probs_diff_mean:0.0028373654931783676 - training/rollout_probs_diff_std:0.008734497241675854 - training/rollout_actor_probs_pearson_corr:0.9992759227752686 - rollout_corr/rollout_is_mean:1.000104308128357 - rollout_corr/rollout_is_ratio_fraction_high:6.5414201344538014e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0567126274108887 - rollout_corr/rollout_is_min:0.6020901799201965 - rollout_corr/rollout_is_std:0.0269706342369318 - rollout_corr/rollout_is_eff_sample_size:0.9992733517354029 - rollout_corr/rollout_is_seq_mean:1.0001739263534546 - rollout_corr/rollout_is_seq_std:0.0016109951538965106 - rollout_corr/rollout_is_seq_max:1.0077435970306396 - rollout_corr/rollout_is_seq_min:0.9901288747787476 - rollout_corr/rollout_is_seq_max_deviation:0.009871125221252441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2825340307317674) - rollout_corr/training_log_ppl:np.float64(0.2307681181409862) - rollout_corr/kl:np.float64(0.00024018173945883348) - rollout_corr/k3_kl:np.float64(0.00046571865181022076) - rollout_corr/rollout_ppl:np.float64(1.2822559108026326) - rollout_corr/rollout_log_ppl:np.float64(0.23052793518581893) - rollout_corr/log_ppl_diff:np.float64(0.0002401829551672563) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012830473860958591) - rollout_corr/log_ppl_diff_max:np.float64(0.011005207896232605) - rollout_corr/log_ppl_diff_min:np.float64(-0.004439152777194977) - rollout_corr/ppl_ratio:np.float64(1.0002417913638055) - rollout_corr/chi2_token:np.float64(0.0013833614066243172) - rollout_corr/chi2_seq:np.float64(0.8579549191053957) - actor/pg_loss:np.float64(0.00013076758477836847) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022705169044456852) - actor/ppo_kl:np.float64(1.2435915886399584e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1974821835756302) - perf/mfu/actor:np.float64(0.10945352184092855) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.5802116394043) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011041100136935711 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011041100136935711 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:597.15625 - response_length/max:2544.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:597.15625 - response_length_non_aborted/max:2544.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001034904271364212 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0850455239415169) - timing_s/agent_loop/generate_sequences/max:np.float64(18.34066752716899) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.139947620045859) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.34066752716899) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:2544 - timing_s/gen:24.80101560242474 - timing_s/reward:0.06913194060325623 - timing_s/old_log_prob:3.2432605400681496 - timing_s/adv:0.15737476013600826 - timing_s/update_actor:13.316460059955716 - timing_s/step:41.67461157031357 - timing_s/stop_profile:0.00011474825441837311 - timing_per_token_ms/gen:0.1622338662568995 - timing_per_token_ms/update_actor:0.06024021089658601 - timing_per_token_ms/adv:0.0007119225903662794 - perf/total_num_tokens:221056 - perf/time_per_step:41.67461157031357 - perf/throughput:663.0415727661715 (TaskRunner pid=2281424) Training Progress: 15%|█▌ | 15/100 [13:04<1:07:32, 47.68s/it] (TaskRunner pid=2281424) step:16 - global_seqlen/min:19774 - global_seqlen/max:37148 - global_seqlen/minmax_diff:17374 - global_seqlen/balanced_min:26832 - global_seqlen/balanced_max:26869 - global_seqlen/mean:26857.125 - actor/entropy:0.269243985414505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29323524236679077 - training/rollout_probs_diff_mean:0.0030834192875772715 - training/rollout_probs_diff_std:0.009029646404087543 - training/rollout_actor_probs_pearson_corr:0.9992733001708984 - rollout_corr/rollout_is_mean:1.000123381614685 - rollout_corr/rollout_is_ratio_fraction_high:1.3476997992256656e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.39180064201355 - rollout_corr/rollout_is_min:0.5188149213790894 - rollout_corr/rollout_is_std:0.028928346931934357 - rollout_corr/rollout_is_eff_sample_size:0.9991640885384842 - rollout_corr/rollout_is_seq_mean:1.0000724792480469 - rollout_corr/rollout_is_seq_std:0.0016198219964280725 - rollout_corr/rollout_is_seq_max:1.0117334127426147 - rollout_corr/rollout_is_seq_min:0.9945037961006165 - rollout_corr/rollout_is_seq_max_deviation:0.011733412742614746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2698871567845345) - rollout_corr/training_log_ppl:np.float64(0.22661357806646265) - rollout_corr/kl:np.float64(0.000278438638499523) - rollout_corr/k3_kl:np.float64(0.0004574081480726022) - rollout_corr/rollout_ppl:np.float64(1.2695180620066822) - rollout_corr/rollout_log_ppl:np.float64(0.22633513917389791) - rollout_corr/log_ppl_diff:np.float64(0.0002784388925647363) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010494471789570525) - rollout_corr/log_ppl_diff_max:np.float64(0.00465761125087738) - rollout_corr/log_ppl_diff_min:np.float64(-0.009550660848617554) - rollout_corr/ppl_ratio:np.float64(1.000279544852674) - rollout_corr/chi2_token:np.float64(0.0013024122454226017) - rollout_corr/chi2_seq:np.float64(0.9144449539016932) - actor/pg_loss:np.float64(-1.0943855158984661e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023535769855698163) - actor/ppo_kl:np.float64(-7.823348023361376e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21825583279132843) - perf/mfu/actor:np.float64(0.10663394115254357) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.46512222290039) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005873444024473429 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005873444024473429 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:579.69140625 - response_length/max:1869.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:579.69140625 - response_length_non_aborted/max:1869.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001010652631521225 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3836285527795553) - timing_s/agent_loop/generate_sequences/max:np.float64(14.655153315514326) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.240751736462698) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.655153315514326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1869 - timing_s/gen:21.27606852352619 - timing_s/reward:0.06742825172841549 - timing_s/old_log_prob:3.215101270005107 - timing_s/adv:0.13886815682053566 - timing_s/update_actor:13.220384620130062 - timing_s/step:38.01398337073624 - timing_s/stop_profile:2.8956681489944458e-05 - timing_per_token_ms/gen:0.14336876788920688 - timing_per_token_ms/update_actor:0.06153108635106169 - timing_per_token_ms/adv:0.0006463282872819394 - perf/total_num_tokens:214857 - perf/time_per_step:38.01398337073624 - perf/throughput:706.5064646888607 (TaskRunner pid=2281424) Training Progress: 16%|█▌ | 16/100 [13:42<1:02:41, 44.77s/it] (TaskRunner pid=2281424) step:17 - global_seqlen/min:21078 - global_seqlen/max:38139 - global_seqlen/minmax_diff:17061 - global_seqlen/balanced_min:27815 - global_seqlen/balanced_max:28195 - global_seqlen/mean:27876.125 - actor/entropy:0.3084404766559601 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105865716934204 - training/rollout_probs_diff_mean:0.003280392149463296 - training/rollout_probs_diff_std:0.008956593461334705 - training/rollout_actor_probs_pearson_corr:0.9993877410888672 - rollout_corr/rollout_is_mean:1.0001431703567505 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.458905318140751e-06 - rollout_corr/rollout_is_max:1.8297889232635498 - rollout_corr/rollout_is_min:0.49442058801651 - rollout_corr/rollout_is_std:0.02931314706802368 - rollout_corr/rollout_is_eff_sample_size:0.9991418341499507 - rollout_corr/rollout_is_seq_mean:1.0001916885375977 - rollout_corr/rollout_is_seq_std:0.0019043951760977507 - rollout_corr/rollout_is_seq_max:1.009162425994873 - rollout_corr/rollout_is_seq_min:0.9893993139266968 - rollout_corr/rollout_is_seq_max_deviation:0.010600686073303223 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3411333807744086) - rollout_corr/training_log_ppl:np.float64(0.2804811333771795) - rollout_corr/kl:np.float64(0.00039379035426634346) - rollout_corr/k3_kl:np.float64(0.0005168410488067821) - rollout_corr/rollout_ppl:np.float64(1.3405519826337695) - rollout_corr/rollout_log_ppl:np.float64(0.28008734311151784) - rollout_corr/log_ppl_diff:np.float64(0.0003937902656616643) - rollout_corr/log_ppl_abs_diff:np.float64(0.001501921404269524) - rollout_corr/log_ppl_diff_max:np.float64(0.011711418628692627) - rollout_corr/log_ppl_diff_min:np.float64(-0.007628694176673889) - rollout_corr/ppl_ratio:np.float64(1.0003962186165154) - rollout_corr/chi2_token:np.float64(0.001294472487643361) - rollout_corr/chi2_seq:np.float64(1.0754117609467357) - actor/pg_loss:np.float64(-0.00010975566692650318) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020725219656014815) - actor/ppo_kl:np.float64(0.00011342834537941826) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2514360435307026) - perf/mfu/actor:np.float64(0.10841431290124035) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.30791854858398) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012801550328731537 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012801550328731537 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.78515625 - response_length/max:2740.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.78515625 - response_length_non_aborted/max:2740.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.358139216899872e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0903181321918964) - timing_s/agent_loop/generate_sequences/max:np.float64(20.050666142255068) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.366174238610256) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.050666142255068) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:2740 - timing_s/gen:27.158069726079702 - timing_s/reward:0.08344477787613869 - timing_s/old_log_prob:3.2945270854979753 - timing_s/adv:0.13789121992886066 - timing_s/update_actor:13.660845940932631 - timing_s/step:44.418967282399535 - timing_s/stop_profile:2.7371570467948914e-05 - timing_per_token_ms/gen:0.17541139819848023 - timing_per_token_ms/update_actor:0.06125692658562045 - timing_per_token_ms/adv:0.0006183213230356652 - perf/total_num_tokens:223009 - perf/time_per_step:44.418967282399535 - perf/throughput:627.5725597755077 (TaskRunner pid=2281424) Training Progress: 17%|█▋ | 17/100 [14:27<1:01:48, 44.68s/it] (TaskRunner pid=2281424) step:18 - global_seqlen/min:18777 - global_seqlen/max:44614 - global_seqlen/minmax_diff:25837 - global_seqlen/balanced_min:32860 - global_seqlen/balanced_max:32946 - global_seqlen/mean:32927.0 - actor/entropy:0.2818203866481781 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2900807857513428 - training/rollout_probs_diff_mean:0.002938726684078574 - training/rollout_probs_diff_std:0.00846165232360363 - training/rollout_actor_probs_pearson_corr:0.9994096159934998 - rollout_corr/rollout_is_mean:1.0000146627426147 - rollout_corr/rollout_is_ratio_fraction_high:5.105583568365546e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.552791738708038e-05 - rollout_corr/rollout_is_max:2.0839157104492188 - rollout_corr/rollout_is_min:0.4158320426940918 - rollout_corr/rollout_is_std:0.02699493244290352 - rollout_corr/rollout_is_eff_sample_size:0.9992719233041721 - rollout_corr/rollout_is_seq_mean:1.0000157356262207 - rollout_corr/rollout_is_seq_std:0.0014222179306671023 - rollout_corr/rollout_is_seq_max:1.0088167190551758 - rollout_corr/rollout_is_seq_min:0.995079755783081 - rollout_corr/rollout_is_seq_max_deviation:0.008816719055175781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.289564796257764) - rollout_corr/training_log_ppl:np.float64(0.2442400810541585) - rollout_corr/kl:np.float64(0.0005266999699861663) - rollout_corr/k3_kl:np.float64(0.0004649700730112727) - rollout_corr/rollout_ppl:np.float64(1.2888782764784992) - rollout_corr/rollout_log_ppl:np.float64(0.24371338007040322) - rollout_corr/log_ppl_diff:np.float64(0.0005267009837552905) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011411418090574443) - rollout_corr/log_ppl_diff_max:np.float64(0.010496377944946289) - rollout_corr/log_ppl_diff_min:np.float64(-0.007471442222595215) - rollout_corr/ppl_ratio:np.float64(1.0005280999466777) - rollout_corr/chi2_token:np.float64(0.0008352911099791527) - rollout_corr/chi2_seq:np.float64(0.4274897330906242) - actor/pg_loss:np.float64(9.232200682163239e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002937083892220471) - actor/ppo_kl:np.float64(0.00015827277887936475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1933615617454052) - perf/mfu/actor:np.float64(0.12426924001280201) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.3476791381836) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007050810847431421 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007050810847431421 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:765.09375 - response_length/max:3008.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:765.09375 - response_length_non_aborted/max:3008.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.749613046646118e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9996219798922539) - timing_s/agent_loop/generate_sequences/max:np.float64(21.299407305195928) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.81618346637697) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.299407305195928) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:330 - timing_s/agent_loop/slowest/response_length:3008 - timing_s/gen:28.408122271299362 - timing_s/reward:0.07211213000118732 - timing_s/old_log_prob:3.439273815602064 - timing_s/adv:0.14084554836153984 - timing_s/update_actor:14.02409834600985 - timing_s/step:46.15811971202493 - timing_s/stop_profile:0.0001232251524925232 - timing_per_token_ms/gen:0.14504003937068252 - timing_per_token_ms/update_actor:0.05323935655392934 - timing_per_token_ms/adv:0.0005346886611350102 - perf/total_num_tokens:263416 - perf/time_per_step:46.15811971202493 - perf/throughput:713.352281363012 (TaskRunner pid=2281424) Training Progress: 18%|█▊ | 18/100 [15:13<1:01:41, 45.14s/it] (TaskRunner pid=2281424) step:19 - global_seqlen/min:21194 - global_seqlen/max:44487 - global_seqlen/minmax_diff:23293 - global_seqlen/balanced_min:30623 - global_seqlen/balanced_max:30828 - global_seqlen/mean:30657.0 - actor/entropy:0.2834802567958832 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35189008712768555 - training/rollout_probs_diff_mean:0.002977204043418169 - training/rollout_probs_diff_std:0.00879612471908331 - training/rollout_actor_probs_pearson_corr:0.9994228482246399 - rollout_corr/rollout_is_mean:1.0000439882278442 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2517451725434512e-05 - rollout_corr/rollout_is_max:1.6279520988464355 - rollout_corr/rollout_is_min:0.2959691882133484 - rollout_corr/rollout_is_std:0.027716144919395447 - rollout_corr/rollout_is_eff_sample_size:0.9992325240192108 - rollout_corr/rollout_is_seq_mean:1.000134825706482 - rollout_corr/rollout_is_seq_std:0.0016275998204946518 - rollout_corr/rollout_is_seq_max:1.009211540222168 - rollout_corr/rollout_is_seq_min:0.9948177337646484 - rollout_corr/rollout_is_seq_max_deviation:0.009211540222167969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2855765600688756) - rollout_corr/training_log_ppl:np.float64(0.23977286802255549) - rollout_corr/kl:np.float64(0.0005027599546281181) - rollout_corr/k3_kl:np.float64(0.0004755220272727456) - rollout_corr/rollout_ppl:np.float64(1.2849308247677982) - rollout_corr/rollout_log_ppl:np.float64(0.2392701047792798) - rollout_corr/log_ppl_diff:np.float64(0.0005027632432756945) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011823297390947118) - rollout_corr/log_ppl_diff_max:np.float64(0.009016215801239014) - rollout_corr/log_ppl_diff_min:np.float64(-0.008047908544540405) - rollout_corr/ppl_ratio:np.float64(1.0005042462144047) - rollout_corr/chi2_token:np.float64(0.0009039849974215031) - rollout_corr/chi2_seq:np.float64(0.45300188730470836) - actor/pg_loss:np.float64(8.254800923168659e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003364918597981159) - actor/ppo_kl:np.float64(0.0002041434732245051) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19276084750890732) - perf/mfu/actor:np.float64(0.11752787272188393) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.36764144897461) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0269034281373024 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0269034281373024 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:693.90625 - response_length/max:3216.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:693.90625 - response_length_non_aborted/max:3216.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013180077075958252 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2591454926878214) - timing_s/agent_loop/generate_sequences/max:np.float64(22.810867067426443) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.210481837464613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.810867067426443) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:3216 - timing_s/gen:29.960727905854583 - timing_s/reward:0.07135201431810856 - timing_s/old_log_prob:3.375285627320409 - timing_s/adv:0.1431940197944641 - timing_s/update_actor:13.87465051561594 - timing_s/step:47.520265359431505 - timing_s/stop_profile:0.00011952407658100128 - timing_per_token_ms/gen:0.16865980581994247 - timing_per_token_ms/update_actor:0.05657211450735534 - timing_per_token_ms/adv:0.0005838553176862712 - perf/total_num_tokens:245256 - perf/time_per_step:47.520265359431505 - perf/throughput:645.1352863482149 (TaskRunner pid=2281424) Training Progress: 19%|█▉ | 19/100 [16:01<1:01:55, 45.87s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] (TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The electric field due to a point charge is given by: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for the charge $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Substituting the given values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Convert to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest option is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2281424) step:20 - global_seqlen/min:19499 - global_seqlen/max:36025 - global_seqlen/minmax_diff:16526 - global_seqlen/balanced_min:27704 - global_seqlen/balanced_max:28173 - global_seqlen/mean:27765.0 - actor/entropy:0.280457079410553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135375022888184 - training/rollout_probs_diff_mean:0.003101746318861842 - training/rollout_probs_diff_std:0.008900674059987068 - training/rollout_actor_probs_pearson_corr:0.999406099319458 - rollout_corr/rollout_is_mean:0.9998956918716431 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9279214029666036e-05 - rollout_corr/rollout_is_max:1.8279300928115845 - rollout_corr/rollout_is_min:0.40598663687705994 - rollout_corr/rollout_is_std:0.028816867619752884 - rollout_corr/rollout_is_eff_sample_size:0.9991700390777238 - rollout_corr/rollout_is_seq_mean:0.999803900718689 - rollout_corr/rollout_is_seq_std:0.001518259639851749 - rollout_corr/rollout_is_seq_max:1.0038286447525024 - rollout_corr/rollout_is_seq_min:0.9914582967758179 - rollout_corr/rollout_is_seq_max_deviation:0.008541703224182129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2934490512125194) - rollout_corr/training_log_ppl:np.float64(0.24451396349468268) - rollout_corr/kl:np.float64(0.0004959776629758217) - rollout_corr/k3_kl:np.float64(0.0004771725971863816) - rollout_corr/rollout_ppl:np.float64(1.2928053527139127) - rollout_corr/rollout_log_ppl:np.float64(0.2440179859258933) - rollout_corr/log_ppl_diff:np.float64(0.0004959775687893853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011907425214303657) - rollout_corr/log_ppl_diff_max:np.float64(0.009751290082931519) - rollout_corr/log_ppl_diff_min:np.float64(-0.004419475793838501) - rollout_corr/ppl_ratio:np.float64(1.0004974079784006) - rollout_corr/chi2_token:np.float64(0.0009594245348125696) - rollout_corr/chi2_seq:np.float64(0.41463141702115536) - actor/pg_loss:np.float64(1.230929046869278e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021071931905680685) - actor/ppo_kl:np.float64(-0.0001457839747551759) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1746744140982628) - perf/mfu/actor:np.float64(0.10426063797607768) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(103.34674835205078) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.59921875) - val-aux/sciknoweval/reward/std@16:np.float64(0.20532643626967628) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.684225) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18261389709388515) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.514475) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1643688800388951) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.60015) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2055792779718459) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7575125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14221272724332604) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4511375) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11570816324542939) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6215249999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17291949688631747) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8132375) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09937488352535821) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4064375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07134373339436899) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6354124999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1400502136179348) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8518374999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06232857449090168) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.37925) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04032492559816903) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6459375) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10760674780586772) - val-aux/sciknoweval/score/mean@16:np.float64(0.59921875) - val-aux/sciknoweval/score/std@16:np.float64(0.20532643626967628) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.684225) - val-aux/sciknoweval/score/best@2/std:np.float64(0.18261389709388515) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.514475) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1643688800388951) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.60015) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2055792779718459) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7575125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14221272724332604) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4511375) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11570816324542939) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6215249999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17291949688631747) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8132375) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09937488352535821) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4064375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07134373339436899) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6354124999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1400502136179348) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8518374999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06232857449090168) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.37925) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04032492559816903) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6459375) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10760674780586772) - val-core/sciknoweval/acc/mean@16:np.float64(0.59921875) - val-aux/sciknoweval/acc/std@16:np.float64(0.20532643626967628) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.684225) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.18261389709388515) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.514475) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1643688800388951) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.60015) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2055792779718459) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7575125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14221272724332604) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4511375) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11570816324542939) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6215249999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17291949688631747) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8132375) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09937488352535821) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4064375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07134373339436899) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6354124999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1400502136179348) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8518374999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06232857449090168) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.37925) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04032492559816903) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6459375) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10760674780586772) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9578125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.08158753118437541) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9860749999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.03951222065525328) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9282874999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0973249508439899) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9574) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.08213995093780886) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9966125) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.012924820131308547) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8906499999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.10540656140217979) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.974175) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.05962872167707507) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.999525) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0032780861692795426) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8437625000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.10035091942020005) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9859375) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.03701549481184431) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0003950870157319778) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7979749999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08009154869844874) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9934750000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.018079184472962695) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011969178915023804 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.011969178915023804 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:607.84375 - response_length/max:2668.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:607.84375 - response_length_non_aborted/max:2668.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.631311357021332e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.728585135191679) - timing_s/agent_loop/generate_sequences/max:np.float64(19.024446960538626) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.399849012530467) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.024446960538626) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:2668 - timing_s/gen:26.06310199201107 - timing_s/reward:0.06901264749467373 - timing_s/old_log_prob:3.238787254318595 - timing_s/adv:0.17001512832939625 - timing_s/update_actor:14.12741300649941 - timing_s/step:43.75453623570502 - timing_s/testing:145.7744435351342 - timing_s/save_checkpoint:13.140905741602182 - timing_s/stop_profile:0.00017121247947216034 - timing_per_token_ms/gen:0.16749204405950252 - timing_per_token_ms/update_actor:0.06360261573248428 - timing_per_token_ms/adv:0.000765420170760833 - perf/total_num_tokens:222120 - perf/time_per_step:43.75453623570502 - perf/throughput:634.5627765411652 (TaskRunner pid=2281424) Training Progress: 20%|██ | 20/100 [19:23<2:03:58, 92.98s/it] (TaskRunner pid=2281424) step:21 - global_seqlen/min:25301 - global_seqlen/max:34398 - global_seqlen/minmax_diff:9097 - global_seqlen/balanced_min:29031 - global_seqlen/balanced_max:29324 - global_seqlen/mean:29106.375 - actor/entropy:0.24065154790878296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37683379650115967 - training/rollout_probs_diff_mean:0.002658553421497345 - training/rollout_probs_diff_std:0.008330690674483776 - training/rollout_actor_probs_pearson_corr:0.9994138479232788 - rollout_corr/rollout_is_mean:1.0000982284545898 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2177079042885453e-05 - rollout_corr/rollout_is_max:1.7682775259017944 - rollout_corr/rollout_is_min:0.3936323821544647 - rollout_corr/rollout_is_std:0.02563127689063549 - rollout_corr/rollout_is_eff_sample_size:0.9993435879743411 - rollout_corr/rollout_is_seq_mean:1.0001851320266724 - rollout_corr/rollout_is_seq_std:0.0012425798922777176 - rollout_corr/rollout_is_seq_max:1.0056188106536865 - rollout_corr/rollout_is_seq_min:0.9965409636497498 - rollout_corr/rollout_is_seq_max_deviation:0.0056188106536865234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2498526307754219) - rollout_corr/training_log_ppl:np.float64(0.21081234834855422) - rollout_corr/kl:np.float64(0.00032340780622153176) - rollout_corr/k3_kl:np.float64(0.0003890872453951033) - rollout_corr/rollout_ppl:np.float64(1.2494138525798917) - rollout_corr/rollout_log_ppl:np.float64(0.21048893986153416) - rollout_corr/log_ppl_diff:np.float64(0.0003234084870200604) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010021366470027715) - rollout_corr/log_ppl_diff_max:np.float64(0.004034250974655151) - rollout_corr/log_ppl_diff_min:np.float64(-0.004479348659515381) - rollout_corr/ppl_ratio:np.float64(1.0003242590464652) - rollout_corr/chi2_token:np.float64(0.0009132004342973232) - rollout_corr/chi2_seq:np.float64(0.7536862585693598) - actor/pg_loss:np.float64(3.263808321207762e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000137664294015849) - actor/ppo_kl:np.float64(0.00017732822421123728) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13653844129294157) - perf/mfu/actor:np.float64(0.11284239253639325) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(104.92810821533203) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021431660279631615 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021431660279631615 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:641.57421875 - response_length/max:3017.0 - response_length/min:180.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:641.57421875 - response_length_non_aborted/max:3017.0 - response_length_non_aborted/min:180.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.9451907873153687e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2093537878245115) - timing_s/agent_loop/generate_sequences/max:np.float64(21.849115535616875) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.7584272636668175) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.849115535616875) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:364 - timing_s/agent_loop/slowest/response_length:3017 - timing_s/gen:27.816161528229713 - timing_s/reward:0.0652250312268734 - timing_s/old_log_prob:3.518872758373618 - timing_s/adv:0.15252801775932312 - timing_s/update_actor:13.777937777340412 - timing_s/step:45.40992152877152 - timing_s/stop_profile:0.00012294389307498932 - timing_per_token_ms/gen:0.16935979937184364 - timing_per_token_ms/update_actor:0.059170618882205414 - timing_per_token_ms/adv:0.0006550455774693822 - perf/total_num_tokens:232851 - perf/time_per_step:45.40992152877152 - perf/throughput:640.9695066651532 (TaskRunner pid=2281424) Training Progress: 21%|██ | 21/100 [20:09<1:43:38, 78.71s/it] (TaskRunner pid=2281424) step:22 - global_seqlen/min:20123 - global_seqlen/max:45886 - global_seqlen/minmax_diff:25763 - global_seqlen/balanced_min:32892 - global_seqlen/balanced_max:32946 - global_seqlen/mean:32919.625 - actor/entropy:0.2954756021499634 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5972118377685547 - training/rollout_probs_diff_mean:0.0029836043249815702 - training/rollout_probs_diff_std:0.008759740740060806 - training/rollout_actor_probs_pearson_corr:0.9994177222251892 - rollout_corr/rollout_is_mean:1.000027060508728 - rollout_corr/rollout_is_ratio_fraction_high:1.0150790330953896e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5226184586936142e-05 - rollout_corr/rollout_is_max:5.339309215545654 - rollout_corr/rollout_is_min:0.3289409577846527 - rollout_corr/rollout_is_std:0.027539236471056938 - rollout_corr/rollout_is_eff_sample_size:0.999242284277643 - rollout_corr/rollout_is_seq_mean:1.000131607055664 - rollout_corr/rollout_is_seq_std:0.0019331261282786727 - rollout_corr/rollout_is_seq_max:1.0212081670761108 - rollout_corr/rollout_is_seq_min:0.990929365158081 - rollout_corr/rollout_is_seq_max_deviation:0.02120816707611084 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2931092074140906) - rollout_corr/training_log_ppl:np.float64(0.24355409591225907) - rollout_corr/kl:np.float64(0.0005864815489502462) - rollout_corr/k3_kl:np.float64(0.0005580453379678829) - rollout_corr/rollout_ppl:np.float64(1.292321430053562) - rollout_corr/rollout_log_ppl:np.float64(0.24296761279401835) - rollout_corr/log_ppl_diff:np.float64(0.0005864831182407215) - rollout_corr/log_ppl_abs_diff:np.float64(0.001289499065023847) - rollout_corr/log_ppl_diff_max:np.float64(0.019712358713150024) - rollout_corr/log_ppl_diff_min:np.float64(-0.007148668169975281) - rollout_corr/ppl_ratio:np.float64(1.0005892012268305) - rollout_corr/chi2_token:np.float64(0.0012211541179567575) - rollout_corr/chi2_seq:np.float64(0.6782732140272856) - actor/pg_loss:np.float64(0.00016800372395664454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002511092553731942) - actor/ppo_kl:np.float64(0.00027240615360524245) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29514269530773163) - perf/mfu/actor:np.float64(0.12364022737507745) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(104.99796676635742) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005714894738048315 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005714894738048315 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:769.64453125 - response_length/max:2870.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:769.64453125 - response_length_non_aborted/max:2870.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.717978537082672e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7690473347902298) - timing_s/agent_loop/generate_sequences/max:np.float64(21.679362626746297) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.051158084097551) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.679362626746297) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:2870 - timing_s/gen:28.594687478616834 - timing_s/reward:0.07660721056163311 - timing_s/old_log_prob:3.3768766596913338 - timing_s/adv:0.14736606925725937 - timing_s/update_actor:14.125624496489763 - timing_s/step:46.4090982247144 - timing_s/stop_profile:0.00013731606304645538 - timing_per_token_ms/gen:0.1451293336443713 - timing_per_token_ms/update_actor:0.0536367914902196 - timing_per_token_ms/adv:0.0005595676942601084 - perf/total_num_tokens:263357 - perf/time_per_step:46.4090982247144 - perf/throughput:709.3355884788383 (TaskRunner pid=2281424) Training Progress: 22%|██▏ | 22/100 [20:55<1:29:44, 69.04s/it] (TaskRunner pid=2281424) step:23 - global_seqlen/min:22934 - global_seqlen/max:53809 - global_seqlen/minmax_diff:30875 - global_seqlen/balanced_min:39243 - global_seqlen/balanced_max:39551 - global_seqlen/mean:39306.5 - actor/entropy:0.33397042751312256 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4463748633861542 - training/rollout_probs_diff_mean:0.003143494250252843 - training/rollout_probs_diff_std:0.008556975051760674 - training/rollout_actor_probs_pearson_corr:0.9994155764579773 - rollout_corr/rollout_is_mean:0.9999798536300659 - rollout_corr/rollout_is_ratio_fraction_high:8.167132364178542e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6334264728357084e-05 - rollout_corr/rollout_is_max:2.7373318672180176 - rollout_corr/rollout_is_min:0.23669613897800446 - rollout_corr/rollout_is_std:0.028452597558498383 - rollout_corr/rollout_is_eff_sample_size:0.9991909855397003 - rollout_corr/rollout_is_seq_mean:0.9999604821205139 - rollout_corr/rollout_is_seq_std:0.001166414120234549 - rollout_corr/rollout_is_seq_max:1.0046206712722778 - rollout_corr/rollout_is_seq_min:0.9961167573928833 - rollout_corr/rollout_is_seq_max_deviation:0.004620671272277832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.385243427939713) - rollout_corr/training_log_ppl:np.float64(0.3101144570391625) - rollout_corr/kl:np.float64(0.0005338342346767888) - rollout_corr/k3_kl:np.float64(0.000465180856878078) - rollout_corr/rollout_ppl:np.float64(1.3844622108153999) - rollout_corr/rollout_log_ppl:np.float64(0.30958062058198266) - rollout_corr/log_ppl_diff:np.float64(0.0005338364571798593) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009809565672185272) - rollout_corr/log_ppl_diff_max:np.float64(0.006667211651802063) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032280683517456055) - rollout_corr/ppl_ratio:np.float64(1.0005347987171263) - rollout_corr/chi2_token:np.float64(0.000802481546998024) - rollout_corr/chi2_seq:np.float64(0.3838173961266875) - actor/pg_loss:np.float64(6.97027426213026e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030751674410112173) - actor/ppo_kl:np.float64(6.243631206404032e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20673737302422523) - perf/mfu/actor:np.float64(0.1370845535950862) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.68359375) - perf/cpu_memory_used_gb:np.float64(105.6169319152832) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0067174662835896015 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0067174662835896015 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:956.578125 - response_length/max:4024.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:956.578125 - response_length_non_aborted/max:4024.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00028231181204319 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8497151639312506) - timing_s/agent_loop/generate_sequences/max:np.float64(29.862245429307222) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.941006362365442) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.862245429307222) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:4024 - timing_s/gen:36.85486473329365 - timing_s/reward:0.08099697157740593 - timing_s/old_log_prob:3.847181210294366 - timing_s/adv:0.1430294755846262 - timing_s/update_actor:15.214192001149058 - timing_s/step:56.236885111778975 - timing_s/stop_profile:0.00012627430260181427 - timing_per_token_ms/gen:0.1504992761196879 - timing_per_token_ms/update_actor:0.04838319362303009 - timing_per_token_ms/adv:0.00045485312729645925 - perf/total_num_tokens:314452 - perf/time_per_step:56.236885111778975 - perf/throughput:698.9451837859195 (TaskRunner pid=2281424) Training Progress: 23%|██▎ | 23/100 [21:52<1:23:49, 65.31s/it] (TaskRunner pid=2281424) step:24 - global_seqlen/min:21247 - global_seqlen/max:40900 - global_seqlen/minmax_diff:19653 - global_seqlen/balanced_min:30630 - global_seqlen/balanced_max:32071 - global_seqlen/mean:30857.125 - actor/entropy:0.31415796279907227 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3094596862792969 - training/rollout_probs_diff_mean:0.003211991162970662 - training/rollout_probs_diff_std:0.008898185566067696 - training/rollout_actor_probs_pearson_corr:0.9993066191673279 - rollout_corr/rollout_is_mean:0.9999200105667114 - rollout_corr/rollout_is_ratio_fraction_high:5.59005866307416e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.59005866307416e-06 - rollout_corr/rollout_is_max:2.404808521270752 - rollout_corr/rollout_is_min:0.19617219269275665 - rollout_corr/rollout_is_std:0.02825077250599861 - rollout_corr/rollout_is_eff_sample_size:0.9992022922335758 - rollout_corr/rollout_is_seq_mean:0.9998365640640259 - rollout_corr/rollout_is_seq_std:0.001564831123687327 - rollout_corr/rollout_is_seq_max:1.0052552223205566 - rollout_corr/rollout_is_seq_min:0.992292582988739 - rollout_corr/rollout_is_seq_max_deviation:0.007707417011260986 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.303541206754744) - rollout_corr/training_log_ppl:np.float64(0.25006278717773966) - rollout_corr/kl:np.float64(0.0005476736968059726) - rollout_corr/k3_kl:np.float64(0.00043206283888252983) - rollout_corr/rollout_ppl:np.float64(1.3028357764706016) - rollout_corr/rollout_log_ppl:np.float64(0.24951511321705766) - rollout_corr/log_ppl_diff:np.float64(0.0005476739606820047) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011344742961227894) - rollout_corr/log_ppl_diff_max:np.float64(0.008412361145019531) - rollout_corr/log_ppl_diff_min:np.float64(-0.00302673876285553) - rollout_corr/ppl_ratio:np.float64(1.0005489422474056) - rollout_corr/chi2_token:np.float64(0.0006379920523613691) - rollout_corr/chi2_seq:np.float64(0.4201601871754974) - actor/pg_loss:np.float64(-3.0018971301615238e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.42312039076387e-05) - actor/ppo_kl:np.float64(-2.9987399832176465e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1152929738163948) - perf/mfu/actor:np.float64(0.11669484239312708) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(138.80078125) - perf/cpu_memory_used_gb:np.float64(105.66312026977539) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013332289643585682 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013332289643585682 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:698.78515625 - response_length/max:4217.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:698.78515625 - response_length_non_aborted/max:4217.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001115109771490097 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1027332954108715) - timing_s/agent_loop/generate_sequences/max:np.float64(28.570936573669314) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.08747353394574) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.570936573669314) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:4217 - timing_s/gen:35.1314688809216 - timing_s/reward:0.07224285788834095 - timing_s/old_log_prob:3.3607450742274523 - timing_s/adv:0.1696465015411377 - timing_s/update_actor:13.863193551078439 - timing_s/step:52.68583432585001 - timing_s/stop_profile:0.00012148357927799225 - timing_per_token_ms/gen:0.19638697114367906 - timing_per_token_ms/update_actor:0.05615880267149985 - timing_per_token_ms/adv:0.0006872258090357482 - perf/total_num_tokens:246857 - perf/time_per_step:52.68583432585001 - perf/throughput:585.6816238147742 (TaskRunner pid=2281424) Training Progress: 24%|██▍ | 24/100 [22:45<1:17:57, 61.54s/it] (TaskRunner pid=2281424) step:25 - global_seqlen/min:23246 - global_seqlen/max:52171 - global_seqlen/minmax_diff:28925 - global_seqlen/balanced_min:34912 - global_seqlen/balanced_max:36421 - global_seqlen/mean:35164.75 - actor/entropy:0.35677048563957214 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5146784782409668 - training/rollout_probs_diff_mean:0.0033477256074547768 - training/rollout_probs_diff_std:0.00895012728869915 - training/rollout_actor_probs_pearson_corr:0.9994227886199951 - rollout_corr/rollout_is_mean:0.9999942779541016 - rollout_corr/rollout_is_ratio_fraction_high:1.4294563698058482e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4294563698058482e-05 - rollout_corr/rollout_is_max:2.550842046737671 - rollout_corr/rollout_is_min:0.2724951505661011 - rollout_corr/rollout_is_std:0.02962060645222664 - rollout_corr/rollout_is_eff_sample_size:0.9991233887504097 - rollout_corr/rollout_is_seq_mean:1.0000044107437134 - rollout_corr/rollout_is_seq_std:0.001464587403461337 - rollout_corr/rollout_is_seq_max:1.0044803619384766 - rollout_corr/rollout_is_seq_min:0.9913833737373352 - rollout_corr/rollout_is_seq_max_deviation:0.008616626262664795 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3856255286373198) - rollout_corr/training_log_ppl:np.float64(0.3002330707240617) - rollout_corr/kl:np.float64(0.00044336768771691126) - rollout_corr/k3_kl:np.float64(0.0005020772942430085) - rollout_corr/rollout_ppl:np.float64(1.3849827772937715) - rollout_corr/rollout_log_ppl:np.float64(0.2997897020104574) - rollout_corr/log_ppl_diff:np.float64(0.0004433687136042863) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010328854841645807) - rollout_corr/log_ppl_diff_max:np.float64(0.010214567184448242) - rollout_corr/log_ppl_diff_min:np.float64(-0.005682602524757385) - rollout_corr/ppl_ratio:np.float64(1.000444594072178) - rollout_corr/chi2_token:np.float64(0.0011366589460521936) - rollout_corr/chi2_seq:np.float64(0.17277534841559827) - actor/pg_loss:np.float64(-0.00010206212755292654) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003034779521726705) - actor/ppo_kl:np.float64(-3.511990449567293e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1835123896598816) - perf/mfu/actor:np.float64(0.1283748813468745) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.58767604827881) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.025912469252943993 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.025912469252943993 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:819.8046875 - response_length/max:5123.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:819.8046875 - response_length_non_aborted/max:5123.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001536533236503601 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2272972408682108) - timing_s/agent_loop/generate_sequences/max:np.float64(33.647482462227345) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.377409354601696) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(33.647482462227345) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:5123 - timing_s/gen:40.239728681743145 - timing_s/reward:0.07882356271147728 - timing_s/old_log_prob:3.5549416448920965 - timing_s/adv:0.13953095860779285 - timing_s/update_actor:14.543323775753379 - timing_s/step:58.64243852160871 - timing_s/stop_profile:0.00014534033834934235 - timing_per_token_ms/gen:0.1917364496199702 - timing_per_token_ms/update_actor:0.05169709643802878 - timing_per_token_ms/adv:0.0004959901556522969 - perf/total_num_tokens:281318 - perf/time_per_step:58.64243852160871 - perf/throughput:599.646789705759 (TaskRunner pid=2281424) Training Progress: 25%|██▌ | 25/100 [23:43<1:15:51, 60.69s/it] (TaskRunner pid=2281424) step:26 - global_seqlen/min:24712 - global_seqlen/max:39123 - global_seqlen/minmax_diff:14411 - global_seqlen/balanced_min:30297 - global_seqlen/balanced_max:30365 - global_seqlen/mean:30344.375 - actor/entropy:0.2876966595649719 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20822957158088684 - training/rollout_probs_diff_mean:0.0029764524661004543 - training/rollout_probs_diff_std:0.008548058569431305 - training/rollout_actor_probs_pearson_corr:0.9995021820068359 - rollout_corr/rollout_is_mean:1.0000873804092407 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9248120784759521 - rollout_corr/rollout_is_min:0.5073025822639465 - rollout_corr/rollout_is_std:0.02774408645927906 - rollout_corr/rollout_is_eff_sample_size:0.9992309766786769 - rollout_corr/rollout_is_seq_mean:1.0000227689743042 - rollout_corr/rollout_is_seq_std:0.0015661150682717562 - rollout_corr/rollout_is_seq_max:1.0046665668487549 - rollout_corr/rollout_is_seq_min:0.9934303760528564 - rollout_corr/rollout_is_seq_max_deviation:0.006569623947143555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3186388644389808) - rollout_corr/training_log_ppl:np.float64(0.2604680637450656) - rollout_corr/kl:np.float64(0.0005275833006166408) - rollout_corr/k3_kl:np.float64(0.0005207556534685409) - rollout_corr/rollout_ppl:np.float64(1.3179062088020146) - rollout_corr/rollout_log_ppl:np.float64(0.259940478586941) - rollout_corr/log_ppl_diff:np.float64(0.0005275851581245661) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012775544892065227) - rollout_corr/log_ppl_diff_max:np.float64(0.009042024612426758) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037787258625030518) - rollout_corr/ppl_ratio:np.float64(1.0005293833091855) - rollout_corr/chi2_token:np.float64(0.0010106251575052738) - rollout_corr/chi2_seq:np.float64(1.5079653931315988) - actor/pg_loss:np.float64(0.00019484246149659157) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004149369976857997) - actor/ppo_kl:np.float64(0.00010909117361990184) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17218420654535294) - perf/mfu/actor:np.float64(0.11675351697015622) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.56155776977539) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007410742342472076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007410742342472076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:688.73046875 - response_length/max:2773.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:688.73046875 - response_length_non_aborted/max:2773.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011996924877166748 - timing_s/agent_loop/generate_sequences/min:np.float64(1.479305861517787) - timing_s/agent_loop/generate_sequences/max:np.float64(20.65613953024149) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.7965023621873115) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.65613953024149) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:295 - timing_s/agent_loop/slowest/response_length:2773 - timing_s/gen:27.100659066811204 - timing_s/reward:0.07199572958052158 - timing_s/old_log_prob:3.321561750024557 - timing_s/adv:0.16026310436427593 - timing_s/update_actor:13.673425955697894 - timing_s/step:44.414213774725795 - timing_s/stop_profile:0.00011827610433101654 - timing_per_token_ms/gen:0.1537059187636401 - timing_per_token_ms/update_actor:0.05632603223702043 - timing_per_token_ms/adv:0.0006601845661851494 - perf/total_num_tokens:242755 - perf/time_per_step:44.414213774725795 - perf/throughput:683.2131523009796 (TaskRunner pid=2281424) Training Progress: 26%|██▌ | 26/100 [24:28<1:08:50, 55.82s/it] (TaskRunner pid=2281424) step:27 - global_seqlen/min:25977 - global_seqlen/max:48183 - global_seqlen/minmax_diff:22206 - global_seqlen/balanced_min:36093 - global_seqlen/balanced_max:36153 - global_seqlen/mean:36121.125 - actor/entropy:0.30439427495002747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2310372292995453 - training/rollout_probs_diff_mean:0.002941888989880681 - training/rollout_probs_diff_std:0.008349236100912094 - training/rollout_actor_probs_pearson_corr:0.99952632188797 - rollout_corr/rollout_is_mean:0.9999545216560364 - rollout_corr/rollout_is_ratio_fraction_high:4.5820902414561715e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.3746271179115865e-05 - rollout_corr/rollout_is_max:2.7575740814208984 - rollout_corr/rollout_is_min:0.4973885416984558 - rollout_corr/rollout_is_std:0.027530577033758163 - rollout_corr/rollout_is_eff_sample_size:0.9992425223351087 - rollout_corr/rollout_is_seq_mean:0.9999428987503052 - rollout_corr/rollout_is_seq_std:0.0012769426684826612 - rollout_corr/rollout_is_seq_max:1.004395604133606 - rollout_corr/rollout_is_seq_min:0.9921810030937195 - rollout_corr/rollout_is_seq_max_deviation:0.007818996906280518 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3105780305340886) - rollout_corr/training_log_ppl:np.float64(0.25283126058639027) - rollout_corr/kl:np.float64(0.00047030424940075477) - rollout_corr/k3_kl:np.float64(0.0004522940477897919) - rollout_corr/rollout_ppl:np.float64(1.3099122606217861) - rollout_corr/rollout_log_ppl:np.float64(0.2523609542840859) - rollout_corr/log_ppl_diff:np.float64(0.00047030630230437964) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010534748871577904) - rollout_corr/log_ppl_diff_max:np.float64(0.009107545018196106) - rollout_corr/log_ppl_diff_min:np.float64(-0.00359538197517395) - rollout_corr/ppl_ratio:np.float64(1.0004714394453913) - rollout_corr/chi2_token:np.float64(0.0008890700992196798) - rollout_corr/chi2_seq:np.float64(1.7405338953249156) - actor/pg_loss:np.float64(-9.15073323994875e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002051412569699096) - actor/ppo_kl:np.float64(1.4041290314459332e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15001486241817474) - perf/mfu/actor:np.float64(0.13349702138291186) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.56867599487305) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006906355731189251 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006906355731189251 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:852.50390625 - response_length/max:3307.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:852.50390625 - response_length_non_aborted/max:3307.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015197880566120148 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5302176997065544) - timing_s/agent_loop/generate_sequences/max:np.float64(24.74932035431266) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.707113990400103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.74932035431266) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:3281 - timing_s/gen:31.510395139455795 - timing_s/reward:0.07958226092159748 - timing_s/old_log_prob:3.521709840744734 - timing_s/adv:0.14874472096562386 - timing_s/update_actor:14.358893560245633 - timing_s/step:49.70740628428757 - timing_s/stop_profile:0.00017487630248069763 - timing_per_token_ms/gen:0.144383480370122 - timing_per_token_ms/update_actor:0.049690082881712686 - timing_per_token_ms/adv:0.0005147428304268757 - perf/total_num_tokens:288969 - perf/time_per_step:49.70740628428757 - perf/throughput:726.6749102420544 (TaskRunner pid=2281424) Training Progress: 27%|██▋ | 27/100 [25:18<1:05:42, 54.01s/it] (TaskRunner pid=2281424) step:28 - global_seqlen/min:21975 - global_seqlen/max:58966 - global_seqlen/minmax_diff:36991 - global_seqlen/balanced_min:38635 - global_seqlen/balanced_max:38827 - global_seqlen/mean:38769.375 - actor/entropy:0.3461316227912903 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34711864590644836 - training/rollout_probs_diff_mean:0.003255465067923069 - training/rollout_probs_diff_std:0.008680691011250019 - training/rollout_actor_probs_pearson_corr:0.9994996190071106 - rollout_corr/rollout_is_mean:1.00006103515625 - rollout_corr/rollout_is_ratio_fraction_high:8.337119652424008e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.084279913106002e-05 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.17730121314525604 - rollout_corr/rollout_is_std:0.028798246756196022 - rollout_corr/rollout_is_eff_sample_size:0.9991714672176878 - rollout_corr/rollout_is_seq_mean:1.0000207424163818 - rollout_corr/rollout_is_seq_std:0.0014608351048082113 - rollout_corr/rollout_is_seq_max:1.006380558013916 - rollout_corr/rollout_is_seq_min:0.9955966472625732 - rollout_corr/rollout_is_seq_max_deviation:0.006380558013916016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3715200861915946) - rollout_corr/training_log_ppl:np.float64(0.2959491069341311) - rollout_corr/kl:np.float64(0.0004920310259990401) - rollout_corr/k3_kl:np.float64(0.000492703045580356) - rollout_corr/rollout_ppl:np.float64(1.3707749480381608) - rollout_corr/rollout_log_ppl:np.float64(0.2954570743313525) - rollout_corr/log_ppl_diff:np.float64(0.0004920326027786359) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011141302093164995) - rollout_corr/log_ppl_diff_max:np.float64(0.00506967306137085) - rollout_corr/log_ppl_diff_min:np.float64(-0.004320159554481506) - rollout_corr/ppl_ratio:np.float64(1.0004931495059282) - rollout_corr/chi2_token:np.float64(0.0010096586775034666) - rollout_corr/chi2_seq:np.float64(1.0986658702604473) - actor/pg_loss:np.float64(-3.765069413930178e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001984241041554924) - actor/ppo_kl:np.float64(4.411382598945579e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1757737323641777) - perf/mfu/actor:np.float64(0.13821070890339499) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.6130256652832) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01265887450426817 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01265887450426817 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:937.07421875 - response_length/max:4442.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:937.07421875 - response_length_non_aborted/max:4442.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013406015932559967 - timing_s/agent_loop/generate_sequences/min:np.float64(1.040765456855297) - timing_s/agent_loop/generate_sequences/max:np.float64(31.025565603747964) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.124727715730842) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(31.025565603747964) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:4440 - timing_s/gen:37.81342346780002 - timing_s/reward:0.08299151435494423 - timing_s/old_log_prob:3.631516432389617 - timing_s/adv:0.1339629665017128 - timing_s/update_actor:15.046643575653434 - timing_s/step:56.79524556733668 - timing_s/stop_profile:0.00011165253818035126 - timing_per_token_ms/gen:0.1576275202812945 - timing_per_token_ms/update_actor:0.04851330326982778 - timing_per_token_ms/adv:0.00043192264029827924 - perf/total_num_tokens:310155 - perf/time_per_step:56.79524556733668 - perf/throughput:682.61655729677 (TaskRunner pid=2281424) Training Progress: 28%|██▊ | 28/100 [26:14<1:05:49, 54.86s/it] (TaskRunner pid=2281424) step:29 - global_seqlen/min:26468 - global_seqlen/max:47121 - global_seqlen/minmax_diff:20653 - global_seqlen/balanced_min:34309 - global_seqlen/balanced_max:34448 - global_seqlen/mean:34354.375 - actor/entropy:0.2958447337150574 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135512113571167 - training/rollout_probs_diff_mean:0.003082486568018794 - training/rollout_probs_diff_std:0.00864423532038927 - training/rollout_actor_probs_pearson_corr:0.9994667768478394 - rollout_corr/rollout_is_mean:0.9999562501907349 - rollout_corr/rollout_is_ratio_fraction_high:4.822275059268577e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.446682472305838e-05 - rollout_corr/rollout_is_max:2.0740530490875244 - rollout_corr/rollout_is_min:0.37482571601867676 - rollout_corr/rollout_is_std:0.027990059927105904 - rollout_corr/rollout_is_eff_sample_size:0.9992171698522248 - rollout_corr/rollout_is_seq_mean:0.9999083280563354 - rollout_corr/rollout_is_seq_std:0.0016167733119800687 - rollout_corr/rollout_is_seq_max:1.008034586906433 - rollout_corr/rollout_is_seq_min:0.993806004524231 - rollout_corr/rollout_is_seq_max_deviation:0.008034586906433105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3131270464509726) - rollout_corr/training_log_ppl:np.float64(0.26043001232028473) - rollout_corr/kl:np.float64(0.0006687698600753578) - rollout_corr/k3_kl:np.float64(0.0005304543523152461) - rollout_corr/rollout_ppl:np.float64(1.3122303052805364) - rollout_corr/rollout_log_ppl:np.float64(0.2597612422541715) - rollout_corr/log_ppl_diff:np.float64(0.0006687700661132112) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013376595597947016) - rollout_corr/log_ppl_diff_max:np.float64(0.009146973490715027) - rollout_corr/log_ppl_diff_min:np.float64(-0.004970654845237732) - rollout_corr/ppl_ratio:np.float64(1.0006705992855132) - rollout_corr/chi2_token:np.float64(0.0007638768292963505) - rollout_corr/chi2_seq:np.float64(0.7327017430216074) - actor/pg_loss:np.float64(4.302803426980972e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002477214987948173) - actor/ppo_kl:np.float64(0.00011317849087255638) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1297485176473856) - perf/mfu/actor:np.float64(0.1281224223428748) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(139.953125) - perf/cpu_memory_used_gb:np.float64(105.53421020507812) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04238719493150711 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04238719493150711 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:810.04296875 - response_length/max:3813.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:810.04296875 - response_length_non_aborted/max:3813.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014430470764636993 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3452266938984394) - timing_s/agent_loop/generate_sequences/max:np.float64(27.088402163237333) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.822816854313714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.088402163237333) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:3813 - timing_s/gen:33.822356663644314 - timing_s/reward:0.073737021535635 - timing_s/old_log_prob:3.5272585432976484 - timing_s/adv:0.16648594662547112 - timing_s/update_actor:14.246411005035043 - timing_s/step:51.92763191089034 - timing_s/stop_profile:6.692670285701752e-05 - timing_per_token_ms/gen:0.16310070677020563 - timing_per_token_ms/update_actor:0.051836232667000354 - timing_per_token_ms/adv:0.0006057669024158899 - perf/total_num_tokens:274835 - perf/time_per_step:51.92763191089034 - perf/throughput:661.5817770961196 (TaskRunner pid=2281424) Training Progress: 29%|██▉ | 29/100 [27:06<1:03:53, 53.99s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] (TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We can solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Substituting the values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0 \, \mathrm{N/C}) \cdot (0.50 \, \mathrm{m})^2}{8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2}} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0) \cdot (0.25)}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now, convert coulombs to picocoulombs (1 pC = $1 \times 10^{-12} \, \mathrm{C}$): (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest option is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2281424) step:30 - global_seqlen/min:21152 - global_seqlen/max:43378 - global_seqlen/minmax_diff:22226 - global_seqlen/balanced_min:32690 - global_seqlen/balanced_max:36508 - global_seqlen/mean:33253.625 - actor/entropy:0.3238883912563324 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535161018371582 - training/rollout_probs_diff_mean:0.0032308220397681 - training/rollout_probs_diff_std:0.00875203125178814 - training/rollout_actor_probs_pearson_corr:0.9993578195571899 - rollout_corr/rollout_is_mean:1.0000488758087158 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9363911151885986 - rollout_corr/rollout_is_min:0.5356025099754333 - rollout_corr/rollout_is_std:0.028555061668157578 - rollout_corr/rollout_is_eff_sample_size:0.9991853917962915 - rollout_corr/rollout_is_seq_mean:1.0000839233398438 - rollout_corr/rollout_is_seq_std:0.0014182842569425702 - rollout_corr/rollout_is_seq_max:1.00834321975708 - rollout_corr/rollout_is_seq_min:0.9960029721260071 - rollout_corr/rollout_is_seq_max_deviation:0.008343219757080078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3260084195062518) - rollout_corr/training_log_ppl:np.float64(0.26798260471878166) - rollout_corr/kl:np.float64(0.00034142893949073994) - rollout_corr/k3_kl:np.float64(0.00046080480333898777) - rollout_corr/rollout_ppl:np.float64(1.3255186867900193) - rollout_corr/rollout_log_ppl:np.float64(0.2676411745214864) - rollout_corr/log_ppl_diff:np.float64(0.00034143019729526713) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010471087371115573) - rollout_corr/log_ppl_diff_max:np.float64(0.0044901371002197266) - rollout_corr/log_ppl_diff_min:np.float64(-0.006970256567001343) - rollout_corr/ppl_ratio:np.float64(1.0003424619790167) - rollout_corr/chi2_token:np.float64(0.0011780557688325644) - rollout_corr/chi2_seq:np.float64(9.059563745046034) - actor/pg_loss:np.float64(-7.204711437225342e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002318803169032435) - actor/ppo_kl:np.float64(-2.2715349961899278e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1330014280974865) - perf/mfu/actor:np.float64(0.11964730091498024) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.54463195800781) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6046875) - val-aux/sciknoweval/reward/std@16:np.float64(0.19331331768802099) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6854) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1699013098775457) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5269125000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1573208332576977) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6056250000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19436455511085113) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.75095) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13208485427187178) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4644124999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11165691249499834) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6343625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16176855495856937) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8039375) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0971781630054231) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.42205000000000004) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07114364007214655) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6537749999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12550218875311617) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.84275) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.061103157160745235) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.39563750000000003) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.038223937353486045) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.666125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09165205807525627) - val-aux/sciknoweval/score/mean@16:np.float64(0.6046875) - val-aux/sciknoweval/score/std@16:np.float64(0.19331331768802099) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6854) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1699013098775457) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5269125000000001) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1573208332576977) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6056250000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.19436455511085113) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.75095) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13208485427187178) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4644124999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11165691249499834) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6343625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.16176855495856937) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8039375) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0971781630054231) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.42205000000000004) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07114364007214655) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6537749999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12550218875311617) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.84275) - val-aux/sciknoweval/score/best@16/std:np.float64(0.061103157160745235) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.39563750000000003) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.038223937353486045) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.666125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09165205807525627) - val-core/sciknoweval/acc/mean@16:np.float64(0.6046875) - val-aux/sciknoweval/acc/std@16:np.float64(0.19331331768802099) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6854) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1699013098775457) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5269125000000001) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1573208332576977) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6056250000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.19436455511085113) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.75095) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13208485427187178) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4644124999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11165691249499834) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6343625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.16176855495856937) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8039375) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0971781630054231) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.42205000000000004) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07114364007214655) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6537749999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12550218875311617) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.84275) - val-core/sciknoweval/acc/best@16/std:np.float64(0.061103157160745235) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.39563750000000003) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.038223937353486045) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.666125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09165205807525627) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9421875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.11810324679239408) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9847249999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.05478145372165707) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9000999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.14152552996328321) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9422875000000002) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.11789115899569932) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9971625) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.012950626299582524) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8410875000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.15275603349423345) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.965475) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.08685302210496705) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9997) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0022688782927956746) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7717375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.1420235663973301) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9818749999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.05503683300798294) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.70755) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.10363469468730849) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9918625000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.02568992930006094) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00041142108966596425 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00041142108966596425 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:779.73828125 - response_length/max:8192.0 - response_length/min:130.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:779.73828125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012162141501903534 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5774449706077576) - timing_s/agent_loop/generate_sequences/max:np.float64(62.31554364040494) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.508535967179341) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(62.31554364040494) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:323 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:69.08000029809773 - timing_s/reward:0.07287852838635445 - timing_s/old_log_prob:3.7097500916570425 - timing_s/adv:0.15018161572515965 - timing_s/update_actor:14.847722072154284 - timing_s/step:87.9640430957079 - timing_s/testing:149.87052939273417 - timing_s/save_checkpoint:12.872908538207412 - timing_s/stop_profile:0.00015308894217014313 - timing_per_token_ms/gen:0.346069646255994 - timing_per_token_ms/update_actor:0.05581241921803368 - timing_per_token_ms/adv:0.0005645309937080531 - perf/total_num_tokens:266029 - perf/time_per_step:87.9640430957079 - perf/throughput:378.03656846262646 (TaskRunner pid=2281424) Training Progress: 30%|███ | 30/100 [31:17<2:11:52, 113.03s/it] (TaskRunner pid=2281424) step:31 - global_seqlen/min:26976 - global_seqlen/max:51365 - global_seqlen/minmax_diff:24389 - global_seqlen/balanced_min:37711 - global_seqlen/balanced_max:37821 - global_seqlen/mean:37765.0 - actor/entropy:0.33241793513298035 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.301638126373291 - training/rollout_probs_diff_mean:0.003111757803708315 - training/rollout_probs_diff_std:0.008627519011497498 - training/rollout_actor_probs_pearson_corr:0.9994876980781555 - rollout_corr/rollout_is_mean:0.9999532699584961 - rollout_corr/rollout_is_ratio_fraction_high:8.658907972858287e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.2988362868782133e-05 - rollout_corr/rollout_is_max:2.167019844055176 - rollout_corr/rollout_is_min:0.3715561032295227 - rollout_corr/rollout_is_std:0.028119659051299095 - rollout_corr/rollout_is_eff_sample_size:0.9992096714770495 - rollout_corr/rollout_is_seq_mean:0.9999525547027588 - rollout_corr/rollout_is_seq_std:0.0012215663446113467 - rollout_corr/rollout_is_seq_max:1.004585862159729 - rollout_corr/rollout_is_seq_min:0.9942855834960938 - rollout_corr/rollout_is_seq_max_deviation:0.00571441650390625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3133017774671316) - rollout_corr/training_log_ppl:np.float64(0.25711654864426237) - rollout_corr/kl:np.float64(0.0004556128830559558) - rollout_corr/k3_kl:np.float64(0.00039641217307462284) - rollout_corr/rollout_ppl:np.float64(1.3126854803413153) - rollout_corr/rollout_log_ppl:np.float64(0.25666093576001003) - rollout_corr/log_ppl_diff:np.float64(0.00045561288425233215) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010043695365311578) - rollout_corr/log_ppl_diff_max:np.float64(0.006338976323604584) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035679787397384644) - rollout_corr/ppl_ratio:np.float64(1.0004564973060042) - rollout_corr/chi2_token:np.float64(0.0006840897258371115) - rollout_corr/chi2_seq:np.float64(0.9153507652226835) - actor/pg_loss:np.float64(1.1453521437942982e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001992799257095612) - actor/ppo_kl:np.float64(2.699381930071354e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14760523661971092) - perf/mfu/actor:np.float64(0.13769359300581646) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.38329315185547) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009976144880056381 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009976144880056381 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:902.25 - response_length/max:3456.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:902.25 - response_length_non_aborted/max:3456.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019257515668869019 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1455713659524918) - timing_s/agent_loop/generate_sequences/max:np.float64(25.956687213853) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.312753760583291) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.956687213853) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:3456 - timing_s/gen:32.13823664188385 - timing_s/reward:0.0789014957845211 - timing_s/old_log_prob:3.81612709723413 - timing_s/adv:0.13714739680290222 - timing_s/update_actor:14.64492060430348 - timing_s/step:50.91287592239678 - timing_s/stop_profile:0.00011721253395080566 - timing_per_token_ms/gen:0.13914102175933366 - timing_per_token_ms/update_actor:0.048473853449965174 - timing_per_token_ms/adv:0.00045395007547630815 - perf/total_num_tokens:302120 - perf/time_per_step:50.91287592239678 - perf/throughput:741.7573514716152 (TaskRunner pid=2281424) Training Progress: 31%|███ | 31/100 [32:08<1:48:34, 94.42s/it] (TaskRunner pid=2281424) step:32 - global_seqlen/min:17749 - global_seqlen/max:40522 - global_seqlen/minmax_diff:22773 - global_seqlen/balanced_min:29749 - global_seqlen/balanced_max:30114 - global_seqlen/mean:29860.25 - actor/entropy:0.3001790940761566 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931319773197174 - training/rollout_probs_diff_mean:0.0032425629906356335 - training/rollout_probs_diff_std:0.009085338562726974 - training/rollout_actor_probs_pearson_corr:0.9993113279342651 - rollout_corr/rollout_is_mean:1.0000972747802734 - rollout_corr/rollout_is_ratio_fraction_high:5.759240593761206e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.3036962375044823e-05 - rollout_corr/rollout_is_max:2.2561657428741455 - rollout_corr/rollout_is_min:0.11440415680408478 - rollout_corr/rollout_is_std:0.028905672952532768 - rollout_corr/rollout_is_eff_sample_size:0.999165278640662 - rollout_corr/rollout_is_seq_mean:1.0000715255737305 - rollout_corr/rollout_is_seq_std:0.001591670559719205 - rollout_corr/rollout_is_seq_max:1.0057041645050049 - rollout_corr/rollout_is_seq_min:0.9924244284629822 - rollout_corr/rollout_is_seq_max_deviation:0.007575571537017822 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3085072333924472) - rollout_corr/training_log_ppl:np.float64(0.25189579716243315) - rollout_corr/kl:np.float64(0.0004256223451228891) - rollout_corr/k3_kl:np.float64(0.0005207587888378384) - rollout_corr/rollout_ppl:np.float64(1.307915877085179) - rollout_corr/rollout_log_ppl:np.float64(0.25147017439303454) - rollout_corr/log_ppl_diff:np.float64(0.00042562276939861476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012707264686468989) - rollout_corr/log_ppl_diff_max:np.float64(0.008575737476348877) - rollout_corr/log_ppl_diff_min:np.float64(-0.005954384803771973) - rollout_corr/ppl_ratio:np.float64(1.0004271555226296) - rollout_corr/chi2_token:np.float64(0.0012784942518919706) - rollout_corr/chi2_seq:np.float64(1.3941792568657547) - actor/pg_loss:np.float64(-4.464993253350258e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011340672926962725) - actor/ppo_kl:np.float64(3.421145366999667e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10089067462831736) - perf/mfu/actor:np.float64(0.11507569937509035) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.87258911132812) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02760116197168827 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02760116197168827 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:678.2578125 - response_length/max:3951.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:678.2578125 - response_length_non_aborted/max:3951.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010407157242298126 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1292062569409609) - timing_s/agent_loop/generate_sequences/max:np.float64(27.004830926656723) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.73606214641768) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.004830926656723) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:3951 - timing_s/gen:34.06722195446491 - timing_s/reward:0.06839162670075893 - timing_s/old_log_prob:3.350517366081476 - timing_s/adv:0.15078148432075977 - timing_s/update_actor:13.741796219721437 - timing_s/step:51.4739043507725 - timing_s/stop_profile:0.00011109933257102966 - timing_per_token_ms/gen:0.1962013312742027 - timing_per_token_ms/update_actor:0.05752545700271028 - timing_per_token_ms/adv:0.0006311965084048181 - perf/total_num_tokens:238882 - perf/time_per_step:51.4739043507725 - perf/throughput:580.104625375127 (TaskRunner pid=2281424) Training Progress: 32%|███▏ | 32/100 [33:00<1:32:25, 81.55s/it] (TaskRunner pid=2281424) step:33 - global_seqlen/min:27211 - global_seqlen/max:50719 - global_seqlen/minmax_diff:23508 - global_seqlen/balanced_min:36245 - global_seqlen/balanced_max:36302 - global_seqlen/mean:36278.125 - actor/entropy:0.30930498242378235 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351723432540894 - training/rollout_probs_diff_mean:0.0030934372916817665 - training/rollout_probs_diff_std:0.00848071463406086 - training/rollout_actor_probs_pearson_corr:0.9993887543678284 - rollout_corr/rollout_is_mean:0.9999690055847168 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3896545169700403e-05 - rollout_corr/rollout_is_max:1.8591357469558716 - rollout_corr/rollout_is_min:0.3342134654521942 - rollout_corr/rollout_is_std:0.027681713923811913 - rollout_corr/rollout_is_eff_sample_size:0.999234190391299 - rollout_corr/rollout_is_seq_mean:0.9999369382858276 - rollout_corr/rollout_is_seq_std:0.0011744012590497732 - rollout_corr/rollout_is_seq_max:1.003006100654602 - rollout_corr/rollout_is_seq_min:0.9960953593254089 - rollout_corr/rollout_is_seq_max_deviation:0.0039046406745910645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.354571030009538) - rollout_corr/training_log_ppl:np.float64(0.2829173822829034) - rollout_corr/kl:np.float64(0.0005067846795761) - rollout_corr/k3_kl:np.float64(0.00047523474137278754) - rollout_corr/rollout_ppl:np.float64(1.353815943468362) - rollout_corr/rollout_log_ppl:np.float64(0.28241059606079943) - rollout_corr/log_ppl_diff:np.float64(0.0005067862221039832) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010225145379081368) - rollout_corr/log_ppl_diff_max:np.float64(0.005544401705265045) - rollout_corr/log_ppl_diff_min:np.float64(-0.003694787621498108) - rollout_corr/ppl_ratio:np.float64(1.000507768476382) - rollout_corr/chi2_token:np.float64(0.0008780690841376781) - rollout_corr/chi2_seq:np.float64(0.740443660877645) - actor/pg_loss:np.float64(-8.333136793226004e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002747284138422401) - actor/ppo_kl:np.float64(2.7713305339016614e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16628609038889408) - perf/mfu/actor:np.float64(0.1349622504868255) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.00208282470703) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004228602629154921 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004228602629154921 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:843.28515625 - response_length/max:3164.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:843.28515625 - response_length_non_aborted/max:3164.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013750791549682617 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8266812413930893) - timing_s/agent_loop/generate_sequences/max:np.float64(24.079156171530485) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.841072585782968) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.079156171530485) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:292 - timing_s/agent_loop/slowest/response_length:3164 - timing_s/gen:30.762977486476302 - timing_s/reward:0.07941771484911442 - timing_s/old_log_prob:3.5987925808876753 - timing_s/adv:0.14997461810708046 - timing_s/update_actor:14.10401264578104 - timing_s/step:48.78351537883282 - timing_s/stop_profile:0.00021011196076869965 - timing_per_token_ms/gen:0.14249969884555055 - timing_per_token_ms/update_actor:0.04859682193395138 - timing_per_token_ms/adv:0.0005167529265469221 - perf/total_num_tokens:290225 - perf/time_per_step:48.78351537883282 - perf/throughput:743.6554073292776 (TaskRunner pid=2281424) Training Progress: 33%|███▎ | 33/100 [33:48<1:20:06, 71.74s/it] (TaskRunner pid=2281424) step:34 - global_seqlen/min:25666 - global_seqlen/max:73494 - global_seqlen/minmax_diff:47828 - global_seqlen/balanced_min:35473 - global_seqlen/balanced_max:36595 - global_seqlen/mean:35820.75 - actor/entropy:0.35148361325263977 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45545026659965515 - training/rollout_probs_diff_mean:0.0031844459008425474 - training/rollout_probs_diff_std:0.00856610108166933 - training/rollout_actor_probs_pearson_corr:0.9994761347770691 - rollout_corr/rollout_is_mean:0.9999700784683228 - rollout_corr/rollout_is_ratio_fraction_high:9.077787581190933e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8155575162381865e-05 - rollout_corr/rollout_is_max:2.0403316020965576 - rollout_corr/rollout_is_min:0.33807873725891113 - rollout_corr/rollout_is_std:0.02805599570274353 - rollout_corr/rollout_is_eff_sample_size:0.9992133611396591 - rollout_corr/rollout_is_seq_mean:0.9998204112052917 - rollout_corr/rollout_is_seq_std:0.0015267465496435761 - rollout_corr/rollout_is_seq_max:1.003962516784668 - rollout_corr/rollout_is_seq_min:0.9919791221618652 - rollout_corr/rollout_is_seq_max_deviation:0.008020877838134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.309415325988084) - rollout_corr/training_log_ppl:np.float64(0.25629871332785115) - rollout_corr/kl:np.float64(0.0007180621188593861) - rollout_corr/k3_kl:np.float64(0.000561066810490729) - rollout_corr/rollout_ppl:np.float64(1.3084843065589666) - rollout_corr/rollout_log_ppl:np.float64(0.2555806517775636) - rollout_corr/log_ppl_diff:np.float64(0.0007180615502875298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012585615331772715) - rollout_corr/log_ppl_diff_max:np.float64(0.009126394987106323) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031400546431541443) - rollout_corr/ppl_ratio:np.float64(1.0007198799867183) - rollout_corr/chi2_token:np.float64(0.0008239294402301311) - rollout_corr/chi2_seq:np.float64(1.3956339918076992) - actor/pg_loss:np.float64(6.344064604490995e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036903620963357753) - actor/ppo_kl:np.float64(0.00010281039390358515) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1495772898197174) - perf/mfu/actor:np.float64(0.12881182116660295) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.90487289428711) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018949881196022034 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0018949881196022034 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:860.6171875 - response_length/max:6530.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:860.6171875 - response_length_non_aborted/max:6530.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010554678738117218 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5495978090912104) - timing_s/agent_loop/generate_sequences/max:np.float64(42.11380794644356) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.242849451620714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(42.11380794644356) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:6530 - timing_s/gen:48.750448731705546 - timing_s/reward:0.07960104756057262 - timing_s/old_log_prob:3.7480743881314993 - timing_s/adv:0.1472205240279436 - timing_s/update_actor:14.973976226523519 - timing_s/step:67.78288715332747 - timing_s/stop_profile:3.6908313632011414e-05 - timing_per_token_ms/gen:0.22127310855992496 - timing_per_token_ms/update_actor:0.05225315015222852 - timing_per_token_ms/adv:0.0005137403740427811 - perf/total_num_tokens:286566 - perf/time_per_step:67.78288715332747 - perf/throughput:528.4630310740837 (TaskRunner pid=2281424) Training Progress: 34%|███▍ | 34/100 [34:56<1:17:37, 70.56s/it] (TaskRunner pid=2281424) step:35 - global_seqlen/min:22711 - global_seqlen/max:52151 - global_seqlen/minmax_diff:29440 - global_seqlen/balanced_min:33570 - global_seqlen/balanced_max:34828 - global_seqlen/mean:33923.125 - actor/entropy:0.356091171503067 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23818081617355347 - training/rollout_probs_diff_mean:0.003347822232171893 - training/rollout_probs_diff_std:0.008754725567996502 - training/rollout_actor_probs_pearson_corr:0.9993891716003418 - rollout_corr/rollout_is_mean:0.999956488609314 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.74996237346204e-06 - rollout_corr/rollout_is_max:1.6463863849639893 - rollout_corr/rollout_is_min:0.14987757802009583 - rollout_corr/rollout_is_std:0.02889123558998108 - rollout_corr/rollout_is_eff_sample_size:0.9991658736928141 - rollout_corr/rollout_is_seq_mean:0.9999445676803589 - rollout_corr/rollout_is_seq_std:0.0016311032231897116 - rollout_corr/rollout_is_seq_max:1.0061169862747192 - rollout_corr/rollout_is_seq_min:0.994307279586792 - rollout_corr/rollout_is_seq_max_deviation:0.006116986274719238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3848898322321475) - rollout_corr/training_log_ppl:np.float64(0.3058403903269209) - rollout_corr/kl:np.float64(0.0004393863629239547) - rollout_corr/k3_kl:np.float64(0.0005089315998816346) - rollout_corr/rollout_ppl:np.float64(1.3842959650792181) - rollout_corr/rollout_log_ppl:np.float64(0.3054010035120882) - rollout_corr/log_ppl_diff:np.float64(0.0004393868148326874) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012084643822163343) - rollout_corr/log_ppl_diff_max:np.float64(0.006951093673706055) - rollout_corr/log_ppl_diff_min:np.float64(-0.005402863025665283) - rollout_corr/ppl_ratio:np.float64(1.0004409453831613) - rollout_corr/chi2_token:np.float64(0.001150362892076373) - rollout_corr/chi2_seq:np.float64(1.6086886166594923) - actor/pg_loss:np.float64(2.0964303985238075e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002854640780469708) - actor/ppo_kl:np.float64(-8.599662149322285e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18192359805107117) - perf/mfu/actor:np.float64(0.12649445809002435) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.95371627807617) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00438565481454134 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00438565481454134 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:801.28515625 - response_length/max:5012.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:801.28515625 - response_length_non_aborted/max:5012.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.070756793022156e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6557252500206232) - timing_s/agent_loop/generate_sequences/max:np.float64(33.535721046850085) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.11258227860526) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(33.535721046850085) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:5012 - timing_s/gen:40.21909198909998 - timing_s/reward:0.07026992179453373 - timing_s/old_log_prob:3.5517083313316107 - timing_s/adv:0.14980567805469036 - timing_s/update_actor:14.293088844045997 - timing_s/step:58.31905349902809 - timing_s/stop_profile:2.7982518076896667e-05 - timing_per_token_ms/gen:0.19606731368602187 - timing_per_token_ms/update_actor:0.05266720284483666 - timing_per_token_ms/adv:0.0005520042672022786 - perf/total_num_tokens:271385 - perf/time_per_step:58.31905349902809 - perf/throughput:581.6816797372294 (TaskRunner pid=2281424) Training Progress: 35%|███▌ | 35/100 [35:55<1:12:28, 66.89s/it] (TaskRunner pid=2281424) step:36 - global_seqlen/min:33717 - global_seqlen/max:49536 - global_seqlen/minmax_diff:15819 - global_seqlen/balanced_min:43612 - global_seqlen/balanced_max:43672 - global_seqlen/mean:43658.375 - actor/entropy:0.362861692905426 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8421130180358887 - training/rollout_probs_diff_mean:0.003083376446738839 - training/rollout_probs_diff_std:0.00820560660213232 - training/rollout_actor_probs_pearson_corr:0.9995745420455933 - rollout_corr/rollout_is_mean:0.9999458193778992 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.136867225199239e-06 - rollout_corr/rollout_is_max:1.5991979837417603 - rollout_corr/rollout_is_min:0.010655050165951252 - rollout_corr/rollout_is_std:0.027175385504961014 - rollout_corr/rollout_is_eff_sample_size:0.9992619243999026 - rollout_corr/rollout_is_seq_mean:0.999880313873291 - rollout_corr/rollout_is_seq_std:0.0011923309648409486 - rollout_corr/rollout_is_seq_max:1.0057610273361206 - rollout_corr/rollout_is_seq_min:0.9923911094665527 - rollout_corr/rollout_is_seq_max_deviation:0.007608890533447266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3939749002456665) - rollout_corr/training_log_ppl:np.float64(0.3143607703241287) - rollout_corr/kl:np.float64(0.0004875935232391271) - rollout_corr/k3_kl:np.float64(0.00043940715849544176) - rollout_corr/rollout_ppl:np.float64(1.3933153660036623) - rollout_corr/rollout_log_ppl:np.float64(0.3138731736544287) - rollout_corr/log_ppl_diff:np.float64(0.0004875966696999967) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008989509078674018) - rollout_corr/log_ppl_diff_max:np.float64(0.008858025074005127) - rollout_corr/log_ppl_diff_min:np.float64(-0.003800034523010254) - rollout_corr/ppl_ratio:np.float64(1.0004885424859822) - rollout_corr/chi2_token:np.float64(0.0007559196092188358) - rollout_corr/chi2_seq:np.float64(0.28486403776332736) - actor/pg_loss:np.float64(4.073767922818661e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001381784221621274) - actor/ppo_kl:np.float64(-3.878047037275678e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11497985571622849) - perf/mfu/actor:np.float64(0.15230643812180578) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.80973815917969) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006595357321202755 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006595357321202755 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1094.66796875 - response_length/max:3514.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1094.66796875 - response_length_non_aborted/max:3514.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7200207114219666e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3092289455235004) - timing_s/agent_loop/generate_sequences/max:np.float64(27.297075459733605) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.346446073104744) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.297075459733605) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:3514 - timing_s/gen:34.776961313560605 - timing_s/reward:0.08570255897939205 - timing_s/old_log_prob:3.737231995910406 - timing_s/adv:0.14758271723985672 - timing_s/update_actor:15.371759809553623 - timing_s/step:54.21316165663302 - timing_s/stop_profile:2.7824193239212036e-05 - timing_per_token_ms/gen:0.12409927851110891 - timing_per_token_ms/update_actor:0.04401148636874833 - timing_per_token_ms/adv:0.00042254984650670326 - perf/total_num_tokens:349267 - perf/time_per_step:54.21316165663302 - perf/throughput:805.3095164697587 (TaskRunner pid=2281424) Training Progress: 36%|███▌ | 36/100 [36:49<1:07:18, 63.10s/it] (TaskRunner pid=2281424) step:37 - global_seqlen/min:34704 - global_seqlen/max:44098 - global_seqlen/minmax_diff:9394 - global_seqlen/balanced_min:39539 - global_seqlen/balanced_max:39624 - global_seqlen/mean:39597.875 - actor/entropy:0.35228487849235535 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413027286529541 - training/rollout_probs_diff_mean:0.003145897062495351 - training/rollout_probs_diff_std:0.0083078658208251 - training/rollout_actor_probs_pearson_corr:0.9995119571685791 - rollout_corr/rollout_is_mean:0.9999536275863647 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.998736247012857e-06 - rollout_corr/rollout_is_max:1.6415951251983643 - rollout_corr/rollout_is_min:0.39738908410072327 - rollout_corr/rollout_is_std:0.027896204963326454 - rollout_corr/rollout_is_eff_sample_size:0.9992224068794121 - rollout_corr/rollout_is_seq_mean:0.9999120235443115 - rollout_corr/rollout_is_seq_std:0.0014193912502378225 - rollout_corr/rollout_is_seq_max:1.0059196949005127 - rollout_corr/rollout_is_seq_min:0.9947477579116821 - rollout_corr/rollout_is_seq_max_deviation:0.005919694900512695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3658450348302722) - rollout_corr/training_log_ppl:np.float64(0.29802232861402445) - rollout_corr/kl:np.float64(0.0005081376050801722) - rollout_corr/k3_kl:np.float64(0.00047723877818839355) - rollout_corr/rollout_ppl:np.float64(1.365188850555569) - rollout_corr/rollout_log_ppl:np.float64(0.2975141915230779) - rollout_corr/log_ppl_diff:np.float64(0.0005081370909465477) - rollout_corr/log_ppl_abs_diff:np.float64(0.001064729891368188) - rollout_corr/log_ppl_diff_max:np.float64(0.005850955843925476) - rollout_corr/log_ppl_diff_min:np.float64(-0.00405728816986084) - rollout_corr/ppl_ratio:np.float64(1.000509355450049) - rollout_corr/chi2_token:np.float64(0.0009017251431941986) - rollout_corr/chi2_seq:np.float64(1.061360603896901) - actor/pg_loss:np.float64(6.26088585704565e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021849221991487866) - actor/ppo_kl:np.float64(-2.4668701797736503e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1671565007418394) - perf/mfu/actor:np.float64(0.14026788981271016) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.84344863891602) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011376905255019665 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011376905255019665 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:976.87109375 - response_length/max:3632.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:976.87109375 - response_length_non_aborted/max:3632.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6365742087364197e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.545783119276166) - timing_s/agent_loop/generate_sequences/max:np.float64(27.446757026016712) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.701898429557332) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.446757026016712) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:3632 - timing_s/gen:33.87001137807965 - timing_s/reward:0.0845334343612194 - timing_s/old_log_prob:3.622224699705839 - timing_s/adv:0.14574186690151691 - timing_s/update_actor:15.075419383123517 - timing_s/step:52.884285079315305 - timing_s/stop_profile:0.00012076087296009064 - timing_per_token_ms/gen:0.13543724734215848 - timing_per_token_ms/update_actor:0.0475891047913667 - timing_per_token_ms/adv:0.0004600684598021892 - perf/total_num_tokens:316783 - perf/time_per_step:52.884285079315305 - perf/throughput:748.7644947948435 (TaskRunner pid=2281424) Training Progress: 37%|███▋ | 37/100 [37:42<1:03:03, 60.05s/it] (TaskRunner pid=2281424) step:38 - global_seqlen/min:25876 - global_seqlen/max:56353 - global_seqlen/minmax_diff:30477 - global_seqlen/balanced_min:36571 - global_seqlen/balanced_max:36714 - global_seqlen/mean:36659.625 - actor/entropy:0.32807788252830505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9003579616546631 - training/rollout_probs_diff_mean:0.002977442229166627 - training/rollout_probs_diff_std:0.008599979802966118 - training/rollout_actor_probs_pearson_corr:0.9994491338729858 - rollout_corr/rollout_is_mean:0.9999948143959045 - rollout_corr/rollout_is_ratio_fraction_high:4.4448197513702326e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.3334459254110698e-05 - rollout_corr/rollout_is_max:3.384216547012329 - rollout_corr/rollout_is_min:0.005166077055037022 - rollout_corr/rollout_is_std:0.027199501171708107 - rollout_corr/rollout_is_eff_sample_size:0.999260734067485 - rollout_corr/rollout_is_seq_mean:0.9999963641166687 - rollout_corr/rollout_is_seq_std:0.0012283420655876398 - rollout_corr/rollout_is_seq_max:1.00580632686615 - rollout_corr/rollout_is_seq_min:0.9936442375183105 - rollout_corr/rollout_is_seq_max_deviation:0.006355762481689453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3002572190016508) - rollout_corr/training_log_ppl:np.float64(0.24828266966505907) - rollout_corr/kl:np.float64(0.0003652476746376365) - rollout_corr/k3_kl:np.float64(0.00043928875018650615) - rollout_corr/rollout_ppl:np.float64(1.2997558075003326) - rollout_corr/rollout_log_ppl:np.float64(0.2479174185282318) - rollout_corr/log_ppl_diff:np.float64(0.0003652511368272826) - rollout_corr/log_ppl_abs_diff:np.float64(0.000950878209550865) - rollout_corr/log_ppl_diff_max:np.float64(0.008495032787322998) - rollout_corr/log_ppl_diff_min:np.float64(-0.004545837640762329) - rollout_corr/ppl_ratio:np.float64(1.0003662216477096) - rollout_corr/chi2_token:np.float64(0.0010224031284451485) - rollout_corr/chi2_seq:np.float64(0.804062022594735) - actor/pg_loss:np.float64(-4.1241059079766273e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001981328330202814) - actor/ppo_kl:np.float64(-5.4830987617826565e-05) - actor/pg_clipfrac_lower:np.float64(3.98596921513672e-06) - actor/grad_norm:np.float64(0.12819637544453144) - perf/mfu/actor:np.float64(0.13466560382170992) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.90673160552979) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004230912774801254 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004230912774801254 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:878.83203125 - response_length/max:3854.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:878.83203125 - response_length_non_aborted/max:3854.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014106743037700653 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5907990671694279) - timing_s/agent_loop/generate_sequences/max:np.float64(27.61575059592724) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.814689949009335) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.61575059592724) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:3674 - timing_s/gen:35.19211849011481 - timing_s/reward:0.0795986708253622 - timing_s/old_log_prob:3.5315878726541996 - timing_s/adv:0.1513067688792944 - timing_s/update_actor:14.461832895874977 - timing_s/step:53.504376608878374 - timing_s/stop_profile:0.00011487491428852081 - timing_per_token_ms/gen:0.15642262453324862 - timing_per_token_ms/update_actor:0.04931117304076002 - timing_per_token_ms/adv:0.0005159176099022235 - perf/total_num_tokens:293277 - perf/time_per_step:53.504376608878374 - perf/throughput:685.1705846044154 (TaskRunner pid=2281424) Training Progress: 38%|███▊ | 38/100 [38:35<1:00:02, 58.10s/it] (TaskRunner pid=2281424) step:39 - global_seqlen/min:19213 - global_seqlen/max:40355 - global_seqlen/minmax_diff:21142 - global_seqlen/balanced_min:28562 - global_seqlen/balanced_max:29340 - global_seqlen/mean:28757.375 - actor/entropy:0.3156418800354004 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22961103916168213 - training/rollout_probs_diff_mean:0.003309106919914484 - training/rollout_probs_diff_std:0.00906699150800705 - training/rollout_actor_probs_pearson_corr:0.9993242025375366 - rollout_corr/rollout_is_mean:1.0000478029251099 - rollout_corr/rollout_is_ratio_fraction_high:1.219846944877645e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.099234724388225e-06 - rollout_corr/rollout_is_max:3.2739274501800537 - rollout_corr/rollout_is_min:0.49741330742836 - rollout_corr/rollout_is_std:0.029006537050008774 - rollout_corr/rollout_is_eff_sample_size:0.9991593281581235 - rollout_corr/rollout_is_seq_mean:1.0000423192977905 - rollout_corr/rollout_is_seq_std:0.0014898530207574368 - rollout_corr/rollout_is_seq_max:1.010595679283142 - rollout_corr/rollout_is_seq_min:0.9943899512290955 - rollout_corr/rollout_is_seq_max_deviation:0.01059567928314209 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3278274578042328) - rollout_corr/training_log_ppl:np.float64(0.26448297980823554) - rollout_corr/kl:np.float64(0.00044818446750127805) - rollout_corr/k3_kl:np.float64(0.00045055161507434605) - rollout_corr/rollout_ppl:np.float64(1.3272340912371874) - rollout_corr/rollout_log_ppl:np.float64(0.264034794352483) - rollout_corr/log_ppl_diff:np.float64(0.00044818545575253665) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011384486278984696) - rollout_corr/log_ppl_diff_max:np.float64(0.006218463182449341) - rollout_corr/log_ppl_diff_min:np.float64(-0.00863182544708252) - rollout_corr/ppl_ratio:np.float64(1.0004495589528233) - rollout_corr/chi2_token:np.float64(0.0009194693993777037) - rollout_corr/chi2_seq:np.float64(0.4609958524815738) - actor/pg_loss:np.float64(1.4589284546673298e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.937741061894485e-05) - actor/ppo_kl:np.float64(4.479127906620306e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10549908224493265) - perf/mfu/actor:np.float64(0.11212622068994285) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.87326049804688) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.87890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.87890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009453050792217255 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009453050792217255 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:640.44921875 - response_length/max:3306.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:640.44921875 - response_length_non_aborted/max:3306.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001562461256980896 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4932099115103483) - timing_s/agent_loop/generate_sequences/max:np.float64(23.016741577535868) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.735508493344241) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.016741577535868) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:3306 - timing_s/gen:30.13746011815965 - timing_s/reward:0.07099534384906292 - timing_s/old_log_prob:3.4354004375636578 - timing_s/adv:0.1418602205812931 - timing_s/update_actor:13.508899280801415 - timing_s/step:47.37992210872471 - timing_s/stop_profile:7.83577561378479e-05 - timing_per_token_ms/gen:0.18381543788331953 - timing_per_token_ms/update_actor:0.05871928192681623 - timing_per_token_ms/adv:0.0006166253899273365 - perf/total_num_tokens:230059 - perf/time_per_step:47.37992210872471 - perf/throughput:606.9527707118056 (TaskRunner pid=2281424) Training Progress: 39%|███▉ | 39/100 [39:23<55:49, 54.90s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] To find the magnitude of the particle's charge, we can use the formula for the electric field due to a point charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Where: (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now, substitute the known values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now, convert coulombs to picocoulombs ($1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C}$): (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) So, the closest answer is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2281424) step:40 - global_seqlen/min:25284 - global_seqlen/max:51779 - global_seqlen/minmax_diff:26495 - global_seqlen/balanced_min:36707 - global_seqlen/balanced_max:38750 - global_seqlen/mean:36970.5 - actor/entropy:0.2809111177921295 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35805636644363403 - training/rollout_probs_diff_mean:0.0029524413403123617 - training/rollout_probs_diff_std:0.008548099547624588 - training/rollout_actor_probs_pearson_corr:0.9994715452194214 - rollout_corr/rollout_is_mean:0.9999876022338867 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.368109330243897e-06 - rollout_corr/rollout_is_max:1.9496772289276123 - rollout_corr/rollout_is_min:0.451233834028244 - rollout_corr/rollout_is_std:0.026926398277282715 - rollout_corr/rollout_is_eff_sample_size:0.999275494389906 - rollout_corr/rollout_is_seq_mean:0.9999891519546509 - rollout_corr/rollout_is_seq_std:0.0013239040272310376 - rollout_corr/rollout_is_seq_max:1.0059114694595337 - rollout_corr/rollout_is_seq_min:0.9929170608520508 - rollout_corr/rollout_is_seq_max_deviation:0.007082939147949219 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2970258207060397) - rollout_corr/training_log_ppl:np.float64(0.24729529570322484) - rollout_corr/kl:np.float64(0.00048714316951858905) - rollout_corr/k3_kl:np.float64(0.00042656138907659624) - rollout_corr/rollout_ppl:np.float64(1.2963653528131545) - rollout_corr/rollout_log_ppl:np.float64(0.24680815133615397) - rollout_corr/log_ppl_diff:np.float64(0.0004871443670708686) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010458430915605277) - rollout_corr/log_ppl_diff_max:np.float64(0.007907375693321228) - rollout_corr/log_ppl_diff_min:np.float64(-0.00467248260974884) - rollout_corr/ppl_ratio:np.float64(1.0004882379435003) - rollout_corr/chi2_token:np.float64(0.0007523114327341318) - rollout_corr/chi2_seq:np.float64(0.4919047087896615) - actor/pg_loss:np.float64(0.00013426528312265873) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.462851380703796e-05) - actor/ppo_kl:np.float64(9.811125503489393e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10963228065520525) - perf/mfu/actor:np.float64(0.13216087132668086) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(105.88793182373047) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62109375) - val-aux/sciknoweval/reward/std@16:np.float64(0.20978195620494514) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7098125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19512127996631243) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5332000000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16365501611769506) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.62185) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.209297903919098) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7927750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15226235422617695) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.47158749999999994) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10047297589092027) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6512499999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1840588452797897) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8536250000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09931525131049548) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.43674999999999997) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.05502894877052068) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6747375) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15323121047661067) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8911) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05002471514195125) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.416225) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03088195571501212) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6931875) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.12136011366811209) - val-aux/sciknoweval/score/mean@16:np.float64(0.62109375) - val-aux/sciknoweval/score/std@16:np.float64(0.20978195620494514) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7098125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.19512127996631243) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5332000000000001) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16365501611769506) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.62185) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.209297903919098) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7927750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15226235422617695) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.47158749999999994) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10047297589092027) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6512499999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1840588452797897) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8536250000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09931525131049548) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.43674999999999997) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.05502894877052068) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6747375) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15323121047661067) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8911) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05002471514195125) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.416225) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03088195571501212) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6931875) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.12136011366811209) - val-core/sciknoweval/acc/mean@16:np.float64(0.62109375) - val-aux/sciknoweval/acc/std@16:np.float64(0.20978195620494514) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7098125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.19512127996631243) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5332000000000001) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16365501611769506) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.62185) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.209297903919098) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7927750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15226235422617695) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.47158749999999994) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10047297589092027) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6512499999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1840588452797897) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8536250000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09931525131049548) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.43674999999999997) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.05502894877052068) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6747375) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15323121047661067) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8911) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05002471514195125) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.416225) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03088195571501212) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6931875) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.12136011366811209) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.93046875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.12202913367678088) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9769125000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.06539542499226024) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.8835625) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.13970718792907938) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9306750000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.1211440536861494) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9951250000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.02218746491637611) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.823675) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.14131489814444675) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.956325) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.09419032337440465) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9994624999999999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.004248820241722828) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.760775) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.12100807936316986) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9767624999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.06242768994051049) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7088500000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08327809515343695) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99055) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.030619804261104205) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.83203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0065811024978756905 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0065811024978756905 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:894.265625 - response_length/max:5469.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:894.265625 - response_length_non_aborted/max:5469.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.171664714813232e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.883589567616582) - timing_s/agent_loop/generate_sequences/max:np.float64(36.48411184921861) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.177588642422052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(36.48411184921861) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:313 - timing_s/agent_loop/slowest/response_length:5469 - timing_s/gen:43.124824890866876 - timing_s/reward:0.08122630603611469 - timing_s/old_log_prob:3.681945353746414 - timing_s/adv:0.15046975389122963 - timing_s/update_actor:14.803198579698801 - timing_s/step:61.928098333999515 - timing_s/testing:162.62985380925238 - timing_s/save_checkpoint:12.925621895119548 - timing_s/stop_profile:0.00014221295714378357 - timing_per_token_ms/gen:0.1883739489930061 - timing_per_token_ms/update_actor:0.0500507113093507 - timing_per_token_ms/adv:0.0005087493876578272 - perf/total_num_tokens:295764 - perf/time_per_step:61.928098333999515 - perf/throughput:596.9907197958088 (TaskRunner pid=2281424) Training Progress: 40%|████ | 40/100 [43:20<1:49:42, 109.70s/it] (TaskRunner pid=2281424) step:41 - global_seqlen/min:29658 - global_seqlen/max:51620 - global_seqlen/minmax_diff:21962 - global_seqlen/balanced_min:38538 - global_seqlen/balanced_max:38782 - global_seqlen/mean:38637.75 - actor/entropy:0.33072540163993835 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35299545526504517 - training/rollout_probs_diff_mean:0.0031244975980371237 - training/rollout_probs_diff_std:0.008380025625228882 - training/rollout_actor_probs_pearson_corr:0.9994889497756958 - rollout_corr/rollout_is_mean:1.000058650970459 - rollout_corr/rollout_is_ratio_fraction_high:1.260218232346233e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.29278564453125 - rollout_corr/rollout_is_min:0.55013507604599 - rollout_corr/rollout_is_std:0.027836313471198082 - rollout_corr/rollout_is_eff_sample_size:0.999225739561659 - rollout_corr/rollout_is_seq_mean:0.9999604225158691 - rollout_corr/rollout_is_seq_std:0.0012300461530685425 - rollout_corr/rollout_is_seq_max:1.006003737449646 - rollout_corr/rollout_is_seq_min:0.9945253133773804 - rollout_corr/rollout_is_seq_max_deviation:0.006003737449645996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3499207119457424) - rollout_corr/training_log_ppl:np.float64(0.2855829408799764) - rollout_corr/kl:np.float64(0.0005269990794536383) - rollout_corr/k3_kl:np.float64(0.0004426697283577141) - rollout_corr/rollout_ppl:np.float64(1.349194892682135) - rollout_corr/rollout_log_ppl:np.float64(0.28505594066518825) - rollout_corr/log_ppl_diff:np.float64(0.0005270002147881314) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011361698125256225) - rollout_corr/log_ppl_diff_max:np.float64(0.005944006145000458) - rollout_corr/log_ppl_diff_min:np.float64(-0.005369067192077637) - rollout_corr/ppl_ratio:np.float64(1.0005282380152494) - rollout_corr/chi2_token:np.float64(0.0007350665982812643) - rollout_corr/chi2_seq:np.float64(4.942351899808273) - actor/pg_loss:np.float64(4.62154857814312e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001543072065146589) - actor/ppo_kl:np.float64(0.0001058366084052409) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12146356515586376) - perf/mfu/actor:np.float64(0.13938522742671805) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.29471588134766) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004173422697931528 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004173422697931528 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:929.8984375 - response_length/max:4125.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:929.8984375 - response_length_non_aborted/max:4125.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015160441398620605 - timing_s/agent_loop/generate_sequences/min:np.float64(1.333501910790801) - timing_s/agent_loop/generate_sequences/max:np.float64(29.424998305737972) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.613472268974874) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.424998305737972) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:4125 - timing_s/gen:35.98993204906583 - timing_s/reward:0.08053253218531609 - timing_s/old_log_prob:3.700353318825364 - timing_s/adv:0.16254926659166813 - timing_s/update_actor:14.687377415597439 - timing_s/step:54.72286199964583 - timing_s/stop_profile:0.00014472007751464844 - timing_per_token_ms/gen:0.15118389965749715 - timing_per_token_ms/update_actor:0.04751628076038796 - timing_per_token_ms/adv:0.0005258758163702213 - perf/total_num_tokens:309102 - perf/time_per_step:54.72286199964583 - perf/throughput:706.0623035441762 (TaskRunner pid=2281424) Training Progress: 41%|████ | 41/100 [44:15<1:31:40, 93.23s/it] (TaskRunner pid=2281424) step:42 - global_seqlen/min:34807 - global_seqlen/max:57617 - global_seqlen/minmax_diff:22810 - global_seqlen/balanced_min:46153 - global_seqlen/balanced_max:46569 - global_seqlen/mean:46354.0 - actor/entropy:0.4125242531299591 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730005979537964 - training/rollout_probs_diff_mean:0.0032492319587618113 - training/rollout_probs_diff_std:0.00808805413544178 - training/rollout_actor_probs_pearson_corr:0.9996025562286377 - rollout_corr/rollout_is_mean:0.9999741911888123 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.913554094964638e-06 - rollout_corr/rollout_is_max:1.8443630933761597 - rollout_corr/rollout_is_min:0.3683568835258484 - rollout_corr/rollout_is_std:0.028246553614735603 - rollout_corr/rollout_is_eff_sample_size:0.9992026492912346 - rollout_corr/rollout_is_seq_mean:1.0000536441802979 - rollout_corr/rollout_is_seq_std:0.0012193896109238267 - rollout_corr/rollout_is_seq_max:1.009535312652588 - rollout_corr/rollout_is_seq_min:0.9968177080154419 - rollout_corr/rollout_is_seq_max_deviation:0.00953531265258789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4070447161793709) - rollout_corr/training_log_ppl:np.float64(0.3194918569060974) - rollout_corr/kl:np.float64(0.00035904505688222343) - rollout_corr/k3_kl:np.float64(0.00043011571953854855) - rollout_corr/rollout_ppl:np.float64(1.4064954705536366) - rollout_corr/rollout_log_ppl:np.float64(0.3191328103421256) - rollout_corr/log_ppl_diff:np.float64(0.0003590465639717877) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009108688100241125) - rollout_corr/log_ppl_diff_max:np.float64(0.0039010122418403625) - rollout_corr/log_ppl_diff_min:np.float64(-0.003787517547607422) - rollout_corr/ppl_ratio:np.float64(1.0003597426693887) - rollout_corr/chi2_token:np.float64(0.0010020083282142878) - rollout_corr/chi2_seq:np.float64(0.939779143081978) - actor/pg_loss:np.float64(0.00010238576214760542) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001587477142379612) - actor/ppo_kl:np.float64(2.258576988367622e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11312472820281982) - perf/mfu/actor:np.float64(0.15362174541859897) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.15521240234375) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01894480176270008 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01894480176270008 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1182.09375 - response_length/max:6248.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1182.09375 - response_length_non_aborted/max:6248.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.07406210899353e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8849900215864182) - timing_s/agent_loop/generate_sequences/max:np.float64(42.688118137419224) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.67329497004539) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(42.688118137419224) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:6248 - timing_s/gen:50.097527500241995 - timing_s/reward:0.09398127906024456 - timing_s/old_log_prob:3.954119984060526 - timing_s/adv:0.14543932490050793 - timing_s/update_actor:16.064200446009636 - timing_s/step:70.44157047756016 - timing_s/stop_profile:0.00011380389332771301 - timing_per_token_ms/gen:0.16554817822006104 - timing_per_token_ms/update_actor:0.04331934796891756 - timing_per_token_ms/adv:0.00039219734246372464 - perf/total_num_tokens:370832 - perf/time_per_step:70.44157047756016 - perf/throughput:658.0489288603596 (TaskRunner pid=2281424) Training Progress: 42%|████▏ | 42/100 [45:26<1:23:31, 86.41s/it] (TaskRunner pid=2281424) step:43 - global_seqlen/min:27945 - global_seqlen/max:57053 - global_seqlen/minmax_diff:29108 - global_seqlen/balanced_min:46139 - global_seqlen/balanced_max:46442 - global_seqlen/mean:46394.125 - actor/entropy:0.3777778148651123 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24276472628116608 - training/rollout_probs_diff_mean:0.0031969803385436535 - training/rollout_probs_diff_std:0.008178768679499626 - training/rollout_actor_probs_pearson_corr:0.9996250867843628 - rollout_corr/rollout_is_mean:0.9999624490737915 - rollout_corr/rollout_is_ratio_fraction_high:3.310370402687113e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.620740805374226e-06 - rollout_corr/rollout_is_max:3.3981943130493164 - rollout_corr/rollout_is_min:0.40390288829803467 - rollout_corr/rollout_is_std:0.027761269360780716 - rollout_corr/rollout_is_eff_sample_size:0.9992297864199887 - rollout_corr/rollout_is_seq_mean:0.9999164938926697 - rollout_corr/rollout_is_seq_std:0.0013294965028762817 - rollout_corr/rollout_is_seq_max:1.0057275295257568 - rollout_corr/rollout_is_seq_min:0.9940543174743652 - rollout_corr/rollout_is_seq_max_deviation:0.005945682525634766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.368367766495794) - rollout_corr/training_log_ppl:np.float64(0.297560689446982) - rollout_corr/kl:np.float64(0.0004662561269233034) - rollout_corr/k3_kl:np.float64(0.00041893447971119713) - rollout_corr/rollout_ppl:np.float64(1.3677249136380851) - rollout_corr/rollout_log_ppl:np.float64(0.29709443140018266) - rollout_corr/log_ppl_diff:np.float64(0.00046625804679933935) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009343025594716892) - rollout_corr/log_ppl_diff_max:np.float64(0.006767094135284424) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033786147832870483) - rollout_corr/ppl_ratio:np.float64(1.000467125326395) - rollout_corr/chi2_token:np.float64(0.0007502012886106968) - rollout_corr/chi2_seq:np.float64(0.8961137712467462) - actor/pg_loss:np.float64(7.715250831097364e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018895000766860903) - actor/ppo_kl:np.float64(-1.2429667751945317e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12724709697067738) - perf/mfu/actor:np.float64(0.15634425835350255) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.0877799987793) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015876950696110725 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015876950696110725 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:1180.00390625 - response_length/max:5656.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1180.00390625 - response_length_non_aborted/max:5656.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011191703379154205 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1672064159065485) - timing_s/agent_loop/generate_sequences/max:np.float64(40.370598992332816) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.840922129631508) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(40.370598992332816) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:5656 - timing_s/gen:47.32844007387757 - timing_s/reward:0.09296286478638649 - timing_s/old_log_prob:3.898196619004011 - timing_s/adv:0.13054160960018635 - timing_s/update_actor:15.71086062118411 - timing_s/step:67.24753057211637 - timing_s/stop_profile:2.9379501938819885e-05 - timing_per_token_ms/gen:0.15667466697302238 - timing_per_token_ms/update_actor:0.04232987641534384 - timing_per_token_ms/adv:0.00035171912823063896 - perf/total_num_tokens:371153 - perf/time_per_step:67.24753057211637 - perf/throughput:689.9007979221907 (TaskRunner pid=2281424) Training Progress: 43%|████▎ | 43/100 [46:33<1:16:37, 80.66s/it] (TaskRunner pid=2281424) step:44 - global_seqlen/min:31039 - global_seqlen/max:81483 - global_seqlen/minmax_diff:50444 - global_seqlen/balanced_min:49966 - global_seqlen/balanced_max:50221 - global_seqlen/mean:50039.375 - actor/entropy:0.42505836486816406 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40381795167922974 - training/rollout_probs_diff_mean:0.0033156690187752247 - training/rollout_probs_diff_std:0.00811709649860859 - training/rollout_actor_probs_pearson_corr:0.9995504021644592 - rollout_corr/rollout_is_mean:0.9999823570251465 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.108491212828085e-06 - rollout_corr/rollout_is_max:1.868242859840393 - rollout_corr/rollout_is_min:0.16989949345588684 - rollout_corr/rollout_is_std:0.028271863237023354 - rollout_corr/rollout_is_eff_sample_size:0.9992012210621306 - rollout_corr/rollout_is_seq_mean:1.0000169277191162 - rollout_corr/rollout_is_seq_std:0.0010278340196236968 - rollout_corr/rollout_is_seq_max:1.0032761096954346 - rollout_corr/rollout_is_seq_min:0.9966418147087097 - rollout_corr/rollout_is_seq_max_deviation:0.003358185291290283 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4229408679530025) - rollout_corr/training_log_ppl:np.float64(0.3282105611579027) - rollout_corr/kl:np.float64(0.0004335679858704644) - rollout_corr/k3_kl:np.float64(0.00045822281549590116) - rollout_corr/rollout_ppl:np.float64(1.422240549698472) - rollout_corr/rollout_log_ppl:np.float64(0.3277769897831604) - rollout_corr/log_ppl_diff:np.float64(0.0004335713747423142) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009576250740792602) - rollout_corr/log_ppl_diff_max:np.float64(0.004134416580200195) - rollout_corr/log_ppl_diff_min:np.float64(-0.004089929163455963) - rollout_corr/ppl_ratio:np.float64(1.000434412388131) - rollout_corr/chi2_token:np.float64(0.0009704537224024534) - rollout_corr/chi2_seq:np.float64(1.733142590848729) - actor/pg_loss:np.float64(2.68502626568079e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020757744624688712) - actor/ppo_kl:np.float64(6.416326252889348e-05) - actor/pg_clipfrac_lower:np.float64(1.8610052165968227e-06) - actor/grad_norm:np.float64(0.14784160628914833) - perf/mfu/actor:np.float64(0.1614180488812359) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.14054107666016) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005857898388057947 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005857898388057947 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1286.57421875 - response_length/max:7615.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1286.57421875 - response_length_non_aborted/max:7615.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.9996037483215332e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0687543358653784) - timing_s/agent_loop/generate_sequences/max:np.float64(51.31303711421788) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.701673694929923) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.31303711421788) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:7615 - timing_s/gen:58.36772031150758 - timing_s/reward:0.09810716658830643 - timing_s/old_log_prob:4.152743987739086 - timing_s/adv:0.15301652811467648 - timing_s/update_actor:16.473277961835265 - timing_s/step:79.3296963069588 - timing_s/stop_profile:3.1422823667526245e-05 - timing_per_token_ms/gen:0.17721395636883192 - timing_per_token_ms/update_actor:0.04115078865851958 - timing_per_token_ms/adv:0.0003822403060456802 - perf/total_num_tokens:400315 - perf/time_per_step:79.3296963069588 - perf/throughput:630.777342275172 (TaskRunner pid=2281424) Training Progress: 44%|████▍ | 44/100 [47:52<1:14:55, 80.27s/it] (TaskRunner pid=2281424) step:45 - global_seqlen/min:29259 - global_seqlen/max:67289 - global_seqlen/minmax_diff:38030 - global_seqlen/balanced_min:45428 - global_seqlen/balanced_max:45525 - global_seqlen/mean:45499.625 - actor/entropy:0.39985886216163635 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25675493478775024 - training/rollout_probs_diff_mean:0.0032300371676683426 - training/rollout_probs_diff_std:0.00817596260458231 - training/rollout_actor_probs_pearson_corr:0.9996156692504883 - rollout_corr/rollout_is_mean:0.9999929666519165 - rollout_corr/rollout_is_ratio_fraction_high:3.4166423574788496e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.4166423574788496e-06 - rollout_corr/rollout_is_max:2.063965082168579 - rollout_corr/rollout_is_min:0.2600027620792389 - rollout_corr/rollout_is_std:0.028221219778060913 - rollout_corr/rollout_is_eff_sample_size:0.9992040775244214 - rollout_corr/rollout_is_seq_mean:0.999981701374054 - rollout_corr/rollout_is_seq_std:0.0011321797501295805 - rollout_corr/rollout_is_seq_max:1.00321364402771 - rollout_corr/rollout_is_seq_min:0.9935727119445801 - rollout_corr/rollout_is_seq_max_deviation:0.006427288055419922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4365097098052502) - rollout_corr/training_log_ppl:np.float64(0.33815656184742693) - rollout_corr/kl:np.float64(0.0006105463397099165) - rollout_corr/k3_kl:np.float64(0.0004829082062087764) - rollout_corr/rollout_ppl:np.float64(1.4356286954134703) - rollout_corr/rollout_log_ppl:np.float64(0.3375460135139292) - rollout_corr/log_ppl_diff:np.float64(0.000610548333497718) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009788531460799277) - rollout_corr/log_ppl_diff_max:np.float64(0.007325723767280579) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033019185066223145) - rollout_corr/ppl_ratio:np.float64(1.0006115036085248) - rollout_corr/chi2_token:np.float64(0.0007435702718794346) - rollout_corr/chi2_seq:np.float64(2.2746842054184526) - actor/pg_loss:np.float64(7.623340934514999e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001888360293378355) - actor/ppo_kl:np.float64(0.0001864508021274247) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12979234382510185) - perf/mfu/actor:np.float64(0.15628703385179005) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.251708984375) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008445513434708118 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008445513434708118 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1143.30078125 - response_length/max:3842.0 - response_length/min:192.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1143.30078125 - response_length_non_aborted/max:3842.0 - response_length_non_aborted/min:192.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002876799553632736 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4185115825384855) - timing_s/agent_loop/generate_sequences/max:np.float64(29.632253983989358) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.899439175627776) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.632253983989358) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:3842 - timing_s/gen:36.76520950347185 - timing_s/reward:0.08381190523505211 - timing_s/old_log_prob:3.9696482811123133 - timing_s/adv:0.12938232719898224 - timing_s/update_actor:15.343466022983193 - timing_s/step:56.40911629982293 - timing_s/stop_profile:0.00011620856821537018 - timing_per_token_ms/gen:0.12561357604069853 - timing_per_token_ms/update_actor:0.0421527265966016 - timing_per_token_ms/adv:0.0003554488833671218 - perf/total_num_tokens:363997 - perf/time_per_step:56.40911629982293 - perf/throughput:806.600563606823 (TaskRunner pid=2281424) Training Progress: 45%|████▌ | 45/100 [48:49<1:07:07, 73.23s/it] (TaskRunner pid=2281424) step:46 - global_seqlen/min:20945 - global_seqlen/max:94884 - global_seqlen/minmax_diff:73939 - global_seqlen/balanced_min:43085 - global_seqlen/balanced_max:44324 - global_seqlen/mean:43425.25 - actor/entropy:0.3762817084789276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5543509721755981 - training/rollout_probs_diff_mean:0.0032524620182812214 - training/rollout_probs_diff_std:0.008445695042610168 - training/rollout_actor_probs_pearson_corr:0.9995606541633606 - rollout_corr/rollout_is_mean:0.999961793422699 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4132578144199215e-05 - rollout_corr/rollout_is_max:1.6447221040725708 - rollout_corr/rollout_is_min:0.002203931100666523 - rollout_corr/rollout_is_std:0.027791311964392662 - rollout_corr/rollout_is_eff_sample_size:0.999228120062589 - rollout_corr/rollout_is_seq_mean:0.9999802112579346 - rollout_corr/rollout_is_seq_std:0.0012473923852667212 - rollout_corr/rollout_is_seq_max:1.0055023431777954 - rollout_corr/rollout_is_seq_min:0.9959372878074646 - rollout_corr/rollout_is_seq_max_deviation:0.00550234317779541 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3645080188289285) - rollout_corr/training_log_ppl:np.float64(0.28979677851020824) - rollout_corr/kl:np.float64(0.0005104832347480048) - rollout_corr/k3_kl:np.float64(0.00045839784411327855) - rollout_corr/rollout_ppl:np.float64(1.3638080032542348) - rollout_corr/rollout_log_ppl:np.float64(0.2892862949083792) - rollout_corr/log_ppl_diff:np.float64(0.0005104836018290371) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010666771559044719) - rollout_corr/log_ppl_diff_max:np.float64(0.005919963121414185) - rollout_corr/log_ppl_diff_min:np.float64(-0.004801034927368164) - rollout_corr/ppl_ratio:np.float64(1.0005116499960423) - rollout_corr/chi2_token:np.float64(0.0008005544077605009) - rollout_corr/chi2_seq:np.float64(1.0131068772170693) - actor/pg_loss:np.float64(-1.758092548698187e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.805006137890814e-05) - actor/ppo_kl:np.float64(7.538383222893685e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10063287615776062) - perf/mfu/actor:np.float64(0.14549136899234336) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.28108978271484) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.027433983981609344 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.027433983981609344 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1105.6015625 - response_length/max:8192.0 - response_length/min:136.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1105.6015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001056082546710968 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1990190539509058) - timing_s/agent_loop/generate_sequences/max:np.float64(59.229342663660645) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.871870818904426) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.229342663660645) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.95301722921431 - timing_s/reward:0.08986181393265724 - timing_s/old_log_prob:3.924274370074272 - timing_s/adv:0.1476494651287794 - timing_s/update_actor:16.007332164794207 - timing_s/step:86.20792456716299 - timing_s/stop_profile:5.9446319937705994e-05 - timing_per_token_ms/gen:0.23302153532513517 - timing_per_token_ms/update_actor:0.04607725967263921 - timing_per_token_ms/adv:0.0004250104061829794 - perf/total_num_tokens:347402 - perf/time_per_step:86.20792456716299 - perf/throughput:503.72689306733275 (TaskRunner pid=2281424) Training Progress: 46%|████▌ | 46/100 [50:15<1:09:24, 77.13s/it] (TaskRunner pid=2281424) step:47 - global_seqlen/min:31035 - global_seqlen/max:67357 - global_seqlen/minmax_diff:36322 - global_seqlen/balanced_min:49399 - global_seqlen/balanced_max:49649 - global_seqlen/mean:49578.625 - actor/entropy:0.34646978974342346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5756693482398987 - training/rollout_probs_diff_mean:0.0030399137176573277 - training/rollout_probs_diff_std:0.00816922727972269 - training/rollout_actor_probs_pearson_corr:0.9995539784431458 - rollout_corr/rollout_is_mean:0.9999984502792358 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.122617484536022e-06 - rollout_corr/rollout_is_max:1.9699045419692993 - rollout_corr/rollout_is_min:0.25761306285858154 - rollout_corr/rollout_is_std:0.026937464252114296 - rollout_corr/rollout_is_eff_sample_size:0.9992746611342851 - rollout_corr/rollout_is_seq_mean:1.0000650882720947 - rollout_corr/rollout_is_seq_std:0.001198867685161531 - rollout_corr/rollout_is_seq_max:1.005571961402893 - rollout_corr/rollout_is_seq_min:0.9964936971664429 - rollout_corr/rollout_is_seq_max_deviation:0.005571961402893066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3529047267511487) - rollout_corr/training_log_ppl:np.float64(0.28667624859372154) - rollout_corr/kl:np.float64(0.0004597161929975613) - rollout_corr/k3_kl:np.float64(0.00046043861678413123) - rollout_corr/rollout_ppl:np.float64(1.3522660555317998) - rollout_corr/rollout_log_ppl:np.float64(0.2862165296683088) - rollout_corr/log_ppl_diff:np.float64(0.0004597189254127443) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009910539374686778) - rollout_corr/log_ppl_diff_max:np.float64(0.007551088929176331) - rollout_corr/log_ppl_diff_min:np.float64(-0.004575580358505249) - rollout_corr/ppl_ratio:np.float64(1.0004606684669852) - rollout_corr/chi2_token:np.float64(0.0009119359310716391) - rollout_corr/chi2_seq:np.float64(954.7251316532493) - actor/pg_loss:np.float64(-8.133123628795147e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001264397735667444) - actor/ppo_kl:np.float64(0.0001422462675852998) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11066612228751183) - perf/mfu/actor:np.float64(0.1590812815304962) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.31835174560547) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024808576330542564 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024808576330542564 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1284.58203125 - response_length/max:7270.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1284.58203125 - response_length_non_aborted/max:7270.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.761533975601196e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.059980768710375) - timing_s/agent_loop/generate_sequences/max:np.float64(50.74220941402018) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.601909340250131) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.74220941402018) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:7270 - timing_s/gen:58.42462229728699 - timing_s/reward:0.09585716761648655 - timing_s/old_log_prob:3.987767066806555 - timing_s/adv:0.1361205969005823 - timing_s/update_actor:16.680994622409344 - timing_s/step:79.40998443961143 - timing_s/stop_profile:0.00011073052883148193 - timing_per_token_ms/gen:0.17766181940650377 - timing_per_token_ms/update_actor:0.04205692125994152 - timing_per_token_ms/adv:0.0003431937576440006 - perf/total_num_tokens:396629 - perf/time_per_step:79.40998443961143 - perf/throughput:624.3374224270607 (TaskRunner pid=2281424) Training Progress: 47%|████▋ | 47/100 [51:35<1:08:44, 77.83s/it] (TaskRunner pid=2281424) step:48 - global_seqlen/min:22232 - global_seqlen/max:57431 - global_seqlen/minmax_diff:35199 - global_seqlen/balanced_min:41913 - global_seqlen/balanced_max:42517 - global_seqlen/mean:42023.25 - actor/entropy:0.34810274839401245 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931787967681885 - training/rollout_probs_diff_mean:0.003345586359500885 - training/rollout_probs_diff_std:0.008651177398860455 - training/rollout_actor_probs_pearson_corr:0.9994678497314453 - rollout_corr/rollout_is_mean:1.0000156164169312 - rollout_corr/rollout_is_ratio_fraction_high:7.471328899555374e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.471328899555374e-06 - rollout_corr/rollout_is_max:2.302506685256958 - rollout_corr/rollout_is_min:0.015659699216485023 - rollout_corr/rollout_is_std:0.02865091897547245 - rollout_corr/rollout_is_eff_sample_size:0.9991797981155129 - rollout_corr/rollout_is_seq_mean:1.0000426769256592 - rollout_corr/rollout_is_seq_std:0.0014892894541844726 - rollout_corr/rollout_is_seq_max:1.0078259706497192 - rollout_corr/rollout_is_seq_min:0.9916344881057739 - rollout_corr/rollout_is_seq_max_deviation:0.008365511894226074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.361631895415485) - rollout_corr/training_log_ppl:np.float64(0.296743152604904) - rollout_corr/kl:np.float64(0.00044437152225995646) - rollout_corr/k3_kl:np.float64(0.0004750857856947732) - rollout_corr/rollout_ppl:np.float64(1.3610092177987099) - rollout_corr/rollout_log_ppl:np.float64(0.2962987821374554) - rollout_corr/log_ppl_diff:np.float64(0.0004443704674486071) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010612811020109802) - rollout_corr/log_ppl_diff_max:np.float64(0.009809985756874084) - rollout_corr/log_ppl_diff_min:np.float64(-0.005429580807685852) - rollout_corr/ppl_ratio:np.float64(1.0004456255119294) - rollout_corr/chi2_token:np.float64(0.0010020604822784662) - rollout_corr/chi2_seq:np.float64(0.4791911744978279) - actor/pg_loss:np.float64(3.651808947324753e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019356490452082653) - actor/ppo_kl:np.float64(3.127409698944206e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2193583231419325) - perf/mfu/actor:np.float64(0.14563750602154726) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.25163650512695) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024314971640706062 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024314971640706062 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1045.6640625 - response_length/max:5423.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1045.6640625 - response_length_non_aborted/max:5423.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001386590301990509 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7352543752640486) - timing_s/agent_loop/generate_sequences/max:np.float64(38.22215409949422) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.838098710140912) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(38.22215409949422) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:5423 - timing_s/gen:45.5628403685987 - timing_s/reward:0.08806743286550045 - timing_s/old_log_prob:3.732063392177224 - timing_s/adv:0.15191545337438583 - timing_s/update_actor:15.251708237454295 - timing_s/step:64.87329536117613 - timing_s/stop_profile:0.00012004934251308441 - timing_per_token_ms/gen:0.17020748017706563 - timing_per_token_ms/update_actor:0.04536687499614587 - timing_per_token_ms/adv:0.00045187917811683365 - perf/total_num_tokens:336186 - perf/time_per_step:64.87329536117613 - perf/throughput:647.7742461831082 (TaskRunner pid=2281424) Training Progress: 48%|████▊ | 48/100 [52:40<1:04:05, 73.96s/it] (TaskRunner pid=2281424) step:49 - global_seqlen/min:23092 - global_seqlen/max:51168 - global_seqlen/minmax_diff:28076 - global_seqlen/balanced_min:37111 - global_seqlen/balanced_max:37334 - global_seqlen/mean:37259.625 - actor/entropy:0.3179267346858978 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20985785126686096 - training/rollout_probs_diff_mean:0.0031237066723406315 - training/rollout_probs_diff_std:0.008550681173801422 - training/rollout_actor_probs_pearson_corr:0.9993894100189209 - rollout_corr/rollout_is_mean:0.9999985098838806 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.58763451105915e-06 - rollout_corr/rollout_is_max:1.7586421966552734 - rollout_corr/rollout_is_min:0.3407052159309387 - rollout_corr/rollout_is_std:0.02756519615650177 - rollout_corr/rollout_is_eff_sample_size:0.9992407369068808 - rollout_corr/rollout_is_seq_mean:0.9999875426292419 - rollout_corr/rollout_is_seq_std:0.0011641301680356264 - rollout_corr/rollout_is_seq_max:1.0047152042388916 - rollout_corr/rollout_is_seq_min:0.9961039423942566 - rollout_corr/rollout_is_seq_max_deviation:0.0047152042388916016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.328462516888976) - rollout_corr/training_log_ppl:np.float64(0.2660138398205163) - rollout_corr/kl:np.float64(0.00038798783518378954) - rollout_corr/k3_kl:np.float64(0.0004385801271951095) - rollout_corr/rollout_ppl:np.float64(1.3279090919531882) - rollout_corr/rollout_log_ppl:np.float64(0.26562585204374045) - rollout_corr/log_ppl_diff:np.float64(0.0003879877767758444) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009870426292764023) - rollout_corr/log_ppl_diff_max:np.float64(0.006948456168174744) - rollout_corr/log_ppl_diff_min:np.float64(-0.004289939999580383) - rollout_corr/ppl_ratio:np.float64(1.0003889380022883) - rollout_corr/chi2_token:np.float64(0.0009887455962598324) - rollout_corr/chi2_seq:np.float64(1.5676374214235693) - actor/pg_loss:np.float64(2.256419975310564e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012307778933973168) - actor/ppo_kl:np.float64(-3.0272832692190832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07936635101214051) - perf/mfu/actor:np.float64(0.13578834962618122) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.31588363647461) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006559879519045353 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006559879519045353 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:909.73828125 - response_length/max:4227.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:909.73828125 - response_length_non_aborted/max:4227.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.641796350479126e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1448177974671125) - timing_s/agent_loop/generate_sequences/max:np.float64(29.94586299546063) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.983509431694984) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.94586299546063) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:4227 - timing_s/gen:37.325577680021524 - timing_s/reward:0.08239317126572132 - timing_s/old_log_prob:3.49440111592412 - timing_s/adv:0.12574291415512562 - timing_s/update_actor:14.33308171108365 - timing_s/step:55.44903611578047 - timing_s/stop_profile:0.00013232603669166565 - timing_per_token_ms/gen:0.16026921238517913 - timing_per_token_ms/update_actor:0.04808516494423807 - timing_per_token_ms/adv:0.00042184708701149576 - perf/total_num_tokens:298077 - perf/time_per_step:55.44903611578047 - perf/throughput:671.9616355855125 (TaskRunner pid=2281424) Training Progress: 49%|████▉ | 49/100 [53:35<58:09, 68.42s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] To find the magnitude of the charge $ q $, we use the formula for the electric field due to a point charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Substituting the given values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) So the closest option is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2281424) step:50 - global_seqlen/min:34234 - global_seqlen/max:75152 - global_seqlen/minmax_diff:40918 - global_seqlen/balanced_min:51555 - global_seqlen/balanced_max:51907 - global_seqlen/mean:51805.25 - actor/entropy:0.4125359058380127 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19821900129318237 - training/rollout_probs_diff_mean:0.0032416246831417084 - training/rollout_probs_diff_std:0.008041433990001678 - training/rollout_actor_probs_pearson_corr:0.999610185623169 - rollout_corr/rollout_is_mean:1.0000464916229248 - rollout_corr/rollout_is_ratio_fraction_high:8.665861969348043e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.167043685913086 - rollout_corr/rollout_is_min:0.5511428117752075 - rollout_corr/rollout_is_std:0.028244443237781525 - rollout_corr/rollout_is_eff_sample_size:0.9992028873298154 - rollout_corr/rollout_is_seq_mean:0.9999783635139465 - rollout_corr/rollout_is_seq_std:0.0010451057460159063 - rollout_corr/rollout_is_seq_max:1.0044012069702148 - rollout_corr/rollout_is_seq_min:0.9962003827095032 - rollout_corr/rollout_is_seq_max_deviation:0.004401206970214844 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4209796008653939) - rollout_corr/training_log_ppl:np.float64(0.3332373514131177) - rollout_corr/kl:np.float64(0.0004479175979907879) - rollout_corr/k3_kl:np.float64(0.0004798759525215246) - rollout_corr/rollout_ppl:np.float64(1.4203380569815636) - rollout_corr/rollout_log_ppl:np.float64(0.3327894336543977) - rollout_corr/log_ppl_diff:np.float64(0.0004479177587199956) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009449601930100471) - rollout_corr/log_ppl_diff_max:np.float64(0.008978396654129028) - rollout_corr/log_ppl_diff_min:np.float64(-0.004376143217086792) - rollout_corr/ppl_ratio:np.float64(1.0004488993436098) - rollout_corr/chi2_token:np.float64(0.0010542639065533876) - rollout_corr/chi2_seq:np.float64(2.937424761708826) - actor/pg_loss:np.float64(0.00014853477478027344) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021279246669791974) - actor/ppo_kl:np.float64(1.2714361647070405e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1673903912305832) - perf/mfu/actor:np.float64(0.16524905376259622) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.29336929321289) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.65390625) - val-aux/sciknoweval/reward/std@16:np.float64(0.21345823452859586) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7464375000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.19899169203769165) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.55885) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1654343787430585) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6524) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21312019792510917) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8329625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14911841799011322) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49555000000000005) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.0995466061193929) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6846875000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.19169144926745516) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8903625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08902310096279493) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4632625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.04804278474577391) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7100125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.16088561753057692) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9214874999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04076217776661745) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.448325) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.02214616706327121) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7325125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.12534302205736753) - val-aux/sciknoweval/score/mean@16:np.float64(0.65390625) - val-aux/sciknoweval/score/std@16:np.float64(0.21345823452859586) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7464375000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.19899169203769165) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.55885) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1654343787430585) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6524) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21312019792510917) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8329625) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14911841799011322) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49555000000000005) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.0995466061193929) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6846875000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.19169144926745516) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8903625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08902310096279493) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4632625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.04804278474577391) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7100125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.16088561753057692) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9214874999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04076217776661745) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.448325) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.02214616706327121) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7325125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.12534302205736753) - val-core/sciknoweval/acc/mean@16:np.float64(0.65390625) - val-aux/sciknoweval/acc/std@16:np.float64(0.21345823452859586) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7464375000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.19899169203769165) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.55885) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1654343787430585) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6524) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21312019792510917) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8329625) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14911841799011322) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49555000000000005) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.0995466061193929) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6846875000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.19169144926745516) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8903625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08902310096279493) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4632625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.04804278474577391) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7100125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.16088561753057692) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9214874999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04076217776661745) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.448325) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.02214616706327121) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7325125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.12534302205736753) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.93359375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.11242025595458031) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9786250000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.06145532564716834) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.8873125) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.12815918862605175) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9337624999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.11286336680590714) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9963750000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.019611055703303716) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8331125) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12538427780463093) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9568625) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.08973386165612807) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0029360164781816363) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7791125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.10222723529017974) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9782) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.06007209345275405) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7367125) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06895219571062065) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9924125) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.03078564025657012) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012263277545571327 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012263277545571327 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1352.2890625 - response_length/max:6204.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1352.2890625 - response_length_non_aborted/max:6204.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010510161519050598 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9486863240599632) - timing_s/agent_loop/generate_sequences/max:np.float64(45.78752861917019) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.385333957761759) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(45.78752861917019) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:6204 - timing_s/gen:52.41064179688692 - timing_s/reward:0.09868243709206581 - timing_s/old_log_prob:4.130571527406573 - timing_s/adv:0.1609333734959364 - timing_s/update_actor:16.788390869274735 - timing_s/step:73.67946309968829 - timing_s/testing:173.9143652804196 - timing_s/save_checkpoint:12.895720075815916 - timing_s/stop_profile:0.00021344982087612152 - timing_per_token_ms/gen:0.15139445788358546 - timing_per_token_ms/update_actor:0.040508420645771266 - timing_per_token_ms/adv:0.00038831337918438863 - perf/total_num_tokens:414442 - perf/time_per_step:73.67946309968829 - perf/throughput:703.1165513503744 (TaskRunner pid=2281424) Training Progress: 50%|█████ | 50/100 [57:56<1:45:03, 126.06s/it] (TaskRunner pid=2281424) step:51 - global_seqlen/min:25192 - global_seqlen/max:76157 - global_seqlen/minmax_diff:50965 - global_seqlen/balanced_min:48271 - global_seqlen/balanced_max:48448 - global_seqlen/mean:48379.375 - actor/entropy:0.41202202439308167 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37214818596839905 - training/rollout_probs_diff_mean:0.003262621583417058 - training/rollout_probs_diff_std:0.008078004233539104 - training/rollout_actor_probs_pearson_corr:0.9995943903923035 - rollout_corr/rollout_is_mean:0.9999638795852661 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.276774456637213e-06 - rollout_corr/rollout_is_max:1.636783480644226 - rollout_corr/rollout_is_min:0.13366740942001343 - rollout_corr/rollout_is_std:0.027932504191994667 - rollout_corr/rollout_is_eff_sample_size:0.999220264452564 - rollout_corr/rollout_is_seq_mean:0.9999167919158936 - rollout_corr/rollout_is_seq_std:0.001000249176286161 - rollout_corr/rollout_is_seq_max:1.0028444528579712 - rollout_corr/rollout_is_seq_min:0.9964448809623718 - rollout_corr/rollout_is_seq_max_deviation:0.003555119037628174 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3898025089874864) - rollout_corr/training_log_ppl:np.float64(0.31166147111798637) - rollout_corr/kl:np.float64(0.0004372116775031021) - rollout_corr/k3_kl:np.float64(0.0003934571469699222) - rollout_corr/rollout_ppl:np.float64(1.3891857019625604) - rollout_corr/rollout_log_ppl:np.float64(0.3112242584757041) - rollout_corr/log_ppl_diff:np.float64(0.00043721264228224754) - rollout_corr/log_ppl_abs_diff:np.float64(0.000884955981746316) - rollout_corr/log_ppl_diff_max:np.float64(0.007482245564460754) - rollout_corr/log_ppl_diff_min:np.float64(-0.002665892243385315) - rollout_corr/ppl_ratio:np.float64(1.0004379730671644) - rollout_corr/chi2_token:np.float64(0.0006916895508766174) - rollout_corr/chi2_seq:np.float64(0.8691392296459526) - actor/pg_loss:np.float64(5.567236803472042e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015443053536046136) - actor/ppo_kl:np.float64(-3.0327652694950302e-05) - actor/pg_clipfrac_lower:np.float64(8.184056241589133e-07) - actor/grad_norm:np.float64(0.10533490870147943) - perf/mfu/actor:np.float64(0.1597418217861688) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.85708999633789) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014979914762079716 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014979914762079716 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1244.66796875 - response_length/max:5675.0 - response_length/min:200.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1244.66796875 - response_length_non_aborted/max:5675.0 - response_length_non_aborted/min:200.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011126697063446045 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4992719031870365) - timing_s/agent_loop/generate_sequences/max:np.float64(40.4043846745044) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.760603568858642) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(40.4043846745044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:5675 - timing_s/gen:46.7182105332613 - timing_s/reward:0.0914076529443264 - timing_s/old_log_prob:4.139765975996852 - timing_s/adv:0.15743766352534294 - timing_s/update_actor:16.27989299967885 - timing_s/step:67.497363332659 - timing_s/stop_profile:0.00012996233999729156 - timing_per_token_ms/gen:0.1466198331421887 - timing_per_token_ms/update_actor:0.04206310281932862 - timing_per_token_ms/adv:0.0004067788792366141 - perf/total_num_tokens:387035 - perf/time_per_step:67.497363332659 - perf/throughput:716.7594793527489 (TaskRunner pid=2281424) Training Progress: 51%|█████ | 51/100 [59:03<1:28:37, 108.52s/it] (TaskRunner pid=2281424) step:52 - global_seqlen/min:38872 - global_seqlen/max:61135 - global_seqlen/minmax_diff:22263 - global_seqlen/balanced_min:50883 - global_seqlen/balanced_max:50962 - global_seqlen/mean:50922.75 - actor/entropy:0.4251244068145752 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31729912757873535 - training/rollout_probs_diff_mean:0.0034149810671806335 - training/rollout_probs_diff_std:0.008329342119395733 - training/rollout_actor_probs_pearson_corr:0.9995748996734619 - rollout_corr/rollout_is_mean:0.9999804496765137 - rollout_corr/rollout_is_ratio_fraction_high:2.9626644391100854e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.925328878220171e-06 - rollout_corr/rollout_is_max:2.0289995670318604 - rollout_corr/rollout_is_min:0.38479822874069214 - rollout_corr/rollout_is_std:0.02867795154452324 - rollout_corr/rollout_is_eff_sample_size:0.9991781319248341 - rollout_corr/rollout_is_seq_mean:1.0001072883605957 - rollout_corr/rollout_is_seq_std:0.0014209682121872902 - rollout_corr/rollout_is_seq_max:1.0089370012283325 - rollout_corr/rollout_is_seq_min:0.9960955381393433 - rollout_corr/rollout_is_seq_max_deviation:0.00893700122833252 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4353607962839305) - rollout_corr/training_log_ppl:np.float64(0.34465374518185854) - rollout_corr/kl:np.float64(0.00030015603282895764) - rollout_corr/k3_kl:np.float64(0.00044390009770722827) - rollout_corr/rollout_ppl:np.float64(1.4348906846717) - rollout_corr/rollout_log_ppl:np.float64(0.3443535875703674) - rollout_corr/log_ppl_diff:np.float64(0.0003001576114911586) - rollout_corr/log_ppl_abs_diff:np.float64(0.000979981996351853) - rollout_corr/log_ppl_diff_max:np.float64(0.006529062986373901) - rollout_corr/log_ppl_diff_min:np.float64(-0.006823122501373291) - rollout_corr/ppl_ratio:np.float64(1.0003011787775904) - rollout_corr/chi2_token:np.float64(0.0011839414946734905) - rollout_corr/chi2_seq:np.float64(2.5429790497291833) - actor/pg_loss:np.float64(3.4721335396170616e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.84185026279738e-05) - actor/ppo_kl:np.float64(-4.0347963830456024e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11110754683613777) - perf/mfu/actor:np.float64(0.16599907427815286) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.80552291870117) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004099216777831316 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004099216777831316 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1318.4921875 - response_length/max:5029.0 - response_length/min:132.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1318.4921875 - response_length_non_aborted/max:5029.0 - response_length_non_aborted/min:132.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001096632331609726 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7103512845933437) - timing_s/agent_loop/generate_sequences/max:np.float64(38.38058450073004) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.082279307935096) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(38.38058450073004) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:365 - timing_s/agent_loop/slowest/response_length:4841 - timing_s/gen:45.04192319884896 - timing_s/reward:0.10605371557176113 - timing_s/old_log_prob:4.176968155428767 - timing_s/adv:0.13944151997566223 - timing_s/update_actor:16.25410868227482 - timing_s/step:65.81346314027905 - timing_s/stop_profile:0.00014881044626235962 - timing_per_token_ms/gen:0.13344410696062903 - timing_per_token_ms/update_actor:0.03989893682655301 - timing_per_token_ms/adv:0.00034228689528664064 - perf/total_num_tokens:407382 - perf/time_per_step:65.81346314027905 - perf/throughput:773.7436623181486 (TaskRunner pid=2281424) Training Progress: 52%|█████▏ | 52/100 [1:00:09<1:16:34, 95.73s/it] (TaskRunner pid=2281424) step:53 - global_seqlen/min:24839 - global_seqlen/max:55902 - global_seqlen/minmax_diff:31063 - global_seqlen/balanced_min:43329 - global_seqlen/balanced_max:43584 - global_seqlen/mean:43520.375 - actor/entropy:0.37126773595809937 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016398251056671 - training/rollout_probs_diff_mean:0.003143634181469679 - training/rollout_probs_diff_std:0.00820545107126236 - training/rollout_actor_probs_pearson_corr:0.9994932413101196 - rollout_corr/rollout_is_mean:0.9999392628669739 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.210143394331681e-06 - rollout_corr/rollout_is_max:1.7527995109558105 - rollout_corr/rollout_is_min:0.30146440863609314 - rollout_corr/rollout_is_std:0.027419937774538994 - rollout_corr/rollout_is_eff_sample_size:0.9992485928388086 - rollout_corr/rollout_is_seq_mean:0.9999774694442749 - rollout_corr/rollout_is_seq_std:0.0011340720811858773 - rollout_corr/rollout_is_seq_max:1.005629062652588 - rollout_corr/rollout_is_seq_min:0.9950371980667114 - rollout_corr/rollout_is_seq_max_deviation:0.005629062652587891 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3701058248989284) - rollout_corr/training_log_ppl:np.float64(0.2918059763032943) - rollout_corr/kl:np.float64(0.0003194374729091898) - rollout_corr/k3_kl:np.float64(0.00042779503252177165) - rollout_corr/rollout_ppl:np.float64(1.3696815236471593) - rollout_corr/rollout_log_ppl:np.float64(0.29148653568699956) - rollout_corr/log_ppl_diff:np.float64(0.00031944061629474163) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009462947491556406) - rollout_corr/log_ppl_diff_max:np.float64(0.006018996238708496) - rollout_corr/log_ppl_diff_min:np.float64(-0.003216482698917389) - rollout_corr/ppl_ratio:np.float64(1.0003202487714589) - rollout_corr/chi2_token:np.float64(0.0010781094897538424) - rollout_corr/chi2_seq:np.float64(1.596792209893465) - actor/pg_loss:np.float64(3.835768438875675e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021167921056530759) - actor/ppo_kl:np.float64(-7.656292218705119e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13869349658489227) - perf/mfu/actor:np.float64(0.14950320003636675) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.62938690185547) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015568501316010952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015568501316010952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1083.54296875 - response_length/max:5024.0 - response_length/min:193.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1083.54296875 - response_length_non_aborted/max:5024.0 - response_length_non_aborted/min:193.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011430680751800537 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2967226803302765) - timing_s/agent_loop/generate_sequences/max:np.float64(36.389772513881326) - timing_s/agent_loop/generate_sequences/mean:np.float64(10.762310426936892) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(36.389772513881326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:5024 - timing_s/gen:43.48752962425351 - timing_s/reward:0.08610256388783455 - timing_s/old_log_prob:3.7902569361031055 - timing_s/adv:0.1312139704823494 - timing_s/update_actor:15.44841111637652 - timing_s/step:63.040359219536185 - timing_s/stop_profile:0.00011725910007953644 - timing_per_token_ms/gen:0.15677565864389287 - timing_per_token_ms/update_actor:0.04437120290317041 - timing_per_token_ms/adv:0.0003768751144789923 - perf/total_num_tokens:348163 - perf/time_per_step:63.040359219536185 - perf/throughput:690.3573447042328 (TaskRunner pid=2281424) Training Progress: 53%|█████▎ | 53/100 [1:01:12<1:07:19, 85.94s/it] (TaskRunner pid=2281424) step:54 - global_seqlen/min:35066 - global_seqlen/max:65296 - global_seqlen/minmax_diff:30230 - global_seqlen/balanced_min:48016 - global_seqlen/balanced_max:48715 - global_seqlen/mean:48320.375 - actor/entropy:0.41513416171073914 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3469814658164978 - training/rollout_probs_diff_mean:0.0032787041272968054 - training/rollout_probs_diff_std:0.008095493540167809 - training/rollout_actor_probs_pearson_corr:0.999546229839325 - rollout_corr/rollout_is_mean:0.9998549818992615 - rollout_corr/rollout_is_ratio_fraction_high:3.120212340945727e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.360637704958208e-06 - rollout_corr/rollout_is_max:3.656163454055786 - rollout_corr/rollout_is_min:0.04287761077284813 - rollout_corr/rollout_is_std:0.027915427461266518 - rollout_corr/rollout_is_eff_sample_size:0.99922121664114 - rollout_corr/rollout_is_seq_mean:0.9998648166656494 - rollout_corr/rollout_is_seq_std:0.0012277925852686167 - rollout_corr/rollout_is_seq_max:1.0036733150482178 - rollout_corr/rollout_is_seq_min:0.9944618940353394 - rollout_corr/rollout_is_seq_max_deviation:0.0055381059646606445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.386646915692836) - rollout_corr/training_log_ppl:np.float64(0.31141754146665335) - rollout_corr/kl:np.float64(0.00046588320381735215) - rollout_corr/k3_kl:np.float64(0.00041930354774422085) - rollout_corr/rollout_ppl:np.float64(1.3860031738877296) - rollout_corr/rollout_log_ppl:np.float64(0.31095165672013536) - rollout_corr/log_ppl_diff:np.float64(0.00046588474651798606) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009574951836839318) - rollout_corr/log_ppl_diff_max:np.float64(0.0065114498138427734) - rollout_corr/log_ppl_diff_min:np.float64(-0.0028219521045684814) - rollout_corr/ppl_ratio:np.float64(1.0004668282344937) - rollout_corr/chi2_token:np.float64(0.000739572336897254) - rollout_corr/chi2_seq:np.float64(1.1254641045816243) - actor/pg_loss:np.float64(-1.3741431757807732e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.72049774898187e-05) - actor/ppo_kl:np.float64(-8.353029136642931e-05) - actor/pg_clipfrac_lower:np.float64(1.2657970955842757e-06) - actor/grad_norm:np.float64(0.09051301795989275) - perf/mfu/actor:np.float64(0.15693746638014805) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.6057014465332) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015124449506402016 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015124449506402016 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1251.91796875 - response_length/max:8192.0 - response_length/min:151.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1251.91796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.027255535125732e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3670800887048244) - timing_s/agent_loop/generate_sequences/max:np.float64(57.39985219575465) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.685668693418847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(57.39985219575465) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:64.27751941606402 - timing_s/reward:0.09590023942291737 - timing_s/old_log_prob:4.017355827614665 - timing_s/adv:0.11962731927633286 - timing_s/update_actor:16.561819057911634 - timing_s/step:85.16014346294105 - timing_s/stop_profile:0.00010659545660018921 - timing_per_token_ms/gen:0.20055951466987848 - timing_per_token_ms/update_actor:0.04284377723142575 - timing_per_token_ms/adv:0.0003094639664849788 - perf/total_num_tokens:386563 - perf/time_per_step:85.16014346294105 - perf/throughput:567.4059840097318 (TaskRunner pid=2281424) Training Progress: 54%|█████▍ | 54/100 [1:02:38<1:05:43, 85.72s/it] (TaskRunner pid=2281424) step:55 - global_seqlen/min:28028 - global_seqlen/max:53093 - global_seqlen/minmax_diff:25065 - global_seqlen/balanced_min:38416 - global_seqlen/balanced_max:38451 - global_seqlen/mean:38438.25 - actor/entropy:0.3102551996707916 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5002540349960327 - training/rollout_probs_diff_mean:0.0031136018224060535 - training/rollout_probs_diff_std:0.008566453121602535 - training/rollout_actor_probs_pearson_corr:0.9994353652000427 - rollout_corr/rollout_is_mean:0.9999341368675232 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.173030447418569e-06 - rollout_corr/rollout_is_max:1.8234162330627441 - rollout_corr/rollout_is_min:0.46666446328163147 - rollout_corr/rollout_is_std:0.02762567438185215 - rollout_corr/rollout_is_eff_sample_size:0.9992372850970654 - rollout_corr/rollout_is_seq_mean:0.999902606010437 - rollout_corr/rollout_is_seq_std:0.0013777980348095298 - rollout_corr/rollout_is_seq_max:1.0075701475143433 - rollout_corr/rollout_is_seq_min:0.9948606491088867 - rollout_corr/rollout_is_seq_max_deviation:0.007570147514343262 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.308470833580941) - rollout_corr/training_log_ppl:np.float64(0.2564443113806192) - rollout_corr/kl:np.float64(0.0004685787036251554) - rollout_corr/k3_kl:np.float64(0.0005076911484707125) - rollout_corr/rollout_ppl:np.float64(1.3078627600334585) - rollout_corr/rollout_log_ppl:np.float64(0.255975730920909) - rollout_corr/log_ppl_diff:np.float64(0.00046858045971021056) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011701608309522271) - rollout_corr/log_ppl_diff_max:np.float64(0.006679356098175049) - rollout_corr/log_ppl_diff_min:np.float64(-0.0075002312660217285) - rollout_corr/ppl_ratio:np.float64(1.000470134196803) - rollout_corr/chi2_token:np.float64(0.0011455377098172903) - rollout_corr/chi2_seq:np.float64(5.852095796493813) - actor/pg_loss:np.float64(-3.334670327603817e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001705758450043504) - actor/ppo_kl:np.float64(-4.5051564853793025e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07974306493997574) - perf/mfu/actor:np.float64(0.1378203781579895) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.55471420288086) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011153988540172577 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011153988540172577 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:936.0703125 - response_length/max:3898.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:936.0703125 - response_length_non_aborted/max:3898.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001405421644449234 - timing_s/agent_loop/generate_sequences/min:np.float64(1.978264905512333) - timing_s/agent_loop/generate_sequences/max:np.float64(29.77366792783141) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.763206802737841) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(29.77366792783141) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:3898 - timing_s/gen:36.85863957554102 - timing_s/reward:0.08288105763494968 - timing_s/old_log_prob:3.500804428011179 - timing_s/adv:0.11686178669333458 - timing_s/update_actor:14.765768744051456 - timing_s/step:55.41219947300851 - timing_s/stop_profile:0.00012343376874923706 - timing_per_token_ms/gen:0.1538122285466212 - timing_per_token_ms/update_actor:0.04801782321012096 - timing_per_token_ms/adv:0.0003800309154726561 - perf/total_num_tokens:307506 - perf/time_per_step:55.41219947300851 - perf/throughput:693.6784745157682 (TaskRunner pid=2281424) Training Progress: 55%|█████▌ | 55/100 [1:03:33<57:28, 76.64s/it] (TaskRunner pid=2281424) step:56 - global_seqlen/min:35222 - global_seqlen/max:67738 - global_seqlen/minmax_diff:32516 - global_seqlen/balanced_min:48431 - global_seqlen/balanced_max:48616 - global_seqlen/mean:48556.375 - actor/entropy:0.3693101406097412 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5576332211494446 - training/rollout_probs_diff_mean:0.003098106477409601 - training/rollout_probs_diff_std:0.008025160990655422 - training/rollout_actor_probs_pearson_corr:0.9995522499084473 - rollout_corr/rollout_is_mean:0.999983012676239 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.090129894189886e-06 - rollout_corr/rollout_is_max:1.6744037866592407 - rollout_corr/rollout_is_min:0.3452942669391632 - rollout_corr/rollout_is_std:0.02711610123515129 - rollout_corr/rollout_is_eff_sample_size:0.9992651383115954 - rollout_corr/rollout_is_seq_mean:0.9999608993530273 - rollout_corr/rollout_is_seq_std:0.0010731675429269671 - rollout_corr/rollout_is_seq_max:1.006026029586792 - rollout_corr/rollout_is_seq_min:0.9958376288414001 - rollout_corr/rollout_is_seq_max_deviation:0.006026029586791992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.373812660574913) - rollout_corr/training_log_ppl:np.float64(0.3022938699577935) - rollout_corr/kl:np.float64(0.0004166627330413064) - rollout_corr/k3_kl:np.float64(0.00039845293167672935) - rollout_corr/rollout_ppl:np.float64(1.3732107440009713) - rollout_corr/rollout_log_ppl:np.float64(0.3018772063078359) - rollout_corr/log_ppl_diff:np.float64(0.00041666364995762706) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008859913214109838) - rollout_corr/log_ppl_diff_max:np.float64(0.00528949499130249) - rollout_corr/log_ppl_diff_min:np.float64(-0.005298733711242676) - rollout_corr/ppl_ratio:np.float64(1.0004174057394266) - rollout_corr/chi2_token:np.float64(0.0007626630831509829) - rollout_corr/chi2_seq:np.float64(3.9297358801122755) - actor/pg_loss:np.float64(-1.1299271136522293e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014216022026403152) - actor/ppo_kl:np.float64(-9.481449527726227e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08582276105880737) - perf/mfu/actor:np.float64(0.15478688954187692) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(117.41515350341797) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01221644226461649 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.01221644226461649 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:1264.10546875 - response_length/max:5770.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1264.10546875 - response_length_non_aborted/max:5770.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013057328760623932 - timing_s/agent_loop/generate_sequences/min:np.float64(2.215219648554921) - timing_s/agent_loop/generate_sequences/max:np.float64(45.5247247479856) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.946847598039312) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(45.5247247479856) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:5770 - timing_s/gen:52.53792614676058 - timing_s/reward:0.09584798477590084 - timing_s/old_log_prob:3.963084263727069 - timing_s/adv:0.13674898445606232 - timing_s/update_actor:16.772162212058902 - timing_s/step:73.59324477799237 - timing_s/stop_profile:3.3739954233169556e-05 - timing_per_token_ms/gen:0.1623490120754875 - timing_per_token_ms/update_actor:0.04317703445752206 - timing_per_token_ms/adv:0.0003520366389996739 - perf/total_num_tokens:388451 - perf/time_per_step:73.59324477799237 - perf/throughput:659.7939137821587 (TaskRunner pid=2281424) Training Progress: 56%|█████▌ | 56/100 [1:04:47<55:32, 75.74s/it] (TaskRunner pid=2281424) step:57 - global_seqlen/min:50111 - global_seqlen/max:61804 - global_seqlen/minmax_diff:11693 - global_seqlen/balanced_min:54058 - global_seqlen/balanced_max:54380 - global_seqlen/mean:54299.125 - actor/entropy:0.3828109800815582 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26735612750053406 - training/rollout_probs_diff_mean:0.003163557732477784 - training/rollout_probs_diff_std:0.007935766130685806 - training/rollout_actor_probs_pearson_corr:0.9995775818824768 - rollout_corr/rollout_is_mean:0.9999682903289795 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.757548145382316e-06 - rollout_corr/rollout_is_max:1.6177020072937012 - rollout_corr/rollout_is_min:0.4519280195236206 - rollout_corr/rollout_is_std:0.027300119400024414 - rollout_corr/rollout_is_eff_sample_size:0.9992552585748898 - rollout_corr/rollout_is_seq_mean:0.9999948740005493 - rollout_corr/rollout_is_seq_std:0.0008971233037300408 - rollout_corr/rollout_is_seq_max:1.004016637802124 - rollout_corr/rollout_is_seq_min:0.9975495934486389 - rollout_corr/rollout_is_seq_max_deviation:0.0040166378021240234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4292010506615043) - rollout_corr/training_log_ppl:np.float64(0.3405683948658407) - rollout_corr/kl:np.float64(0.00034688604483079644) - rollout_corr/k3_kl:np.float64(0.00040535901956673115) - rollout_corr/rollout_ppl:np.float64(1.4287002789787948) - rollout_corr/rollout_log_ppl:np.float64(0.34022150505916215) - rollout_corr/log_ppl_diff:np.float64(0.00034688980667851865) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007467021641787142) - rollout_corr/log_ppl_diff_max:np.float64(0.0035548806190490723) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031991004943847656) - rollout_corr/ppl_ratio:np.float64(1.0003474049735814) - rollout_corr/chi2_token:np.float64(0.0009404013399034739) - rollout_corr/chi2_seq:np.float64(1.3481947318650782) - actor/pg_loss:np.float64(-1.1662370525300503e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001704909616933037) - actor/ppo_kl:np.float64(-4.2620445977803456e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11611825972795486) - perf/mfu/actor:np.float64(0.13975899720323387) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(142.28152465820312) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023128935135900974 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023128935135900974 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1416.56640625 - response_length/max:6110.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1416.56640625 - response_length_non_aborted/max:6110.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.9925798773765564e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7666615955531597) - timing_s/agent_loop/generate_sequences/max:np.float64(44.93153551965952) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.538133056579682) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(44.93153551965952) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:382 - timing_s/agent_loop/slowest/response_length:6110 - timing_s/gen:52.147955069318414 - timing_s/reward:0.10221655108034611 - timing_s/old_log_prob:4.09379106014967 - timing_s/adv:0.1320952270179987 - timing_s/update_actor:20.559559671208262 - timing_s/step:77.12406794913113 - timing_s/stop_profile:0.00012364424765110016 - timing_per_token_ms/gen:0.14380049434376813 - timing_per_token_ms/update_actor:0.04732939911832894 - timing_per_token_ms/adv:0.0003040915185511707 - perf/total_num_tokens:434393 - perf/time_per_step:77.12406794913113 - perf/throughput:704.0490270276482 (TaskRunner pid=2281424) Training Progress: 57%|█████▋ | 57/100 [1:06:04<54:35, 76.17s/it] (TaskRunner pid=2281424) step:58 - global_seqlen/min:26330 - global_seqlen/max:76091 - global_seqlen/minmax_diff:49761 - global_seqlen/balanced_min:58973 - global_seqlen/balanced_max:59329 - global_seqlen/mean:59224.75 - actor/entropy:0.41788530349731445 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2639843225479126 - training/rollout_probs_diff_mean:0.0031849679071456194 - training/rollout_probs_diff_std:0.007847386412322521 - training/rollout_actor_probs_pearson_corr:0.9995084404945374 - rollout_corr/rollout_is_mean:1.0000232458114624 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4740838853176683e-06 - rollout_corr/rollout_is_max:1.8915661573410034 - rollout_corr/rollout_is_min:0.4832708239555359 - rollout_corr/rollout_is_std:0.027686020359396935 - rollout_corr/rollout_is_eff_sample_size:0.9992339523378033 - rollout_corr/rollout_is_seq_mean:0.999971866607666 - rollout_corr/rollout_is_seq_std:0.001041158102452755 - rollout_corr/rollout_is_seq_max:1.0040910243988037 - rollout_corr/rollout_is_seq_min:0.9945273399353027 - rollout_corr/rollout_is_seq_max_deviation:0.005472660064697266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4465448674745858) - rollout_corr/training_log_ppl:np.float64(0.35632477820036) - rollout_corr/kl:np.float64(0.0003769699203459709) - rollout_corr/k3_kl:np.float64(0.0005920725309920272) - rollout_corr/rollout_ppl:np.float64(1.4459977843798697) - rollout_corr/rollout_log_ppl:np.float64(0.35594780783867463) - rollout_corr/log_ppl_diff:np.float64(0.00037697036168538034) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009633789595682174) - rollout_corr/log_ppl_diff_max:np.float64(0.006280913949012756) - rollout_corr/log_ppl_diff_min:np.float64(-0.004786491394042969) - rollout_corr/ppl_ratio:np.float64(1.000377933960408) - rollout_corr/chi2_token:np.float64(0.0016758001875132322) - rollout_corr/chi2_seq:np.float64(7.071388645563275) - actor/pg_loss:np.float64(1.6489182598888874e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025524779363195194) - actor/ppo_kl:np.float64(-7.475169755899458e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11395814083516598) - perf/mfu/actor:np.float64(0.17468793390598217) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(112.19358444213867) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017182203009724617 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017182203009724617 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1578.8671875 - response_length/max:6944.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1578.8671875 - response_length_non_aborted/max:6944.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018575415015220642 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6479607988148928) - timing_s/agent_loop/generate_sequences/max:np.float64(53.533365273848176) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.731221175505198) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(53.533365273848176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:6944 - timing_s/gen:60.268137166276574 - timing_s/reward:0.11793672852218151 - timing_s/old_log_prob:4.338053205981851 - timing_s/adv:0.14384562522172928 - timing_s/update_actor:18.135621577501297 - timing_s/step:83.09424239955842 - timing_s/stop_profile:0.00011111609637737274 - timing_per_token_ms/gen:0.14910843208955338 - timing_per_token_ms/update_actor:0.03827711720501416 - timing_per_token_ms/adv:0.0003036011659435651 - perf/total_num_tokens:473798 - perf/time_per_step:83.09424239955842 - perf/throughput:712.7419215788498 (TaskRunner pid=2281424) Training Progress: 58%|█████▊ | 58/100 [1:07:27<54:46, 78.26s/it] (TaskRunner pid=2281424) step:59 - global_seqlen/min:34689 - global_seqlen/max:56180 - global_seqlen/minmax_diff:21491 - global_seqlen/balanced_min:47141 - global_seqlen/balanced_max:49075 - global_seqlen/mean:47480.625 - actor/entropy:0.42323192954063416 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22112640738487244 - training/rollout_probs_diff_mean:0.0033911745995283127 - training/rollout_probs_diff_std:0.008138471283018589 - training/rollout_actor_probs_pearson_corr:0.9995591044425964 - rollout_corr/rollout_is_mean:0.9999943971633911 - rollout_corr/rollout_is_ratio_fraction_high:3.213068112017936e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.426136224035872e-06 - rollout_corr/rollout_is_max:2.264416217803955 - rollout_corr/rollout_is_min:0.44598260521888733 - rollout_corr/rollout_is_std:0.02836867794394493 - rollout_corr/rollout_is_eff_sample_size:0.9991957462217225 - rollout_corr/rollout_is_seq_mean:0.9999998211860657 - rollout_corr/rollout_is_seq_std:0.001030960469506681 - rollout_corr/rollout_is_seq_max:1.004229187965393 - rollout_corr/rollout_is_seq_min:0.9960595369338989 - rollout_corr/rollout_is_seq_max_deviation:0.004229187965393066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4299974106252193) - rollout_corr/training_log_ppl:np.float64(0.3359124011185486) - rollout_corr/kl:np.float64(0.00043199253835979334) - rollout_corr/k3_kl:np.float64(0.0004072271172077535) - rollout_corr/rollout_ppl:np.float64(1.429351010825485) - rollout_corr/rollout_log_ppl:np.float64(0.3354804104164941) - rollout_corr/log_ppl_diff:np.float64(0.0004319907020544633) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008698585879756138) - rollout_corr/log_ppl_diff_max:np.float64(0.0032856017351150513) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038261711597442627) - rollout_corr/ppl_ratio:np.float64(1.0004326105117798) - rollout_corr/chi2_token:np.float64(0.0007886665407568216) - rollout_corr/chi2_seq:np.float64(1.102270802250132) - actor/pg_loss:np.float64(5.7405210100114346e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012484539280421814) - actor/ppo_kl:np.float64(4.3178232203011646e-05) - actor/pg_clipfrac_lower:np.float64(2.1677303720935015e-06) - actor/grad_norm:np.float64(0.12459838949143887) - perf/mfu/actor:np.float64(0.15411530957231614) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.97130966186523) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04919448494911194 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04919448494911194 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1215.73828125 - response_length/max:7645.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1215.73828125 - response_length_non_aborted/max:7645.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017377547919750214 - timing_s/agent_loop/generate_sequences/min:np.float64(2.034050405025482) - timing_s/agent_loop/generate_sequences/max:np.float64(52.7514441665262) - timing_s/agent_loop/generate_sequences/mean:np.float64(12.213702496665064) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(52.7514441665262) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:7645 - timing_s/gen:60.28812167607248 - timing_s/reward:0.09481470286846161 - timing_s/old_log_prob:3.9277980905026197 - timing_s/adv:0.1367920432239771 - timing_s/update_actor:16.420462617650628 - timing_s/step:80.95495403930545 - timing_s/stop_profile:0.00012053921818733215 - timing_per_token_ms/gen:0.19370984604928357 - timing_per_token_ms/update_actor:0.04322937676591933 - timing_per_token_ms/adv:0.00036012595459720964 - perf/total_num_tokens:379845 - perf/time_per_step:80.95495403930545 - perf/throughput:586.5067254185221 (TaskRunner pid=2281424) Training Progress: 59%|█████▉ | 59/100 [1:08:48<54:02, 79.08s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] To find the magnitude of the charge $ q $, we are given: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We use the formula for the electric field due to a point charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now substitute the known values: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) First, compute $ (0.50)^2 $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (0.50)^2 = 0.25 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now compute the numerator: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 2.0 \cdot 0.25 = 0.50 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now compute the denominator: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 8.99 \times 10^9 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now compute the charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} = \frac{0.50}{8.99} \times 10^{-9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) \frac{0.50}{8.99} \approx 0.0556 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 0.0556 \times 10^{-9} = 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now convert to picocoulombs (pC), since $ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \times 10^{-12} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now compare with the answer choices: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest match is **B: 56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We use the formula $ E = k \frac{q}{r^2} $ to find the charge $ q $. Substituting the given values and simplifying, we find that the charge is approximately 55.6 pC, which is closest to option B. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2281424) step:60 - global_seqlen/min:30111 - global_seqlen/max:89796 - global_seqlen/minmax_diff:59685 - global_seqlen/balanced_min:50047 - global_seqlen/balanced_max:50180 - global_seqlen/mean:50131.125 - actor/entropy:0.4219725430011749 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26761937141418457 - training/rollout_probs_diff_mean:0.0032737525179982185 - training/rollout_probs_diff_std:0.008023563772439957 - training/rollout_actor_probs_pearson_corr:0.9995866417884827 - rollout_corr/rollout_is_mean:0.9999962449073792 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.0261113503656816e-06 - rollout_corr/rollout_is_max:1.9552534818649292 - rollout_corr/rollout_is_min:0.36368343234062195 - rollout_corr/rollout_is_std:0.028026238083839417 - rollout_corr/rollout_is_eff_sample_size:0.999215146470059 - rollout_corr/rollout_is_seq_mean:0.9999516606330872 - rollout_corr/rollout_is_seq_std:0.0010303977178409696 - rollout_corr/rollout_is_seq_max:1.0035020112991333 - rollout_corr/rollout_is_seq_min:0.9954249262809753 - rollout_corr/rollout_is_seq_max_deviation:0.004575073719024658 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4140595034696162) - rollout_corr/training_log_ppl:np.float64(0.3272325733851176) - rollout_corr/kl:np.float64(0.00043046686714831583) - rollout_corr/k3_kl:np.float64(0.0003949974453973937) - rollout_corr/rollout_ppl:np.float64(1.4134283671155572) - rollout_corr/rollout_log_ppl:np.float64(0.32680210378021) - rollout_corr/log_ppl_diff:np.float64(0.00043046960490755737) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008739173354115337) - rollout_corr/log_ppl_diff_max:np.float64(0.006730854511260986) - rollout_corr/log_ppl_diff_min:np.float64(-0.002438843250274658) - rollout_corr/ppl_ratio:np.float64(1.0004312125965953) - rollout_corr/chi2_token:np.float64(0.000722584780305624) - rollout_corr/chi2_seq:np.float64(1.0466190420556813) - actor/pg_loss:np.float64(1.1199153959751129e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015968021341450367) - actor/ppo_kl:np.float64(6.704961512227214e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12008443288505077) - perf/mfu/actor:np.float64(0.1647866551242615) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.77543354034424) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.671875) - val-aux/sciknoweval/reward/std@16:np.float64(0.2240626490983253) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7666999999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1971325056706902) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5781125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18312620134038018) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.671675) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2244978550800715) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8490500000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14877363384575418) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5088) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1262359553956552) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6999125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.19120944443302226) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.9071374999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09393121099917975) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.46338749999999995) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08209148087342806) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7191875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15548933759230438) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9402375) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.050091787700358235) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4305) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05314562557979722) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7322124999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.12204692054822548) - val-aux/sciknoweval/score/mean@16:np.float64(0.671875) - val-aux/sciknoweval/score/std@16:np.float64(0.2240626490983253) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7666999999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1971325056706902) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5781125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.18312620134038018) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.671675) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2244978550800715) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8490500000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14877363384575418) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5088) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1262359553956552) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6999125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.19120944443302226) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.9071374999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09393121099917975) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.46338749999999995) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08209148087342806) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7191875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15548933759230438) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9402375) - val-aux/sciknoweval/score/best@16/std:np.float64(0.050091787700358235) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4305) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05314562557979722) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7322124999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.12204692054822548) - val-core/sciknoweval/acc/mean@16:np.float64(0.671875) - val-aux/sciknoweval/acc/std@16:np.float64(0.2240626490983253) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7666999999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1971325056706902) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5781125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.18312620134038018) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.671675) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2244978550800715) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8490500000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14877363384575418) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5088) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1262359553956552) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6999125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.19120944443302226) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.9071374999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09393121099917975) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.46338749999999995) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08209148087342806) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7191875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15548933759230438) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9402375) - val-core/sciknoweval/acc/best@16/std:np.float64(0.050091787700358235) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4305) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05314562557979722) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7322124999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.12204692054822548) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.95078125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.09649250755143636) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.98285) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.043210438568887635) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9181125) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.11503551772140294) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9507125000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.09563050360637051) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9944124999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.014654216053618907) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8752125) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12749716856579513) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9671624999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0726457037962381) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9986749999999999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0056780934585833245) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8206) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.12838320809724896) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9803000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.047900368785141687) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9998750000000001) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.001243734296383275) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7596999999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.10958385062712987) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9878125000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.028513078887782435) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015586912631988525 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015586912631988525 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1296.44140625 - response_length/max:4897.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1296.44140625 - response_length_non_aborted/max:4897.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010100007057189941 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3318876773118973) - timing_s/agent_loop/generate_sequences/max:np.float64(37.898281663656235) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.311515245914052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(37.898281663656235) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:317 - timing_s/agent_loop/slowest/response_length:4897 - timing_s/gen:44.70199620164931 - timing_s/reward:0.09790181741118431 - timing_s/old_log_prob:3.9458488430827856 - timing_s/adv:0.12571285665035248 - timing_s/update_actor:16.154612444341183 - timing_s/step:65.11406415887177 - timing_s/testing:175.5087940171361 - timing_s/save_checkpoint:12.78239992260933 - timing_s/stop_profile:0.00016752071678638458 - timing_per_token_ms/gen:0.13468959863583702 - timing_per_token_ms/update_actor:0.040280894465118185 - timing_per_token_ms/adv:0.00031346009253321286 - perf/total_num_tokens:401049 - perf/time_per_step:65.11406415887177 - perf/throughput:769.8970360333383 (TaskRunner pid=2281424) Training Progress: 60%|██████ | 60/100 [1:13:01<1:27:36, 131.40s/it] (TaskRunner pid=2281424) step:61 - global_seqlen/min:45752 - global_seqlen/max:77521 - global_seqlen/minmax_diff:31769 - global_seqlen/balanced_min:56607 - global_seqlen/balanced_max:56958 - global_seqlen/mean:56856.125 - actor/entropy:0.39492928981781006 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6552004814147949 - training/rollout_probs_diff_mean:0.003135534469038248 - training/rollout_probs_diff_std:0.007907739840447903 - training/rollout_actor_probs_pearson_corr:0.9994683861732483 - rollout_corr/rollout_is_mean:0.9999828338623047 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.567888486737502e-06 - rollout_corr/rollout_is_max:1.8672653436660767 - rollout_corr/rollout_is_min:0.29101717472076416 - rollout_corr/rollout_is_std:0.027300119400024414 - rollout_corr/rollout_is_eff_sample_size:0.9992550205113556 - rollout_corr/rollout_is_seq_mean:1.0000195503234863 - rollout_corr/rollout_is_seq_std:0.0009980873437598348 - rollout_corr/rollout_is_seq_max:1.0043768882751465 - rollout_corr/rollout_is_seq_min:0.9967591166496277 - rollout_corr/rollout_is_seq_max_deviation:0.004376888275146484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.408340530935675) - rollout_corr/training_log_ppl:np.float64(0.324529722915031) - rollout_corr/kl:np.float64(0.00038801542386091015) - rollout_corr/k3_kl:np.float64(0.00042000259992391875) - rollout_corr/rollout_ppl:np.float64(1.407751803752035) - rollout_corr/rollout_log_ppl:np.float64(0.32414170513220597) - rollout_corr/log_ppl_diff:np.float64(0.0003880177828250453) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008422878890996799) - rollout_corr/log_ppl_diff_max:np.float64(0.005800724029541016) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036716610193252563) - rollout_corr/ppl_ratio:np.float64(1.0003886730410159) - rollout_corr/chi2_token:np.float64(0.0008997754193842411) - rollout_corr/chi2_seq:np.float64(0.5856562717817724) - actor/pg_loss:np.float64(9.93476714938879e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020609253107295444) - actor/ppo_kl:np.float64(1.0061916269954452e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15778088383376598) - perf/mfu/actor:np.float64(0.17557393231026636) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.95418548583984) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009713038452900946 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009713038452900946 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1521.19140625 - response_length/max:6659.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1521.19140625 - response_length_non_aborted/max:6659.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001254584640264511 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8240225315093994) - timing_s/agent_loop/generate_sequences/max:np.float64(49.38348316960037) - timing_s/agent_loop/generate_sequences/mean:np.float64(15.205760123368236) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.38348316960037) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:6659 - timing_s/gen:55.99783328920603 - timing_s/reward:0.10260644741356373 - timing_s/old_log_prob:4.418327359482646 - timing_s/adv:0.13144686073064804 - timing_s/update_actor:17.245223624631763 - timing_s/step:77.9999400805682 - timing_s/stop_profile:0.00013169646263122559 - timing_per_token_ms/gen:0.14379619513181235 - timing_per_token_ms/update_actor:0.03791417288953425 - timing_per_token_ms/adv:0.00028899010601462913 - perf/total_num_tokens:454849 - perf/time_per_step:77.9999400805682 - perf/throughput:728.9252394459767 (TaskRunner pid=2281424) Training Progress: 61%|██████ | 61/100 [1:14:19<1:15:00, 115.40s/it] (TaskRunner pid=2281424) step:62 - global_seqlen/min:28896 - global_seqlen/max:104683 - global_seqlen/minmax_diff:75787 - global_seqlen/balanced_min:67049 - global_seqlen/balanced_max:67213 - global_seqlen/mean:67134.0 - actor/entropy:0.42600366473197937 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8567732572555542 - training/rollout_probs_diff_mean:0.0032150347251445055 - training/rollout_probs_diff_std:0.00793883204460144 - training/rollout_actor_probs_pearson_corr:0.9996106028556824 - rollout_corr/rollout_is_mean:1.0000700950622559 - rollout_corr/rollout_is_ratio_fraction_high:2.15829822991509e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.47489468974527e-06 - rollout_corr/rollout_is_max:2.125422954559326 - rollout_corr/rollout_is_min:0.003053532214835286 - rollout_corr/rollout_is_std:0.027791311964392662 - rollout_corr/rollout_is_eff_sample_size:0.9992283581133058 - rollout_corr/rollout_is_seq_mean:1.0000699758529663 - rollout_corr/rollout_is_seq_std:0.001128942472860217 - rollout_corr/rollout_is_seq_max:1.0039699077606201 - rollout_corr/rollout_is_seq_min:0.9920714497566223 - rollout_corr/rollout_is_seq_max_deviation:0.007928550243377686 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4691616958007216) - rollout_corr/training_log_ppl:np.float64(0.3653776543796994) - rollout_corr/kl:np.float64(0.00039364140611319876) - rollout_corr/k3_kl:np.float64(0.00042572843989319153) - rollout_corr/rollout_ppl:np.float64(1.4685669397003949) - rollout_corr/rollout_log_ppl:np.float64(0.36498401110293344) - rollout_corr/log_ppl_diff:np.float64(0.0003936432767659426) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008307227399200201) - rollout_corr/log_ppl_diff_max:np.float64(0.009543061256408691) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032494962215423584) - rollout_corr/ppl_ratio:np.float64(1.0003944088239223) - rollout_corr/chi2_token:np.float64(0.0009197669569402933) - rollout_corr/chi2_seq:np.float64(1.6601045140996575) - actor/pg_loss:np.float64(-9.474251419305801e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011590343615353049) - actor/ppo_kl:np.float64(4.716461066323063e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10818999446928501) - perf/mfu/actor:np.float64(0.19350604047505943) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.04900360107422) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0014814019668847322 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0014814019668847322 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1809.875 - response_length/max:6139.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1809.875 - response_length_non_aborted/max:6139.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.605474770069122e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1590842586010695) - timing_s/agent_loop/generate_sequences/max:np.float64(48.22148149460554) - timing_s/agent_loop/generate_sequences/mean:np.float64(18.0422284047454) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(48.22148149460554) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:6139 - timing_s/gen:55.47001213952899 - timing_s/reward:0.11935079656541348 - timing_s/old_log_prob:4.546598572283983 - timing_s/adv:0.14351257495582104 - timing_s/update_actor:18.579031608998775 - timing_s/step:78.9453232511878 - timing_s/stop_profile:0.00011393986642360687 - timing_per_token_ms/gen:0.11972082874233586 - timing_per_token_ms/update_actor:0.0345931860327829 - timing_per_token_ms/adv:0.00026721291550447807 - perf/total_num_tokens:537072 - perf/time_per_step:78.9453232511878 - perf/throughput:850.386029662497 (TaskRunner pid=2281424) Training Progress: 62%|██████▏ | 62/100 [1:15:38<1:06:10, 104.48s/it] (TaskRunner pid=2281424) step:63 - global_seqlen/min:33404 - global_seqlen/max:94971 - global_seqlen/minmax_diff:61567 - global_seqlen/balanced_min:53259 - global_seqlen/balanced_max:54267 - global_seqlen/mean:53515.0 - actor/entropy:0.45275378227233887 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18870103359222412 - training/rollout_probs_diff_mean:0.00339776324108243 - training/rollout_probs_diff_std:0.008000901900231838 - training/rollout_actor_probs_pearson_corr:0.9995976686477661 - rollout_corr/rollout_is_mean:1.0000234842300415 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.779074748104904e-06 - rollout_corr/rollout_is_max:1.6216226816177368 - rollout_corr/rollout_is_min:0.3960243761539459 - rollout_corr/rollout_is_std:0.028292939066886902 - rollout_corr/rollout_is_eff_sample_size:0.9992000308743295 - rollout_corr/rollout_is_seq_mean:1.0000338554382324 - rollout_corr/rollout_is_seq_std:0.0010206034639850259 - rollout_corr/rollout_is_seq_max:1.0040816068649292 - rollout_corr/rollout_is_seq_min:0.9965076446533203 - rollout_corr/rollout_is_seq_max_deviation:0.004081606864929199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.392845130059868) - rollout_corr/training_log_ppl:np.float64(0.30986753129400313) - rollout_corr/kl:np.float64(0.00039362613700077986) - rollout_corr/k3_kl:np.float64(0.0003788294339130971) - rollout_corr/rollout_ppl:np.float64(1.3922858135774732) - rollout_corr/rollout_log_ppl:np.float64(0.30947390163782984) - rollout_corr/log_ppl_diff:np.float64(0.0003936296561732888) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008133103256113827) - rollout_corr/log_ppl_diff_max:np.float64(0.004159331321716309) - rollout_corr/log_ppl_diff_min:np.float64(-0.0025495290756225586) - rollout_corr/ppl_ratio:np.float64(1.0003942425828427) - rollout_corr/chi2_token:np.float64(0.0007350770756602287) - rollout_corr/chi2_seq:np.float64(0.4594551851041615) - actor/pg_loss:np.float64(7.723795715719461e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010548596083026496) - actor/ppo_kl:np.float64(5.877050272280271e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08551348559558392) - perf/mfu/actor:np.float64(0.16753065811491627) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.80081176757812) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03102802112698555 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03102802112698555 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1405.59375 - response_length/max:8192.0 - response_length/min:152.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1405.59375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017119385302066803 - timing_s/agent_loop/generate_sequences/min:np.float64(1.869371073320508) - timing_s/agent_loop/generate_sequences/max:np.float64(67.03272437117994) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.693403950666834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(67.03272437117994) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:74.319850102067 - timing_s/reward:0.1025898028165102 - timing_s/old_log_prob:4.21283820644021 - timing_s/adv:0.12729890830814838 - timing_s/update_actor:17.184931615367532 - timing_s/step:96.03458793275058 - timing_s/stop_profile:0.00012183934450149536 - timing_per_token_ms/gen:0.2065404135876381 - timing_per_token_ms/update_actor:0.04014045504850867 - timing_per_token_ms/adv:0.0002973439883867803 - perf/total_num_tokens:428120 - perf/time_per_step:96.03458793275058 - perf/throughput:557.2471455542095 (TaskRunner pid=2281424) Training Progress: 63%|██████▎ | 63/100 [1:17:15<1:02:52, 101.96s/it] (TaskRunner pid=2281424) step:64 - global_seqlen/min:30970 - global_seqlen/max:90543 - global_seqlen/minmax_diff:59573 - global_seqlen/balanced_min:59596 - global_seqlen/balanced_max:59937 - global_seqlen/mean:59830.0 - actor/entropy:0.43621382117271423 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4681967496871948 - training/rollout_probs_diff_mean:0.0032422388903796673 - training/rollout_probs_diff_std:0.007834543474018574 - training/rollout_actor_probs_pearson_corr:0.999615490436554 - rollout_corr/rollout_is_mean:1.000067949295044 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.873009402217576e-06 - rollout_corr/rollout_is_max:1.9102002382278442 - rollout_corr/rollout_is_min:0.2423228621482849 - rollout_corr/rollout_is_std:0.02796875685453415 - rollout_corr/rollout_is_eff_sample_size:0.999218360080856 - rollout_corr/rollout_is_seq_mean:1.0001271963119507 - rollout_corr/rollout_is_seq_std:0.0008784870151430368 - rollout_corr/rollout_is_seq_max:1.0027687549591064 - rollout_corr/rollout_is_seq_min:0.9961645007133484 - rollout_corr/rollout_is_seq_max_deviation:0.0038354992866516113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4255081894807518) - rollout_corr/training_log_ppl:np.float64(0.33409654538263567) - rollout_corr/kl:np.float64(0.0003069534103872229) - rollout_corr/k3_kl:np.float64(0.0004051391514963143) - rollout_corr/rollout_ppl:np.float64(1.425030098296702) - rollout_corr/rollout_log_ppl:np.float64(0.3337895911245141) - rollout_corr/log_ppl_diff:np.float64(0.0003069542581215501) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007710371864959598) - rollout_corr/log_ppl_diff_max:np.float64(0.00480864942073822) - rollout_corr/log_ppl_diff_min:np.float64(-0.002487003803253174) - rollout_corr/ppl_ratio:np.float64(1.0003075052518398) - rollout_corr/chi2_token:np.float64(0.0010234767105430365) - rollout_corr/chi2_seq:np.float64(1.0248696226626635) - actor/pg_loss:np.float64(3.9162347093224525e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00013449225230033335) - actor/ppo_kl:np.float64(7.314733183516609e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09787004068493843) - perf/mfu/actor:np.float64(0.18222994298278014) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.78503465652466) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012692666612565517 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012692666612565517 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1603.21875 - response_length/max:6504.0 - response_length/min:235.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1603.21875 - response_length_non_aborted/max:6504.0 - response_length_non_aborted/min:235.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016041100025177002 - timing_s/agent_loop/generate_sequences/min:np.float64(3.038218954578042) - timing_s/agent_loop/generate_sequences/max:np.float64(48.57417339272797) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.26644155262329) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(48.57417339272797) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:212 - timing_s/agent_loop/slowest/response_length:6504 - timing_s/gen:55.43104035779834 - timing_s/reward:0.1111034657806158 - timing_s/old_log_prob:4.2981742061674595 - timing_s/adv:0.1171770989894867 - timing_s/update_actor:17.636247349902987 - timing_s/step:77.67978408560157 - timing_s/stop_profile:0.00011309981346130371 - timing_per_token_ms/gen:0.1350579896833478 - timing_per_token_ms/update_actor:0.03684658062406608 - timing_per_token_ms/adv:0.0002448125919051619 - perf/total_num_tokens:478640 - perf/time_per_step:77.67978408560157 - perf/throughput:770.2132633899772 (TaskRunner pid=2281424) Training Progress: 64%|██████▍ | 64/100 [1:18:32<56:48, 94.69s/it] (TaskRunner pid=2281424) step:65 - global_seqlen/min:21582 - global_seqlen/max:63673 - global_seqlen/minmax_diff:42091 - global_seqlen/balanced_min:47500 - global_seqlen/balanced_max:47999 - global_seqlen/mean:47802.125 - actor/entropy:0.39129528403282166 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21986928582191467 - training/rollout_probs_diff_mean:0.003225617576390505 - training/rollout_probs_diff_std:0.008090500719845295 - training/rollout_actor_probs_pearson_corr:0.9995739459991455 - rollout_corr/rollout_is_mean:0.9999448657035828 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6904900074005127 - rollout_corr/rollout_is_min:0.5123902559280396 - rollout_corr/rollout_is_std:0.027487242594361305 - rollout_corr/rollout_is_eff_sample_size:0.9992447838866899 - rollout_corr/rollout_is_seq_mean:0.9998865127563477 - rollout_corr/rollout_is_seq_std:0.0011093486100435257 - rollout_corr/rollout_is_seq_max:1.0034520626068115 - rollout_corr/rollout_is_seq_min:0.9955258369445801 - rollout_corr/rollout_is_seq_max_deviation:0.004474163055419922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3684682152234018) - rollout_corr/training_log_ppl:np.float64(0.29601700889179483) - rollout_corr/kl:np.float64(0.00043714539492256677) - rollout_corr/k3_kl:np.float64(0.00038878725348467924) - rollout_corr/rollout_ppl:np.float64(1.3678660304285586) - rollout_corr/rollout_log_ppl:np.float64(0.295579861412989) - rollout_corr/log_ppl_diff:np.float64(0.0004371474788058549) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009100331517402083) - rollout_corr/log_ppl_diff_max:np.float64(0.005311831831932068) - rollout_corr/log_ppl_diff_min:np.float64(-0.004062667489051819) - rollout_corr/ppl_ratio:np.float64(1.0004379414021969) - rollout_corr/chi2_token:np.float64(0.0006793311331421137) - rollout_corr/chi2_seq:np.float64(1.2524537781719118) - actor/pg_loss:np.float64(0.0001103521790355444) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.197616297891727e-05) - actor/ppo_kl:np.float64(-5.6069447737172595e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08994254469871521) - perf/mfu/actor:np.float64(0.15645029922787607) - perf/max_memory_allocated_gb:np.float64(131.87133502960205) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.88603591918945) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.8671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006594820413738489 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006594820413738489 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1230.47265625 - response_length/max:7182.0 - response_length/min:212.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1230.47265625 - response_length_non_aborted/max:7182.0 - response_length_non_aborted/min:212.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.158067405223846e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5644865073263645) - timing_s/agent_loop/generate_sequences/max:np.float64(49.72097803838551) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.574511633109068) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.72097803838551) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:7182 - timing_s/gen:57.23068424500525 - timing_s/reward:0.09001462161540985 - timing_s/old_log_prob:3.9613787718117237 - timing_s/adv:0.12929501570761204 - timing_s/update_actor:16.40677917189896 - timing_s/step:77.87616940587759 - timing_s/stop_profile:0.0001377630978822708 - timing_per_token_ms/gen:0.18168413511387344 - timing_per_token_ms/update_actor:0.04290284995671992 - timing_per_token_ms/adv:0.00033809955024910514 - perf/total_num_tokens:382417 - perf/time_per_step:77.87616940587759 - perf/throughput:613.8222432444425 (TaskRunner pid=2281424) Training Progress: 65%|██████▌ | 65/100 [1:19:50<52:18, 89.66s/it] (TaskRunner pid=2281424) step:66 - global_seqlen/min:33934 - global_seqlen/max:97523 - global_seqlen/minmax_diff:63589 - global_seqlen/balanced_min:63235 - global_seqlen/balanced_max:63618 - global_seqlen/mean:63460.5 - actor/entropy:0.4836350083351135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38506191968917847 - training/rollout_probs_diff_mean:0.00350581924431026 - training/rollout_probs_diff_std:0.00794005673378706 - training/rollout_actor_probs_pearson_corr:0.9995872378349304 - rollout_corr/rollout_is_mean:0.9999473094940186 - rollout_corr/rollout_is_ratio_fraction_high:2.293304532940965e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.293304532940965e-06 - rollout_corr/rollout_is_max:3.7363619804382324 - rollout_corr/rollout_is_min:0.45649924874305725 - rollout_corr/rollout_is_std:0.02884167805314064 - rollout_corr/rollout_is_eff_sample_size:0.9991688489642058 - rollout_corr/rollout_is_seq_mean:0.9999408721923828 - rollout_corr/rollout_is_seq_std:0.0010089804418385029 - rollout_corr/rollout_is_seq_max:1.0059617757797241 - rollout_corr/rollout_is_seq_min:0.9970559477806091 - rollout_corr/rollout_is_seq_max_deviation:0.005961775779724121 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4541357709094882) - rollout_corr/training_log_ppl:np.float64(0.3529185401275754) - rollout_corr/kl:np.float64(0.0004487401768882293) - rollout_corr/k3_kl:np.float64(0.00045227643718703803) - rollout_corr/rollout_ppl:np.float64(1.4534654365852475) - rollout_corr/rollout_log_ppl:np.float64(0.3524697999237105) - rollout_corr/log_ppl_diff:np.float64(0.00044874020386487246) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009008654160425067) - rollout_corr/log_ppl_diff_max:np.float64(0.004554241895675659) - rollout_corr/log_ppl_diff_min:np.float64(-0.005111396312713623) - rollout_corr/ppl_ratio:np.float64(1.000449490500614) - rollout_corr/chi2_token:np.float64(0.0009134733118116856) - rollout_corr/chi2_seq:np.float64(13.181665818439797) - actor/pg_loss:np.float64(5.611649248749018e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.645304444598878e-05) - actor/ppo_kl:np.float64(5.086660891162609e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08397138444706798) - perf/mfu/actor:np.float64(0.1864176210922637) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.92216110229492) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02002083510160446 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02002083510160446 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1703.328125 - response_length/max:8192.0 - response_length/min:198.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1703.328125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:198.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.438022971153259e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.710974918678403) - timing_s/agent_loop/generate_sequences/max:np.float64(59.87396731041372) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.808275037481508) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.87396731041372) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:66.76294595561922 - timing_s/reward:0.1163025051355362 - timing_s/old_log_prob:4.507007392123342 - timing_s/adv:0.12916515208780766 - timing_s/update_actor:18.44666711241007 - timing_s/step:90.04977263323963 - timing_s/stop_profile:0.00011244788765907288 - timing_per_token_ms/gen:0.15310776227518558 - timing_per_token_ms/update_actor:0.03633493888405005 - timing_per_token_ms/adv:0.00025442037190025223 - perf/total_num_tokens:507684 - perf/time_per_step:90.04977263323963 - perf/throughput:704.7269320541865 (TaskRunner pid=2281424) Training Progress: 66%|██████▌ | 66/100 [1:21:20<50:52, 89.79s/it] (TaskRunner pid=2281424) step:67 - global_seqlen/min:29846 - global_seqlen/max:66602 - global_seqlen/minmax_diff:36756 - global_seqlen/balanced_min:48064 - global_seqlen/balanced_max:48303 - global_seqlen/mean:48233.875 - actor/entropy:0.4240332841873169 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18446603417396545 - training/rollout_probs_diff_mean:0.0033334451727569103 - training/rollout_probs_diff_std:0.00801022257655859 - training/rollout_actor_probs_pearson_corr:0.9995075464248657 - rollout_corr/rollout_is_mean:1.000016450881958 - rollout_corr/rollout_is_ratio_fraction_high:3.1456927445105975e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.291385489021195e-06 - rollout_corr/rollout_is_max:2.192669630050659 - rollout_corr/rollout_is_min:0.3415142297744751 - rollout_corr/rollout_is_std:0.028202205896377563 - rollout_corr/rollout_is_eff_sample_size:0.9992053867417628 - rollout_corr/rollout_is_seq_mean:0.9999523758888245 - rollout_corr/rollout_is_seq_std:0.0010149587178602815 - rollout_corr/rollout_is_seq_max:1.0026092529296875 - rollout_corr/rollout_is_seq_min:0.9962970018386841 - rollout_corr/rollout_is_seq_max_deviation:0.003702998161315918 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.456437811255455) - rollout_corr/training_log_ppl:np.float64(0.3491874962928705) - rollout_corr/kl:np.float64(0.000549241650548149) - rollout_corr/k3_kl:np.float64(0.0004329512739786878) - rollout_corr/rollout_ppl:np.float64(1.4555674134753644) - rollout_corr/rollout_log_ppl:np.float64(0.3486382510745898) - rollout_corr/log_ppl_diff:np.float64(0.0005492452182807028) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009298735531046987) - rollout_corr/log_ppl_diff_max:np.float64(0.005370736122131348) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034429728984832764) - rollout_corr/ppl_ratio:np.float64(1.0005500647239387) - rollout_corr/chi2_token:np.float64(0.0006130961701273918) - rollout_corr/chi2_seq:np.float64(3.0569681401830167) - actor/pg_loss:np.float64(-7.326365448534489e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.657254923287837e-05) - actor/ppo_kl:np.float64(0.00010237288070413797) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0874917283654213) - perf/mfu/actor:np.float64(0.15933752928759692) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.65837478637695) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010827867314219475 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010827867314219475 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1241.77734375 - response_length/max:6316.0 - response_length/min:226.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1241.77734375 - response_length_non_aborted/max:6316.0 - response_length_non_aborted/min:226.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00024794600903987885 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7115938980132341) - timing_s/agent_loop/generate_sequences/max:np.float64(44.07025810703635) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.514710396375449) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(44.07025810703635) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:6316 - timing_s/gen:51.214709755033255 - timing_s/reward:0.09305312111973763 - timing_s/old_log_prob:4.1541140507906675 - timing_s/adv:0.1196837779134512 - timing_s/update_actor:16.18540153838694 - timing_s/step:71.87573814950883 - timing_s/stop_profile:7.292069494724274e-05 - timing_per_token_ms/gen:0.1611057416915436 - timing_per_token_ms/update_actor:0.041945109993720545 - timing_per_token_ms/adv:0.0003101652570767204 - perf/total_num_tokens:385871 - perf/time_per_step:71.87573814950883 - perf/throughput:671.0731081421194 (TaskRunner pid=2281424) Training Progress: 67%|██████▋ | 67/100 [1:22:33<46:30, 84.55s/it] (TaskRunner pid=2281424) step:68 - global_seqlen/min:29366 - global_seqlen/max:71127 - global_seqlen/minmax_diff:41761 - global_seqlen/balanced_min:45241 - global_seqlen/balanced_max:45620 - global_seqlen/mean:45320.875 - actor/entropy:0.3679567873477936 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846073567867279 - training/rollout_probs_diff_mean:0.003109095385298133 - training/rollout_probs_diff_std:0.007897583767771721 - training/rollout_actor_probs_pearson_corr:0.9995740056037903 - rollout_corr/rollout_is_mean:0.9999691247940063 - rollout_corr/rollout_is_ratio_fraction_high:3.393983888599905e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.393983888599905e-06 - rollout_corr/rollout_is_max:3.361013650894165 - rollout_corr/rollout_is_min:0.496554970741272 - rollout_corr/rollout_is_std:0.02703685127198696 - rollout_corr/rollout_is_eff_sample_size:0.9992695425945294 - rollout_corr/rollout_is_seq_mean:1.0000405311584473 - rollout_corr/rollout_is_seq_std:0.001029360806569457 - rollout_corr/rollout_is_seq_max:1.0070128440856934 - rollout_corr/rollout_is_seq_min:0.9971495866775513 - rollout_corr/rollout_is_seq_max_deviation:0.007012844085693359 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3687643725425005) - rollout_corr/training_log_ppl:np.float64(0.3007245181070175) - rollout_corr/kl:np.float64(0.0003466152552178414) - rollout_corr/k3_kl:np.float64(0.0004064006222961325) - rollout_corr/rollout_ppl:np.float64(1.368270150385797) - rollout_corr/rollout_log_ppl:np.float64(0.3003779045538977) - rollout_corr/log_ppl_diff:np.float64(0.00034661355311982334) - rollout_corr/log_ppl_abs_diff:np.float64(0.000815852457890287) - rollout_corr/log_ppl_diff_max:np.float64(0.0050161778926849365) - rollout_corr/log_ppl_diff_min:np.float64(-0.00573354959487915) - rollout_corr/ppl_ratio:np.float64(1.0003472932148725) - rollout_corr/chi2_token:np.float64(0.0009375796653330326) - rollout_corr/chi2_seq:np.float64(0.4990305930841714) - actor/pg_loss:np.float64(4.833098500967026e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.405877534618412e-05) - actor/ppo_kl:np.float64(4.350960717935948e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09933188185095787) - perf/mfu/actor:np.float64(0.15303388314863475) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.6871109008789) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008664840832352638 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008664840832352638 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1150.93359375 - response_length/max:6380.0 - response_length/min:199.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1150.93359375 - response_length_non_aborted/max:6380.0 - response_length_non_aborted/min:199.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.791895091533661e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0759412702172995) - timing_s/agent_loop/generate_sequences/max:np.float64(44.35951707884669) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.321704763286107) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(44.35951707884669) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:317 - timing_s/agent_loop/slowest/response_length:6380 - timing_s/gen:51.27926462888718 - timing_s/reward:0.09004293940961361 - timing_s/old_log_prob:3.896944200620055 - timing_s/adv:0.12336954846978188 - timing_s/update_actor:15.747877143323421 - timing_s/step:71.23202810995281 - timing_s/stop_profile:0.00013591721653938293 - timing_per_token_ms/gen:0.1740409946710625 - timing_per_token_ms/update_actor:0.04343439183191913 - timing_per_token_ms/adv:0.0003402668981727015 - perf/total_num_tokens:362567 - perf/time_per_step:71.23202810995281 - perf/throughput:636.2429401847621 (TaskRunner pid=2281424) Training Progress: 68%|██████▊ | 68/100 [1:23:44<42:58, 80.57s/it] (TaskRunner pid=2281424) step:69 - global_seqlen/min:35402 - global_seqlen/max:71677 - global_seqlen/minmax_diff:36275 - global_seqlen/balanced_min:51417 - global_seqlen/balanced_max:51537 - global_seqlen/mean:51488.375 - actor/entropy:0.41125714778900146 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462836742401123 - training/rollout_probs_diff_mean:0.0032182156573981047 - training/rollout_probs_diff_std:0.007813959382474422 - training/rollout_actor_probs_pearson_corr:0.9995726346969604 - rollout_corr/rollout_is_mean:0.9999262094497681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.918830205089762e-06 - rollout_corr/rollout_is_max:1.644830346107483 - rollout_corr/rollout_is_min:0.48185646533966064 - rollout_corr/rollout_is_std:0.027313215658068657 - rollout_corr/rollout_is_eff_sample_size:0.9992545443846275 - rollout_corr/rollout_is_seq_mean:0.9999979734420776 - rollout_corr/rollout_is_seq_std:0.0008565172320231795 - rollout_corr/rollout_is_seq_max:1.0034431219100952 - rollout_corr/rollout_is_seq_min:0.9972632527351379 - rollout_corr/rollout_is_seq_max_deviation:0.003443121910095215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3975837221369147) - rollout_corr/training_log_ppl:np.float64(0.3145434020116227) - rollout_corr/kl:np.float64(0.0003842490903210871) - rollout_corr/k3_kl:np.float64(0.000364644451536833) - rollout_corr/rollout_ppl:np.float64(1.3969996976666152) - rollout_corr/rollout_log_ppl:np.float64(0.31415915075922385) - rollout_corr/log_ppl_diff:np.float64(0.0003842512523988262) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007727831398369744) - rollout_corr/log_ppl_diff_max:np.float64(0.0032014399766921997) - rollout_corr/log_ppl_diff_min:np.float64(-0.002889871597290039) - rollout_corr/ppl_ratio:np.float64(1.0003847263287753) - rollout_corr/chi2_token:np.float64(0.0007037864997982979) - rollout_corr/chi2_seq:np.float64(0.6845616844948381) - actor/pg_loss:np.float64(-3.3041927963495255e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.254580268385325e-05) - actor/ppo_kl:np.float64(3.3652320112764755e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08329814206808805) - perf/mfu/actor:np.float64(0.166977802348238) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.75886535644531) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006434926297515631 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006434926297515631 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1338.29296875 - response_length/max:5606.0 - response_length/min:269.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1338.29296875 - response_length_non_aborted/max:5606.0 - response_length_non_aborted/min:269.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010468810796737671 - timing_s/agent_loop/generate_sequences/min:np.float64(3.5091678723692894) - timing_s/agent_loop/generate_sequences/max:np.float64(41.96201452054083) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.428402047728014) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(41.96201452054083) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:5606 - timing_s/gen:48.85266746021807 - timing_s/reward:0.09144083969295025 - timing_s/old_log_prob:3.999005449935794 - timing_s/adv:0.11515455320477486 - timing_s/update_actor:16.556827584281564 - timing_s/step:69.7196654882282 - timing_s/stop_profile:3.0202791094779968e-05 - timing_per_token_ms/gen:0.14259264355600526 - timing_per_token_ms/update_actor:0.04019554798602977 - timing_per_token_ms/adv:0.00027956444829724876 - perf/total_num_tokens:411907 - perf/time_per_step:69.7196654882282 - perf/throughput:738.5057664783768 (TaskRunner pid=2281424) Training Progress: 69%|██████▉ | 69/100 [1:24:54<39:56, 77.32s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] To find the magnitude of the particle's charge, we can use the formula for the electric field due to a point charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given: (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 1: Calculate $ r^2 $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) r^2 = (0.50)^2 = 0.25 \, \mathrm{m^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 2: Plug into the formula (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 3: Convert to picocoulombs (pC) (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 4: Choose the closest answer (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest option to 55.6 pC is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) **B: 56 pC** (TaskRunner pid=2281424) (TaskRunner pid=2281424) --- (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We use the formula for the electric field due to a point charge, $ E = k \frac{q}{r^2} $, and solve for $ q $. We calculate $ r^2 = 0.25 \, \mathrm{m^2} $, plug into the formula, and find $ q \approx 5.56 \times 10^{-11} \, \mathrm{C} $, which is approximately 55.6 pC. The closest answer is B. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2281424) step:70 - global_seqlen/min:28634 - global_seqlen/max:74589 - global_seqlen/minmax_diff:45955 - global_seqlen/balanced_min:51601 - global_seqlen/balanced_max:52935 - global_seqlen/mean:51969.25 - actor/entropy:0.463895708322525 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6431382894515991 - training/rollout_probs_diff_mean:0.0034441002644598484 - training/rollout_probs_diff_std:0.008035892620682716 - training/rollout_actor_probs_pearson_corr:0.9996177554130554 - rollout_corr/rollout_is_mean:0.9999815225601196 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.8799022402381524e-06 - rollout_corr/rollout_is_max:1.7186620235443115 - rollout_corr/rollout_is_min:0.18310591578483582 - rollout_corr/rollout_is_std:0.028500739485025406 - rollout_corr/rollout_is_eff_sample_size:0.9991883671839248 - rollout_corr/rollout_is_seq_mean:1.0000474452972412 - rollout_corr/rollout_is_seq_std:0.0009564801002852619 - rollout_corr/rollout_is_seq_max:1.0044219493865967 - rollout_corr/rollout_is_seq_min:0.9975866675376892 - rollout_corr/rollout_is_seq_max_deviation:0.00442194938659668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4960705381818116) - rollout_corr/training_log_ppl:np.float64(0.38029307904071175) - rollout_corr/kl:np.float64(0.00034536135440355054) - rollout_corr/k3_kl:np.float64(0.0003967561805495734) - rollout_corr/rollout_ppl:np.float64(1.4955097283236682) - rollout_corr/rollout_log_ppl:np.float64(0.3799477120919619) - rollout_corr/log_ppl_diff:np.float64(0.00034536694874987006) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008130436181090772) - rollout_corr/log_ppl_diff_max:np.float64(0.0031409859657287598) - rollout_corr/log_ppl_diff_min:np.float64(-0.002258613705635071) - rollout_corr/ppl_ratio:np.float64(1.0003458971623331) - rollout_corr/chi2_token:np.float64(0.0009016424883157015) - rollout_corr/chi2_seq:np.float64(2.9101600360590965) - actor/pg_loss:np.float64(-6.507872603833675e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.506012740852384e-05) - actor/ppo_kl:np.float64(7.824350067409114e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11454777047038078) - perf/mfu/actor:np.float64(0.16590592651685154) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(106.71920394897461) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.68125) - val-aux/sciknoweval/reward/std@16:np.float64(0.2137127463842002) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.769025) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1809424350916951) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5937875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1784672156379507) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6828625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21320174521790633) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8434625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1347694924147861) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5284375) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13266359016202112) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.711225) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17773838845597045) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8962875) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0854184909297855) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47755000000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10070722739093489) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7298625000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14406533830862692) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9257875) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.045417892588033096) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4352875) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07488497149607146) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7414875) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11564756114569612) - val-aux/sciknoweval/score/mean@16:np.float64(0.68125) - val-aux/sciknoweval/score/std@16:np.float64(0.2137127463842002) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.769025) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1809424350916951) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5937875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1784672156379507) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6828625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21320174521790633) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8434625) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1347694924147861) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5284375) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13266359016202112) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.711225) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17773838845597045) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8962875) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0854184909297855) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.47755000000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10070722739093489) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7298625000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14406533830862692) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9257875) - val-aux/sciknoweval/score/best@16/std:np.float64(0.045417892588033096) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4352875) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07488497149607146) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7414875) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11564756114569612) - val-core/sciknoweval/acc/mean@16:np.float64(0.68125) - val-aux/sciknoweval/acc/std@16:np.float64(0.2137127463842002) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.769025) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1809424350916951) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5937875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1784672156379507) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6828625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21320174521790633) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8434625) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1347694924147861) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5284375) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13266359016202112) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.711225) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17773838845597045) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8962875) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0854184909297855) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.47755000000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10070722739093489) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7298625000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14406533830862692) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9257875) - val-core/sciknoweval/acc/best@16/std:np.float64(0.045417892588033096) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4352875) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07488497149607146) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7414875) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11564756114569612) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.12121925691237849) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9806625) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.05945683002695111) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.89495) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.1434002448368515) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.93955) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.11972140703255145) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9960875000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.01660654402930483) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8371375000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.15424155659422464) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9604250000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.09268990099722656) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.999625) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0033009411325088988) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7685375000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.14442824247076974) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9791624999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.061597817355926635) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7031375) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.11075323791783456) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9912124999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.03468685742406086) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004505247343331575 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.004505247343331575 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:1356.3828125 - response_length/max:7204.0 - response_length/min:235.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1356.3828125 - response_length_non_aborted/max:7204.0 - response_length_non_aborted/min:235.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.293583333492279e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.054271874949336) - timing_s/agent_loop/generate_sequences/max:np.float64(50.83749855682254) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.613497315665882) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.83749855682254) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:225 - timing_s/agent_loop/slowest/response_length:7204 - timing_s/gen:57.93693597614765 - timing_s/reward:0.1021033264696598 - timing_s/old_log_prob:4.041707821190357 - timing_s/adv:0.12145847268402576 - timing_s/update_actor:16.889128616079688 - timing_s/step:79.18048135004938 - timing_s/testing:186.86356358602643 - timing_s/save_checkpoint:13.130839332938194 - timing_s/stop_profile:0.00013073720037937164 - timing_per_token_ms/gen:0.16685271596717963 - timing_per_token_ms/update_actor:0.04062288905477683 - timing_per_token_ms/adv:0.00029214023841989676 - perf/total_num_tokens:415754 - perf/time_per_step:79.18048135004938 - perf/throughput:656.3391521989982 (TaskRunner pid=2281424) Training Progress: 70%|███████ | 70/100 [1:29:33<1:08:57, 137.90s/it] (TaskRunner pid=2281424) step:71 - global_seqlen/min:28875 - global_seqlen/max:77315 - global_seqlen/minmax_diff:48440 - global_seqlen/balanced_min:54005 - global_seqlen/balanced_max:54274 - global_seqlen/mean:54174.25 - actor/entropy:0.420095831155777 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42047184705734253 - training/rollout_probs_diff_mean:0.00322735495865345 - training/rollout_probs_diff_std:0.007836757227778435 - training/rollout_actor_probs_pearson_corr:0.999700129032135 - rollout_corr/rollout_is_mean:0.9999887943267822 - rollout_corr/rollout_is_ratio_fraction_high:2.7597932330536423e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.519586466107285e-06 - rollout_corr/rollout_is_max:2.1141860485076904 - rollout_corr/rollout_is_min:0.4562623202800751 - rollout_corr/rollout_is_std:0.02742645889520645 - rollout_corr/rollout_is_eff_sample_size:0.9992484738086154 - rollout_corr/rollout_is_seq_mean:0.9999967813491821 - rollout_corr/rollout_is_seq_std:0.0009134845458902419 - rollout_corr/rollout_is_seq_max:1.0040614604949951 - rollout_corr/rollout_is_seq_min:0.997020423412323 - rollout_corr/rollout_is_seq_max_deviation:0.004061460494995117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4078892222605646) - rollout_corr/training_log_ppl:np.float64(0.3253569998778403) - rollout_corr/kl:np.float64(0.00031806869935557813) - rollout_corr/k3_kl:np.float64(0.00037636621712522356) - rollout_corr/rollout_ppl:np.float64(1.4074036730453372) - rollout_corr/rollout_log_ppl:np.float64(0.3250389306922443) - rollout_corr/log_ppl_diff:np.float64(0.0003180691855959594) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007455509621649981) - rollout_corr/log_ppl_diff_max:np.float64(0.004639908671379089) - rollout_corr/log_ppl_diff_min:np.float64(-0.003470480442047119) - rollout_corr/ppl_ratio:np.float64(1.0003185661043972) - rollout_corr/chi2_token:np.float64(0.0008722511120140553) - rollout_corr/chi2_seq:np.float64(0.7835535886697471) - actor/pg_loss:np.float64(8.003273978829384e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.337292018723019e-05) - actor/ppo_kl:np.float64(-4.260811079248583e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10855482891201973) - perf/mfu/actor:np.float64(0.16889551806942366) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.48225784301758) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017720285803079605 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017720285803079605 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1415.4140625 - response_length/max:7054.0 - response_length/min:226.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1415.4140625 - response_length_non_aborted/max:7054.0 - response_length_non_aborted/min:226.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001040436327457428 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8338771276175976) - timing_s/agent_loop/generate_sequences/max:np.float64(49.44516071304679) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.63207975136902) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.44516071304679) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:7054 - timing_s/gen:55.94673459418118 - timing_s/reward:0.09744138084352016 - timing_s/old_log_prob:4.294410843402147 - timing_s/adv:0.1294803023338318 - timing_s/update_actor:17.14490484073758 - timing_s/step:77.71968797035515 - timing_s/stop_profile:0.00013727881014347076 - timing_per_token_ms/gen:0.15440141354998035 - timing_per_token_ms/update_actor:0.03955962666935302 - timing_per_token_ms/adv:0.00029875887145145477 - perf/total_num_tokens:433394 - perf/time_per_step:77.71968797035515 - perf/throughput:697.0466739478398 (TaskRunner pid=2281424) Training Progress: 71%|███████ | 71/100 [1:30:51<57:56, 119.87s/it] (TaskRunner pid=2281424) step:72 - global_seqlen/min:33672 - global_seqlen/max:79500 - global_seqlen/minmax_diff:45828 - global_seqlen/balanced_min:55942 - global_seqlen/balanced_max:56271 - global_seqlen/mean:56194.875 - actor/entropy:0.40107372403144836 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7170437574386597 - training/rollout_probs_diff_mean:0.003104705596342683 - training/rollout_probs_diff_std:0.007854383438825607 - training/rollout_actor_probs_pearson_corr:0.9995812177658081 - rollout_corr/rollout_is_mean:0.9999414682388306 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.225766017247224e-06 - rollout_corr/rollout_is_max:1.9946351051330566 - rollout_corr/rollout_is_min:0.23694215714931488 - rollout_corr/rollout_is_std:0.026999348774552345 - rollout_corr/rollout_is_eff_sample_size:0.9992714471613361 - rollout_corr/rollout_is_seq_mean:0.9999212026596069 - rollout_corr/rollout_is_seq_std:0.0008411617600359023 - rollout_corr/rollout_is_seq_max:1.0026910305023193 - rollout_corr/rollout_is_seq_min:0.9967179298400879 - rollout_corr/rollout_is_seq_max_deviation:0.0032820701599121094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.386727336794138) - rollout_corr/training_log_ppl:np.float64(0.31231660785852) - rollout_corr/kl:np.float64(0.00041095774418131725) - rollout_corr/k3_kl:np.float64(0.0003522394192145839) - rollout_corr/rollout_ppl:np.float64(1.3861299636773765) - rollout_corr/rollout_log_ppl:np.float64(0.3119056501018349) - rollout_corr/log_ppl_diff:np.float64(0.00041095775668509305) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007225902227219194) - rollout_corr/log_ppl_diff_max:np.float64(0.0031125247478485107) - rollout_corr/log_ppl_diff_min:np.float64(-0.00223328173160553) - rollout_corr/ppl_ratio:np.float64(1.00041141663678) - rollout_corr/chi2_token:np.float64(0.0005854771006852388) - rollout_corr/chi2_seq:np.float64(3.2485340386629105) - actor/pg_loss:np.float64(1.788930967450142e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.609930801459996e-05) - actor/ppo_kl:np.float64(-7.207803651265721e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07063721679151058) - perf/mfu/actor:np.float64(0.1745573387991699) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.36584854125977) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016829580068588257 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016829580068588257 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1494.99609375 - response_length/max:7029.0 - response_length/min:275.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1494.99609375 - response_length_non_aborted/max:7029.0 - response_length_non_aborted/min:275.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001059994101524353 - timing_s/agent_loop/generate_sequences/min:np.float64(3.654697211459279) - timing_s/agent_loop/generate_sequences/max:np.float64(50.904972886666656) - timing_s/agent_loop/generate_sequences/mean:np.float64(15.098586545231228) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.904972886666656) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:7029 - timing_s/gen:58.266811951994896 - timing_s/reward:0.106016606092453 - timing_s/old_log_prob:4.172686280682683 - timing_s/adv:0.13374079205095768 - timing_s/update_actor:17.26791531406343 - timing_s/step:80.03169925138354 - timing_s/stop_profile:3.802776336669922e-05 - timing_per_token_ms/gen:0.1522443671518657 - timing_per_token_ms/update_actor:0.038410787714323213 - timing_per_token_ms/adv:0.000297493303550719 - perf/total_num_tokens:449559 - perf/time_per_step:80.03169925138354 - perf/throughput:702.1577140763825 (TaskRunner pid=2281424) Training Progress: 72%|███████▏ | 72/100 [1:32:11<50:21, 107.92s/it] (TaskRunner pid=2281424) step:73 - global_seqlen/min:50710 - global_seqlen/max:100431 - global_seqlen/minmax_diff:49721 - global_seqlen/balanced_min:71047 - global_seqlen/balanced_max:71387 - global_seqlen/mean:71271.125 - actor/entropy:0.4723150432109833 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5466041564941406 - training/rollout_probs_diff_mean:0.0032528568990528584 - training/rollout_probs_diff_std:0.007730447221547365 - training/rollout_actor_probs_pearson_corr:0.9995112419128418 - rollout_corr/rollout_is_mean:0.9999789595603943 - rollout_corr/rollout_is_ratio_fraction_high:4.0056320358417e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.0084480537625495e-06 - rollout_corr/rollout_is_max:2.142465353012085 - rollout_corr/rollout_is_min:0.40851542353630066 - rollout_corr/rollout_is_std:0.028115419670939445 - rollout_corr/rollout_is_eff_sample_size:0.9992101475610164 - rollout_corr/rollout_is_seq_mean:0.9999721646308899 - rollout_corr/rollout_is_seq_std:0.0010153069160878658 - rollout_corr/rollout_is_seq_max:1.0036121606826782 - rollout_corr/rollout_is_seq_min:0.9954786896705627 - rollout_corr/rollout_is_seq_max_deviation:0.004521310329437256 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5221755541861057) - rollout_corr/training_log_ppl:np.float64(0.39488993302802555) - rollout_corr/kl:np.float64(0.0004714057424735074) - rollout_corr/k3_kl:np.float64(0.0004070275241190302) - rollout_corr/rollout_ppl:np.float64(1.5213952930644155) - rollout_corr/rollout_log_ppl:np.float64(0.39441852559684776) - rollout_corr/log_ppl_diff:np.float64(0.00047140743117779493) - rollout_corr/log_ppl_abs_diff:np.float64(0.000822892296127975) - rollout_corr/log_ppl_diff_max:np.float64(0.0034640729427337646) - rollout_corr/log_ppl_diff_min:np.float64(-0.0030289292335510254) - rollout_corr/ppl_ratio:np.float64(1.0004719870630652) - rollout_corr/chi2_token:np.float64(0.0006970379035919905) - rollout_corr/chi2_seq:np.float64(1.1741749560460448) - actor/pg_loss:np.float64(8.69284849613905e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010152803969276647) - actor/ppo_kl:np.float64(4.153459515521263e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0953609012067318) - perf/mfu/actor:np.float64(0.19884977160610307) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.27853107452393) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005314472131431103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005314472131431103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1950.37890625 - response_length/max:7405.0 - response_length/min:180.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1950.37890625 - response_length_non_aborted/max:7405.0 - response_length_non_aborted/min:180.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.626864731311798e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4811857379972935) - timing_s/agent_loop/generate_sequences/max:np.float64(56.79083485901356) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.615477024730353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(56.79083485901356) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:7405 - timing_s/gen:64.19048333540559 - timing_s/reward:0.1299944929778576 - timing_s/old_log_prob:4.7613958436995745 - timing_s/adv:0.13127423450350761 - timing_s/update_actor:19.186476746574044 - timing_s/step:88.48862772993743 - timing_s/stop_profile:0.00012197904288768768 - timing_per_token_ms/gen:0.12856172445539546 - timing_per_token_ms/update_actor:0.03365050843973286 - timing_per_token_ms/adv:0.00023023741119476438 - perf/total_num_tokens:570169 - perf/time_per_step:88.48862772993743 - perf/throughput:805.426943872558 (TaskRunner pid=2281424) Training Progress: 73%|███████▎ | 73/100 [1:33:39<45:56, 102.11s/it] (TaskRunner pid=2281424) step:74 - global_seqlen/min:37867 - global_seqlen/max:74272 - global_seqlen/minmax_diff:36405 - global_seqlen/balanced_min:53139 - global_seqlen/balanced_max:53734 - global_seqlen/mean:53577.125 - actor/entropy:0.44166502356529236 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30947351455688477 - training/rollout_probs_diff_mean:0.0033270702697336674 - training/rollout_probs_diff_std:0.007980842143297195 - training/rollout_actor_probs_pearson_corr:0.9996140599250793 - rollout_corr/rollout_is_mean:0.9999847412109375 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7457967917143833e-06 - rollout_corr/rollout_is_max:1.913217306137085 - rollout_corr/rollout_is_min:0.3732314705848694 - rollout_corr/rollout_is_std:0.02833504043519497 - rollout_corr/rollout_is_eff_sample_size:0.9991978885434327 - rollout_corr/rollout_is_seq_mean:1.000065565109253 - rollout_corr/rollout_is_seq_std:0.0012911416124552488 - rollout_corr/rollout_is_seq_max:1.006355881690979 - rollout_corr/rollout_is_seq_min:0.9940794706344604 - rollout_corr/rollout_is_seq_max_deviation:0.006355881690979004 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.46239750739187) - rollout_corr/training_log_ppl:np.float64(0.35442679983680137) - rollout_corr/kl:np.float64(0.0003367374671343626) - rollout_corr/k3_kl:np.float64(0.00044517193822457557) - rollout_corr/rollout_ppl:np.float64(1.4618395678699017) - rollout_corr/rollout_log_ppl:np.float64(0.3540900625521317) - rollout_corr/log_ppl_diff:np.float64(0.0003367372846696526) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008827941201161593) - rollout_corr/log_ppl_diff_max:np.float64(0.006804674863815308) - rollout_corr/log_ppl_diff_min:np.float64(-0.005605608224868774) - rollout_corr/ppl_ratio:np.float64(1.0003375951200724) - rollout_corr/chi2_token:np.float64(0.0011235442943871021) - rollout_corr/chi2_seq:np.float64(0.879344102460891) - actor/pg_loss:np.float64(2.2439053282141685e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.060110855012681e-05) - actor/ppo_kl:np.float64(-3.356134554266532e-05) - actor/pg_clipfrac_lower:np.float64(1.6035509133871528e-06) - actor/grad_norm:np.float64(0.09511143900454044) - perf/mfu/actor:np.float64(0.16871793536919505) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.22051620483398) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0102184833958745 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0102184833958745 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1422.62890625 - response_length/max:7451.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1422.62890625 - response_length_non_aborted/max:7451.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001026596873998642 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8556665536016226) - timing_s/agent_loop/generate_sequences/max:np.float64(52.814375065267086) - timing_s/agent_loop/generate_sequences/mean:np.float64(13.79034505609161) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(52.814375065267086) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:7451 - timing_s/gen:60.13706563971937 - timing_s/reward:0.1039069052785635 - timing_s/old_log_prob:4.208096336573362 - timing_s/adv:0.12475542351603508 - timing_s/update_actor:17.123554781079292 - timing_s/step:81.78595715388656 - timing_s/stop_profile:0.00013553164899349213 - timing_per_token_ms/gen:0.16512416669106592 - timing_per_token_ms/update_actor:0.03995071306336261 - timing_per_token_ms/adv:0.00029106503828834387 - perf/total_num_tokens:428617 - perf/time_per_step:81.78595715388656 - perf/throughput:655.0895393837665 (TaskRunner pid=2281424) Training Progress: 74%|███████▍ | 74/100 [1:35:01<41:36, 96.02s/it] (TaskRunner pid=2281424) step:75 - global_seqlen/min:29725 - global_seqlen/max:63241 - global_seqlen/minmax_diff:33516 - global_seqlen/balanced_min:43431 - global_seqlen/balanced_max:43829 - global_seqlen/mean:43649.625 - actor/entropy:0.40172678232192993 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4140506982803345 - training/rollout_probs_diff_mean:0.0032765574287623167 - training/rollout_probs_diff_std:0.008161391131579876 - training/rollout_actor_probs_pearson_corr:0.9995269775390625 - rollout_corr/rollout_is_mean:1.0000414848327637 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0609216587909032e-05 - rollout_corr/rollout_is_max:1.553276777267456 - rollout_corr/rollout_is_min:0.2633557915687561 - rollout_corr/rollout_is_std:0.027534905821084976 - rollout_corr/rollout_is_eff_sample_size:0.9992425223351087 - rollout_corr/rollout_is_seq_mean:1.0000547170639038 - rollout_corr/rollout_is_seq_std:0.0010406997753307223 - rollout_corr/rollout_is_seq_max:1.0065134763717651 - rollout_corr/rollout_is_seq_min:0.9971379041671753 - rollout_corr/rollout_is_seq_max_deviation:0.006513476371765137 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.368973680306226) - rollout_corr/training_log_ppl:np.float64(0.29217190519557334) - rollout_corr/kl:np.float64(0.0003027983755394814) - rollout_corr/k3_kl:np.float64(0.0003719866110429848) - rollout_corr/rollout_ppl:np.float64(1.3685665079392493) - rollout_corr/rollout_log_ppl:np.float64(0.29186910606222227) - rollout_corr/log_ppl_diff:np.float64(0.00030279913335107267) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008351847936864942) - rollout_corr/log_ppl_diff_max:np.float64(0.00448000431060791) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031926631927490234) - rollout_corr/ppl_ratio:np.float64(1.000303411623463) - rollout_corr/chi2_token:np.float64(0.0008776339236646891) - rollout_corr/chi2_seq:np.float64(2.008884164271876) - actor/pg_loss:np.float64(2.7343048714101315e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.3585604796826374e-05) - actor/ppo_kl:np.float64(-8.158876836361628e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.034894744865596294) - perf/mfu/actor:np.float64(0.14880178274967848) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.15599822998047) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.85546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.85546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015577865997329354 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015577865997329354 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1104.58203125 - response_length/max:7265.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1104.58203125 - response_length_non_aborted/max:7265.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001430269330739975 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6985986921936274) - timing_s/agent_loop/generate_sequences/max:np.float64(49.74266072921455) - timing_s/agent_loop/generate_sequences/mean:np.float64(11.12252202147647) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.74266072921455) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:7265 - timing_s/gen:56.838472582399845 - timing_s/reward:0.08878145180642605 - timing_s/old_log_prob:3.778964426368475 - timing_s/adv:0.15880484133958817 - timing_s/update_actor:15.498764231801033 - timing_s/step:76.44660468026996 - timing_s/stop_profile:0.00011175312101840973 - timing_per_token_ms/gen:0.20100388856927587 - timing_per_token_ms/update_actor:0.04438401312669076 - timing_per_token_ms/adv:0.0004547714938547243 - perf/total_num_tokens:349197 - perf/time_per_step:76.44660468026996 - perf/throughput:570.9818661346708 (TaskRunner pid=2281424) Training Progress: 75%|███████▌ | 75/100 [1:36:18<37:34, 90.17s/it] (TaskRunner pid=2281424) step:76 - global_seqlen/min:42462 - global_seqlen/max:96092 - global_seqlen/minmax_diff:53630 - global_seqlen/balanced_min:72056 - global_seqlen/balanced_max:72271 - global_seqlen/mean:72168.875 - actor/entropy:0.4893510341644287 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2547096014022827 - training/rollout_probs_diff_mean:0.003322131233289838 - training/rollout_probs_diff_std:0.0077032484114170074 - training/rollout_actor_probs_pearson_corr:0.9996566772460938 - rollout_corr/rollout_is_mean:1.0001403093338013 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9754140794248087e-06 - rollout_corr/rollout_is_max:1.7543652057647705 - rollout_corr/rollout_is_min:0.48901525139808655 - rollout_corr/rollout_is_std:0.028295045718550682 - rollout_corr/rollout_is_eff_sample_size:0.9992002689116629 - rollout_corr/rollout_is_seq_mean:1.0001996755599976 - rollout_corr/rollout_is_seq_std:0.001209214678965509 - rollout_corr/rollout_is_seq_max:1.009247064590454 - rollout_corr/rollout_is_seq_min:0.9936361908912659 - rollout_corr/rollout_is_seq_max_deviation:0.009247064590454102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5176880699582398) - rollout_corr/training_log_ppl:np.float64(0.3912102092872374) - rollout_corr/kl:np.float64(0.0003245452827291828) - rollout_corr/k3_kl:np.float64(0.0004195083137688016) - rollout_corr/rollout_ppl:np.float64(1.5171602945774794) - rollout_corr/rollout_log_ppl:np.float64(0.39088566205464303) - rollout_corr/log_ppl_diff:np.float64(0.00032454723259434104) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007810101378709078) - rollout_corr/log_ppl_diff_max:np.float64(0.007137387990951538) - rollout_corr/log_ppl_diff_min:np.float64(-0.008129239082336426) - rollout_corr/ppl_ratio:np.float64(1.000325285596773) - rollout_corr/chi2_token:np.float64(0.0010407264344394207) - rollout_corr/chi2_seq:np.float64(1.3129587173461914) - actor/pg_loss:np.float64(1.1760275810956955e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.053675117025705e-05) - actor/ppo_kl:np.float64(0.00012599972318838581) - actor/pg_clipfrac_lower:np.float64(2.287031520609162e-06) - actor/grad_norm:np.float64(0.09297847654670477) - perf/mfu/actor:np.float64(0.20044092158481716) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.21371841430664) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.027344420552253723 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.027344420552253723 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1977.43359375 - response_length/max:6976.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1977.43359375 - response_length_non_aborted/max:6976.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.447336196899414e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7599249705672264) - timing_s/agent_loop/generate_sequences/max:np.float64(53.188171891495585) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.02186474698101) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(53.188171891495585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:6976 - timing_s/gen:60.21184345707297 - timing_s/reward:0.11902121268212795 - timing_s/old_log_prob:4.799457537010312 - timing_s/adv:0.14908457174897194 - timing_s/update_actor:19.46632076613605 - timing_s/step:84.86317491158843 - timing_s/stop_profile:0.00012650899589061737 - timing_per_token_ms/gen:0.11894331837366728 - timing_per_token_ms/update_actor:0.03371661392486728 - timing_per_token_ms/adv:0.0002582217260366258 - perf/total_num_tokens:577351 - perf/time_per_step:84.86317491158843 - perf/throughput:850.414506353156 (TaskRunner pid=2281424) Training Progress: 76%|███████▌ | 76/100 [1:37:43<35:26, 88.59s/it] (TaskRunner pid=2281424) step:77 - global_seqlen/min:27511 - global_seqlen/max:128154 - global_seqlen/minmax_diff:100643 - global_seqlen/balanced_min:64748 - global_seqlen/balanced_max:65174 - global_seqlen/mean:65054.25 - actor/entropy:0.462461918592453 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6478930115699768 - training/rollout_probs_diff_mean:0.0033272244036197662 - training/rollout_probs_diff_std:0.007839487865567207 - training/rollout_actor_probs_pearson_corr:0.9997078776359558 - rollout_corr/rollout_is_mean:1.000058650970459 - rollout_corr/rollout_is_ratio_fraction_high:2.2016636194166495e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.604990630876273e-06 - rollout_corr/rollout_is_max:4.972353458404541 - rollout_corr/rollout_is_min:0.14135953783988953 - rollout_corr/rollout_is_std:0.02817683294415474 - rollout_corr/rollout_is_eff_sample_size:0.9992068149827754 - rollout_corr/rollout_is_seq_mean:1.0000437498092651 - rollout_corr/rollout_is_seq_std:0.0011115683009847999 - rollout_corr/rollout_is_seq_max:1.0068318843841553 - rollout_corr/rollout_is_seq_min:0.9928021430969238 - rollout_corr/rollout_is_seq_max_deviation:0.007197856903076172 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4656865671277046) - rollout_corr/training_log_ppl:np.float64(0.36309026755043305) - rollout_corr/kl:np.float64(0.00044725021728453385) - rollout_corr/k3_kl:np.float64(0.0003817609845384595) - rollout_corr/rollout_ppl:np.float64(1.4650072930380702) - rollout_corr/rollout_log_ppl:np.float64(0.3626430144358892) - rollout_corr/log_ppl_diff:np.float64(0.0004472531145438552) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008061645785346627) - rollout_corr/log_ppl_diff_max:np.float64(0.008196413516998291) - rollout_corr/log_ppl_diff_min:np.float64(-0.0022748708724975586) - rollout_corr/ppl_ratio:np.float64(1.0004479088820517) - rollout_corr/chi2_token:np.float64(0.0006361156702041626) - rollout_corr/chi2_seq:np.float64(3.7252589049749076) - actor/pg_loss:np.float64(4.542234819382429e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.581747235008152e-05) - actor/ppo_kl:np.float64(0.00011224961554034962) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09250594861805439) - perf/mfu/actor:np.float64(0.1876084141306772) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.58203125) - perf/cpu_memory_used_gb:np.float64(107.25350570678711) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009082688018679619 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009082688018679619 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1774.2265625 - response_length/max:8192.0 - response_length/min:214.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1774.2265625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:214.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010951049625873566 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6410596389323473) - timing_s/agent_loop/generate_sequences/max:np.float64(66.44215983897448) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.0781091665267) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(66.44215983897448) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:73.64733845368028 - timing_s/reward:0.11625036410987377 - timing_s/old_log_prob:4.594569396227598 - timing_s/adv:0.1430046670138836 - timing_s/update_actor:18.86780060455203 - timing_s/step:97.45536208339036 - timing_s/stop_profile:0.0001203492283821106 - timing_per_token_ms/gen:0.16214666261636954 - timing_per_token_ms/update_actor:0.036253973807537615 - timing_per_token_ms/adv:0.0002747796397120165 - perf/total_num_tokens:520434 - perf/time_per_step:97.45536208339036 - perf/throughput:667.5286881016823 (TaskRunner pid=2281424) Training Progress: 77%|███████▋ | 77/100 [1:39:20<34:59, 91.27s/it] (TaskRunner pid=2281424) step:78 - global_seqlen/min:25325 - global_seqlen/max:84216 - global_seqlen/minmax_diff:58891 - global_seqlen/balanced_min:56720 - global_seqlen/balanced_max:56942 - global_seqlen/mean:56839.875 - actor/entropy:0.44374775886535645 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491605162620544 - training/rollout_probs_diff_mean:0.003467120695859194 - training/rollout_probs_diff_std:0.008032586425542831 - training/rollout_actor_probs_pearson_corr:0.9996151328086853 - rollout_corr/rollout_is_mean:0.9999576210975647 - rollout_corr/rollout_is_ratio_fraction_high:7.75131684349617e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.75131684349617e-06 - rollout_corr/rollout_is_max:4.747284889221191 - rollout_corr/rollout_is_min:0.10899676382541656 - rollout_corr/rollout_is_std:0.02860928140580654 - rollout_corr/rollout_is_eff_sample_size:0.9991820593831796 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.001185454661026597 - rollout_corr/rollout_is_seq_max:1.005151629447937 - rollout_corr/rollout_is_seq_min:0.9958641529083252 - rollout_corr/rollout_is_seq_max_deviation:0.005151629447937012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4343888186849654) - rollout_corr/training_log_ppl:np.float64(0.3459277451329399) - rollout_corr/kl:np.float64(0.0004222604492056803) - rollout_corr/k3_kl:np.float64(0.0003943129925687572) - rollout_corr/rollout_ppl:np.float64(1.4337338921613991) - rollout_corr/rollout_log_ppl:np.float64(0.34550548443803564) - rollout_corr/log_ppl_diff:np.float64(0.0004222606949042529) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008823849784675986) - rollout_corr/log_ppl_diff_max:np.float64(0.004985526204109192) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038795918226242065) - rollout_corr/ppl_ratio:np.float64(1.0004229948390275) - rollout_corr/chi2_token:np.float64(0.0007329215295612812) - rollout_corr/chi2_seq:np.float64(0.7212347930762917) - actor/pg_loss:np.float64(6.53313472867012e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.447403512765959e-05) - actor/ppo_kl:np.float64(2.688655172633503e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08514224551618099) - perf/mfu/actor:np.float64(0.17315237393578076) - perf/max_memory_allocated_gb:np.float64(132.44677352905273) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.3226547241211) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06087884306907654 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06087884306907654 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1511.83984375 - response_length/max:8192.0 - response_length/min:229.0 - response_length/clip_ratio:0.01953125 - response_length_non_aborted/mean:1511.83984375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:229.0 - response_length_non_aborted/clip_ratio:0.01953125 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010377727448940277 - timing_s/agent_loop/generate_sequences/min:np.float64(3.003926744684577) - timing_s/agent_loop/generate_sequences/max:np.float64(69.25940157100558) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.432622163862106) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(69.25940157100558) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.2985665909946 - timing_s/reward:0.10811152122914791 - timing_s/old_log_prob:4.204658040776849 - timing_s/adv:0.17462857253849506 - timing_s/update_actor:17.65695309266448 - timing_s/step:98.5296196397394 - timing_s/stop_profile:0.00012289360165596008 - timing_per_token_ms/gen:0.19713812741355238 - timing_per_token_ms/update_actor:0.0388304713299081 - timing_per_token_ms/adv:0.00038403623455033785 - perf/total_num_tokens:454719 - perf/time_per_step:98.5296196397394 - perf/throughput:576.8810963426788 (TaskRunner pid=2281424) Training Progress: 78%|███████▊ | 78/100 [1:40:59<34:16, 93.46s/it] (TaskRunner pid=2281424) step:79 - global_seqlen/min:37448 - global_seqlen/max:108817 - global_seqlen/minmax_diff:71369 - global_seqlen/balanced_min:70970 - global_seqlen/balanced_max:71124 - global_seqlen/mean:71023.25 - actor/entropy:0.4368029832839966 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45647865533828735 - training/rollout_probs_diff_mean:0.0031526105012744665 - training/rollout_probs_diff_std:0.007653329521417618 - training/rollout_actor_probs_pearson_corr:0.9996246695518494 - rollout_corr/rollout_is_mean:1.0000195503234863 - rollout_corr/rollout_is_ratio_fraction_high:6.0216539168322925e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.014435944554862e-06 - rollout_corr/rollout_is_max:3.049748420715332 - rollout_corr/rollout_is_min:1.032992599903082e-06 - rollout_corr/rollout_is_std:0.02727172151207924 - rollout_corr/rollout_is_eff_sample_size:0.9992569250228056 - rollout_corr/rollout_is_seq_mean:1.0000171661376953 - rollout_corr/rollout_is_seq_std:0.001091465470381081 - rollout_corr/rollout_is_seq_max:1.005435824394226 - rollout_corr/rollout_is_seq_min:0.9938186407089233 - rollout_corr/rollout_is_seq_max_deviation:0.00618135929107666 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3994069588370621) - rollout_corr/training_log_ppl:np.float64(0.3162277852825355) - rollout_corr/kl:np.float64(0.00035690159927703746) - rollout_corr/k3_kl:np.float64(0.0004051223161241069) - rollout_corr/rollout_ppl:np.float64(1.3988550025969744) - rollout_corr/rollout_log_ppl:np.float64(0.31587088617379777) - rollout_corr/log_ppl_diff:np.float64(0.00035689910873770714) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008177596027962863) - rollout_corr/log_ppl_diff_max:np.float64(0.008748382329940796) - rollout_corr/log_ppl_diff_min:np.float64(-0.00391659140586853) - rollout_corr/ppl_ratio:np.float64(1.0003577324096113) - rollout_corr/chi2_token:np.float64(0.0008362571243196726) - rollout_corr/chi2_seq:np.float64(119.7799857633654) - actor/pg_loss:np.float64(3.9536505937576294e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.417913894234516e-05) - actor/ppo_kl:np.float64(-1.3414459802518763e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.056897282134741545) - perf/mfu/actor:np.float64(0.19757145862440487) - perf/max_memory_allocated_gb:np.float64(133.4042763710022) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.21912050247192) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017608821392059326 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017608821392059326 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1946.1015625 - response_length/max:8192.0 - response_length/min:209.0 - response_length/clip_ratio:0.015625 - response_length_non_aborted/mean:1946.1015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:209.0 - response_length_non_aborted/clip_ratio:0.015625 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014724023640155792 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6493716575205326) - timing_s/agent_loop/generate_sequences/max:np.float64(73.67174332775176) - timing_s/agent_loop/generate_sequences/mean:np.float64(18.573477208301483) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(73.67174332775176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:80.96080801077187 - timing_s/reward:0.12271188385784626 - timing_s/old_log_prob:4.715622756630182 - timing_s/adv:0.1407144833356142 - timing_s/update_actor:19.42637729458511 - timing_s/step:105.45251621864736 - timing_s/stop_profile:0.00012031197547912598 - timing_per_token_ms/gen:0.16250598755278356 - timing_per_token_ms/update_actor:0.034190172398800935 - timing_per_token_ms/adv:0.00024765566792496506 - perf/total_num_tokens:568186 - perf/time_per_step:105.45251621864736 - perf/throughput:673.5092963807423 (TaskRunner pid=2281424) Training Progress: 79%|███████▉ | 79/100 [1:42:44<33:58, 97.08s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] To find the magnitude of the charge, we use the formula for the electric field due to a point charge: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 1: Plug in the values (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 2: Compute $ (0.50)^2 $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (0.50)^2 = 0.25 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 3: Multiply by 2.0 (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 2.0 \cdot 0.25 = 0.50 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 4: Divide by $ 8.99 \times 10^9 $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 5: Convert to picocoulombs (pC) (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 1 \, \mathrm{pC} = 1 \times 10^{-12} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \times 10^{-12} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 6: Choose the closest answer (TaskRunner pid=2281424) (TaskRunner pid=2281424) The options are: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - A: 50 pC (TaskRunner pid=2281424) - B: 56 pC (TaskRunner pid=2281424) - C: 60 pC (TaskRunner pid=2281424) - D: 64 pC (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest value to 55.6 pC is **56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) --- (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We use the formula $ E = k \frac{q}{r^2} $ to find the charge $ q $. Given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $, we rearrange the formula to solve for $ q $. Calculating the result gives $ q \approx 5.56 \times 10^{-11} \, \mathrm{C} $, which is approximately 55.6 pC. The closest option is B: 56 pC. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2281424) step:80 - global_seqlen/min:29962 - global_seqlen/max:105447 - global_seqlen/minmax_diff:75485 - global_seqlen/balanced_min:60292 - global_seqlen/balanced_max:60692 - global_seqlen/mean:60551.375 - actor/entropy:0.37467148900032043 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3974228799343109 - training/rollout_probs_diff_mean:0.0030430646147578955 - training/rollout_probs_diff_std:0.007756151724606752 - training/rollout_actor_probs_pearson_corr:0.9996011257171631 - rollout_corr/rollout_is_mean:0.9999578595161438 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.408228510830668e-06 - rollout_corr/rollout_is_max:1.7704508304595947 - rollout_corr/rollout_is_min:0.40431222319602966 - rollout_corr/rollout_is_std:0.02627207152545452 - rollout_corr/rollout_is_eff_sample_size:0.9993102542910165 - rollout_corr/rollout_is_seq_mean:0.9999442100524902 - rollout_corr/rollout_is_seq_std:0.0010222762357443571 - rollout_corr/rollout_is_seq_max:1.0053536891937256 - rollout_corr/rollout_is_seq_min:0.9961909055709839 - rollout_corr/rollout_is_seq_max_deviation:0.005353689193725586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3535777013748884) - rollout_corr/training_log_ppl:np.float64(0.2919448420871049) - rollout_corr/kl:np.float64(0.0002914607577544359) - rollout_corr/k3_kl:np.float64(0.00035389692166631903) - rollout_corr/rollout_ppl:np.float64(1.3531782091595232) - rollout_corr/rollout_log_ppl:np.float64(0.2916533798270393) - rollout_corr/log_ppl_diff:np.float64(0.0002914622600656003) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007385123462881893) - rollout_corr/log_ppl_diff_max:np.float64(0.004037708044052124) - rollout_corr/log_ppl_diff_min:np.float64(-0.004292726516723633) - rollout_corr/ppl_ratio:np.float64(1.0002919645048678) - rollout_corr/chi2_token:np.float64(0.0008356906473636627) - rollout_corr/chi2_seq:np.float64(10.826386786764488) - actor/pg_loss:np.float64(2.2610998712480068e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.29967929121267e-05) - actor/ppo_kl:np.float64(-0.00010968260576760258) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08275915030390024) - perf/mfu/actor:np.float64(0.17912262687715078) - perf/max_memory_allocated_gb:np.float64(133.4042763710022) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.1332893371582) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7125) - val-aux/sciknoweval/reward/std@16:np.float64(0.21208410796593385) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.8003625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1838824791651191) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6199874999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1767685182330923) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7112875) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21264807771425337) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8766124999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13289130750751482) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.551875) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12414401859208643) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7451) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18095144568917293) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.9285499999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08117133451240169) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5055999999999999) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0797599664446933) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7729874999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14284952333254633) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9576999999999998) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03905052666534377) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.47392500000000004) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0470622776131008) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7932874999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10751795426918813) - val-aux/sciknoweval/score/mean@16:np.float64(0.7125) - val-aux/sciknoweval/score/std@16:np.float64(0.21208410796593385) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.8003625) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1838824791651191) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6199874999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1767685182330923) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7112875) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21264807771425337) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8766124999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13289130750751482) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.551875) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12414401859208643) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7451) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18095144568917293) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.9285499999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08117133451240169) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5055999999999999) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0797599664446933) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7729874999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14284952333254633) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9576999999999998) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03905052666534377) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.47392500000000004) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0470622776131008) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7932874999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10751795426918813) - val-core/sciknoweval/acc/mean@16:np.float64(0.7125) - val-aux/sciknoweval/acc/std@16:np.float64(0.21208410796593385) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.8003625) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1838824791651191) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6199874999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1767685182330923) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7112875) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21264807771425337) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8766124999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13289130750751482) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.551875) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12414401859208643) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7451) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18095144568917293) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.9285499999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08117133451240169) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5055999999999999) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0797599664446933) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7729874999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14284952333254633) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9576999999999998) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03905052666534377) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.47392500000000004) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0470622776131008) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7932874999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10751795426918813) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.92890625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.11586940894478473) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.972475) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.06345428095007741) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.88225) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.13239906930031728) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.92675) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.11753222858603023) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.990875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.02434920966995723) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8277999999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.13386162236149496) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.952825) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.09039223284283485) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9973500000000002) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.008420884420166629) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7695875) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.11560936592748024) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9721375) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.06158716693942362) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9996875) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0019515618744994994) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.720375) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08171985491238151) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9849) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.03299085003096019) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013781989924609661 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013781989924609661 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1622.04296875 - response_length/max:8192.0 - response_length/min:248.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1622.04296875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:248.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010387226939201355 - timing_s/agent_loop/generate_sequences/min:np.float64(2.926741849631071) - timing_s/agent_loop/generate_sequences/max:np.float64(69.41407606191933) - timing_s/agent_loop/generate_sequences/mean:np.float64(15.920171771431342) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(69.41407606191933) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.53145934268832 - timing_s/reward:0.1108447965234518 - timing_s/old_log_prob:4.27453831396997 - timing_s/adv:0.18335014954209328 - timing_s/update_actor:18.237299781292677 - timing_s/step:99.4225641246885 - timing_s/testing:192.12527464702725 - timing_s/save_checkpoint:12.946154896169901 - timing_s/stop_profile:0.00015969760715961456 - timing_per_token_ms/gen:0.18430523655471212 - timing_per_token_ms/update_actor:0.03764840142212435 - timing_per_token_ms/adv:0.00037850120980343813 - perf/total_num_tokens:484411 - perf/time_per_step:99.4225641246885 - perf/throughput:609.0305106601445 (TaskRunner pid=2281424) Training Progress: 80%|████████ | 80/100 [1:47:49<53:06, 159.33s/it] (TaskRunner pid=2281424) step:81 - global_seqlen/min:47325 - global_seqlen/max:105881 - global_seqlen/minmax_diff:58556 - global_seqlen/balanced_min:76357 - global_seqlen/balanced_max:76567 - global_seqlen/mean:76471.5 - actor/entropy:0.46122434735298157 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2159644365310669 - training/rollout_probs_diff_mean:0.0033320188522338867 - training/rollout_probs_diff_std:0.00773474620655179 - training/rollout_actor_probs_pearson_corr:0.9996447563171387 - rollout_corr/rollout_is_mean:1.0000454187393188 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8291126489639282 - rollout_corr/rollout_is_min:0.5626341700553894 - rollout_corr/rollout_is_std:0.02797088772058487 - rollout_corr/rollout_is_eff_sample_size:0.9992181220349029 - rollout_corr/rollout_is_seq_mean:1.0000308752059937 - rollout_corr/rollout_is_seq_std:0.0010952734155580401 - rollout_corr/rollout_is_seq_max:1.0038789510726929 - rollout_corr/rollout_is_seq_min:0.9951615929603577 - rollout_corr/rollout_is_seq_max_deviation:0.004838407039642334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4837682861834764) - rollout_corr/training_log_ppl:np.float64(0.3760610534227453) - rollout_corr/kl:np.float64(0.0004281692851382246) - rollout_corr/k3_kl:np.float64(0.0004241228809291897) - rollout_corr/rollout_ppl:np.float64(1.483103048056364) - rollout_corr/rollout_log_ppl:np.float64(0.37563288104138337) - rollout_corr/log_ppl_diff:np.float64(0.00042817238136194646) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008447545405942947) - rollout_corr/log_ppl_diff_max:np.float64(0.007132202386856079) - rollout_corr/log_ppl_diff_min:np.float64(-0.004386395215988159) - rollout_corr/ppl_ratio:np.float64(1.0004289988428354) - rollout_corr/chi2_token:np.float64(0.0008449205197393894) - rollout_corr/chi2_seq:np.float64(8.882342195371166) - actor/pg_loss:np.float64(8.517643436789513e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.26423552603228e-05) - actor/ppo_kl:np.float64(4.755724401039885e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.05668603768572211) - perf/mfu/actor:np.float64(0.20224053663035063) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.5755615234375) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.021595492959022522 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.021595492959022522 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2130.390625 - response_length/max:8192.0 - response_length/min:178.0 - response_length/clip_ratio:0.01953125 - response_length_non_aborted/mean:2130.390625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.01953125 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.419792771339417e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.191812541335821) - timing_s/agent_loop/generate_sequences/max:np.float64(73.02005716599524) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.716089782734343) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(73.02005716599524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:79.11253042705357 - timing_s/reward:0.1305206846445799 - timing_s/old_log_prob:5.078640753403306 - timing_s/adv:0.14080399833619595 - timing_s/update_actor:20.568581249564886 - timing_s/step:105.1306124124676 - timing_s/stop_profile:0.0001254640519618988 - timing_per_token_ms/gen:0.14505946390966587 - timing_per_token_ms/update_actor:0.033621318480683796 - timing_per_token_ms/adv:0.00023015763770848607 - perf/total_num_tokens:611772 - perf/time_per_step:105.1306124124676 - perf/throughput:727.395172968013 (TaskRunner pid=2281424) Training Progress: 81%|████████ | 81/100 [1:49:34<45:18, 143.08s/it] (TaskRunner pid=2281424) step:82 - global_seqlen/min:38375 - global_seqlen/max:108226 - global_seqlen/minmax_diff:69851 - global_seqlen/balanced_min:78960 - global_seqlen/balanced_max:79182 - global_seqlen/mean:79033.125 - actor/entropy:0.4809735119342804 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40625 - training/rollout_probs_diff_mean:0.0033739369828253984 - training/rollout_probs_diff_std:0.0077420733869075775 - training/rollout_actor_probs_pearson_corr:0.99964839220047 - rollout_corr/rollout_is_mean:1.0000008344650269 - rollout_corr/rollout_is_ratio_fraction_high:1.7803699847718235e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.901850378606468e-06 - rollout_corr/rollout_is_max:3.1266798973083496 - rollout_corr/rollout_is_min:0.04171716049313545 - rollout_corr/rollout_is_std:0.028216995298862457 - rollout_corr/rollout_is_eff_sample_size:0.9992043155636828 - rollout_corr/rollout_is_seq_mean:1.0000370740890503 - rollout_corr/rollout_is_seq_std:0.0007998693035915494 - rollout_corr/rollout_is_seq_max:1.0032259225845337 - rollout_corr/rollout_is_seq_min:0.9969495534896851 - rollout_corr/rollout_is_seq_max_deviation:0.0032259225845336914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.447751184925437) - rollout_corr/training_log_ppl:np.float64(0.3510043573041912) - rollout_corr/kl:np.float64(0.0003127274083301046) - rollout_corr/k3_kl:np.float64(0.0003627495793239177) - rollout_corr/rollout_ppl:np.float64(1.4472675155848265) - rollout_corr/rollout_log_ppl:np.float64(0.3506916323385667) - rollout_corr/log_ppl_diff:np.float64(0.00031272496562451124) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007065368699841201) - rollout_corr/log_ppl_diff_max:np.float64(0.0035865604877471924) - rollout_corr/log_ppl_diff_min:np.float64(-0.0028453469276428223) - rollout_corr/ppl_ratio:np.float64(1.0003131539560854) - rollout_corr/chi2_token:np.float64(0.000826970674097538) - rollout_corr/chi2_seq:np.float64(2.990220111096278) - actor/pg_loss:np.float64(-1.262675505131483e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.107881173671558e-05) - actor/ppo_kl:np.float64(-1.3940705438253431e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0625862805172801) - perf/mfu/actor:np.float64(0.20966418106155954) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.6513671875) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017938785254955292 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017938785254955292 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2194.06640625 - response_length/max:8192.0 - response_length/min:305.0 - response_length/clip_ratio:0.01171875 - response_length_non_aborted/mean:2194.06640625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:305.0 - response_length_non_aborted/clip_ratio:0.01171875 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.206309914588928e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.8525668401271105) - timing_s/agent_loop/generate_sequences/max:np.float64(73.49001097865403) - timing_s/agent_loop/generate_sequences/mean:np.float64(21.367409962149395) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(73.49001097865403) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:81.01238861493766 - timing_s/reward:0.1340006049722433 - timing_s/old_log_prob:5.036404974758625 - timing_s/adv:0.17358910478651524 - timing_s/update_actor:20.482164518907666 - timing_s/step:106.89824829623103 - timing_s/stop_profile:0.00012538954615592957 - timing_per_token_ms/gen:0.14423202603423949 - timing_per_token_ms/update_actor:0.032394904856203756 - timing_per_token_ms/adv:0.0002745511846876155 - perf/total_num_tokens:632265 - perf/time_per_step:106.89824829623103 - perf/throughput:739.3304030669183 (TaskRunner pid=2281424) Training Progress: 82%|████████▏ | 82/100 [1:51:21<39:40, 132.24s/it] (TaskRunner pid=2281424) step:83 - global_seqlen/min:46708 - global_seqlen/max:91538 - global_seqlen/minmax_diff:44830 - global_seqlen/balanced_min:69220 - global_seqlen/balanced_max:69538 - global_seqlen/mean:69476.25 - actor/entropy:0.5335648059844971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21911153197288513 - training/rollout_probs_diff_mean:0.0035760728642344475 - training/rollout_probs_diff_std:0.007884988561272621 - training/rollout_actor_probs_pearson_corr:0.9996548891067505 - rollout_corr/rollout_is_mean:1.0000134706497192 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9868483543395996 - rollout_corr/rollout_is_min:0.5721929669380188 - rollout_corr/rollout_is_std:0.029507704079151154 - rollout_corr/rollout_is_eff_sample_size:0.9991300528161801 - rollout_corr/rollout_is_seq_mean:0.9999951124191284 - rollout_corr/rollout_is_seq_std:0.0008592059020884335 - rollout_corr/rollout_is_seq_max:1.0029661655426025 - rollout_corr/rollout_is_seq_min:0.9966848492622375 - rollout_corr/rollout_is_seq_max_deviation:0.003315150737762451 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5849693166092038) - rollout_corr/training_log_ppl:np.float64(0.43137292834580876) - rollout_corr/kl:np.float64(0.0004295566333016865) - rollout_corr/k3_kl:np.float64(0.000427562826786243) - rollout_corr/rollout_ppl:np.float64(1.5842498331330717) - rollout_corr/rollout_log_ppl:np.float64(0.43094336806098) - rollout_corr/log_ppl_diff:np.float64(0.0004295602848287672) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007582434045616537) - rollout_corr/log_ppl_diff_max:np.float64(0.0037308931350708008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0021752119064331055) - rollout_corr/ppl_ratio:np.float64(1.000430047744885) - rollout_corr/chi2_token:np.float64(0.0008571695070713758) - rollout_corr/chi2_seq:np.float64(14.75534682860598) - actor/pg_loss:np.float64(5.458039231598377e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001093973785941671) - actor/ppo_kl:np.float64(3.278788307525815e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11635340005159378) - perf/mfu/actor:np.float64(0.19497208017666126) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.30697250366211) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005556248594075441 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005556248594075441 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1894.0703125 - response_length/max:7501.0 - response_length/min:209.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1894.0703125 - response_length_non_aborted/max:7501.0 - response_length_non_aborted/min:209.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014759600162506104 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8615881502628326) - timing_s/agent_loop/generate_sequences/max:np.float64(58.530355628579855) - timing_s/agent_loop/generate_sequences/mean:np.float64(18.284697317627433) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.530355628579855) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:7501 - timing_s/gen:66.29873481579125 - timing_s/reward:0.12525632046163082 - timing_s/old_log_prob:4.649687595665455 - timing_s/adv:0.13868866860866547 - timing_s/update_actor:19.150086453184485 - timing_s/step:90.4499344099313 - timing_s/stop_profile:0.0001360122114419937 - timing_per_token_ms/gen:0.13673168897956875 - timing_per_token_ms/update_actor:0.034454375511747695 - timing_per_token_ms/adv:0.00024952532089862626 - perf/total_num_tokens:555810 - perf/time_per_step:90.4499344099313 - perf/throughput:768.1183016134015 (TaskRunner pid=2281424) Training Progress: 83%|████████▎ | 83/100 [1:52:51<33:55, 119.72s/it] (TaskRunner pid=2281424) step:84 - global_seqlen/min:46333 - global_seqlen/max:120390 - global_seqlen/minmax_diff:74057 - global_seqlen/balanced_min:81591 - global_seqlen/balanced_max:81854 - global_seqlen/mean:81763.5 - actor/entropy:0.46362319588661194 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21383929252624512 - training/rollout_probs_diff_mean:0.003217406803742051 - training/rollout_probs_diff_std:0.007578108459711075 - training/rollout_actor_probs_pearson_corr:0.9996676445007324 - rollout_corr/rollout_is_mean:1.0000669956207275 - rollout_corr/rollout_is_ratio_fraction_high:3.410757472011028e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.705378736005514e-06 - rollout_corr/rollout_is_max:2.3830087184906006 - rollout_corr/rollout_is_min:0.15366080403327942 - rollout_corr/rollout_is_std:0.027716144919395447 - rollout_corr/rollout_is_eff_sample_size:0.9992325240192108 - rollout_corr/rollout_is_seq_mean:1.0001479387283325 - rollout_corr/rollout_is_seq_std:0.0008669063681736588 - rollout_corr/rollout_is_seq_max:1.0025583505630493 - rollout_corr/rollout_is_seq_min:0.9951901435852051 - rollout_corr/rollout_is_seq_max_deviation:0.004809856414794922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5030197105370462) - rollout_corr/training_log_ppl:np.float64(0.3850575773685705) - rollout_corr/kl:np.float64(0.00027459628167836314) - rollout_corr/k3_kl:np.float64(0.00038053442332852683) - rollout_corr/rollout_ppl:np.float64(1.5025763548910618) - rollout_corr/rollout_log_ppl:np.float64(0.38478297731489874) - rollout_corr/log_ppl_diff:np.float64(0.00027460005367174745) - rollout_corr/log_ppl_abs_diff:np.float64(0.0006311151082627475) - rollout_corr/log_ppl_diff_max:np.float64(0.005537629127502441) - rollout_corr/log_ppl_diff_min:np.float64(-0.002191781997680664) - rollout_corr/ppl_ratio:np.float64(1.0002749729901552) - rollout_corr/chi2_token:np.float64(0.0009795920923352242) - rollout_corr/chi2_seq:np.float64(5.221234666882083) - actor/pg_loss:np.float64(-1.2737466022372246e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.926910533091359e-05) - actor/ppo_kl:np.float64(4.087090245974423e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08847346901893616) - perf/mfu/actor:np.float64(0.21121760504386758) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.6875) - perf/cpu_memory_used_gb:np.float64(107.31388092041016) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.028147703036665916 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.028147703036665916 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:2290.546875 - response_length/max:8192.0 - response_length/min:261.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:2290.546875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:261.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.132176637649536e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.4179056379944086) - timing_s/agent_loop/generate_sequences/max:np.float64(73.46726548857987) - timing_s/agent_loop/generate_sequences/mean:np.float64(22.820737626556365) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(73.46726548857987) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:80.69648661091924 - timing_s/reward:0.13805697858333588 - timing_s/old_log_prob:5.1129378993064165 - timing_s/adv:0.1517045572400093 - timing_s/update_actor:20.95853779092431 - timing_s/step:107.14519646950066 - timing_s/stop_profile:0.0001168474555015564 - timing_per_token_ms/gen:0.13761807464599618 - timing_per_token_ms/update_actor:0.03204140262911371 - timing_per_token_ms/adv:0.0002319258551187408 - perf/total_num_tokens:654108 - perf/time_per_step:107.14519646950066 - perf/throughput:763.1093384879306 (TaskRunner pid=2281424) Training Progress: 84%|████████▍ | 84/100 [1:54:39<30:55, 115.97s/it] (TaskRunner pid=2281424) step:85 - global_seqlen/min:32778 - global_seqlen/max:118304 - global_seqlen/minmax_diff:85526 - global_seqlen/balanced_min:71971 - global_seqlen/balanced_max:72255 - global_seqlen/mean:72087.625 - actor/entropy:0.43251606822013855 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4627995491027832 - training/rollout_probs_diff_mean:0.0032046264968812466 - training/rollout_probs_diff_std:0.0077904630452394485 - training/rollout_actor_probs_pearson_corr:0.9996163845062256 - rollout_corr/rollout_is_mean:1.0000158548355103 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.962150236067828e-06 - rollout_corr/rollout_is_max:1.9618604183197021 - rollout_corr/rollout_is_min:0.2549246847629547 - rollout_corr/rollout_is_std:0.027508918195962906 - rollout_corr/rollout_is_eff_sample_size:0.9992438316531975 - rollout_corr/rollout_is_seq_mean:1.0000420808792114 - rollout_corr/rollout_is_seq_std:0.0008891079341992736 - rollout_corr/rollout_is_seq_max:1.0028163194656372 - rollout_corr/rollout_is_seq_min:0.9960147738456726 - rollout_corr/rollout_is_seq_max_deviation:0.003985226154327393 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.42215026775375) - rollout_corr/training_log_ppl:np.float64(0.33671512818546034) - rollout_corr/kl:np.float64(0.0003456190911617085) - rollout_corr/k3_kl:np.float64(0.00036502632008250657) - rollout_corr/rollout_ppl:np.float64(1.4216335988603532) - rollout_corr/rollout_log_ppl:np.float64(0.33636950654909015) - rollout_corr/log_ppl_diff:np.float64(0.00034562163637019694) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007374820706900209) - rollout_corr/log_ppl_diff_max:np.float64(0.004794985055923462) - rollout_corr/log_ppl_diff_min:np.float64(-0.00254783034324646) - rollout_corr/ppl_ratio:np.float64(1.000346165150404) - rollout_corr/chi2_token:np.float64(0.0007752671372145414) - rollout_corr/chi2_seq:np.float64(2.035842107143253) - actor/pg_loss:np.float64(-3.1908624805510044e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.119389224248152e-05) - actor/ppo_kl:np.float64(2.0877532892260575e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0993273975327611) - perf/mfu/actor:np.float64(0.19706691211589414) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.26990127563477) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01401048805564642 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01401048805564642 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1990.80078125 - response_length/max:8192.0 - response_length/min:199.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1990.80078125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:199.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.484799087047577e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4878016971051693) - timing_s/agent_loop/generate_sequences/max:np.float64(64.35746593214571) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.05917956525809) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(64.35746593214571) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:71.24950071051717 - timing_s/reward:0.11637670919299126 - timing_s/old_log_prob:4.78818198479712 - timing_s/adv:0.16108788177371025 - timing_s/update_actor:19.909558206796646 - timing_s/step:96.25776395574212 - timing_s/stop_profile:2.9178336262702942e-05 - timing_per_token_ms/gen:0.13980221666163148 - timing_per_token_ms/update_actor:0.034523190018391936 - timing_per_token_ms/adv:0.0002793265171617706 - perf/total_num_tokens:576701 - perf/time_per_step:96.25776395574212 - perf/throughput:748.9019278813167 (TaskRunner pid=2281424) Training Progress: 85%|████████▌ | 85/100 [1:56:15<27:30, 110.06s/it] (TaskRunner pid=2281424) step:86 - global_seqlen/min:29204 - global_seqlen/max:122423 - global_seqlen/minmax_diff:93219 - global_seqlen/balanced_min:71868 - global_seqlen/balanced_max:72149 - global_seqlen/mean:72006.125 - actor/entropy:0.447162926197052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48606076836586 - training/rollout_probs_diff_mean:0.003287011291831732 - training/rollout_probs_diff_std:0.007797359488904476 - training/rollout_actor_probs_pearson_corr:0.9996358752250671 - rollout_corr/rollout_is_mean:1.0000190734863281 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.949065103370231e-06 - rollout_corr/rollout_is_max:1.695367455482483 - rollout_corr/rollout_is_min:0.47366517782211304 - rollout_corr/rollout_is_std:0.02779559977352619 - rollout_corr/rollout_is_eff_sample_size:0.9992278820119855 - rollout_corr/rollout_is_seq_mean:1.0000596046447754 - rollout_corr/rollout_is_seq_std:0.0009104225318878889 - rollout_corr/rollout_is_seq_max:1.0035185813903809 - rollout_corr/rollout_is_seq_min:0.9944294691085815 - rollout_corr/rollout_is_seq_max_deviation:0.005570530891418457 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4478634255938232) - rollout_corr/training_log_ppl:np.float64(0.3556695947772823) - rollout_corr/kl:np.float64(0.00028983059334386496) - rollout_corr/k3_kl:np.float64(0.0003808262275128982) - rollout_corr/rollout_ppl:np.float64(1.4474190385080874) - rollout_corr/rollout_log_ppl:np.float64(0.35537976160412654) - rollout_corr/log_ppl_diff:np.float64(0.0002898331731557846) - rollout_corr/log_ppl_abs_diff:np.float64(0.0006754108471795917) - rollout_corr/log_ppl_diff_max:np.float64(0.006349921226501465) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031312406063079834) - rollout_corr/ppl_ratio:np.float64(1.000290306750685) - rollout_corr/chi2_token:np.float64(0.0009537497535347939) - rollout_corr/chi2_seq:np.float64(1.8716313727200031) - actor/pg_loss:np.float64(-4.055839963257313e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.612786794832573e-05) - actor/ppo_kl:np.float64(-2.927031754040854e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08380211982876062) - perf/mfu/actor:np.float64(0.19894113002162722) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.34819412231445) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012212483212351799 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012212483212351799 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1978.31640625 - response_length/max:7625.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1978.31640625 - response_length_non_aborted/max:7625.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6423484086990356e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2320269010961056) - timing_s/agent_loop/generate_sequences/max:np.float64(58.958134239539504) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.39341227179102) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.958134239539504) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:7625 - timing_s/gen:66.23730275034904 - timing_s/reward:0.1252636183053255 - timing_s/old_log_prob:4.83253344334662 - timing_s/adv:0.144756231456995 - timing_s/update_actor:19.455502839758992 - timing_s/step:90.88217270746827 - timing_s/stop_profile:0.00012291967868804932 - timing_per_token_ms/gen:0.13078770567292866 - timing_per_token_ms/update_actor:0.03377404151341117 - timing_per_token_ms/adv:0.00025129152460466906 - perf/total_num_tokens:576049 - perf/time_per_step:90.88217270746827 - perf/throughput:792.301975787633 (TaskRunner pid=2281424) Training Progress: 86%|████████▌ | 86/100 [1:57:46<24:20, 104.32s/it] (TaskRunner pid=2281424) step:87 - global_seqlen/min:43406 - global_seqlen/max:108646 - global_seqlen/minmax_diff:65240 - global_seqlen/balanced_min:77124 - global_seqlen/balanced_max:77249 - global_seqlen/mean:77219.0 - actor/entropy:0.47046366333961487 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28602373600006104 - training/rollout_probs_diff_mean:0.003333775093778968 - training/rollout_probs_diff_std:0.007726138457655907 - training/rollout_actor_probs_pearson_corr:0.9997732043266296 - rollout_corr/rollout_is_mean:1.0000189542770386 - rollout_corr/rollout_is_ratio_fraction_high:1.8291968899575295e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0640363693237305 - rollout_corr/rollout_is_min:0.6168027520179749 - rollout_corr/rollout_is_std:0.02792823687195778 - rollout_corr/rollout_is_eff_sample_size:0.9992206215230673 - rollout_corr/rollout_is_seq_mean:0.9999931454658508 - rollout_corr/rollout_is_seq_std:0.0008395964978262782 - rollout_corr/rollout_is_seq_max:1.0021971464157104 - rollout_corr/rollout_is_seq_min:0.9952844977378845 - rollout_corr/rollout_is_seq_max_deviation:0.0047155022621154785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.507551294285804) - rollout_corr/training_log_ppl:np.float64(0.3906772773480043) - rollout_corr/kl:np.float64(0.0004402913548275933) - rollout_corr/k3_kl:np.float64(0.0004143168899304328) - rollout_corr/rollout_ppl:np.float64(1.5068849357776344) - rollout_corr/rollout_log_ppl:np.float64(0.39023698499659076) - rollout_corr/log_ppl_diff:np.float64(0.0004402923514135182) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007724418537691236) - rollout_corr/log_ppl_diff_max:np.float64(0.00550532341003418) - rollout_corr/log_ppl_diff_min:np.float64(-0.0020297616720199585) - rollout_corr/ppl_ratio:np.float64(1.0004408617969602) - rollout_corr/chi2_token:np.float64(0.0008024710696190596) - rollout_corr/chi2_seq:np.float64(1.297804655507207) - actor/pg_loss:np.float64(-6.791017949581146e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000107869764519819) - actor/ppo_kl:np.float64(5.49935483284969e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10479225032031536) - perf/mfu/actor:np.float64(0.20496773282402933) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.29073524475098) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003920197486877441 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003920197486877441 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2135.5 - response_length/max:7668.0 - response_length/min:246.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:2135.5 - response_length_non_aborted/max:7668.0 - response_length_non_aborted/min:246.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001415777951478958 - timing_s/agent_loop/generate_sequences/min:np.float64(3.136248679831624) - timing_s/agent_loop/generate_sequences/max:np.float64(58.29445764608681) - timing_s/agent_loop/generate_sequences/mean:np.float64(20.58206392160355) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.29445764608681) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:7668 - timing_s/gen:65.97938266210258 - timing_s/reward:0.13106519170105457 - timing_s/old_log_prob:4.976280787959695 - timing_s/adv:0.12150675244629383 - timing_s/update_actor:20.37396141141653 - timing_s/step:91.67027896083891 - timing_s/stop_profile:0.00010777637362480164 - timing_per_token_ms/gen:0.12068928284890573 - timing_per_token_ms/update_actor:0.03298081011703164 - timing_per_token_ms/adv:0.0001966917993730394 - perf/total_num_tokens:617752 - perf/time_per_step:91.67027896083891 - perf/throughput:842.3558963204156 (TaskRunner pid=2281424) Training Progress: 87%|████████▋ | 87/100 [1:59:18<21:47, 100.54s/it] (TaskRunner pid=2281424) step:88 - global_seqlen/min:36604 - global_seqlen/max:113322 - global_seqlen/minmax_diff:76718 - global_seqlen/balanced_min:70235 - global_seqlen/balanced_max:70460 - global_seqlen/mean:70355.125 - actor/entropy:0.45366615056991577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292335391044617 - training/rollout_probs_diff_mean:0.0033171819522976875 - training/rollout_probs_diff_std:0.007824771106243134 - training/rollout_actor_probs_pearson_corr:0.9996778964996338 - rollout_corr/rollout_is_mean:1.0 - rollout_corr/rollout_is_ratio_fraction_high:2.0178824797767447e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.0178824797767447e-06 - rollout_corr/rollout_is_max:4.592259407043457 - rollout_corr/rollout_is_min:0.19030924141407013 - rollout_corr/rollout_is_std:0.02784273773431778 - rollout_corr/rollout_is_eff_sample_size:0.9992253824874978 - rollout_corr/rollout_is_seq_mean:1.0000131130218506 - rollout_corr/rollout_is_seq_std:0.0010293829254806042 - rollout_corr/rollout_is_seq_max:1.0040936470031738 - rollout_corr/rollout_is_seq_min:0.9965201020240784 - rollout_corr/rollout_is_seq_max_deviation:0.004093647003173828 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4986823541112244) - rollout_corr/training_log_ppl:np.float64(0.38453248073346913) - rollout_corr/kl:np.float64(0.00033108328769060336) - rollout_corr/k3_kl:np.float64(0.00040329458698806775) - rollout_corr/rollout_ppl:np.float64(1.4981776834465563) - rollout_corr/rollout_log_ppl:np.float64(0.3842013974208385) - rollout_corr/log_ppl_diff:np.float64(0.0003310833126306534) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007660561241209507) - rollout_corr/log_ppl_diff_max:np.float64(0.004498302936553955) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034332573413848877) - rollout_corr/ppl_ratio:np.float64(1.0003316532820463) - rollout_corr/chi2_token:np.float64(0.0009584731888025999) - rollout_corr/chi2_seq:np.float64(4.4919839494396) - actor/pg_loss:np.float64(3.3170217648148537e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010095978973367892) - actor/ppo_kl:np.float64(-5.047958963944055e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0923737701959908) - perf/mfu/actor:np.float64(0.19545780170266114) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.3228530883789) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.021646076813340187 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.021646076813340187 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1935.81640625 - response_length/max:8192.0 - response_length/min:217.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1935.81640625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:217.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.361468255519867e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7692256066948175) - timing_s/agent_loop/generate_sequences/max:np.float64(67.97121464461088) - timing_s/agent_loop/generate_sequences/mean:np.float64(18.578832377446815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(67.97121464461088) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:212 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:75.0351722035557 - timing_s/reward:0.1233233455568552 - timing_s/old_log_prob:4.722648134455085 - timing_s/adv:0.11689798347651958 - timing_s/update_actor:19.538657259196043 - timing_s/step:99.62376413680613 - timing_s/stop_profile:0.00011381879448890686 - timing_per_token_ms/gen:0.15141215895981328 - timing_per_token_ms/update_actor:0.034714346074994615 - timing_per_token_ms/adv:0.00020769272934366827 - perf/total_num_tokens:562841 - perf/time_per_step:99.62376413680613 - perf/throughput:706.2082587382101 (TaskRunner pid=2281424) Training Progress: 88%|████████▊ | 88/100 [2:00:57<20:03, 100.28s/it] (TaskRunner pid=2281424) step:89 - global_seqlen/min:53363 - global_seqlen/max:96134 - global_seqlen/minmax_diff:42771 - global_seqlen/balanced_min:73474 - global_seqlen/balanced_max:73759 - global_seqlen/mean:73580.375 - actor/entropy:0.5425375699996948 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3450472354888916 - training/rollout_probs_diff_mean:0.0035728770308196545 - training/rollout_probs_diff_std:0.007812545634806156 - training/rollout_actor_probs_pearson_corr:0.999666154384613 - rollout_corr/rollout_is_mean:1.0000165700912476 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.851049314107513e-06 - rollout_corr/rollout_is_max:1.8271527290344238 - rollout_corr/rollout_is_min:0.16427424550056458 - rollout_corr/rollout_is_std:0.029384229332208633 - rollout_corr/rollout_is_eff_sample_size:0.9991374309926928 - rollout_corr/rollout_is_seq_mean:0.9999986290931702 - rollout_corr/rollout_is_seq_std:0.0008472701301798224 - rollout_corr/rollout_is_seq_max:1.0029807090759277 - rollout_corr/rollout_is_seq_min:0.9957017302513123 - rollout_corr/rollout_is_seq_max_deviation:0.004298269748687744 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5710625080391765) - rollout_corr/training_log_ppl:np.float64(0.4208992576750461) - rollout_corr/kl:np.float64(0.0004893235657874584) - rollout_corr/k3_kl:np.float64(0.0004040306061483534) - rollout_corr/rollout_ppl:np.float64(1.5702631180174649) - rollout_corr/rollout_log_ppl:np.float64(0.42040993153932504) - rollout_corr/log_ppl_diff:np.float64(0.0004893261357210577) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007742471643723547) - rollout_corr/log_ppl_diff_max:np.float64(0.0040569007396698) - rollout_corr/log_ppl_diff_min:np.float64(-0.0024844110012054443) - rollout_corr/ppl_ratio:np.float64(1.0004898344632238) - rollout_corr/chi2_token:np.float64(0.0006377112586051226) - rollout_corr/chi2_seq:np.float64(1.1348994506988674) - actor/pg_loss:np.float64(-6.997038144618273e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.692707725936998e-05) - actor/ppo_kl:np.float64(9.186447801567965e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0774675034917891) - perf/mfu/actor:np.float64(0.20087032175496733) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.02946090698242) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.029441511258482933 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.029441511258482933 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2028.66796875 - response_length/max:8192.0 - response_length/min:153.0 - response_length/clip_ratio:0.015625 - response_length_non_aborted/mean:2028.66796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.015625 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00038411468267440796 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9795979037880898) - timing_s/agent_loop/generate_sequences/max:np.float64(68.82650169543922) - timing_s/agent_loop/generate_sequences/mean:np.float64(19.44817877533933) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(68.82650169543922) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:330 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.69867212511599 - timing_s/reward:0.12369080632925034 - timing_s/old_log_prob:4.9405558202415705 - timing_s/adv:0.16437353193759918 - timing_s/update_actor:19.898798257112503 - timing_s/step:101.944374775514 - timing_s/stop_profile:0.00012306682765483856 - timing_per_token_ms/gen:0.14768517697518574 - timing_per_token_ms/update_actor:0.03380452711934484 - timing_per_token_ms/adv:0.00027924146203658104 - perf/total_num_tokens:588643 - perf/time_per_step:101.944374775514 - perf/throughput:721.769839307242 (TaskRunner pid=2281424) Training Progress: 89%|████████▉ | 89/100 [2:02:40<18:30, 100.93s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] We are given the electric field $ E $ at a distance $ r $ from a point charge $ q $, and we are to find the magnitude of the charge. The formula for the electric field due to a point charge is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given: (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are to solve for $ q $. Rearranging the formula: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 1: Compute $ r^2 $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) r^2 = (0.50 \, \mathrm{m})^2 = 0.25 \, \mathrm{m^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 2: Plug into the formula (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0 \, \mathrm{N/C}) \cdot (0.25 \, \mathrm{m^2})}{8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2}} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 3: Convert to picocoulombs (pC) (TaskRunner pid=2281424) (TaskRunner pid=2281424) Since $ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \times 10^{-12} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Now, we check the options: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - A: 50 pC (TaskRunner pid=2281424) - B: 56 pC (TaskRunner pid=2281424) - C: 60 pC (TaskRunner pid=2281424) - D: 64 pC (TaskRunner pid=2281424) (TaskRunner pid=2281424) Our calculated value is approximately **55.6 pC**, which is **closest to option B: 56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) --- (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We used the formula for the electric field due to a point charge, $ E = k \frac{q}{r^2} $, and rearranged it to solve for $ q $. We converted the given distance into meters, calculated $ r^2 $, and then used the values of $ E $ and $ k $ to compute the charge. We then converted the result into picocoulombs and matched it with the given options. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2281424) step:90 - global_seqlen/min:31582 - global_seqlen/max:87120 - global_seqlen/minmax_diff:55538 - global_seqlen/balanced_min:57199 - global_seqlen/balanced_max:57669 - global_seqlen/mean:57438.75 - actor/entropy:0.45833009481430054 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28802943229675293 - training/rollout_probs_diff_mean:0.0033582851756364107 - training/rollout_probs_diff_std:0.007876868359744549 - training/rollout_actor_probs_pearson_corr:0.9996347427368164 - rollout_corr/rollout_is_mean:1.0000334978103638 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.112814051244641e-06 - rollout_corr/rollout_is_max:1.816962718963623 - rollout_corr/rollout_is_min:0.38643011450767517 - rollout_corr/rollout_is_std:0.028212770819664 - rollout_corr/rollout_is_eff_sample_size:0.9992047916425459 - rollout_corr/rollout_is_seq_mean:1.0000895261764526 - rollout_corr/rollout_is_seq_std:0.0008514981018379331 - rollout_corr/rollout_is_seq_max:1.0030163526535034 - rollout_corr/rollout_is_seq_min:0.996926486492157 - rollout_corr/rollout_is_seq_max_deviation:0.0030735135078430176 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4863704326562583) - rollout_corr/training_log_ppl:np.float64(0.3669586373725906) - rollout_corr/kl:np.float64(0.0003789341422653081) - rollout_corr/k3_kl:np.float64(0.0003786877977631775) - rollout_corr/rollout_ppl:np.float64(1.4857277404516935) - rollout_corr/rollout_log_ppl:np.float64(0.3665797036665026) - rollout_corr/log_ppl_diff:np.float64(0.0003789337060879916) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007170522294472903) - rollout_corr/log_ppl_diff_max:np.float64(0.0034637898206710815) - rollout_corr/log_ppl_diff_min:np.float64(-0.0026997029781341553) - rollout_corr/ppl_ratio:np.float64(1.000379371456802) - rollout_corr/chi2_token:np.float64(0.0007569114677608013) - rollout_corr/chi2_seq:np.float64(19.86091840430163) - actor/pg_loss:np.float64(-9.84076177701354e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.184421224337711e-05) - actor/ppo_kl:np.float64(0.00010069883839824989) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08238446805626154) - perf/mfu/actor:np.float64(0.17647744927476752) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.05528259277344) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.72265625) - val-aux/sciknoweval/reward/std@16:np.float64(0.2057458394439798) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.8098875000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17178028077323276) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6335) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17894301231008136) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.720275) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.20532848626214437) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8814499999999998) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11395992699905108) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5634875) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13094232605191994) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7604375) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.17656682501213306) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.9239125000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06209606071752287) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5146) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08708453812159662) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7867500000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14109642832776273) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9435625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.024301931121292062) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.48135000000000006) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.054652451285185435) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.8034125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11089874407322804) - val-aux/sciknoweval/score/mean@16:np.float64(0.72265625) - val-aux/sciknoweval/score/std@16:np.float64(0.2057458394439798) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.8098875000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17178028077323276) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6335) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17894301231008136) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.720275) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.20532848626214437) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8814499999999998) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11395992699905108) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5634875) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13094232605191994) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7604375) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.17656682501213306) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.9239125000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06209606071752287) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5146) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08708453812159662) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7867500000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14109642832776273) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9435625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.024301931121292062) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.48135000000000006) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.054652451285185435) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.8034125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11089874407322804) - val-core/sciknoweval/acc/mean@16:np.float64(0.72265625) - val-aux/sciknoweval/acc/std@16:np.float64(0.2057458394439798) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.8098875000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17178028077323276) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6335) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17894301231008136) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.720275) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.20532848626214437) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8814499999999998) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11395992699905108) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5634875) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13094232605191994) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7604375) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.17656682501213306) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.9239125000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06209606071752287) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5146) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08708453812159662) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7867500000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14109642832776273) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9435625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.024301931121292062) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.48135000000000006) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.054652451285185435) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.8034125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11089874407322804) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9390625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.1062386511391431) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9786999999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.05641481524452378) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.8982875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.12304830102207487) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9378250000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.10790044372990266) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.994575) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.018566940076233975) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8458500000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12479054833478716) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9613875000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.08324671963700653) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9990874999999999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.004891743956974362) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.792075) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.10839186005437869) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9796250000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.05464609964655156) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.000683625445693766) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.745775) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0776266069855637) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.991225) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.02706783139141474) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017509790137410164 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017509790137410164 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1528.0234375 - response_length/max:8192.0 - response_length/min:305.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:1528.0234375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:305.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001458972692489624 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6096901912242174) - timing_s/agent_loop/generate_sequences/max:np.float64(68.86912310123444) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.475064634047158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(68.86912310123444) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:256 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.93200786039233 - timing_s/reward:0.10610510408878326 - timing_s/old_log_prob:4.28859911672771 - timing_s/adv:0.1231590025126934 - timing_s/update_actor:17.333123948425055 - timing_s/step:98.87560060434043 - timing_s/testing:198.1766846049577 - timing_s/save_checkpoint:12.776142889633775 - timing_s/stop_profile:0.00014507584273815155 - timing_per_token_ms/gen:0.19666953289429343 - timing_per_token_ms/update_actor:0.03772088517861429 - timing_per_token_ms/adv:0.0002680224641742147 - perf/total_num_tokens:459510 - perf/time_per_step:98.87560060434043 - perf/throughput:580.919353702298 (TaskRunner pid=2281424) Training Progress: 90%|█████████ | 90/100 [2:07:49<27:16, 163.62s/it] (TaskRunner pid=2281424) step:91 - global_seqlen/min:30432 - global_seqlen/max:80533 - global_seqlen/minmax_diff:50101 - global_seqlen/balanced_min:59087 - global_seqlen/balanced_max:59360 - global_seqlen/mean:59265.0 - actor/entropy:0.4014686048030853 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6135859489440918 - training/rollout_probs_diff_mean:0.0031206756830215454 - training/rollout_probs_diff_std:0.007770925760269165 - training/rollout_actor_probs_pearson_corr:0.9996281266212463 - rollout_corr/rollout_is_mean:1.000016450881958 - rollout_corr/rollout_is_ratio_fraction_high:2.4419784949714085e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.8256072998046875 - rollout_corr/rollout_is_min:0.535260796546936 - rollout_corr/rollout_is_std:0.026806596666574478 - rollout_corr/rollout_is_eff_sample_size:0.9992819224085475 - rollout_corr/rollout_is_seq_mean:1.0000858306884766 - rollout_corr/rollout_is_seq_std:0.0011405966943129897 - rollout_corr/rollout_is_seq_max:1.0064619779586792 - rollout_corr/rollout_is_seq_min:0.9956552982330322 - rollout_corr/rollout_is_seq_max_deviation:0.006461977958679199 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3793772477656603) - rollout_corr/training_log_ppl:np.float64(0.3077376027358696) - rollout_corr/kl:np.float64(0.0003082244062286854) - rollout_corr/k3_kl:np.float64(0.00036910291433400744) - rollout_corr/rollout_ppl:np.float64(1.3789437217637897) - rollout_corr/rollout_log_ppl:np.float64(0.3074293795507401) - rollout_corr/log_ppl_diff:np.float64(0.00030822318512946367) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008478707168251276) - rollout_corr/log_ppl_diff_max:np.float64(0.00515410304069519) - rollout_corr/log_ppl_diff_min:np.float64(-0.00571519136428833) - rollout_corr/ppl_ratio:np.float64(1.00030901376158) - rollout_corr/chi2_token:np.float64(0.000863607507199049) - rollout_corr/chi2_seq:np.float64(2.677014630055055) - actor/pg_loss:np.float64(8.078990504145622e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.9724320180976065e-05) - actor/ppo_kl:np.float64(3.992081970327632e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.056282463716343045) - perf/mfu/actor:np.float64(0.17517399995117178) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.51257610321045) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017548669129610062 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017548669129610062 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1599.625 - response_length/max:8192.0 - response_length/min:181.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1599.625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001323893666267395 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8850416410714388) - timing_s/agent_loop/generate_sequences/max:np.float64(68.87393903173506) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.232906831581204) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(68.87393903173506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:75.83647374995053 - timing_s/reward:0.10788371786475182 - timing_s/old_log_prob:4.422369368374348 - timing_s/adv:0.13758745603263378 - timing_s/update_actor:18.241979939863086 - timing_s/step:98.84038063883781 - timing_s/stop_profile:0.00011936575174331665 - timing_per_token_ms/gen:0.18519104514229537 - timing_per_token_ms/update_actor:0.038475449126514565 - timing_per_token_ms/adv:0.00029019542738680876 - perf/total_num_tokens:474120 - perf/time_per_step:98.84038063883781 - perf/throughput:599.6031137977299 (TaskRunner pid=2281424) Training Progress: 91%|█████████ | 91/100 [2:09:28<21:37, 144.20s/it] (TaskRunner pid=2281424) step:92 - global_seqlen/min:56361 - global_seqlen/max:133413 - global_seqlen/minmax_diff:77052 - global_seqlen/balanced_min:93215 - global_seqlen/balanced_max:93343 - global_seqlen/mean:93298.75 - actor/entropy:0.5294249653816223 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932349443435669 - training/rollout_probs_diff_mean:0.0034848994109779596 - training/rollout_probs_diff_std:0.0077215456403791904 - training/rollout_actor_probs_pearson_corr:0.9996566772460938 - rollout_corr/rollout_is_mean:1.0000396966934204 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.548328161239624 - rollout_corr/rollout_is_min:0.5352661609649658 - rollout_corr/rollout_is_std:0.028887107968330383 - rollout_corr/rollout_is_eff_sample_size:0.9991662307244454 - rollout_corr/rollout_is_seq_mean:0.999986469745636 - rollout_corr/rollout_is_seq_std:0.000814978324342519 - rollout_corr/rollout_is_seq_max:1.0046956539154053 - rollout_corr/rollout_is_seq_min:0.9965773224830627 - rollout_corr/rollout_is_seq_max_deviation:0.0046956539154052734 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.608520980924368) - rollout_corr/training_log_ppl:np.float64(0.45163694545044564) - rollout_corr/kl:np.float64(0.0004835717547688745) - rollout_corr/k3_kl:np.float64(0.00041193552203822037) - rollout_corr/rollout_ppl:np.float64(1.6077243499457836) - rollout_corr/rollout_log_ppl:np.float64(0.45115337034803815) - rollout_corr/log_ppl_diff:np.float64(0.00048357510240748525) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007026950479485095) - rollout_corr/log_ppl_diff_max:np.float64(0.003002941608428955) - rollout_corr/log_ppl_diff_min:np.float64(-0.0039307475090026855) - rollout_corr/ppl_ratio:np.float64(1.0004840346518904) - rollout_corr/chi2_token:np.float64(0.0006826643366366625) - rollout_corr/chi2_seq:np.float64(2.7582137731369585) - actor/pg_loss:np.float64(-2.270552795380354e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.691421706113033e-05) - actor/ppo_kl:np.float64(6.0937326300969374e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06705359183251858) - perf/mfu/actor:np.float64(0.22324610163002523) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.4158935546875) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004625823814421892 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004625823814421892 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:2637.3046875 - response_length/max:8192.0 - response_length/min:370.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:2637.3046875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:370.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.772313594818115e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(4.7102756798267365) - timing_s/agent_loop/generate_sequences/max:np.float64(70.24139930866659) - timing_s/agent_loop/generate_sequences/mean:np.float64(26.584116022888338) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(70.24139930866659) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:77.78476001881063 - timing_s/reward:0.14629439264535904 - timing_s/old_log_prob:5.78052232414484 - timing_s/adv:0.15388775803148746 - timing_s/update_actor:22.8142747040838 - timing_s/step:106.79661073349416 - timing_s/stop_profile:0.0001097414642572403 - timing_per_token_ms/gen:0.11521107904733856 - timing_per_token_ms/update_actor:0.030566158046173984 - timing_per_token_ms/adv:0.00020617607153296195 - perf/total_num_tokens:746390 - perf/time_per_step:106.79661073349416 - perf/throughput:873.6115253022642 (TaskRunner pid=2281424) Training Progress: 92%|█████████▏| 92/100 [2:11:15<17:43, 132.99s/it] (TaskRunner pid=2281424) step:93 - global_seqlen/min:32833 - global_seqlen/max:94956 - global_seqlen/minmax_diff:62123 - global_seqlen/balanced_min:68640 - global_seqlen/balanced_max:68729 - global_seqlen/mean:68692.75 - actor/entropy:0.5789089202880859 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18479880690574646 - training/rollout_probs_diff_mean:0.003710611490532756 - training/rollout_probs_diff_std:0.00791421253234148 - training/rollout_actor_probs_pearson_corr:0.9996806979179382 - rollout_corr/rollout_is_mean:0.9999197721481323 - rollout_corr/rollout_is_ratio_fraction_high:2.0674228835559916e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.97680926322937 - rollout_corr/rollout_is_min:0.6035233736038208 - rollout_corr/rollout_is_std:0.029875071719288826 - rollout_corr/rollout_is_eff_sample_size:0.9991082759306559 - rollout_corr/rollout_is_seq_mean:0.9998905658721924 - rollout_corr/rollout_is_seq_std:0.0011505308793857694 - rollout_corr/rollout_is_seq_max:1.0048060417175293 - rollout_corr/rollout_is_seq_min:0.994641125202179 - rollout_corr/rollout_is_seq_max_deviation:0.005358874797821045 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.601950139272958) - rollout_corr/training_log_ppl:np.float64(0.43084850793820806) - rollout_corr/kl:np.float64(0.0005708589492365856) - rollout_corr/k3_kl:np.float64(0.0004376700993589111) - rollout_corr/rollout_ppl:np.float64(1.600947569590062) - rollout_corr/rollout_log_ppl:np.float64(0.4302776500408072) - rollout_corr/log_ppl_diff:np.float64(0.000570857897400856) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009169939439743757) - rollout_corr/log_ppl_diff_max:np.float64(0.005955815315246582) - rollout_corr/log_ppl_diff_min:np.float64(-0.002580583095550537) - rollout_corr/ppl_ratio:np.float64(1.0005716276355088) - rollout_corr/chi2_token:np.float64(0.0006089142989367247) - rollout_corr/chi2_seq:np.float64(0.175237572286278) - actor/pg_loss:np.float64(-2.8330832719802856e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.197003153829428e-05) - actor/ppo_kl:np.float64(2.893346624155413e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.057069569593295455) - perf/mfu/actor:np.float64(0.19020234349041532) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.21023941040039) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008691961877048016 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008691961877048016 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1889.4296875 - response_length/max:8192.0 - response_length/min:174.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1889.4296875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002087317407131195 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1920957434922457) - timing_s/agent_loop/generate_sequences/max:np.float64(69.09191367775202) - timing_s/agent_loop/generate_sequences/mean:np.float64(17.2113787043927) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(69.09191367775202) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.51329581439495 - timing_s/reward:0.12370793335139751 - timing_s/old_log_prob:4.726794863119721 - timing_s/adv:0.12588824145495892 - timing_s/update_actor:19.58357641287148 - timing_s/step:101.16370816528797 - timing_s/stop_profile:8.470378816127777e-05 - timing_per_token_ms/gen:0.15818533166505053 - timing_per_token_ms/update_actor:0.03563617778599539 - timing_per_token_ms/adv:0.0002290784716272076 - perf/total_num_tokens:549542 - perf/time_per_step:101.16370816528797 - perf/throughput:679.0256233763716 (TaskRunner pid=2281424) Training Progress: 93%|█████████▎| 93/100 [2:12:56<14:24, 123.46s/it] (TaskRunner pid=2281424) step:94 - global_seqlen/min:33314 - global_seqlen/max:97152 - global_seqlen/minmax_diff:63838 - global_seqlen/balanced_min:55347 - global_seqlen/balanced_max:55703 - global_seqlen/mean:55618.375 - actor/entropy:0.44463133811950684 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5926223993301392 - training/rollout_probs_diff_mean:0.00327357673086226 - training/rollout_probs_diff_std:0.007880967110395432 - training/rollout_actor_probs_pearson_corr:0.9996113777160645 - rollout_corr/rollout_is_mean:1.0000139474868774 - rollout_corr/rollout_is_ratio_fraction_high:2.65053017756145e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.3252651115180925e-05 - rollout_corr/rollout_is_max:2.4800922870635986 - rollout_corr/rollout_is_min:1.9368031644262373e-05 - rollout_corr/rollout_is_std:0.027902614325284958 - rollout_corr/rollout_is_eff_sample_size:0.9992220498076327 - rollout_corr/rollout_is_seq_mean:1.0000684261322021 - rollout_corr/rollout_is_seq_std:0.000999571057036519 - rollout_corr/rollout_is_seq_max:1.0037225484848022 - rollout_corr/rollout_is_seq_min:0.9954401254653931 - rollout_corr/rollout_is_seq_max_deviation:0.004559874534606934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4411272373981774) - rollout_corr/training_log_ppl:np.float64(0.34707161964615807) - rollout_corr/kl:np.float64(0.0003712050064246242) - rollout_corr/k3_kl:np.float64(0.0004094531111888955) - rollout_corr/rollout_ppl:np.float64(1.4405805715359747) - rollout_corr/rollout_log_ppl:np.float64(0.3467004152480513) - rollout_corr/log_ppl_diff:np.float64(0.00037120439810678363) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007890164270065725) - rollout_corr/log_ppl_diff_max:np.float64(0.0038557052612304688) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032331347465515137) - rollout_corr/ppl_ratio:np.float64(1.000371773261577) - rollout_corr/chi2_token:np.float64(0.0008935551159083843) - rollout_corr/chi2_seq:np.float64(0.36217507324181497) - actor/pg_loss:np.float64(6.495625711977482e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.518509802484914e-05) - actor/ppo_kl:np.float64(3.6960312083778035e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.057536555686965585) - perf/mfu/actor:np.float64(0.17357501625996624) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.26631164550781) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010005089454352856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010005089454352856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1473.76171875 - response_length/max:7020.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1473.76171875 - response_length_non_aborted/max:7020.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.296353578567505e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.030196225270629) - timing_s/agent_loop/generate_sequences/max:np.float64(52.45747355185449) - timing_s/agent_loop/generate_sequences/mean:np.float64(14.197725277575955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(52.45747355185449) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:7020 - timing_s/gen:59.61554795503616 - timing_s/reward:0.10644834861159325 - timing_s/old_log_prob:4.186762785539031 - timing_s/adv:0.14022722467780113 - timing_s/update_actor:17.202740924432874 - timing_s/step:81.33951114118099 - timing_s/stop_profile:0.00011213496327400208 - timing_per_token_ms/gen:0.15801281254399527 - timing_per_token_ms/update_actor:0.038662449515184674 - timing_per_token_ms/adv:0.0003151548941285167 - perf/total_num_tokens:444947 - perf/time_per_step:81.33951114118099 - perf/throughput:683.7805418262618 (TaskRunner pid=2281424) Training Progress: 94%|█████████▍| 94/100 [2:14:18<11:05, 110.84s/it] (TaskRunner pid=2281424) step:95 - global_seqlen/min:41511 - global_seqlen/max:154263 - global_seqlen/minmax_diff:112752 - global_seqlen/balanced_min:88799 - global_seqlen/balanced_max:88957 - global_seqlen/mean:88866.125 - actor/entropy:0.4982995390892029 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2708369493484497 - training/rollout_probs_diff_mean:0.00339501746930182 - training/rollout_probs_diff_std:0.007665873970836401 - training/rollout_actor_probs_pearson_corr:0.999656081199646 - rollout_corr/rollout_is_mean:1.000037670135498 - rollout_corr/rollout_is_ratio_fraction_high:1.5620679505445878e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.5386104583740234 - rollout_corr/rollout_is_min:0.503195583820343 - rollout_corr/rollout_is_std:0.02833504043519497 - rollout_corr/rollout_is_eff_sample_size:0.9991978885434327 - rollout_corr/rollout_is_seq_mean:1.000043511390686 - rollout_corr/rollout_is_seq_std:0.0009036674164235592 - rollout_corr/rollout_is_seq_max:1.0054752826690674 - rollout_corr/rollout_is_seq_min:0.9958840012550354 - rollout_corr/rollout_is_seq_max_deviation:0.005475282669067383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.540353175252676) - rollout_corr/training_log_ppl:np.float64(0.41098539560334757) - rollout_corr/kl:np.float64(0.0004560288263313339) - rollout_corr/k3_kl:np.float64(0.0003980033748689493) - rollout_corr/rollout_ppl:np.float64(1.5396650568582118) - rollout_corr/rollout_log_ppl:np.float64(0.41052936331834644) - rollout_corr/log_ppl_diff:np.float64(0.0004560322850011289) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007419722969643772) - rollout_corr/log_ppl_diff_max:np.float64(0.005006879568099976) - rollout_corr/log_ppl_diff_min:np.float64(-0.004607051610946655) - rollout_corr/ppl_ratio:np.float64(1.0004565534181893) - rollout_corr/chi2_token:np.float64(0.0006959009915590286) - rollout_corr/chi2_seq:np.float64(27.76160763693042) - actor/pg_loss:np.float64(4.626181907951832e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.550116794798669e-05) - actor/ppo_kl:np.float64(0.00011306502917474148) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06149573251605034) - perf/mfu/actor:np.float64(0.21980245524914177) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.26933860778809) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004443692974746227 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004443692974746227 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2500.69140625 - response_length/max:8192.0 - response_length/min:209.0 - response_length/clip_ratio:0.01953125 - response_length_non_aborted/mean:2500.69140625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:209.0 - response_length_non_aborted/clip_ratio:0.01953125 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010070018470287323 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8333873972296715) - timing_s/agent_loop/generate_sequences/max:np.float64(72.34981802105904) - timing_s/agent_loop/generate_sequences/mean:np.float64(24.544051925644453) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(72.34981802105904) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:79.49770486913621 - timing_s/reward:0.14676685817539692 - timing_s/old_log_prob:5.38677878677845 - timing_s/adv:0.12247935496270657 - timing_s/update_actor:22.04177999868989 - timing_s/step:107.2831785902381 - timing_s/stop_profile:0.000116763636469841 - timing_per_token_ms/gen:0.12418082009996644 - timing_per_token_ms/update_actor:0.03100419310323519 - timing_per_token_ms/adv:0.00017228071293013306 - perf/total_num_tokens:710929 - perf/time_per_step:107.2831785902381 - perf/throughput:828.332327283283 (TaskRunner pid=2281424) Training Progress: 95%|█████████▌| 95/100 [2:16:05<09:08, 109.79s/it] (TaskRunner pid=2281424) step:96 - global_seqlen/min:36997 - global_seqlen/max:166139 - global_seqlen/minmax_diff:129142 - global_seqlen/balanced_min:75846 - global_seqlen/balanced_max:76107 - global_seqlen/mean:75997.125 - actor/entropy:0.471091628074646 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2679024338722229 - training/rollout_probs_diff_mean:0.00336635485291481 - training/rollout_probs_diff_std:0.0077355667017400265 - training/rollout_actor_probs_pearson_corr:0.999660849571228 - rollout_corr/rollout_is_mean:0.9999645948410034 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.7283869005477754e-06 - rollout_corr/rollout_is_max:1.7948318719863892 - rollout_corr/rollout_is_min:0.12595275044441223 - rollout_corr/rollout_is_std:0.028181063011288643 - rollout_corr/rollout_is_eff_sample_size:0.9992062198818573 - rollout_corr/rollout_is_seq_mean:0.9999157190322876 - rollout_corr/rollout_is_seq_std:0.0007819948368705809 - rollout_corr/rollout_is_seq_max:1.0023274421691895 - rollout_corr/rollout_is_seq_min:0.9979436993598938 - rollout_corr/rollout_is_seq_max_deviation:0.002327442169189453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4727759365923703) - rollout_corr/training_log_ppl:np.float64(0.37059994105948135) - rollout_corr/kl:np.float64(0.00044277470152032095) - rollout_corr/k3_kl:np.float64(0.00038263250905856694) - rollout_corr/rollout_ppl:np.float64(1.4721139534376562) - rollout_corr/rollout_log_ppl:np.float64(0.3701571657729801) - rollout_corr/log_ppl_diff:np.float64(0.0004427752865012735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007409804675262421) - rollout_corr/log_ppl_diff_max:np.float64(0.002924293279647827) - rollout_corr/log_ppl_diff_min:np.float64(-0.001986294984817505) - rollout_corr/ppl_ratio:np.float64(1.0004432362038642) - rollout_corr/chi2_token:np.float64(0.00064484728500247) - rollout_corr/chi2_seq:np.float64(1.0331782191060483) - actor/pg_loss:np.float64(3.8074213080108166e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.83946193855445e-05) - actor/ppo_kl:np.float64(-1.7157419575752897e-05) - actor/pg_clipfrac_lower:np.float64(6.802943062211853e-07) - actor/grad_norm:np.float64(0.08832370489835739) - perf/mfu/actor:np.float64(0.20191487584379592) - perf/max_memory_allocated_gb:np.float64(134.68813514709473) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.39874267578125) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.016185393556952477 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.016185393556952477 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2095.41015625 - response_length/max:8192.0 - response_length/min:297.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:2095.41015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:297.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.160566747188568e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.8253472223877907) - timing_s/agent_loop/generate_sequences/max:np.float64(69.03189334087074) - timing_s/agent_loop/generate_sequences/mean:np.float64(20.723504332338052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(69.03189334087074) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:355 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.10101301595569 - timing_s/reward:0.13098066858947277 - timing_s/old_log_prob:4.933196693658829 - timing_s/adv:0.12136180512607098 - timing_s/update_actor:20.294502025470138 - timing_s/step:101.66740370728076 - timing_s/stop_profile:3.110058605670929e-05 - timing_per_token_ms/gen:0.1418670140578006 - timing_per_token_ms/update_actor:0.03338037791803002 - timing_per_token_ms/adv:0.00019961578337021133 - perf/total_num_tokens:607977 - perf/time_per_step:101.66740370728076 - perf/throughput:747.5072858043053 (TaskRunner pid=2281424) Training Progress: 96%|█████████▌| 96/100 [2:17:47<07:09, 107.36s/it] (TaskRunner pid=2281424) step:97 - global_seqlen/min:33062 - global_seqlen/max:136692 - global_seqlen/minmax_diff:103630 - global_seqlen/balanced_min:83301 - global_seqlen/balanced_max:83461 - global_seqlen/mean:83356.5 - actor/entropy:0.5467954874038696 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25347256660461426 - training/rollout_probs_diff_mean:0.0037388440687209368 - training/rollout_probs_diff_std:0.007841377519071102 - training/rollout_actor_probs_pearson_corr:0.9996518492698669 - rollout_corr/rollout_is_mean:1.000118374824524 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.344011929584667e-06 - rollout_corr/rollout_is_max:1.951640248298645 - rollout_corr/rollout_is_min:0.06364211440086365 - rollout_corr/rollout_is_std:0.029398424550890923 - rollout_corr/rollout_is_eff_sample_size:0.9991367169708485 - rollout_corr/rollout_is_seq_mean:1.0000510215759277 - rollout_corr/rollout_is_seq_std:0.00096407305682078 - rollout_corr/rollout_is_seq_max:1.0034452676773071 - rollout_corr/rollout_is_seq_min:0.9948387145996094 - rollout_corr/rollout_is_seq_max_deviation:0.005161285400390625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5564615204930305) - rollout_corr/training_log_ppl:np.float64(0.42052613582927734) - rollout_corr/kl:np.float64(0.000411179766622638) - rollout_corr/k3_kl:np.float64(0.00041175296956907914) - rollout_corr/rollout_ppl:np.float64(1.5558123434893787) - rollout_corr/rollout_log_ppl:np.float64(0.42011495429323986) - rollout_corr/log_ppl_diff:np.float64(0.00041118153603747487) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007968471036292613) - rollout_corr/log_ppl_diff_max:np.float64(0.005975067615509033) - rollout_corr/log_ppl_diff_min:np.float64(-0.0025658011436462402) - rollout_corr/ppl_ratio:np.float64(1.0004117703065276) - rollout_corr/chi2_token:np.float64(0.000819820212200284) - rollout_corr/chi2_seq:np.float64(4.5130012354347855) - actor/pg_loss:np.float64(1.147331204265356e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.38316911426773e-05) - actor/ppo_kl:np.float64(6.123169056237998e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09202379267662764) - perf/mfu/actor:np.float64(0.2117913412574436) - perf/max_memory_allocated_gb:np.float64(135.608256816864) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.30643844604492) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.039547327905893326 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.039547327905893326 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:2336.265625 - response_length/max:8192.0 - response_length/min:270.0 - response_length/clip_ratio:0.0546875 - response_length_non_aborted/mean:2336.265625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:270.0 - response_length_non_aborted/clip_ratio:0.0546875 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.753873407840729e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.633798986673355) - timing_s/agent_loop/generate_sequences/max:np.float64(78.11388960666955) - timing_s/agent_loop/generate_sequences/mean:np.float64(22.929097295367683) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(78.11388960666955) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:85.37431517802179 - timing_s/reward:0.1406618170440197 - timing_s/old_log_prob:5.179867882281542 - timing_s/adv:0.11821933090686798 - timing_s/update_actor:21.53932829387486 - timing_s/step:112.4388401247561 - timing_s/stop_profile:2.911873161792755e-05 - timing_per_token_ms/gen:0.1427463620127303 - timing_per_token_ms/update_actor:0.03230001303718795 - timing_per_token_ms/adv:0.0001772797126001991 - perf/total_num_tokens:666852 - perf/time_per_step:112.4388401247561 - perf/throughput:741.3496964884386 (TaskRunner pid=2281424) Training Progress: 97%|█████████▋| 97/100 [2:19:39<05:26, 108.89s/it] (TaskRunner pid=2281424) step:98 - global_seqlen/min:36746 - global_seqlen/max:101541 - global_seqlen/minmax_diff:64795 - global_seqlen/balanced_min:66846 - global_seqlen/balanced_max:67095 - global_seqlen/mean:66942.25 - actor/entropy:0.46268388628959656 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19511407613754272 - training/rollout_probs_diff_mean:0.0033976633567363024 - training/rollout_probs_diff_std:0.007802109699696302 - training/rollout_actor_probs_pearson_corr:0.9996366500854492 - rollout_corr/rollout_is_mean:1.000012755393982 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.758634090423584 - rollout_corr/rollout_is_min:0.5404731035232544 - rollout_corr/rollout_is_std:0.02801772952079773 - rollout_corr/rollout_is_eff_sample_size:0.9992156225592431 - rollout_corr/rollout_is_seq_mean:1.0000487565994263 - rollout_corr/rollout_is_seq_std:0.0007411489496007562 - rollout_corr/rollout_is_seq_max:1.0029963254928589 - rollout_corr/rollout_is_seq_min:0.9981603622436523 - rollout_corr/rollout_is_seq_max_deviation:0.0029963254928588867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4537494746036828) - rollout_corr/training_log_ppl:np.float64(0.3591947711247485) - rollout_corr/kl:np.float64(0.00034263647683197007) - rollout_corr/k3_kl:np.float64(0.0003613955028072269) - rollout_corr/rollout_ppl:np.float64(1.453213640023023) - rollout_corr/rollout_log_ppl:np.float64(0.35885213530855253) - rollout_corr/log_ppl_diff:np.float64(0.0003426358161959797) - rollout_corr/log_ppl_abs_diff:np.float64(0.0006818583060521632) - rollout_corr/log_ppl_diff_max:np.float64(0.0025803744792938232) - rollout_corr/log_ppl_diff_min:np.float64(-0.0025287121534347534) - rollout_corr/ppl_ratio:np.float64(1.0003430175129324) - rollout_corr/chi2_token:np.float64(0.0007636058144271374) - rollout_corr/chi2_seq:np.float64(1.1469660245347768) - actor/pg_loss:np.float64(3.07583250105381e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.736449529398669e-05) - actor/ppo_kl:np.float64(3.588981383906287e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06043858523480594) - perf/mfu/actor:np.float64(0.19140438179546104) - perf/max_memory_allocated_gb:np.float64(135.608256816864) - perf/max_memory_reserved_gb:np.float64(144.69140625) - perf/cpu_memory_used_gb:np.float64(107.1692008972168) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015156281180679798 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015156281180679798 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1822.2578125 - response_length/max:8192.0 - response_length/min:417.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:1822.2578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:417.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.0846487283706665e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(5.219669699668884) - timing_s/agent_loop/generate_sequences/max:np.float64(69.51818436384201) - timing_s/agent_loop/generate_sequences/mean:np.float64(17.624326372162614) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(69.51818436384201) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:76.9898233152926 - timing_s/reward:0.12057511694729328 - timing_s/old_log_prob:4.5568945575505495 - timing_s/adv:0.12083770520985126 - timing_s/update_actor:18.797641713172197 - timing_s/step:100.67088041082025 - timing_s/stop_profile:3.077089786529541e-05 - timing_per_token_ms/gen:0.16503784221002576 - timing_per_token_ms/update_actor:0.03510048159639876 - timing_per_token_ms/adv:0.00022563796632517444 - perf/total_num_tokens:535538 - perf/time_per_step:100.67088041082025 - perf/throughput:664.9614041997089 (TaskRunner pid=2281424) Training Progress: 98%|█████████▊| 98/100 [2:21:20<03:32, 106.43s/it] (TaskRunner pid=2281424) step:99 - global_seqlen/min:29160 - global_seqlen/max:172289 - global_seqlen/minmax_diff:143129 - global_seqlen/balanced_min:67431 - global_seqlen/balanced_max:67700 - global_seqlen/mean:67597.625 - actor/entropy:0.47339898347854614 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3376702070236206 - training/rollout_probs_diff_mean:0.003389441641047597 - training/rollout_probs_diff_std:0.007820799946784973 - training/rollout_actor_probs_pearson_corr:0.9996418952941895 - rollout_corr/rollout_is_mean:1.0000227689743042 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.224498297844548e-06 - rollout_corr/rollout_is_max:1.6443817615509033 - rollout_corr/rollout_is_min:0.14922289550304413 - rollout_corr/rollout_is_std:0.028145084157586098 - rollout_corr/rollout_is_eff_sample_size:0.9992084812691172 - rollout_corr/rollout_is_seq_mean:1.0001269578933716 - rollout_corr/rollout_is_seq_std:0.0011187855852767825 - rollout_corr/rollout_is_seq_max:1.0088918209075928 - rollout_corr/rollout_is_seq_min:0.9973447322845459 - rollout_corr/rollout_is_seq_max_deviation:0.008891820907592773 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4072618028149009) - rollout_corr/training_log_ppl:np.float64(0.3233967706328258) - rollout_corr/kl:np.float64(0.00018692354644933573) - rollout_corr/k3_kl:np.float64(0.00037853698717071893) - rollout_corr/rollout_ppl:np.float64(1.406971170566976) - rollout_corr/rollout_log_ppl:np.float64(0.3232098486623727) - rollout_corr/log_ppl_diff:np.float64(0.00018692197045311332) - rollout_corr/log_ppl_abs_diff:np.float64(0.0007679337286390364) - rollout_corr/log_ppl_diff_max:np.float64(0.0030442774295806885) - rollout_corr/log_ppl_diff_min:np.float64(-0.007332652807235718) - rollout_corr/ppl_ratio:np.float64(1.000187524361536) - rollout_corr/chi2_token:np.float64(0.00115422741509974) - rollout_corr/chi2_seq:np.float64(0.9885839202906936) - actor/pg_loss:np.float64(1.733703538775444e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.918888794236409e-05) - actor/ppo_kl:np.float64(-5.2708012820090744e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.047866626060567796) - perf/mfu/actor:np.float64(0.18982802399143323) - perf/max_memory_allocated_gb:np.float64(135.608256816864) - perf/max_memory_reserved_gb:np.float64(144.7109375) - perf/cpu_memory_used_gb:np.float64(107.20830154418945) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.056533027440309525 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.056533027440309525 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1849.33203125 - response_length/max:8192.0 - response_length/min:178.0 - response_length/clip_ratio:0.02734375 - response_length_non_aborted/mean:1849.33203125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.02734375 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.295259714126587e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2005517426878214) - timing_s/agent_loop/generate_sequences/max:np.float64(71.4071844201535) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.98496175607579) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(71.4071844201535) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:78.54274812154472 - timing_s/reward:0.12314253114163876 - timing_s/old_log_prob:4.655178176239133 - timing_s/adv:0.1377553455531597 - timing_s/update_actor:19.54215112142265 - timing_s/step:103.08848447538912 - timing_s/stop_profile:0.00012132711708545685 - timing_per_token_ms/gen:0.16590185248800712 - timing_per_token_ms/update_actor:0.03613690407285509 - timing_per_token_ms/adv:0.00025473407082194033 - perf/total_num_tokens:540781 - perf/time_per_step:103.08848447538912 - perf/throughput:655.7243065896264 (TaskRunner pid=2281424) Training Progress: 99%|█████████▉| 99/100 [2:23:03<01:45, 105.44s/it] (TaskRunner pid=2281424) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2281424) validation generation end (TaskRunner pid=2281424) [prompt] system (TaskRunner pid=2281424) (TaskRunner pid=2281424) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) ... (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2281424) (TaskRunner pid=2281424) A (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) user (TaskRunner pid=2281424) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2281424) (TaskRunner pid=2281424) A: 50 pC (TaskRunner pid=2281424) B: 56 pC (TaskRunner pid=2281424) C: 60 pC (TaskRunner pid=2281424) D: 64 pC (TaskRunner pid=2281424) Please reason step by step. (TaskRunner pid=2281424) assistant (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) [response] We are given the electric field $ E $ due to a point charge $ q $ at a distance $ r $, and we are asked to find the magnitude of the charge. The formula for the electric field due to a point charge is: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) E = k \frac{q}{r^2} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given: (TaskRunner pid=2281424) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2281424) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2281424) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2281424) (TaskRunner pid=2281424) We need to solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E r^2}{k} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 1: Plug in the known values (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 2: Compute the square of the distance (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (0.50)^2 = 0.25 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 3: Multiply by the electric field (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 2.0 \cdot 0.25 = 0.50 (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 4: Divide by Coulomb's constant (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{0.50}{8.99 \times 10^9} \approx \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 5: Perform the division (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 6: Convert to picocoulombs (pC) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We know that $ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $, so: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \times 10^{-12} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) ### Step 7: Choose the closest option (TaskRunner pid=2281424) (TaskRunner pid=2281424) Looking at the answer choices: (TaskRunner pid=2281424) (TaskRunner pid=2281424) - A: 50 pC (TaskRunner pid=2281424) - B: 56 pC (TaskRunner pid=2281424) - C: 60 pC (TaskRunner pid=2281424) - D: 64 pC (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest value is **B: 56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) --- (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $ at a distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ from a point charge. Using the formula $ E = k \frac{q}{r^2} $, we solve for $ q $: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) q = \frac{E r^2}{k} = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) Converting to picocoulombs: (TaskRunner pid=2281424) (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2281424) $$ (TaskRunner pid=2281424) (TaskRunner pid=2281424) The closest answer is **B: 56 pC**. (TaskRunner pid=2281424) (TaskRunner pid=2281424) (TaskRunner pid=2281424) B (TaskRunner pid=2281424) (TaskRunner pid=2281424) [ground_truth] B (TaskRunner pid=2281424) [score] 1.0 (TaskRunner pid=2281424) [acc] 1.0 (TaskRunner pid=2281424) [pred] B (TaskRunner pid=2281424) [incorrect_format] 1 (TaskRunner pid=2281424) [feedback] (TaskRunner pid=2281424) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2281424) step:100 - global_seqlen/min:33724 - global_seqlen/max:110235 - global_seqlen/minmax_diff:76511 - global_seqlen/balanced_min:67861 - global_seqlen/balanced_max:68027 - global_seqlen/mean:67950.125 - actor/entropy:0.45818907022476196 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5070421695709229 - training/rollout_probs_diff_mean:0.0032882497180253267 - training/rollout_probs_diff_std:0.007731531746685505 - training/rollout_actor_probs_pearson_corr:0.9995552897453308 - rollout_corr/rollout_is_mean:1.0000216960906982 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.213497959426604e-06 - rollout_corr/rollout_is_max:1.5396182537078857 - rollout_corr/rollout_is_min:0.46327918767929077 - rollout_corr/rollout_is_std:0.02762351557612419 - rollout_corr/rollout_is_eff_sample_size:0.9992375231521491 - rollout_corr/rollout_is_seq_mean:1.000037670135498 - rollout_corr/rollout_is_seq_std:0.0009224399691447616 - rollout_corr/rollout_is_seq_max:1.0046027898788452 - rollout_corr/rollout_is_seq_min:0.9973651766777039 - rollout_corr/rollout_is_seq_max_deviation:0.004602789878845215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4568524220958352) - rollout_corr/training_log_ppl:np.float64(0.35900981855229475) - rollout_corr/kl:np.float64(0.0003901403655550695) - rollout_corr/k3_kl:np.float64(0.00036932118109689327) - rollout_corr/rollout_ppl:np.float64(1.4562692898325622) - rollout_corr/rollout_log_ppl:np.float64(0.3586196781543549) - rollout_corr/log_ppl_diff:np.float64(0.000390140397939831) - rollout_corr/log_ppl_abs_diff:np.float64(0.0008009577286429703) - rollout_corr/log_ppl_diff_max:np.float64(0.003635212779045105) - rollout_corr/log_ppl_diff_min:np.float64(-0.003899216651916504) - rollout_corr/ppl_ratio:np.float64(1.00039070751518) - rollout_corr/chi2_token:np.float64(0.0006887984927743673) - rollout_corr/chi2_seq:np.float64(3.438023897819221) - actor/pg_loss:np.float64(-1.3369019143283367e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.500477859892271e-05) - actor/ppo_kl:np.float64(6.788543978900052e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.04646363575011492) - perf/mfu/actor:np.float64(0.1938518522795187) - perf/max_memory_allocated_gb:np.float64(135.608256816864) - perf/max_memory_reserved_gb:np.float64(144.7109375) - perf/cpu_memory_used_gb:np.float64(107.14533233642578) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7296875) - val-aux/sciknoweval/reward/std@16:np.float64(0.21113028788197769) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.8195625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18312693662059415) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6398875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1769580686853267) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7299875) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.21049045089342963) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8962625000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1329636020160485) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5704125) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12181204475036858) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.76555) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18390036772261203) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.94765) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07747202634384016) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.52575) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07452105831546266) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7921125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15178369995448104) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.9732) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03523755515622687) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.49838750000000004) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.045061773818715276) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.8134375) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11988946359651878) - val-aux/sciknoweval/score/mean@16:np.float64(0.7296875) - val-aux/sciknoweval/score/std@16:np.float64(0.21113028788197769) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.8195625) - val-aux/sciknoweval/score/best@2/std:np.float64(0.18312693662059415) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6398875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1769580686853267) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7299875) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.21049045089342963) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8962625000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1329636020160485) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5704125) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12181204475036858) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.76555) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18390036772261203) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.94765) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07747202634384016) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.52575) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07452105831546266) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7921125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15178369995448104) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.9732) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03523755515622687) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.49838750000000004) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.045061773818715276) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.8134375) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11988946359651878) - val-core/sciknoweval/acc/mean@16:np.float64(0.7296875) - val-aux/sciknoweval/acc/std@16:np.float64(0.21113028788197769) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.8195625) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.18312693662059415) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6398875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1769580686853267) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7299875) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.21049045089342963) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8962625000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1329636020160485) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5704125) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12181204475036858) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.76555) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18390036772261203) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.94765) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07747202634384016) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.52575) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07452105831546266) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7921125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15178369995448104) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.9732) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03523755515622687) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.49838750000000004) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.045061773818715276) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.8134375) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11988946359651878) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.91796875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.12921982287160344) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.969875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.07402285219911191) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.8673375) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.14334823308252115) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9178625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.1290617057662198) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.99245) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.02834928965541194) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.80435) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.13896621816207683) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9458125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.10365239649604263) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9992375000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.006519985280372976) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7432874999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.112524959047603) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9701000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.07166964518321907) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(0.9999500000000001) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0010787124614257437) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.695975) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.07060170833571836) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9865999999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.03874947109834628) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023091811686754227 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023091811686754227 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1854.16015625 - response_length/max:8192.0 - response_length/min:336.0 - response_length/clip_ratio:0.01171875 - response_length_non_aborted/mean:1854.16015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:336.0 - response_length_non_aborted/clip_ratio:0.01171875 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.544752538204193e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.829353081062436) - timing_s/agent_loop/generate_sequences/max:np.float64(67.18781132809818) - timing_s/agent_loop/generate_sequences/mean:np.float64(17.904436235730827) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(67.18781132809818) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:74.55190750397742 - timing_s/reward:0.11913642473518848 - timing_s/old_log_prob:4.5896848775446415 - timing_s/adv:0.12158972769975662 - timing_s/update_actor:19.037666445598006 - timing_s/step:98.50110033527017 - timing_s/testing:211.57785162888467 - timing_s/save_checkpoint:12.600243583321571 - timing_s/stop_profile:0.00014124438166618347 - timing_per_token_ms/gen:0.15706215436987647 - timing_per_token_ms/update_actor:0.035021397027595616 - timing_per_token_ms/adv:0.000223674584299434 - perf/total_num_tokens:543601 - perf/time_per_step:98.50110033527017 - perf/throughput:689.8412786122876 (TaskRunner pid=2281424) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2281424) "np.float64(0.7296875), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2281424) "np.float64(0.21113028788197769), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2281424) "np.float64(0.8195625), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2281424) "np.float64(0.18312693662059415), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2281424) "np.float64(0.6398875), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2281424) "np.float64(0.1769580686853267), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2281424) "np.float64(0.7299875), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2281424) "np.float64(0.21049045089342963), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2281424) "np.float64(0.8962625000000001), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2281424) "np.float64(0.1329636020160485), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2281424) "np.float64(0.5704125), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2281424) "np.float64(0.12181204475036858), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2281424) "np.float64(0.76555), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2281424) "np.float64(0.18390036772261203), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2281424) "np.float64(0.94765), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2281424) "np.float64(0.07747202634384016), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2281424) "np.float64(0.52575), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2281424) "np.float64(0.07452105831546266), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2281424) "np.float64(0.7921125), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2281424) "np.float64(0.15178369995448104), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2281424) "np.float64(0.9732), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2281424) "np.float64(0.03523755515622687), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2281424) "np.float64(0.49838750000000004), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2281424) "np.float64(0.045061773818715276), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2281424) "np.float64(0.8134375), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2281424) "np.float64(0.11988946359651878), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2281424) "np.float64(0.7296875), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2281424) "np.float64(0.21113028788197769), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2281424) "np.float64(0.8195625), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2281424) "np.float64(0.18312693662059415), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2281424) "np.float64(0.6398875), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2281424) "np.float64(0.1769580686853267), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2281424) "np.float64(0.7299875), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2281424) "np.float64(0.21049045089342963), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2281424) "np.float64(0.8962625000000001), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2281424) "np.float64(0.1329636020160485), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2281424) "np.float64(0.5704125), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2281424) "np.float64(0.12181204475036858), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2281424) "np.float64(0.76555), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2281424) "np.float64(0.18390036772261203), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2281424) "np.float64(0.94765), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2281424) "np.float64(0.07747202634384016), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2281424) "np.float64(0.52575), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2281424) "np.float64(0.07452105831546266), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2281424) "np.float64(0.7921125), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2281424) "np.float64(0.15178369995448104), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2281424) "np.float64(0.9732), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2281424) "np.float64(0.03523755515622687), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2281424) "np.float64(0.49838750000000004), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2281424) "np.float64(0.045061773818715276), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2281424) "np.float64(0.8134375), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2281424) "np.float64(0.11988946359651878), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2281424) "np.float64(0.7296875), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2281424) "np.float64(0.21113028788197769), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2281424) "np.float64(0.8195625), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2281424) "np.float64(0.18312693662059415), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2281424) "np.float64(0.6398875), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2281424) "np.float64(0.1769580686853267), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2281424) "np.float64(0.7299875), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2281424) "np.float64(0.21049045089342963), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2281424) "np.float64(0.8962625000000001), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2281424) "np.float64(0.1329636020160485), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2281424) "np.float64(0.5704125), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2281424) "np.float64(0.12181204475036858), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2281424) "np.float64(0.76555), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2281424) "np.float64(0.18390036772261203), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2281424) "np.float64(0.94765), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2281424) "np.float64(0.07747202634384016), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2281424) "np.float64(0.52575), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2281424) "np.float64(0.07452105831546266), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2281424) "np.float64(0.7921125), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2281424) "np.float64(0.15178369995448104), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2281424) "np.float64(0.9732), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2281424) "np.float64(0.03523755515622687), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2281424) "np.float64(0.49838750000000004), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2281424) "np.float64(0.045061773818715276), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2281424) "np.float64(0.8134375), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2281424) 'np.float64(0.11988946359651878), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.91796875), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2281424) 'np.float64(0.12921982287160344), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(0.969875), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2281424) 'np.float64(0.07402285219911191), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': np.float64(0.8673375), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2281424) 'np.float64(0.14334823308252115), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9178625), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2281424) 'np.float64(0.1290617057662198), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(0.99245), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2281424) 'np.float64(0.02834928965541194), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(0.80435), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2281424) 'np.float64(0.13896621816207683), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(0.9458125), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2281424) 'np.float64(0.10365239649604263), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@8/mean': " (TaskRunner pid=2281424) 'np.float64(0.9992375000000001), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@8/std': " (TaskRunner pid=2281424) 'np.float64(0.006519985280372976), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2281424) 'np.float64(0.7432874999999999), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2281424) 'np.float64(0.112524959047603), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2281424) 'np.float64(0.9701000000000001), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2281424) 'np.float64(0.07166964518321907), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@16/mean': " (TaskRunner pid=2281424) 'np.float64(0.9999500000000001), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/best@16/std': " (TaskRunner pid=2281424) 'np.float64(0.0010787124614257437), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(0.695975), " (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2281424) 'np.float64(0.07060170833571836), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2281424) 'np.float64(0.9865999999999999), ' (TaskRunner pid=2281424) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2281424) "np.float64(0.03874947109834628), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2281424) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2281424) 'np.float64(2.0)}') (TaskRunner pid=2281424) Training Progress: 100%|██████████| 100/100 [2:28:26<00:00, 170.63s/it] Training Progress: 100%|██████████| 100/100 [2:28:26<00:00, 89.06s/it] (TaskRunner pid=2281424) wandb: updating run metadata (TaskRunner pid=2281424) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2281424) wandb: uploading output.log (TaskRunner pid=2281424) wandb: uploading history steps 98-99, summary, console lines 1063-1188 (TaskRunner pid=2281424) wandb: (TaskRunner pid=2281424) wandb: Run history: (TaskRunner pid=2281424) wandb: actor/entropy ▂▂▂▂▂▁▂▃▂▃▄▃▄▃▂▃▄▅▄▄▅▄▅▅▅▅▆▅▆▅▆▄▇█▆▅▆▇▇▆ (TaskRunner pid=2281424) wandb: actor/grad_norm ▄▄▄▅▅▅▄█▆▃▃▃▄▄▅▃▃▃▃▃▃▃▄▃▄▂▂▂▂▁▁▃▂▂▂▂▁▁▂▁ (TaskRunner pid=2281424) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2281424) wandb: actor/lr ▁▂▅▅▆███████████████████████████████████ (TaskRunner pid=2281424) wandb: actor/pg_clipfrac ▂▄█▄▆▃█▄▄▅▄▇▆▃▄▃▄▄▂▄▂▃▃▅▂▂▂▃▂▂▂▁▂▁▂▂▂▂▂▁ (TaskRunner pid=2281424) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▂▁▁▁▁▁▁▁▁▄▁▁▁▁▁▁▁▂ (TaskRunner pid=2281424) wandb: actor/pg_loss ▁▅▃▅▄▄▄▇▃▁▅▄▁█▃▂▄▄▅▄▃▃▅▄▅▄▅▂▃▅▄▅▅▄▄▅▃▁▄▄ (TaskRunner pid=2281424) wandb: actor/ppo_kl ▃▂▄▅▅▇▁▆█▅▄▄▅▃▄▄▃▅▆▃▃▂▄▄▄▄▃▄▄▃▂▄▃▃▄▄▄▄▄▄ (TaskRunner pid=2281424) wandb: critic/advantages/max ███▁████████████████████████████████████ (TaskRunner pid=2281424) wandb: critic/advantages/mean ▇▆▇▆▇█▇█▄▇▇▄▇▇▆██▇▅▅▂█▅▇▄▁▅▅▆▄▆▇▄▅▇▆▆▆▅▅ (TaskRunner pid=2281424) wandb: +204 ... (TaskRunner pid=2281424) wandb: (TaskRunner pid=2281424) wandb: Run summary: (TaskRunner pid=2281424) wandb: actor/entropy 0.45819 (TaskRunner pid=2281424) wandb: actor/grad_norm 0.04646 (TaskRunner pid=2281424) wandb: actor/kl_loss 0 (TaskRunner pid=2281424) wandb: actor/lr 0.0 (TaskRunner pid=2281424) wandb: actor/pg_clipfrac 5e-05 (TaskRunner pid=2281424) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2281424) wandb: actor/pg_loss -1e-05 (TaskRunner pid=2281424) wandb: actor/ppo_kl 7e-05 (TaskRunner pid=2281424) wandb: critic/advantages/max 0.875 (TaskRunner pid=2281424) wandb: critic/advantages/mean -0.02309 (TaskRunner pid=2281424) wandb: +204 ... (TaskRunner pid=2281424) wandb: (TaskRunner pid=2281424) wandb: 🚀 View run qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/6ig3l55l (TaskRunner pid=2281424) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2281424) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2281424) wandb: Find logs at: ./wandb/run-20260703_032040-6ig3l55l/logs (TaskRunner pid=2281424) Exception ignored in atexit callback: .teardown_atexit at 0x7d739de7b920> (TaskRunner pid=2281424) Traceback (most recent call last): (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2281424) conn.teardown(hooks.exit_code) (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2281424) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2281424) return future.result() (TaskRunner pid=2281424) ^^^^^^^^^^^^^^^ (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2281424) return self.__get_result() (TaskRunner pid=2281424) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2281424) raise self._exception (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2281424) return await fn() (TaskRunner pid=2281424) ^^^^^^^^^^ (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2281424) await self._client.close() (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2281424) await self._writer.wait_closed() (TaskRunner pid=2281424) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2281424) await self._protocol._get_close_waiter(self) (TaskRunner pid=2281424) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-03 05:49:13] Finished Qwen/Qwen3-8B physics grpo. [2026-07-03 05:49:13] Starting Qwen/Qwen3-8B physics rlsd_tr. Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_rlsd_tr_Qwen3_8B 2026-07-03 05:49:15,332 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 05:49:25,205 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2320396) TaskRunner hostname: mole-workspace-rw, PID: 2320396 (TaskRunner pid=2320396) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2320396) 'calculate_entropy': False, (TaskRunner pid=2320396) 'calculate_sum_pi_squared': False, (TaskRunner pid=2320396) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2320396) 'async_save': False, (TaskRunner pid=2320396) 'load_contents': ['model', (TaskRunner pid=2320396) 'optimizer', (TaskRunner pid=2320396) 'extra'], (TaskRunner pid=2320396) 'save_contents': ['model', (TaskRunner pid=2320396) 'optimizer', (TaskRunner pid=2320396) 'extra']}, (TaskRunner pid=2320396) 'clip_ratio': 0.2, (TaskRunner pid=2320396) 'clip_ratio_c': 3.0, (TaskRunner pid=2320396) 'clip_ratio_high': 0.28, (TaskRunner pid=2320396) 'clip_ratio_low': 0.2, (TaskRunner pid=2320396) 'data_loader_seed': 42, (TaskRunner pid=2320396) 'entropy_checkpointing': False, (TaskRunner pid=2320396) 'entropy_coeff': 0, (TaskRunner pid=2320396) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2320396) 'freeze_vision_tower': False, (TaskRunner pid=2320396) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2320396) 'dtype': 'bfloat16', (TaskRunner pid=2320396) 'entropy_checkpointing': False, (TaskRunner pid=2320396) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2320396) 'forward_only': False, (TaskRunner pid=2320396) 'forward_prefetch': False, (TaskRunner pid=2320396) 'fsdp_size': -1, (TaskRunner pid=2320396) 'full_determinism': False, (TaskRunner pid=2320396) 'model_dtype': 'fp32', (TaskRunner pid=2320396) 'offload_policy': False, (TaskRunner pid=2320396) 'optimizer_offload': False, (TaskRunner pid=2320396) 'param_offload': False, (TaskRunner pid=2320396) 'reshard_after_forward': True, (TaskRunner pid=2320396) 'seed': 42, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_orig_params': False, (TaskRunner pid=2320396) 'use_torch_compile': True, (TaskRunner pid=2320396) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2320396) 'grad_clip': 1.0, (TaskRunner pid=2320396) 'kl_loss_coef': 0.001, (TaskRunner pid=2320396) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2320396) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2320396) 'loss_scale_factor': None, (TaskRunner pid=2320396) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2320396) 'betas': [0.9, 0.999], (TaskRunner pid=2320396) 'clip_grad': 1.0, (TaskRunner pid=2320396) 'lr': 1e-06, (TaskRunner pid=2320396) 'lr_scheduler_type': 'constant', (TaskRunner pid=2320396) 'lr_warmup_steps': 10, (TaskRunner pid=2320396) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2320396) 'min_lr_ratio': 0.0, (TaskRunner pid=2320396) 'num_cycles': 0.5, (TaskRunner pid=2320396) 'optimizer': 'AdamW', (TaskRunner pid=2320396) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2320396) 'override_optimizer_config': None, (TaskRunner pid=2320396) 'total_training_steps': -1, (TaskRunner pid=2320396) 'warmup_style': None, (TaskRunner pid=2320396) 'weight_decay': 0.01}, (TaskRunner pid=2320396) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2320396) 'clip_cov_lb': 1.0, (TaskRunner pid=2320396) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2320396) 'clip_cov_ub': 5.0, (TaskRunner pid=2320396) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2320396) 'loss_mode': 'rlsd', (TaskRunner pid=2320396) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2320396) 'ppo_epochs': 1, (TaskRunner pid=2320396) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2320396) 'ppo_micro_batch_size': None, (TaskRunner pid=2320396) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2320396) 'ppo_mini_batch_size': 8, (TaskRunner pid=2320396) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'all_ranks': False, (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'ranks': [], (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'tool': None, (TaskRunner pid=2320396) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2320396) 'analysis': True, (TaskRunner pid=2320396) 'contents': [], (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'level': 'level0'}, (TaskRunner pid=2320396) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'discrete': False}, (TaskRunner pid=2320396) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2320396) 'step_end': None, (TaskRunner pid=2320396) 'step_start': 0}, (TaskRunner pid=2320396) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2320396) 'rollout_n': 8, (TaskRunner pid=2320396) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2320396) 'mode': 'disabled', (TaskRunner pid=2320396) 'record_file': None, (TaskRunner pid=2320396) 'replay_file': None}, (TaskRunner pid=2320396) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2320396) 'alpha': 0.5, (TaskRunner pid=2320396) 'distillation_add_tail': True, (TaskRunner pid=2320396) 'distillation_topk': 100, (TaskRunner pid=2320396) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2320396) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2320396) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'loss_weight': 0.0, (TaskRunner pid=2320396) 'mask': 'all'}, (TaskRunner pid=2320396) 'feedback_template': '\n' (TaskRunner pid=2320396) 'The ' (TaskRunner pid=2320396) 'following ' (TaskRunner pid=2320396) 'is ' (TaskRunner pid=2320396) 'feedback ' (TaskRunner pid=2320396) 'from ' (TaskRunner pid=2320396) 'your ' (TaskRunner pid=2320396) 'unsuccessful ' (TaskRunner pid=2320396) 'earlier ' (TaskRunner pid=2320396) 'attempt:\n' (TaskRunner pid=2320396) '\n' (TaskRunner pid=2320396) '{feedback_raw}', (TaskRunner pid=2320396) 'full_logit_distillation': True, (TaskRunner pid=2320396) 'include_environment_feedback': True, (TaskRunner pid=2320396) 'is_clip': 2, (TaskRunner pid=2320396) 'max_reprompt_len': 10240, (TaskRunner pid=2320396) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2320396) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2320396) '\n' (TaskRunner pid=2320396) 'Correctly ' (TaskRunner pid=2320396) 'solve ' (TaskRunner pid=2320396) 'the ' (TaskRunner pid=2320396) 'original ' (TaskRunner pid=2320396) 'question.', (TaskRunner pid=2320396) 'reprompt_truncation': 'right', (TaskRunner pid=2320396) 'solution_template': '\n' (TaskRunner pid=2320396) 'Correct ' (TaskRunner pid=2320396) 'solution:\n' (TaskRunner pid=2320396) '\n' (TaskRunner pid=2320396) '{successful_previous_attempt}', (TaskRunner pid=2320396) 'srpo_dynamic_weighting': False, (TaskRunner pid=2320396) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2320396) 'success_reward_threshold': 0.5, (TaskRunner pid=2320396) 'teacher_regularization': 'trust-region', (TaskRunner pid=2320396) 'teacher_update_rate': 0.1, (TaskRunner pid=2320396) 'token_reweight_decay_steps': 0, (TaskRunner pid=2320396) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2320396) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2320396) 'shuffle': False, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2320396) 'tau_neg': 1.05, (TaskRunner pid=2320396) 'tau_pos': 1.0, (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_dynamic_bsz': False, (TaskRunner pid=2320396) 'use_fused_kernels': False, (TaskRunner pid=2320396) 'use_kl_loss': False, (TaskRunner pid=2320396) 'use_prefix_grouper': False, (TaskRunner pid=2320396) 'use_remove_padding': True, (TaskRunner pid=2320396) 'use_torch_compile': True}, (TaskRunner pid=2320396) 'hybrid_engine': True, (TaskRunner pid=2320396) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2320396) 'custom_chat_template': None, (TaskRunner pid=2320396) 'enable_activation_offload': False, (TaskRunner pid=2320396) 'enable_gradient_checkpointing': True, (TaskRunner pid=2320396) 'exclude_modules': None, (TaskRunner pid=2320396) 'external_lib': None, (TaskRunner pid=2320396) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2320396) 'hf_config_path': None, (TaskRunner pid=2320396) 'lora_adapter_path': None, (TaskRunner pid=2320396) 'lora_alpha': 16, (TaskRunner pid=2320396) 'lora_rank': 0, (TaskRunner pid=2320396) 'override_config': {}, (TaskRunner pid=2320396) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2320396) 'target_modules': 'all-linear', (TaskRunner pid=2320396) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2320396) 'num_shards': 4}, (TaskRunner pid=2320396) 'tokenizer_path': None, (TaskRunner pid=2320396) 'trust_remote_code': True, (TaskRunner pid=2320396) 'use_fused_kernels': False, (TaskRunner pid=2320396) 'use_liger': False, (TaskRunner pid=2320396) 'use_remove_padding': True, (TaskRunner pid=2320396) 'use_shm': False}, (TaskRunner pid=2320396) 'nccl_timeout': 600, (TaskRunner pid=2320396) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2320396) 'entropy_checkpointing': False, (TaskRunner pid=2320396) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2320396) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2320396) 'dtype': 'bfloat16', (TaskRunner pid=2320396) 'entropy_checkpointing': False, (TaskRunner pid=2320396) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2320396) 'forward_only': True, (TaskRunner pid=2320396) 'forward_prefetch': False, (TaskRunner pid=2320396) 'fsdp_size': -1, (TaskRunner pid=2320396) 'full_determinism': False, (TaskRunner pid=2320396) 'model_dtype': 'fp32', (TaskRunner pid=2320396) 'offload_policy': False, (TaskRunner pid=2320396) 'optimizer_offload': False, (TaskRunner pid=2320396) 'param_offload': False, (TaskRunner pid=2320396) 'reshard_after_forward': True, (TaskRunner pid=2320396) 'seed': 42, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_orig_params': False, (TaskRunner pid=2320396) 'use_torch_compile': True, (TaskRunner pid=2320396) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2320396) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2320396) 'log_prob_micro_batch_size': None, (TaskRunner pid=2320396) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2320396) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2320396) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'all_ranks': False, (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'ranks': [], (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'tool': None, (TaskRunner pid=2320396) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2320396) 'analysis': True, (TaskRunner pid=2320396) 'contents': [], (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'level': 'level0'}, (TaskRunner pid=2320396) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'discrete': False}, (TaskRunner pid=2320396) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2320396) 'step_end': None, (TaskRunner pid=2320396) 'step_start': 0}, (TaskRunner pid=2320396) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2320396) 'rollout_n': 8, (TaskRunner pid=2320396) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2320396) 'mode': 'disabled', (TaskRunner pid=2320396) 'record_file': None, (TaskRunner pid=2320396) 'replay_file': None}, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_torch_compile': True}, (TaskRunner pid=2320396) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2320396) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2320396) 'agent_loop_config_path': None, (TaskRunner pid=2320396) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2320396) 'name': None, (TaskRunner pid=2320396) 'path': None}, (TaskRunner pid=2320396) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2320396) 'num_workers': 8}, (TaskRunner pid=2320396) 'calculate_log_probs': True, (TaskRunner pid=2320396) 'cudagraph_capture_sizes': None, (TaskRunner pid=2320396) 'data_parallel_size': 1, (TaskRunner pid=2320396) 'disable_log_stats': True, (TaskRunner pid=2320396) 'do_sample': True, (TaskRunner pid=2320396) 'dtype': 'bfloat16', (TaskRunner pid=2320396) 'enable_chunked_prefill': True, (TaskRunner pid=2320396) 'enable_prefix_caching': True, (TaskRunner pid=2320396) 'enable_rollout_routing_replay': False, (TaskRunner pid=2320396) 'enforce_eager': False, (TaskRunner pid=2320396) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2320396) 'expert_parallel_size': 1, (TaskRunner pid=2320396) 'free_cache_engine': True, (TaskRunner pid=2320396) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2320396) 'ignore_eos': False, (TaskRunner pid=2320396) 'layered_summon': False, (TaskRunner pid=2320396) 'load_format': 'dummy', (TaskRunner pid=2320396) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2320396) 'log_prob_micro_batch_size': None, (TaskRunner pid=2320396) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2320396) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2320396) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2320396) 'max_model_len': 10240, (TaskRunner pid=2320396) 'max_num_batched_tokens': 10240, (TaskRunner pid=2320396) 'max_num_seqs': 1024, (TaskRunner pid=2320396) 'mode': 'async', (TaskRunner pid=2320396) 'multi_stage_wake_up': False, (TaskRunner pid=2320396) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'format': 'hermes', (TaskRunner pid=2320396) 'interaction_config_path': None, (TaskRunner pid=2320396) 'max_assistant_turns': None, (TaskRunner pid=2320396) 'max_parallel_calls': 1, (TaskRunner pid=2320396) 'max_tool_response_length': 256, (TaskRunner pid=2320396) 'max_user_turns': None, (TaskRunner pid=2320396) 'num_repeat_rollouts': None, (TaskRunner pid=2320396) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2320396) 'tool_config_path': None, (TaskRunner pid=2320396) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2320396) 'use_inference_chat_template': False}, (TaskRunner pid=2320396) 'n': 8, (TaskRunner pid=2320396) 'name': 'vllm', (TaskRunner pid=2320396) 'over_sample_rate': 0, (TaskRunner pid=2320396) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2320396) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'all_ranks': False, (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'ranks': [], (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'tool': None, (TaskRunner pid=2320396) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2320396) 'analysis': True, (TaskRunner pid=2320396) 'contents': [], (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'level': 'level0'}, (TaskRunner pid=2320396) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'discrete': False}, (TaskRunner pid=2320396) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2320396) 'step_end': None, (TaskRunner pid=2320396) 'step_start': 0}, (TaskRunner pid=2320396) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2320396) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2320396) 'port': 9090, (TaskRunner pid=2320396) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2320396) 'prompt_length': 2048, (TaskRunner pid=2320396) 'quantization': None, (TaskRunner pid=2320396) 'quantization_config_file': None, (TaskRunner pid=2320396) 'response_length': 8192, (TaskRunner pid=2320396) 'scheduling_policy': 'fcfs', (TaskRunner pid=2320396) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2320396) 'skip_rollout': False, (TaskRunner pid=2320396) 'skip_tokenizer_init': True, (TaskRunner pid=2320396) 'temperature': 1.0, (TaskRunner pid=2320396) 'tensor_model_parallel_size': 2, (TaskRunner pid=2320396) 'top_k': -1, (TaskRunner pid=2320396) 'top_p': 1.0, (TaskRunner pid=2320396) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2320396) 'backend': None, (TaskRunner pid=2320396) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2320396) 'token2text': False}, (TaskRunner pid=2320396) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2320396) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2320396) 'do_sample': True, (TaskRunner pid=2320396) 'n': 16, (TaskRunner pid=2320396) 'temperature': 0.6, (TaskRunner pid=2320396) 'top_k': -1, (TaskRunner pid=2320396) 'top_p': 0.95}}}, (TaskRunner pid=2320396) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2320396) 'adv_estimator': 'grpo', (TaskRunner pid=2320396) 'gamma': 1.0, (TaskRunner pid=2320396) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2320396) 'horizon': 10000, (TaskRunner pid=2320396) 'kl_coef': 0.001, (TaskRunner pid=2320396) 'target_kl': 0.1, (TaskRunner pid=2320396) 'type': 'fixed'}, (TaskRunner pid=2320396) 'kl_penalty': 'kl', (TaskRunner pid=2320396) 'lam': 1.0, (TaskRunner pid=2320396) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2320396) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2320396) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2320396) 'loss_type': 'ppo_clip', (TaskRunner pid=2320396) 'rollout_is': 'token', (TaskRunner pid=2320396) 'rollout_is_batch_normalize': False, (TaskRunner pid=2320396) 'rollout_is_threshold': 2.0, (TaskRunner pid=2320396) 'rollout_rs': None, (TaskRunner pid=2320396) 'rollout_rs_threshold': None}, (TaskRunner pid=2320396) 'use_kl_in_reward': False, (TaskRunner pid=2320396) 'use_pf_ppo': False}, (TaskRunner pid=2320396) 'critic': (TaskRunner pid=2320396) {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2320396) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2320396) 'async_save': False, (TaskRunner pid=2320396) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2320396) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2320396) 'cliprange_value': 0.5, (TaskRunner pid=2320396) 'data_loader_seed': 42, (TaskRunner pid=2320396) 'enable': None, (TaskRunner pid=2320396) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2320396) 'forward_micro_batch_size': None, (TaskRunner pid=2320396) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2320396) 'grad_clip': 1.0, (TaskRunner pid=2320396) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2320396) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2320396) 'enable_activation_offload': False, (TaskRunner pid=2320396) 'enable_gradient_checkpointing': True, (TaskRunner pid=2320396) 'external_lib': None, (TaskRunner pid=2320396) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2320396) 'dtype': 'bfloat16', (TaskRunner pid=2320396) 'entropy_checkpointing': False, (TaskRunner pid=2320396) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2320396) 'forward_only': False, (TaskRunner pid=2320396) 'forward_prefetch': False, (TaskRunner pid=2320396) 'fsdp_size': -1, (TaskRunner pid=2320396) 'full_determinism': False, (TaskRunner pid=2320396) 'model_dtype': 'fp32', (TaskRunner pid=2320396) 'offload_policy': False, (TaskRunner pid=2320396) 'optimizer_offload': False, (TaskRunner pid=2320396) 'param_offload': False, (TaskRunner pid=2320396) 'reshard_after_forward': True, (TaskRunner pid=2320396) 'seed': 42, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_orig_params': False, (TaskRunner pid=2320396) 'use_torch_compile': True, (TaskRunner pid=2320396) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2320396) 'lora_alpha': 16, (TaskRunner pid=2320396) 'lora_rank': 0, (TaskRunner pid=2320396) 'override_config': {}, (TaskRunner pid=2320396) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2320396) 'target_modules': 'all-linear', (TaskRunner pid=2320396) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2320396) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2320396) 'trust_remote_code': True, (TaskRunner pid=2320396) 'use_remove_padding': False, (TaskRunner pid=2320396) 'use_shm': False}, (TaskRunner pid=2320396) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2320396) 'betas': [0.9, 0.999], (TaskRunner pid=2320396) 'clip_grad': 1.0, (TaskRunner pid=2320396) 'lr': 1e-05, (TaskRunner pid=2320396) 'lr_scheduler_type': 'constant', (TaskRunner pid=2320396) 'lr_warmup_steps': -1, (TaskRunner pid=2320396) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2320396) 'min_lr_ratio': 0.0, (TaskRunner pid=2320396) 'num_cycles': 0.5, (TaskRunner pid=2320396) 'optimizer': 'AdamW', (TaskRunner pid=2320396) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2320396) 'override_optimizer_config': None, (TaskRunner pid=2320396) 'total_training_steps': -1, (TaskRunner pid=2320396) 'warmup_style': None, (TaskRunner pid=2320396) 'weight_decay': 0.01}, (TaskRunner pid=2320396) 'ppo_epochs': 1, (TaskRunner pid=2320396) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2320396) 'ppo_micro_batch_size': None, (TaskRunner pid=2320396) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2320396) 'ppo_mini_batch_size': 8, (TaskRunner pid=2320396) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'all_ranks': False, (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'ranks': [], (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'tool': None, (TaskRunner pid=2320396) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2320396) 'analysis': True, (TaskRunner pid=2320396) 'contents': [], (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'level': 'level0'}, (TaskRunner pid=2320396) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'discrete': False}, (TaskRunner pid=2320396) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2320396) 'step_end': None, (TaskRunner pid=2320396) 'step_start': 0}, (TaskRunner pid=2320396) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2320396) 'rollout_n': 8, (TaskRunner pid=2320396) 'shuffle': False, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_dynamic_bsz': False}, (TaskRunner pid=2320396) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2320396) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2320396) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2320396) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2320396) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2320396) 'dataloader_num_workers': 8, (TaskRunner pid=2320396) 'filter_overlong_prompts': True, (TaskRunner pid=2320396) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2320396) 'image_key': 'images', (TaskRunner pid=2320396) 'image_patch_size': 14, (TaskRunner pid=2320396) 'max_prompt_length': 2048, (TaskRunner pid=2320396) 'max_response_length': 8192, (TaskRunner pid=2320396) 'prompt_key': 'prompt', (TaskRunner pid=2320396) 'return_full_prompt': False, (TaskRunner pid=2320396) 'return_multi_modal_inputs': True, (TaskRunner pid=2320396) 'return_raw_chat': True, (TaskRunner pid=2320396) 'return_raw_input_ids': False, (TaskRunner pid=2320396) 'reward_fn_key': 'data_source', (TaskRunner pid=2320396) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2320396) 'seed': None, (TaskRunner pid=2320396) 'shuffle': True, (TaskRunner pid=2320396) 'tokenizer': None, (TaskRunner pid=2320396) 'tool_config_path': None, (TaskRunner pid=2320396) 'train_batch_size': 32, (TaskRunner pid=2320396) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2320396) 'train_max_samples': 3200, (TaskRunner pid=2320396) 'truncation': 'error', (TaskRunner pid=2320396) 'trust_remote_code': True, (TaskRunner pid=2320396) 'use_shm': False, (TaskRunner pid=2320396) 'val_batch_size': None, (TaskRunner pid=2320396) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2320396) 'val_max_samples': -1, (TaskRunner pid=2320396) 'validation_shuffle': False, (TaskRunner pid=2320396) 'video_key': 'videos'}, (TaskRunner pid=2320396) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2320396) 'cuda-memory-usage': 'true', (TaskRunner pid=2320396) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2320396) 'capture-range-end': None, (TaskRunner pid=2320396) 'cuda-graph-trace': 'graph', (TaskRunner pid=2320396) 'cuda-memory-usage': 'true', (TaskRunner pid=2320396) 'kill': 'none', (TaskRunner pid=2320396) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2320396) 'torch_memory': {'context': 'all', (TaskRunner pid=2320396) 'kw_args': {}, (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'stacks': 'all', (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2320396) 'profile_continuous_steps': False, (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'steps': None, (TaskRunner pid=2320396) 'tool': None}, (TaskRunner pid=2320396) 'max_model_len': 10240, (TaskRunner pid=2320396) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_8B', (TaskRunner pid=2320396) 'include_dashboard': False, (TaskRunner pid=2320396) 'num_cpus': None, (TaskRunner pid=2320396) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2320396) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2320396) 'USER': 'root'}}}, (TaskRunner pid=2320396) 'timeline_json_file': None}, (TaskRunner pid=2320396) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2320396) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2320396) 'name': 'custom_reward_manager', (TaskRunner pid=2320396) 'path': None}, (TaskRunner pid=2320396) 'name': 'naive', (TaskRunner pid=2320396) 'source': 'register'}, (TaskRunner pid=2320396) 'reward_model': {'enable': False, (TaskRunner pid=2320396) 'enable_resource_pool': False, (TaskRunner pid=2320396) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2320396) 'launch_reward_fn_async': False, (TaskRunner pid=2320396) 'max_length': None, (TaskRunner pid=2320396) 'micro_batch_size': None, (TaskRunner pid=2320396) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2320396) 'model': {'external_lib': None, (TaskRunner pid=2320396) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2320396) 'forward_prefetch': False, (TaskRunner pid=2320396) 'fsdp_size': -1, (TaskRunner pid=2320396) 'param_offload': False, (TaskRunner pid=2320396) 'reshard_after_forward': True, (TaskRunner pid=2320396) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2320396) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2320396) 'override_config': {}, (TaskRunner pid=2320396) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2320396) 'trust_remote_code': False, (TaskRunner pid=2320396) 'use_fused_kernels': False, (TaskRunner pid=2320396) 'use_remove_padding': False, (TaskRunner pid=2320396) 'use_shm': False}, (TaskRunner pid=2320396) 'n_gpus_per_node': 8, (TaskRunner pid=2320396) 'nnodes': 0, (TaskRunner pid=2320396) 'num_workers': 1, (TaskRunner pid=2320396) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2320396) 'all_ranks': False, (TaskRunner pid=2320396) 'enable': False, (TaskRunner pid=2320396) 'ranks': [], (TaskRunner pid=2320396) 'save_path': 'outputs/profile', (TaskRunner pid=2320396) 'tool': None, (TaskRunner pid=2320396) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2320396) 'analysis': True, (TaskRunner pid=2320396) 'contents': [], (TaskRunner pid=2320396) 'discrete': False, (TaskRunner pid=2320396) 'level': 'level0'}, (TaskRunner pid=2320396) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2320396) 'discrete': False}, (TaskRunner pid=2320396) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2320396) 'step_end': None, (TaskRunner pid=2320396) 'step_start': 0}, (TaskRunner pid=2320396) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2320396) 'stack_depth': 32, (TaskRunner pid=2320396) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2320396) 'reward_loop_class_name': None, (TaskRunner pid=2320396) 'reward_loop_module_path': None, (TaskRunner pid=2320396) 'reward_loop_source': 'register', (TaskRunner pid=2320396) 'reward_manager': 'naive', (TaskRunner pid=2320396) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2320396) 'cudagraph_capture_sizes': None, (TaskRunner pid=2320396) 'data_parallel_size': 1, (TaskRunner pid=2320396) 'disable_log_stats': True, (TaskRunner pid=2320396) 'dtype': 'bfloat16', (TaskRunner pid=2320396) 'enable_chunked_prefill': True, (TaskRunner pid=2320396) 'enable_prefix_caching': True, (TaskRunner pid=2320396) 'enforce_eager': True, (TaskRunner pid=2320396) 'engine_kwargs': {}, (TaskRunner pid=2320396) 'expert_parallel_size': 1, (TaskRunner pid=2320396) 'free_cache_engine': True, (TaskRunner pid=2320396) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2320396) 'limit_images': None, (TaskRunner pid=2320396) 'load_format': 'auto', (TaskRunner pid=2320396) 'max_model_len': None, (TaskRunner pid=2320396) 'max_num_batched_tokens': 8192, (TaskRunner pid=2320396) 'max_num_seqs': 1024, (TaskRunner pid=2320396) 'name': '???', (TaskRunner pid=2320396) 'prompt_length': 2048, (TaskRunner pid=2320396) 'response_length': 2048, (TaskRunner pid=2320396) 'skip_tokenizer_init': False, (TaskRunner pid=2320396) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2320396) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2320396) 'memory_limit_mb': 1024, (TaskRunner pid=2320396) 'url': None}, (TaskRunner pid=2320396) 'strategy': 'fsdp', (TaskRunner pid=2320396) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2320396) 'use_dynamic_bsz': False, (TaskRunner pid=2320396) 'use_reward_loop': False}, (TaskRunner pid=2320396) 'trainer': {'balance_batch': True, (TaskRunner pid=2320396) 'critic_warmup': 0, (TaskRunner pid=2320396) 'default_hdfs_dir': None, (TaskRunner pid=2320396) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2320396) 'del_local_ckpt_after_load': False, (TaskRunner pid=2320396) 'device': 'cuda', (TaskRunner pid=2320396) 'esi_redundant_time': 0, (TaskRunner pid=2320396) 'experiment_name': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2320396) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2320396) 'log_val_generations': 0, (TaskRunner pid=2320396) 'logger': ['console', 'wandb'], (TaskRunner pid=2320396) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2320396) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2320396) 'n_gpus_per_node': 8, (TaskRunner pid=2320396) 'nnodes': 1, (TaskRunner pid=2320396) 'project_name': 'SDPO-root', (TaskRunner pid=2320396) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2320396) 'resume_from_path': None, (TaskRunner pid=2320396) 'resume_mode': 'auto', (TaskRunner pid=2320396) 'rollout_data_dir': None, (TaskRunner pid=2320396) 'save_freq': 10, (TaskRunner pid=2320396) 'test_freq': 10, (TaskRunner pid=2320396) 'total_epochs': 30, (TaskRunner pid=2320396) 'total_training_steps': 100, (TaskRunner pid=2320396) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2320396) 'val_before_train': False, (TaskRunner pid=2320396) 'val_only': False, (TaskRunner pid=2320396) 'validation_data_dir': None}, (TaskRunner pid=2320396) 'transfer_queue': {'enable': False}, (TaskRunner pid=2320396) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2320396) 'dir': '/users/root/SDPO', (TaskRunner pid=2320396) 'log_dir': '/users/root/output', (TaskRunner pid=2320396) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2320396) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2320396) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2320396) use_critic=need_critic(config), (TaskRunner pid=2320396) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2320396) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2320396) Using dataset class: RLHFDataset (TaskRunner pid=2320396) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2320396) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2320396) dataset len: 720 (TaskRunner pid=2320396) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2320396) WARNING:2026-07-03 05:49:34,055:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2320396) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2320396) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2320396) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2320396) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2320396) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2320396) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (WorkerDict pid=2320775) [W703 05:49:43.667210084 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:35715 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2320772) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2320775) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2320775) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2320779) [W703 05:49:43.905871872 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:35715 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2320775) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2320775) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2320775) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2320775) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2320772) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2320773) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2320773) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2320774) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.91s/it] [repeated 15x across cluster] (WorkerDict pid=2320774) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.22s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.62s/it] [repeated 7x across cluster] (WorkerDict pid=2320772) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2320772) Actor use_remove_padding=True (WorkerDict pid=2320772) Actor use_fused_kernels=False (WorkerDict pid=2320772) Actor use_prefix_grouper=False (WorkerDict pid=2320774) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2320774) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2320774) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2320774) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2320773) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2320772) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2320772) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2320778) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2320778) warnings.warn( (WorkerDict pid=2320772) reference model: Qwen/Qwen3-8B (WorkerDict pid=2320778) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2320778) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2320772) Model config after override: Qwen3Config { (WorkerDict pid=2320772) "architectures": [ (WorkerDict pid=2320772) "Qwen3ForCausalLM" (WorkerDict pid=2320772) ], (WorkerDict pid=2320772) "attention_bias": false, (WorkerDict pid=2320772) "attention_dropout": 0.0, (WorkerDict pid=2320772) "dtype": "bfloat16", (WorkerDict pid=2320772) "eos_token_id": 151645, (WorkerDict pid=2320772) "head_dim": 128, (WorkerDict pid=2320772) "hidden_act": "silu", (WorkerDict pid=2320772) "hidden_size": 4096, (WorkerDict pid=2320772) "initializer_range": 0.02, (WorkerDict pid=2320772) "intermediate_size": 12288, (WorkerDict pid=2320772) "layer_types": [ (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention", (WorkerDict pid=2320772) "full_attention" (WorkerDict pid=2320772) ], (WorkerDict pid=2320772) "max_position_embeddings": 40960, (WorkerDict pid=2320772) "max_window_layers": 36, (WorkerDict pid=2320772) "model_type": "qwen3", (WorkerDict pid=2320772) "num_attention_heads": 32, (WorkerDict pid=2320772) "num_hidden_layers": 36, (WorkerDict pid=2320772) "num_key_value_heads": 8, (WorkerDict pid=2320772) "pad_token_id": 151643, (WorkerDict pid=2320772) "rms_norm_eps": 1e-06, (WorkerDict pid=2320772) "rope_scaling": null, (WorkerDict pid=2320772) "rope_theta": 1000000, (WorkerDict pid=2320772) "sliding_window": null, (WorkerDict pid=2320772) "tie_word_embeddings": false, (WorkerDict pid=2320772) "transformers_version": "4.57.1", (WorkerDict pid=2320772) "use_cache": true, (WorkerDict pid=2320772) "use_sliding_window": false, (WorkerDict pid=2320772) "vocab_size": 151936 (WorkerDict pid=2320772) } (WorkerDict pid=2320772) (WorkerDict pid=2320776) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (WorkerDict pid=2320774) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2320772) Ref use_remove_padding=True (WorkerDict pid=2320772) Ref use_fused_kernels=False (WorkerDict pid=2320772) Ref use_prefix_grouper=False (WorkerDict pid=2320774) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2320774) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2320396) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2320396) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2320775) Loading checkpoint shards: 80%|████████ | 4/5 [00:11<00:02, 2.86s/it] [repeated 15x across cluster] (WorkerDict pid=2320775) Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00, 2.18s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00, 2.58s/it] [repeated 6x across cluster] (vLLMHttpServer pid=2321709) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2321709) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2321709) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2321709) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2321706) ['serve', (vLLMHttpServer pid=2321706) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2321706) '--dtype', (vLLMHttpServer pid=2321706) 'bfloat16', (vLLMHttpServer pid=2321706) '--load_format', (vLLMHttpServer pid=2321706) 'dummy', (vLLMHttpServer pid=2321706) '--trust_remote_code', (vLLMHttpServer pid=2321706) '--max_model_len', (vLLMHttpServer pid=2321706) '40960', (vLLMHttpServer pid=2321706) '--max_num_seqs', (vLLMHttpServer pid=2321706) '1024', (vLLMHttpServer pid=2321706) '--enable_chunked_prefill', (vLLMHttpServer pid=2321706) '--max_num_batched_tokens', (vLLMHttpServer pid=2321706) '10240', (vLLMHttpServer pid=2321706) '--enable_prefix_caching', (vLLMHttpServer pid=2321706) '--enable_sleep_mode', (vLLMHttpServer pid=2321706) '--logprobs_mode', (vLLMHttpServer pid=2321706) 'processed_logprobs', (vLLMHttpServer pid=2321706) '--disable_custom_all_reduce', (vLLMHttpServer pid=2321706) '--gpu_memory_utilization', (vLLMHttpServer pid=2321706) '0.8', (vLLMHttpServer pid=2321706) '--disable_log_stats', (vLLMHttpServer pid=2321706) '--tensor_parallel_size', (vLLMHttpServer pid=2321706) '2', (vLLMHttpServer pid=2321706) '--seed', (vLLMHttpServer pid=2321706) '0', (vLLMHttpServer pid=2321706) '--override_generation_config', (vLLMHttpServer pid=2321706) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2321706) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2321706) '--hf_overrides', (vLLMHttpServer pid=2321706) '{}', (vLLMHttpServer pid=2321706) '--scheduling_policy', (vLLMHttpServer pid=2321706) 'fcfs'] (vLLMHttpServer pid=2321709) WARNING 07-03 05:50:57 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2320778) WARNING 07-03 05:51:05 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2321708) WARNING 07-03 05:50:57 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2320774) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2320773) [rank1]:W0703 05:51:16.018000 2320773 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/aj/faj75csb7ofozv5in77ssqc762na7cdfwj3wkvfkkmz6awioro7y/.2320778.137164054263488.tmp' (vLLMHttpServer pid=2321708) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2321708) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2321708) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2321708) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2320776) 2026-07-03 05:51:18,431 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2320776) 2026-07-03 05:51:18,445 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2320772) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We are to find the magnitude of the charge $ q $ using the formula: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Rearranging to solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the given values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2320396) step:10 - global_seqlen/min:16491 - global_seqlen/max:30025 - global_seqlen/minmax_diff:13534 - global_seqlen/balanced_min:25337 - global_seqlen/balanced_max:25538 - global_seqlen/mean:25375.0 - actor/entropy:0.31971633434295654 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2820093035697937 - training/rollout_probs_diff_mean:0.0033090896904468536 - training/rollout_probs_diff_std:0.009006068110466003 - training/rollout_actor_probs_pearson_corr:0.9993484020233154 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.95703125 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.95703125 - rollout_corr/rollout_is_mean:0.9999304413795471 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7673910856246948 - rollout_corr/rollout_is_min:0.5399474501609802 - rollout_corr/rollout_is_std:0.02919088862836361 - rollout_corr/rollout_is_eff_sample_size:0.9991483794554346 - rollout_corr/rollout_is_seq_mean:0.9999055862426758 - rollout_corr/rollout_is_seq_std:0.0017869374714791775 - rollout_corr/rollout_is_seq_max:1.0064821243286133 - rollout_corr/rollout_is_seq_min:0.9924333095550537 - rollout_corr/rollout_is_seq_max_deviation:0.007566690444946289 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36662180069834) - rollout_corr/training_log_ppl:np.float64(0.2933628953032894) - rollout_corr/kl:np.float64(0.0005606463353959867) - rollout_corr/k3_kl:np.float64(0.0005264203514343535) - rollout_corr/rollout_ppl:np.float64(1.3658109386451542) - rollout_corr/rollout_log_ppl:np.float64(0.2928022450651042) - rollout_corr/log_ppl_diff:np.float64(0.0005606502381851897) - rollout_corr/log_ppl_abs_diff:np.float64(0.001444778943550773) - rollout_corr/log_ppl_diff_max:np.float64(0.009412601590156555) - rollout_corr/log_ppl_diff_min:np.float64(-0.004387408494949341) - rollout_corr/ppl_ratio:np.float64(1.0005626599304378) - rollout_corr/chi2_token:np.float64(0.0009579714387655258) - rollout_corr/chi2_seq:np.float64(0.6561292943079025) - actor/pg_loss:np.float64(0.0034951333655044436) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002724248687400177) - actor/ppo_kl:np.float64(6.655702073254943e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.95703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.95703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(2924.901734317187) - self_distillation/token_reweight_w_std:np.float64(77176.66680757602) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011562679428607) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08802842449222226) - self_distillation/empty_target_batch:np.float64(0.04296875) - actor/grad_norm:np.float64(0.22159437276422977) - perf/mfu/actor:np.float64(0.058286523178669794) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(191.5031852722168) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.58671875) - val-aux/sciknoweval/reward/std@16:np.float64(0.2080057629527512) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6764374999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1762086184055891) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.500725) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17562203990145614) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5888625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.20769851667696293) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7466875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12697903624119594) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4284625) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12684135690194748) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6092875) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1746588055788758) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7941499999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0796429418879614) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3798125) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07572344702548002) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6249750000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13772658944496724) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8243375000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04642849444514059) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3520625) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03889823328050272) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6363624999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10581089168215327) - val-aux/sciknoweval/score/mean@16:np.float64(0.58671875) - val-aux/sciknoweval/score/std@16:np.float64(0.2080057629527512) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6764374999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1762086184055891) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.500725) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17562203990145614) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5888625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.20769851667696293) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7466875) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12697903624119594) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4284625) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12684135690194748) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6092875) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1746588055788758) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7941499999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0796429418879614) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3798125) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07572344702548002) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6249750000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13772658944496724) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8243375000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04642849444514059) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3520625) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03889823328050272) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6363624999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10581089168215327) - val-core/sciknoweval/acc/mean@16:np.float64(0.58671875) - val-aux/sciknoweval/acc/std@16:np.float64(0.2080057629527512) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6764374999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1762086184055891) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.500725) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17562203990145614) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5888625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.20769851667696293) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7466875) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12697903624119594) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4284625) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12684135690194748) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6092875) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1746588055788758) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7941499999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0796429418879614) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3798125) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07572344702548002) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6249750000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13772658944496724) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8243375000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04642849444514059) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3520625) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03889823328050272) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6363624999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10581089168215327) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98828125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.026208725517679103) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9966750000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.01120071940534392) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9786374999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.032547605155072164) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9869625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.027445106555870208) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9994875000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003073578374100516) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9667125000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03659620343144428) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9929625) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.018372295927089957) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9515499999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.037149139284508935) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9968) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.011264995674402745) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9349999999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03219017290513628) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9991749999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004546032142798204) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016472239047288895 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016472239047288895 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:523.875 - response_length/max:2394.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:523.875 - response_length_non_aborted/max:2394.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015249475836753845 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2428864315152168) - timing_s/agent_loop/generate_sequences/max:np.float64(17.01486107893288) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.637755410964019) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.01486107893288) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:2394 - timing_s/gen:23.984429011121392 - timing_s/reward:0.0665527768433094 - timing_s/old_log_prob:3.143460402265191 - timing_s/adv:0.6190033722668886 - timing_s/update_actor:22.729416826739907 - timing_s/step:50.62902601622045 - timing_s/testing:111.75804004445672 - timing_s/save_checkpoint:12.780955523252487 - timing_s/stop_profile:0.0001423358917236328 - timing_per_token_ms/gen:0.1788387989972664 - timing_per_token_ms/update_actor:0.11196757057507344 - timing_per_token_ms/adv:0.003049277695896003 - perf/total_num_tokens:203000 - perf/time_per_step:50.62902601622045 - perf/throughput:501.1947097672864 (TaskRunner pid=2320396) Training Progress: 10%|█ | 10/100 [11:29<2:16:44, 91.16s/it] (TaskRunner pid=2320396) step:11 - global_seqlen/min:20882 - global_seqlen/max:36506 - global_seqlen/minmax_diff:15624 - global_seqlen/balanced_min:26779 - global_seqlen/balanced_max:26829 - global_seqlen/mean:26804.75 - actor/entropy:0.2590366303920746 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016822338104248 - training/rollout_probs_diff_mean:0.002952962415292859 - training/rollout_probs_diff_std:0.008740789256989956 - training/rollout_actor_probs_pearson_corr:0.9993800520896912 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000083446502686 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.9364482442324515e-06 - rollout_corr/rollout_is_max:1.9402036666870117 - rollout_corr/rollout_is_min:0.47953054308891296 - rollout_corr/rollout_is_std:0.027369895949959755 - rollout_corr/rollout_is_eff_sample_size:0.999251449571954 - rollout_corr/rollout_is_seq_mean:0.9999634027481079 - rollout_corr/rollout_is_seq_std:0.001723091583698988 - rollout_corr/rollout_is_seq_max:1.0086556673049927 - rollout_corr/rollout_is_seq_min:0.9935302734375 - rollout_corr/rollout_is_seq_max_deviation:0.008655667304992676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2878905679099262) - rollout_corr/training_log_ppl:np.float64(0.23974193456524517) - rollout_corr/kl:np.float64(0.0006918362495197672) - rollout_corr/k3_kl:np.float64(0.0005159022044267658) - rollout_corr/rollout_ppl:np.float64(1.287003616336733) - rollout_corr/rollout_log_ppl:np.float64(0.2390500977926422) - rollout_corr/log_ppl_diff:np.float64(0.0006918367726029828) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013698859984287992) - rollout_corr/log_ppl_diff_max:np.float64(0.014145970344543457) - rollout_corr/log_ppl_diff_min:np.float64(-0.007170185446739197) - rollout_corr/ppl_ratio:np.float64(1.0006940583698452) - rollout_corr/chi2_token:np.float64(0.0006612862925976515) - rollout_corr/chi2_seq:np.float64(0.25394133082590997) - actor/pg_loss:np.float64(0.0022024653153494) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030940706426463294) - actor/ppo_kl:np.float64(0.00023035600926712974) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(6060.7103929820005) - self_distillation/token_reweight_w_std:np.float64(178119.12218089768) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995679485145956) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06338222158956341) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.18062633275985718) - perf/mfu/actor:np.float64(0.06158845832096557) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.91942977905273) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010679529048502445 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010679529048502445 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:563.1484375 - response_length/max:2086.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:563.1484375 - response_length_non_aborted/max:2086.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.1552734375e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3518368285149336) - timing_s/agent_loop/generate_sequences/max:np.float64(16.211577968671918) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.0544039794185665) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.211577968671918) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:2063 - timing_s/gen:22.56175279431045 - timing_s/reward:0.06287666037678719 - timing_s/old_log_prob:3.2755117435008287 - timing_s/adv:0.7134620249271393 - timing_s/update_actor:22.722809998318553 - timing_s/step:49.44606627896428 - timing_s/stop_profile:0.00011546537280082703 - timing_per_token_ms/gen:0.1564984309359381 - timing_per_token_ms/update_actor:0.10596447457222392 - timing_per_token_ms/adv:0.0033271249728459473 - perf/total_num_tokens:214438 - perf/time_per_step:49.44606627896428 - perf/throughput:542.1007577988763 (TaskRunner pid=2320396) Training Progress: 11%|█ | 11/100 [12:18<1:56:18, 78.41s/it] (TaskRunner pid=2320396) step:12 - global_seqlen/min:18709 - global_seqlen/max:30849 - global_seqlen/minmax_diff:12140 - global_seqlen/balanced_min:25947 - global_seqlen/balanced_max:25976 - global_seqlen/mean:25967.5 - actor/entropy:0.2872362732887268 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19511616230010986 - training/rollout_probs_diff_mean:0.0031433426775038242 - training/rollout_probs_diff_std:0.008930543437600136 - training/rollout_actor_probs_pearson_corr:0.9993438720703125 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.5703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9999904632568359 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.423082358087413e-05 - rollout_corr/rollout_is_max:1.6279520988464355 - rollout_corr/rollout_is_min:0.480317085981369 - rollout_corr/rollout_is_std:0.028058120980858803 - rollout_corr/rollout_is_eff_sample_size:0.9992131230960879 - rollout_corr/rollout_is_seq_mean:1.0000054836273193 - rollout_corr/rollout_is_seq_std:0.0013679719995707273 - rollout_corr/rollout_is_seq_max:1.0060700178146362 - rollout_corr/rollout_is_seq_min:0.9948147535324097 - rollout_corr/rollout_is_seq_max_deviation:0.0060700178146362305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2835775869898498) - rollout_corr/training_log_ppl:np.float64(0.23835550290823448) - rollout_corr/kl:np.float64(0.00044148623862128034) - rollout_corr/k3_kl:np.float64(0.0004740920864492182) - rollout_corr/rollout_ppl:np.float64(1.2829798916354775) - rollout_corr/rollout_log_ppl:np.float64(0.23791401369089726) - rollout_corr/log_ppl_diff:np.float64(0.0004414892173372209) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011379214993212372) - rollout_corr/log_ppl_diff_max:np.float64(0.006191082298755646) - rollout_corr/log_ppl_diff_min:np.float64(-0.00492994487285614) - rollout_corr/ppl_ratio:np.float64(1.0004427630919963) - rollout_corr/chi2_token:np.float64(0.0010108782444149256) - rollout_corr/chi2_seq:np.float64(0.6784807899966836) - actor/pg_loss:np.float64(0.0032307334477081895) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003089785006977763) - actor/ppo_kl:np.float64(6.130353927158438e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5703125) - self_distillation/token_reweight_w_mean:np.float64(11269.828745884588) - self_distillation/token_reweight_w_std:np.float64(256559.5942551297) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003378372639418) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0678707295737695) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.19498711451888084) - perf/mfu/actor:np.float64(0.05924376023749836) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.71937561035156) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006124590989202261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006124590989202261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:548.984375 - response_length/max:2057.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:548.984375 - response_length_non_aborted/max:2057.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.933005690574646e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3749659731984138) - timing_s/agent_loop/generate_sequences/max:np.float64(15.617194678634405) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.826853218160977) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.617194678634405) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:2057 - timing_s/gen:22.177545238286257 - timing_s/reward:0.06833486817777157 - timing_s/old_log_prob:3.190919019281864 - timing_s/adv:0.6755244601517916 - timing_s/update_actor:22.886284664273262 - timing_s/step:49.08796023763716 - timing_s/stop_profile:0.00012080557644367218 - timing_per_token_ms/gen:0.15780237112769502 - timing_per_token_ms/update_actor:0.11016792463788035 - timing_per_token_ms/adv:0.003251778473822045 - perf/total_num_tokens:207740 - perf/time_per_step:49.08796023763716 - perf/throughput:528.9993691791244 (TaskRunner pid=2320396) Training Progress: 12%|█▏ | 12/100 [13:08<1:41:56, 69.51s/it] (TaskRunner pid=2320396) step:13 - global_seqlen/min:22787 - global_seqlen/max:30986 - global_seqlen/minmax_diff:8199 - global_seqlen/balanced_min:26391 - global_seqlen/balanced_max:26417 - global_seqlen/mean:26409.25 - actor/entropy:0.30309635400772095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41023290157318115 - training/rollout_probs_diff_mean:0.0032512678299099207 - training/rollout_probs_diff_std:0.00912945345044136 - training/rollout_actor_probs_pearson_corr:0.9993520975112915 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8828125 - self_distillation/correct_sample_fraction:0.55078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8828125 - rollout_corr/rollout_is_mean:0.9999639391899109 - rollout_corr/rollout_is_ratio_fraction_high:7.1484737418359146e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.1484737418359146e-06 - rollout_corr/rollout_is_max:4.591783046722412 - rollout_corr/rollout_is_min:0.43883800506591797 - rollout_corr/rollout_is_std:0.028721565380692482 - rollout_corr/rollout_is_eff_sample_size:0.9991755136364232 - rollout_corr/rollout_is_seq_mean:0.9999345541000366 - rollout_corr/rollout_is_seq_std:0.001229618676006794 - rollout_corr/rollout_is_seq_max:1.0057193040847778 - rollout_corr/rollout_is_seq_min:0.9962084889411926 - rollout_corr/rollout_is_seq_max_deviation:0.005719304084777832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3604151918552816) - rollout_corr/training_log_ppl:np.float64(0.28699639250407927) - rollout_corr/kl:np.float64(0.000334256598858218) - rollout_corr/k3_kl:np.float64(0.0004752778168608529) - rollout_corr/rollout_ppl:np.float64(1.3599663516506553) - rollout_corr/rollout_log_ppl:np.float64(0.2866621360881254) - rollout_corr/log_ppl_diff:np.float64(0.0003342564159538597) - rollout_corr/log_ppl_abs_diff:np.float64(0.001109582808567211) - rollout_corr/log_ppl_diff_max:np.float64(0.004641130566596985) - rollout_corr/log_ppl_diff_min:np.float64(-0.003542080521583557) - rollout_corr/ppl_ratio:np.float64(1.0003352970816195) - rollout_corr/chi2_token:np.float64(0.0012653181329369545) - rollout_corr/chi2_seq:np.float64(0.7056709041353315) - actor/pg_loss:np.float64(0.003511187736876309) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003105056030108244) - actor/ppo_kl:np.float64(-0.0001611898588009808) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55078125) - self_distillation/token_reweight_w_mean:np.float64(4473.108445552643) - self_distillation/token_reweight_w_std:np.float64(105375.92507209204) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019248176831752) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10184741955890786) - self_distillation/empty_target_batch:np.float64(0.1171875) - actor/grad_norm:np.float64(0.26514898985624313) - perf/mfu/actor:np.float64(0.061133796091133896) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.83147811889648) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009976589120924473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009976589120924473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:546.4453125 - response_length/max:1478.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:546.4453125 - response_length_non_aborted/max:1478.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017240829765796661 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4114775881171227) - timing_s/agent_loop/generate_sequences/max:np.float64(12.42954215966165) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.215870589490805) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.42954215966165) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:363 - timing_s/agent_loop/slowest/response_length:1478 - timing_s/gen:19.216635648161173 - timing_s/reward:0.06841671280562878 - timing_s/old_log_prob:3.1306111067533493 - timing_s/adv:0.6264998782426119 - timing_s/update_actor:22.610267801210284 - timing_s/step:45.73972590826452 - timing_s/stop_profile:0.00011229142546653748 - timing_per_token_ms/gen:0.13736961647123577 - timing_per_token_ms/update_actor:0.1070186951598885 - timing_per_token_ms/adv:0.0029653430059667155 - perf/total_num_tokens:211274 - perf/time_per_step:45.73972590826452 - perf/throughput:577.3810287575034 (TaskRunner pid=2320396) Training Progress: 13%|█▎ | 13/100 [13:53<1:30:22, 62.33s/it] (TaskRunner pid=2320396) step:14 - global_seqlen/min:17015 - global_seqlen/max:32433 - global_seqlen/minmax_diff:15418 - global_seqlen/balanced_min:25084 - global_seqlen/balanced_max:25111 - global_seqlen/mean:25101.875 - actor/entropy:0.2703498601913452 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4553069472312927 - training/rollout_probs_diff_mean:0.003260517492890358 - training/rollout_probs_diff_std:0.00941763911396265 - training/rollout_actor_probs_pearson_corr:0.9992024898529053 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000470876693726 - rollout_corr/rollout_is_ratio_fraction_high:7.63831030781148e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.527662061562296e-05 - rollout_corr/rollout_is_max:2.148858070373535 - rollout_corr/rollout_is_min:0.38240405917167664 - rollout_corr/rollout_is_std:0.029475368559360504 - rollout_corr/rollout_is_eff_sample_size:0.9991319568512997 - rollout_corr/rollout_is_seq_mean:1.0000320672988892 - rollout_corr/rollout_is_seq_std:0.0016445777146145701 - rollout_corr/rollout_is_seq_max:1.0072376728057861 - rollout_corr/rollout_is_seq_min:0.9936115741729736 - rollout_corr/rollout_is_seq_max_deviation:0.007237672805786133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3087067985907197) - rollout_corr/training_log_ppl:np.float64(0.2531808679632377) - rollout_corr/kl:np.float64(0.0006406861673453612) - rollout_corr/k3_kl:np.float64(0.0005106621201207417) - rollout_corr/rollout_ppl:np.float64(1.3078300086781383) - rollout_corr/rollout_log_ppl:np.float64(0.2525401824968867) - rollout_corr/log_ppl_diff:np.float64(0.0006406854663509876) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012924426409881562) - rollout_corr/log_ppl_diff_max:np.float64(0.007752135396003723) - rollout_corr/log_ppl_diff_min:np.float64(-0.005377739667892456) - rollout_corr/ppl_ratio:np.float64(1.0006423369050026) - rollout_corr/chi2_token:np.float64(0.0007580590900033712) - rollout_corr/chi2_seq:np.float64(0.2396695853676647) - actor/pg_loss:np.float64(0.004213686450384557) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00043405349424574524) - actor/ppo_kl:np.float64(0.0002313976682248864) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(1120.801469426835) - self_distillation/token_reweight_w_std:np.float64(23381.567357088614) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000115639762953) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08497079348308034) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.2399461343884468) - perf/mfu/actor:np.float64(0.05799008474978051) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.85115432739258) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.018892407417297363 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.018892407417297363 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:511.40234375 - response_length/max:1582.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:511.40234375 - response_length_non_aborted/max:1582.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.643604815006256e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0278006736189127) - timing_s/agent_loop/generate_sequences/max:np.float64(12.826374592259526) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.649861370380677) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.826374592259526) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:249 - timing_s/agent_loop/slowest/response_length:1582 - timing_s/gen:20.82091149315238 - timing_s/reward:0.06341278553009033 - timing_s/old_log_prob:3.1353542301803827 - timing_s/adv:0.6522118151187897 - timing_s/update_actor:22.489721328020096 - timing_s/step:47.25435267388821 - timing_s/stop_profile:0.00012648850679397583 - timing_per_token_ms/gen:0.15903659127515776 - timing_per_token_ms/update_actor:0.11199223826915368 - timing_per_token_ms/adv:0.0032478241920115016 - perf/total_num_tokens:200815 - perf/time_per_step:47.25435267388821 - perf/throughput:531.207678861524 (TaskRunner pid=2320396) Training Progress: 14%|█▍ | 14/100 [14:41<1:22:50, 57.80s/it] (TaskRunner pid=2320396) step:15 - global_seqlen/min:15276 - global_seqlen/max:29060 - global_seqlen/minmax_diff:13784 - global_seqlen/balanced_min:23721 - global_seqlen/balanced_max:23925 - global_seqlen/mean:23760.875 - actor/entropy:0.24135680496692657 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35413694381713867 - training/rollout_probs_diff_mean:0.002892824588343501 - training/rollout_probs_diff_std:0.009023798629641533 - training/rollout_actor_probs_pearson_corr:0.999297022819519 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999739527702332 - rollout_corr/rollout_is_ratio_fraction_high:8.203243851312436e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6406487702624872e-05 - rollout_corr/rollout_is_max:2.057389497756958 - rollout_corr/rollout_is_min:0.30289867520332336 - rollout_corr/rollout_is_std:0.02816413715481758 - rollout_corr/rollout_is_eff_sample_size:0.9992072910640203 - rollout_corr/rollout_is_seq_mean:1.0000672340393066 - rollout_corr/rollout_is_seq_std:0.0018630841514095664 - rollout_corr/rollout_is_seq_max:1.0082615613937378 - rollout_corr/rollout_is_seq_min:0.9932234883308411 - rollout_corr/rollout_is_seq_max_deviation:0.008261561393737793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2723506367765367) - rollout_corr/training_log_ppl:np.float64(0.2237588390125893) - rollout_corr/kl:np.float64(0.00038307747586419083) - rollout_corr/k3_kl:np.float64(0.00043506306940344075) - rollout_corr/rollout_ppl:np.float64(1.271862001158297) - rollout_corr/rollout_log_ppl:np.float64(0.22337575981509872) - rollout_corr/log_ppl_diff:np.float64(0.00038307919749058783) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012621376663446426) - rollout_corr/log_ppl_diff_max:np.float64(0.007576450705528259) - rollout_corr/log_ppl_diff_min:np.float64(-0.006189495325088501) - rollout_corr/ppl_ratio:np.float64(1.00038459058851) - rollout_corr/chi2_token:np.float64(0.0009704220574349165) - rollout_corr/chi2_seq:np.float64(0.41881759371608496) - actor/pg_loss:np.float64(0.0019638556987047195) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002059341913991375) - actor/ppo_kl:np.float64(2.1355168600578622e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(14600.623928362038) - self_distillation/token_reweight_w_std:np.float64(243156.56867134484) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005333223380148) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05767766255303286) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.17628217674791813) - perf/mfu/actor:np.float64(0.05489578998408176) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.80868911743164) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008619558066129684 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008619558066129684 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:476.18359375 - response_length/max:1702.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:476.18359375 - response_length_non_aborted/max:1702.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.830878555774689e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8086316455155611) - timing_s/agent_loop/generate_sequences/max:np.float64(13.160807492211461) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.187484379501257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.160807492211461) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1702 - timing_s/gen:20.94913785532117 - timing_s/reward:0.06456178799271584 - timing_s/old_log_prob:3.1010314859449863 - timing_s/adv:0.6535162795335054 - timing_s/update_actor:22.56844968162477 - timing_s/step:47.42496529221535 - timing_s/stop_profile:0.0001125577837228775 - timing_per_token_ms/gen:0.1718508802516851 - timing_per_token_ms/update_actor:0.11872694966844008 - timing_per_token_ms/adv:0.003437985130669143 - perf/total_num_tokens:190087 - perf/time_per_step:47.42496529221535 - perf/throughput:501.02039829853646 (TaskRunner pid=2320396) Training Progress: 15%|█▌ | 15/100 [15:28<1:17:28, 54.68s/it] (TaskRunner pid=2320396) step:16 - global_seqlen/min:18153 - global_seqlen/max:26318 - global_seqlen/minmax_diff:8165 - global_seqlen/balanced_min:22294 - global_seqlen/balanced_max:22350 - global_seqlen/mean:22311.75 - actor/entropy:0.2570214569568634 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34247446060180664 - training/rollout_probs_diff_mean:0.003073042258620262 - training/rollout_probs_diff_std:0.00942500215023756 - training/rollout_actor_probs_pearson_corr:0.9992203712463379 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.9609375 - self_distillation/correct_sample_fraction:0.76953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9609375 - rollout_corr/rollout_is_mean:1.0000674724578857 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6776629965752363e-05 - rollout_corr/rollout_is_max:1.8510113954544067 - rollout_corr/rollout_is_min:0.30873462557792664 - rollout_corr/rollout_is_std:0.029871081933379173 - rollout_corr/rollout_is_eff_sample_size:0.9991086329211257 - rollout_corr/rollout_is_seq_mean:1.0001213550567627 - rollout_corr/rollout_is_seq_std:0.0016576494090259075 - rollout_corr/rollout_is_seq_max:1.0090272426605225 - rollout_corr/rollout_is_seq_min:0.9953111410140991 - rollout_corr/rollout_is_seq_max_deviation:0.009027242660522461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2797890407964587) - rollout_corr/training_log_ppl:np.float64(0.23128712813195307) - rollout_corr/kl:np.float64(0.00038973505982442536) - rollout_corr/k3_kl:np.float64(0.000491941271206997) - rollout_corr/rollout_ppl:np.float64(1.2792712301015854) - rollout_corr/rollout_log_ppl:np.float64(0.23089739288843703) - rollout_corr/log_ppl_diff:np.float64(0.00038973524351604283) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012824619188904762) - rollout_corr/log_ppl_diff_max:np.float64(0.005027711391448975) - rollout_corr/log_ppl_diff_min:np.float64(-0.007322400808334351) - rollout_corr/ppl_ratio:np.float64(1.0003912099637091) - rollout_corr/chi2_token:np.float64(0.0011921904515475035) - rollout_corr/chi2_seq:np.float64(0.29298205487430096) - actor/pg_loss:np.float64(0.0026901537785306573) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002558936457717209) - actor/ppo_kl:np.float64(4.6419355181015476e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_w_mean:np.float64(25693.85642775311) - self_distillation/token_reweight_w_std:np.float64(431982.68447569985) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010720288846642) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060745136928744614) - self_distillation/empty_target_batch:np.float64(0.0390625) - actor/grad_norm:np.float64(0.23966870829463005) - perf/mfu/actor:np.float64(0.05296940936700055) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.85827255249023) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012607329990714788 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012607329990714788 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:437.6484375 - response_length/max:1591.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.6484375 - response_length_non_aborted/max:1591.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013866089284420013 - timing_s/agent_loop/generate_sequences/min:np.float64(1.381975892931223) - timing_s/agent_loop/generate_sequences/max:np.float64(11.990967459976673) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.791547902925231) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.990967459976673) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:295 - timing_s/agent_loop/slowest/response_length:1591 - timing_s/gen:18.93207285180688 - timing_s/reward:0.05992904677987099 - timing_s/old_log_prob:3.1903146971017122 - timing_s/adv:0.584565894678235 - timing_s/update_actor:21.913405856117606 - timing_s/step:44.769447296857834 - timing_s/stop_profile:8.477084338665009e-05 - timing_per_token_ms/gen:0.16897903257650868 - timing_per_token_ms/update_actor:0.12276830513136355 - timing_per_token_ms/adv:0.0032749890454482227 - perf/total_num_tokens:178494 - perf/time_per_step:44.769447296857834 - perf/throughput:498.370012300017 (TaskRunner pid=2320396) Training Progress: 16%|█▌ | 16/100 [16:13<1:12:23, 51.71s/it] (TaskRunner pid=2320396) step:17 - global_seqlen/min:17426 - global_seqlen/max:28967 - global_seqlen/minmax_diff:11541 - global_seqlen/balanced_min:22692 - global_seqlen/balanced_max:22697 - global_seqlen/mean:22695.125 - actor/entropy:0.29208648204803467 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.279821515083313 - training/rollout_probs_diff_mean:0.003486455651000142 - training/rollout_probs_diff_std:0.00968213938176632 - training/rollout_actor_probs_pearson_corr:0.9992047548294067 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999521374702454 - rollout_corr/rollout_is_ratio_fraction_high:8.820131370157469e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.5134615898132324 - rollout_corr/rollout_is_min:0.5504810214042664 - rollout_corr/rollout_is_std:0.03140411525964737 - rollout_corr/rollout_is_eff_sample_size:0.999014634241615 - rollout_corr/rollout_is_seq_mean:1.0000044107437134 - rollout_corr/rollout_is_seq_std:0.0021442328579723835 - rollout_corr/rollout_is_seq_max:1.0077778100967407 - rollout_corr/rollout_is_seq_min:0.9927151203155518 - rollout_corr/rollout_is_seq_max_deviation:0.007777810096740723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3378607337363064) - rollout_corr/training_log_ppl:np.float64(0.27729781922244) - rollout_corr/kl:np.float64(0.0006814117384088547) - rollout_corr/k3_kl:np.float64(0.000675479077244745) - rollout_corr/rollout_ppl:np.float64(1.336908448021859) - rollout_corr/rollout_log_ppl:np.float64(0.27661640573933255) - rollout_corr/log_ppl_diff:np.float64(0.0006814134831074625) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016361892339773476) - rollout_corr/log_ppl_diff_max:np.float64(0.01305815577507019) - rollout_corr/log_ppl_diff_min:np.float64(-0.006799712777137756) - rollout_corr/ppl_ratio:np.float64(1.0006845023017377) - rollout_corr/chi2_token:np.float64(0.0012903332244604826) - rollout_corr/chi2_seq:np.float64(0.641126612201333) - actor/pg_loss:np.float64(0.0035220705904066563) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00032299184999828867) - actor/ppo_kl:np.float64(0.00013300533843363382) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(1380.9805241154972) - self_distillation/token_reweight_w_std:np.float64(30178.533250448236) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999288574326783) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08566995974979363) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.26250072568655014) - perf/mfu/actor:np.float64(0.05411806700287069) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.87701416015625) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021399843972176313 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021399843972176313 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:442.87890625 - response_length/max:1282.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:442.87890625 - response_length_non_aborted/max:1282.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.532218635082245e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.001680351793766) - timing_s/agent_loop/generate_sequences/max:np.float64(10.440772091969848) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.841584001922456) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.440772091969848) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:1282 - timing_s/gen:17.069299457594752 - timing_s/reward:0.06137548200786114 - timing_s/old_log_prob:3.0410669166594744 - timing_s/adv:0.6068573668599129 - timing_s/update_actor:21.700714791193604 - timing_s/step:42.56168849952519 - timing_s/stop_profile:0.00012058578431606293 - timing_per_token_ms/gen:0.15055345844037815 - timing_per_token_ms/update_actor:0.11952299663029838 - timing_per_token_ms/adv:0.003342443403924372 - perf/total_num_tokens:181561 - perf/time_per_step:42.56168849952519 - perf/throughput:533.228962480029 (TaskRunner pid=2320396) Training Progress: 17%|█▋ | 17/100 [16:56<1:07:44, 48.97s/it] (TaskRunner pid=2320396) step:18 - global_seqlen/min:18525 - global_seqlen/max:28543 - global_seqlen/minmax_diff:10018 - global_seqlen/balanced_min:24608 - global_seqlen/balanced_max:24629 - global_seqlen/mean:24623.75 - actor/entropy:0.291051983833313 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846030056476593 - training/rollout_probs_diff_mean:0.003182345302775502 - training/rollout_probs_diff_std:0.009170476347208023 - training/rollout_actor_probs_pearson_corr:0.9992757439613342 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:0.9999815821647644 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.633814811706543 - rollout_corr/rollout_is_min:0.6136082410812378 - rollout_corr/rollout_is_std:0.02904965728521347 - rollout_corr/rollout_is_eff_sample_size:0.9991567099682599 - rollout_corr/rollout_is_seq_mean:1.0001029968261719 - rollout_corr/rollout_is_seq_std:0.0016350590158253908 - rollout_corr/rollout_is_seq_max:1.0074173212051392 - rollout_corr/rollout_is_seq_min:0.9932276010513306 - rollout_corr/rollout_is_seq_max_deviation:0.00741732120513916 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3274645488709211) - rollout_corr/training_log_ppl:np.float64(0.2682839465123834) - rollout_corr/kl:np.float64(0.0003594352448370586) - rollout_corr/k3_kl:np.float64(0.00047769542220521544) - rollout_corr/rollout_ppl:np.float64(1.3270041062496603) - rollout_corr/rollout_log_ppl:np.float64(0.26792451113578863) - rollout_corr/log_ppl_diff:np.float64(0.0003594353765947744) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012344932329142466) - rollout_corr/log_ppl_diff_max:np.float64(0.008019745349884033) - rollout_corr/log_ppl_diff_min:np.float64(-0.0066003501415252686) - rollout_corr/ppl_ratio:np.float64(1.0003609699197114) - rollout_corr/chi2_token:np.float64(0.0011884842533618212) - rollout_corr/chi2_seq:np.float64(0.5264976168982685) - actor/pg_loss:np.float64(0.003481487277895212) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036702533452626085) - actor/ppo_kl:np.float64(1.649830553063225e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(23783.103630624246) - self_distillation/token_reweight_w_std:np.float64(391530.74363069737) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014330511912704) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09469849267043173) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.24629244580864906) - perf/mfu/actor:np.float64(0.057109274387849074) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.89093780517578) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014965659007430077 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014965659007430077 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:505.6171875 - response_length/max:1449.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:505.6171875 - response_length_non_aborted/max:1449.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002104882150888443 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9661016371101141) - timing_s/agent_loop/generate_sequences/max:np.float64(12.230161782354116) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.689887342115981) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.230161782354116) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:246 - timing_s/agent_loop/slowest/response_length:1449 - timing_s/gen:19.692222712561488 - timing_s/reward:0.06511449627578259 - timing_s/old_log_prob:3.095972517505288 - timing_s/adv:0.610324090346694 - timing_s/update_actor:22.506673527881503 - timing_s/step:46.059743424877524 - timing_s/stop_profile:0.00011930987238883972 - timing_per_token_ms/gen:0.15213633332221982 - timing_per_token_ms/update_actor:0.11425287338383422 - timing_per_token_ms/adv:0.00309824910069899 - perf/total_num_tokens:196990 - perf/time_per_step:46.059743424877524 - perf/throughput:534.6045845904639 (TaskRunner pid=2320396) Training Progress: 18%|█▊ | 18/100 [17:42<1:05:45, 48.11s/it] (TaskRunner pid=2320396) step:19 - global_seqlen/min:18204 - global_seqlen/max:34489 - global_seqlen/minmax_diff:16285 - global_seqlen/balanced_min:24308 - global_seqlen/balanced_max:24325 - global_seqlen/mean:24319.5 - actor/entropy:0.22665835916996002 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28219449520111084 - training/rollout_probs_diff_mean:0.002949574962258339 - training/rollout_probs_diff_std:0.009281001053750515 - training/rollout_actor_probs_pearson_corr:0.9991229772567749 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.999921977519989 - rollout_corr/rollout_is_ratio_fraction_high:1.5755475033074617e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5755475033074617e-05 - rollout_corr/rollout_is_max:2.033278226852417 - rollout_corr/rollout_is_min:0.34410393238067627 - rollout_corr/rollout_is_std:0.028740236535668373 - rollout_corr/rollout_is_eff_sample_size:0.9991745615349487 - rollout_corr/rollout_is_seq_mean:0.999957263469696 - rollout_corr/rollout_is_seq_std:0.0018070649821311235 - rollout_corr/rollout_is_seq_max:1.008998155593872 - rollout_corr/rollout_is_seq_min:0.9937487840652466 - rollout_corr/rollout_is_seq_max_deviation:0.00899815559387207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.236382881179452) - rollout_corr/training_log_ppl:np.float64(0.20459451123315375) - rollout_corr/kl:np.float64(0.00044612464541060604) - rollout_corr/k3_kl:np.float64(0.00047015344770784395) - rollout_corr/rollout_ppl:np.float64(1.235796622466296) - rollout_corr/rollout_log_ppl:np.float64(0.20414838583383244) - rollout_corr/log_ppl_diff:np.float64(0.00044612539932131767) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013511724537238479) - rollout_corr/log_ppl_diff_max:np.float64(0.00844089686870575) - rollout_corr/log_ppl_diff_min:np.float64(-0.005872577428817749) - rollout_corr/ppl_ratio:np.float64(1.000447863014415) - rollout_corr/chi2_token:np.float64(0.000981275923550129) - rollout_corr/chi2_seq:np.float64(0.4389119232073426) - actor/pg_loss:np.float64(0.0033063190057873726) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039961243192010443) - actor/ppo_kl:np.float64(-5.261532975175953e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(9821.60917656147) - self_distillation/token_reweight_w_std:np.float64(211790.16289496643) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992890893481672) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07317971339216456) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.2147277519106865) - perf/mfu/actor:np.float64(0.05726284160695215) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.81607341766357) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015608752146363258 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015608752146363258 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:495.859375 - response_length/max:1619.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:495.859375 - response_length_non_aborted/max:1619.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011139735579490662 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4400303233414888) - timing_s/agent_loop/generate_sequences/max:np.float64(12.841846983879805) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.404720055048529) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.841846983879805) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1619 - timing_s/gen:19.706364968791604 - timing_s/reward:0.0602475106716156 - timing_s/old_log_prob:3.1028328090906143 - timing_s/adv:0.6127728633582592 - timing_s/update_actor:21.8852920550853 - timing_s/step:45.466720102354884 - timing_s/stop_profile:7.124058902263641e-05 - timing_per_token_ms/gen:0.1552415705750087 - timing_per_token_ms/update_actor:0.11248839437018289 - timing_per_token_ms/adv:0.0031495963288629453 - perf/total_num_tokens:194556 - perf/time_per_step:45.466720102354884 - perf/throughput:534.8857350002778 (TaskRunner pid=2320396) Training Progress: 19%|█▉ | 19/100 [18:27<1:03:53, 47.33s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2320396) (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The electric field due to a point charge is given by: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Solving for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substitute the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \times (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest answer is **B: 56 pC**. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2320396) step:20 - global_seqlen/min:17811 - global_seqlen/max:27757 - global_seqlen/minmax_diff:9946 - global_seqlen/balanced_min:22789 - global_seqlen/balanced_max:22795 - global_seqlen/mean:22792.875 - actor/entropy:0.25308850407600403 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31527847051620483 - training/rollout_probs_diff_mean:0.0031136958859860897 - training/rollout_probs_diff_std:0.009490901604294777 - training/rollout_actor_probs_pearson_corr:0.9991628527641296 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000277757644653 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6279141902923584 - rollout_corr/rollout_is_min:0.5579100847244263 - rollout_corr/rollout_is_std:0.028953060507774353 - rollout_corr/rollout_is_eff_sample_size:0.9991623033905332 - rollout_corr/rollout_is_seq_mean:1.000030755996704 - rollout_corr/rollout_is_seq_std:0.001790462527424097 - rollout_corr/rollout_is_seq_max:1.0065312385559082 - rollout_corr/rollout_is_seq_min:0.9912646412849426 - rollout_corr/rollout_is_seq_max_deviation:0.008735358715057373 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2623859508894384) - rollout_corr/training_log_ppl:np.float64(0.22125808593409602) - rollout_corr/kl:np.float64(0.0004030627003750453) - rollout_corr/k3_kl:np.float64(0.000487179084075251) - rollout_corr/rollout_ppl:np.float64(1.2618781537748873) - rollout_corr/rollout_log_ppl:np.float64(0.22085502157278825) - rollout_corr/log_ppl_diff:np.float64(0.00040306436130777) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013739361602347344) - rollout_corr/log_ppl_diff_max:np.float64(0.009082533419132233) - rollout_corr/log_ppl_diff_min:np.float64(-0.005283474922180176) - rollout_corr/ppl_ratio:np.float64(1.0004048922564834) - rollout_corr/chi2_token:np.float64(0.0011564446613192558) - rollout_corr/chi2_seq:np.float64(1.532239300897345) - actor/pg_loss:np.float64(0.003527254913933575) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003930750553990947) - actor/ppo_kl:np.float64(-8.912188441456692e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(4924.831674827263) - self_distillation/token_reweight_w_std:np.float64(105272.75272310019) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003256590571254) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07200192401069216) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.2361677922308445) - perf/mfu/actor:np.float64(0.0536922197326441) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(195.8451690673828) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6171875) - val-aux/sciknoweval/reward/std@16:np.float64(0.202710008221792) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7003625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17668100954882657) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5318375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16688353844816986) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6167999999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.20131107144087576) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7715250000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13401208491393962) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4669) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11627459214311622) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6370125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1686258674491296) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.82355) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08975117780023725) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4231249999999999) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07250128837645714) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6521874999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13003496063039865) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8590249999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05442224021498587) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.39719999999999994) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.041142257051020134) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6623249999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09169010950465917) - val-aux/sciknoweval/score/mean@16:np.float64(0.6171875) - val-aux/sciknoweval/score/std@16:np.float64(0.202710008221792) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7003625) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17668100954882657) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5318375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16688353844816986) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6167999999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.20131107144087576) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7715250000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13401208491393962) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4669) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11627459214311622) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6370125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1686258674491296) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.82355) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08975117780023725) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4231249999999999) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07250128837645714) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6521874999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13003496063039865) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8590249999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05442224021498587) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.39719999999999994) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.041142257051020134) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6623249999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09169010950465917) - val-core/sciknoweval/acc/mean@16:np.float64(0.6171875) - val-aux/sciknoweval/acc/std@16:np.float64(0.202710008221792) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7003625) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17668100954882657) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5318375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16688353844816986) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6167999999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.20131107144087576) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7715250000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13401208491393962) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4669) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11627459214311622) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6370125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1686258674491296) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.82355) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08975117780023725) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4231249999999999) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07250128837645714) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6521874999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13003496063039865) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8590249999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05442224021498587) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.39719999999999994) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.041142257051020134) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6623249999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09169010950465917) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99296875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.012252516113756723) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9973749999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006596626091455685) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.988175) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.014320320245906525) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.993) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0124317209813641) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9995625) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0022972469936861388) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9827) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01450548118607801) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9957874999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.009331159818661627) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9774375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.01359035860325003) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9982875) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.006036525988007402) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.971575) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.012021621146916916) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9996500000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002062159062730128) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003338916227221489 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003338916227221489 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:452.46484375 - response_length/max:1161.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:452.46484375 - response_length_non_aborted/max:1161.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.982738316059113e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4439123030751944) - timing_s/agent_loop/generate_sequences/max:np.float64(9.813925538212061) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.9232439590850845) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.813925538212061) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:244 - timing_s/agent_loop/slowest/response_length:1151 - timing_s/gen:16.630947606638074 - timing_s/reward:0.06120440363883972 - timing_s/old_log_prob:3.0552370827645063 - timing_s/adv:0.5773577280342579 - timing_s/update_actor:22.215389225631952 - timing_s/step:42.61977862007916 - timing_s/testing:104.57180244289339 - timing_s/save_checkpoint:12.73552224598825 - timing_s/stop_profile:0.00013989396393299103 - timing_per_token_ms/gen:0.14357941834774865 - timing_per_token_ms/update_actor:0.12183296987343606 - timing_per_token_ms/adv:0.0031663278987087953 - perf/total_num_tokens:182343 - perf/time_per_step:42.61977862007916 - perf/throughput:534.7957154630022 (TaskRunner pid=2320396) Training Progress: 20%|██ | 20/100 [21:07<1:48:12, 81.16s/it] (TaskRunner pid=2320396) step:21 - global_seqlen/min:21441 - global_seqlen/max:27978 - global_seqlen/minmax_diff:6537 - global_seqlen/balanced_min:23375 - global_seqlen/balanced_max:23403 - global_seqlen/mean:23390.75 - actor/entropy:0.21263273060321808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4400324523448944 - training/rollout_probs_diff_mean:0.002692604437470436 - training/rollout_probs_diff_std:0.009109113365411758 - training/rollout_actor_probs_pearson_corr:0.9992377758026123 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9999504089355469 - rollout_corr/rollout_is_ratio_fraction_high:1.6875073924893513e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.375014784978703e-05 - rollout_corr/rollout_is_max:2.1655259132385254 - rollout_corr/rollout_is_min:0.2829708158969879 - rollout_corr/rollout_is_std:0.0275435633957386 - rollout_corr/rollout_is_eff_sample_size:0.9992418081630519 - rollout_corr/rollout_is_seq_mean:0.9999712109565735 - rollout_corr/rollout_is_seq_std:0.0014922807458788157 - rollout_corr/rollout_is_seq_max:1.0057662725448608 - rollout_corr/rollout_is_seq_min:0.9951524138450623 - rollout_corr/rollout_is_seq_max_deviation:0.00576627254486084 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2243655044585466) - rollout_corr/training_log_ppl:np.float64(0.191109212799347) - rollout_corr/kl:np.float64(0.00025866517576123016) - rollout_corr/k3_kl:np.float64(0.00041524350728394666) - rollout_corr/rollout_ppl:np.float64(1.2240168727003038) - rollout_corr/rollout_log_ppl:np.float64(0.19085054672905244) - rollout_corr/log_ppl_diff:np.float64(0.00025866607029456645) - rollout_corr/log_ppl_abs_diff:np.float64(0.001202312865643762) - rollout_corr/log_ppl_diff_max:np.float64(0.005112379789352417) - rollout_corr/log_ppl_diff_min:np.float64(-0.00403188169002533) - rollout_corr/ppl_ratio:np.float64(1.0002599137369543) - rollout_corr/chi2_token:np.float64(0.0011958186514675617) - rollout_corr/chi2_seq:np.float64(0.7327827827539295) - actor/pg_loss:np.float64(0.0018519839504733682) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023439306892214518) - actor/ppo_kl:np.float64(-0.00015845946509696063) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(232.10363201983273) - self_distillation/token_reweight_w_std:np.float64(5613.1256408341) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000109274405986) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04587709561747033) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.1534450314939022) - perf/mfu/actor:np.float64(0.05609792420550409) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(196.3541603088379) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004018377047032118 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004018377047032118 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:462.9609375 - response_length/max:1575.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:462.9609375 - response_length_non_aborted/max:1575.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001241639256477356 - timing_s/agent_loop/generate_sequences/min:np.float64(2.064604979008436) - timing_s/agent_loop/generate_sequences/max:np.float64(12.253852516412735) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.151852088274609) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.253852516412735) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1575 - timing_s/gen:18.77224212884903 - timing_s/reward:0.05801803804934025 - timing_s/old_log_prob:3.214152639731765 - timing_s/adv:0.6185661982744932 - timing_s/update_actor:21.78351651877165 - timing_s/step:44.54678510688245 - timing_s/stop_profile:0.00012024678289890289 - timing_per_token_ms/gen:0.15839148592491462 - timing_per_token_ms/update_actor:0.11641095581999107 - timing_per_token_ms/adv:0.0033056133208345887 - perf/total_num_tokens:187126 - perf/time_per_step:44.54678510688245 - perf/throughput:525.0827852981504 (TaskRunner pid=2320396) Training Progress: 21%|██ | 21/100 [21:52<1:32:24, 70.19s/it] (TaskRunner pid=2320396) step:22 - global_seqlen/min:16968 - global_seqlen/max:30809 - global_seqlen/minmax_diff:13841 - global_seqlen/balanced_min:23449 - global_seqlen/balanced_max:23463 - global_seqlen/mean:23457.625 - actor/entropy:0.22844623029232025 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.768605649471283 - training/rollout_probs_diff_mean:0.0029645978938788176 - training/rollout_probs_diff_std:0.009484783746302128 - training/rollout_actor_probs_pearson_corr:0.9990792870521545 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:0.9999969601631165 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4725342882447876e-05 - rollout_corr/rollout_is_max:1.8854188919067383 - rollout_corr/rollout_is_min:0.12654437124729156 - rollout_corr/rollout_is_std:0.02832241728901863 - rollout_corr/rollout_is_eff_sample_size:0.9991984836344271 - rollout_corr/rollout_is_seq_mean:1.0000041723251343 - rollout_corr/rollout_is_seq_std:0.0015170971164479852 - rollout_corr/rollout_is_seq_max:1.0070810317993164 - rollout_corr/rollout_is_seq_min:0.993159294128418 - rollout_corr/rollout_is_seq_max_deviation:0.007081031799316406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2414707052521408) - rollout_corr/training_log_ppl:np.float64(0.20471240562619641) - rollout_corr/kl:np.float64(0.00047142372690922585) - rollout_corr/k3_kl:np.float64(0.0006191272363480493) - rollout_corr/rollout_ppl:np.float64(1.2409044564701617) - rollout_corr/rollout_log_ppl:np.float64(0.20424098065996077) - rollout_corr/log_ppl_diff:np.float64(0.0004714249662356451) - rollout_corr/log_ppl_abs_diff:np.float64(0.00149485188012477) - rollout_corr/log_ppl_diff_max:np.float64(0.007620379328727722) - rollout_corr/log_ppl_diff_min:np.float64(-0.0099639892578125) - rollout_corr/ppl_ratio:np.float64(1.000473550753668) - rollout_corr/chi2_token:np.float64(0.001530948095023632) - rollout_corr/chi2_seq:np.float64(1.3715695906430483) - actor/pg_loss:np.float64(0.0013342974707484245) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005119512775308976) - actor/ppo_kl:np.float64(4.771902877465095e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(48177.57957301638) - self_distillation/token_reweight_w_std:np.float64(715592.1982030283) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001572305103764) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04621635359944776) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.26206739246845245) - perf/mfu/actor:np.float64(0.05585179811843127) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(196.36561012268066) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010570083977654576 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010570083977654576 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:473.95703125 - response_length/max:1318.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:473.95703125 - response_length_non_aborted/max:1318.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001383572816848755 - timing_s/agent_loop/generate_sequences/min:np.float64(1.216639457270503) - timing_s/agent_loop/generate_sequences/max:np.float64(11.079553108662367) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.304368509227061) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.079553108662367) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1318 - timing_s/gen:17.77311465330422 - timing_s/reward:0.0648870412260294 - timing_s/old_log_prob:3.055829044431448 - timing_s/adv:0.5942137930542231 - timing_s/update_actor:21.682081462815404 - timing_s/step:43.26949073560536 - timing_s/stop_profile:0.00012202374637126923 - timing_per_token_ms/gen:0.14648211659898147 - timing_per_token_ms/update_actor:0.11553855869261809 - timing_per_token_ms/adv:0.0031664213291745383 - perf/total_num_tokens:187661 - perf/time_per_step:43.26949073560536 - perf/throughput:542.1285206090332 (TaskRunner pid=2320396) Training Progress: 22%|██▏ | 22/100 [22:35<1:20:45, 62.12s/it] (TaskRunner pid=2320396) step:23 - global_seqlen/min:17017 - global_seqlen/max:34235 - global_seqlen/minmax_diff:17218 - global_seqlen/balanced_min:25934 - global_seqlen/balanced_max:25946 - global_seqlen/mean:25942.125 - actor/entropy:0.26053303480148315 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3661079704761505 - training/rollout_probs_diff_mean:0.0031567697878926992 - training/rollout_probs_diff_std:0.009550051763653755 - training/rollout_actor_probs_pearson_corr:0.9991575479507446 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8515625 - self_distillation/correct_sample_fraction:0.4453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8515625 - rollout_corr/rollout_is_mean:1.0000665187835693 - rollout_corr/rollout_is_ratio_fraction_high:1.4496010408038273e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.248005204019137e-06 - rollout_corr/rollout_is_max:2.1038994789123535 - rollout_corr/rollout_is_min:0.38240915536880493 - rollout_corr/rollout_is_std:0.02985311858355999 - rollout_corr/rollout_is_eff_sample_size:0.9991097038940658 - rollout_corr/rollout_is_seq_mean:1.0001295804977417 - rollout_corr/rollout_is_seq_std:0.0018321149982511997 - rollout_corr/rollout_is_seq_max:1.0110899209976196 - rollout_corr/rollout_is_seq_min:0.9960669875144958 - rollout_corr/rollout_is_seq_max_deviation:0.011089920997619629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.29484834196046) - rollout_corr/training_log_ppl:np.float64(0.2494393489760114) - rollout_corr/kl:np.float64(0.00033169940679478316) - rollout_corr/k3_kl:np.float64(0.0005022480274305963) - rollout_corr/rollout_ppl:np.float64(1.2943815663456917) - rollout_corr/rollout_log_ppl:np.float64(0.24910764716332778) - rollout_corr/log_ppl_diff:np.float64(0.00033170181268360466) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013077733310637996) - rollout_corr/log_ppl_diff_max:np.float64(0.010803669691085815) - rollout_corr/log_ppl_diff_min:np.float64(-0.011415451765060425) - rollout_corr/ppl_ratio:np.float64(1.000333657488227) - rollout_corr/chi2_token:np.float64(0.0013864929787814617) - rollout_corr/chi2_seq:np.float64(0.9093728845473379) - actor/pg_loss:np.float64(0.0041787896770983934) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038857600884512067) - actor/ppo_kl:np.float64(-1.400662002115638e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4453125) - self_distillation/token_reweight_w_mean:np.float64(9336.005591196474) - self_distillation/token_reweight_w_std:np.float64(221650.8600916579) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002693643560633) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09581344586331397) - self_distillation/empty_target_batch:np.float64(0.1484375) - actor/grad_norm:np.float64(0.29776711761951447) - perf/mfu/actor:np.float64(0.060611924379589) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.15872192382812) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002980742137879133 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002980742137879133 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:538.94140625 - response_length/max:1397.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:538.94140625 - response_length_non_aborted/max:1397.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003536306321620941 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1290383003652096) - timing_s/agent_loop/generate_sequences/max:np.float64(11.685129918158054) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.777066974442278) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.685129918158054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1397 - timing_s/gen:18.946874292567372 - timing_s/reward:0.06363841146230698 - timing_s/old_log_prob:3.2305244877934456 - timing_s/adv:0.6419652253389359 - timing_s/update_actor:22.296060722321272 - timing_s/step:45.296368503943086 - timing_s/stop_profile:0.00012127123773097992 - timing_per_token_ms/gen:0.13732703935353138 - timing_per_token_ms/update_actor:0.10743173854455482 - timing_per_token_ms/adv:0.003093256746213619 - perf/total_num_tokens:207537 - perf/time_per_step:45.296368503943086 - perf/throughput:572.719753411175 (TaskRunner pid=2320396) Training Progress: 23%|██▎ | 23/100 [23:21<1:13:23, 57.19s/it] (TaskRunner pid=2320396) step:24 - global_seqlen/min:17547 - global_seqlen/max:27928 - global_seqlen/minmax_diff:10381 - global_seqlen/balanced_min:22587 - global_seqlen/balanced_max:22607 - global_seqlen/mean:22601.75 - actor/entropy:0.23046009242534637 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7216058373451233 - training/rollout_probs_diff_mean:0.0030117868445813656 - training/rollout_probs_diff_std:0.009528078138828278 - training/rollout_actor_probs_pearson_corr:0.9991067051887512 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999849200248718 - rollout_corr/rollout_is_ratio_fraction_high:8.861634341883473e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.7723268683766946e-05 - rollout_corr/rollout_is_max:2.567455768585205 - rollout_corr/rollout_is_min:0.043630991131067276 - rollout_corr/rollout_is_std:0.028607197105884552 - rollout_corr/rollout_is_eff_sample_size:0.9991822974119504 - rollout_corr/rollout_is_seq_mean:0.9998600482940674 - rollout_corr/rollout_is_seq_std:0.0020710325334221125 - rollout_corr/rollout_is_seq_max:1.0104851722717285 - rollout_corr/rollout_is_seq_min:0.9908909201622009 - rollout_corr/rollout_is_seq_max_deviation:0.010485172271728516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2293109917081892) - rollout_corr/training_log_ppl:np.float64(0.19703222493990324) - rollout_corr/kl:np.float64(0.0006336333097620717) - rollout_corr/k3_kl:np.float64(0.0004862117235546748) - rollout_corr/rollout_ppl:np.float64(1.2285275124013424) - rollout_corr/rollout_log_ppl:np.float64(0.196398589774617) - rollout_corr/log_ppl_diff:np.float64(0.0006336351652862504) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014370774588314816) - rollout_corr/log_ppl_diff_max:np.float64(0.010363861918449402) - rollout_corr/log_ppl_diff_min:np.float64(-0.00926206260919571) - rollout_corr/ppl_ratio:np.float64(1.000636067474261) - rollout_corr/chi2_token:np.float64(0.0006539849564433098) - rollout_corr/chi2_seq:np.float64(0.4699943282175809) - actor/pg_loss:np.float64(0.0021280853543430567) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015898636911515496) - actor/ppo_kl:np.float64(2.8684361065955954e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(4608.7780102991965) - self_distillation/token_reweight_w_std:np.float64(110638.40136062028) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000740052200854) - self_distillation/token_reweight_w_clip_frac:np.float64(0.050635051098652184) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.1550346463918686) - perf/mfu/actor:np.float64(0.05328909417965846) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.19468307495117) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002256393665447831 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002256393665447831 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.8046875 - response_length/max:1444.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.8046875 - response_length_non_aborted/max:1444.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001823175698518753 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9695372581481934) - timing_s/agent_loop/generate_sequences/max:np.float64(11.480458768084645) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.8320430894818855) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.480458768084645) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:239 - timing_s/agent_loop/slowest/response_length:1444 - timing_s/gen:18.5911016035825 - timing_s/reward:0.061533816158771515 - timing_s/old_log_prob:3.102441059425473 - timing_s/adv:0.6918395888060331 - timing_s/update_actor:21.863283349201083 - timing_s/step:44.40328153781593 - timing_s/stop_profile:0.00011542066931724548 - timing_per_token_ms/gen:0.16474754624517043 - timing_per_token_ms/update_actor:0.12091587680821775 - timing_per_token_ms/adv:0.0038262501178339793 - perf/total_num_tokens:180814 - perf/time_per_step:44.40328153781593 - perf/throughput:509.0108031937073 (TaskRunner pid=2320396) Training Progress: 24%|██▍ | 24/100 [24:05<1:07:36, 53.37s/it] (TaskRunner pid=2320396) step:25 - global_seqlen/min:18579 - global_seqlen/max:32122 - global_seqlen/minmax_diff:13543 - global_seqlen/balanced_min:24530 - global_seqlen/balanced_max:24557 - global_seqlen/mean:24549.25 - actor/entropy:0.2588360011577606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24279403686523438 - training/rollout_probs_diff_mean:0.00314454291947186 - training/rollout_probs_diff_std:0.00929185375571251 - training/rollout_actor_probs_pearson_corr:0.9992793798446655 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000556707382202 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.201383151463233e-05 - rollout_corr/rollout_is_max:1.6370543241500854 - rollout_corr/rollout_is_min:0.29653897881507874 - rollout_corr/rollout_is_std:0.028675872832536697 - rollout_corr/rollout_is_eff_sample_size:0.999178488965226 - rollout_corr/rollout_is_seq_mean:1.0000877380371094 - rollout_corr/rollout_is_seq_std:0.001914774882607162 - rollout_corr/rollout_is_seq_max:1.0101841688156128 - rollout_corr/rollout_is_seq_min:0.9926535487174988 - rollout_corr/rollout_is_seq_max_deviation:0.010184168815612793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2983261444605887) - rollout_corr/training_log_ppl:np.float64(0.2460111120599322) - rollout_corr/kl:np.float64(0.0005979655367260861) - rollout_corr/k3_kl:np.float64(0.0004908691531397835) - rollout_corr/rollout_ppl:np.float64(1.297467083670199) - rollout_corr/rollout_log_ppl:np.float64(0.24541314343514387) - rollout_corr/log_ppl_diff:np.float64(0.0005979686247883365) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014427673449972644) - rollout_corr/log_ppl_diff_max:np.float64(0.009425044059753418) - rollout_corr/log_ppl_diff_min:np.float64(-0.006913349032402039) - rollout_corr/ppl_ratio:np.float64(1.0006001458968967) - rollout_corr/chi2_token:np.float64(0.0007500855717808008) - rollout_corr/chi2_seq:np.float64(0.46263502025976777) - actor/pg_loss:np.float64(0.0036563018802553415) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040037721146291005) - actor/ppo_kl:np.float64(0.00023973096506502145) - actor/pg_clipfrac_lower:np.float64(7.063743396429345e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(5786.897568458458) - self_distillation/token_reweight_w_std:np.float64(129991.75501095905) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001443214714527) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07592839169956278) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.2528732307255268) - perf/mfu/actor:np.float64(0.058185086481743295) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.12001419067383) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021869447082281113 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021869447082281113 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:488.0703125 - response_length/max:1537.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:488.0703125 - response_length_non_aborted/max:1537.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014443323016166687 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8763598762452602) - timing_s/agent_loop/generate_sequences/max:np.float64(12.1195951346308) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.420748021701002) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.1195951346308) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:1537 - timing_s/gen:19.121401023119688 - timing_s/reward:0.06362378038465977 - timing_s/old_log_prob:3.279709119349718 - timing_s/adv:0.657275116071105 - timing_s/update_actor:22.008095744997263 - timing_s/step:45.220784513279796 - timing_s/stop_profile:0.00011463277041912079 - timing_per_token_ms/gen:0.15303732030733028 - timing_per_token_ms/update_actor:0.11206093742679137 - timing_per_token_ms/adv:0.003346716885806618 - perf/total_num_tokens:196394 - perf/time_per_step:45.220784513279796 - perf/throughput:542.8753672504449 (TaskRunner pid=2320396) Training Progress: 25%|██▌ | 25/100 [24:51<1:03:40, 50.94s/it] (TaskRunner pid=2320396) step:26 - global_seqlen/min:19341 - global_seqlen/max:24825 - global_seqlen/minmax_diff:5484 - global_seqlen/balanced_min:21767 - global_seqlen/balanced_max:21774 - global_seqlen/mean:21770.625 - actor/entropy:0.2076856642961502 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105889558792114 - training/rollout_probs_diff_mean:0.002844884293153882 - training/rollout_probs_diff_std:0.009245309047400951 - training/rollout_actor_probs_pearson_corr:0.9991264343261719 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.9999702572822571 - rollout_corr/rollout_is_ratio_fraction_high:9.282895916840062e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8565791833680123e-05 - rollout_corr/rollout_is_max:2.44268798828125 - rollout_corr/rollout_is_min:0.3941245973110199 - rollout_corr/rollout_is_std:0.027870552614331245 - rollout_corr/rollout_is_eff_sample_size:0.9992238351690815 - rollout_corr/rollout_is_seq_mean:1.0000226497650146 - rollout_corr/rollout_is_seq_std:0.0018026805482804775 - rollout_corr/rollout_is_seq_max:1.0095715522766113 - rollout_corr/rollout_is_seq_min:0.9939136505126953 - rollout_corr/rollout_is_seq_max_deviation:0.009571552276611328 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2315654191188514) - rollout_corr/training_log_ppl:np.float64(0.19640630582580343) - rollout_corr/kl:np.float64(0.0003411613332708896) - rollout_corr/k3_kl:np.float64(0.00045860192925317733) - rollout_corr/rollout_ppl:np.float64(1.2310992125421762) - rollout_corr/rollout_log_ppl:np.float64(0.19606514342012815) - rollout_corr/log_ppl_diff:np.float64(0.0003411624056752771) - rollout_corr/log_ppl_abs_diff:np.float64(0.001351964398054406) - rollout_corr/log_ppl_diff_max:np.float64(0.007199376821517944) - rollout_corr/log_ppl_diff_min:np.float64(-0.008631736040115356) - rollout_corr/ppl_ratio:np.float64(1.0003430233336985) - rollout_corr/chi2_token:np.float64(0.0012167126405984163) - rollout_corr/chi2_seq:np.float64(0.3726298357360065) - actor/pg_loss:np.float64(0.001817713608033955) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024271494748973055) - actor/ppo_kl:np.float64(-6.930642252278574e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(21209.821032542968) - self_distillation/token_reweight_w_std:np.float64(407491.57075386524) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005513478536159) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04697586127440445) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.18191703036427498) - perf/mfu/actor:np.float64(0.052533244830375574) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.11370468139648) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007448364049196243 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007448364049196243 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.80078125 - response_length/max:1230.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.80078125 - response_length_non_aborted/max:1230.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011184252798557281 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7711089085787535) - timing_s/agent_loop/generate_sequences/max:np.float64(9.832402851432562) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.4123942583973985) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.832402851432562) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:316 - timing_s/agent_loop/slowest/response_length:1202 - timing_s/gen:16.367364613339305 - timing_s/reward:0.06243230774998665 - timing_s/old_log_prob:3.00197671353817 - timing_s/adv:0.577815530821681 - timing_s/update_actor:21.668468771502376 - timing_s/step:41.76663071103394 - timing_s/stop_profile:0.00011580996215343475 - timing_per_token_ms/gen:0.15193654781470695 - timing_per_token_ms/update_actor:0.12441345144835286 - timing_per_token_ms/adv:0.003317632881587466 - perf/total_num_tokens:174165 - perf/time_per_step:41.76663071103394 - perf/throughput:521.2444630887743 (TaskRunner pid=2320396) Training Progress: 26%|██▌ | 26/100 [25:32<59:27, 48.20s/it] (TaskRunner pid=2320396) step:27 - global_seqlen/min:20266 - global_seqlen/max:29377 - global_seqlen/minmax_diff:9111 - global_seqlen/balanced_min:25441 - global_seqlen/balanced_max:25448 - global_seqlen/mean:25444.0 - actor/entropy:0.21156783401966095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2872914671897888 - training/rollout_probs_diff_mean:0.002737778937444091 - training/rollout_probs_diff_std:0.009022722952067852 - training/rollout_actor_probs_pearson_corr:0.9992183446884155 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.9921875 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:0.9998476505279541 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6285487413406372 - rollout_corr/rollout_is_min:0.5106052756309509 - rollout_corr/rollout_is_std:0.02745903842151165 - rollout_corr/rollout_is_eff_sample_size:0.9992465693293799 - rollout_corr/rollout_is_seq_mean:0.9997648000717163 - rollout_corr/rollout_is_seq_std:0.0015325150452554226 - rollout_corr/rollout_is_seq_max:1.0047353506088257 - rollout_corr/rollout_is_seq_min:0.9945488572120667 - rollout_corr/rollout_is_seq_max_deviation:0.00545114278793335 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2360313851386309) - rollout_corr/training_log_ppl:np.float64(0.20034451349056326) - rollout_corr/kl:np.float64(0.0006052363555104634) - rollout_corr/k3_kl:np.float64(0.00048079977509019045) - rollout_corr/rollout_ppl:np.float64(1.2352807368151844) - rollout_corr/rollout_log_ppl:np.float64(0.19973927503451705) - rollout_corr/log_ppl_diff:np.float64(0.0006052384560462087) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012537067814264446) - rollout_corr/log_ppl_diff_max:np.float64(0.005964487791061401) - rollout_corr/log_ppl_diff_min:np.float64(-0.004271447658538818) - rollout_corr/ppl_ratio:np.float64(1.0006067522335798) - rollout_corr/chi2_token:np.float64(0.0007339166477322578) - rollout_corr/chi2_seq:np.float64(0.17413755250163376) - actor/pg_loss:np.float64(0.0029487464344128966) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036103266734244244) - actor/ppo_kl:np.float64(-5.398572019643666e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(677.8268313717563) - self_distillation/token_reweight_w_std:np.float64(18296.45410767995) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002783685922623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0762047362368321) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.38731055706739426) - perf/mfu/actor:np.float64(0.05949076540086539) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.24272918701172) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0031912531703710556 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0031912531703710556 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:518.84375 - response_length/max:1403.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:518.84375 - response_length_non_aborted/max:1403.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010640919208526611 - timing_s/agent_loop/generate_sequences/min:np.float64(1.03832764364779) - timing_s/agent_loop/generate_sequences/max:np.float64(11.360929055139422) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.670121224604372) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.360929055139422) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:1403 - timing_s/gen:17.88263091444969 - timing_s/reward:0.06664908863604069 - timing_s/old_log_prob:3.0836950354278088 - timing_s/adv:0.6352357696741819 - timing_s/update_actor:22.302965883165598 - timing_s/step:44.05930548161268 - timing_s/stop_profile:5.204230546951294e-05 - timing_per_token_ms/gen:0.13463403386774747 - timing_per_token_ms/update_actor:0.10956888600045982 - timing_per_token_ms/adv:0.003120754252840463 - perf/total_num_tokens:203552 - perf/time_per_step:44.05930548161268 - perf/throughput:577.494350441329 (TaskRunner pid=2320396) Training Progress: 27%|██▋ | 27/100 [26:16<57:08, 46.97s/it] (TaskRunner pid=2320396) step:28 - global_seqlen/min:17086 - global_seqlen/max:33653 - global_seqlen/minmax_diff:16567 - global_seqlen/balanced_min:24718 - global_seqlen/balanced_max:24765 - global_seqlen/mean:24755.125 - actor/entropy:0.24302850663661957 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4545377194881439 - training/rollout_probs_diff_mean:0.002987111918628216 - training/rollout_probs_diff_std:0.009371647611260414 - training/rollout_actor_probs_pearson_corr:0.9992448687553406 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0000720024108887 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5652269212296233e-05 - rollout_corr/rollout_is_max:1.6478630304336548 - rollout_corr/rollout_is_min:0.32801195979118347 - rollout_corr/rollout_is_std:0.02821488305926323 - rollout_corr/rollout_is_eff_sample_size:0.9992046726227877 - rollout_corr/rollout_is_seq_mean:0.999945878982544 - rollout_corr/rollout_is_seq_std:0.0020114099606871605 - rollout_corr/rollout_is_seq_max:1.0073790550231934 - rollout_corr/rollout_is_seq_min:0.987631618976593 - rollout_corr/rollout_is_seq_max_deviation:0.012368381023406982 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2567434725351632) - rollout_corr/training_log_ppl:np.float64(0.21964200896036346) - rollout_corr/kl:np.float64(0.0004823495658996535) - rollout_corr/k3_kl:np.float64(0.0004897397312362273) - rollout_corr/rollout_ppl:np.float64(1.256141950841993) - rollout_corr/rollout_log_ppl:np.float64(0.21915965757216327) - rollout_corr/log_ppl_diff:np.float64(0.0004823513882001862) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014439801598200575) - rollout_corr/log_ppl_diff_max:np.float64(0.015101104974746704) - rollout_corr/log_ppl_diff_min:np.float64(-0.006693243980407715) - rollout_corr/ppl_ratio:np.float64(1.000484771328047) - rollout_corr/chi2_token:np.float64(0.0009968443773686886) - rollout_corr/chi2_seq:np.float64(1.4560303187463433) - actor/pg_loss:np.float64(0.0032396138412877917) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000316952940693227) - actor/ppo_kl:np.float64(-2.395193692983355e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(23524.618457379518) - self_distillation/token_reweight_w_std:np.float64(496158.66897797544) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004901748616248) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08251259801909328) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.25753332301974297) - perf/mfu/actor:np.float64(0.05782911225939355) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.28218793869019) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01441084872931242 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01441084872931242 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:499.12890625 - response_length/max:1897.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:499.12890625 - response_length_non_aborted/max:1897.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001032557338476181 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9153902158141136) - timing_s/agent_loop/generate_sequences/max:np.float64(14.23232613503933) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.135112953190401) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.23232613503933) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1897 - timing_s/gen:21.24835624359548 - timing_s/reward:0.06606246717274189 - timing_s/old_log_prob:3.0791484899818897 - timing_s/adv:0.6036070547997952 - timing_s/update_actor:22.402966856956482 - timing_s/step:47.48915730416775 - timing_s/stop_profile:0.00010910257697105408 - timing_per_token_ms/gen:0.16629249586072203 - timing_per_token_ms/update_actor:0.11312287282409442 - timing_per_token_ms/adv:0.0030478893501840283 - perf/total_num_tokens:198041 - perf/time_per_step:47.48915730416775 - perf/throughput:521.279517373694 (TaskRunner pid=2320396) Training Progress: 28%|██▊ | 28/100 [27:04<56:33, 47.14s/it] (TaskRunner pid=2320396) step:29 - global_seqlen/min:20180 - global_seqlen/max:30311 - global_seqlen/minmax_diff:10131 - global_seqlen/balanced_min:24016 - global_seqlen/balanced_max:24053 - global_seqlen/mean:24044.5 - actor/entropy:0.22302956879138947 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6587663888931274 - training/rollout_probs_diff_mean:0.002952124923467636 - training/rollout_probs_diff_std:0.009483864530920982 - training/rollout_actor_probs_pearson_corr:0.9991285800933838 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.9999752640724182 - rollout_corr/rollout_is_ratio_fraction_high:2.402075369900558e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.006917596503627e-06 - rollout_corr/rollout_is_max:3.0540003776550293 - rollout_corr/rollout_is_min:0.08800696581602097 - rollout_corr/rollout_is_std:0.02860928140580654 - rollout_corr/rollout_is_eff_sample_size:0.9991820593831796 - rollout_corr/rollout_is_seq_mean:0.9998377561569214 - rollout_corr/rollout_is_seq_std:0.0019047566456720233 - rollout_corr/rollout_is_seq_max:1.0071954727172852 - rollout_corr/rollout_is_seq_min:0.9904940724372864 - rollout_corr/rollout_is_seq_max_deviation:0.009505927562713623 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.239888684824109) - rollout_corr/training_log_ppl:np.float64(0.2059190910149482) - rollout_corr/kl:np.float64(0.0005806351006372168) - rollout_corr/k3_kl:np.float64(0.0005054707310918616) - rollout_corr/rollout_ppl:np.float64(1.2391317044384778) - rollout_corr/rollout_log_ppl:np.float64(0.20533845498721348) - rollout_corr/log_ppl_diff:np.float64(0.0005806360277347267) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014441340463235974) - rollout_corr/log_ppl_diff_max:np.float64(0.009319275617599487) - rollout_corr/log_ppl_diff_min:np.float64(-0.008508995175361633) - rollout_corr/ppl_ratio:np.float64(1.0005828321445733) - rollout_corr/chi2_token:np.float64(0.0008565832395106554) - rollout_corr/chi2_seq:np.float64(0.48222310887649655) - actor/pg_loss:np.float64(0.002814142149873078) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031159500281319197) - actor/ppo_kl:np.float64(-3.463015074700593e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(1034.8058356901165) - self_distillation/token_reweight_w_std:np.float64(28329.355002246273) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994706646539271) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05690540156501811) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.188645763322711) - perf/mfu/actor:np.float64(0.05621638661845592) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.2503433227539) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013596748001873493 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013596748001873493 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:487.859375 - response_length/max:1808.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:487.859375 - response_length_non_aborted/max:1808.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.099580347537994e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1761888433247805) - timing_s/agent_loop/generate_sequences/max:np.float64(13.156388711184263) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.122582192576374) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.156388711184263) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:1808 - timing_s/gen:20.01565883681178 - timing_s/reward:0.06360073015093803 - timing_s/old_log_prob:3.1809177827090025 - timing_s/adv:0.6148683913052082 - timing_s/update_actor:22.266462163999677 - timing_s/step:46.22870226763189 - timing_s/stop_profile:2.8520822525024414e-05 - timing_per_token_ms/gen:0.16026373856461407 - timing_per_token_ms/update_actor:0.11575652521366464 - timing_per_token_ms/adv:0.003196512670804177 - perf/total_num_tokens:192356 - perf/time_per_step:46.22870226763189 - perf/throughput:520.1205921983088 (TaskRunner pid=2320396) Training Progress: 29%|██▉ | 29/100 [27:50<55:27, 46.87s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) We solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substitute the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \times (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Convert to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is **B: 56 pC**. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2320396) step:30 - global_seqlen/min:17742 - global_seqlen/max:26480 - global_seqlen/minmax_diff:8738 - global_seqlen/balanced_min:22366 - global_seqlen/balanced_max:22378 - global_seqlen/mean:22373.5 - actor/entropy:0.24863006174564362 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24456164240837097 - training/rollout_probs_diff_mean:0.0031601106747984886 - training/rollout_probs_diff_std:0.009502153843641281 - training/rollout_actor_probs_pearson_corr:0.9991329312324524 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.0000206232070923 - rollout_corr/rollout_is_ratio_fraction_high:1.776640783646144e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.664961175469216e-05 - rollout_corr/rollout_is_max:3.497854709625244 - rollout_corr/rollout_is_min:0.16539886593818665 - rollout_corr/rollout_is_std:0.029274486005306244 - rollout_corr/rollout_is_eff_sample_size:0.9991437382299739 - rollout_corr/rollout_is_seq_mean:0.9998601078987122 - rollout_corr/rollout_is_seq_std:0.0016287954058498144 - rollout_corr/rollout_is_seq_max:1.0044320821762085 - rollout_corr/rollout_is_seq_min:0.992756724357605 - rollout_corr/rollout_is_seq_max_deviation:0.0072432756423950195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2682397873140872) - rollout_corr/training_log_ppl:np.float64(0.22362850421632174) - rollout_corr/kl:np.float64(0.00048620960521006396) - rollout_corr/k3_kl:np.float64(0.0004756366115827859) - rollout_corr/rollout_ppl:np.float64(1.2676585195586085) - rollout_corr/rollout_log_ppl:np.float64(0.22314229208859615) - rollout_corr/log_ppl_diff:np.float64(0.00048621212772559375) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014107342140050605) - rollout_corr/log_ppl_diff_max:np.float64(0.007874712347984314) - rollout_corr/log_ppl_diff_min:np.float64(-0.010280773043632507) - rollout_corr/ppl_ratio:np.float64(1.0004882055800408) - rollout_corr/chi2_token:np.float64(0.0009464134927839041) - rollout_corr/chi2_seq:np.float64(0.9949460891075432) - actor/pg_loss:np.float64(0.002644557738676667) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018046644299829495) - actor/ppo_kl:np.float64(-0.00010434456421570815) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(15509.348625806859) - self_distillation/token_reweight_w_std:np.float64(335266.9792876573) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003502645995468) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06214348939829506) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.1956712994724512) - perf/mfu/actor:np.float64(0.05303688677384744) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.29561233520508) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6140625) - val-aux/sciknoweval/reward/std@16:np.float64(0.20064917766957618) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6982125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17558868106714795) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.528725) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1636992413789501) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6131) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.20074825670766394) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.77105) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13181042805409973) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46654999999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11489070749110727) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6290749999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16748543115238934) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.82365) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08642785589755841) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.42318750000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07302527970826564) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6379374999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12699065808758647) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8556874999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04432075210710901) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3948625) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04169575957732149) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6434875) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09359792416990569) - val-aux/sciknoweval/score/mean@16:np.float64(0.6140625) - val-aux/sciknoweval/score/std@16:np.float64(0.20064917766957618) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6982125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17558868106714795) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.528725) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1636992413789501) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6131) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.20074825670766394) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.77105) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13181042805409973) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.46654999999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11489070749110727) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6290749999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.16748543115238934) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.82365) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08642785589755841) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.42318750000000005) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07302527970826564) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6379374999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12699065808758647) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8556874999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04432075210710901) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3948625) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04169575957732149) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6434875) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09359792416990569) - val-core/sciknoweval/acc/mean@16:np.float64(0.6140625) - val-aux/sciknoweval/acc/std@16:np.float64(0.20064917766957618) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6982125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17558868106714795) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.528725) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1636992413789501) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6131) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.20074825670766394) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.77105) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13181042805409973) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.46654999999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11489070749110727) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6290749999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.16748543115238934) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.82365) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08642785589755841) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.42318750000000005) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07302527970826564) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6379374999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12699065808758647) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8556874999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04432075210710901) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3948625) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04169575957732149) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6434875) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09359792416990569) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98984375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.028446172895248756) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.998475) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.00989590809890792) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9818999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.036282721326102566) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9904250000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.027632795745506478) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999925) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0013474443894951921) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9682624999999998) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04318228991168178) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9961500000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01706630999890189) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.94895) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.045543356359162894) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9989000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008950291396294888) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9274875) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03923536913417337) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999125000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0016718450301930732) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00767730874940753 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00767730874940753 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.734375 - response_length/max:1411.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.734375 - response_length_non_aborted/max:1411.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.119906902313232e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9468586817383766) - timing_s/agent_loop/generate_sequences/max:np.float64(11.205822797492146) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.667787884332938) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.205822797492146) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1407 - timing_s/gen:18.225317746400833 - timing_s/reward:0.06284921430051327 - timing_s/old_log_prob:3.0297752507030964 - timing_s/adv:0.6036888118833303 - timing_s/update_actor:22.02555533312261 - timing_s/step:44.033918507397175 - timing_s/testing:111.50396230816841 - timing_s/save_checkpoint:13.47470897063613 - timing_s/stop_profile:5.4797157645225525e-05 - timing_per_token_ms/gen:0.16189920891874385 - timing_per_token_ms/update_actor:0.12305604472435365 - timing_per_token_ms/adv:0.003372789303659074 - perf/total_num_tokens:178988 - perf/time_per_step:44.033918507397175 - perf/throughput:508.0969570364608 (TaskRunner pid=2320396) Training Progress: 30%|███ | 30/100 [30:39<1:37:26, 83.53s/it] (TaskRunner pid=2320396) step:31 - global_seqlen/min:19933 - global_seqlen/max:31808 - global_seqlen/minmax_diff:11875 - global_seqlen/balanced_min:24800 - global_seqlen/balanced_max:24806 - global_seqlen/mean:24803.625 - actor/entropy:0.23857657611370087 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21513324975967407 - training/rollout_probs_diff_mean:0.0027538202702999115 - training/rollout_probs_diff_std:0.008704227395355701 - training/rollout_actor_probs_pearson_corr:0.9992588758468628 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.9999880194664001 - rollout_corr/rollout_is_ratio_fraction_high:7.856385309423786e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.571277061884757e-05 - rollout_corr/rollout_is_max:2.55385684967041 - rollout_corr/rollout_is_min:0.49398577213287354 - rollout_corr/rollout_is_std:0.027300119400024414 - rollout_corr/rollout_is_eff_sample_size:0.9992551395431086 - rollout_corr/rollout_is_seq_mean:0.9999855756759644 - rollout_corr/rollout_is_seq_std:0.0014757115859538317 - rollout_corr/rollout_is_seq_max:1.0052125453948975 - rollout_corr/rollout_is_seq_min:0.995438277721405 - rollout_corr/rollout_is_seq_max_deviation:0.005212545394897461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2466215244494379) - rollout_corr/training_log_ppl:np.float64(0.20910465411725454) - rollout_corr/kl:np.float64(0.0004072134899462121) - rollout_corr/k3_kl:np.float64(0.0004494452254277803) - rollout_corr/rollout_ppl:np.float64(1.2461073002777994) - rollout_corr/rollout_log_ppl:np.float64(0.20869743956427556) - rollout_corr/log_ppl_diff:np.float64(0.000407214552978985) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012124801141908392) - rollout_corr/log_ppl_diff_max:np.float64(0.006325840950012207) - rollout_corr/log_ppl_diff_min:np.float64(-0.004628986120223999) - rollout_corr/ppl_ratio:np.float64(1.0004085870459676) - rollout_corr/chi2_token:np.float64(0.0010155059862881899) - rollout_corr/chi2_seq:np.float64(0.6279427350964397) - actor/pg_loss:np.float64(0.002768305013887584) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030953091891205986) - actor/ppo_kl:np.float64(2.230859727170298e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(27233.19396411348) - self_distillation/token_reweight_w_std:np.float64(526200.2407925605) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011748706456274) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06611918151611462) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.22732961922883987) - perf/mfu/actor:np.float64(0.057831701660071295) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(199.57726669311523) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005538751720450819 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005538751720450819 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:497.20703125 - response_length/max:1315.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:497.20703125 - response_length_non_aborted/max:1315.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.376819849014282e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8423037379980087) - timing_s/agent_loop/generate_sequences/max:np.float64(11.606234759092331) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.555152973211079) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.606234759092331) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:1315 - timing_s/gen:18.41877550072968 - timing_s/reward:0.05963750369846821 - timing_s/old_log_prob:3.2094839587807655 - timing_s/adv:0.6232157163321972 - timing_s/update_actor:22.237760320305824 - timing_s/step:44.65447077155113 - timing_s/stop_profile:0.00010682083666324615 - timing_per_token_ms/gen:0.1447049966667689 - timing_per_token_ms/update_actor:0.11206910441672248 - timing_per_token_ms/adv:0.0031407491663627655 - perf/total_num_tokens:198429 - perf/time_per_step:44.65447077155113 - perf/throughput:555.4567005595801 (TaskRunner pid=2320396) Training Progress: 31%|███ | 31/100 [31:24<1:22:39, 71.88s/it] (TaskRunner pid=2320396) step:32 - global_seqlen/min:14922 - global_seqlen/max:26458 - global_seqlen/minmax_diff:11536 - global_seqlen/balanced_min:21089 - global_seqlen/balanced_max:21197 - global_seqlen/mean:21116.75 - actor/entropy:0.23724991083145142 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22528886795043945 - training/rollout_probs_diff_mean:0.003095978172495961 - training/rollout_probs_diff_std:0.00945594534277916 - training/rollout_actor_probs_pearson_corr:0.9991330504417419 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.000195860862732 - rollout_corr/rollout_is_ratio_fraction_high:9.644503734307364e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.9289007468614727e-05 - rollout_corr/rollout_is_max:2.3271644115448 - rollout_corr/rollout_is_min:0.296479195356369 - rollout_corr/rollout_is_std:0.029066067188978195 - rollout_corr/rollout_is_eff_sample_size:0.9991562339352135 - rollout_corr/rollout_is_seq_mean:1.0002284049987793 - rollout_corr/rollout_is_seq_std:0.0022042470518499613 - rollout_corr/rollout_is_seq_max:1.0126055479049683 - rollout_corr/rollout_is_seq_min:0.9936482906341553 - rollout_corr/rollout_is_seq_max_deviation:0.012605547904968262 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.261983500327915) - rollout_corr/training_log_ppl:np.float64(0.21683665785531048) - rollout_corr/kl:np.float64(0.0002809870581792495) - rollout_corr/k3_kl:np.float64(0.00047995293701319497) - rollout_corr/rollout_ppl:np.float64(1.261621960438788) - rollout_corr/rollout_log_ppl:np.float64(0.2165556696272688) - rollout_corr/log_ppl_diff:np.float64(0.00028098822804167867) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015070391818881035) - rollout_corr/log_ppl_diff_max:np.float64(0.007703110575675964) - rollout_corr/log_ppl_diff_min:np.float64(-0.010310009121894836) - rollout_corr/ppl_ratio:np.float64(1.0002832827158272) - rollout_corr/chi2_token:np.float64(0.0013595439959317446) - rollout_corr/chi2_seq:np.float64(0.5350547896232456) - actor/pg_loss:np.float64(0.001982162822969258) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023478754496863985) - actor/ppo_kl:np.float64(4.076835085786712e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(17187.49791721138) - self_distillation/token_reweight_w_std:np.float64(332695.3507974972) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003862450830638) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05358828179305419) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.1963564045727253) - perf/mfu/actor:np.float64(0.05060214812294124) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.50678253173828) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010506480932235718 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010506480932235718 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:405.0234375 - response_length/max:1677.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:405.0234375 - response_length_non_aborted/max:1677.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001492369920015335 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9843822922557592) - timing_s/agent_loop/generate_sequences/max:np.float64(12.267383886501193) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.2397227422043215) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.267383886501193) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:1677 - timing_s/gen:19.05756193958223 - timing_s/reward:0.05966056138277054 - timing_s/old_log_prob:3.105440441519022 - timing_s/adv:0.6150863282382488 - timing_s/update_actor:21.63486903347075 - timing_s/step:44.565065210685134 - timing_s/stop_profile:0.00013065338134765625 - timing_per_token_ms/gen:0.1838007246839711 - timing_per_token_ms/update_actor:0.1280669908572031 - timing_per_token_ms/adv:0.0036409859959407155 - perf/total_num_tokens:168934 - perf/time_per_step:44.565065210685134 - perf/throughput:473.840886357257 (TaskRunner pid=2320396) Training Progress: 32%|███▏ | 32/100 [32:09<1:12:12, 63.71s/it] (TaskRunner pid=2320396) step:33 - global_seqlen/min:20278 - global_seqlen/max:32793 - global_seqlen/minmax_diff:12515 - global_seqlen/balanced_min:25854 - global_seqlen/balanced_max:25862 - global_seqlen/mean:25858.5 - actor/entropy:0.2247491031885147 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24133732914924622 - training/rollout_probs_diff_mean:0.0028703457210212946 - training/rollout_probs_diff_std:0.008939353749155998 - training/rollout_actor_probs_pearson_corr:0.9992457628250122 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.92578125 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.92578125 - rollout_corr/rollout_is_mean:0.9999353885650635 - rollout_corr/rollout_is_ratio_fraction_high:7.54580287321005e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.54580287321005e-06 - rollout_corr/rollout_is_max:2.0470011234283447 - rollout_corr/rollout_is_min:0.4843989908695221 - rollout_corr/rollout_is_std:0.02728046104311943 - rollout_corr/rollout_is_eff_sample_size:0.9992563298621977 - rollout_corr/rollout_is_seq_mean:0.9998895525932312 - rollout_corr/rollout_is_seq_std:0.001413932885043323 - rollout_corr/rollout_is_seq_max:1.0037660598754883 - rollout_corr/rollout_is_seq_min:0.9910454750061035 - rollout_corr/rollout_is_seq_max_deviation:0.008954524993896484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2669287519529462) - rollout_corr/training_log_ppl:np.float64(0.22308755599078722) - rollout_corr/kl:np.float64(0.0005338981650639951) - rollout_corr/k3_kl:np.float64(0.0004518400677682166) - rollout_corr/rollout_ppl:np.float64(1.2662203731015325) - rollout_corr/rollout_log_ppl:np.float64(0.22255365665478166) - rollout_corr/log_ppl_diff:np.float64(0.000533899336005561) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012277777277631685) - rollout_corr/log_ppl_diff_max:np.float64(0.01045304536819458) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037618353962898254) - rollout_corr/ppl_ratio:np.float64(1.0005353440064937) - rollout_corr/chi2_token:np.float64(0.0007296854164451361) - rollout_corr/chi2_seq:np.float64(0.41097239311784506) - actor/pg_loss:np.float64(0.0023052702890709043) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027347075024408696) - actor/ppo_kl:np.float64(1.4708585448275358e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.92578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.92578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(3411.1585964083206) - self_distillation/token_reweight_w_std:np.float64(86631.71416301752) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000493316212669) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05668039701413363) - self_distillation/empty_target_batch:np.float64(0.07421875) - actor/grad_norm:np.float64(0.19899238646030426) - perf/mfu/actor:np.float64(0.06038801604738149) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.45315170288086) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006789336446672678 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006789336446672678 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:517.671875 - response_length/max:1329.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:517.671875 - response_length_non_aborted/max:1329.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013659894466400146 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2961205057799816) - timing_s/agent_loop/generate_sequences/max:np.float64(11.317749934270978) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.581255443590635) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.317749934270978) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1329 - timing_s/gen:18.672686226665974 - timing_s/reward:0.06610148772597313 - timing_s/old_log_prob:3.063822403550148 - timing_s/adv:0.623656140640378 - timing_s/update_actor:22.36642457358539 - timing_s/step:44.8817024435848 - timing_s/stop_profile:0.00011933408677577972 - timing_per_token_ms/gen:0.14090041220206131 - timing_per_token_ms/update_actor:0.1081193059032107 - timing_per_token_ms/adv:0.0030147540491539436 - perf/total_num_tokens:206868 - perf/time_per_step:44.8817024435848 - perf/throughput:576.1479309414232 (TaskRunner pid=2320396) Training Progress: 33%|███▎ | 33/100 [32:54<1:04:51, 58.08s/it] (TaskRunner pid=2320396) step:34 - global_seqlen/min:18809 - global_seqlen/max:30541 - global_seqlen/minmax_diff:11732 - global_seqlen/balanced_min:22984 - global_seqlen/balanced_max:23001 - global_seqlen/mean:22996.375 - actor/entropy:0.2210918515920639 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4279317259788513 - training/rollout_probs_diff_mean:0.002838705899193883 - training/rollout_probs_diff_std:0.009284661151468754 - training/rollout_actor_probs_pearson_corr:0.9991373419761658 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0000059604644775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.49451680551283e-06 - rollout_corr/rollout_is_max:1.964918613433838 - rollout_corr/rollout_is_min:0.022180255502462387 - rollout_corr/rollout_is_std:0.027658019214868546 - rollout_corr/rollout_is_eff_sample_size:0.9992357377417861 - rollout_corr/rollout_is_seq_mean:1.000097632408142 - rollout_corr/rollout_is_seq_std:0.0018793637864291668 - rollout_corr/rollout_is_seq_max:1.0091444253921509 - rollout_corr/rollout_is_seq_min:0.9919560551643372 - rollout_corr/rollout_is_seq_max_deviation:0.009144425392150879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2141821277327836) - rollout_corr/training_log_ppl:np.float64(0.18632135271036532) - rollout_corr/kl:np.float64(0.00037837233885004706) - rollout_corr/k3_kl:np.float64(0.0005182297692556403) - rollout_corr/rollout_ppl:np.float64(1.2136791995726526) - rollout_corr/rollout_log_ppl:np.float64(0.18594297835079487) - rollout_corr/log_ppl_diff:np.float64(0.00037837435957044363) - rollout_corr/log_ppl_abs_diff:np.float64(0.001449274888727814) - rollout_corr/log_ppl_diff_max:np.float64(0.010157078504562378) - rollout_corr/log_ppl_diff_min:np.float64(-0.007487893104553223) - rollout_corr/ppl_ratio:np.float64(1.0003805679734796) - rollout_corr/chi2_token:np.float64(0.0012797676026821136) - rollout_corr/chi2_seq:np.float64(0.4169660920742899) - actor/pg_loss:np.float64(0.0021550224628299475) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045285703208719497) - actor/ppo_kl:np.float64(4.1904223541777696e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(5599.005817859666) - self_distillation/token_reweight_w_std:np.float64(112004.81754589331) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004518062341958) - self_distillation/token_reweight_w_clip_frac:np.float64(0.057579226355301216) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.2537267468869686) - perf/mfu/actor:np.float64(0.05430294950212631) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.41295623779297) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006035353988409042 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006035353988409042 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:459.85546875 - response_length/max:1588.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:459.85546875 - response_length_non_aborted/max:1588.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012004747986793518 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2501347120851278) - timing_s/agent_loop/generate_sequences/max:np.float64(12.419921029359102) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.819320628768764) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.419921029359102) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1588 - timing_s/gen:19.884517431259155 - timing_s/reward:0.06309028342366219 - timing_s/old_log_prob:3.156838323920965 - timing_s/adv:0.5906004216521978 - timing_s/update_actor:22.091687720268965 - timing_s/step:45.87465010210872 - timing_s/stop_profile:0.00013749301433563232 - timing_per_token_ms/gen:0.1689093671691951 - timing_per_token_ms/update_actor:0.1200824462565783 - timing_per_token_ms/adv:0.003210290870040375 - perf/total_num_tokens:183971 - perf/time_per_step:45.87465010210872 - perf/throughput:501.2872021653398 (TaskRunner pid=2320396) Training Progress: 34%|███▍ | 34/100 [33:39<59:52, 54.43s/it] (TaskRunner pid=2320396) step:35 - global_seqlen/min:18781 - global_seqlen/max:27491 - global_seqlen/minmax_diff:8710 - global_seqlen/balanced_min:22623 - global_seqlen/balanced_max:22626 - global_seqlen/mean:22624.75 - actor/entropy:0.23807327449321747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18412302434444427 - training/rollout_probs_diff_mean:0.003115108236670494 - training/rollout_probs_diff_std:0.00949728861451149 - training/rollout_actor_probs_pearson_corr:0.999127209186554 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.71875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9999211430549622 - rollout_corr/rollout_is_ratio_fraction_high:1.743040957080666e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6145613446715288e-05 - rollout_corr/rollout_is_max:2.6255035400390625 - rollout_corr/rollout_is_min:0.4158304035663605 - rollout_corr/rollout_is_std:0.0292867012321949 - rollout_corr/rollout_is_eff_sample_size:0.9991427861890552 - rollout_corr/rollout_is_seq_mean:0.9998403787612915 - rollout_corr/rollout_is_seq_std:0.001774416770786047 - rollout_corr/rollout_is_seq_max:1.0043466091156006 - rollout_corr/rollout_is_seq_min:0.9918447732925415 - rollout_corr/rollout_is_seq_max_deviation:0.008155226707458496 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.265632627531886) - rollout_corr/training_log_ppl:np.float64(0.22240729215263855) - rollout_corr/kl:np.float64(0.000700322022321842) - rollout_corr/k3_kl:np.float64(0.0004940029620996711) - rollout_corr/rollout_ppl:np.float64(1.2647189851850271) - rollout_corr/rollout_log_ppl:np.float64(0.22170696874673013) - rollout_corr/log_ppl_diff:np.float64(0.0007003234059084207) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015176253218669444) - rollout_corr/log_ppl_diff_max:np.float64(0.008863285183906555) - rollout_corr/log_ppl_diff_min:np.float64(-0.003641173243522644) - rollout_corr/ppl_ratio:np.float64(1.0007026130333543) - rollout_corr/chi2_token:np.float64(0.0005808144342154264) - rollout_corr/chi2_seq:np.float64(0.4950171331875026) - actor/pg_loss:np.float64(0.0020264311460778117) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021731189508500393) - actor/ppo_kl:np.float64(7.550312483939337e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_w_mean:np.float64(530.5515484747011) - self_distillation/token_reweight_w_std:np.float64(11889.211997900158) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006914176046848) - self_distillation/token_reweight_w_clip_frac:np.float64(0.046933838588302024) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.15660720132291317) - perf/mfu/actor:np.float64(0.0544778590588236) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.24555206298828) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003924021031707525 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003924021031707525 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:448.2109375 - response_length/max:1032.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:448.2109375 - response_length_non_aborted/max:1032.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000113682821393013 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9868113528937101) - timing_s/agent_loop/generate_sequences/max:np.float64(9.072009176015854) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.959669176765601) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.072009176015854) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:994 - timing_s/gen:15.984084367752075 - timing_s/reward:0.06343846768140793 - timing_s/old_log_prob:2.9839373845607042 - timing_s/adv:0.5477243233472109 - timing_s/update_actor:21.675551645457745 - timing_s/step:41.344858787953854 - timing_s/stop_profile:0.00011985935270786285 - timing_per_token_ms/gen:0.13930456474309386 - timing_per_token_ms/update_actor:0.11975575224841017 - timing_per_token_ms/adv:0.003026134671914667 - perf/total_num_tokens:180998 - perf/time_per_step:41.344858787953854 - perf/throughput:547.2203960360822 (TaskRunner pid=2320396) Training Progress: 35%|███▌ | 35/100 [34:21<54:43, 50.52s/it] (TaskRunner pid=2320396) step:36 - global_seqlen/min:20139 - global_seqlen/max:31367 - global_seqlen/minmax_diff:11228 - global_seqlen/balanced_min:27336 - global_seqlen/balanced_max:27346 - global_seqlen/mean:27343.75 - actor/entropy:0.2504202723503113 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40465080738067627 - training/rollout_probs_diff_mean:0.0030166946817189455 - training/rollout_probs_diff_std:0.009248757734894753 - training/rollout_actor_probs_pearson_corr:0.9992465972900391 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8515625 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8515625 - rollout_corr/rollout_is_mean:0.9999235272407532 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0037670765304938e-05 - rollout_corr/rollout_is_max:1.5995163917541504 - rollout_corr/rollout_is_min:0.43956276774406433 - rollout_corr/rollout_is_std:0.028227554634213448 - rollout_corr/rollout_is_eff_sample_size:0.9992037204657419 - rollout_corr/rollout_is_seq_mean:0.9999057054519653 - rollout_corr/rollout_is_seq_std:0.0012788664316758513 - rollout_corr/rollout_is_seq_max:1.0031697750091553 - rollout_corr/rollout_is_seq_min:0.9950219392776489 - rollout_corr/rollout_is_seq_max_deviation:0.004978060722351074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2692863936536014) - rollout_corr/training_log_ppl:np.float64(0.22850381008174736) - rollout_corr/kl:np.float64(0.0005316972360809302) - rollout_corr/k3_kl:np.float64(0.0004293374125552418) - rollout_corr/rollout_ppl:np.float64(1.2685975637286901) - rollout_corr/rollout_log_ppl:np.float64(0.22797211322176736) - rollout_corr/log_ppl_diff:np.float64(0.000531696859980002) - rollout_corr/log_ppl_abs_diff:np.float64(0.001130868069594726) - rollout_corr/log_ppl_diff_max:np.float64(0.008397668600082397) - rollout_corr/log_ppl_diff_min:np.float64(-0.0027961134910583496) - rollout_corr/ppl_ratio:np.float64(1.000532856443897) - rollout_corr/chi2_token:np.float64(0.0006337745580822229) - rollout_corr/chi2_seq:np.float64(0.33934940421022475) - actor/pg_loss:np.float64(0.0029051536694169044) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002513188455850468) - actor/ppo_kl:np.float64(3.129442966631757e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(6897.516162533779) - self_distillation/token_reweight_w_std:np.float64(169786.0625592887) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015529317315668) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07552138640312478) - self_distillation/empty_target_batch:np.float64(0.1484375) - actor/grad_norm:np.float64(0.1749717816710472) - perf/mfu/actor:np.float64(0.06304641565475115) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.2574691772461) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006652506534010172 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006652506534010172 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:584.8359375 - response_length/max:1433.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:584.8359375 - response_length_non_aborted/max:1433.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.780377149581909e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0670874547213316) - timing_s/agent_loop/generate_sequences/max:np.float64(12.71995124220848) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.654472718997567) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.71995124220848) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:1433 - timing_s/gen:19.51580721512437 - timing_s/reward:0.0687258280813694 - timing_s/old_log_prob:3.301211541518569 - timing_s/adv:0.6224024724215269 - timing_s/update_actor:22.644038517028093 - timing_s/step:46.24068879336119 - timing_s/stop_profile:0.00012005679309368134 - timing_per_token_ms/gen:0.13035044026185474 - timing_per_token_ms/update_actor:0.10351560464927129 - timing_per_token_ms/adv:0.0028452684453555517 - perf/total_num_tokens:218750 - perf/time_per_step:46.24068879336119 - perf/throughput:591.3352658346596 (TaskRunner pid=2320396) Training Progress: 36%|███▌ | 36/100 [35:07<52:32, 49.25s/it] (TaskRunner pid=2320396) step:37 - global_seqlen/min:20086 - global_seqlen/max:24803 - global_seqlen/minmax_diff:4717 - global_seqlen/balanced_min:23730 - global_seqlen/balanced_max:23741 - global_seqlen/mean:23735.75 - actor/entropy:0.23037061095237732 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6908016204833984 - training/rollout_probs_diff_mean:0.002985295606777072 - training/rollout_probs_diff_std:0.009590771049261093 - training/rollout_actor_probs_pearson_corr:0.9990717768669128 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000369548797607 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.05903447244782e-05 - rollout_corr/rollout_is_max:1.6410112380981445 - rollout_corr/rollout_is_min:0.2311713546514511 - rollout_corr/rollout_is_std:0.028421156108379364 - rollout_corr/rollout_is_eff_sample_size:0.9991930088240167 - rollout_corr/rollout_is_seq_mean:0.9999706745147705 - rollout_corr/rollout_is_seq_std:0.0017531486228108406 - rollout_corr/rollout_is_seq_max:1.00639009475708 - rollout_corr/rollout_is_seq_min:0.9936760663986206 - rollout_corr/rollout_is_seq_max_deviation:0.006390094757080078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2430013921111822) - rollout_corr/training_log_ppl:np.float64(0.20931774778000545) - rollout_corr/kl:np.float64(0.0005845249421625454) - rollout_corr/k3_kl:np.float64(0.0004938606622033603) - rollout_corr/rollout_ppl:np.float64(1.242268514353782) - rollout_corr/rollout_log_ppl:np.float64(0.20873322064289823) - rollout_corr/log_ppl_diff:np.float64(0.0005845271371072158) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014318034081952646) - rollout_corr/log_ppl_diff_max:np.float64(0.008943051099777222) - rollout_corr/log_ppl_diff_min:np.float64(-0.009509354829788208) - rollout_corr/ppl_ratio:np.float64(1.0005866941064596) - rollout_corr/chi2_token:np.float64(0.0007902453653514385) - rollout_corr/chi2_seq:np.float64(0.5003544613718987) - actor/pg_loss:np.float64(0.002713275025598705) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003606915511227271) - actor/ppo_kl:np.float64(7.810856640055874e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(4877.020437426167) - self_distillation/token_reweight_w_std:np.float64(107261.42743945995) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000146112870425) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05535905514261685) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.17725222185254097) - perf/mfu/actor:np.float64(0.055516710245582246) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.18002700805664) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007146945223212242 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.007146945223212242 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:481.1796875 - response_length/max:1472.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:481.1796875 - response_length_non_aborted/max:1472.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.340047836303711e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.017892599105835) - timing_s/agent_loop/generate_sequences/max:np.float64(11.753218919038773) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.032719595379604) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.753218919038773) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:1472 - timing_s/gen:18.823624592274427 - timing_s/reward:0.0637381449341774 - timing_s/old_log_prob:3.0538349244743586 - timing_s/adv:0.6095426809042692 - timing_s/update_actor:22.17010228894651 - timing_s/step:44.80895793251693 - timing_s/stop_profile:0.0001234728842973709 - timing_per_token_ms/gen:0.15281148700519903 - timing_per_token_ms/update_actor:0.11675480177025431 - timing_per_token_ms/adv:0.0032100454004206167 - perf/total_num_tokens:189886 - perf/time_per_step:44.80895793251693 - perf/throughput:529.7099306738276 (TaskRunner pid=2320396) Training Progress: 37%|███▋ | 37/100 [35:52<50:19, 47.93s/it] (TaskRunner pid=2320396) step:38 - global_seqlen/min:19691 - global_seqlen/max:29519 - global_seqlen/minmax_diff:9828 - global_seqlen/balanced_min:23230 - global_seqlen/balanced_max:23236 - global_seqlen/mean:23233.125 - actor/entropy:0.22175385057926178 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424724340438843 - training/rollout_probs_diff_mean:0.002874217461794615 - training/rollout_probs_diff_std:0.00937693938612938 - training/rollout_actor_probs_pearson_corr:0.999193012714386 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999099969863892 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8694218397140503 - rollout_corr/rollout_is_min:0.5457053184509277 - rollout_corr/rollout_is_std:0.02795809879899025 - rollout_corr/rollout_is_eff_sample_size:0.9992187171499983 - rollout_corr/rollout_is_seq_mean:0.9998998045921326 - rollout_corr/rollout_is_seq_std:0.00174828025046736 - rollout_corr/rollout_is_seq_max:1.0071884393692017 - rollout_corr/rollout_is_seq_min:0.9907048940658569 - rollout_corr/rollout_is_seq_max_deviation:0.009295105934143066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2191084562800825) - rollout_corr/training_log_ppl:np.float64(0.18708222404529806) - rollout_corr/kl:np.float64(0.0005426743667129585) - rollout_corr/k3_kl:np.float64(0.00042263647131335347) - rollout_corr/rollout_ppl:np.float64(1.2184375426732004) - rollout_corr/rollout_log_ppl:np.float64(0.1865395491477102) - rollout_corr/log_ppl_diff:np.float64(0.0005426748975878581) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012689346476690844) - rollout_corr/log_ppl_diff_max:np.float64(0.010417088866233826) - rollout_corr/log_ppl_diff_min:np.float64(-0.005928635597229004) - rollout_corr/ppl_ratio:np.float64(1.0005444309208542) - rollout_corr/chi2_token:np.float64(0.0006026304326951504) - rollout_corr/chi2_seq:np.float64(1.5354293449781835) - actor/pg_loss:np.float64(0.0027565507916733623) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002350132526771631) - actor/ppo_kl:np.float64(3.495901299288562e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(6817.583530688891) - self_distillation/token_reweight_w_std:np.float64(138689.49364839046) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017913763877004) - self_distillation/token_reweight_w_clip_frac:np.float64(0.056382376729743555) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.1824763622134924) - perf/mfu/actor:np.float64(0.05564749761275871) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.17073440551758) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007229797076433897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007229797076433897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:459.25390625 - response_length/max:1222.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:459.25390625 - response_length_non_aborted/max:1222.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010543875396251678 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1841194070875645) - timing_s/agent_loop/generate_sequences/max:np.float64(10.229186763986945) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.03015022159525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.229186763986945) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:1222 - timing_s/gen:16.99055285193026 - timing_s/reward:0.06129678152501583 - timing_s/old_log_prob:3.0377829782664776 - timing_s/adv:0.5637715607881546 - timing_s/update_actor:21.657408459112048 - timing_s/step:42.39016137458384 - timing_s/stop_profile:0.00011087954044342041 - timing_per_token_ms/gen:0.14451558533227518 - timing_per_token_ms/update_actor:0.11652225249031312 - timing_per_token_ms/adv:0.0030332314356557428 - perf/total_num_tokens:185865 - perf/time_per_step:42.39016137458384 - perf/throughput:548.0782390682298 (TaskRunner pid=2320396) Training Progress: 38%|███▊ | 38/100 [36:34<47:49, 46.28s/it] (TaskRunner pid=2320396) step:39 - global_seqlen/min:14393 - global_seqlen/max:25409 - global_seqlen/minmax_diff:11016 - global_seqlen/balanced_min:20262 - global_seqlen/balanced_max:20274 - global_seqlen/mean:20270.375 - actor/entropy:0.20056596398353577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28917062282562256 - training/rollout_probs_diff_mean:0.002990458393469453 - training/rollout_probs_diff_std:0.009769958443939686 - training/rollout_actor_probs_pearson_corr:0.9990027546882629 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.80859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.000001072883606 - rollout_corr/rollout_is_ratio_fraction_high:2.082053651975002e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.164107303950004e-05 - rollout_corr/rollout_is_max:2.3594627380371094 - rollout_corr/rollout_is_min:0.1703754961490631 - rollout_corr/rollout_is_std:0.02931111305952072 - rollout_corr/rollout_is_eff_sample_size:0.999141596140458 - rollout_corr/rollout_is_seq_mean:0.9999600052833557 - rollout_corr/rollout_is_seq_std:0.001872700871899724 - rollout_corr/rollout_is_seq_max:1.0076165199279785 - rollout_corr/rollout_is_seq_min:0.993287205696106 - rollout_corr/rollout_is_seq_max_deviation:0.007616519927978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2189809894189239) - rollout_corr/training_log_ppl:np.float64(0.18452461713604862) - rollout_corr/kl:np.float64(0.0004751513001508556) - rollout_corr/k3_kl:np.float64(0.00048812701186662366) - rollout_corr/rollout_ppl:np.float64(1.2183574447408319) - rollout_corr/rollout_log_ppl:np.float64(0.1840494627977023) - rollout_corr/log_ppl_diff:np.float64(0.0004751543383463286) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013876069351681508) - rollout_corr/log_ppl_diff_max:np.float64(0.01167362928390503) - rollout_corr/log_ppl_diff_min:np.float64(-0.006826907396316528) - rollout_corr/ppl_ratio:np.float64(1.0004770983941853) - rollout_corr/chi2_token:np.float64(0.0010034546721726656) - rollout_corr/chi2_seq:np.float64(0.23432688671164215) - actor/pg_loss:np.float64(0.001627549878321588) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016317091376549797) - actor/ppo_kl:np.float64(-2.4777720572188855e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_w_mean:np.float64(10545.353267262224) - self_distillation/token_reweight_w_std:np.float64(130564.52004474157) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002583481837064) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03637115124729462) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.1302459230646491) - perf/mfu/actor:np.float64(0.04886280426766004) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.26232528686523) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027457084506750107 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027457084506750107 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:375.23046875 - response_length/max:1107.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:375.23046875 - response_length_non_aborted/max:1107.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001499038189649582 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0094370376318693) - timing_s/agent_loop/generate_sequences/max:np.float64(8.979632189497352) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.206847889901837) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.979632189497352) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:1107 - timing_s/gen:16.14283774420619 - timing_s/reward:0.05978567153215408 - timing_s/old_log_prob:3.030688526108861 - timing_s/adv:0.5634818132966757 - timing_s/update_actor:21.313188266009092 - timing_s/step:41.19874492101371 - timing_s/stop_profile:0.00010902248322963715 - timing_per_token_ms/gen:0.1680512783206799 - timing_per_token_ms/update_actor:0.1314306485820384 - timing_per_token_ms/adv:0.003474786562265595 - perf/total_num_tokens:162163 - perf/time_per_step:41.19874492101371 - perf/throughput:492.01438147842583 (TaskRunner pid=2320396) Training Progress: 39%|███▉ | 39/100 [37:16<45:31, 44.78s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) We solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) This is closest to option B: 56 pC. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2320396) step:40 - global_seqlen/min:17964 - global_seqlen/max:33806 - global_seqlen/minmax_diff:15842 - global_seqlen/balanced_min:23957 - global_seqlen/balanced_max:24000 - global_seqlen/mean:23991.125 - actor/entropy:0.17409266531467438 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22586679458618164 - training/rollout_probs_diff_mean:0.002714834176003933 - training/rollout_probs_diff_std:0.009270383976399899 - training/rollout_actor_probs_pearson_corr:0.9989473819732666 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.77734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:1.0000334978103638 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.398139076831285e-05 - rollout_corr/rollout_is_max:1.8032424449920654 - rollout_corr/rollout_is_min:0.3792487382888794 - rollout_corr/rollout_is_std:0.027413414791226387 - rollout_corr/rollout_is_eff_sample_size:0.9992491879902002 - rollout_corr/rollout_is_seq_mean:0.9999297857284546 - rollout_corr/rollout_is_seq_std:0.0016750209033489227 - rollout_corr/rollout_is_seq_max:1.0049023628234863 - rollout_corr/rollout_is_seq_min:0.9892888069152832 - rollout_corr/rollout_is_seq_max_deviation:0.010711193084716797 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1875572656281292) - rollout_corr/training_log_ppl:np.float64(0.16661910359107424) - rollout_corr/kl:np.float64(0.0007166074137661127) - rollout_corr/k3_kl:np.float64(0.0006648261616248874) - rollout_corr/rollout_ppl:np.float64(1.1866890429519117) - rollout_corr/rollout_log_ppl:np.float64(0.16590249516593758) - rollout_corr/log_ppl_diff:np.float64(0.0007166084251366556) - rollout_corr/log_ppl_abs_diff:np.float64(0.001578556781169027) - rollout_corr/log_ppl_diff_max:np.float64(0.07116454839706421) - rollout_corr/log_ppl_diff_min:np.float64(-0.004071809351444244) - rollout_corr/ppl_ratio:np.float64(1.0007289675995708) - rollout_corr/chi2_token:np.float64(0.000788895646110177) - rollout_corr/chi2_seq:np.float64(1.0485429209657013) - actor/pg_loss:np.float64(0.0022394302068278193) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00032143645762516826) - actor/ppo_kl:np.float64(0.0002396634859533897) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_w_mean:np.float64(14596.109324033838) - self_distillation/token_reweight_w_std:np.float64(170215.50741613266) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997718355152756) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045137047200114466) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.2175825722515583) - perf/mfu/actor:np.float64(0.056676871566027895) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.3233880996704) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.61796875) - val-aux/sciknoweval/reward/std@16:np.float64(0.20523842662574854) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.705825) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18368841387292015) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5308499999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16680965099369396) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.618025) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2043784429066801) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.783475) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13910305561491254) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46548750000000005) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10917650565978679) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6298874999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1789536010234551) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8374375000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08838687383578706) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.42711249999999995) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06170735155075495) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6369999999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14868065619864407) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8685500000000002) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04271179547353976) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4054375) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03508296536724832) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6418125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11962085895290062) - val-aux/sciknoweval/score/mean@16:np.float64(0.61796875) - val-aux/sciknoweval/score/std@16:np.float64(0.20523842662574854) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.705825) - val-aux/sciknoweval/score/best@2/std:np.float64(0.18368841387292015) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5308499999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16680965099369396) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.618025) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2043784429066801) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.783475) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13910305561491254) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.46548750000000005) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10917650565978679) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6298874999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1789536010234551) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8374375000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08838687383578706) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.42711249999999995) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06170735155075495) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6369999999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14868065619864407) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8685500000000002) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04271179547353976) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4054375) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03508296536724832) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6418125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11962085895290062) - val-core/sciknoweval/acc/mean@16:np.float64(0.61796875) - val-aux/sciknoweval/acc/std@16:np.float64(0.20523842662574854) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.705825) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.18368841387292015) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5308499999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16680965099369396) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.618025) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2043784429066801) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.783475) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13910305561491254) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.46548750000000005) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10917650565978679) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6298874999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1789536010234551) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8374375000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08838687383578706) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.42711249999999995) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06170735155075495) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6369999999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14868065619864407) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8685500000000002) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04271179547353976) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4054375) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03508296536724832) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6418125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11962085895290062) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98984375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.017180441164697836) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9962500000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.009611894943540346) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9837250000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.018769314003885828) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9903625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.016320258609184714) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9991875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003454498723887649) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9764875) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.019219229161406868) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99375) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.012089324810506192) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999500000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.000788986691902975) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9676874999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.017749891876663433) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.996725) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008829929148801091) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9596875) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.014249930113123078) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9987) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0038157568056677834) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009700472466647625 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009700472466647625 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:488.66015625 - response_length/max:1799.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:488.66015625 - response_length_non_aborted/max:1799.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014634616672992706 - timing_s/agent_loop/generate_sequences/min:np.float64(1.002776077017188) - timing_s/agent_loop/generate_sequences/max:np.float64(13.396277887746692) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.303398293312057) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.396277887746692) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1799 - timing_s/gen:20.2838581148535 - timing_s/reward:0.06532224267721176 - timing_s/old_log_prob:3.0899851340800524 - timing_s/adv:0.610138576477766 - timing_s/update_actor:21.942959178239107 - timing_s/step:46.080426421016455 - timing_s/testing:112.31239799968898 - timing_s/save_checkpoint:12.967904256656766 - timing_s/stop_profile:0.0001502949744462967 - timing_per_token_ms/gen:0.16214504036750282 - timing_per_token_ms/update_actor:0.1143285234552314 - timing_per_token_ms/adv:0.0031789806463732215 - perf/total_num_tokens:191929 - perf/time_per_step:46.080426421016455 - perf/throughput:520.6359155795069 (TaskRunner pid=2320396) Training Progress: 40%|████ | 40/100 [40:07<1:22:46, 82.78s/it] (TaskRunner pid=2320396) step:41 - global_seqlen/min:21172 - global_seqlen/max:28257 - global_seqlen/minmax_diff:7085 - global_seqlen/balanced_min:24104 - global_seqlen/balanced_max:24111 - global_seqlen/mean:24108.25 - actor/entropy:0.19510897994041443 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34874776005744934 - training/rollout_probs_diff_mean:0.00296596996486187 - training/rollout_probs_diff_std:0.009650228545069695 - training/rollout_actor_probs_pearson_corr:0.999030590057373 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.9609375 - self_distillation/correct_sample_fraction:0.7734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9609375 - rollout_corr/rollout_is_mean:1.0000007152557373 - rollout_corr/rollout_is_ratio_fraction_high:8.208967301470693e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.208967301470693e-06 - rollout_corr/rollout_is_max:3.2528834342956543 - rollout_corr/rollout_is_min:0.25628554821014404 - rollout_corr/rollout_is_std:0.02881893701851368 - rollout_corr/rollout_is_eff_sample_size:0.9991700390777238 - rollout_corr/rollout_is_seq_mean:0.9999620914459229 - rollout_corr/rollout_is_seq_std:0.001639653230085969 - rollout_corr/rollout_is_seq_max:1.0059510469436646 - rollout_corr/rollout_is_seq_min:0.99149489402771 - rollout_corr/rollout_is_seq_max_deviation:0.008505105972290039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.217791025992483) - rollout_corr/training_log_ppl:np.float64(0.18964533759572078) - rollout_corr/kl:np.float64(0.0005734493415801012) - rollout_corr/k3_kl:np.float64(0.00044943573641376133) - rollout_corr/rollout_ppl:np.float64(1.2171180993318558) - rollout_corr/rollout_log_ppl:np.float64(0.18907188744924497) - rollout_corr/log_ppl_diff:np.float64(0.0005734501464758068) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012757474614772946) - rollout_corr/log_ppl_diff_max:np.float64(0.010674156248569489) - rollout_corr/log_ppl_diff_min:np.float64(-0.005041897296905518) - rollout_corr/ppl_ratio:np.float64(1.0005749089177698) - rollout_corr/chi2_token:np.float64(0.0006430486682802439) - rollout_corr/chi2_seq:np.float64(0.34916975046508014) - actor/pg_loss:np.float64(0.0028609149158000946) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00037430567363117007) - actor/ppo_kl:np.float64(0.00010280042299903869) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_w_mean:np.float64(15439.003869137494) - self_distillation/token_reweight_w_std:np.float64(381955.13341238984) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000298652332276) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05494706741592381) - self_distillation/empty_target_batch:np.float64(0.0390625) - actor/grad_norm:np.float64(0.19181200489401817) - perf/mfu/actor:np.float64(0.05644916039033881) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.62611770629883) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020902082324028015 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020902082324028015 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:475.8515625 - response_length/max:1251.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:475.8515625 - response_length_non_aborted/max:1251.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010300613939762115 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0740045364946127) - timing_s/agent_loop/generate_sequences/max:np.float64(10.960649330168962) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.377686426785658) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.960649330168962) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1251 - timing_s/gen:17.190705524757504 - timing_s/reward:0.06419655494391918 - timing_s/old_log_prob:3.0940296556800604 - timing_s/adv:0.6143710687756538 - timing_s/update_actor:22.368526071310043 - timing_s/step:43.42171490751207 - timing_s/stop_profile:0.00011404231190681458 - timing_per_token_ms/gen:0.141117942543446 - timing_per_token_ms/update_actor:0.11597962352778636 - timing_per_token_ms/adv:0.0031854814678359787 - perf/total_num_tokens:192866 - perf/time_per_step:43.42171490751207 - perf/throughput:555.2118347087487 (TaskRunner pid=2320396) Training Progress: 41%|████ | 41/100 [40:51<1:09:48, 70.99s/it] (TaskRunner pid=2320396) step:42 - global_seqlen/min:22569 - global_seqlen/max:27311 - global_seqlen/minmax_diff:4742 - global_seqlen/balanced_min:25038 - global_seqlen/balanced_max:25054 - global_seqlen/mean:25047.0 - actor/entropy:0.24892261624336243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.670967161655426 - training/rollout_probs_diff_mean:0.003025430254638195 - training/rollout_probs_diff_std:0.009795612655580044 - training/rollout_actor_probs_pearson_corr:0.9991313219070435 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9998930096626282 - rollout_corr/rollout_is_ratio_fraction_high:1.5133171473280527e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.783293141168542e-05 - rollout_corr/rollout_is_max:4.227712631225586 - rollout_corr/rollout_is_min:0.18747776746749878 - rollout_corr/rollout_is_std:0.029035290703177452 - rollout_corr/rollout_is_eff_sample_size:0.9991574240186801 - rollout_corr/rollout_is_seq_mean:0.9998325109481812 - rollout_corr/rollout_is_seq_std:0.0015463787131011486 - rollout_corr/rollout_is_seq_max:1.0083459615707397 - rollout_corr/rollout_is_seq_min:0.9951817393302917 - rollout_corr/rollout_is_seq_max_deviation:0.008345961570739746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2542899283580482) - rollout_corr/training_log_ppl:np.float64(0.2118087612761883) - rollout_corr/kl:np.float64(0.0006046219645430284) - rollout_corr/k3_kl:np.float64(0.0004428104627720586) - rollout_corr/rollout_ppl:np.float64(1.2534998757764697) - rollout_corr/rollout_log_ppl:np.float64(0.21120413720200304) - rollout_corr/log_ppl_diff:np.float64(0.0006046240741852671) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012860278657171875) - rollout_corr/log_ppl_diff_max:np.float64(0.005592525005340576) - rollout_corr/log_ppl_diff_min:np.float64(-0.007412515580654144) - rollout_corr/ppl_ratio:np.float64(1.0006060795858502) - rollout_corr/chi2_token:np.float64(0.0005524898879230022) - rollout_corr/chi2_seq:np.float64(0.7299846305977553) - actor/pg_loss:np.float64(0.004190075560472906) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002905324188304803) - actor/ppo_kl:np.float64(3.560879972663855e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(3159.014352656668) - self_distillation/token_reweight_w_std:np.float64(83420.70022474515) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017929573077708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06845499108021613) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.227945476770401) - perf/mfu/actor:np.float64(0.057657091310341184) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.51451110839844) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014856046065688133 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014856046065688133 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:516.25 - response_length/max:1516.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:516.25 - response_length_non_aborted/max:1516.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010149739682674408 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0408725664019585) - timing_s/agent_loop/generate_sequences/max:np.float64(12.283551223576069) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.583720707065368) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.283551223576069) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:1516 - timing_s/gen:18.965983686968684 - timing_s/reward:0.07659243419766426 - timing_s/old_log_prob:3.0534052439033985 - timing_s/adv:0.6341004334390163 - timing_s/update_actor:22.555470870807767 - timing_s/step:45.37643878906965 - timing_s/stop_profile:0.00011052936315536499 - timing_per_token_ms/gen:0.143507745815441 - timing_per_token_ms/update_actor:0.11256573078017211 - timing_per_token_ms/adv:0.0031645528079161994 - perf/total_num_tokens:200376 - perf/time_per_step:45.37643878906965 - perf/throughput:551.9824972697804 (TaskRunner pid=2320396) Training Progress: 42%|████▏ | 42/100 [41:36<1:01:12, 63.32s/it] (TaskRunner pid=2320396) step:43 - global_seqlen/min:19052 - global_seqlen/max:31919 - global_seqlen/minmax_diff:12867 - global_seqlen/balanced_min:24552 - global_seqlen/balanced_max:24917 - global_seqlen/mean:24612.5 - actor/entropy:0.23190994560718536 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5026711821556091 - training/rollout_probs_diff_mean:0.003001613076776266 - training/rollout_probs_diff_std:0.00961999874562025 - training/rollout_actor_probs_pearson_corr:0.9990864396095276 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999968409538269 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.564602462167386e-05 - rollout_corr/rollout_is_max:1.974767804145813 - rollout_corr/rollout_is_min:0.15729260444641113 - rollout_corr/rollout_is_std:0.02876925654709339 - rollout_corr/rollout_is_eff_sample_size:0.9991730143739226 - rollout_corr/rollout_is_seq_mean:1.0000637769699097 - rollout_corr/rollout_is_seq_std:0.001507977838627994 - rollout_corr/rollout_is_seq_max:1.007026195526123 - rollout_corr/rollout_is_seq_min:0.9949943423271179 - rollout_corr/rollout_is_seq_max_deviation:0.007026195526123047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2377484519965947) - rollout_corr/training_log_ppl:np.float64(0.2034379402466584) - rollout_corr/kl:np.float64(0.0003022583822458458) - rollout_corr/k3_kl:np.float64(0.0004391857282328715) - rollout_corr/rollout_ppl:np.float64(1.2373604252934456) - rollout_corr/rollout_log_ppl:np.float64(0.20313568067649612) - rollout_corr/log_ppl_diff:np.float64(0.00030225957016227767) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011025687199435197) - rollout_corr/log_ppl_diff_max:np.float64(0.003672093152999878) - rollout_corr/log_ppl_diff_min:np.float64(-0.00567355751991272) - rollout_corr/ppl_ratio:np.float64(1.0003033550456166) - rollout_corr/chi2_token:np.float64(0.001162565080448985) - rollout_corr/chi2_seq:np.float64(0.39001374156214297) - actor/pg_loss:np.float64(0.004258588422089815) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024049899820965948) - actor/ppo_kl:np.float64(-5.5626800181229896e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(2705.9410314837005) - self_distillation/token_reweight_w_std:np.float64(74205.18795861094) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006196503527462) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06989276569220237) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.19171869941055775) - perf/mfu/actor:np.float64(0.05718318820002838) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.54368352890015) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008528061211109161 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008528061211109161 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:499.328125 - response_length/max:2008.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:499.328125 - response_length_non_aborted/max:2008.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013203918933868408 - timing_s/agent_loop/generate_sequences/min:np.float64(1.648325640708208) - timing_s/agent_loop/generate_sequences/max:np.float64(14.80245334841311) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.24944398264779) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.80245334841311) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:2008 - timing_s/gen:21.46511185541749 - timing_s/reward:0.06564498879015446 - timing_s/old_log_prob:3.178984245285392 - timing_s/adv:0.6894926764070988 - timing_s/update_actor:22.444915840402246 - timing_s/step:47.931692676618695 - timing_s/stop_profile:0.00011303648352622986 - timing_per_token_ms/gen:0.16792183133130056 - timing_per_token_ms/update_actor:0.11399144662469399 - timing_per_token_ms/adv:0.003501740357577952 - perf/total_num_tokens:196900 - perf/time_per_step:47.931692676618695 - perf/throughput:513.4911501259394 (TaskRunner pid=2320396) Training Progress: 43%|████▎ | 43/100 [42:24<55:46, 58.72s/it] (TaskRunner pid=2320396) step:44 - global_seqlen/min:19418 - global_seqlen/max:31085 - global_seqlen/minmax_diff:11667 - global_seqlen/balanced_min:25774 - global_seqlen/balanced_max:25804 - global_seqlen/mean:25789.125 - actor/entropy:0.23197732865810394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35145077109336853 - training/rollout_probs_diff_mean:0.002846738323569298 - training/rollout_probs_diff_std:0.008968722075223923 - training/rollout_actor_probs_pearson_corr:0.999229371547699 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.000036597251892 - rollout_corr/rollout_is_ratio_fraction_high:7.387652203760808e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.387652203760808e-06 - rollout_corr/rollout_is_max:2.6363072395324707 - rollout_corr/rollout_is_min:0.4945003390312195 - rollout_corr/rollout_is_std:0.027107305824756622 - rollout_corr/rollout_is_eff_sample_size:0.9992657334826961 - rollout_corr/rollout_is_seq_mean:1.0000344514846802 - rollout_corr/rollout_is_seq_std:0.001453010132536292 - rollout_corr/rollout_is_seq_max:1.0084365606307983 - rollout_corr/rollout_is_seq_min:0.9961057901382446 - rollout_corr/rollout_is_seq_max_deviation:0.00843656063079834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.260404644999653) - rollout_corr/training_log_ppl:np.float64(0.21625306174973957) - rollout_corr/kl:np.float64(0.00030714693509725777) - rollout_corr/k3_kl:np.float64(0.0004374984173551866) - rollout_corr/rollout_ppl:np.float64(1.260006621479988) - rollout_corr/rollout_log_ppl:np.float64(0.2159459152026102) - rollout_corr/log_ppl_diff:np.float64(0.0003071465471293777) - rollout_corr/log_ppl_abs_diff:np.float64(0.001081082969903946) - rollout_corr/log_ppl_diff_max:np.float64(0.004448056221008301) - rollout_corr/log_ppl_diff_min:np.float64(-0.0050467848777771) - rollout_corr/ppl_ratio:np.float64(1.0003082156181335) - rollout_corr/chi2_token:np.float64(0.0011717674788087606) - rollout_corr/chi2_seq:np.float64(0.48563542030751705) - actor/pg_loss:np.float64(0.002838829648680985) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029677772226932575) - actor/ppo_kl:np.float64(-4.0592053640153836e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(11010.470829747384) - self_distillation/token_reweight_w_std:np.float64(210548.85107363865) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000365384388715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06577898818068206) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.20856376737356186) - perf/mfu/actor:np.float64(0.05939696075292037) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.48828506469727) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.020655876025557518 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.020655876025557518 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:528.75390625 - response_length/max:1759.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:528.75390625 - response_length_non_aborted/max:1759.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.023956954479218e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3401075471192598) - timing_s/agent_loop/generate_sequences/max:np.float64(13.932623855769634) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.293072881489934) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.932623855769634) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:282 - timing_s/agent_loop/slowest/response_length:1759 - timing_s/gen:21.01163581199944 - timing_s/reward:0.06546073779463768 - timing_s/old_log_prob:3.0930407010018826 - timing_s/adv:0.6480591148138046 - timing_s/update_actor:22.687276219949126 - timing_s/step:47.591358166188 - timing_s/stop_profile:0.00011670403182506561 - timing_per_token_ms/gen:0.15522665917065803 - timing_per_token_ms/update_actor:0.10996532559726788 - timing_per_token_ms/adv:0.0031411453219807022 - perf/total_num_tokens:206313 - perf/time_per_step:47.591358166188 - perf/throughput:541.8867204828433 (TaskRunner pid=2320396) Training Progress: 44%|████▍ | 44/100 [43:12<51:41, 55.39s/it] (TaskRunner pid=2320396) step:45 - global_seqlen/min:19411 - global_seqlen/max:31128 - global_seqlen/minmax_diff:11717 - global_seqlen/balanced_min:24868 - global_seqlen/balanced_max:24877 - global_seqlen/mean:24872.0 - actor/entropy:0.2606814205646515 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37384867668151855 - training/rollout_probs_diff_mean:0.0030074005480855703 - training/rollout_probs_diff_std:0.009297839365899563 - training/rollout_actor_probs_pearson_corr:0.9992638230323792 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.999846875667572 - rollout_corr/rollout_is_ratio_fraction_high:7.83306222729152e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.566612445458304e-05 - rollout_corr/rollout_is_max:2.1816329956054688 - rollout_corr/rollout_is_min:0.03358318656682968 - rollout_corr/rollout_is_std:0.02886233665049076 - rollout_corr/rollout_is_eff_sample_size:0.9991675398426103 - rollout_corr/rollout_is_seq_mean:0.9997819662094116 - rollout_corr/rollout_is_seq_std:0.0016762863378971815 - rollout_corr/rollout_is_seq_max:1.0064592361450195 - rollout_corr/rollout_is_seq_min:0.9929512143135071 - rollout_corr/rollout_is_seq_max_deviation:0.00704878568649292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3119843625463545) - rollout_corr/training_log_ppl:np.float64(0.2495773843547795) - rollout_corr/kl:np.float64(0.0005540147624278902) - rollout_corr/k3_kl:np.float64(0.0005072801855448006) - rollout_corr/rollout_ppl:np.float64(1.3112443177960813) - rollout_corr/rollout_log_ppl:np.float64(0.24902336824743543) - rollout_corr/log_ppl_diff:np.float64(0.0005540161073440686) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013045282248640433) - rollout_corr/log_ppl_diff_max:np.float64(0.0060444176197052) - rollout_corr/log_ppl_diff_min:np.float64(-0.005649633705615997) - rollout_corr/ppl_ratio:np.float64(1.0005556086543947) - rollout_corr/chi2_token:np.float64(0.0009730139281600714) - rollout_corr/chi2_seq:np.float64(1.1036222074180841) - actor/pg_loss:np.float64(0.0031190799782052636) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035551090218177706) - actor/ppo_kl:np.float64(-0.00010970474031912758) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(530.7625214133877) - self_distillation/token_reweight_w_std:np.float64(14348.925151058473) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011988517362624) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07025789209001232) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.22478391602635384) - perf/mfu/actor:np.float64(0.05839612980819302) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.5486536026001) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001738939667120576 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001738939667120576 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:498.6875 - response_length/max:1215.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:498.6875 - response_length_non_aborted/max:1215.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00029014796018600464 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8514383714646101) - timing_s/agent_loop/generate_sequences/max:np.float64(10.650067988783121) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.062247704801848) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.650067988783121) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1204 - timing_s/gen:17.77479348704219 - timing_s/reward:0.06109056994318962 - timing_s/old_log_prob:3.223510628566146 - timing_s/adv:0.6315231639891863 - timing_s/update_actor:22.08882056735456 - timing_s/step:43.915783969685435 - timing_s/stop_profile:4.657916724681854e-05 - timing_per_token_ms/gen:0.13923105563856833 - timing_per_token_ms/update_actor:0.11101248676903024 - timing_per_token_ms/adv:0.0031738660139372906 - perf/total_num_tokens:198976 - perf/time_per_step:43.915783969685435 - perf/throughput:566.356734452671 (TaskRunner pid=2320396) Training Progress: 45%|████▌ | 45/100 [43:56<47:45, 52.10s/it] (TaskRunner pid=2320396) step:46 - global_seqlen/min:15575 - global_seqlen/max:30167 - global_seqlen/minmax_diff:14592 - global_seqlen/balanced_min:22514 - global_seqlen/balanced_max:22538 - global_seqlen/mean:22530.625 - actor/entropy:0.21788272261619568 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967324733734131 - training/rollout_probs_diff_mean:0.002842165995389223 - training/rollout_probs_diff_std:0.009226741269230843 - training/rollout_actor_probs_pearson_corr:0.9992075562477112 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9999492168426514 - rollout_corr/rollout_is_ratio_fraction_high:2.5889521566568874e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.314920079195872e-05 - rollout_corr/rollout_is_max:2.2939116954803467 - rollout_corr/rollout_is_min:0.26859140396118164 - rollout_corr/rollout_is_std:0.02792610228061676 - rollout_corr/rollout_is_eff_sample_size:0.9992207405466251 - rollout_corr/rollout_is_seq_mean:1.0000059604644775 - rollout_corr/rollout_is_seq_std:0.0019772870000451803 - rollout_corr/rollout_is_seq_max:1.0133771896362305 - rollout_corr/rollout_is_seq_min:0.9926490783691406 - rollout_corr/rollout_is_seq_max_deviation:0.013377189636230469 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.231029165443033) - rollout_corr/training_log_ppl:np.float64(0.19566736919659888) - rollout_corr/kl:np.float64(0.0004418784923347907) - rollout_corr/k3_kl:np.float64(0.0004674396851669371) - rollout_corr/rollout_ppl:np.float64(1.230477964039892) - rollout_corr/rollout_log_ppl:np.float64(0.19522549020621227) - rollout_corr/log_ppl_diff:np.float64(0.0004418789903866127) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013463952200254425) - rollout_corr/log_ppl_diff_max:np.float64(0.009284377098083496) - rollout_corr/log_ppl_diff_min:np.float64(-0.01020503044128418) - rollout_corr/ppl_ratio:np.float64(1.000443896977231) - rollout_corr/chi2_token:np.float64(0.0009975701104849577) - rollout_corr/chi2_seq:np.float64(0.27493038563989103) - actor/pg_loss:np.float64(0.0021532203536480665) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017491115090706444) - actor/ppo_kl:np.float64(3.2276031305400465e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_w_mean:np.float64(4078.0635640691034) - self_distillation/token_reweight_w_std:np.float64(119085.47364705714) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001369349192828) - self_distillation/token_reweight_w_clip_frac:np.float64(0.044050666445400566) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.1551105249673128) - perf/mfu/actor:np.float64(0.053146217542449595) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.63757705688477) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01592421345412731 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01592421345412731 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:452.64453125 - response_length/max:1570.0 - response_length/min:60.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:452.64453125 - response_length_non_aborted/max:1570.0 - response_length_non_aborted/min:60.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2947551012039185e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7519746031612158) - timing_s/agent_loop/generate_sequences/max:np.float64(12.470940450206399) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.772140024790133) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.470940450206399) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1570 - timing_s/gen:19.47098889015615 - timing_s/reward:0.05715354159474373 - timing_s/old_log_prob:2.9865529257804155 - timing_s/adv:0.6135058924555779 - timing_s/update_actor:21.954095283523202 - timing_s/step:45.127909518778324 - timing_s/stop_profile:2.9237940907478333e-05 - timing_per_token_ms/gen:0.16803152385854095 - timing_per_token_ms/update_actor:0.1218014107660307 - timing_per_token_ms/adv:0.0034037332101061212 - perf/total_num_tokens:180245 - perf/time_per_step:45.127909518778324 - perf/throughput:499.2614379938142 (TaskRunner pid=2320396) Training Progress: 46%|████▌ | 46/100 [44:41<45:00, 50.01s/it] (TaskRunner pid=2320396) step:47 - global_seqlen/min:21565 - global_seqlen/max:31125 - global_seqlen/minmax_diff:9560 - global_seqlen/balanced_min:25397 - global_seqlen/balanced_max:25437 - global_seqlen/mean:25423.0 - actor/entropy:0.20809735357761383 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37773603200912476 - training/rollout_probs_diff_mean:0.002722726669162512 - training/rollout_probs_diff_std:0.008913216181099415 - training/rollout_actor_probs_pearson_corr:0.999236524105072 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9998671412467957 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9537394046783447 - rollout_corr/rollout_is_min:0.5497885942459106 - rollout_corr/rollout_is_std:0.02662140503525734 - rollout_corr/rollout_is_eff_sample_size:0.9992915645915715 - rollout_corr/rollout_is_seq_mean:0.9997721314430237 - rollout_corr/rollout_is_seq_std:0.0015414318768307567 - rollout_corr/rollout_is_seq_max:1.005649447441101 - rollout_corr/rollout_is_seq_min:0.9940074682235718 - rollout_corr/rollout_is_seq_max_deviation:0.005992531776428223 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2369549586437643) - rollout_corr/training_log_ppl:np.float64(0.20200137386564165) - rollout_corr/kl:np.float64(0.0005988720004523884) - rollout_corr/k3_kl:np.float64(0.0004104271441178753) - rollout_corr/rollout_ppl:np.float64(1.2361970636993647) - rollout_corr/rollout_log_ppl:np.float64(0.201402501334087) - rollout_corr/log_ppl_diff:np.float64(0.0005988725315546617) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012694383185589686) - rollout_corr/log_ppl_diff_max:np.float64(0.0073181986808776855) - rollout_corr/log_ppl_diff_min:np.float64(-0.00609000027179718) - rollout_corr/ppl_ratio:np.float64(1.0006005712784827) - rollout_corr/chi2_token:np.float64(0.0004317518323659897) - rollout_corr/chi2_seq:np.float64(0.32166874944232404) - actor/pg_loss:np.float64(0.0022747202310711145) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001154373726421909) - actor/ppo_kl:np.float64(-2.0942095368292613e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(300.36585143371485) - self_distillation/token_reweight_w_std:np.float64(6000.846875203308) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002250391989946) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03434523963369429) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.13008191250264645) - perf/mfu/actor:np.float64(0.05954238275847209) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.58963775634766) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004297312814742327 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004297312814742327 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:529.71875 - response_length/max:1681.0 - response_length/min:63.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:529.71875 - response_length_non_aborted/max:1681.0 - response_length_non_aborted/min:63.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.859058678150177e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8061241153627634) - timing_s/agent_loop/generate_sequences/max:np.float64(13.01833027601242) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.616919153006165) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.01833027601242) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1681 - timing_s/gen:20.68991795182228 - timing_s/reward:0.06652592867612839 - timing_s/old_log_prob:3.264441102743149 - timing_s/adv:0.6679162699729204 - timing_s/update_actor:22.31015120446682 - timing_s/step:47.09043016284704 - timing_s/stop_profile:0.0001293029636144638 - timing_per_token_ms/gen:0.15257151459959797 - timing_per_token_ms/update_actor:0.10969472133730687 - timing_per_token_ms/adv:0.00328401580248653 - perf/total_num_tokens:203384 - perf/time_per_step:47.09043016284704 - perf/throughput:539.8761470660337 (TaskRunner pid=2320396) Training Progress: 47%|████▋ | 47/100 [45:28<43:25, 49.16s/it] (TaskRunner pid=2320396) step:48 - global_seqlen/min:15612 - global_seqlen/max:27953 - global_seqlen/minmax_diff:12341 - global_seqlen/balanced_min:23320 - global_seqlen/balanced_max:23770 - global_seqlen/mean:23382.0 - actor/entropy:0.20437420904636383 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2153259515762329 - training/rollout_probs_diff_mean:0.002861631801351905 - training/rollout_probs_diff_std:0.009165971539914608 - training/rollout_actor_probs_pearson_corr:0.9991044402122498 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:0.9999480843544006 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.373819345142692e-05 - rollout_corr/rollout_is_max:1.8078258037567139 - rollout_corr/rollout_is_min:0.1487177610397339 - rollout_corr/rollout_is_std:0.026906467974185944 - rollout_corr/rollout_is_eff_sample_size:0.9992764466837457 - rollout_corr/rollout_is_seq_mean:0.9999303221702576 - rollout_corr/rollout_is_seq_std:0.0016568535938858986 - rollout_corr/rollout_is_seq_max:1.0050488710403442 - rollout_corr/rollout_is_seq_min:0.9914178252220154 - rollout_corr/rollout_is_seq_max_deviation:0.00858217477798462 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2299613454379141) - rollout_corr/training_log_ppl:np.float64(0.19597684718610253) - rollout_corr/kl:np.float64(0.0004885526310606236) - rollout_corr/k3_kl:np.float64(0.0004654208404986093) - rollout_corr/rollout_ppl:np.float64(1.2293341453187168) - rollout_corr/rollout_log_ppl:np.float64(0.19548829441191629) - rollout_corr/log_ppl_diff:np.float64(0.0004885527741862461) - rollout_corr/log_ppl_abs_diff:np.float64(0.001387346550473012) - rollout_corr/log_ppl_diff_max:np.float64(0.006539776921272278) - rollout_corr/log_ppl_diff_min:np.float64(-0.006212443113327026) - rollout_corr/ppl_ratio:np.float64(1.0004903129301965) - rollout_corr/chi2_token:np.float64(0.0008849361911416054) - rollout_corr/chi2_seq:np.float64(0.5547701527830213) - actor/pg_loss:np.float64(0.0020671726670116186) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001482926384142047) - actor/ppo_kl:np.float64(7.87036429628074e-06) - actor/pg_clipfrac_lower:np.float64(7.038288458716124e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(3750.297277679201) - self_distillation/token_reweight_w_std:np.float64(98113.6687944993) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994098702445626) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04493811877910048) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.17269694805145264) - perf/mfu/actor:np.float64(0.05559806881598567) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.6170768737793) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016268134117126465 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016268134117126465 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:463.125 - response_length/max:2002.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.125 - response_length_non_aborted/max:2002.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.217990398406982e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2065272722393274) - timing_s/agent_loop/generate_sequences/max:np.float64(14.751506268978119) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.25816673142981) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.751506268978119) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:2002 - timing_s/gen:21.466947183012962 - timing_s/reward:0.06394496373832226 - timing_s/old_log_prob:3.1414225921034813 - timing_s/adv:0.5687259417027235 - timing_s/update_actor:21.953529139980674 - timing_s/step:47.28480998799205 - timing_s/stop_profile:0.00011818297207355499 - timing_per_token_ms/gen:0.18106399445861135 - timing_per_token_ms/update_actor:0.117363405290291 - timing_per_token_ms/adv:0.0030404047007458915 - perf/total_num_tokens:187056 - perf/time_per_step:47.28480998799205 - perf/throughput:494.4928404267215 (TaskRunner pid=2320396) Training Progress: 48%|████▊ | 48/100 [46:16<42:07, 48.61s/it] (TaskRunner pid=2320396) step:49 - global_seqlen/min:16311 - global_seqlen/max:29943 - global_seqlen/minmax_diff:13632 - global_seqlen/balanced_min:23478 - global_seqlen/balanced_max:23546 - global_seqlen/mean:23497.5 - actor/entropy:0.2099330872297287 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583489418029785 - training/rollout_probs_diff_mean:0.002830114448443055 - training/rollout_probs_diff_std:0.009117886424064636 - training/rollout_actor_probs_pearson_corr:0.9990902543067932 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.0000429153442383 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6360359191894531 - rollout_corr/rollout_is_min:0.5087555050849915 - rollout_corr/rollout_is_std:0.027530577033758163 - rollout_corr/rollout_is_eff_sample_size:0.9992427603926878 - rollout_corr/rollout_is_seq_mean:1.0000454187393188 - rollout_corr/rollout_is_seq_std:0.001672466634772718 - rollout_corr/rollout_is_seq_max:1.0071189403533936 - rollout_corr/rollout_is_seq_min:0.9933210611343384 - rollout_corr/rollout_is_seq_max_deviation:0.007118940353393555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2388078104704618) - rollout_corr/training_log_ppl:np.float64(0.19887281095725484) - rollout_corr/kl:np.float64(0.00033442378822035046) - rollout_corr/k3_kl:np.float64(0.0004594542516542788) - rollout_corr/rollout_ppl:np.float64(1.2383832414634526) - rollout_corr/rollout_log_ppl:np.float64(0.19853838607377838) - rollout_corr/log_ppl_diff:np.float64(0.0003344248834764585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013376152055570856) - rollout_corr/log_ppl_diff_max:np.float64(0.006600901484489441) - rollout_corr/log_ppl_diff_min:np.float64(-0.006887331604957581) - rollout_corr/ppl_ratio:np.float64(1.000336108962074) - rollout_corr/chi2_token:np.float64(0.0011650859378278255) - rollout_corr/chi2_seq:np.float64(0.6430712481960654) - actor/pg_loss:np.float64(0.0028300664853304625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023176581339612312) - actor/ppo_kl:np.float64(-4.170142055848203e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_w_mean:np.float64(5994.0513384339865) - self_distillation/token_reweight_w_std:np.float64(110882.28832101094) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000301810214296) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04926858190447092) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.16475248709321022) - perf/mfu/actor:np.float64(0.05549702752766853) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.6996307373047) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001991107128560543 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001991107128560543 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:479.671875 - response_length/max:1847.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:479.671875 - response_length_non_aborted/max:1847.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010908395051956177 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7865940872579813) - timing_s/agent_loop/generate_sequences/max:np.float64(13.637167720124125) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.768220525184006) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.637167720124125) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1815 - timing_s/gen:20.72220243141055 - timing_s/reward:0.06068837456405163 - timing_s/old_log_prob:3.109410423785448 - timing_s/adv:0.724044319242239 - timing_s/update_actor:22.068346058949828 - timing_s/step:46.765394140034914 - timing_s/stop_profile:0.00011418946087360382 - timing_per_token_ms/gen:0.16875307364580727 - timing_per_token_ms/update_actor:0.11739730853787546 - timing_per_token_ms/adv:0.0038517093267487978 - perf/total_num_tokens:187980 - perf/time_per_step:46.765394140034914 - perf/throughput:502.4548692915701 (TaskRunner pid=2320396) Training Progress: 49%|████▉ | 49/100 [47:03<40:51, 48.07s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) We solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is **B: 56 pC**. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2320396) step:50 - global_seqlen/min:19168 - global_seqlen/max:29440 - global_seqlen/minmax_diff:10272 - global_seqlen/balanced_min:25436 - global_seqlen/balanced_max:25450 - global_seqlen/mean:25439.375 - actor/entropy:0.27149584889411926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37974900007247925 - training/rollout_probs_diff_mean:0.0030482823494821787 - training/rollout_probs_diff_std:0.009097245521843433 - training/rollout_actor_probs_pearson_corr:0.999318540096283 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9999427199363708 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4786446627113037e-05 - rollout_corr/rollout_is_max:1.8260700702667236 - rollout_corr/rollout_is_min:0.42270684242248535 - rollout_corr/rollout_is_std:0.02846306934952736 - rollout_corr/rollout_is_eff_sample_size:0.9991903904576372 - rollout_corr/rollout_is_seq_mean:0.9999755024909973 - rollout_corr/rollout_is_seq_std:0.0014398901257663965 - rollout_corr/rollout_is_seq_max:1.0053409337997437 - rollout_corr/rollout_is_seq_min:0.994831383228302 - rollout_corr/rollout_is_seq_max_deviation:0.005340933799743652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3109961580485106) - rollout_corr/training_log_ppl:np.float64(0.253154604928568) - rollout_corr/kl:np.float64(0.00036529192135503763) - rollout_corr/k3_kl:np.float64(0.0004534678223535593) - rollout_corr/rollout_ppl:np.float64(1.3104890743270516) - rollout_corr/rollout_log_ppl:np.float64(0.2527893086516997) - rollout_corr/log_ppl_diff:np.float64(0.000365296276868321) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012172957794973627) - rollout_corr/log_ppl_diff_max:np.float64(0.006820358335971832) - rollout_corr/log_ppl_diff_min:np.float64(-0.006113961338996887) - rollout_corr/ppl_ratio:np.float64(1.0003666609991342) - rollout_corr/chi2_token:np.float64(0.0010919254273176193) - rollout_corr/chi2_seq:np.float64(0.44652247754856944) - actor/pg_loss:np.float64(0.004591187578625977) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002630960186706943) - actor/ppo_kl:np.float64(-8.892219416978264e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(3386.0691014186013) - self_distillation/token_reweight_w_std:np.float64(81954.25167127652) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989137353841215) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09482813796785194) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.2807828411459923) - perf/mfu/actor:np.float64(0.05904541053648982) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.66108226776123) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6421875) - val-aux/sciknoweval/reward/std@16:np.float64(0.20277213467419983) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7293499999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17546086487058532) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5562374999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16899377189330053) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.641375) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.20366229536255137) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8003374999999998) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12873103397768565) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4905375) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1195036280366466) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.662125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1700630423843838) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8505125) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0808261666845073) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.445125) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07380946263817979) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6723125000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12969198776088287) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8791499999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04230892033532858) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4182125) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.040390744834128765) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6767) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09448682040636207) - val-aux/sciknoweval/score/mean@16:np.float64(0.6421875) - val-aux/sciknoweval/score/std@16:np.float64(0.20277213467419983) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7293499999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17546086487058532) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5562374999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16899377189330053) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.641375) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.20366229536255137) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8003374999999998) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12873103397768565) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4905375) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1195036280366466) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.662125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1700630423843838) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8505125) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0808261666845073) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.445125) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07380946263817979) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6723125000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12969198776088287) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8791499999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04230892033532858) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4182125) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.040390744834128765) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6767) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09448682040636207) - val-core/sciknoweval/acc/mean@16:np.float64(0.6421875) - val-aux/sciknoweval/acc/std@16:np.float64(0.20277213467419983) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7293499999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17546086487058532) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5562374999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16899377189330053) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.641375) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.20366229536255137) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8003374999999998) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12873103397768565) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4905375) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1195036280366466) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.662125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1700630423843838) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8505125) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0808261666845073) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.445125) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07380946263817979) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6723125000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12969198776088287) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8791499999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04230892033532858) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4182125) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.040390744834128765) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6767) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09448682040636207) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98671875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03262356191773175) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9967750000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.013393238265410088) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.977025) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04044627703253543) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.986825) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.032195435957587636) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9994125) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0030136505532463915) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.96235) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.046578683223695114) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9938750000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021421915469836496) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.00039508701573197775) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9415125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.049659373319305505) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9979375000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.011968339769143646) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9173) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04406214749247993) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998000000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0015684387141358125) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.019525503739714622 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.019525503739714622 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:528.35546875 - response_length/max:1825.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:528.35546875 - response_length_non_aborted/max:1825.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001132078468799591 - timing_s/agent_loop/generate_sequences/min:np.float64(0.971835458651185) - timing_s/agent_loop/generate_sequences/max:np.float64(14.26541281864047) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.613275811665517) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.26541281864047) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:1825 - timing_s/gen:20.7291506472975 - timing_s/reward:0.06667716056108475 - timing_s/old_log_prob:3.100318057462573 - timing_s/adv:0.6697617433965206 - timing_s/update_actor:22.497836330905557 - timing_s/step:47.153244730085135 - timing_s/testing:112.05036744847894 - timing_s/save_checkpoint:12.590073201805353 - timing_s/stop_profile:0.00015703029930591583 - timing_per_token_ms/gen:0.15325524103606786 - timing_per_token_ms/update_actor:0.11054632990642241 - timing_per_token_ms/adv:0.0032909699206275737 - perf/total_num_tokens:203515 - perf/time_per_step:47.153244730085135 - perf/throughput:539.5042302098235 (TaskRunner pid=2320396) Training Progress: 50%|█████ | 50/100 [49:54<1:11:00, 85.21s/it] (TaskRunner pid=2320396) step:51 - global_seqlen/min:17511 - global_seqlen/max:28028 - global_seqlen/minmax_diff:10517 - global_seqlen/balanced_min:23155 - global_seqlen/balanced_max:23188 - global_seqlen/mean:23179.125 - actor/entropy:0.2698791027069092 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38078969717025757 - training/rollout_probs_diff_mean:0.0030010524205863476 - training/rollout_probs_diff_std:0.00926926452666521 - training/rollout_actor_probs_pearson_corr:0.99923175573349 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.6328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0001506805419922 - rollout_corr/rollout_is_ratio_fraction_high:8.544598131265957e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.544598131265957e-06 - rollout_corr/rollout_is_max:2.1633248329162598 - rollout_corr/rollout_is_min:0.3055838942527771 - rollout_corr/rollout_is_std:0.028341351076960564 - rollout_corr/rollout_is_eff_sample_size:0.999197769525319 - rollout_corr/rollout_is_seq_mean:1.0000841617584229 - rollout_corr/rollout_is_seq_std:0.0014276247238740325 - rollout_corr/rollout_is_seq_max:1.0039843320846558 - rollout_corr/rollout_is_seq_min:0.9958688020706177 - rollout_corr/rollout_is_seq_max_deviation:0.004131197929382324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.283448894508183) - rollout_corr/training_log_ppl:np.float64(0.23242870725516696) - rollout_corr/kl:np.float64(0.00039944934866742443) - rollout_corr/k3_kl:np.float64(0.0004781505269306763) - rollout_corr/rollout_ppl:np.float64(1.2829442485235631) - rollout_corr/rollout_log_ppl:np.float64(0.23202925745863467) - rollout_corr/log_ppl_diff:np.float64(0.00039944979653228074) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011975742381764576) - rollout_corr/log_ppl_diff_max:np.float64(0.005612224340438843) - rollout_corr/log_ppl_diff_min:np.float64(-0.0040902793407440186) - rollout_corr/ppl_ratio:np.float64(1.000400707591325) - rollout_corr/chi2_token:np.float64(0.0011282062623649836) - rollout_corr/chi2_seq:np.float64(0.39218471315689385) - actor/pg_loss:np.float64(0.002458873437717557) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036319168407317193) - actor/ppo_kl:np.float64(6.448495334332449e-05) - actor/pg_clipfrac_lower:np.float64(7.689468475291505e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_w_mean:np.float64(5380.610399987083) - self_distillation/token_reweight_w_std:np.float64(106550.12923576194) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000015466939658) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06583685430814512) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.19366929307579994) - perf/mfu/actor:np.float64(0.05463014221907142) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.95783233642578) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001417335239239037 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001417335239239037 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:457.16015625 - response_length/max:1647.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:457.16015625 - response_length_non_aborted/max:1647.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.342469274997711e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1378375869244337) - timing_s/agent_loop/generate_sequences/max:np.float64(12.7565373852849) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.867362762641278) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.7565373852849) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1647 - timing_s/gen:19.348504519090056 - timing_s/reward:0.06200408935546875 - timing_s/old_log_prob:3.208234280347824 - timing_s/adv:0.5986099690198898 - timing_s/update_actor:22.022641595453024 - timing_s/step:45.33466789312661 - timing_s/stop_profile:9.629316627979279e-05 - timing_per_token_ms/gen:0.16532520331094697 - timing_per_token_ms/update_actor:0.11876333551985366 - timing_per_token_ms/adv:0.0032281738904072622 - perf/total_num_tokens:185433 - perf/time_per_step:45.33466789312661 - perf/throughput:511.2891761916776 (TaskRunner pid=2320396) Training Progress: 51%|█████ | 51/100 [50:40<59:49, 73.26s/it] (TaskRunner pid=2320396) step:52 - global_seqlen/min:20249 - global_seqlen/max:26904 - global_seqlen/minmax_diff:6655 - global_seqlen/balanced_min:23741 - global_seqlen/balanced_max:23754 - global_seqlen/mean:23748.875 - actor/entropy:0.26206788420677185 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2897759675979614 - training/rollout_probs_diff_mean:0.0031796968542039394 - training/rollout_probs_diff_std:0.009452592581510544 - training/rollout_actor_probs_pearson_corr:0.9992661476135254 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9998950362205505 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.323414476762991e-06 - rollout_corr/rollout_is_max:1.6328537464141846 - rollout_corr/rollout_is_min:0.2245272547006607 - rollout_corr/rollout_is_std:0.02913775108754635 - rollout_corr/rollout_is_eff_sample_size:0.9991514736296966 - rollout_corr/rollout_is_seq_mean:0.9998642206192017 - rollout_corr/rollout_is_seq_std:0.0018541476456448436 - rollout_corr/rollout_is_seq_max:1.0078346729278564 - rollout_corr/rollout_is_seq_min:0.990766167640686 - rollout_corr/rollout_is_seq_max_deviation:0.009233832359313965 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2924986188299954) - rollout_corr/training_log_ppl:np.float64(0.24121209807344712) - rollout_corr/kl:np.float64(0.0005627371857137575) - rollout_corr/k3_kl:np.float64(0.000526310548963238) - rollout_corr/rollout_ppl:np.float64(1.2917516590096056) - rollout_corr/rollout_log_ppl:np.float64(0.24064936087233946) - rollout_corr/log_ppl_diff:np.float64(0.0005627372011076659) - rollout_corr/log_ppl_abs_diff:np.float64(0.001493871764978394) - rollout_corr/log_ppl_diff_max:np.float64(0.008751854300498962) - rollout_corr/log_ppl_diff_min:np.float64(-0.008186154067516327) - rollout_corr/ppl_ratio:np.float64(1.000564940739423) - rollout_corr/chi2_token:np.float64(0.0010216198861598969) - rollout_corr/chi2_seq:np.float64(0.5899208602495492) - actor/pg_loss:np.float64(0.0034945248626172543) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027937310051129316) - actor/ppo_kl:np.float64(-5.135034418657369e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(15624.9816659051) - self_distillation/token_reweight_w_std:np.float64(329907.70011061645) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006199525669217) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06881534481362905) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.2124406099319458) - perf/mfu/actor:np.float64(0.05564996634226216) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.88951110839844) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004192920168861747 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004192920168861747 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:469.30859375 - response_length/max:1387.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:469.30859375 - response_length_non_aborted/max:1387.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.375089287757874e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0267762243747711) - timing_s/agent_loop/generate_sequences/max:np.float64(11.22788168117404) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.981896769080777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.22788168117404) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:1387 - timing_s/gen:18.050167409703135 - timing_s/reward:0.06669834069907665 - timing_s/old_log_prob:2.978366069495678 - timing_s/adv:0.6107301041483879 - timing_s/update_actor:21.93880164436996 - timing_s/step:43.7340450193733 - timing_s/stop_profile:0.00011033564805984497 - timing_per_token_ms/gen:0.15023902690712848 - timing_per_token_ms/update_actor:0.11547284684206073 - timing_per_token_ms/adv:0.003214521235997431 - perf/total_num_tokens:189991 - perf/time_per_step:43.7340450193733 - perf/throughput:543.0294634187103 (TaskRunner pid=2320396) Training Progress: 52%|█████▏ | 52/100 [51:24<51:31, 64.41s/it] (TaskRunner pid=2320396) step:53 - global_seqlen/min:17256 - global_seqlen/max:25754 - global_seqlen/minmax_diff:8498 - global_seqlen/balanced_min:22977 - global_seqlen/balanced_max:22994 - global_seqlen/mean:22989.0 - actor/entropy:0.230458065867424 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44532376527786255 - training/rollout_probs_diff_mean:0.0028626120183616877 - training/rollout_probs_diff_std:0.009131461381912231 - training/rollout_actor_probs_pearson_corr:0.999158501625061 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000241994857788 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.838919711706694e-06 - rollout_corr/rollout_is_max:1.7926321029663086 - rollout_corr/rollout_is_min:0.4609784185886383 - rollout_corr/rollout_is_std:0.02715124748647213 - rollout_corr/rollout_is_eff_sample_size:0.9992634718362852 - rollout_corr/rollout_is_seq_mean:1.000015377998352 - rollout_corr/rollout_is_seq_std:0.0015237050829455256 - rollout_corr/rollout_is_seq_max:1.0050256252288818 - rollout_corr/rollout_is_seq_min:0.9932184815406799 - rollout_corr/rollout_is_seq_max_deviation:0.006781518459320068 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2424569255672395) - rollout_corr/training_log_ppl:np.float64(0.20219183068547864) - rollout_corr/kl:np.float64(0.00047282653207858516) - rollout_corr/k3_kl:np.float64(0.00042950060326774064) - rollout_corr/rollout_ppl:np.float64(1.241812152788043) - rollout_corr/rollout_log_ppl:np.float64(0.20171900314744562) - rollout_corr/log_ppl_diff:np.float64(0.000472827538033016) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012220797216286883) - rollout_corr/log_ppl_diff_max:np.float64(0.01025158166885376) - rollout_corr/log_ppl_diff_min:np.float64(-0.00387602299451828) - rollout_corr/ppl_ratio:np.float64(1.000474382424727) - rollout_corr/chi2_token:np.float64(0.0006975317373871803) - rollout_corr/chi2_seq:np.float64(0.14690965181216598) - actor/pg_loss:np.float64(0.0025837309658527374) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020541453568512225) - actor/ppo_kl:np.float64(0.00013539861986799906) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(1052.9588372900616) - self_distillation/token_reweight_w_std:np.float64(28003.405932930298) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998704448807985) - self_distillation/token_reweight_w_clip_frac:np.float64(0.055031316907843575) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.15601524617522955) - perf/mfu/actor:np.float64(0.054306824026305565) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7872657775879) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014584216987714171 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014584216987714171 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:441.9375 - response_length/max:1407.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:441.9375 - response_length_non_aborted/max:1407.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001513790339231491 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0363005455583334) - timing_s/agent_loop/generate_sequences/max:np.float64(11.363907653838396) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.789611810541828) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.363907653838396) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:256 - timing_s/agent_loop/slowest/response_length:1407 - timing_s/gen:18.20484925620258 - timing_s/reward:0.0619586817920208 - timing_s/old_log_prob:3.02254974283278 - timing_s/adv:0.648742325603962 - timing_s/update_actor:21.905866038054228 - timing_s/step:43.93234056420624 - timing_s/stop_profile:0.00011724047362804413 - timing_per_token_ms/gen:0.16091119764003128 - timing_per_token_ms/update_actor:0.11911058570432723 - timing_per_token_ms/adv:0.0035274605550696093 - perf/total_num_tokens:183912 - perf/time_per_step:43.93234056420624 - perf/throughput:523.2819309137885 (TaskRunner pid=2320396) Training Progress: 53%|█████▎ | 53/100 [52:08<45:39, 58.29s/it] (TaskRunner pid=2320396) step:54 - global_seqlen/min:17834 - global_seqlen/max:25657 - global_seqlen/minmax_diff:7823 - global_seqlen/balanced_min:21174 - global_seqlen/balanced_max:21207 - global_seqlen/mean:21197.75 - actor/entropy:0.23790472745895386 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3006957769393921 - training/rollout_probs_diff_mean:0.0029614968225359917 - training/rollout_probs_diff_std:0.009239520877599716 - training/rollout_actor_probs_pearson_corr:0.9991896152496338 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000056028366089 - rollout_corr/rollout_is_ratio_fraction_high:9.66090192378033e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.932180384756066e-05 - rollout_corr/rollout_is_max:2.61974835395813 - rollout_corr/rollout_is_min:0.3203427791595459 - rollout_corr/rollout_is_std:0.02868834137916565 - rollout_corr/rollout_is_eff_sample_size:0.9991775368580812 - rollout_corr/rollout_is_seq_mean:0.9999408721923828 - rollout_corr/rollout_is_seq_std:0.0016452306881546974 - rollout_corr/rollout_is_seq_max:1.0042672157287598 - rollout_corr/rollout_is_seq_min:0.9935493469238281 - rollout_corr/rollout_is_seq_max_deviation:0.006450653076171875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.244489336386323) - rollout_corr/training_log_ppl:np.float64(0.20888344780541956) - rollout_corr/kl:np.float64(0.0004943577070619654) - rollout_corr/k3_kl:np.float64(0.00044278011073117796) - rollout_corr/rollout_ppl:np.float64(1.243890237994492) - rollout_corr/rollout_log_ppl:np.float64(0.20838908667792566) - rollout_corr/log_ppl_diff:np.float64(0.000494361127493903) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013888106041122228) - rollout_corr/log_ppl_diff_max:np.float64(0.008533328771591187) - rollout_corr/log_ppl_diff_min:np.float64(-0.005737587809562683) - rollout_corr/ppl_ratio:np.float64(1.0004961658269167) - rollout_corr/chi2_token:np.float64(0.0007693876978009939) - rollout_corr/chi2_seq:np.float64(0.3788678168784827) - actor/pg_loss:np.float64(0.0019641465041786432) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015006033072495484) - actor/ppo_kl:np.float64(-2.1946025228203325e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(1024.2289772662334) - self_distillation/token_reweight_w_std:np.float64(25291.225553450407) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008950042538345) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04413453795132227) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.13687882665544748) - perf/mfu/actor:np.float64(0.050549113817286606) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.81171035766602) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004241135902702808 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.004241135902702808 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:404.3359375 - response_length/max:1449.0 - response_length/min:65.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:404.3359375 - response_length_non_aborted/max:1449.0 - response_length_non_aborted/min:65.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.37432050704956e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7421483434736729) - timing_s/agent_loop/generate_sequences/max:np.float64(10.745056714862585) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.232009753388411) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.745056714862585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1449 - timing_s/gen:18.027256151661277 - timing_s/reward:0.07766947150230408 - timing_s/old_log_prob:3.0949815083295107 - timing_s/adv:0.6324186753481627 - timing_s/update_actor:21.854929784312844 - timing_s/step:43.77818885631859 - timing_s/stop_profile:8.252449333667755e-05 - timing_per_token_ms/gen:0.1741595609280386 - timing_per_token_ms/update_actor:0.12887529209652465 - timing_per_token_ms/adv:0.0037292794951596436 - perf/total_num_tokens:169582 - perf/time_per_step:43.77818885631859 - perf/throughput:484.2080166809022 (TaskRunner pid=2320396) Training Progress: 54%|█████▍ | 54/100 [52:51<41:21, 53.95s/it] (TaskRunner pid=2320396) step:55 - global_seqlen/min:15068 - global_seqlen/max:26886 - global_seqlen/minmax_diff:11818 - global_seqlen/balanced_min:21975 - global_seqlen/balanced_max:21993 - global_seqlen/mean:21988.75 - actor/entropy:0.17557011544704437 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29323479533195496 - training/rollout_probs_diff_mean:0.0027691253926604986 - training/rollout_probs_diff_std:0.00955580361187458 - training/rollout_actor_probs_pearson_corr:0.9989780187606812 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.80078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:1.000060796737671 - rollout_corr/rollout_is_ratio_fraction_high:9.256002158508636e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.7768008294515312e-05 - rollout_corr/rollout_is_max:2.0412697792053223 - rollout_corr/rollout_is_min:0.40601131319999695 - rollout_corr/rollout_is_std:0.02833504043519497 - rollout_corr/rollout_is_eff_sample_size:0.9991978885434327 - rollout_corr/rollout_is_seq_mean:1.0000414848327637 - rollout_corr/rollout_is_seq_std:0.00220958748832345 - rollout_corr/rollout_is_seq_max:1.0099327564239502 - rollout_corr/rollout_is_seq_min:0.9892379641532898 - rollout_corr/rollout_is_seq_max_deviation:0.010762035846710205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2014198061078787) - rollout_corr/training_log_ppl:np.float64(0.17472496219124878) - rollout_corr/kl:np.float64(0.0003758088305049201) - rollout_corr/k3_kl:np.float64(0.0004877579008848443) - rollout_corr/rollout_ppl:np.float64(1.2009721314534545) - rollout_corr/rollout_log_ppl:np.float64(0.17434915404010098) - rollout_corr/log_ppl_diff:np.float64(0.000375808151147794) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015464641401194967) - rollout_corr/log_ppl_diff_max:np.float64(0.014139816164970398) - rollout_corr/log_ppl_diff_min:np.float64(-0.009086787700653076) - rollout_corr/ppl_ratio:np.float64(1.0003787553869188) - rollout_corr/chi2_token:np.float64(0.0011782660149037838) - rollout_corr/chi2_seq:np.float64(0.4314419652801007) - actor/pg_loss:np.float64(0.0012743137776851654) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001162844778264116) - actor/ppo_kl:np.float64(-5.910023932820252e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_w_mean:np.float64(9333.47411274095) - self_distillation/token_reweight_w_std:np.float64(244293.20970271114) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000400455435738) - self_distillation/token_reweight_w_clip_frac:np.float64(0.033853263084893115) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.10476882196962833) - perf/mfu/actor:np.float64(0.052601070214590546) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.73566818237305) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.80078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00026263907784596086 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00026263907784596086 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:422.0234375 - response_length/max:1367.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:422.0234375 - response_length_non_aborted/max:1367.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.639561176300049e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0234655812382698) - timing_s/agent_loop/generate_sequences/max:np.float64(10.340585360303521) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.485789840742655) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.340585360303521) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:1367 - timing_s/gen:17.07848149165511 - timing_s/reward:0.06134553626179695 - timing_s/old_log_prob:3.1349700447171926 - timing_s/adv:0.5768703576177359 - timing_s/update_actor:21.743792919442058 - timing_s/step:42.68766647577286 - timing_s/stop_profile:0.00012461841106414795 - timing_per_token_ms/gen:0.1580784676841029 - timing_per_token_ms/update_actor:0.12360748632506428 - timing_per_token_ms/adv:0.0032793494265120566 - perf/total_num_tokens:175910 - perf/time_per_step:42.68766647577286 - perf/throughput:515.1078008089196 (TaskRunner pid=2320396) Training Progress: 55%|█████▌ | 55/100 [53:34<37:56, 50.59s/it] (TaskRunner pid=2320396) step:56 - global_seqlen/min:18311 - global_seqlen/max:32456 - global_seqlen/minmax_diff:14145 - global_seqlen/balanced_min:22088 - global_seqlen/balanced_max:22120 - global_seqlen/mean:22110.5 - actor/entropy:0.21576471626758575 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19307291507720947 - training/rollout_probs_diff_mean:0.0028168810531497 - training/rollout_probs_diff_std:0.009117874316871166 - training/rollout_actor_probs_pearson_corr:0.9991778135299683 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.921875 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.921875 - rollout_corr/rollout_is_mean:1.0000052452087402 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9838331937789917 - rollout_corr/rollout_is_min:0.5898699164390564 - rollout_corr/rollout_is_std:0.027591131627559662 - rollout_corr/rollout_is_eff_sample_size:0.9992394275969021 - rollout_corr/rollout_is_seq_mean:1.0000158548355103 - rollout_corr/rollout_is_seq_std:0.0018200587946921587 - rollout_corr/rollout_is_seq_max:1.0058069229125977 - rollout_corr/rollout_is_seq_min:0.9924702644348145 - rollout_corr/rollout_is_seq_max_deviation:0.007529735565185547 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2427338524721563) - rollout_corr/training_log_ppl:np.float64(0.20696487333043478) - rollout_corr/kl:np.float64(0.0005580308752195229) - rollout_corr/k3_kl:np.float64(0.0004463659485622884) - rollout_corr/rollout_ppl:np.float64(1.2419886952266097) - rollout_corr/rollout_log_ppl:np.float64(0.20640684351383243) - rollout_corr/log_ppl_diff:np.float64(0.0005580298166023567) - rollout_corr/log_ppl_abs_diff:np.float64(0.00135869525547605) - rollout_corr/log_ppl_diff_max:np.float64(0.0075851380825042725) - rollout_corr/log_ppl_diff_min:np.float64(-0.0047438740730285645) - rollout_corr/ppl_ratio:np.float64(1.0005598990246654) - rollout_corr/chi2_token:np.float64(0.0006612758152186871) - rollout_corr/chi2_seq:np.float64(0.2137361967470497) - actor/pg_loss:np.float64(0.00270246050786227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002699654398838902) - actor/ppo_kl:np.float64(0.00015283571616642178) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(3888.0846324316226) - self_distillation/token_reweight_w_std:np.float64(94961.07842942793) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017440444789827) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060424512790632434) - self_distillation/empty_target_batch:np.float64(0.078125) - actor/grad_norm:np.float64(0.18024816177785397) - perf/mfu/actor:np.float64(0.052331017405947575) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7208366394043) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007848478853702545 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007848478853702545 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:437.671875 - response_length/max:1662.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.671875 - response_length_non_aborted/max:1662.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.252876043319702e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9589528422802687) - timing_s/agent_loop/generate_sequences/max:np.float64(12.19803000986576) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.432029293398955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.19803000986576) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1662 - timing_s/gen:19.05821064300835 - timing_s/reward:0.0605444461107254 - timing_s/old_log_prob:3.044057449325919 - timing_s/adv:0.641364049166441 - timing_s/update_actor:22.09057642146945 - timing_s/step:44.98519729450345 - timing_s/stop_profile:3.0197203159332275e-05 - timing_per_token_ms/gen:0.17009577168798284 - timing_per_token_ms/update_actor:0.12488736359122052 - timing_per_token_ms/adv:0.0036259019988605016 - perf/total_num_tokens:176884 - perf/time_per_step:44.98519729450345 - perf/throughput:491.50612489814705 (TaskRunner pid=2320396) Training Progress: 56%|█████▌ | 56/100 [54:19<35:52, 48.92s/it] (TaskRunner pid=2320396) step:57 - global_seqlen/min:20660 - global_seqlen/max:27723 - global_seqlen/minmax_diff:7063 - global_seqlen/balanced_min:24017 - global_seqlen/balanced_max:24026 - global_seqlen/mean:24022.5 - actor/entropy:0.21244722604751587 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48743531107902527 - training/rollout_probs_diff_mean:0.0028552731964737177 - training/rollout_probs_diff_std:0.009358067996799946 - training/rollout_actor_probs_pearson_corr:0.9991409778594971 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.999923825263977 - rollout_corr/rollout_is_ratio_fraction_high:8.303716640511993e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4911150831030682e-05 - rollout_corr/rollout_is_max:2.55444073677063 - rollout_corr/rollout_is_min:0.3948735296726227 - rollout_corr/rollout_is_std:0.028060244396328926 - rollout_corr/rollout_is_eff_sample_size:0.9992131230960879 - rollout_corr/rollout_is_seq_mean:0.9999536871910095 - rollout_corr/rollout_is_seq_std:0.001707185059785843 - rollout_corr/rollout_is_seq_max:1.0068445205688477 - rollout_corr/rollout_is_seq_min:0.9935453534126282 - rollout_corr/rollout_is_seq_max_deviation:0.006844520568847656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2550159809179604) - rollout_corr/training_log_ppl:np.float64(0.21568975165428128) - rollout_corr/kl:np.float64(0.0006072405742267861) - rollout_corr/k3_kl:np.float64(0.0005212998507886368) - rollout_corr/rollout_ppl:np.float64(1.2542184651829302) - rollout_corr/rollout_log_ppl:np.float64(0.2150825107237324) - rollout_corr/log_ppl_diff:np.float64(0.0006072409305488691) - rollout_corr/log_ppl_abs_diff:np.float64(0.001481423867517151) - rollout_corr/log_ppl_diff_max:np.float64(0.010948315262794495) - rollout_corr/log_ppl_diff_min:np.float64(-0.006019413471221924) - rollout_corr/ppl_ratio:np.float64(1.0006093410775065) - rollout_corr/chi2_token:np.float64(0.0008280288893729448) - rollout_corr/chi2_seq:np.float64(0.5540966941043735) - actor/pg_loss:np.float64(0.001621344476006925) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026958688931699726) - actor/ppo_kl:np.float64(0.000122946722594941) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(1276.8657940202393) - self_distillation/token_reweight_w_std:np.float64(25753.365622778947) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988068183884025) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05870081028842833) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.16179810464382172) - perf/mfu/actor:np.float64(0.05671102563507794) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.77534103393555) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0032415634486824274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0032415634486824274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.421875 - response_length/max:1224.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.421875 - response_length_non_aborted/max:1224.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.60230153799057e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7831499483436346) - timing_s/agent_loop/generate_sequences/max:np.float64(10.127770842984319) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.204904470600013) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.127770842984319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1224 - timing_s/gen:18.107756910845637 - timing_s/reward:0.05700390413403511 - timing_s/old_log_prob:3.0384509321302176 - timing_s/adv:0.5749542769044638 - timing_s/update_actor:22.06851520948112 - timing_s/step:43.88189716823399 - timing_s/stop_profile:3.085285425186157e-05 - timing_per_token_ms/gen:0.15036168424988905 - timing_per_token_ms/update_actor:0.11483252788781934 - timing_per_token_ms/adv:0.002991748761080569 - perf/total_num_tokens:192180 - perf/time_per_step:43.88189716823399 - perf/throughput:547.4353104630543 (TaskRunner pid=2320396) Training Progress: 57%|█████▋ | 57/100 [55:03<33:59, 47.42s/it] (TaskRunner pid=2320396) step:58 - global_seqlen/min:15671 - global_seqlen/max:26285 - global_seqlen/minmax_diff:10614 - global_seqlen/balanced_min:22595 - global_seqlen/balanced_max:22603 - global_seqlen/mean:22600.5 - actor/entropy:0.2078225016593933 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45647239685058594 - training/rollout_probs_diff_mean:0.002780613023787737 - training/rollout_probs_diff_std:0.009222094900906086 - training/rollout_actor_probs_pearson_corr:0.9990860223770142 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9997680187225342 - rollout_corr/rollout_is_ratio_fraction_high:1.7986258171731606e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.697938725759741e-05 - rollout_corr/rollout_is_max:2.3788044452667236 - rollout_corr/rollout_is_min:0.22833219170570374 - rollout_corr/rollout_is_std:0.028010282665491104 - rollout_corr/rollout_is_eff_sample_size:0.9992158606040054 - rollout_corr/rollout_is_seq_mean:0.9997212886810303 - rollout_corr/rollout_is_seq_std:0.001959129935130477 - rollout_corr/rollout_is_seq_max:1.0098477602005005 - rollout_corr/rollout_is_seq_min:0.9926185607910156 - rollout_corr/rollout_is_seq_max_deviation:0.009847760200500488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.232383776921779) - rollout_corr/training_log_ppl:np.float64(0.19788227328535868) - rollout_corr/kl:np.float64(0.0007298278635232691) - rollout_corr/k3_kl:np.float64(0.00048722387552402324) - rollout_corr/rollout_ppl:np.float64(1.2314080968499184) - rollout_corr/rollout_log_ppl:np.float64(0.19715244446706492) - rollout_corr/log_ppl_diff:np.float64(0.000729828818293754) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014633310711360537) - rollout_corr/log_ppl_diff_max:np.float64(0.008250519633293152) - rollout_corr/log_ppl_diff_min:np.float64(-0.008565150201320648) - rollout_corr/ppl_ratio:np.float64(1.000732037704438) - rollout_corr/chi2_token:np.float64(0.0004544253461062908) - rollout_corr/chi2_seq:np.float64(0.14290119265206158) - actor/pg_loss:np.float64(0.0023564978037029505) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047837036345299566) - actor/ppo_kl:np.float64(1.843932994916031e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(189.77452703402378) - self_distillation/token_reweight_w_std:np.float64(4119.465276905161) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004234984517097) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06107424577930942) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.18996725231409073) - perf/mfu/actor:np.float64(0.05371021553273776) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.75103378295898) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012600498273968697 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012600498273968697 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.359375 - response_length/max:1079.0 - response_length/min:56.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.359375 - response_length_non_aborted/max:1079.0 - response_length_non_aborted/min:56.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.9674341678619385e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7030046246945858) - timing_s/agent_loop/generate_sequences/max:np.float64(9.170371565967798) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.49670002021594) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.170371565967798) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1079 - timing_s/gen:15.731107097119093 - timing_s/reward:0.06631920486688614 - timing_s/old_log_prob:3.030934613198042 - timing_s/adv:0.5863243769854307 - timing_s/update_actor:22.018998054787517 - timing_s/step:41.53536947630346 - timing_s/stop_profile:0.00011384859681129456 - timing_per_token_ms/gen:0.14147187935824215 - timing_per_token_ms/update_actor:0.12178379933401648 - timing_per_token_ms/adv:0.0032428728179986657 - perf/total_num_tokens:180804 - perf/time_per_step:41.53536947630346 - perf/throughput:544.1266150983421 (TaskRunner pid=2320396) Training Progress: 58%|█████▊ | 58/100 [55:45<31:58, 45.67s/it] (TaskRunner pid=2320396) step:59 - global_seqlen/min:16079 - global_seqlen/max:27248 - global_seqlen/minmax_diff:11169 - global_seqlen/balanced_min:22015 - global_seqlen/balanced_max:22027 - global_seqlen/mean:22023.625 - actor/entropy:0.1970500946044922 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20963798463344574 - training/rollout_probs_diff_mean:0.002791076432913542 - training/rollout_probs_diff_std:0.009237607941031456 - training/rollout_actor_probs_pearson_corr:0.9990670084953308 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.000091314315796 - rollout_corr/rollout_is_ratio_fraction_high:9.29601264942903e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.29601264942903e-06 - rollout_corr/rollout_is_max:2.8204736709594727 - rollout_corr/rollout_is_min:0.44322657585144043 - rollout_corr/rollout_is_std:0.028172601014375687 - rollout_corr/rollout_is_eff_sample_size:0.9992070530233411 - rollout_corr/rollout_is_seq_mean:0.9999852180480957 - rollout_corr/rollout_is_seq_std:0.0020917868241667747 - rollout_corr/rollout_is_seq_max:1.011966347694397 - rollout_corr/rollout_is_seq_min:0.9925203323364258 - rollout_corr/rollout_is_seq_max_deviation:0.011966347694396973 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2375959348864853) - rollout_corr/training_log_ppl:np.float64(0.19846132550446782) - rollout_corr/kl:np.float64(0.000528362165669094) - rollout_corr/k3_kl:np.float64(0.0005359913304374686) - rollout_corr/rollout_ppl:np.float64(1.2368552833795547) - rollout_corr/rollout_log_ppl:np.float64(0.19793296113493852) - rollout_corr/log_ppl_diff:np.float64(0.0005283643695292994) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013686856691492721) - rollout_corr/log_ppl_diff_max:np.float64(0.01003563404083252) - rollout_corr/log_ppl_diff_min:np.float64(-0.009744033217430115) - rollout_corr/ppl_ratio:np.float64(1.0005305462982506) - rollout_corr/chi2_token:np.float64(0.0012348862364888191) - rollout_corr/chi2_seq:np.float64(0.18860497209243476) - actor/pg_loss:np.float64(0.0033428192837163806) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003301967922197946) - actor/ppo_kl:np.float64(7.350512616555172e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(5227.270080071641) - self_distillation/token_reweight_w_std:np.float64(87981.93152151996) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998072686139494) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06207750209432561) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.25658440589904785) - perf/mfu/actor:np.float64(0.05250423204988193) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7232894897461) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007577412761747837 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007577412761747837 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.20703125 - response_length/max:1145.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.20703125 - response_length_non_aborted/max:1145.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014202482998371124 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9425349365919828) - timing_s/agent_loop/generate_sequences/max:np.float64(9.21257615275681) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.516783446379122) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.21257615275681) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1145 - timing_s/gen:16.408081464469433 - timing_s/reward:0.06142246536910534 - timing_s/old_log_prob:3.069812884554267 - timing_s/adv:0.5685435552150011 - timing_s/update_actor:21.84132767468691 - timing_s/step:42.037929989397526 - timing_s/stop_profile:0.00012028217315673828 - timing_per_token_ms/gen:0.1525297376151026 - timing_per_token_ms/update_actor:0.12396533083613 - timing_per_token_ms/adv:0.0032268958630504805 - perf/total_num_tokens:176189 - perf/time_per_step:42.037929989397526 - perf/throughput:523.8988933459527 (TaskRunner pid=2320396) Training Progress: 59%|█████▉ | 59/100 [56:27<30:28, 44.59s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Solving for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substitute the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Rounding to the nearest whole number, the magnitude of the charge is approximately $ 56 \, \mathrm{pC} $. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2320396) step:60 - global_seqlen/min:16861 - global_seqlen/max:28185 - global_seqlen/minmax_diff:11324 - global_seqlen/balanced_min:22957 - global_seqlen/balanced_max:22976 - global_seqlen/mean:22970.875 - actor/entropy:0.20119284093379974 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41020888090133667 - training/rollout_probs_diff_mean:0.0027175727300345898 - training/rollout_probs_diff_std:0.00927771907299757 - training/rollout_actor_probs_pearson_corr:0.9991341829299927 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0001078844070435 - rollout_corr/rollout_is_ratio_fraction_high:2.6176410756306723e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.7450940504204482e-05 - rollout_corr/rollout_is_max:2.1670923233032227 - rollout_corr/rollout_is_min:0.39598017930984497 - rollout_corr/rollout_is_std:0.028364475816488266 - rollout_corr/rollout_is_eff_sample_size:0.999196341310165 - rollout_corr/rollout_is_seq_mean:1.0001099109649658 - rollout_corr/rollout_is_seq_std:0.0016272783977910876 - rollout_corr/rollout_is_seq_max:1.0065743923187256 - rollout_corr/rollout_is_seq_min:0.9956563115119934 - rollout_corr/rollout_is_seq_max_deviation:0.006574392318725586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2329197567887604) - rollout_corr/training_log_ppl:np.float64(0.19494881299033295) - rollout_corr/kl:np.float64(0.0005096376898059418) - rollout_corr/k3_kl:np.float64(0.000489828710598772) - rollout_corr/rollout_ppl:np.float64(1.2322118538431823) - rollout_corr/rollout_log_ppl:np.float64(0.19443917492753826) - rollout_corr/log_ppl_diff:np.float64(0.0005096380627946928) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013880194601370022) - rollout_corr/log_ppl_diff_max:np.float64(0.0056622326374053955) - rollout_corr/log_ppl_diff_min:np.float64(-0.005925208330154419) - rollout_corr/ppl_ratio:np.float64(1.000511354766786) - rollout_corr/chi2_token:np.float64(0.00095889694057405) - rollout_corr/chi2_seq:np.float64(0.9751317095942795) - actor/pg_loss:np.float64(0.001680221175774932) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000293630393116473) - actor/ppo_kl:np.float64(0.00020125376903123993) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(45.16269864095375) - self_distillation/token_reweight_w_std:np.float64(918.4515415658243) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998249302152544) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03986541925405618) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.17679162323474884) - perf/mfu/actor:np.float64(0.054896138579939024) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.72972869873047) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.1910811494167271) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7283999999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15493982751146623) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.561325) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1694866363964564) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6451) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19198155285050436) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7913375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10282284928700713) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49381249999999993) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12633365609527897) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.66425) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16169326315203264) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8276625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05186472248141516) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4475625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08316721871278766) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6774875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12662312734272863) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8432000000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.019071806002745252) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41564999999999996) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05142809598951441) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6827375) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09880717472092095) - val-aux/sciknoweval/score/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/score/std@16:np.float64(0.1910811494167271) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7283999999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15493982751146623) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.561325) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1694866363964564) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6451) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.19198155285050436) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7913375) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10282284928700713) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49381249999999993) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12633365609527897) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.66425) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.16169326315203264) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8276625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05186472248141516) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4475625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08316721871278766) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6774875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12662312734272863) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8432000000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.019071806002745252) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41564999999999996) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05142809598951441) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6827375) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09880717472092095) - val-core/sciknoweval/acc/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/acc/std@16:np.float64(0.1910811494167271) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7283999999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15493982751146623) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.561325) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1694866363964564) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6451) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.19198155285050436) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7913375) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10282284928700713) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49381249999999993) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12633365609527897) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.66425) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.16169326315203264) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8276625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05186472248141516) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4475625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08316721871278766) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6774875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12662312734272863) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8432000000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.019071806002745252) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41564999999999996) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05142809598951441) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6827375) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09880717472092095) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98671875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.030342711941217525) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9966250000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.013654021672028597) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.976625) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03787881465318859) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9869125000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.030202946000179105) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9995125) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0027850399441387277) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.962275) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04219717155049714) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9939125000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.020819912791213174) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0003950870157319778) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9445375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04278961022314208) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9984000000000002) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010747648297244363) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9239875000000002) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0361678881045997) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99985) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0021922653339917476) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005840611644089222 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005840611644089222 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:447.68359375 - response_length/max:1305.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:447.68359375 - response_length_non_aborted/max:1305.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.737614214420319e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5804355461150408) - timing_s/agent_loop/generate_sequences/max:np.float64(10.886121707037091) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.964501895803551) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.886121707037091) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:1305 - timing_s/gen:17.71578363329172 - timing_s/reward:0.07900326326489449 - timing_s/old_log_prob:3.0846715457737446 - timing_s/adv:0.5956523399800062 - timing_s/update_actor:21.899688495323062 - timing_s/step:43.44701548665762 - timing_s/testing:107.27109943144023 - timing_s/save_checkpoint:12.707542151212692 - timing_s/stop_profile:0.00014159828424453735 - timing_per_token_ms/gen:0.154578547848663 - timing_per_token_ms/update_actor:0.11917095286598281 - timing_per_token_ms/adv:0.003241345508061873 - perf/total_num_tokens:183767 - perf/time_per_step:43.44701548665762 - perf/throughput:528.7100792240666 (TaskRunner pid=2320396) Training Progress: 60%|██████ | 60/100 [59:10<53:30, 80.27s/it] (TaskRunner pid=2320396) step:61 - global_seqlen/min:19183 - global_seqlen/max:30554 - global_seqlen/minmax_diff:11371 - global_seqlen/balanced_min:24395 - global_seqlen/balanced_max:24418 - global_seqlen/mean:24411.25 - actor/entropy:0.20756813883781433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3475205600261688 - training/rollout_probs_diff_mean:0.0026950256433337927 - training/rollout_probs_diff_std:0.009178821928799152 - training/rollout_actor_probs_pearson_corr:0.9992281794548035 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9999772906303406 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5400490156025626e-05 - rollout_corr/rollout_is_max:1.9424793720245361 - rollout_corr/rollout_is_min:0.32020506262779236 - rollout_corr/rollout_is_std:0.02756519615650177 - rollout_corr/rollout_is_eff_sample_size:0.9992406178785591 - rollout_corr/rollout_is_seq_mean:0.9998874664306641 - rollout_corr/rollout_is_seq_std:0.0018777569057419896 - rollout_corr/rollout_is_seq_max:1.0054404735565186 - rollout_corr/rollout_is_seq_min:0.9923277497291565 - rollout_corr/rollout_is_seq_max_deviation:0.007672250270843506 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2361338967457414) - rollout_corr/training_log_ppl:np.float64(0.19852098800765816) - rollout_corr/kl:np.float64(0.0005635680740212479) - rollout_corr/k3_kl:np.float64(0.00048129173163147243) - rollout_corr/rollout_ppl:np.float64(1.2354488540440798) - rollout_corr/rollout_log_ppl:np.float64(0.1979574196448084) - rollout_corr/log_ppl_diff:np.float64(0.0005635683628497645) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013617147196782753) - rollout_corr/log_ppl_diff_max:np.float64(0.009035363793373108) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046346187591552734) - rollout_corr/ppl_ratio:np.float64(1.0005656315479428) - rollout_corr/chi2_token:np.float64(0.0008184912148863077) - rollout_corr/chi2_seq:np.float64(0.38796538999304175) - actor/pg_loss:np.float64(0.0034785300958901644) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003513928604661487) - actor/ppo_kl:np.float64(2.1584473852698238e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(21164.804104407784) - self_distillation/token_reweight_w_std:np.float64(310615.7949758964) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992343427147716) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06176384340506047) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.20081432536244392) - perf/mfu/actor:np.float64(0.056794247237666955) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.04122829437256) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004993608687072992 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004993608687072992 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:507.2890625 - response_length/max:1599.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:507.2890625 - response_length_non_aborted/max:1599.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015490874648094177 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9240107480436563) - timing_s/agent_loop/generate_sequences/max:np.float64(12.865415262058377) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.402125497661473) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.865415262058377) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1599 - timing_s/gen:19.278599567711353 - timing_s/reward:0.06761794164776802 - timing_s/old_log_prob:3.1490181647241116 - timing_s/adv:0.639240212738514 - timing_s/update_actor:22.43888548016548 - timing_s/step:45.67836303450167 - timing_s/stop_profile:0.00011006183922290802 - timing_per_token_ms/gen:0.14844993737938608 - timing_per_token_ms/update_actor:0.1149003301764836 - timing_per_token_ms/adv:0.003273286971880352 - perf/total_num_tokens:195290 - perf/time_per_step:45.67836303450167 - perf/throughput:534.4160424829969 (TaskRunner pid=2320396) Training Progress: 61%|██████ | 61/100 [59:56<45:26, 69.91s/it] (TaskRunner pid=2320396) step:62 - global_seqlen/min:20758 - global_seqlen/max:30529 - global_seqlen/minmax_diff:9771 - global_seqlen/balanced_min:25475 - global_seqlen/balanced_max:25505 - global_seqlen/mean:25489.375 - actor/entropy:0.23646610975265503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163541436195374 - training/rollout_probs_diff_mean:0.0028907926753163338 - training/rollout_probs_diff_std:0.009120183065533638 - training/rollout_actor_probs_pearson_corr:0.9992669224739075 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0000348091125488 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.682202522119042e-06 - rollout_corr/rollout_is_max:1.6338132619857788 - rollout_corr/rollout_is_min:0.04254038631916046 - rollout_corr/rollout_is_std:0.027586810290813446 - rollout_corr/rollout_is_eff_sample_size:0.9992395466249402 - rollout_corr/rollout_is_seq_mean:1.0000940561294556 - rollout_corr/rollout_is_seq_std:0.0016261922428384423 - rollout_corr/rollout_is_seq_max:1.0076667070388794 - rollout_corr/rollout_is_seq_min:0.9946310520172119 - rollout_corr/rollout_is_seq_max_deviation:0.0076667070388793945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2761757029220462) - rollout_corr/training_log_ppl:np.float64(0.22961067700816784) - rollout_corr/kl:np.float64(0.0003324805734532532) - rollout_corr/k3_kl:np.float64(0.00043630713193465454) - rollout_corr/rollout_ppl:np.float64(1.275736337993294) - rollout_corr/rollout_log_ppl:np.float64(0.229278196071391) - rollout_corr/log_ppl_diff:np.float64(0.00033248093677684665) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012090119707863778) - rollout_corr/log_ppl_diff_max:np.float64(0.006240293383598328) - rollout_corr/log_ppl_diff_min:np.float64(-0.007163435220718384) - rollout_corr/ppl_ratio:np.float64(1.0003339080139995) - rollout_corr/chi2_token:np.float64(0.0010929901618510485) - rollout_corr/chi2_seq:np.float64(0.3629838922061026) - actor/pg_loss:np.float64(0.0025408605579286814) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021029063645983115) - actor/ppo_kl:np.float64(2.6390286969046883e-06) - actor/pg_clipfrac_lower:np.float64(4.116174750379287e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(9723.37822908233) - self_distillation/token_reweight_w_std:np.float64(233810.96705805685) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999671149533242) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0595514630258549) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.18411173671483994) - perf/mfu/actor:np.float64(0.058899028442019795) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.08804321289062) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0028481765184551477 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0028481765184551477 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:508.48046875 - response_length/max:1441.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:508.48046875 - response_length_non_aborted/max:1441.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.119642734527588e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0143952444195747) - timing_s/agent_loop/generate_sequences/max:np.float64(11.363589746877551) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.538877304374182) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.363589746877551) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1379 - timing_s/gen:18.385584201663733 - timing_s/reward:0.06554795429110527 - timing_s/old_log_prob:3.051215363666415 - timing_s/adv:0.6370475739240646 - timing_s/update_actor:22.51959772594273 - timing_s/step:44.75173839740455 - timing_s/stop_profile:0.00012248754501342773 - timing_per_token_ms/gen:0.1412417835129463 - timing_per_token_ms/update_actor:0.11043620001443116 - timing_per_token_ms/adv:0.0031240839267541115 - perf/total_num_tokens:203915 - perf/time_per_step:44.75173839740455 - perf/throughput:569.5728459450929 (TaskRunner pid=2320396) Training Progress: 62%|██████▏ | 62/100 [1:00:41<39:30, 62.37s/it] (TaskRunner pid=2320396) step:63 - global_seqlen/min:18323 - global_seqlen/max:25373 - global_seqlen/minmax_diff:7050 - global_seqlen/balanced_min:22190 - global_seqlen/balanced_max:22198 - global_seqlen/mean:22195.25 - actor/entropy:0.22693386673927307 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47528380155563354 - training/rollout_probs_diff_mean:0.0029311031103134155 - training/rollout_probs_diff_std:0.009294695220887661 - training/rollout_actor_probs_pearson_corr:0.9991291165351868 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.000087022781372 - rollout_corr/rollout_is_ratio_fraction_high:1.8302614989806898e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.348695755004883 - rollout_corr/rollout_is_min:0.5427963733673096 - rollout_corr/rollout_is_std:0.02828240394592285 - rollout_corr/rollout_is_eff_sample_size:0.9992008640054926 - rollout_corr/rollout_is_seq_mean:1.0000746250152588 - rollout_corr/rollout_is_seq_std:0.001608627731911838 - rollout_corr/rollout_is_seq_max:1.0082793235778809 - rollout_corr/rollout_is_seq_min:0.9936819672584534 - rollout_corr/rollout_is_seq_max_deviation:0.00827932357788086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2453211299143732) - rollout_corr/training_log_ppl:np.float64(0.2045088292506989) - rollout_corr/kl:np.float64(0.00037927413301552804) - rollout_corr/k3_kl:np.float64(0.0004077912325044508) - rollout_corr/rollout_ppl:np.float64(1.2448103982023895) - rollout_corr/rollout_log_ppl:np.float64(0.204129554957035) - rollout_corr/log_ppl_diff:np.float64(0.0003792742936639115) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012008958583464846) - rollout_corr/log_ppl_diff_max:np.float64(0.005300462245941162) - rollout_corr/log_ppl_diff_min:np.float64(-0.009547919034957886) - rollout_corr/ppl_ratio:np.float64(1.0003806757740676) - rollout_corr/chi2_token:np.float64(0.0008865287527441978) - rollout_corr/chi2_seq:np.float64(0.3732549585402012) - actor/pg_loss:np.float64(0.001522935344837606) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011153514515171992) - actor/ppo_kl:np.float64(4.508457355001383e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(3185.1990010682493) - self_distillation/token_reweight_w_std:np.float64(66056.73461075983) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00030661188066) - self_distillation/token_reweight_w_clip_frac:np.float64(0.032705511490348727) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.1341842170804739) - perf/mfu/actor:np.float64(0.05382390301504096) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.0361328125) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006129088345915079 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006129088345915079 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:426.8515625 - response_length/max:1065.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:426.8515625 - response_length_non_aborted/max:1065.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015802867710590363 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8849795944988728) - timing_s/agent_loop/generate_sequences/max:np.float64(9.163215219974518) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.294443838567531) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.163215219974518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:1065 - timing_s/gen:15.782489329576492 - timing_s/reward:0.07164940424263477 - timing_s/old_log_prob:3.163828060030937 - timing_s/adv:0.542920557782054 - timing_s/update_actor:21.563510172069073 - timing_s/step:41.211437748745084 - timing_s/stop_profile:0.0001256987452507019 - timing_per_token_ms/gen:0.14443041647213878 - timing_per_token_ms/update_actor:0.12144214512152979 - timing_per_token_ms/adv:0.0030576393472818166 - perf/total_num_tokens:177562 - perf/time_per_step:41.211437748745084 - perf/throughput:538.5701449029368 (TaskRunner pid=2320396) Training Progress: 63%|██████▎ | 63/100 [1:01:22<34:33, 56.04s/it] (TaskRunner pid=2320396) step:64 - global_seqlen/min:19205 - global_seqlen/max:31322 - global_seqlen/minmax_diff:12117 - global_seqlen/balanced_min:24735 - global_seqlen/balanced_max:24761 - global_seqlen/mean:24748.75 - actor/entropy:0.23320099711418152 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35835739970207214 - training/rollout_probs_diff_mean:0.002753478242084384 - training/rollout_probs_diff_std:0.008778145536780357 - training/rollout_actor_probs_pearson_corr:0.9993224143981934 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999129772186279 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.705703865212854e-06 - rollout_corr/rollout_is_max:1.6222718954086304 - rollout_corr/rollout_is_min:0.47236648201942444 - rollout_corr/rollout_is_std:0.02701038494706154 - rollout_corr/rollout_is_eff_sample_size:0.9992709710189538 - rollout_corr/rollout_is_seq_mean:0.9999531507492065 - rollout_corr/rollout_is_seq_std:0.0013188592856749892 - rollout_corr/rollout_is_seq_max:1.0045053958892822 - rollout_corr/rollout_is_seq_min:0.9948081970214844 - rollout_corr/rollout_is_seq_max_deviation:0.005191802978515625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2561526354402304) - rollout_corr/training_log_ppl:np.float64(0.212319816739182) - rollout_corr/kl:np.float64(0.00036533900832669275) - rollout_corr/k3_kl:np.float64(0.000400368980734811) - rollout_corr/rollout_ppl:np.float64(1.255633560474962) - rollout_corr/rollout_log_ppl:np.float64(0.21195447669015266) - rollout_corr/log_ppl_diff:np.float64(0.00036534004902932793) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011143776500830427) - rollout_corr/log_ppl_diff_max:np.float64(0.005632832646369934) - rollout_corr/log_ppl_diff_min:np.float64(-0.005937211215496063) - rollout_corr/ppl_ratio:np.float64(1.0003664672840387) - rollout_corr/chi2_token:np.float64(0.0008791075088083744) - rollout_corr/chi2_seq:np.float64(0.31506598158739507) - actor/pg_loss:np.float64(0.0030615170253440738) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000200204512111668) - actor/ppo_kl:np.float64(-5.050095456127224e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(12580.71558534773) - self_distillation/token_reweight_w_std:np.float64(320267.26721710706) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003797668032348) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05963162177067716) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.16736289486289024) - perf/mfu/actor:np.float64(0.05846823715557507) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.96532821655273) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009651394211687148 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009651394211687148 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:506.9296875 - response_length/max:1500.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:506.9296875 - response_length_non_aborted/max:1500.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.725987911224365e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8472258783876896) - timing_s/agent_loop/generate_sequences/max:np.float64(12.097647780552506) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.7089943648461485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.097647780552506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:349 - timing_s/agent_loop/slowest/response_length:1500 - timing_s/gen:18.741990983486176 - timing_s/reward:0.06568616442382336 - timing_s/old_log_prob:3.0562352687120438 - timing_s/adv:0.6292859483510256 - timing_s/update_actor:22.034651696681976 - timing_s/step:44.61622857116163 - timing_s/stop_profile:0.00011373311281204224 - timing_per_token_ms/gen:0.14442023042740593 - timing_per_token_ms/update_actor:0.1112917404751855 - timing_per_token_ms/adv:0.0031783723842165037 - perf/total_num_tokens:197990 - perf/time_per_step:44.61622857116163 - perf/throughput:554.7028691707199 (TaskRunner pid=2320396) Training Progress: 64%|██████▍ | 64/100 [1:02:07<31:34, 52.63s/it] (TaskRunner pid=2320396) step:65 - global_seqlen/min:12805 - global_seqlen/max:27893 - global_seqlen/minmax_diff:15088 - global_seqlen/balanced_min:21975 - global_seqlen/balanced_max:21991 - global_seqlen/mean:21986.0 - actor/entropy:0.19277343153953552 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135476350784302 - training/rollout_probs_diff_mean:0.002630578586831689 - training/rollout_probs_diff_std:0.008825007826089859 - training/rollout_actor_probs_pearson_corr:0.9991601705551147 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.84375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:1.0000410079956055 - rollout_corr/rollout_is_ratio_fraction_high:9.218969353241846e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8437938706483692e-05 - rollout_corr/rollout_is_max:2.167001485824585 - rollout_corr/rollout_is_min:0.4516155421733856 - rollout_corr/rollout_is_std:0.02641010284423828 - rollout_corr/rollout_is_eff_sample_size:0.9993031116481851 - rollout_corr/rollout_is_seq_mean:1.000004768371582 - rollout_corr/rollout_is_seq_std:0.0014547642786055803 - rollout_corr/rollout_is_seq_max:1.0044742822647095 - rollout_corr/rollout_is_seq_min:0.9930375218391418 - rollout_corr/rollout_is_seq_max_deviation:0.006962478160858154 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.188718719407916) - rollout_corr/training_log_ppl:np.float64(0.16565465025632875) - rollout_corr/kl:np.float64(0.0005832098385161189) - rollout_corr/k3_kl:np.float64(0.0003921816246865717) - rollout_corr/rollout_ppl:np.float64(1.1879985383711755) - rollout_corr/rollout_log_ppl:np.float64(0.16507143871422159) - rollout_corr/log_ppl_diff:np.float64(0.0005832115421071649) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012660615611821413) - rollout_corr/log_ppl_diff_max:np.float64(0.012264132499694824) - rollout_corr/log_ppl_diff_min:np.float64(-0.003919497132301331) - rollout_corr/ppl_ratio:np.float64(1.0005849439185113) - rollout_corr/chi2_token:np.float64(0.0003987816162407398) - rollout_corr/chi2_seq:np.float64(0.1664372819941491) - actor/pg_loss:np.float64(0.0017755826702341437) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001691463867246057) - actor/ppo_kl:np.float64(0.00022979352656449237) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_w_mean:np.float64(11.275148867396638) - self_distillation/token_reweight_w_std:np.float64(238.43248290149495) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994188658893108) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03806430572876707) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.10744893923401833) - perf/mfu/actor:np.float64(0.05209502512150299) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.95738220214844) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.84375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008053922094404697 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008053922094404697 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.71875 - response_length/max:1412.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.71875 - response_length_non_aborted/max:1412.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001363661140203476 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9663147907704115) - timing_s/agent_loop/generate_sequences/max:np.float64(10.955320874229074) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.436157472024206) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.955320874229074) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:363 - timing_s/agent_loop/slowest/response_length:1412 - timing_s/gen:18.540402602404356 - timing_s/reward:0.061292244121432304 - timing_s/old_log_prob:3.1427970565855503 - timing_s/adv:0.5885716322809458 - timing_s/update_actor:21.8789913803339 - timing_s/step:44.30024716630578 - timing_s/stop_profile:0.00010786019265651703 - timing_per_token_ms/gen:0.1709233959215683 - timing_per_token_ms/update_actor:0.12439160932146537 - timing_per_token_ms/adv:0.003346286456614128 - perf/total_num_tokens:175888 - perf/time_per_step:44.30024716630578 - perf/throughput:496.29519938033843 (TaskRunner pid=2320396) Training Progress: 65%|██████▌ | 65/100 [1:02:51<29:15, 50.14s/it] (TaskRunner pid=2320396) step:66 - global_seqlen/min:21157 - global_seqlen/max:30988 - global_seqlen/minmax_diff:9831 - global_seqlen/balanced_min:24722 - global_seqlen/balanced_max:24738 - global_seqlen/mean:24733.625 - actor/entropy:0.22358284890651703 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413550317287445 - training/rollout_probs_diff_mean:0.002847072435542941 - training/rollout_probs_diff_std:0.00899769738316536 - training/rollout_actor_probs_pearson_corr:0.9991993308067322 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.000038504600525 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.874790906906128 - rollout_corr/rollout_is_min:0.5706772804260254 - rollout_corr/rollout_is_std:0.027571681886911392 - rollout_corr/rollout_is_eff_sample_size:0.9992406178785591 - rollout_corr/rollout_is_seq_mean:0.999980628490448 - rollout_corr/rollout_is_seq_std:0.0014468131121248007 - rollout_corr/rollout_is_seq_max:1.0045393705368042 - rollout_corr/rollout_is_seq_min:0.9940057396888733 - rollout_corr/rollout_is_seq_max_deviation:0.005994260311126709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.241884677670896) - rollout_corr/training_log_ppl:np.float64(0.20733708786428906) - rollout_corr/kl:np.float64(0.0005020291563688772) - rollout_corr/k3_kl:np.float64(0.0004314393266895422) - rollout_corr/rollout_ppl:np.float64(1.2412674692459404) - rollout_corr/rollout_log_ppl:np.float64(0.20683505745546427) - rollout_corr/log_ppl_diff:np.float64(0.000502030408824794) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011577872064663097) - rollout_corr/log_ppl_diff_max:np.float64(0.006901249289512634) - rollout_corr/log_ppl_diff_min:np.float64(-0.005114339292049408) - rollout_corr/ppl_ratio:np.float64(1.0005033700726926) - rollout_corr/chi2_token:np.float64(0.0007429802790284157) - rollout_corr/chi2_seq:np.float64(0.2530661830678582) - actor/pg_loss:np.float64(0.0027268020203337073) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020032282054671668) - actor/ppo_kl:np.float64(9.502547750450674e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(5488.480364457937) - self_distillation/token_reweight_w_std:np.float64(106593.19712470478) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994499702006578) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06123181653674692) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.1668576430529356) - perf/mfu/actor:np.float64(0.05822141794440938) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.87117385864258) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008164207451045513 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008164207451045513 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:493.11328125 - response_length/max:1246.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:493.11328125 - response_length_non_aborted/max:1246.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001009814441204071 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1982299461960793) - timing_s/agent_loop/generate_sequences/max:np.float64(10.634438088163733) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.621325811313) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.634438088163733) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:1246 - timing_s/gen:17.873955199494958 - timing_s/reward:0.06542827747762203 - timing_s/old_log_prob:3.105355206876993 - timing_s/adv:0.5925860479474068 - timing_s/update_actor:22.105792071670294 - timing_s/step:43.8312210701406 - timing_s/stop_profile:0.00012839213013648987 - timing_per_token_ms/gen:0.14159046238024475 - timing_per_token_ms/update_actor:0.11171932981755754 - timing_per_token_ms/adv:0.0029948402627364913 - perf/total_num_tokens:197869 - perf/time_per_step:43.8312210701406 - perf/throughput:564.292401537712 (TaskRunner pid=2320396) Training Progress: 66%|██████▌ | 66/100 [1:03:35<27:21, 48.27s/it] (TaskRunner pid=2320396) step:67 - global_seqlen/min:12971 - global_seqlen/max:31231 - global_seqlen/minmax_diff:18260 - global_seqlen/balanced_min:21701 - global_seqlen/balanced_max:21708 - global_seqlen/mean:21704.75 - actor/entropy:0.2337634563446045 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23616445064544678 - training/rollout_probs_diff_mean:0.002843605587258935 - training/rollout_probs_diff_std:0.008973178453743458 - training/rollout_actor_probs_pearson_corr:0.9992346167564392 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.81640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:0.9999405741691589 - rollout_corr/rollout_is_ratio_fraction_high:9.46414002100937e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.378192901611328 - rollout_corr/rollout_is_min:0.5167847275733948 - rollout_corr/rollout_is_std:0.027302302420139313 - rollout_corr/rollout_is_eff_sample_size:0.999254901479631 - rollout_corr/rollout_is_seq_mean:0.9998087286949158 - rollout_corr/rollout_is_seq_std:0.0017661856254562736 - rollout_corr/rollout_is_seq_max:1.0052413940429688 - rollout_corr/rollout_is_seq_min:0.9917988777160645 - rollout_corr/rollout_is_seq_max_deviation:0.008201122283935547 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3157822396606207) - rollout_corr/training_log_ppl:np.float64(0.2406519346841378) - rollout_corr/kl:np.float64(0.0006637728164324841) - rollout_corr/k3_kl:np.float64(0.0004682636699158138) - rollout_corr/rollout_ppl:np.float64(1.3148044827394187) - rollout_corr/rollout_log_ppl:np.float64(0.23998815877712332) - rollout_corr/log_ppl_diff:np.float64(0.0006637759070144966) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014726822410011664) - rollout_corr/log_ppl_diff_max:np.float64(0.010123491287231445) - rollout_corr/log_ppl_diff_min:np.float64(-0.007408738136291504) - rollout_corr/ppl_ratio:np.float64(1.0006661214865744) - rollout_corr/chi2_token:np.float64(0.000572551041841507) - rollout_corr/chi2_seq:np.float64(1.99370268243365) - actor/pg_loss:np.float64(0.0007992174942046404) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001359532384412887) - actor/ppo_kl:np.float64(5.996200779279093e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_w_mean:np.float64(4817.5678551825695) - self_distillation/token_reweight_w_std:np.float64(95755.82945584616) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006719485390931) - self_distillation/token_reweight_w_clip_frac:np.float64(0.034751594881527126) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.12752852961421013) - perf/mfu/actor:np.float64(0.05151760153021234) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.8641757965088) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015237266197800636 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0015237266197800636 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:412.7421875 - response_length/max:1256.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.7421875 - response_length_non_aborted/max:1256.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005677081644535065 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8790447302162647) - timing_s/agent_loop/generate_sequences/max:np.float64(9.784351877868176) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.49863724395982) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.784351877868176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1199 - timing_s/gen:16.717525171115994 - timing_s/reward:0.05824257992208004 - timing_s/old_log_prob:3.199490273371339 - timing_s/adv:0.6258921548724174 - timing_s/update_actor:21.930379945784807 - timing_s/step:42.64087346568704 - timing_s/stop_profile:3.0914321541786194e-05 - timing_per_token_ms/gen:0.15821700489405835 - timing_per_token_ms/update_actor:0.12629942723243073 - timing_per_token_ms/adv:0.003604580534631921 - perf/total_num_tokens:173638 - perf/time_per_step:42.64087346568704 - perf/throughput:509.0127906846406 (TaskRunner pid=2320396) Training Progress: 67%|██████▋ | 67/100 [1:04:18<25:41, 46.73s/it] (TaskRunner pid=2320396) step:68 - global_seqlen/min:19038 - global_seqlen/max:30016 - global_seqlen/minmax_diff:10978 - global_seqlen/balanced_min:22411 - global_seqlen/balanced_max:22428 - global_seqlen/mean:22421.25 - actor/entropy:0.19149050116539001 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553967833518982 - training/rollout_probs_diff_mean:0.0027065107133239508 - training/rollout_probs_diff_std:0.00906725600361824 - training/rollout_actor_probs_pearson_corr:0.9990512132644653 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.7578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.9999111890792847 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6474685668945312 - rollout_corr/rollout_is_min:0.5086522102355957 - rollout_corr/rollout_is_std:0.026639310643076897 - rollout_corr/rollout_is_eff_sample_size:0.9992909693897 - rollout_corr/rollout_is_seq_mean:0.9998083710670471 - rollout_corr/rollout_is_seq_std:0.0016350612277165055 - rollout_corr/rollout_is_seq_max:1.0051226615905762 - rollout_corr/rollout_is_seq_min:0.9900025129318237 - rollout_corr/rollout_is_seq_max_deviation:0.00999748706817627 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2052938612177968) - rollout_corr/training_log_ppl:np.float64(0.17810455984727014) - rollout_corr/kl:np.float64(0.0006475814677733283) - rollout_corr/k3_kl:np.float64(0.0004210113900739998) - rollout_corr/rollout_ppl:np.float64(1.204493265133351) - rollout_corr/rollout_log_ppl:np.float64(0.1774569788685767) - rollout_corr/log_ppl_diff:np.float64(0.0006475809786934406) - rollout_corr/log_ppl_abs_diff:np.float64(0.001370340964058414) - rollout_corr/log_ppl_diff_max:np.float64(0.01119665801525116) - rollout_corr/log_ppl_diff_min:np.float64(-0.0046173036098480225) - rollout_corr/ppl_ratio:np.float64(1.0006494410336018) - rollout_corr/chi2_token:np.float64(0.0003732861950993538) - rollout_corr/chi2_seq:np.float64(0.1820344750303775) - actor/pg_loss:np.float64(0.0016421022592112422) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019774771476477326) - actor/ppo_kl:np.float64(6.871426979415673e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7578125) - self_distillation/token_reweight_w_mean:np.float64(55.99475798616186) - self_distillation/token_reweight_w_std:np.float64(1029.008185678511) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001285537146032) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04700022886390798) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.21867019683122635) - perf/mfu/actor:np.float64(0.053644370184561974) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7801856994629) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010703998617827892 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010703998617827892 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:435.3203125 - response_length/max:1265.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:435.3203125 - response_length_non_aborted/max:1265.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.28882509469986e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0393309127539396) - timing_s/agent_loop/generate_sequences/max:np.float64(10.404665313661098) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.840117222447589) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.404665313661098) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:1265 - timing_s/gen:17.57954941689968 - timing_s/reward:0.061834998428821564 - timing_s/old_log_prob:3.1406824830919504 - timing_s/adv:0.5611031148582697 - timing_s/update_actor:21.558307776227593 - timing_s/step:42.98878578469157 - timing_s/stop_profile:0.00012563355267047882 - timing_per_token_ms/gen:0.15774617663806895 - timing_per_token_ms/update_actor:0.1201890381681864 - timing_per_token_ms/adv:0.0031281881856401275 - perf/total_num_tokens:179370 - perf/time_per_step:42.98878578469157 - perf/throughput:521.5604393270459 (TaskRunner pid=2320396) Training Progress: 68%|██████▊ | 68/100 [1:05:01<24:19, 45.62s/it] (TaskRunner pid=2320396) step:69 - global_seqlen/min:18619 - global_seqlen/max:27775 - global_seqlen/minmax_diff:9156 - global_seqlen/balanced_min:23471 - global_seqlen/balanced_max:23483 - global_seqlen/mean:23479.0 - actor/entropy:0.1945514678955078 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18275725841522217 - training/rollout_probs_diff_mean:0.002515253610908985 - training/rollout_probs_diff_std:0.00857163593173027 - training/rollout_actor_probs_pearson_corr:0.9991545677185059 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.80859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9998260140419006 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.606383204460144 - rollout_corr/rollout_is_min:0.5698836445808411 - rollout_corr/rollout_is_std:0.025290658697485924 - rollout_corr/rollout_is_eff_sample_size:0.9993606128205051 - rollout_corr/rollout_is_seq_mean:0.9998338222503662 - rollout_corr/rollout_is_seq_std:0.0015274977777153254 - rollout_corr/rollout_is_seq_max:1.0084797143936157 - rollout_corr/rollout_is_seq_min:0.9939998984336853 - rollout_corr/rollout_is_seq_max_deviation:0.008479714393615723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2252938458696008) - rollout_corr/training_log_ppl:np.float64(0.19016908761113882) - rollout_corr/kl:np.float64(0.000461935019760773) - rollout_corr/k3_kl:np.float64(0.0003738991998147867) - rollout_corr/rollout_ppl:np.float64(1.2247161795385182) - rollout_corr/rollout_log_ppl:np.float64(0.18970715206523892) - rollout_corr/log_ppl_diff:np.float64(0.00046193554589990526) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011312234710203484) - rollout_corr/log_ppl_diff_max:np.float64(0.006538271903991699) - rollout_corr/log_ppl_diff_min:np.float64(-0.007022887468338013) - rollout_corr/ppl_ratio:np.float64(1.0004631178453565) - rollout_corr/chi2_token:np.float64(0.0005808186251670122) - rollout_corr/chi2_seq:np.float64(0.09659922705031931) - actor/pg_loss:np.float64(0.0018763046246021986) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010173328519158531) - actor/ppo_kl:np.float64(-5.9943213223334624e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_w_mean:np.float64(5404.409017624101) - self_distillation/token_reweight_w_std:np.float64(155274.98073743377) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990867469459772) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04784997092792764) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.14408646896481514) - perf/mfu/actor:np.float64(0.055353113989175064) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7887077331543) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011332765221595764 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011332765221595764 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:463.0 - response_length/max:1312.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.0 - response_length_non_aborted/max:1312.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010467134416103363 - timing_s/agent_loop/generate_sequences/min:np.float64(1.175532167777419) - timing_s/agent_loop/generate_sequences/max:np.float64(10.851345924660563) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.075913681110251) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.851345924660563) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1312 - timing_s/gen:17.613020185381174 - timing_s/reward:0.061091553419828415 - timing_s/old_log_prob:3.0520095359534025 - timing_s/adv:0.5751816276460886 - timing_s/update_actor:22.079808918759227 - timing_s/step:43.46936789341271 - timing_s/stop_profile:0.00011400319635868073 - timing_per_token_ms/gen:0.14859796997655553 - timing_per_token_ms/update_actor:0.11755083755036004 - timing_per_token_ms/adv:0.003062213188626478 - perf/total_num_tokens:187832 - perf/time_per_step:43.46936789341271 - perf/throughput:540.1274768377291 (TaskRunner pid=2320396) Training Progress: 69%|██████▉ | 69/100 [1:05:45<23:14, 44.99s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Solving for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substitute the known values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Convert to picocoulombs ($1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C}$): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2320396) step:70 - global_seqlen/min:18028 - global_seqlen/max:25510 - global_seqlen/minmax_diff:7482 - global_seqlen/balanced_min:22348 - global_seqlen/balanced_max:22365 - global_seqlen/mean:22360.125 - actor/entropy:0.27961182594299316 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4933590888977051 - training/rollout_probs_diff_mean:0.0031229450833052397 - training/rollout_probs_diff_std:0.00926915556192398 - training/rollout_actor_probs_pearson_corr:0.9992610216140747 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.999971330165863 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8122344044968486e-05 - rollout_corr/rollout_is_max:1.530380129814148 - rollout_corr/rollout_is_min:0.4454325735569 - rollout_corr/rollout_is_std:0.02873816154897213 - rollout_corr/rollout_is_eff_sample_size:0.9991745615349487 - rollout_corr/rollout_is_seq_mean:0.9999191164970398 - rollout_corr/rollout_is_seq_std:0.0015768433222547174 - rollout_corr/rollout_is_seq_max:1.0047082901000977 - rollout_corr/rollout_is_seq_min:0.9932003021240234 - rollout_corr/rollout_is_seq_max_deviation:0.0067996978759765625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3106959639117122) - rollout_corr/training_log_ppl:np.float64(0.25108718674164265) - rollout_corr/kl:np.float64(0.0005516339078734944) - rollout_corr/k3_kl:np.float64(0.000464749626843286) - rollout_corr/rollout_ppl:np.float64(1.309946612920612) - rollout_corr/rollout_log_ppl:np.float64(0.25053555284102913) - rollout_corr/log_ppl_diff:np.float64(0.000551633900613524) - rollout_corr/log_ppl_abs_diff:np.float64(0.00132319655676838) - rollout_corr/log_ppl_diff_max:np.float64(0.005755379796028137) - rollout_corr/log_ppl_diff_min:np.float64(-0.005698829889297485) - rollout_corr/ppl_ratio:np.float64(1.0005531306378543) - rollout_corr/chi2_token:np.float64(0.0007509859278798103) - rollout_corr/chi2_seq:np.float64(0.20920691825449467) - actor/pg_loss:np.float64(0.0028931591659784317) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023804989859854686) - actor/ppo_kl:np.float64(4.7358733134217346e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(771.5895430163946) - self_distillation/token_reweight_w_std:np.float64(12027.290180076088) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004990971647203) - self_distillation/token_reweight_w_clip_frac:np.float64(0.053546549650491215) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.18996156752109528) - perf/mfu/actor:np.float64(0.05299450687498227) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.74568939208984) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.653125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18547879406462497) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7277750000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1578272523157477) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5789625) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15746179142845634) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6547125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1856290415296593) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7875875000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11695555066473791) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5169125) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12130525026673644) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.66805) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14816289329687846) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.833) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08474610225017794) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4673375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0844809112181804) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6747) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10936134760034488) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8664124999999998) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0552941679483645) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4340625) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04729254434864337) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6728999999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07496741716168553) - val-aux/sciknoweval/score/mean@16:np.float64(0.653125) - val-aux/sciknoweval/score/std@16:np.float64(0.18547879406462497) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7277750000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1578272523157477) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5789625) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15746179142845634) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6547125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1856290415296593) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7875875000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11695555066473791) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5169125) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12130525026673644) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.66805) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14816289329687846) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.833) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08474610225017794) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4673375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0844809112181804) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6747) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10936134760034488) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8664124999999998) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0552941679483645) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4340625) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04729254434864337) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6728999999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07496741716168553) - val-core/sciknoweval/acc/mean@16:np.float64(0.653125) - val-aux/sciknoweval/acc/std@16:np.float64(0.18547879406462497) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7277750000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1578272523157477) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5789625) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15746179142845634) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6547125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1856290415296593) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7875875000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11695555066473791) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5169125) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12130525026673644) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.66805) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14816289329687846) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.833) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08474610225017794) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4673375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0844809112181804) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6747) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10936134760034488) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8664124999999998) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0552941679483645) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4340625) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04729254434864337) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6728999999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07496741716168553) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9953125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0162370593804366) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995499999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.005113892629358849) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9904624999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.022182150680960806) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9950374999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.01666058299867771) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.982925) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.02741957965577288) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9982875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.010091735838382115) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9716000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.03053533788791813) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997125) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.004125729993095772) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9568249999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.02802487396893062) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010168899782001972 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.010168899782001972 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:431.09765625 - response_length/max:1341.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:431.09765625 - response_length_non_aborted/max:1341.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016302429139614105 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1931192688643932) - timing_s/agent_loop/generate_sequences/max:np.float64(10.98846198990941) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.887176220894617) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.98846198990941) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1341 - timing_s/gen:17.629647402092814 - timing_s/reward:0.06261598318815231 - timing_s/old_log_prob:3.0896036606281996 - timing_s/adv:0.5849062670022249 - timing_s/update_actor:21.95095776952803 - timing_s/step:43.40484565123916 - timing_s/testing:101.84621452912688 - timing_s/save_checkpoint:12.727051159366965 - timing_s/stop_profile:0.00017385371029376984 - timing_per_token_ms/gen:0.15974526691578378 - timing_per_token_ms/update_actor:0.12271262889590304 - timing_per_token_ms/adv:0.0032698065585625357 - perf/total_num_tokens:178881 - perf/time_per_step:43.40484565123916 - perf/throughput:515.1527361637247 (TaskRunner pid=2320396) Training Progress: 70%|███████ | 70/100 [1:08:23<39:27, 78.91s/it] (TaskRunner pid=2320396) step:71 - global_seqlen/min:19647 - global_seqlen/max:25487 - global_seqlen/minmax_diff:5840 - global_seqlen/balanced_min:22565 - global_seqlen/balanced_max:22577 - global_seqlen/mean:22573.375 - actor/entropy:0.23593924939632416 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2151264250278473 - training/rollout_probs_diff_mean:0.0029731541872024536 - training/rollout_probs_diff_std:0.009116576053202152 - training/rollout_actor_probs_pearson_corr:0.9992273449897766 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.79296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.9999300241470337 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9040335416793823 - rollout_corr/rollout_is_min:0.5837352871894836 - rollout_corr/rollout_is_std:0.028094211593270302 - rollout_corr/rollout_is_eff_sample_size:0.9992112187516006 - rollout_corr/rollout_is_seq_mean:0.9998959302902222 - rollout_corr/rollout_is_seq_std:0.0014871498569846153 - rollout_corr/rollout_is_seq_max:1.0080996751785278 - rollout_corr/rollout_is_seq_min:0.9946392774581909 - rollout_corr/rollout_is_seq_max_deviation:0.008099675178527832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2749516679905355) - rollout_corr/training_log_ppl:np.float64(0.22600234394485597) - rollout_corr/kl:np.float64(0.0004519108627860646) - rollout_corr/k3_kl:np.float64(0.00046148597355966103) - rollout_corr/rollout_ppl:np.float64(1.274333464447409) - rollout_corr/rollout_log_ppl:np.float64(0.22555043052852852) - rollout_corr/log_ppl_diff:np.float64(0.000451913416327443) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013323303937795572) - rollout_corr/log_ppl_diff_max:np.float64(0.00794076919555664) - rollout_corr/log_ppl_diff_min:np.float64(-0.004279732704162598) - rollout_corr/ppl_ratio:np.float64(1.0004535210318863) - rollout_corr/chi2_token:np.float64(0.0009360732510685921) - rollout_corr/chi2_seq:np.float64(0.4728632946498692) - actor/pg_loss:np.float64(0.002302838722243905) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025980098098443705) - actor/ppo_kl:np.float64(-5.338420736222815e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_w_mean:np.float64(32.36409391206689) - self_distillation/token_reweight_w_std:np.float64(696.1186733208597) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998391503468156) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045971906249178573) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.19924570620059967) - perf/mfu/actor:np.float64(0.053960876631776955) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.2897071838379) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003918695729225874 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003918695729225874 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:427.88671875 - response_length/max:1067.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:427.88671875 - response_length_non_aborted/max:1067.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011662766337394714 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5011277925223112) - timing_s/agent_loop/generate_sequences/max:np.float64(9.15895122103393) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.785454357996059) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.15895122103393) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:1067 - timing_s/gen:15.603826524689794 - timing_s/reward:0.05770381726324558 - timing_s/old_log_prob:3.1228378247469664 - timing_s/adv:0.6005111709237099 - timing_s/update_actor:21.588089572265744 - timing_s/step:41.09169472195208 - timing_s/stop_profile:2.989359200000763e-05 - timing_per_token_ms/gen:0.14244996325226444 - timing_per_token_ms/update_actor:0.11954398474012938 - timing_per_token_ms/adv:0.0033253289047589798 - perf/total_num_tokens:180587 - perf/time_per_step:41.09169472195208 - perf/throughput:549.341543412684 (TaskRunner pid=2320396) Training Progress: 71%|███████ | 71/100 [1:09:04<32:39, 67.57s/it] (TaskRunner pid=2320396) step:72 - global_seqlen/min:20651 - global_seqlen/max:27731 - global_seqlen/minmax_diff:7080 - global_seqlen/balanced_min:23755 - global_seqlen/balanced_max:23770 - global_seqlen/mean:23766.375 - actor/entropy:0.20936648547649384 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26683688163757324 - training/rollout_probs_diff_mean:0.002599770436063409 - training/rollout_probs_diff_std:0.008832783438265324 - training/rollout_actor_probs_pearson_corr:0.9991433620452881 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999456405639648 - rollout_corr/rollout_is_ratio_fraction_high:8.11089194030501e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.622178388061002e-05 - rollout_corr/rollout_is_max:2.0051612854003906 - rollout_corr/rollout_is_min:0.18509429693222046 - rollout_corr/rollout_is_std:0.02668849006295204 - rollout_corr/rollout_is_eff_sample_size:0.9992881124305862 - rollout_corr/rollout_is_seq_mean:0.9999668598175049 - rollout_corr/rollout_is_seq_std:0.0013168805744498968 - rollout_corr/rollout_is_seq_max:1.0059409141540527 - rollout_corr/rollout_is_seq_min:0.996266782283783 - rollout_corr/rollout_is_seq_max_deviation:0.005940914154052734 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2467334070242941) - rollout_corr/training_log_ppl:np.float64(0.1992745844036108) - rollout_corr/kl:np.float64(0.00042391312367939804) - rollout_corr/k3_kl:np.float64(0.00039017095170379434) - rollout_corr/rollout_ppl:np.float64(1.2461699997074902) - rollout_corr/rollout_log_ppl:np.float64(0.1988506696507102) - rollout_corr/log_ppl_diff:np.float64(0.00042391475290060043) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011057191295549273) - rollout_corr/log_ppl_diff_max:np.float64(0.0045042484998703) - rollout_corr/log_ppl_diff_min:np.float64(-0.004313677549362183) - rollout_corr/ppl_ratio:np.float64(1.0004249857738614) - rollout_corr/chi2_token:np.float64(0.0007191256154328585) - rollout_corr/chi2_seq:np.float64(0.29579704883508384) - actor/pg_loss:np.float64(0.001024734228849411) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015756982065795455) - actor/ppo_kl:np.float64(2.5644786637357697e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_w_mean:np.float64(8350.803719572723) - self_distillation/token_reweight_w_std:np.float64(178395.78476184432) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000230914214626) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0306084052863298) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.11315388977527618) - perf/mfu/actor:np.float64(0.05624797651716909) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.9417266845703) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008435327559709549 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008435327559709549 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:481.60546875 - response_length/max:1367.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:481.60546875 - response_length_non_aborted/max:1367.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.057718276977539e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6083013713359833) - timing_s/agent_loop/generate_sequences/max:np.float64(10.97674540989101) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.481123702607874) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.97674540989101) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:1367 - timing_s/gen:17.8749917820096 - timing_s/reward:0.06322882324457169 - timing_s/old_log_prob:3.2556111719459295 - timing_s/adv:0.5855676289647818 - timing_s/update_actor:21.827434290200472 - timing_s/step:43.699696861207485 - timing_s/stop_profile:0.00011280179023742676 - timing_per_token_ms/gen:0.1449821299365696 - timing_per_token_ms/update_actor:0.11480208009320138 - timing_per_token_ms/adv:0.0030798114403478745 - perf/total_num_tokens:190131 - perf/time_per_step:43.699696861207485 - perf/throughput:543.8567474617329 (TaskRunner pid=2320396) Training Progress: 72%|███████▏ | 72/100 [1:09:48<28:11, 60.42s/it] (TaskRunner pid=2320396) step:73 - global_seqlen/min:17296 - global_seqlen/max:32968 - global_seqlen/minmax_diff:15672 - global_seqlen/balanced_min:25404 - global_seqlen/balanced_max:25442 - global_seqlen/mean:25428.5 - actor/entropy:0.254859983921051 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4140661358833313 - training/rollout_probs_diff_mean:0.0028677647933363914 - training/rollout_probs_diff_std:0.009029688313603401 - training/rollout_actor_probs_pearson_corr:0.9993501901626587 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:1.000032901763916 - rollout_corr/rollout_is_ratio_fraction_high:7.54398161006975e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.50879632201395e-05 - rollout_corr/rollout_is_max:2.0585145950317383 - rollout_corr/rollout_is_min:0.19157066941261292 - rollout_corr/rollout_is_std:0.028284510597586632 - rollout_corr/rollout_is_eff_sample_size:0.9992006259678756 - rollout_corr/rollout_is_seq_mean:0.9999836683273315 - rollout_corr/rollout_is_seq_std:0.0019239388639107347 - rollout_corr/rollout_is_seq_max:1.0077039003372192 - rollout_corr/rollout_is_seq_min:0.9925651550292969 - rollout_corr/rollout_is_seq_max_deviation:0.007703900337219238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3189547401852906) - rollout_corr/training_log_ppl:np.float64(0.2570040342397988) - rollout_corr/kl:np.float64(0.0005966623022146678) - rollout_corr/k3_kl:np.float64(0.0004898175338325927) - rollout_corr/rollout_ppl:np.float64(1.3181753954850137) - rollout_corr/rollout_log_ppl:np.float64(0.256407369626686) - rollout_corr/log_ppl_diff:np.float64(0.0005966646131128073) - rollout_corr/log_ppl_abs_diff:np.float64(0.00133663194719702) - rollout_corr/log_ppl_diff_max:np.float64(0.008886903524398804) - rollout_corr/log_ppl_diff_min:np.float64(-0.004705339670181274) - rollout_corr/ppl_ratio:np.float64(1.0005985633470118) - rollout_corr/chi2_token:np.float64(0.0007987350691109896) - rollout_corr/chi2_seq:np.float64(0.6767518501728773) - actor/pg_loss:np.float64(0.00291269866283983) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024224099979619496) - actor/ppo_kl:np.float64(0.00012902852559548705) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(7661.186577946413) - self_distillation/token_reweight_w_std:np.float64(198827.132493558) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001774257980287) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08138004870852455) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.22882431373000145) - perf/mfu/actor:np.float64(0.05922365864354845) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.86934280395508) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0030854884535074234 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0030854884535074234 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:517.796875 - response_length/max:1611.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:517.796875 - response_length_non_aborted/max:1611.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.911399006843567e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0221941787749529) - timing_s/agent_loop/generate_sequences/max:np.float64(12.645636655390263) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.6147617847746005) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.645636655390263) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:1611 - timing_s/gen:19.4119477532804 - timing_s/reward:0.06393673457205296 - timing_s/old_log_prob:3.1552902571856976 - timing_s/adv:0.6077743247151375 - timing_s/update_actor:22.31188929080963 - timing_s/step:45.63956543803215 - timing_s/stop_profile:0.00011852756142616272 - timing_per_token_ms/gen:0.14644337301427623 - timing_per_token_ms/update_actor:0.10967953915296631 - timing_per_token_ms/adv:0.0029876630784117105 - perf/total_num_tokens:203428 - perf/time_per_step:45.63956543803215 - perf/throughput:557.1591174444014 (TaskRunner pid=2320396) Training Progress: 73%|███████▎ | 73/100 [1:10:33<25:12, 56.01s/it] (TaskRunner pid=2320396) step:74 - global_seqlen/min:14990 - global_seqlen/max:28721 - global_seqlen/minmax_diff:13731 - global_seqlen/balanced_min:21228 - global_seqlen/balanced_max:21262 - global_seqlen/mean:21251.0 - actor/entropy:0.22410832345485687 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24179348349571228 - training/rollout_probs_diff_mean:0.0028740845154970884 - training/rollout_probs_diff_std:0.009236778132617474 - training/rollout_actor_probs_pearson_corr:0.999184250831604 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.81640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.9999187588691711 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.471132216276601e-06 - rollout_corr/rollout_is_max:1.6295045614242554 - rollout_corr/rollout_is_min:0.3649487793445587 - rollout_corr/rollout_is_std:0.027934638783335686 - rollout_corr/rollout_is_eff_sample_size:0.9992200264057035 - rollout_corr/rollout_is_seq_mean:0.9999990463256836 - rollout_corr/rollout_is_seq_std:0.0019640212412923574 - rollout_corr/rollout_is_seq_max:1.006421446800232 - rollout_corr/rollout_is_seq_min:0.9917288422584534 - rollout_corr/rollout_is_seq_max_deviation:0.00827115774154663 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.25452196970582) - rollout_corr/training_log_ppl:np.float64(0.21193978351948317) - rollout_corr/kl:np.float64(0.0006391342091154328) - rollout_corr/k3_kl:np.float64(0.0005525115491096244) - rollout_corr/rollout_ppl:np.float64(1.2536878855898976) - rollout_corr/rollout_log_ppl:np.float64(0.21130064985482022) - rollout_corr/log_ppl_diff:np.float64(0.0006391336646629497) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016215195209952071) - rollout_corr/log_ppl_diff_max:np.float64(0.01168176531791687) - rollout_corr/log_ppl_diff_min:np.float64(-0.00469110906124115) - rollout_corr/ppl_ratio:np.float64(1.0006418717093766) - rollout_corr/chi2_token:np.float64(0.0008710839319974184) - rollout_corr/chi2_seq:np.float64(0.41612958326004446) - actor/pg_loss:np.float64(0.0029548348393291235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002011250385294261) - actor/ppo_kl:np.float64(0.00016125536095801252) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_w_mean:np.float64(7153.264800689416) - self_distillation/token_reweight_w_std:np.float64(146011.8352878823) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995592245832086) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05097539708367549) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.20852339267730713) - perf/mfu/actor:np.float64(0.05059562385832781) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.92523574829102) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014293122105300426 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.014293122105300426 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:412.4375 - response_length/max:1764.0 - response_length/min:65.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.4375 - response_length_non_aborted/max:1764.0 - response_length_non_aborted/min:65.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.712654769420624e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6892342232167721) - timing_s/agent_loop/generate_sequences/max:np.float64(12.871262649074197) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.185971406070166) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.871262649074197) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1764 - timing_s/gen:20.324090790003538 - timing_s/reward:0.07555966824293137 - timing_s/old_log_prob:2.993329456076026 - timing_s/adv:0.5724999159574509 - timing_s/update_actor:21.911697022616863 - timing_s/step:45.928413731977344 - timing_s/stop_profile:0.00012182816863059998 - timing_per_token_ms/gen:0.19249214644267634 - timing_per_token_ms/update_actor:0.12888627019091375 - timing_per_token_ms/adv:0.0033674880944276203 - perf/total_num_tokens:170008 - perf/time_per_step:45.928413731977344 - perf/throughput:462.6983227422927 (TaskRunner pid=2320396) Training Progress: 74%|███████▍ | 74/100 [1:11:19<22:57, 53.00s/it] (TaskRunner pid=2320396) step:75 - global_seqlen/min:16466 - global_seqlen/max:23453 - global_seqlen/minmax_diff:6987 - global_seqlen/balanced_min:20955 - global_seqlen/balanced_max:20971 - global_seqlen/mean:20966.375 - actor/entropy:0.16500282287597656 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31862035393714905 - training/rollout_probs_diff_mean:0.0026397460605949163 - training/rollout_probs_diff_std:0.009367442689836025 - training/rollout_actor_probs_pearson_corr:0.9988722801208496 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.8515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999569654464722 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.948394441977143e-05 - rollout_corr/rollout_is_max:1.7520591020584106 - rollout_corr/rollout_is_min:0.27226293087005615 - rollout_corr/rollout_is_std:0.026879871264100075 - rollout_corr/rollout_is_eff_sample_size:0.9992779941651064 - rollout_corr/rollout_is_seq_mean:1.0000392198562622 - rollout_corr/rollout_is_seq_std:0.0016926469979807734 - rollout_corr/rollout_is_seq_max:1.0069493055343628 - rollout_corr/rollout_is_seq_min:0.9934012293815613 - rollout_corr/rollout_is_seq_max_deviation:0.006949305534362793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1966069960035384) - rollout_corr/training_log_ppl:np.float64(0.1667857915381319) - rollout_corr/kl:np.float64(0.0006250800824640912) - rollout_corr/k3_kl:np.float64(0.0005256048687840575) - rollout_corr/rollout_ppl:np.float64(1.195797340478748) - rollout_corr/rollout_log_ppl:np.float64(0.16616071118187392) - rollout_corr/log_ppl_diff:np.float64(0.0006250803562579677) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014622663875343278) - rollout_corr/log_ppl_diff_max:np.float64(0.022766180336475372) - rollout_corr/log_ppl_diff_min:np.float64(-0.005373738706111908) - rollout_corr/ppl_ratio:np.float64(1.000627742614597) - rollout_corr/chi2_token:np.float64(0.0006986984517425299) - rollout_corr/chi2_seq:np.float64(0.22746214317157865) - actor/pg_loss:np.float64(0.000660582329146564) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.942104457470123e-05) - actor/ppo_kl:np.float64(0.00026853492200018536) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_w_mean:np.float64(7268.040664529195) - self_distillation/token_reweight_w_std:np.float64(129162.24056490284) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002146703191102) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02054209852940403) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.12643406353890896) - perf/mfu/actor:np.float64(0.05026049795772646) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.7984733581543) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.8515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003491861280053854 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0003491861280053854 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:395.73046875 - response_length/max:1260.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:395.73046875 - response_length_non_aborted/max:1260.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002998523414134979 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2730316687375307) - timing_s/agent_loop/generate_sequences/max:np.float64(9.413474356755614) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.387236144888448) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.413474356755614) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:210 - timing_s/agent_loop/slowest/response_length:1260 - timing_s/gen:16.036076923832297 - timing_s/reward:0.08017385751008987 - timing_s/old_log_prob:3.003221595659852 - timing_s/adv:0.5643693823367357 - timing_s/update_actor:21.520387353375554 - timing_s/step:41.29089017212391 - timing_s/stop_profile:0.00011784397065639496 - timing_per_token_ms/gen:0.15829189418137243 - timing_per_token_ms/update_actor:0.12830298128178783 - timing_per_token_ms/adv:0.0033647291337721456 - perf/total_num_tokens:167731 - perf/time_per_step:41.29089017212391 - perf/throughput:507.7724145108092 (TaskRunner pid=2320396) Training Progress: 75%|███████▌ | 75/100 [1:12:01<20:37, 49.50s/it] (TaskRunner pid=2320396) step:76 - global_seqlen/min:21205 - global_seqlen/max:29111 - global_seqlen/minmax_diff:7906 - global_seqlen/balanced_min:24229 - global_seqlen/balanced_max:24247 - global_seqlen/mean:24241.625 - actor/entropy:0.23836244642734528 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6271570920944214 - training/rollout_probs_diff_mean:0.0028990982100367546 - training/rollout_probs_diff_std:0.009411140345036983 - training/rollout_actor_probs_pearson_corr:0.9991864562034607 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.000035047531128 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.142990736814681e-06 - rollout_corr/rollout_is_max:1.7983005046844482 - rollout_corr/rollout_is_min:0.0746283084154129 - rollout_corr/rollout_is_std:0.0281493179500103 - rollout_corr/rollout_is_eff_sample_size:0.9992081242072904 - rollout_corr/rollout_is_seq_mean:1.0001025199890137 - rollout_corr/rollout_is_seq_std:0.0014056930085644126 - rollout_corr/rollout_is_seq_max:1.005411148071289 - rollout_corr/rollout_is_seq_min:0.9962686896324158 - rollout_corr/rollout_is_seq_max_deviation:0.0054111480712890625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2666503265500069) - rollout_corr/training_log_ppl:np.float64(0.22336477947828826) - rollout_corr/kl:np.float64(0.0004707449971448341) - rollout_corr/k3_kl:np.float64(0.00046742754221895666) - rollout_corr/rollout_ppl:np.float64(1.2659660731442273) - rollout_corr/rollout_log_ppl:np.float64(0.2228940339118708) - rollout_corr/log_ppl_diff:np.float64(0.0004707455664174631) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013132981694070622) - rollout_corr/log_ppl_diff_max:np.float64(0.00734102725982666) - rollout_corr/log_ppl_diff_min:np.float64(-0.004764363169670105) - rollout_corr/ppl_ratio:np.float64(1.0004723304882646) - rollout_corr/chi2_token:np.float64(0.0009256789926439524) - rollout_corr/chi2_seq:np.float64(0.6073360876180232) - actor/pg_loss:np.float64(0.0020736060105264187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002479879030943266) - actor/ppo_kl:np.float64(0.00013921088918422697) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(6886.147386563243) - self_distillation/token_reweight_w_std:np.float64(160346.53925711656) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005711058620363) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05048915263614617) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.15480851382017136) - perf/mfu/actor:np.float64(0.05722938996950414) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.76996278762817) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004289320670068264 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004289320670068264 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:479.70703125 - response_length/max:1260.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:479.70703125 - response_length_non_aborted/max:1260.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015643984079360962 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1107019297778606) - timing_s/agent_loop/generate_sequences/max:np.float64(10.500982820987701) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.217866806182428) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.500982820987701) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:1260 - timing_s/gen:17.480207895860076 - timing_s/reward:0.06496796384453773 - timing_s/old_log_prob:3.222787020727992 - timing_s/adv:0.5987678561359644 - timing_s/update_actor:21.95581777766347 - timing_s/step:43.41250845976174 - timing_s/stop_profile:0.0001221839338541031 - timing_per_token_ms/gen:0.14234117418557937 - timing_per_token_ms/update_actor:0.11321341792094934 - timing_per_token_ms/adv:0.0030874985491688594 - perf/total_num_tokens:193933 - perf/time_per_step:43.41250845976174 - perf/throughput:558.4018491460616 (TaskRunner pid=2320396) Training Progress: 76%|███████▌ | 76/100 [1:12:44<19:04, 47.70s/it] (TaskRunner pid=2320396) step:77 - global_seqlen/min:15619 - global_seqlen/max:27776 - global_seqlen/minmax_diff:12157 - global_seqlen/balanced_min:22253 - global_seqlen/balanced_max:22829 - global_seqlen/mean:22363.375 - actor/entropy:0.21923139691352844 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37990033626556396 - training/rollout_probs_diff_mean:0.002951013157144189 - training/rollout_probs_diff_std:0.009562992490828037 - training/rollout_actor_probs_pearson_corr:0.9991053342819214 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.999911367893219 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.550033216015436e-05 - rollout_corr/rollout_is_max:1.8682562112808228 - rollout_corr/rollout_is_min:0.2955019176006317 - rollout_corr/rollout_is_std:0.028421156108379364 - rollout_corr/rollout_is_eff_sample_size:0.9991928898070654 - rollout_corr/rollout_is_seq_mean:0.9998927712440491 - rollout_corr/rollout_is_seq_std:0.0016912176506593823 - rollout_corr/rollout_is_seq_max:1.005102276802063 - rollout_corr/rollout_is_seq_min:0.9950426816940308 - rollout_corr/rollout_is_seq_max_deviation:0.005102276802062988 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2357686357572675) - rollout_corr/training_log_ppl:np.float64(0.20082158614241052) - rollout_corr/kl:np.float64(0.0005429030619339414) - rollout_corr/k3_kl:np.float64(0.00043220897507012523) - rollout_corr/rollout_ppl:np.float64(1.2350516240112484) - rollout_corr/rollout_log_ppl:np.float64(0.20027868168835994) - rollout_corr/log_ppl_diff:np.float64(0.0005429044540505856) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013006862718611956) - rollout_corr/log_ppl_diff_max:np.float64(0.005264118313789368) - rollout_corr/log_ppl_diff_min:np.float64(-0.003429308533668518) - rollout_corr/ppl_ratio:np.float64(1.0005443205591291) - rollout_corr/chi2_token:np.float64(0.0006337540689855814) - rollout_corr/chi2_seq:np.float64(0.21167059405706823) - actor/pg_loss:np.float64(0.0019600478699430823) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016945441325333377) - actor/ppo_kl:np.float64(1.5344409057149733e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(5170.421208809828) - self_distillation/token_reweight_w_std:np.float64(99334.34433582227) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000085497740656) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03653434028092306) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.11112173949368298) - perf/mfu/actor:np.float64(0.05219431599433301) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.65237426757812) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015117594972252846 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015117594972252846 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.13671875 - response_length/max:2226.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.13671875 - response_length_non_aborted/max:2226.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011080503463745117 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0522634759545326) - timing_s/agent_loop/generate_sequences/max:np.float64(15.5052381567657) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.6413939066260355) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.5052381567657) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2226 - timing_s/gen:22.334846772253513 - timing_s/reward:0.06260121613740921 - timing_s/old_log_prob:3.061686584725976 - timing_s/adv:0.5881753247231245 - timing_s/update_actor:22.3996890950948 - timing_s/step:48.53496636450291 - timing_s/stop_profile:0.00013917125761508942 - timing_per_token_ms/gen:0.19822362345021977 - timing_per_token_ms/update_actor:0.12520297749721812 - timing_per_token_ms/adv:0.0032876037534759653 - perf/total_num_tokens:178907 - perf/time_per_step:48.53496636450291 - perf/throughput:460.7683217919346 (TaskRunner pid=2320396) Training Progress: 77%|███████▋ | 77/100 [1:13:33<18:23, 47.97s/it] (TaskRunner pid=2320396) step:78 - global_seqlen/min:16885 - global_seqlen/max:27989 - global_seqlen/minmax_diff:11104 - global_seqlen/balanced_min:22881 - global_seqlen/balanced_max:22910 - global_seqlen/mean:22902.875 - actor/entropy:0.19331404566764832 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35333073139190674 - training/rollout_probs_diff_mean:0.0028317312244325876 - training/rollout_probs_diff_std:0.009370970539748669 - training/rollout_actor_probs_pearson_corr:0.9989901781082153 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.84765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.0001708269119263 - rollout_corr/rollout_is_ratio_fraction_high:1.731077099975664e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.731077099975664e-05 - rollout_corr/rollout_is_max:2.9112932682037354 - rollout_corr/rollout_is_min:0.43616390228271484 - rollout_corr/rollout_is_std:0.028227554634213448 - rollout_corr/rollout_is_eff_sample_size:0.9992043155636828 - rollout_corr/rollout_is_seq_mean:1.0001126527786255 - rollout_corr/rollout_is_seq_std:0.001920209964737296 - rollout_corr/rollout_is_seq_max:1.0091001987457275 - rollout_corr/rollout_is_seq_min:0.9932641983032227 - rollout_corr/rollout_is_seq_max_deviation:0.009100198745727539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1990957544185221) - rollout_corr/training_log_ppl:np.float64(0.17567761093960144) - rollout_corr/kl:np.float64(0.0006657836468448863) - rollout_corr/k3_kl:np.float64(0.0007186990048140274) - rollout_corr/rollout_ppl:np.float64(1.1982321050018072) - rollout_corr/rollout_log_ppl:np.float64(0.17501182701380458) - rollout_corr/log_ppl_diff:np.float64(0.000665783925796859) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016977235936792567) - rollout_corr/log_ppl_diff_max:np.float64(0.049715906381607056) - rollout_corr/log_ppl_diff_min:np.float64(-0.008080065250396729) - rollout_corr/ppl_ratio:np.float64(1.0006730714812875) - rollout_corr/chi2_token:np.float64(0.0011810176074504852) - rollout_corr/chi2_seq:np.float64(0.5433423959184438) - actor/pg_loss:np.float64(0.002108511747792363) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003408425704947149) - actor/ppo_kl:np.float64(0.00035315673385127866) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_w_mean:np.float64(6268.631206417689) - self_distillation/token_reweight_w_std:np.float64(116329.06896551698) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998350939713418) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0443158886191668) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.25990286469459534) - perf/mfu/actor:np.float64(0.05410870160542751) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.07009887695312) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0028173280879855156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0028173280879855156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:451.30859375 - response_length/max:1525.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:451.30859375 - response_length_non_aborted/max:1525.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001499764621257782 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2836046293377876) - timing_s/agent_loop/generate_sequences/max:np.float64(11.574323669075966) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.801704231867916) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.574323669075966) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:1525 - timing_s/gen:18.21794650144875 - timing_s/reward:0.06405134312808514 - timing_s/old_log_prob:3.1706241723150015 - timing_s/adv:0.5903529282659292 - timing_s/update_actor:22.47433365881443 - timing_s/step:44.60598493553698 - timing_s/stop_profile:0.00011158548295497894 - timing_per_token_ms/gen:0.1576833557056195 - timing_per_token_ms/update_actor:0.12266109417930299 - timing_per_token_ms/adv:0.003222045967296296 - perf/total_num_tokens:183223 - perf/time_per_step:44.60598493553698 - perf/throughput:513.4484763221446 (TaskRunner pid=2320396) Training Progress: 78%|███████▊ | 78/100 [1:14:17<17:13, 46.98s/it] (TaskRunner pid=2320396) step:79 - global_seqlen/min:16558 - global_seqlen/max:33044 - global_seqlen/minmax_diff:16486 - global_seqlen/balanced_min:24636 - global_seqlen/balanced_max:24664 - global_seqlen/mean:24657.875 - actor/entropy:0.19059540331363678 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016378581523895 - training/rollout_probs_diff_mean:0.0026734904386103153 - training/rollout_probs_diff_std:0.009236500598490238 - training/rollout_actor_probs_pearson_corr:0.9990987777709961 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.8203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.000053882598877 - rollout_corr/rollout_is_ratio_fraction_high:1.5713510947534814e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.856755473767407e-06 - rollout_corr/rollout_is_max:2.6087822914123535 - rollout_corr/rollout_is_min:0.4246424436569214 - rollout_corr/rollout_is_std:0.027441667392849922 - rollout_corr/rollout_is_eff_sample_size:0.9992476405980566 - rollout_corr/rollout_is_seq_mean:0.9999700784683228 - rollout_corr/rollout_is_seq_std:0.001676858519203961 - rollout_corr/rollout_is_seq_max:1.0092583894729614 - rollout_corr/rollout_is_seq_min:0.9936667680740356 - rollout_corr/rollout_is_seq_max_deviation:0.009258389472961426 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2020120695233345) - rollout_corr/training_log_ppl:np.float64(0.17426249934942462) - rollout_corr/kl:np.float64(0.00046253042679245127) - rollout_corr/k3_kl:np.float64(0.0004328242781781455) - rollout_corr/rollout_ppl:np.float64(1.201419445220381) - rollout_corr/rollout_log_ppl:np.float64(0.1737999671822763) - rollout_corr/log_ppl_diff:np.float64(0.0004625321671483107) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013120870062266476) - rollout_corr/log_ppl_diff_max:np.float64(0.00897689163684845) - rollout_corr/log_ppl_diff_min:np.float64(-0.007229626178741455) - rollout_corr/ppl_ratio:np.float64(1.0004643604625016) - rollout_corr/chi2_token:np.float64(0.0007903354708105326) - rollout_corr/chi2_seq:np.float64(0.6352600273676217) - actor/pg_loss:np.float64(0.003112686681561172) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002658791520389059) - actor/ppo_kl:np.float64(2.6398326252063953e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_w_mean:np.float64(408.7717815169599) - self_distillation/token_reweight_w_std:np.float64(9884.658681469678) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990864836145192) - self_distillation/token_reweight_w_clip_frac:np.float64(0.054150840849615633) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.16461476869881153) - perf/mfu/actor:np.float64(0.05762919844007125) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.98615264892578) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006058540660887957 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006058540660887957 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:497.18359375 - response_length/max:1686.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:497.18359375 - response_length_non_aborted/max:1686.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.457976996898651e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4331816025078297) - timing_s/agent_loop/generate_sequences/max:np.float64(13.189415600150824) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.385525229481573) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.189415600150824) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:1686 - timing_s/gen:19.859036806970835 - timing_s/reward:0.06584840640425682 - timing_s/old_log_prob:3.203118473291397 - timing_s/adv:0.6456434894353151 - timing_s/update_actor:22.172156451269984 - timing_s/step:46.03451617434621 - timing_s/stop_profile:0.00011187046766281128 - timing_per_token_ms/gen:0.1560275992659499 - timing_per_token_ms/update_actor:0.11239896205203198 - timing_per_token_ms/adv:0.003273008569449492 - perf/total_num_tokens:197263 - perf/time_per_step:46.03451617434621 - perf/throughput:535.6388433977105 (TaskRunner pid=2320396) Training Progress: 79%|███████▉ | 79/100 [1:15:03<16:20, 46.71s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. Using the formula for the electric field due to a point charge: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Rearranging to solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the given values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) This value is closest to option **B: 56 pC**. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2320396) step:80 - global_seqlen/min:17092 - global_seqlen/max:30502 - global_seqlen/minmax_diff:13410 - global_seqlen/balanced_min:23444 - global_seqlen/balanced_max:23520 - global_seqlen/mean:23455.875 - actor/entropy:0.1628781259059906 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5851323008537292 - training/rollout_probs_diff_mean:0.00254165125079453 - training/rollout_probs_diff_std:0.009256764315068722 - training/rollout_actor_probs_pearson_corr:0.9989363551139832 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:0.9999262094497681 - rollout_corr/rollout_is_ratio_fraction_high:8.440314559265971e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.440314559265971e-06 - rollout_corr/rollout_is_max:2.55444073677063 - rollout_corr/rollout_is_min:0.05996687710285187 - rollout_corr/rollout_is_std:0.026260726153850555 - rollout_corr/rollout_is_eff_sample_size:0.9993107304708356 - rollout_corr/rollout_is_seq_mean:0.9999236464500427 - rollout_corr/rollout_is_seq_std:0.0016612117178738117 - rollout_corr/rollout_is_seq_max:1.0112076997756958 - rollout_corr/rollout_is_seq_min:0.9951014518737793 - rollout_corr/rollout_is_seq_max_deviation:0.0112076997756958 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1665030946023762) - rollout_corr/training_log_ppl:np.float64(0.14808018820622237) - rollout_corr/kl:np.float64(0.0003466195788832671) - rollout_corr/k3_kl:np.float64(0.000394960443784953) - rollout_corr/rollout_ppl:np.float64(1.1661016917787492) - rollout_corr/rollout_log_ppl:np.float64(0.14773356900695944) - rollout_corr/log_ppl_diff:np.float64(0.00034661919926293194) - rollout_corr/log_ppl_abs_diff:np.float64(0.001185306638944894) - rollout_corr/log_ppl_diff_max:np.float64(0.0072307586669921875) - rollout_corr/log_ppl_diff_min:np.float64(-0.009507015347480774) - rollout_corr/ppl_ratio:np.float64(1.000348127214238) - rollout_corr/chi2_token:np.float64(0.0008883415721356869) - rollout_corr/chi2_seq:np.float64(0.43197265616618097) - actor/pg_loss:np.float64(0.001996967475861311) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033745979953891947) - actor/ppo_kl:np.float64(-0.00012091028241578705) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(23806.097967893817) - self_distillation/token_reweight_w_std:np.float64(548794.9111263388) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001761829480529) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04754826093267184) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.15800323337316513) - perf/mfu/actor:np.float64(0.05552541876185921) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.99964141845703) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.66640625) - val-aux/sciknoweval/reward/std@16:np.float64(0.15799927809691777) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7303875) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12982374930438345) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.602) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13671534569253624) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6660875) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.15808013005567143) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7812875000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09494219426925349) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.54905) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10555818229721874) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6775375) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12453197346122731) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8188374999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06332935205697901) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.506175) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0748531273029532) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6804125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09154521211754381) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8425) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03640773357727453) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.47509999999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04919681483385159) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06299643554923988) - val-aux/sciknoweval/score/mean@16:np.float64(0.66640625) - val-aux/sciknoweval/score/std@16:np.float64(0.15799927809691777) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7303875) - val-aux/sciknoweval/score/best@2/std:np.float64(0.12982374930438345) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.602) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13671534569253624) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6660875) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.15808013005567143) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7812875000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09494219426925349) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.54905) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10555818229721874) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6775375) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12453197346122731) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8188374999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06332935205697901) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.506175) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0748531273029532) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6804125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09154521211754381) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8425) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03640773357727453) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.47509999999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04919681483385159) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06299643554923988) - val-core/sciknoweval/acc/mean@16:np.float64(0.66640625) - val-aux/sciknoweval/acc/std@16:np.float64(0.15799927809691777) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7303875) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.12982374930438345) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.602) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13671534569253624) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6660875) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.15808013005567143) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7812875000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09494219426925349) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.54905) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10555818229721874) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6775375) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12453197346122731) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8188374999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06332935205697901) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.506175) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0748531273029532) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6804125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09154521211754381) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8425) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03640773357727453) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.47509999999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04919681483385159) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6779125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06299643554923988) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9953125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.00881967330726803) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.004383097536411576) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9921) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.01028015103724269) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.995725) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.008610222519927225) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998750000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.001243734296383275) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.987675) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01048649118826383) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9985875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.004890128461522432) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9830249999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.008839286092192807) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0023919993524231077) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9794750000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.006665041979425554) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008666092529892921 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008666092529892921 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:462.80859375 - response_length/max:1515.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:462.80859375 - response_length_non_aborted/max:1515.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010422617197036743 - timing_s/agent_loop/generate_sequences/min:np.float64(1.178299294784665) - timing_s/agent_loop/generate_sequences/max:np.float64(11.539570804685354) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.233297567705449) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.539570804685354) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1515 - timing_s/gen:18.76097808778286 - timing_s/reward:0.06370272673666477 - timing_s/old_log_prob:3.0534738190472126 - timing_s/adv:0.6413047388195992 - timing_s/update_actor:22.003460492938757 - timing_s/step:44.61081040836871 - timing_s/testing:101.52909150719643 - timing_s/save_checkpoint:13.23071464523673 - timing_s/stop_profile:0.00013984739780426025 - timing_per_token_ms/gen:0.15834855196096237 - timing_per_token_ms/update_actor:0.11725985756734057 - timing_per_token_ms/adv:0.003417612532145993 - perf/total_num_tokens:187647 - perf/time_per_step:44.61081040836871 - perf/throughput:525.7890360045965 (TaskRunner pid=2320396) Training Progress: 80%|████████ | 80/100 [1:17:43<26:50, 80.53s/it] (TaskRunner pid=2320396) step:81 - global_seqlen/min:18108 - global_seqlen/max:30762 - global_seqlen/minmax_diff:12654 - global_seqlen/balanced_min:22819 - global_seqlen/balanced_max:22852 - global_seqlen/mean:22838.625 - actor/entropy:0.20225822925567627 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3244411051273346 - training/rollout_probs_diff_mean:0.002944577718153596 - training/rollout_probs_diff_std:0.009473892860114574 - training/rollout_actor_probs_pearson_corr:0.9991171360015869 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.0000029802322388 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.597196028858889e-06 - rollout_corr/rollout_is_max:1.6389256715774536 - rollout_corr/rollout_is_min:0.3698889911174774 - rollout_corr/rollout_is_std:0.028404373675584793 - rollout_corr/rollout_is_eff_sample_size:0.9991939609606479 - rollout_corr/rollout_is_seq_mean:0.9999433755874634 - rollout_corr/rollout_is_seq_std:0.0018434900557622313 - rollout_corr/rollout_is_seq_max:1.0091862678527832 - rollout_corr/rollout_is_seq_min:0.9904931783676147 - rollout_corr/rollout_is_seq_max_deviation:0.009506821632385254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2227952620014548) - rollout_corr/training_log_ppl:np.float64(0.19357780196150998) - rollout_corr/kl:np.float64(0.0005325028031997192) - rollout_corr/k3_kl:np.float64(0.0005149519024598703) - rollout_corr/rollout_ppl:np.float64(1.2221037549898028) - rollout_corr/rollout_log_ppl:np.float64(0.19304529835790163) - rollout_corr/log_ppl_diff:np.float64(0.0005325036036083475) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014249040541471913) - rollout_corr/log_ppl_diff_max:np.float64(0.010488063097000122) - rollout_corr/log_ppl_diff_min:np.float64(-0.008635297417640686) - rollout_corr/ppl_ratio:np.float64(1.0005347745027393) - rollout_corr/chi2_token:np.float64(0.0009839723352342844) - rollout_corr/chi2_seq:np.float64(0.3460533053148538) - actor/pg_loss:np.float64(0.0018761196406558156) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002735087759901944) - actor/ppo_kl:np.float64(2.139973337733636e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(36.513663759920746) - self_distillation/token_reweight_w_std:np.float64(801.7813025001087) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010359066072851) - self_distillation/token_reweight_w_clip_frac:np.float64(0.035162036278052256) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.14915302395820618) - perf/mfu/actor:np.float64(0.05383724667218048) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.4105339050293) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00048574156244285405 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00048574156244285405 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:454.36328125 - response_length/max:1584.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:454.36328125 - response_length_non_aborted/max:1584.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.142312824726105e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1165193412452936) - timing_s/agent_loop/generate_sequences/max:np.float64(12.071419509127736) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.783302926065517) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.071419509127736) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:1584 - timing_s/gen:18.30917154625058 - timing_s/reward:0.059204936027526855 - timing_s/old_log_prob:3.1515744253993034 - timing_s/adv:0.6499936617910862 - timing_s/update_actor:22.073494616895914 - timing_s/step:44.343954378739 - timing_s/stop_profile:0.0001219436526298523 - timing_per_token_ms/gen:0.15740752896180765 - timing_per_token_ms/update_actor:0.12081230052649795 - timing_per_token_ms/adv:0.0035575349971325235 - perf/total_num_tokens:182709 - perf/time_per_step:44.343954378739 - perf/throughput:515.0335670323107 (TaskRunner pid=2320396) Training Progress: 81%|████████ | 81/100 [1:18:27<22:04, 69.69s/it] (TaskRunner pid=2320396) step:82 - global_seqlen/min:18946 - global_seqlen/max:27759 - global_seqlen/minmax_diff:8813 - global_seqlen/balanced_min:24293 - global_seqlen/balanced_max:24305 - global_seqlen/mean:24300.75 - actor/entropy:0.1955031305551529 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19416940212249756 - training/rollout_probs_diff_mean:0.0027140500023961067 - training/rollout_probs_diff_std:0.009143715724349022 - training/rollout_actor_probs_pearson_corr:0.9990493059158325 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.00007963180542 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.615221844986081e-05 - rollout_corr/rollout_is_max:1.8455089330673218 - rollout_corr/rollout_is_min:0.38921883702278137 - rollout_corr/rollout_is_std:0.02782774716615677 - rollout_corr/rollout_is_eff_sample_size:0.9992262156609376 - rollout_corr/rollout_is_seq_mean:1.0001100301742554 - rollout_corr/rollout_is_seq_std:0.0014895005151629448 - rollout_corr/rollout_is_seq_max:1.0084476470947266 - rollout_corr/rollout_is_seq_min:0.9956014156341553 - rollout_corr/rollout_is_seq_max_deviation:0.008447647094726562 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.214292073622346) - rollout_corr/training_log_ppl:np.float64(0.18476674790144898) - rollout_corr/kl:np.float64(0.0003596869540132275) - rollout_corr/k3_kl:np.float64(0.00045131250168140014) - rollout_corr/rollout_ppl:np.float64(1.2138330452144146) - rollout_corr/rollout_log_ppl:np.float64(0.18440705924876966) - rollout_corr/log_ppl_diff:np.float64(0.00035968865267932415) - rollout_corr/log_ppl_abs_diff:np.float64(0.001154881319962442) - rollout_corr/log_ppl_diff_max:np.float64(0.0048035383224487305) - rollout_corr/log_ppl_diff_min:np.float64(-0.0042536258697509766) - rollout_corr/ppl_ratio:np.float64(1.0003608274273574) - rollout_corr/chi2_token:np.float64(0.0010943859815597534) - rollout_corr/chi2_seq:np.float64(0.6024468285031617) - actor/pg_loss:np.float64(0.0022809606743976474) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001371511980323703) - actor/ppo_kl:np.float64(7.002319382820588e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(3974.396525104763) - self_distillation/token_reweight_w_std:np.float64(79235.03549689136) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997991595882922) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03545600798679516) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.140237957239151) - perf/mfu/actor:np.float64(0.056121153281023836) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.15105438232422) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014725775457918644 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014725775457918644 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:483.6796875 - response_length/max:1464.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:483.6796875 - response_length_non_aborted/max:1464.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001256559044122696 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1039730682969093) - timing_s/agent_loop/generate_sequences/max:np.float64(11.90431728027761) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.50781928942888) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.90431728027761) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1464 - timing_s/gen:18.580683009698987 - timing_s/reward:0.0608761440962553 - timing_s/old_log_prob:3.0198480412364006 - timing_s/adv:0.5776665359735489 - timing_s/update_actor:22.33436026982963 - timing_s/step:44.61843103542924 - timing_s/stop_profile:0.00011967122554779053 - timing_per_token_ms/gen:0.15005962599294945 - timing_per_token_ms/update_actor:0.114885138677971 - timing_per_token_ms/adv:0.002971443967642711 - perf/total_num_tokens:194406 - perf/time_per_step:44.61843103542924 - perf/throughput:544.6347940989679 (TaskRunner pid=2320396) Training Progress: 82%|████████▏ | 82/100 [1:19:12<18:39, 62.18s/it] (TaskRunner pid=2320396) step:83 - global_seqlen/min:17814 - global_seqlen/max:27538 - global_seqlen/minmax_diff:9724 - global_seqlen/balanced_min:22306 - global_seqlen/balanced_max:22318 - global_seqlen/mean:22313.375 - actor/entropy:0.21886374056339264 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016381561756134 - training/rollout_probs_diff_mean:0.003045517485588789 - training/rollout_probs_diff_std:0.00972580537199974 - training/rollout_actor_probs_pearson_corr:0.9990494847297668 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.77734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:1.0000078678131104 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.788648271234706e-05 - rollout_corr/rollout_is_max:1.8235840797424316 - rollout_corr/rollout_is_min:0.38655397295951843 - rollout_corr/rollout_is_std:0.029688945040106773 - rollout_corr/rollout_is_eff_sample_size:0.9991191047550716 - rollout_corr/rollout_is_seq_mean:0.9999637007713318 - rollout_corr/rollout_is_seq_std:0.0019075077725574374 - rollout_corr/rollout_is_seq_max:1.0059213638305664 - rollout_corr/rollout_is_seq_min:0.9916067719459534 - rollout_corr/rollout_is_seq_max_deviation:0.00839322805404663 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2587532331235707) - rollout_corr/training_log_ppl:np.float64(0.215623820280598) - rollout_corr/kl:np.float64(0.00042866195441604305) - rollout_corr/k3_kl:np.float64(0.0005887536341475652) - rollout_corr/rollout_ppl:np.float64(1.2582748807035387) - rollout_corr/rollout_log_ppl:np.float64(0.21519515893305652) - rollout_corr/log_ppl_diff:np.float64(0.00042866134754149243) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015262856832123362) - rollout_corr/log_ppl_diff_max:np.float64(0.010632097721099854) - rollout_corr/log_ppl_diff_min:np.float64(-0.0100230872631073) - rollout_corr/ppl_ratio:np.float64(1.000431136926636) - rollout_corr/chi2_token:np.float64(0.0015211885329335928) - rollout_corr/chi2_seq:np.float64(1.5993264480493963) - actor/pg_loss:np.float64(0.003375511965714395) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044200276761330315) - actor/ppo_kl:np.float64(-9.806516564747625e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_w_mean:np.float64(13916.900597623782) - self_distillation/token_reweight_w_std:np.float64(302939.21583341405) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994024811312556) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06193480538786389) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.2252168245613575) - perf/mfu/actor:np.float64(0.05314491856218738) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.17446899414062) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007381784729659557 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007381784729659557 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.23046875 - response_length/max:1116.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.23046875 - response_length_non_aborted/max:1116.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001424010843038559 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7883939687162638) - timing_s/agent_loop/generate_sequences/max:np.float64(9.522131288424134) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.651366425387096) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.522131288424134) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1116 - timing_s/gen:16.62836728617549 - timing_s/reward:0.06175902858376503 - timing_s/old_log_prob:3.2153992895036936 - timing_s/adv:0.6615845076739788 - timing_s/update_actor:21.87908287346363 - timing_s/step:42.53540291264653 - timing_s/stop_profile:0.00012497231364250183 - timing_per_token_ms/gen:0.15456889621743547 - timing_per_token_ms/update_actor:0.12256708629613197 - timing_per_token_ms/adv:0.0037062104437023693 - perf/total_num_tokens:178507 - perf/time_per_step:42.53540291264653 - perf/throughput:524.5836049989746 (TaskRunner pid=2320396) Training Progress: 83%|████████▎ | 83/100 [1:19:54<15:57, 56.30s/it] (TaskRunner pid=2320396) step:84 - global_seqlen/min:19307 - global_seqlen/max:29887 - global_seqlen/minmax_diff:10580 - global_seqlen/balanced_min:23081 - global_seqlen/balanced_max:23100 - global_seqlen/mean:23097.0 - actor/entropy:0.2354067713022232 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35205382108688354 - training/rollout_probs_diff_mean:0.003032247070223093 - training/rollout_probs_diff_std:0.009490408934652805 - training/rollout_actor_probs_pearson_corr:0.9991567730903625 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000667572021484 - rollout_corr/rollout_is_ratio_fraction_high:8.543503099645022e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.543503099645022e-06 - rollout_corr/rollout_is_max:2.8205838203430176 - rollout_corr/rollout_is_min:0.21296259760856628 - rollout_corr/rollout_is_std:0.028636353090405464 - rollout_corr/rollout_is_eff_sample_size:0.9991808692410266 - rollout_corr/rollout_is_seq_mean:1.000049352645874 - rollout_corr/rollout_is_seq_std:0.0015556849539279938 - rollout_corr/rollout_is_seq_max:1.00594961643219 - rollout_corr/rollout_is_seq_min:0.9951849579811096 - rollout_corr/rollout_is_seq_max_deviation:0.005949616432189941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2848193626850843) - rollout_corr/training_log_ppl:np.float64(0.22980090814235155) - rollout_corr/kl:np.float64(0.0004010426303544534) - rollout_corr/k3_kl:np.float64(0.0005172141459865998) - rollout_corr/rollout_ppl:np.float64(1.284251406788826) - rollout_corr/rollout_log_ppl:np.float64(0.22939986467099516) - rollout_corr/log_ppl_diff:np.float64(0.0004010434713563882) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013402195836533792) - rollout_corr/log_ppl_diff_max:np.float64(0.0082988440990448) - rollout_corr/log_ppl_diff_min:np.float64(-0.015117630362510681) - rollout_corr/ppl_ratio:np.float64(1.0004031020216644) - rollout_corr/chi2_token:np.float64(0.0013102053198963404) - rollout_corr/chi2_seq:np.float64(0.600376995280385) - actor/pg_loss:np.float64(0.002821797039359808) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017985764498007484) - actor/ppo_kl:np.float64(1.2684636487847456e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(3089.401668926468) - self_distillation/token_reweight_w_std:np.float64(78661.03726228699) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996353969909251) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05245249951258302) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.16984390653669834) - perf/mfu/actor:np.float64(0.05433426907698578) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.13277435302734) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003203813685104251 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0003203813685104251 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:457.21875 - response_length/max:1337.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:457.21875 - response_length_non_aborted/max:1337.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015847571194171906 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1861316971480846) - timing_s/agent_loop/generate_sequences/max:np.float64(10.804910905659199) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.034344409927144) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.804910905659199) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:1337 - timing_s/gen:17.917205162346363 - timing_s/reward:0.05784008279442787 - timing_s/old_log_prob:3.1344593465328217 - timing_s/adv:0.6041016634553671 - timing_s/update_actor:22.013284035027027 - timing_s/step:43.842269690707326 - timing_s/stop_profile:0.00012374669313430786 - timing_per_token_ms/gen:0.15307570537169676 - timing_per_token_ms/update_actor:0.11913497442864347 - timing_per_token_ms/adv:0.003269372989215954 - perf/total_num_tokens:184776 - perf/time_per_step:43.842269690707326 - perf/throughput:526.8203531190716 (TaskRunner pid=2320396) Training Progress: 84%|████████▍ | 84/100 [1:20:38<14:01, 52.58s/it] (TaskRunner pid=2320396) step:85 - global_seqlen/min:15110 - global_seqlen/max:28870 - global_seqlen/minmax_diff:13760 - global_seqlen/balanced_min:22104 - global_seqlen/balanced_max:22120 - global_seqlen/mean:22112.25 - actor/entropy:0.21829085052013397 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25521230697631836 - training/rollout_probs_diff_mean:0.002716940827667713 - training/rollout_probs_diff_std:0.009013369679450989 - training/rollout_actor_probs_pearson_corr:0.9992413520812988 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0001059770584106 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8207971152151003e-05 - rollout_corr/rollout_is_max:1.9953315258026123 - rollout_corr/rollout_is_min:0.2880904972553253 - rollout_corr/rollout_is_std:0.0274242851883173 - rollout_corr/rollout_is_eff_sample_size:0.9992487118690302 - rollout_corr/rollout_is_seq_mean:1.0001178979873657 - rollout_corr/rollout_is_seq_std:0.0018713462632149458 - rollout_corr/rollout_is_seq_max:1.0108190774917603 - rollout_corr/rollout_is_seq_min:0.9933320879936218 - rollout_corr/rollout_is_seq_max_deviation:0.010819077491760254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2405682974494994) - rollout_corr/training_log_ppl:np.float64(0.2008585182193201) - rollout_corr/kl:np.float64(0.00034136320786615215) - rollout_corr/k3_kl:np.float64(0.00045656332018495505) - rollout_corr/rollout_ppl:np.float64(1.240147914737463) - rollout_corr/rollout_log_ppl:np.float64(0.2005171545824851) - rollout_corr/log_ppl_diff:np.float64(0.0003413636368350126) - rollout_corr/log_ppl_abs_diff:np.float64(0.001429379561159294) - rollout_corr/log_ppl_diff_max:np.float64(0.006210342049598694) - rollout_corr/log_ppl_diff_min:np.float64(-0.009608462452888489) - rollout_corr/ppl_ratio:np.float64(1.000343416351825) - rollout_corr/chi2_token:np.float64(0.0011162424925714731) - rollout_corr/chi2_seq:np.float64(0.7462084121070802) - actor/pg_loss:np.float64(0.002974406350404024) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002645291015141993) - actor/ppo_kl:np.float64(5.648398862057036e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(819.4280955540016) - self_distillation/token_reweight_w_std:np.float64(19350.410570348613) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996866395231336) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0477723392250482) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.1905206125229597) - perf/mfu/actor:np.float64(0.051957974999849506) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.11687469482422) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004917290061712265 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004917290061712265 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:429.0703125 - response_length/max:1578.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:429.0703125 - response_length_non_aborted/max:1578.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001225452870130539 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7567968033254147) - timing_s/agent_loop/generate_sequences/max:np.float64(12.397239990532398) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.498732063300849) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.397239990532398) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:363 - timing_s/agent_loop/slowest/response_length:1578 - timing_s/gen:19.835766257718205 - timing_s/reward:0.0627602580934763 - timing_s/old_log_prob:3.027434177696705 - timing_s/adv:0.630172599107027 - timing_s/update_actor:22.07317166402936 - timing_s/step:45.71889763139188 - timing_s/stop_profile:0.00011015869677066803 - timing_per_token_ms/gen:0.18058453285371903 - timing_per_token_ms/update_actor:0.12477909113743152 - timing_per_token_ms/adv:0.003562350049785905 - perf/total_num_tokens:176898 - perf/time_per_step:45.71889763139188 - perf/throughput:483.65667471424564 (TaskRunner pid=2320396) Training Progress: 85%|████████▌ | 85/100 [1:21:24<12:38, 50.54s/it] (TaskRunner pid=2320396) step:86 - global_seqlen/min:18259 - global_seqlen/max:24454 - global_seqlen/minmax_diff:6195 - global_seqlen/balanced_min:22310 - global_seqlen/balanced_max:22317 - global_seqlen/mean:22315.25 - actor/entropy:0.19693319499492645 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9269568920135498 - training/rollout_probs_diff_mean:0.002871491713449359 - training/rollout_probs_diff_std:0.010464465245604515 - training/rollout_actor_probs_pearson_corr:0.99885493516922 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0000848770141602 - rollout_corr/rollout_is_ratio_fraction_high:1.8361763068242e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.5085289204725996e-05 - rollout_corr/rollout_is_max:4.428065299987793 - rollout_corr/rollout_is_min:0.00090052280575037 - rollout_corr/rollout_is_std:0.0283602736890316 - rollout_corr/rollout_is_eff_sample_size:0.999196341310165 - rollout_corr/rollout_is_seq_mean:1.0001143217086792 - rollout_corr/rollout_is_seq_std:0.001787590328603983 - rollout_corr/rollout_is_seq_max:1.0075844526290894 - rollout_corr/rollout_is_seq_min:0.9934707283973694 - rollout_corr/rollout_is_seq_max_deviation:0.0075844526290893555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2275358876213431) - rollout_corr/training_log_ppl:np.float64(0.19244293621886754) - rollout_corr/kl:np.float64(0.0004759999551815497) - rollout_corr/k3_kl:np.float64(0.0006213987889509553) - rollout_corr/rollout_ppl:np.float64(1.22697224188596) - rollout_corr/rollout_log_ppl:np.float64(0.19196693430421874) - rollout_corr/log_ppl_diff:np.float64(0.0004760019146488048) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014411285155802034) - rollout_corr/log_ppl_diff_max:np.float64(0.009107738733291626) - rollout_corr/log_ppl_diff_min:np.float64(-0.007479369640350342) - rollout_corr/ppl_ratio:np.float64(1.0004782637115568) - rollout_corr/chi2_token:np.float64(0.001509897643700242) - rollout_corr/chi2_seq:np.float64(0.7807369504589587) - actor/pg_loss:np.float64(0.003120149136520922) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000514148305001072) - actor/ppo_kl:np.float64(1.555324082502807e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(28969.59762649727) - self_distillation/token_reweight_w_std:np.float64(541568.6326414472) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999872143380344) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05786374416493345) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.21097707375884056) - perf/mfu/actor:np.float64(0.05366693062582211) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.08077239990234) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0003144452057313174 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0003144452057313174 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:425.4765625 - response_length/max:1046.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:425.4765625 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001625940203666687 - timing_s/agent_loop/generate_sequences/min:np.float64(1.250269953161478) - timing_s/agent_loop/generate_sequences/max:np.float64(9.116639155894518) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7879958484263625) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.116639155894518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:244 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:15.873301655054092 - timing_s/reward:0.06210778281092644 - timing_s/old_log_prob:2.9981462005525827 - timing_s/adv:0.5681248959153891 - timing_s/update_actor:21.514174001291394 - timing_s/step:41.10377608239651 - timing_s/stop_profile:2.7652829885482788e-05 - timing_per_token_ms/gen:0.14573090518953097 - timing_per_token_ms/update_actor:0.12051273233154118 - timing_per_token_ms/adv:0.0031823802999932167 - perf/total_num_tokens:178522 - perf/time_per_step:41.10377608239651 - perf/throughput:542.9002424319098 (TaskRunner pid=2320396) Training Progress: 86%|████████▌ | 86/100 [1:22:05<11:07, 47.71s/it] (TaskRunner pid=2320396) step:87 - global_seqlen/min:20010 - global_seqlen/max:25380 - global_seqlen/minmax_diff:5370 - global_seqlen/balanced_min:22327 - global_seqlen/balanced_max:22346 - global_seqlen/mean:22340.875 - actor/entropy:0.2229112833738327 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28752514719963074 - training/rollout_probs_diff_mean:0.00292000244371593 - training/rollout_probs_diff_std:0.009206824004650116 - training/rollout_actor_probs_pearson_corr:0.9992163777351379 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.99995356798172 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.715296770678833e-05 - rollout_corr/rollout_is_max:1.7399653196334839 - rollout_corr/rollout_is_min:0.386408269405365 - rollout_corr/rollout_is_std:0.027554381638765335 - rollout_corr/rollout_is_eff_sample_size:0.999241213020451 - rollout_corr/rollout_is_seq_mean:0.9999908208847046 - rollout_corr/rollout_is_seq_std:0.0016598277725279331 - rollout_corr/rollout_is_seq_max:1.0081655979156494 - rollout_corr/rollout_is_seq_min:0.9906253814697266 - rollout_corr/rollout_is_seq_max_deviation:0.009374618530273438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2702338360249996) - rollout_corr/training_log_ppl:np.float64(0.22234329585626256) - rollout_corr/kl:np.float64(0.0005667585143394049) - rollout_corr/k3_kl:np.float64(0.0004980311200313281) - rollout_corr/rollout_ppl:np.float64(1.2695111935026944) - rollout_corr/rollout_log_ppl:np.float64(0.22177653654216556) - rollout_corr/log_ppl_diff:np.float64(0.0005667593140969984) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014052593251108192) - rollout_corr/log_ppl_diff_max:np.float64(0.012764573097229004) - rollout_corr/log_ppl_diff_min:np.float64(-0.004403352737426758) - rollout_corr/ppl_ratio:np.float64(1.000568661140278) - rollout_corr/chi2_token:np.float64(0.000875755213201046) - rollout_corr/chi2_seq:np.float64(0.24442025600001216) - actor/pg_loss:np.float64(0.0031820654403418303) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029767151454507257) - actor/ppo_kl:np.float64(0.00015311227060976762) - actor/pg_clipfrac_lower:np.float64(4.398930286697578e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(1033.2029310131911) - self_distillation/token_reweight_w_std:np.float64(21538.62358322338) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992069352883846) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06586394051555544) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.21494160592556) - perf/mfu/actor:np.float64(0.05288507261784649) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.0767822265625) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029490168672055006 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0029490168672055006 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:420.55859375 - response_length/max:1262.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.55859375 - response_length_non_aborted/max:1262.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.940774619579315e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.18855987675488) - timing_s/agent_loop/generate_sequences/max:np.float64(9.835301402956247) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.683391153987031) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.835301402956247) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:256 - timing_s/agent_loop/slowest/response_length:1262 - timing_s/gen:16.892137264832854 - timing_s/reward:0.06300836428999901 - timing_s/old_log_prob:2.9979468770325184 - timing_s/adv:0.5653424803167582 - timing_s/update_actor:22.004743106663227 - timing_s/step:42.61258958093822 - timing_s/stop_profile:0.00011842325329780579 - timing_per_token_ms/gen:0.15689825905680554 - timing_per_token_ms/update_actor:0.12311929986327319 - timing_per_token_ms/adv:0.0031631621429149383 - perf/total_num_tokens:178727 - perf/time_per_step:42.61258958093822 - perf/throughput:524.2787453122465 (TaskRunner pid=2320396) Training Progress: 87%|████████▋ | 87/100 [1:22:48<10:00, 46.20s/it] (TaskRunner pid=2320396) step:88 - global_seqlen/min:18099 - global_seqlen/max:28822 - global_seqlen/minmax_diff:10723 - global_seqlen/balanced_min:23419 - global_seqlen/balanced_max:23442 - global_seqlen/mean:23436.25 - actor/entropy:0.21247567236423492 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29311496019363403 - training/rollout_probs_diff_mean:0.0028727473691105843 - training/rollout_probs_diff_std:0.00918804481625557 - training/rollout_actor_probs_pearson_corr:0.9991670250892639 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9998396039009094 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.663644434302114e-05 - rollout_corr/rollout_is_max:1.599510669708252 - rollout_corr/rollout_is_min:0.4783361852169037 - rollout_corr/rollout_is_std:0.027528412640094757 - rollout_corr/rollout_is_eff_sample_size:0.9992425223351087 - rollout_corr/rollout_is_seq_mean:0.9999282360076904 - rollout_corr/rollout_is_seq_std:0.0018976305145770311 - rollout_corr/rollout_is_seq_max:1.0152655839920044 - rollout_corr/rollout_is_seq_min:0.9939031600952148 - rollout_corr/rollout_is_seq_max_deviation:0.015265583992004395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2466978626325727) - rollout_corr/training_log_ppl:np.float64(0.2076665610074997) - rollout_corr/kl:np.float64(0.00044316515954800906) - rollout_corr/k3_kl:np.float64(0.00045984440038182584) - rollout_corr/rollout_ppl:np.float64(1.2460913262329996) - rollout_corr/rollout_log_ppl:np.float64(0.20722339424537495) - rollout_corr/log_ppl_diff:np.float64(0.00044316676212474704) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013816133432555944) - rollout_corr/log_ppl_diff_max:np.float64(0.006623566150665283) - rollout_corr/log_ppl_diff_min:np.float64(-0.01356533169746399) - rollout_corr/ppl_ratio:np.float64(1.0004452236462384) - rollout_corr/chi2_token:np.float64(0.0009807520546019077) - rollout_corr/chi2_seq:np.float64(2.0590124474838376) - actor/pg_loss:np.float64(0.003491878043860197) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002180453784603742) - actor/ppo_kl:np.float64(-3.431808095299971e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(4024.9873863377143) - self_distillation/token_reweight_w_std:np.float64(95798.2680301803) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008977930992842) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06488000725221355) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.18007886037230492) - perf/mfu/actor:np.float64(0.0547133357028863) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.0287628173828) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004015621729195118 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004015621729195118 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:469.6015625 - response_length/max:1509.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:469.6015625 - response_length_non_aborted/max:1509.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010225735604763031 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9620911218225956) - timing_s/agent_loop/generate_sequences/max:np.float64(11.938467379659414) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.014028608202352) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.938467379659414) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1509 - timing_s/gen:18.666382912546396 - timing_s/reward:0.06460434012115002 - timing_s/old_log_prob:3.040660046041012 - timing_s/adv:0.6120849438011646 - timing_s/update_actor:22.133079420775175 - timing_s/step:44.60495771467686 - timing_s/stop_profile:0.00011262297630310059 - timing_per_token_ms/gen:0.15527111507882677 - timing_per_token_ms/update_actor:0.11804938621139888 - timing_per_token_ms/adv:0.0032646271470540542 - perf/total_num_tokens:187490 - perf/time_per_step:44.60495771467686 - perf/throughput:525.4180521796238 (TaskRunner pid=2320396) Training Progress: 88%|████████▊ | 88/100 [1:23:33<09:08, 45.73s/it] (TaskRunner pid=2320396) step:89 - global_seqlen/min:14328 - global_seqlen/max:25446 - global_seqlen/minmax_diff:11118 - global_seqlen/balanced_min:20881 - global_seqlen/balanced_max:20887 - global_seqlen/mean:20883.875 - actor/entropy:0.2559927701950073 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2545143961906433 - training/rollout_probs_diff_mean:0.0030213401187211275 - training/rollout_probs_diff_std:0.009110929444432259 - training/rollout_actor_probs_pearson_corr:0.9992585778236389 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:1.0 - self_distillation/correct_sample_fraction:0.8671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999969601631165 - rollout_corr/rollout_is_ratio_fraction_high:2.045680048468057e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0228400242340285e-05 - rollout_corr/rollout_is_max:2.7041423320770264 - rollout_corr/rollout_is_min:0.4614633619785309 - rollout_corr/rollout_is_std:0.028393879532814026 - rollout_corr/rollout_is_eff_sample_size:0.9991943180123525 - rollout_corr/rollout_is_seq_mean:0.9998977780342102 - rollout_corr/rollout_is_seq_std:0.0016858673188835382 - rollout_corr/rollout_is_seq_max:1.0047959089279175 - rollout_corr/rollout_is_seq_min:0.992766797542572 - rollout_corr/rollout_is_seq_max_deviation:0.0072332024574279785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.305208371952176) - rollout_corr/training_log_ppl:np.float64(0.24501134573074523) - rollout_corr/kl:np.float64(0.0007130547365559181) - rollout_corr/k3_kl:np.float64(0.0005163975527082698) - rollout_corr/rollout_ppl:np.float64(1.3042262229137123) - rollout_corr/rollout_log_ppl:np.float64(0.24429829073051224) - rollout_corr/log_ppl_diff:np.float64(0.0007130550002329983) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015240707798511721) - rollout_corr/log_ppl_diff_max:np.float64(0.009647458791732788) - rollout_corr/log_ppl_diff_min:np.float64(-0.011320427060127258) - rollout_corr/ppl_ratio:np.float64(1.000715468544513) - rollout_corr/chi2_token:np.float64(0.000679319491609931) - rollout_corr/chi2_seq:np.float64(0.30747226788662374) - actor/pg_loss:np.float64(0.0026361719937995076) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025667395266282256) - actor/ppo_kl:np.float64(0.00020520936391221767) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_w_mean:np.float64(5014.568350150017) - self_distillation/token_reweight_w_std:np.float64(103249.80037083285) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.998729647602886) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07348214878584258) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.2412368506193161) - perf/mfu/actor:np.float64(0.050846935709280464) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.622971534729) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.8671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010872789658606052 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010872789658606052 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:381.90234375 - response_length/max:1035.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.90234375 - response_length_non_aborted/max:1035.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002514813095331192 - timing_s/agent_loop/generate_sequences/min:np.float64(0.941523926332593) - timing_s/agent_loop/generate_sequences/max:np.float64(8.54596209153533) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.18014469731861) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.54596209153533) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1035 - timing_s/gen:15.545465314760804 - timing_s/reward:0.059370189905166626 - timing_s/old_log_prob:3.289316363632679 - timing_s/adv:0.6208392586559057 - timing_s/update_actor:21.38644278794527 - timing_s/step:40.9937635269016 - timing_s/stop_profile:0.0001326780766248703 - timing_per_token_ms/gen:0.15900524016038953 - timing_per_token_ms/update_actor:0.12800810905510393 - timing_per_token_ms/adv:0.0037160204862358263 - perf/total_num_tokens:167071 - perf/time_per_step:40.9937635269016 - perf/throughput:509.44029538286327 (TaskRunner pid=2320396) Training Progress: 89%|████████▉ | 89/100 [1:24:14<08:09, 44.46s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Rearranging to solve for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the given values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is B: 56 pC. (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2320396) step:90 - global_seqlen/min:17689 - global_seqlen/max:24738 - global_seqlen/minmax_diff:7049 - global_seqlen/balanced_min:20971 - global_seqlen/balanced_max:20992 - global_seqlen/mean:20984.875 - actor/entropy:0.21077261865139008 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3020694851875305 - training/rollout_probs_diff_mean:0.002769892103970051 - training/rollout_probs_diff_std:0.008872252888977528 - training/rollout_actor_probs_pearson_corr:0.9991670846939087 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.77734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999796748161316 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8738949298858643 - rollout_corr/rollout_is_min:0.609788715839386 - rollout_corr/rollout_is_std:0.026957370340824127 - rollout_corr/rollout_is_eff_sample_size:0.9992737088438486 - rollout_corr/rollout_is_seq_mean:0.9999539852142334 - rollout_corr/rollout_is_seq_std:0.0014758005272597075 - rollout_corr/rollout_is_seq_max:1.004751205444336 - rollout_corr/rollout_is_seq_min:0.9945812821388245 - rollout_corr/rollout_is_seq_max_deviation:0.005418717861175537 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2639609524048865) - rollout_corr/training_log_ppl:np.float64(0.20912245583895128) - rollout_corr/kl:np.float64(0.0004431889581955062) - rollout_corr/k3_kl:np.float64(0.0004347639480215548) - rollout_corr/rollout_ppl:np.float64(1.2633680049329996) - rollout_corr/rollout_log_ppl:np.float64(0.20867926441133022) - rollout_corr/log_ppl_diff:np.float64(0.0004431914276210591) - rollout_corr/log_ppl_abs_diff:np.float64(0.001261206591152586) - rollout_corr/log_ppl_diff_max:np.float64(0.006049171090126038) - rollout_corr/log_ppl_diff_min:np.float64(-0.005435943603515625) - rollout_corr/ppl_ratio:np.float64(1.0004446292296052) - rollout_corr/chi2_token:np.float64(0.0008622340392321348) - rollout_corr/chi2_seq:np.float64(0.27305450290441513) - actor/pg_loss:np.float64(0.0014611287042498589) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015134929981286405) - actor/ppo_kl:np.float64(2.253385616413084e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_w_mean:np.float64(232.24152618390508) - self_distillation/token_reweight_w_std:np.float64(5283.819025855686) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993226558435708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045578876917716116) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.14182636328041553) - perf/mfu/actor:np.float64(0.05110420901267789) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(197.68716478347778) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.68046875) - val-aux/sciknoweval/reward/std@16:np.float64(0.17636502357475656) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7504125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13927109847218053) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6064499999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1588835992693667) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.67705) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1764893123600096) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8047875000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09485965224518725) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.54345) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12541633861008067) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7016249999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14140393577456317) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8390375000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.061201619852513364) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.493675) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09032488761637099) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7152875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10853346823413709) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8626250000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03995442637984035) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.45824999999999994) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05945111823670979) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7266375) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.08030918738156603) - val-aux/sciknoweval/score/mean@16:np.float64(0.68046875) - val-aux/sciknoweval/score/std@16:np.float64(0.17636502357475656) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7504125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13927109847218053) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6064499999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1588835992693667) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.67705) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1764893123600096) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8047875000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09485965224518725) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.54345) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12541633861008067) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7016249999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14140393577456317) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8390375000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.061201619852513364) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.493675) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09032488761637099) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7152875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10853346823413709) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8626250000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03995442637984035) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.45824999999999994) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05945111823670979) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7266375) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.08030918738156603) - val-core/sciknoweval/acc/mean@16:np.float64(0.68046875) - val-aux/sciknoweval/acc/std@16:np.float64(0.17636502357475656) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7504125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13927109847218053) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6064499999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1588835992693667) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.67705) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1764893123600096) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8047875000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09485965224518725) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.54345) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12541633861008067) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7016249999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14140393577456317) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8390375000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.061201619852513364) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.493675) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09032488761637099) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7152875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10853346823413709) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8626250000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03995442637984035) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.45824999999999994) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05945111823670979) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7266375) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.08030918738156603) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.984375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.035649330156956295) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9954750000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.013924756097101748) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9727875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.044533748832844) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9836625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.036533904346640875) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9990249999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0033521448357730606) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.957075) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05169379024898031) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9919875000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.024094702579460316) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0011135528725660043) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9350624999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.054899137382862784) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9971375) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.01159806300293624) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9084125000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04978662691013876) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9994375) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00405710688339734) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013788010692223907 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013788010692223907 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:388.83984375 - response_length/max:1336.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:388.83984375 - response_length_non_aborted/max:1336.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011170096695423126 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5990730039775372) - timing_s/agent_loop/generate_sequences/max:np.float64(10.28107376769185) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.295882520542364) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.28107376769185) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:318 - timing_s/agent_loop/slowest/response_length:1336 - timing_s/gen:16.937936026602983 - timing_s/reward:0.06041034683585167 - timing_s/old_log_prob:2.971561659127474 - timing_s/adv:0.5751631949096918 - timing_s/update_actor:21.22050972469151 - timing_s/step:41.851907912641764 - timing_s/testing:106.41391924209893 - timing_s/save_checkpoint:12.91794090718031 - timing_s/stop_profile:0.00016064755618572235 - timing_per_token_ms/gen:0.1701569776539082 - timing_per_token_ms/update_actor:0.12640359857213535 - timing_per_token_ms/adv:0.0034260580233959684 - perf/total_num_tokens:167879 - perf/time_per_step:41.851907912641764 - perf/throughput:501.40784606049755 (TaskRunner pid=2320396) Training Progress: 90%|█████████ | 90/100 [1:26:55<13:15, 79.50s/it] (TaskRunner pid=2320396) step:91 - global_seqlen/min:18599 - global_seqlen/max:26740 - global_seqlen/minmax_diff:8141 - global_seqlen/balanced_min:21857 - global_seqlen/balanced_max:21878 - global_seqlen/mean:21869.625 - actor/entropy:0.18502238392829895 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016291558742523 - training/rollout_probs_diff_mean:0.002582877641543746 - training/rollout_probs_diff_std:0.008864011615514755 - training/rollout_actor_probs_pearson_corr:0.9991196990013123 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.0000965595245361 - rollout_corr/rollout_is_ratio_fraction_high:9.062814569915645e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2347850799560547 - rollout_corr/rollout_is_min:0.5558784604072571 - rollout_corr/rollout_is_std:0.02634231001138687 - rollout_corr/rollout_is_eff_sample_size:0.9993068020008989 - rollout_corr/rollout_is_seq_mean:1.0000598430633545 - rollout_corr/rollout_is_seq_std:0.0019468516111373901 - rollout_corr/rollout_is_seq_max:1.01114022731781 - rollout_corr/rollout_is_seq_min:0.9909047484397888 - rollout_corr/rollout_is_seq_max_deviation:0.011140227317810059 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1936893849633634) - rollout_corr/training_log_ppl:np.float64(0.1685559979378013) - rollout_corr/kl:np.float64(0.00032212701280531775) - rollout_corr/k3_kl:np.float64(0.0004202743424741584) - rollout_corr/rollout_ppl:np.float64(1.1933098202571273) - rollout_corr/rollout_log_ppl:np.float64(0.1682338698883541) - rollout_corr/log_ppl_diff:np.float64(0.0003221280494472012) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013454183208523318) - rollout_corr/log_ppl_diff_max:np.float64(0.010432571172714233) - rollout_corr/log_ppl_diff_min:np.float64(-0.004743427038192749) - rollout_corr/ppl_ratio:np.float64(1.0003239274956286) - rollout_corr/chi2_token:np.float64(0.0010666304733604193) - rollout_corr/chi2_seq:np.float64(0.7431910112500191) - actor/pg_loss:np.float64(0.0014502984704449773) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001065377177837945) - actor/ppo_kl:np.float64(9.484854068375626e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_w_mean:np.float64(88.4975810085889) - self_distillation/token_reweight_w_std:np.float64(2492.829599480203) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999885968863964) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037433789286296815) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.14290286786854267) - perf/mfu/actor:np.float64(0.04989908695015073) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.37800979614258) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003633055603131652 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003633055603131652 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:431.01953125 - response_length/max:1378.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:431.01953125 - response_length_non_aborted/max:1378.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011483579874038696 - timing_s/agent_loop/generate_sequences/min:np.float64(1.07878920994699) - timing_s/agent_loop/generate_sequences/max:np.float64(10.949054036289454) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.725024271836446) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.949054036289454) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:1378 - timing_s/gen:17.278234027326107 - timing_s/reward:0.057705922052264214 - timing_s/old_log_prob:3.1367397028952837 - timing_s/adv:0.6056630592793226 - timing_s/update_actor:22.86518513597548 - timing_s/step:44.04848053306341 - timing_s/stop_profile:0.00012228451669216156 - timing_per_token_ms/gen:0.15658942756841163 - timing_per_token_ms/update_actor:0.13069031325397373 - timing_per_token_ms/adv:0.0034617823766944027 - perf/total_num_tokens:174957 - perf/time_per_step:44.04848053306341 - perf/throughput:496.48988422164416 (TaskRunner pid=2320396) Training Progress: 91%|█████████ | 91/100 [1:27:39<10:19, 68.88s/it] (TaskRunner pid=2320396) step:92 - global_seqlen/min:18621 - global_seqlen/max:25406 - global_seqlen/minmax_diff:6785 - global_seqlen/balanced_min:22801 - global_seqlen/balanced_max:22815 - global_seqlen/mean:22805.625 - actor/entropy:0.2772010862827301 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9653425812721252 - training/rollout_probs_diff_mean:0.003166171023622155 - training/rollout_probs_diff_std:0.009848006069660187 - training/rollout_actor_probs_pearson_corr:0.9991579055786133 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.0000650882720947 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6977204470313154e-05 - rollout_corr/rollout_is_max:1.9052884578704834 - rollout_corr/rollout_is_min:0.0294669046998024 - rollout_corr/rollout_is_std:0.029451090842485428 - rollout_corr/rollout_is_eff_sample_size:0.9991336228879825 - rollout_corr/rollout_is_seq_mean:1.0001696348190308 - rollout_corr/rollout_is_seq_std:0.002067823428660631 - rollout_corr/rollout_is_seq_max:1.0114904642105103 - rollout_corr/rollout_is_seq_min:0.9938461184501648 - rollout_corr/rollout_is_seq_max_deviation:0.011490464210510254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3618069398216903) - rollout_corr/training_log_ppl:np.float64(0.28330526626086794) - rollout_corr/kl:np.float64(0.0004403062517628342) - rollout_corr/k3_kl:np.float64(0.000519837122439526) - rollout_corr/rollout_ppl:np.float64(1.36120031401515) - rollout_corr/rollout_log_ppl:np.float64(0.28286495895008557) - rollout_corr/log_ppl_diff:np.float64(0.00044030731078237295) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013980344228912145) - rollout_corr/log_ppl_diff_max:np.float64(0.007395714521408081) - rollout_corr/log_ppl_diff_min:np.float64(-0.006245464086532593) - rollout_corr/ppl_ratio:np.float64(1.000442171934992) - rollout_corr/chi2_token:np.float64(0.0012069738004356623) - rollout_corr/chi2_seq:np.float64(0.3731236432213336) - actor/pg_loss:np.float64(0.002277109189890325) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020634635779970267) - actor/ppo_kl:np.float64(9.285395801583718e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(1179.5885476176627) - self_distillation/token_reweight_w_std:np.float64(24205.86808684515) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999315890017897) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06046442501246929) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.1943492740392685) - perf/mfu/actor:np.float64(0.053056719866905494) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.21900939941406) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0032271479722112417 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0032271479722112417 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.39453125 - response_length/max:1221.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.39453125 - response_length_non_aborted/max:1221.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001361556351184845 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0998735800385475) - timing_s/agent_loop/generate_sequences/max:np.float64(10.115503538399935) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7110357238780125) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.115503538399935) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1221 - timing_s/gen:16.898210391402245 - timing_s/reward:0.06314759328961372 - timing_s/old_log_prob:2.9814101308584213 - timing_s/adv:0.6085547208786011 - timing_s/update_actor:22.09982367232442 - timing_s/step:42.74151246063411 - timing_s/stop_profile:0.00012245215475559235 - timing_per_token_ms/gen:0.15195549113261314 - timing_per_token_ms/update_actor:0.1211314295942581 - timing_per_token_ms/adv:0.003335551650517148 - perf/total_num_tokens:182445 - perf/time_per_step:42.74151246063411 - perf/throughput:533.5708468670707 (TaskRunner pid=2320396) Training Progress: 92%|█████████▏| 92/100 [1:28:22<08:08, 61.05s/it] (TaskRunner pid=2320396) step:93 - global_seqlen/min:14223 - global_seqlen/max:23452 - global_seqlen/minmax_diff:9229 - global_seqlen/balanced_min:19017 - global_seqlen/balanced_max:19031 - global_seqlen/mean:19026.25 - actor/entropy:0.27642500400543213 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34021443128585815 - training/rollout_probs_diff_mean:0.003276179078966379 - training/rollout_probs_diff_std:0.009843645617365837 - training/rollout_actor_probs_pearson_corr:0.9992181062698364 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:1.00007164478302 - rollout_corr/rollout_is_ratio_fraction_high:1.1579166312003508e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.3158332624007016e-05 - rollout_corr/rollout_is_max:2.0102672576904297 - rollout_corr/rollout_is_min:0.3711838126182556 - rollout_corr/rollout_is_std:0.02958839386701584 - rollout_corr/rollout_is_eff_sample_size:0.9991255307618554 - rollout_corr/rollout_is_seq_mean:0.9999958872795105 - rollout_corr/rollout_is_seq_std:0.002170138293877244 - rollout_corr/rollout_is_seq_max:1.0074241161346436 - rollout_corr/rollout_is_seq_min:0.9875465035438538 - rollout_corr/rollout_is_seq_max_deviation:0.01245349645614624 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3184588616713881) - rollout_corr/training_log_ppl:np.float64(0.25437020364188356) - rollout_corr/kl:np.float64(0.0006354141595537044) - rollout_corr/k3_kl:np.float64(0.0005414221820103648) - rollout_corr/rollout_ppl:np.float64(1.3176015447825193) - rollout_corr/rollout_log_ppl:np.float64(0.2537347881880123) - rollout_corr/log_ppl_diff:np.float64(0.0006354154538712464) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016181653118110262) - rollout_corr/log_ppl_diff_max:np.float64(0.014689624309539795) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058724284172058105) - rollout_corr/ppl_ratio:np.float64(1.0006382004357874) - rollout_corr/chi2_token:np.float64(0.0008942568674683571) - rollout_corr/chi2_seq:np.float64(0.2074549775570631) - actor/pg_loss:np.float64(0.0034695685608312488) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003058491590763879) - actor/ppo_kl:np.float64(0.0001464155437389536) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(5357.345607595518) - self_distillation/token_reweight_w_std:np.float64(96275.07195161912) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001250740606338) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0668551324197324) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.22902002558112144) - perf/mfu/actor:np.float64(0.04600961155742432) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.25037384033203) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00025763644953258336 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00025763644953258336 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:337.3515625 - response_length/max:1248.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:337.3515625 - response_length_non_aborted/max:1248.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010005012154579163 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9751391801983118) - timing_s/agent_loop/generate_sequences/max:np.float64(9.368280753493309) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.263870596201741) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.368280753493309) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1248 - timing_s/gen:16.309176307171583 - timing_s/reward:0.059800758957862854 - timing_s/old_log_prob:3.056093478575349 - timing_s/adv:0.5573611091822386 - timing_s/update_actor:21.186741953715682 - timing_s/step:41.25715699791908 - timing_s/stop_profile:2.9325485229492188e-05 - timing_per_token_ms/gen:0.1888466722305132 - timing_per_token_ms/update_actor:0.13919415251110756 - timing_per_token_ms/adv:0.0036617903500574113 - perf/total_num_tokens:152210 - perf/time_per_step:41.25715699791908 - perf/throughput:461.16241118988495 (TaskRunner pid=2320396) Training Progress: 93%|█████████▎| 93/100 [1:29:04<06:25, 55.12s/it] (TaskRunner pid=2320396) step:94 - global_seqlen/min:15297 - global_seqlen/max:24845 - global_seqlen/minmax_diff:9548 - global_seqlen/balanced_min:20488 - global_seqlen/balanced_max:20493 - global_seqlen/mean:20491.75 - actor/entropy:0.20188386738300323 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17276859283447266 - training/rollout_probs_diff_mean:0.002807691926136613 - training/rollout_probs_diff_std:0.009180913679301739 - training/rollout_actor_probs_pearson_corr:0.9991471767425537 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000237226486206 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6338140964508057 - rollout_corr/rollout_is_min:0.5351300239562988 - rollout_corr/rollout_is_std:0.027619199827313423 - rollout_corr/rollout_is_eff_sample_size:0.9992377612073462 - rollout_corr/rollout_is_seq_mean:1.0000065565109253 - rollout_corr/rollout_is_seq_std:0.0017605639295652509 - rollout_corr/rollout_is_seq_max:1.00704026222229 - rollout_corr/rollout_is_seq_min:0.9916078448295593 - rollout_corr/rollout_is_seq_max_deviation:0.008392155170440674 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2156826448626816) - rollout_corr/training_log_ppl:np.float64(0.18709617016429547) - rollout_corr/kl:np.float64(0.0004157023055952891) - rollout_corr/k3_kl:np.float64(0.00043933037052568125) - rollout_corr/rollout_ppl:np.float64(1.2151662367396057) - rollout_corr/rollout_log_ppl:np.float64(0.18668046647508163) - rollout_corr/log_ppl_diff:np.float64(0.00041570368921384215) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013877422898076475) - rollout_corr/log_ppl_diff_max:np.float64(0.00699692964553833) - rollout_corr/log_ppl_diff_min:np.float64(-0.006185248494148254) - rollout_corr/ppl_ratio:np.float64(1.0004174259956926) - rollout_corr/chi2_token:np.float64(0.0009500652085989714) - rollout_corr/chi2_seq:np.float64(0.4689605936873704) - actor/pg_loss:np.float64(0.001823109108954668) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010713179335652967) - actor/ppo_kl:np.float64(9.207377788200688e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(708.5707060070708) - self_distillation/token_reweight_w_std:np.float64(19318.053093866678) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000523213064298) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04706526521476917) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.1663790736347437) - perf/mfu/actor:np.float64(0.049119620321140894) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.20769119262695) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024228731635957956 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024228731635957956 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:376.0546875 - response_length/max:1038.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:376.0546875 - response_length_non_aborted/max:1038.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.8605183362960815e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9258252773433924) - timing_s/agent_loop/generate_sequences/max:np.float64(8.57844446413219) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.102110257896129) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.57844446413219) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:1038 - timing_s/gen:15.732666831463575 - timing_s/reward:0.06416278146207333 - timing_s/old_log_prob:3.082074398174882 - timing_s/adv:0.5425643008202314 - timing_s/update_actor:21.5194071829319 - timing_s/step:41.027859184890985 - timing_s/stop_profile:0.00012390315532684326 - timing_per_token_ms/gen:0.1634223208835938 - timing_per_token_ms/update_actor:0.13126872511456988 - timing_per_token_ms/adv:0.003309650840095596 - perf/total_num_tokens:163934 - perf/time_per_step:41.027859184890985 - perf/throughput:499.4594016630129 (TaskRunner pid=2320396) Training Progress: 94%|█████████▍| 94/100 [1:29:45<05:05, 50.91s/it] (TaskRunner pid=2320396) step:95 - global_seqlen/min:18552 - global_seqlen/max:26721 - global_seqlen/minmax_diff:8169 - global_seqlen/balanced_min:21840 - global_seqlen/balanced_max:21848 - global_seqlen/mean:21846.0 - actor/entropy:0.24933725595474243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2370302975177765 - training/rollout_probs_diff_mean:0.0031582678202539682 - training/rollout_probs_diff_std:0.009481647983193398 - training/rollout_actor_probs_pearson_corr:0.9992415308952332 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000104904174805 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.613905604055617e-06 - rollout_corr/rollout_is_max:1.6393728256225586 - rollout_corr/rollout_is_min:0.27337026596069336 - rollout_corr/rollout_is_std:0.029562193900346756 - rollout_corr/rollout_is_eff_sample_size:0.9991268397733714 - rollout_corr/rollout_is_seq_mean:0.9999439716339111 - rollout_corr/rollout_is_seq_std:0.0017291044350713491 - rollout_corr/rollout_is_seq_max:1.0078022480010986 - rollout_corr/rollout_is_seq_min:0.9936612844467163 - rollout_corr/rollout_is_seq_max_deviation:0.007802248001098633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2998783034272492) - rollout_corr/training_log_ppl:np.float64(0.24693955569819082) - rollout_corr/kl:np.float64(0.0004962469523377422) - rollout_corr/k3_kl:np.float64(0.0005026913720271864) - rollout_corr/rollout_ppl:np.float64(1.2991761877201498) - rollout_corr/rollout_log_ppl:np.float64(0.24644330685259774) - rollout_corr/log_ppl_diff:np.float64(0.0004962488455930725) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013393320696195588) - rollout_corr/log_ppl_diff_max:np.float64(0.0051915645599365234) - rollout_corr/log_ppl_diff_min:np.float64(-0.0062499940395355225) - rollout_corr/ppl_ratio:np.float64(1.0004977483768016) - rollout_corr/chi2_token:np.float64(0.0009992511477321386) - rollout_corr/chi2_seq:np.float64(0.4181784482207149) - actor/pg_loss:np.float64(0.002324818167835474) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000250535831128218) - actor/ppo_kl:np.float64(-3.197742662219838e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(973.5383979976177) - self_distillation/token_reweight_w_std:np.float64(22118.911819823494) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994016685523093) - self_distillation/token_reweight_w_clip_frac:np.float64(0.052211643022019416) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.19546930491924286) - perf/mfu/actor:np.float64(0.05242335858753856) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.08116912841797) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008467447012662888 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008467447012662888 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.3125 - response_length/max:1203.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.3125 - response_length_non_aborted/max:1203.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015277229249477386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1555323153734207) - timing_s/agent_loop/generate_sequences/max:np.float64(9.825576543807983) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.590243221617129) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.825576543807983) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1203 - timing_s/gen:16.539687406271696 - timing_s/reward:0.06275773420929909 - timing_s/old_log_prob:2.922587675973773 - timing_s/adv:0.5635931380093098 - timing_s/update_actor:21.523274278268218 - timing_s/step:41.69783091172576 - timing_s/stop_profile:2.7721747756004333e-05 - timing_per_token_ms/gen:0.15901099259990478 - timing_per_token_ms/update_actor:0.12315340496125274 - timing_per_token_ms/adv:0.0032248073904222155 - perf/total_num_tokens:174768 - perf/time_per_step:41.69783091172576 - perf/throughput:523.9121441651952 (TaskRunner pid=2320396) Training Progress: 95%|█████████▌| 95/100 [1:30:26<04:00, 48.15s/it] (TaskRunner pid=2320396) step:96 - global_seqlen/min:15331 - global_seqlen/max:35009 - global_seqlen/minmax_diff:19678 - global_seqlen/balanced_min:23384 - global_seqlen/balanced_max:23414 - global_seqlen/mean:23404.625 - actor/entropy:0.20848210155963898 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28976500034332275 - training/rollout_probs_diff_mean:0.0027812193147838116 - training/rollout_probs_diff_std:0.009169485419988632 - training/rollout_actor_probs_pearson_corr:0.9991740584373474 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0001304149627686 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.644163244753145e-06 - rollout_corr/rollout_is_max:1.65871000289917 - rollout_corr/rollout_is_min:0.3381444215774536 - rollout_corr/rollout_is_std:0.027441667392849922 - rollout_corr/rollout_is_eff_sample_size:0.9992477596280513 - rollout_corr/rollout_is_seq_mean:1.0001225471496582 - rollout_corr/rollout_is_seq_std:0.0015613530995324254 - rollout_corr/rollout_is_seq_max:1.006011962890625 - rollout_corr/rollout_is_seq_min:0.9935289025306702 - rollout_corr/rollout_is_seq_max_deviation:0.006471097469329834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2423579399473965) - rollout_corr/training_log_ppl:np.float64(0.20239087475056294) - rollout_corr/kl:np.float64(0.00034805676968829857) - rollout_corr/k3_kl:np.float64(0.0004387744726113141) - rollout_corr/rollout_ppl:np.float64(1.2419147840701044) - rollout_corr/rollout_log_ppl:np.float64(0.20204281566111604) - rollout_corr/log_ppl_diff:np.float64(0.000348059089446906) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012117344085709192) - rollout_corr/log_ppl_diff_max:np.float64(0.007875531911849976) - rollout_corr/log_ppl_diff_min:np.float64(-0.0057569146156311035) - rollout_corr/ppl_ratio:np.float64(1.0003494743723422) - rollout_corr/chi2_token:np.float64(0.0010582820978015661) - rollout_corr/chi2_seq:np.float64(0.7725066640414298) - actor/pg_loss:np.float64(0.0019937818869948387) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022862237619847292) - actor/ppo_kl:np.float64(7.256892491014355e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(18855.210086595966) - self_distillation/token_reweight_w_std:np.float64(398993.84403990314) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002333554439247) - self_distillation/token_reweight_w_clip_frac:np.float64(0.047663526580436155) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.16047444380819798) - perf/mfu/actor:np.float64(0.05551098602326122) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.10414505004883) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005147599149495363 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005147599149495363 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:451.89453125 - response_length/max:1508.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:451.89453125 - response_length_non_aborted/max:1508.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5307759642601013e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3833144959062338) - timing_s/agent_loop/generate_sequences/max:np.float64(12.088890012353659) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.130557603857596) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.088890012353659) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:1508 - timing_s/gen:18.805637188255787 - timing_s/reward:0.06366230174899101 - timing_s/old_log_prob:2.9904247242957354 - timing_s/adv:0.5898833721876144 - timing_s/update_actor:21.912752106785774 - timing_s/step:44.431054543703794 - timing_s/stop_profile:0.00012172572314739227 - timing_per_token_ms/gen:0.16255899371790455 - timing_per_token_ms/update_actor:0.11703216835767383 - timing_per_token_ms/adv:0.0031504637020867373 - perf/total_num_tokens:187237 - perf/time_per_step:44.431054543703794 - perf/throughput:526.7627617746157 (TaskRunner pid=2320396) Training Progress: 96%|█████████▌| 96/100 [1:31:11<03:08, 47.05s/it] (TaskRunner pid=2320396) step:97 - global_seqlen/min:18399 - global_seqlen/max:32963 - global_seqlen/minmax_diff:14564 - global_seqlen/balanced_min:23676 - global_seqlen/balanced_max:23712 - global_seqlen/mean:23702.25 - actor/entropy:0.2419547587633133 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163779854774475 - training/rollout_probs_diff_mean:0.003042820608243346 - training/rollout_probs_diff_std:0.00934920459985733 - training/rollout_actor_probs_pearson_corr:0.9992281794548035 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.84765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.00001859664917 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4824161300784908e-05 - rollout_corr/rollout_is_max:1.64872407913208 - rollout_corr/rollout_is_min:0.289579302072525 - rollout_corr/rollout_is_std:0.028276080265641212 - rollout_corr/rollout_is_eff_sample_size:0.9992009830243436 - rollout_corr/rollout_is_seq_mean:0.9999649524688721 - rollout_corr/rollout_is_seq_std:0.0021841987036168575 - rollout_corr/rollout_is_seq_max:1.009124755859375 - rollout_corr/rollout_is_seq_min:0.9882916808128357 - rollout_corr/rollout_is_seq_max_deviation:0.011708319187164307 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.255678509362042) - rollout_corr/training_log_ppl:np.float64(0.2168065948790172) - rollout_corr/kl:np.float64(0.0004864873664445213) - rollout_corr/k3_kl:np.float64(0.00045971487596574434) - rollout_corr/rollout_ppl:np.float64(1.2550409259274602) - rollout_corr/rollout_log_ppl:np.float64(0.21632010873872787) - rollout_corr/log_ppl_diff:np.float64(0.0004864861402893439) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013925857347203419) - rollout_corr/log_ppl_diff_max:np.float64(0.00998617708683014) - rollout_corr/log_ppl_diff_min:np.float64(-0.006745591759681702) - rollout_corr/ppl_ratio:np.float64(1.0004884300287813) - rollout_corr/chi2_token:np.float64(0.0008650836534798145) - rollout_corr/chi2_seq:np.float64(0.7680268259719014) - actor/pg_loss:np.float64(0.0021499351132661104) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.822644940664759e-05) - actor/ppo_kl:np.float64(-1.3785715332836901e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_w_mean:np.float64(5079.578742446611) - self_distillation/token_reweight_w_std:np.float64(103047.53704692877) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000506963115185) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04400453166454099) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.15901292487978935) - perf/mfu/actor:np.float64(0.05632942768262873) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.07713317871094) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006603226996958256 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006603226996958256 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:472.0703125 - response_length/max:1746.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:472.0703125 - response_length_non_aborted/max:1746.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000186799094080925 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9205618463456631) - timing_s/agent_loop/generate_sequences/max:np.float64(12.9535857681185) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.061531551196822) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.9535857681185) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1746 - timing_s/gen:20.029064597561955 - timing_s/reward:0.06482006795704365 - timing_s/old_log_prob:3.03081613779068 - timing_s/adv:0.6125223767012358 - timing_s/update_actor:21.9055054243654 - timing_s/step:45.73275777883828 - timing_s/stop_profile:0.00010943599045276642 - timing_per_token_ms/gen:0.16573491599141046 - timing_per_token_ms/update_actor:0.11552439865606325 - timing_per_token_ms/adv:0.003230296578917802 - perf/total_num_tokens:189618 - perf/time_per_step:45.73275777883828 - perf/throughput:518.2772951201215 (TaskRunner pid=2320396) Training Progress: 97%|█████████▋| 97/100 [1:31:57<02:20, 46.67s/it] (TaskRunner pid=2320396) step:98 - global_seqlen/min:18831 - global_seqlen/max:27260 - global_seqlen/minmax_diff:8429 - global_seqlen/balanced_min:22401 - global_seqlen/balanced_max:22407 - global_seqlen/mean:22404.5 - actor/entropy:0.2321578562259674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5020689964294434 - training/rollout_probs_diff_mean:0.0029261824674904346 - training/rollout_probs_diff_std:0.009089733473956585 - training/rollout_actor_probs_pearson_corr:0.9992458820343018 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999915361404419 - rollout_corr/rollout_is_ratio_fraction_high:2.722421777434647e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.074739864445291e-06 - rollout_corr/rollout_is_max:2.632174015045166 - rollout_corr/rollout_is_min:0.4950760304927826 - rollout_corr/rollout_is_std:0.02858218364417553 - rollout_corr/rollout_is_eff_sample_size:0.999183725586957 - rollout_corr/rollout_is_seq_mean:0.999980092048645 - rollout_corr/rollout_is_seq_std:0.001532103167846799 - rollout_corr/rollout_is_seq_max:1.0066100358963013 - rollout_corr/rollout_is_seq_min:0.9930747151374817 - rollout_corr/rollout_is_seq_max_deviation:0.0069252848625183105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2669148072600365) - rollout_corr/training_log_ppl:np.float64(0.22072085265244823) - rollout_corr/kl:np.float64(0.00041741177986587275) - rollout_corr/k3_kl:np.float64(0.0004389282450176779) - rollout_corr/rollout_ppl:np.float64(1.2663823189213872) - rollout_corr/rollout_log_ppl:np.float64(0.22030344046652317) - rollout_corr/log_ppl_diff:np.float64(0.000417412185925059) - rollout_corr/log_ppl_abs_diff:np.float64(0.001250314715434797) - rollout_corr/log_ppl_diff_max:np.float64(0.008546322584152222) - rollout_corr/log_ppl_diff_min:np.float64(-0.0056915283203125) - rollout_corr/ppl_ratio:np.float64(1.0004189505707473) - rollout_corr/chi2_token:np.float64(0.0009120509494096041) - rollout_corr/chi2_seq:np.float64(0.8078791201114655) - actor/pg_loss:np.float64(0.0020201385486871004) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023760700810271373) - actor/ppo_kl:np.float64(-2.3586566145539223e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(1225.9335982890334) - self_distillation/token_reweight_w_std:np.float64(22503.530439826485) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999883871059865) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04507940106850583) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.4081057645380497) - perf/mfu/actor:np.float64(0.05368987949063367) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.05130767822266) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022709534969180822 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022709534969180822 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:430.453125 - response_length/max:1176.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:430.453125 - response_length_non_aborted/max:1176.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015125982463359833 - timing_s/agent_loop/generate_sequences/min:np.float64(1.054880429059267) - timing_s/agent_loop/generate_sequences/max:np.float64(9.709577007219195) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.776259749698511) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.709577007219195) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1176 - timing_s/gen:16.545982444658875 - timing_s/reward:0.06403978727757931 - timing_s/old_log_prob:2.9795034248381853 - timing_s/adv:0.5723136328160763 - timing_s/update_actor:21.520603019744158 - timing_s/step:41.77067055925727 - timing_s/stop_profile:0.00011765211820602417 - timing_per_token_ms/gen:0.15015048136646408 - timing_per_token_ms/update_actor:0.12006852986980382 - timing_per_token_ms/adv:0.0031930730032810163 - perf/total_num_tokens:179236 - perf/time_per_step:41.77067055925727 - perf/throughput:536.3691724368233 (TaskRunner pid=2320396) Training Progress: 98%|█████████▊| 98/100 [1:32:38<01:30, 45.22s/it] (TaskRunner pid=2320396) step:99 - global_seqlen/min:17790 - global_seqlen/max:27332 - global_seqlen/minmax_diff:9542 - global_seqlen/balanced_min:21975 - global_seqlen/balanced_max:21983 - global_seqlen/mean:21978.625 - actor/entropy:0.21217578649520874 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30725976824760437 - training/rollout_probs_diff_mean:0.0027088799979537725 - training/rollout_probs_diff_std:0.009153125807642937 - training/rollout_actor_probs_pearson_corr:0.9991687536239624 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.8828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:1.0000417232513428 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8437087419442832e-05 - rollout_corr/rollout_is_max:1.9659626483917236 - rollout_corr/rollout_is_min:0.05042092502117157 - rollout_corr/rollout_is_std:0.027372073382139206 - rollout_corr/rollout_is_eff_sample_size:0.9992512115102347 - rollout_corr/rollout_is_seq_mean:1.0000988245010376 - rollout_corr/rollout_is_seq_std:0.0017440631054341793 - rollout_corr/rollout_is_seq_max:1.0088425874710083 - rollout_corr/rollout_is_seq_min:0.9949665069580078 - rollout_corr/rollout_is_seq_max_deviation:0.0088425874710083 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2107150228694081) - rollout_corr/training_log_ppl:np.float64(0.1793389076483436) - rollout_corr/kl:np.float64(0.0004552967537865271) - rollout_corr/k3_kl:np.float64(0.00041060264781833666) - rollout_corr/rollout_ppl:np.float64(1.2101408634334803) - rollout_corr/rollout_log_ppl:np.float64(0.1788836106788949) - rollout_corr/log_ppl_diff:np.float64(0.0004552969694486819) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012980137325939722) - rollout_corr/log_ppl_diff_max:np.float64(0.007373355329036713) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074315667152404785) - rollout_corr/ppl_ratio:np.float64(1.0004568246658891) - rollout_corr/chi2_token:np.float64(0.0007171817123889923) - rollout_corr/chi2_seq:np.float64(0.3282004913780838) - actor/pg_loss:np.float64(0.001477871905080974) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.6140283504646504e-05) - actor/ppo_kl:np.float64(0.00016689537152991818) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8828125) - self_distillation/token_reweight_w_mean:np.float64(748.2231652874034) - self_distillation/token_reweight_w_std:np.float64(19586.47575551673) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993815938942134) - self_distillation/token_reweight_w_clip_frac:np.float64(0.033909484758623876) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.1121530244126916) - perf/mfu/actor:np.float64(0.05194370252387481) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.09511184692383) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.8828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007213510572910309 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007213510572910309 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:423.73828125 - response_length/max:1213.0 - response_length/min:52.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.73828125 - response_length_non_aborted/max:1213.0 - response_length_non_aborted/min:52.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012556836009025574 - timing_s/agent_loop/generate_sequences/min:np.float64(0.719259187579155) - timing_s/agent_loop/generate_sequences/max:np.float64(9.758743185549974) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.64678174258006) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.758743185549974) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:310 - timing_s/agent_loop/slowest/response_length:1213 - timing_s/gen:16.68287494778633 - timing_s/reward:0.06371786259114742 - timing_s/old_log_prob:2.987398575991392 - timing_s/adv:0.5760593675076962 - timing_s/update_actor:21.90517684444785 - timing_s/step:42.30455153994262 - timing_s/stop_profile:3.0780211091041565e-05 - timing_per_token_ms/gen:0.15379181713899104 - timing_per_token_ms/update_actor:0.12458227507662474 - timing_per_token_ms/adv:0.003276247760652089 - perf/total_num_tokens:175829 - perf/time_per_step:42.30455153994262 - perf/throughput:519.5333409751071 (TaskRunner pid=2320396) Training Progress: 99%|█████████▉| 99/100 [1:33:21<00:44, 44.35s/it] (TaskRunner pid=2320396) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2320396) validation generation end (TaskRunner pid=2320396) [prompt] system (TaskRunner pid=2320396) (TaskRunner pid=2320396) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) ... (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2320396) (TaskRunner pid=2320396) A (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) user (TaskRunner pid=2320396) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2320396) (TaskRunner pid=2320396) A: 50 pC (TaskRunner pid=2320396) B: 56 pC (TaskRunner pid=2320396) C: 60 pC (TaskRunner pid=2320396) D: 64 pC (TaskRunner pid=2320396) Please reason step by step. (TaskRunner pid=2320396) assistant (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) [response] (TaskRunner pid=2320396) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. Using the formula for the electric field due to a point charge: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) E = k \frac{q}{r^2} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Solving for $ q $: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Substituting the values: (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) Converting to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2320396) (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2320396) $$ (TaskRunner pid=2320396) (TaskRunner pid=2320396) The closest option is: (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) (TaskRunner pid=2320396) B (TaskRunner pid=2320396) (TaskRunner pid=2320396) [ground_truth] B (TaskRunner pid=2320396) [score] 1.0 (TaskRunner pid=2320396) [acc] 1.0 (TaskRunner pid=2320396) [pred] B (TaskRunner pid=2320396) [incorrect_format] 1 (TaskRunner pid=2320396) [feedback] (TaskRunner pid=2320396) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2320396) step:100 - global_seqlen/min:15293 - global_seqlen/max:27160 - global_seqlen/minmax_diff:11867 - global_seqlen/balanced_min:22042 - global_seqlen/balanced_max:22077 - global_seqlen/mean:22062.125 - actor/entropy:0.2097870260477066 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4133709967136383 - training/rollout_probs_diff_mean:0.0028145588003098965 - training/rollout_probs_diff_std:0.009487012401223183 - training/rollout_actor_probs_pearson_corr:0.9991060495376587 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.953125 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.953125 - rollout_corr/rollout_is_mean:0.9999947547912598 - rollout_corr/rollout_is_ratio_fraction_high:1.8594100765767507e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.789115023915656e-05 - rollout_corr/rollout_is_max:4.497957706451416 - rollout_corr/rollout_is_min:0.3550756275653839 - rollout_corr/rollout_is_std:0.028181063011288643 - rollout_corr/rollout_is_eff_sample_size:0.9992064579221395 - rollout_corr/rollout_is_seq_mean:0.9999136328697205 - rollout_corr/rollout_is_seq_std:0.0020781357306987047 - rollout_corr/rollout_is_seq_max:1.0099788904190063 - rollout_corr/rollout_is_seq_min:0.9891766309738159 - rollout_corr/rollout_is_seq_max_deviation:0.010823369026184082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2158178966492414) - rollout_corr/training_log_ppl:np.float64(0.185322040655592) - rollout_corr/kl:np.float64(0.0004785535343856395) - rollout_corr/k3_kl:np.float64(0.0005256261923278771) - rollout_corr/rollout_ppl:np.float64(1.2151897735893726) - rollout_corr/rollout_log_ppl:np.float64(0.18484348608762957) - rollout_corr/log_ppl_diff:np.float64(0.00047855456796241924) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015097330397111364) - rollout_corr/log_ppl_diff_max:np.float64(0.011803552508354187) - rollout_corr/log_ppl_diff_min:np.float64(-0.010602578520774841) - rollout_corr/ppl_ratio:np.float64(1.0004810797981918) - rollout_corr/chi2_token:np.float64(0.0010942460503429174) - rollout_corr/chi2_seq:np.float64(0.5077934148721397) - actor/pg_loss:np.float64(0.0021155468421056867) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039214472985804605) - actor/ppo_kl:np.float64(-6.8683105657108e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(395.0223485589959) - self_distillation/token_reweight_w_std:np.float64(8410.495676852704) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012438350822777) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05156318433000706) - self_distillation/empty_target_batch:np.float64(0.046875) - actor/grad_norm:np.float64(0.16905477456748486) - perf/mfu/actor:np.float64(0.05262459257753069) - perf/max_memory_allocated_gb:np.float64(131.24971342086792) - perf/max_memory_reserved_gb:np.float64(143.283203125) - perf/cpu_memory_used_gb:np.float64(198.11225509643555) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6578125) - val-aux/sciknoweval/reward/std@16:np.float64(0.1791601964104201) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.72985) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14243945466552344) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5846375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16119285610681686) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.657775) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17855468574283387) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7821) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09939721158640422) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5175124999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12756100353900784) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6794749999999998) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13936384594983817) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8189875000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06830858426455963) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4656625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08893811125770881) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6921250000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09885195998023369) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8481500000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04638693035505402) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4310375000000001) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04764229560029223) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.70175) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06840959900793887) - val-aux/sciknoweval/score/mean@16:np.float64(0.6578125) - val-aux/sciknoweval/score/std@16:np.float64(0.1791601964104201) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.72985) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14243945466552344) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5846375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16119285610681686) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.657775) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17855468574283387) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7821) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09939721158640422) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5175124999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12756100353900784) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6794749999999998) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13936384594983817) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8189875000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06830858426455963) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4656625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08893811125770881) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6921250000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09885195998023369) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8481500000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04638693035505402) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4310375000000001) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04764229560029223) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.70175) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06840959900793887) - val-core/sciknoweval/acc/mean@16:np.float64(0.6578125) - val-aux/sciknoweval/acc/std@16:np.float64(0.1791601964104201) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.72985) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14243945466552344) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5846375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16119285610681686) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.657775) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17855468574283387) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7821) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09939721158640422) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5175124999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12756100353900784) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6794749999999998) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13936384594983817) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8189875000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06830858426455963) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4656625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08893811125770881) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6921250000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09885195998023369) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8481500000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04638693035505402) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4310375000000001) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04764229560029223) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.70175) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06840959900793887) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.036863588899184146) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9937625000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.018150023927969855) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9683875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04360969010040857) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9808749999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03671858855568414) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9987124999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.005042814272451763) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9510124999999998) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.047638230389618993) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.990175) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.024837100384058737) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0008816709987291178) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9298375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04533008795714807) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9957125) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.013499621923874276) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.90865) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03526981610931457) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9989250000000002) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004970325714929565) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0074690175242722034 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0074690175242722034 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.16015625 - response_length/max:1576.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.16015625 - response_length_non_aborted/max:1576.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5948509573936462e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9642361830919981) - timing_s/agent_loop/generate_sequences/max:np.float64(11.51597017236054) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3494094031411805) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.51597017236054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:1576 - timing_s/gen:18.236607283353806 - timing_s/reward:0.06037502735853195 - timing_s/old_log_prob:2.968335896730423 - timing_s/adv:0.5751208234578371 - timing_s/update_actor:21.881923211738467 - timing_s/step:43.809186674654484 - timing_s/testing:113.72614772059023 - timing_s/save_checkpoint:12.852769210934639 - timing_s/stop_profile:0.0001350194215774536 - timing_per_token_ms/gen:0.16954665058296042 - timing_per_token_ms/update_actor:0.12397900934145321 - timing_per_token_ms/adv:0.003258530306225245 - perf/total_num_tokens:176497 - perf/time_per_step:43.809186674654484 - perf/throughput:503.5958591023991 (TaskRunner pid=2320396) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2320396) "np.float64(0.6578125), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2320396) "np.float64(0.1791601964104201), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2320396) "np.float64(0.72985), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2320396) "np.float64(0.14243945466552344), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2320396) "np.float64(0.5846375), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2320396) "np.float64(0.16119285610681686), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2320396) "np.float64(0.657775), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2320396) "np.float64(0.17855468574283387), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2320396) "np.float64(0.7821), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2320396) "np.float64(0.09939721158640422), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2320396) "np.float64(0.5175124999999999), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2320396) "np.float64(0.12756100353900784), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2320396) "np.float64(0.6794749999999998), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2320396) "np.float64(0.13936384594983817), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2320396) "np.float64(0.8189875000000001), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2320396) "np.float64(0.06830858426455963), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2320396) "np.float64(0.4656625), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2320396) "np.float64(0.08893811125770881), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2320396) "np.float64(0.6921250000000001), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2320396) "np.float64(0.09885195998023369), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2320396) "np.float64(0.8481500000000001), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2320396) "np.float64(0.04638693035505402), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2320396) "np.float64(0.4310375000000001), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2320396) "np.float64(0.04764229560029223), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2320396) "np.float64(0.70175), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2320396) "np.float64(0.06840959900793887), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2320396) "np.float64(0.6578125), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2320396) "np.float64(0.1791601964104201), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2320396) "np.float64(0.72985), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2320396) "np.float64(0.14243945466552344), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2320396) "np.float64(0.5846375), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2320396) "np.float64(0.16119285610681686), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2320396) "np.float64(0.657775), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2320396) "np.float64(0.17855468574283387), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2320396) "np.float64(0.7821), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2320396) "np.float64(0.09939721158640422), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2320396) "np.float64(0.5175124999999999), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2320396) "np.float64(0.12756100353900784), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2320396) "np.float64(0.6794749999999998), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2320396) "np.float64(0.13936384594983817), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2320396) "np.float64(0.8189875000000001), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2320396) "np.float64(0.06830858426455963), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2320396) "np.float64(0.4656625), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2320396) "np.float64(0.08893811125770881), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2320396) "np.float64(0.6921250000000001), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2320396) "np.float64(0.09885195998023369), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2320396) "np.float64(0.8481500000000001), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2320396) "np.float64(0.04638693035505402), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2320396) "np.float64(0.4310375000000001), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2320396) "np.float64(0.04764229560029223), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2320396) "np.float64(0.70175), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2320396) "np.float64(0.06840959900793887), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2320396) "np.float64(0.6578125), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2320396) "np.float64(0.1791601964104201), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2320396) "np.float64(0.72985), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2320396) "np.float64(0.14243945466552344), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2320396) "np.float64(0.5846375), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2320396) "np.float64(0.16119285610681686), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2320396) "np.float64(0.657775), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2320396) "np.float64(0.17855468574283387), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2320396) "np.float64(0.7821), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2320396) "np.float64(0.09939721158640422), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2320396) "np.float64(0.5175124999999999), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2320396) "np.float64(0.12756100353900784), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2320396) "np.float64(0.6794749999999998), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2320396) "np.float64(0.13936384594983817), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2320396) "np.float64(0.8189875000000001), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2320396) "np.float64(0.06830858426455963), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2320396) "np.float64(0.4656625), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2320396) "np.float64(0.08893811125770881), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2320396) "np.float64(0.6921250000000001), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2320396) "np.float64(0.09885195998023369), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2320396) "np.float64(0.8481500000000001), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2320396) "np.float64(0.04638693035505402), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2320396) "np.float64(0.4310375000000001), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2320396) "np.float64(0.04764229560029223), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2320396) "np.float64(0.70175), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2320396) 'np.float64(0.06840959900793887), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.98125), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2320396) 'np.float64(0.036863588899184146), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2320396) 'np.float64(0.9937625000000001), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2320396) 'np.float64(0.018150023927969855), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2320396) 'np.float64(0.9683875000000001), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2320396) 'np.float64(0.04360969010040857), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': " (TaskRunner pid=2320396) 'np.float64(0.9808749999999999), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2320396) 'np.float64(0.03671858855568414), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@4/mean': " (TaskRunner pid=2320396) 'np.float64(0.9987124999999999), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2320396) 'np.float64(0.005042814272451763), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2320396) 'np.float64(0.9510124999999998), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2320396) 'np.float64(0.047638230389618993), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(0.990175), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2320396) 'np.float64(0.024837100384058737), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(0.9999375), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@8/std': " (TaskRunner pid=2320396) 'np.float64(0.0008816709987291178), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(0.9298375), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2320396) 'np.float64(0.04533008795714807), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(0.9957125), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2320396) 'np.float64(0.013499621923874276), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(0.90865), " (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2320396) 'np.float64(0.03526981610931457), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2320396) 'np.float64(0.9989250000000002), ' (TaskRunner pid=2320396) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2320396) "np.float64(0.004970325714929565), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2320396) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2320396) 'np.float64(2.0)}') (TaskRunner pid=2320396) Training Progress: 100%|██████████| 100/100 [1:36:11<00:00, 82.17s/it] Training Progress: 100%|██████████| 100/100 [1:36:11<00:00, 57.72s/it] (TaskRunner pid=2320396) Exception ignored in: (TaskRunner pid=2320396) Traceback (most recent call last): (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/threading.py", line 1594, in _shutdown (TaskRunner pid=2320396) atexit_call() (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/process.py", line 101, in _python_exit (TaskRunner pid=2320396) items = list(_threads_wakeups.items()) (TaskRunner pid=2320396) Exception ignored in: (TaskRunner pid=2320396) ^^Traceback (most recent call last): (TaskRunner pid=2320396) ^^ File "/mnt/mole/SDPO/L2T/verl/utils/tracking.py", line 169, in __del__ (TaskRunner pid=2320396) ^^^^^ ^self.logger["wandb"].finish(exit_code=0)^ (TaskRunner pid=2320396) ^^ File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 4097, in finish (TaskRunner pid=2320396) ^^^^ ^wandb.run.finish(exit_code=exit_code, quiet=quiet)^ (TaskRunner pid=2320396) ^ File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2320396) ^^ ^return func(self, *args, **kwargs)^ (TaskRunner pid=2320396) ^ ^ ^ ^ ^ ^ (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/weakref.py", line 462, in items (TaskRunner pid=2320396) ^^^^^^^^^^^ ^with _IterationGuard(self):^ (TaskRunner pid=2320396) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^^^^^^^^^^ (TaskRunner pid=2320396) ^^ File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 445, in wrapper (TaskRunner pid=2320396) ^^ ^return func(self, *args, **kwargs)^ (TaskRunner pid=2320396) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/_weakrefset.py", line 31, in __exit__ (TaskRunner pid=2320396) ^^^^^^^^^^^^^ ^s.remove(self)^ (TaskRunner pid=2320396) ^ File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/signal_handling.py", line 73, in handler (TaskRunner pid=2320396) ^^^^^^ ^_error_if_any_worker_fails()^ (TaskRunner pid=2320396) ^RuntimeError^: ^DataLoader worker (pid 2342222) is killed by signal: Killed. (TaskRunner pid=2320396) (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2275, in finish (TaskRunner pid=2320396) return self._finish(exit_code) (TaskRunner pid=2320396) ^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2320396) return func(self, *args, **kwargs) (TaskRunner pid=2320396) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2288, in _finish (TaskRunner pid=2320396) with telemetry.context(run=self) as tel: (TaskRunner pid=2320396) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/telemetry.py", line 42, in __exit__ (TaskRunner pid=2320396) self._run._telemetry_callback(self._obj) (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 752, in _telemetry_callback (TaskRunner pid=2320396) self._telemetry_flush() (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 765, in _telemetry_flush (TaskRunner pid=2320396) self._backend.interface._publish_telemetry(self._telemetry_obj) (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_shared.py", line 103, in _publish_telemetry (TaskRunner pid=2320396) self._publish(rec) (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_sock.py", line 46, in _publish (TaskRunner pid=2320396) self._asyncer.run(lambda: self._client.publish(request)) (TaskRunner pid=2320396) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 131, in run (TaskRunner pid=2320396) raise ValueError("Cannot use run() inside async loop.") (TaskRunner pid=2320396) ValueError: Cannot use run() inside async loop. main_ppo exit code: 0 [2026-07-03 07:27:53] Finished Qwen/Qwen3-8B physics rlsd_tr. [2026-07-03 07:27:53] Starting Qwen/Qwen3-8B biology grpo. Experiment: qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: biology Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet Ray tmp: /tmp/ray_q3g_biology_grpo_Qwen3_8B 2026-07-03 07:27:55,383 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/biology Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_biology_grpo_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/biology +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_biology_grpo_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/biology', 'EXPERIMENT': 'qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 07:28:05,194 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2347384) TaskRunner hostname: mole-workspace-rw, PID: 2347384 (TaskRunner pid=2347384) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2347384) 'calculate_entropy': False, (TaskRunner pid=2347384) 'calculate_sum_pi_squared': False, (TaskRunner pid=2347384) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2347384) 'async_save': False, (TaskRunner pid=2347384) 'load_contents': ['model', (TaskRunner pid=2347384) 'optimizer', (TaskRunner pid=2347384) 'extra'], (TaskRunner pid=2347384) 'save_contents': ['model', (TaskRunner pid=2347384) 'optimizer', (TaskRunner pid=2347384) 'extra']}, (TaskRunner pid=2347384) 'clip_ratio': 0.2, (TaskRunner pid=2347384) 'clip_ratio_c': 3.0, (TaskRunner pid=2347384) 'clip_ratio_high': 0.28, (TaskRunner pid=2347384) 'clip_ratio_low': 0.2, (TaskRunner pid=2347384) 'data_loader_seed': 42, (TaskRunner pid=2347384) 'entropy_checkpointing': False, (TaskRunner pid=2347384) 'entropy_coeff': 0, (TaskRunner pid=2347384) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2347384) 'freeze_vision_tower': False, (TaskRunner pid=2347384) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2347384) 'dtype': 'bfloat16', (TaskRunner pid=2347384) 'entropy_checkpointing': False, (TaskRunner pid=2347384) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2347384) 'forward_only': False, (TaskRunner pid=2347384) 'forward_prefetch': False, (TaskRunner pid=2347384) 'fsdp_size': -1, (TaskRunner pid=2347384) 'full_determinism': False, (TaskRunner pid=2347384) 'model_dtype': 'fp32', (TaskRunner pid=2347384) 'offload_policy': False, (TaskRunner pid=2347384) 'optimizer_offload': False, (TaskRunner pid=2347384) 'param_offload': False, (TaskRunner pid=2347384) 'reshard_after_forward': True, (TaskRunner pid=2347384) 'seed': 42, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_orig_params': False, (TaskRunner pid=2347384) 'use_torch_compile': True, (TaskRunner pid=2347384) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2347384) 'grad_clip': 1.0, (TaskRunner pid=2347384) 'kl_loss_coef': 0.0, (TaskRunner pid=2347384) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2347384) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2347384) 'loss_scale_factor': None, (TaskRunner pid=2347384) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2347384) 'betas': [0.9, 0.999], (TaskRunner pid=2347384) 'clip_grad': 1.0, (TaskRunner pid=2347384) 'lr': 1e-06, (TaskRunner pid=2347384) 'lr_scheduler_type': 'constant', (TaskRunner pid=2347384) 'lr_warmup_steps': 10, (TaskRunner pid=2347384) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2347384) 'min_lr_ratio': 0.0, (TaskRunner pid=2347384) 'num_cycles': 0.5, (TaskRunner pid=2347384) 'optimizer': 'AdamW', (TaskRunner pid=2347384) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2347384) 'override_optimizer_config': None, (TaskRunner pid=2347384) 'total_training_steps': -1, (TaskRunner pid=2347384) 'warmup_style': None, (TaskRunner pid=2347384) 'weight_decay': 0.01}, (TaskRunner pid=2347384) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2347384) 'clip_cov_lb': 1.0, (TaskRunner pid=2347384) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2347384) 'clip_cov_ub': 5.0, (TaskRunner pid=2347384) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2347384) 'loss_mode': 'vanilla', (TaskRunner pid=2347384) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2347384) 'ppo_epochs': 1, (TaskRunner pid=2347384) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2347384) 'ppo_micro_batch_size': None, (TaskRunner pid=2347384) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2347384) 'ppo_mini_batch_size': 8, (TaskRunner pid=2347384) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'all_ranks': False, (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'ranks': [], (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'tool': None, (TaskRunner pid=2347384) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2347384) 'analysis': True, (TaskRunner pid=2347384) 'contents': [], (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'level': 'level0'}, (TaskRunner pid=2347384) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'discrete': False}, (TaskRunner pid=2347384) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2347384) 'step_end': None, (TaskRunner pid=2347384) 'step_start': 0}, (TaskRunner pid=2347384) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2347384) 'rollout_n': 8, (TaskRunner pid=2347384) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2347384) 'mode': 'disabled', (TaskRunner pid=2347384) 'record_file': None, (TaskRunner pid=2347384) 'replay_file': None}, (TaskRunner pid=2347384) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2347384) 'alpha': 0.5, (TaskRunner pid=2347384) 'distillation_add_tail': True, (TaskRunner pid=2347384) 'distillation_topk': 100, (TaskRunner pid=2347384) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2347384) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2347384) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'loss_weight': 0.0, (TaskRunner pid=2347384) 'mask': 'all'}, (TaskRunner pid=2347384) 'feedback_template': '\n' (TaskRunner pid=2347384) 'The ' (TaskRunner pid=2347384) 'following ' (TaskRunner pid=2347384) 'is ' (TaskRunner pid=2347384) 'feedback ' (TaskRunner pid=2347384) 'from ' (TaskRunner pid=2347384) 'your ' (TaskRunner pid=2347384) 'unsuccessful ' (TaskRunner pid=2347384) 'earlier ' (TaskRunner pid=2347384) 'attempt:\n' (TaskRunner pid=2347384) '\n' (TaskRunner pid=2347384) '{feedback_raw}', (TaskRunner pid=2347384) 'full_logit_distillation': True, (TaskRunner pid=2347384) 'include_environment_feedback': True, (TaskRunner pid=2347384) 'is_clip': 2, (TaskRunner pid=2347384) 'max_reprompt_len': 22528, (TaskRunner pid=2347384) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2347384) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2347384) '\n' (TaskRunner pid=2347384) 'Correctly ' (TaskRunner pid=2347384) 'solve ' (TaskRunner pid=2347384) 'the ' (TaskRunner pid=2347384) 'original ' (TaskRunner pid=2347384) 'question.', (TaskRunner pid=2347384) 'reprompt_truncation': 'right', (TaskRunner pid=2347384) 'solution_template': '\n' (TaskRunner pid=2347384) 'Correct ' (TaskRunner pid=2347384) 'solution:\n' (TaskRunner pid=2347384) '\n' (TaskRunner pid=2347384) '{successful_previous_attempt}', (TaskRunner pid=2347384) 'srpo_dynamic_weighting': False, (TaskRunner pid=2347384) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2347384) 'success_reward_threshold': 0.5, (TaskRunner pid=2347384) 'teacher_regularization': 'ema', (TaskRunner pid=2347384) 'teacher_update_rate': 0.0, (TaskRunner pid=2347384) 'token_reweight_decay_steps': None, (TaskRunner pid=2347384) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2347384) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2347384) 'shuffle': False, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2347384) 'tau_neg': 1.05, (TaskRunner pid=2347384) 'tau_pos': 1.0, (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_dynamic_bsz': False, (TaskRunner pid=2347384) 'use_fused_kernels': False, (TaskRunner pid=2347384) 'use_kl_loss': False, (TaskRunner pid=2347384) 'use_prefix_grouper': False, (TaskRunner pid=2347384) 'use_remove_padding': True, (TaskRunner pid=2347384) 'use_torch_compile': True}, (TaskRunner pid=2347384) 'hybrid_engine': True, (TaskRunner pid=2347384) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2347384) 'custom_chat_template': None, (TaskRunner pid=2347384) 'enable_activation_offload': False, (TaskRunner pid=2347384) 'enable_gradient_checkpointing': True, (TaskRunner pid=2347384) 'exclude_modules': None, (TaskRunner pid=2347384) 'external_lib': None, (TaskRunner pid=2347384) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2347384) 'hf_config_path': None, (TaskRunner pid=2347384) 'lora_adapter_path': None, (TaskRunner pid=2347384) 'lora_alpha': 16, (TaskRunner pid=2347384) 'lora_rank': 0, (TaskRunner pid=2347384) 'override_config': {}, (TaskRunner pid=2347384) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2347384) 'target_modules': 'all-linear', (TaskRunner pid=2347384) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2347384) 'num_shards': 4}, (TaskRunner pid=2347384) 'tokenizer_path': None, (TaskRunner pid=2347384) 'trust_remote_code': True, (TaskRunner pid=2347384) 'use_fused_kernels': False, (TaskRunner pid=2347384) 'use_liger': False, (TaskRunner pid=2347384) 'use_remove_padding': True, (TaskRunner pid=2347384) 'use_shm': False}, (TaskRunner pid=2347384) 'nccl_timeout': 600, (TaskRunner pid=2347384) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2347384) 'entropy_checkpointing': False, (TaskRunner pid=2347384) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2347384) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2347384) 'dtype': 'bfloat16', (TaskRunner pid=2347384) 'entropy_checkpointing': False, (TaskRunner pid=2347384) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2347384) 'forward_only': True, (TaskRunner pid=2347384) 'forward_prefetch': False, (TaskRunner pid=2347384) 'fsdp_size': -1, (TaskRunner pid=2347384) 'full_determinism': False, (TaskRunner pid=2347384) 'model_dtype': 'fp32', (TaskRunner pid=2347384) 'offload_policy': False, (TaskRunner pid=2347384) 'optimizer_offload': False, (TaskRunner pid=2347384) 'param_offload': False, (TaskRunner pid=2347384) 'reshard_after_forward': True, (TaskRunner pid=2347384) 'seed': 42, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_orig_params': False, (TaskRunner pid=2347384) 'use_torch_compile': True, (TaskRunner pid=2347384) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2347384) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2347384) 'log_prob_micro_batch_size': None, (TaskRunner pid=2347384) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2347384) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2347384) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'all_ranks': False, (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'ranks': [], (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'tool': None, (TaskRunner pid=2347384) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2347384) 'analysis': True, (TaskRunner pid=2347384) 'contents': [], (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'level': 'level0'}, (TaskRunner pid=2347384) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'discrete': False}, (TaskRunner pid=2347384) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2347384) 'step_end': None, (TaskRunner pid=2347384) 'step_start': 0}, (TaskRunner pid=2347384) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2347384) 'rollout_n': 8, (TaskRunner pid=2347384) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2347384) 'mode': 'disabled', (TaskRunner pid=2347384) 'record_file': None, (TaskRunner pid=2347384) 'replay_file': None}, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_torch_compile': True}, (TaskRunner pid=2347384) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2347384) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2347384) 'agent_loop_config_path': None, (TaskRunner pid=2347384) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2347384) 'name': None, (TaskRunner pid=2347384) 'path': None}, (TaskRunner pid=2347384) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2347384) 'num_workers': 8}, (TaskRunner pid=2347384) 'calculate_log_probs': True, (TaskRunner pid=2347384) 'cudagraph_capture_sizes': None, (TaskRunner pid=2347384) 'data_parallel_size': 1, (TaskRunner pid=2347384) 'disable_log_stats': True, (TaskRunner pid=2347384) 'do_sample': True, (TaskRunner pid=2347384) 'dtype': 'bfloat16', (TaskRunner pid=2347384) 'enable_chunked_prefill': True, (TaskRunner pid=2347384) 'enable_prefix_caching': True, (TaskRunner pid=2347384) 'enable_rollout_routing_replay': False, (TaskRunner pid=2347384) 'enforce_eager': False, (TaskRunner pid=2347384) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2347384) 'expert_parallel_size': 1, (TaskRunner pid=2347384) 'free_cache_engine': True, (TaskRunner pid=2347384) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2347384) 'ignore_eos': False, (TaskRunner pid=2347384) 'layered_summon': False, (TaskRunner pid=2347384) 'load_format': 'dummy', (TaskRunner pid=2347384) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2347384) 'log_prob_micro_batch_size': None, (TaskRunner pid=2347384) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2347384) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2347384) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2347384) 'max_model_len': 10240, (TaskRunner pid=2347384) 'max_num_batched_tokens': 10240, (TaskRunner pid=2347384) 'max_num_seqs': 1024, (TaskRunner pid=2347384) 'mode': 'async', (TaskRunner pid=2347384) 'multi_stage_wake_up': False, (TaskRunner pid=2347384) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'format': 'hermes', (TaskRunner pid=2347384) 'interaction_config_path': None, (TaskRunner pid=2347384) 'max_assistant_turns': None, (TaskRunner pid=2347384) 'max_parallel_calls': 1, (TaskRunner pid=2347384) 'max_tool_response_length': 256, (TaskRunner pid=2347384) 'max_user_turns': None, (TaskRunner pid=2347384) 'num_repeat_rollouts': None, (TaskRunner pid=2347384) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2347384) 'tool_config_path': None, (TaskRunner pid=2347384) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2347384) 'use_inference_chat_template': False}, (TaskRunner pid=2347384) 'n': 8, (TaskRunner pid=2347384) 'name': 'vllm', (TaskRunner pid=2347384) 'over_sample_rate': 0, (TaskRunner pid=2347384) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2347384) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'all_ranks': False, (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'ranks': [], (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'tool': None, (TaskRunner pid=2347384) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2347384) 'analysis': True, (TaskRunner pid=2347384) 'contents': [], (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'level': 'level0'}, (TaskRunner pid=2347384) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'discrete': False}, (TaskRunner pid=2347384) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2347384) 'step_end': None, (TaskRunner pid=2347384) 'step_start': 0}, (TaskRunner pid=2347384) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2347384) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2347384) 'port': 9090, (TaskRunner pid=2347384) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2347384) 'prompt_length': 2048, (TaskRunner pid=2347384) 'quantization': None, (TaskRunner pid=2347384) 'quantization_config_file': None, (TaskRunner pid=2347384) 'response_length': 8192, (TaskRunner pid=2347384) 'scheduling_policy': 'fcfs', (TaskRunner pid=2347384) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2347384) 'skip_rollout': False, (TaskRunner pid=2347384) 'skip_tokenizer_init': True, (TaskRunner pid=2347384) 'temperature': 1.0, (TaskRunner pid=2347384) 'tensor_model_parallel_size': 2, (TaskRunner pid=2347384) 'top_k': -1, (TaskRunner pid=2347384) 'top_p': 1.0, (TaskRunner pid=2347384) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2347384) 'backend': None, (TaskRunner pid=2347384) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2347384) 'token2text': False}, (TaskRunner pid=2347384) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2347384) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2347384) 'do_sample': True, (TaskRunner pid=2347384) 'n': 16, (TaskRunner pid=2347384) 'temperature': 0.6, (TaskRunner pid=2347384) 'top_k': -1, (TaskRunner pid=2347384) 'top_p': 0.95}}}, (TaskRunner pid=2347384) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2347384) 'adv_estimator': 'grpo', (TaskRunner pid=2347384) 'gamma': 1.0, (TaskRunner pid=2347384) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2347384) 'horizon': 10000, (TaskRunner pid=2347384) 'kl_coef': 0.001, (TaskRunner pid=2347384) 'target_kl': 0.1, (TaskRunner pid=2347384) 'type': 'fixed'}, (TaskRunner pid=2347384) 'kl_penalty': 'kl', (TaskRunner pid=2347384) 'lam': 1.0, (TaskRunner pid=2347384) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2347384) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2347384) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2347384) 'loss_type': 'ppo_clip', (TaskRunner pid=2347384) 'rollout_is': 'token', (TaskRunner pid=2347384) 'rollout_is_batch_normalize': False, (TaskRunner pid=2347384) 'rollout_is_threshold': 2.0, (TaskRunner pid=2347384) 'rollout_rs': None, (TaskRunner pid=2347384) 'rollout_rs_threshold': None}, (TaskRunner pid=2347384) 'use_kl_in_reward': False, (TaskRunner pid=2347384) 'use_pf_ppo': False}, (TaskRunner pid=2347384) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2347384) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2347384) 'async_save': False, (TaskRunner pid=2347384) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2347384) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2347384) 'cliprange_value': 0.5, (TaskRunner pid=2347384) 'data_loader_seed': 42, (TaskRunner pid=2347384) 'enable': None, (TaskRunner pid=2347384) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2347384) 'forward_micro_batch_size': None, (TaskRunner pid=2347384) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2347384) 'grad_clip': 1.0, (TaskRunner pid=2347384) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2347384) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2347384) 'enable_activation_offload': False, (TaskRunner pid=2347384) 'enable_gradient_checkpointing': True, (TaskRunner pid=2347384) 'external_lib': None, (TaskRunner pid=2347384) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2347384) 'dtype': 'bfloat16', (TaskRunner pid=2347384) 'entropy_checkpointing': False, (TaskRunner pid=2347384) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2347384) 'forward_only': False, (TaskRunner pid=2347384) 'forward_prefetch': False, (TaskRunner pid=2347384) 'fsdp_size': -1, (TaskRunner pid=2347384) 'full_determinism': False, (TaskRunner pid=2347384) 'model_dtype': 'fp32', (TaskRunner pid=2347384) 'offload_policy': False, (TaskRunner pid=2347384) 'optimizer_offload': False, (TaskRunner pid=2347384) 'param_offload': False, (TaskRunner pid=2347384) 'reshard_after_forward': True, (TaskRunner pid=2347384) 'seed': 42, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_orig_params': False, (TaskRunner pid=2347384) 'use_torch_compile': True, (TaskRunner pid=2347384) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2347384) 'lora_alpha': 16, (TaskRunner pid=2347384) 'lora_rank': 0, (TaskRunner pid=2347384) 'override_config': {}, (TaskRunner pid=2347384) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2347384) 'target_modules': 'all-linear', (TaskRunner pid=2347384) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2347384) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2347384) 'trust_remote_code': True, (TaskRunner pid=2347384) 'use_remove_padding': False, (TaskRunner pid=2347384) 'use_shm': False}, (TaskRunner pid=2347384) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2347384) 'betas': [0.9, 0.999], (TaskRunner pid=2347384) 'clip_grad': 1.0, (TaskRunner pid=2347384) 'lr': 1e-05, (TaskRunner pid=2347384) 'lr_scheduler_type': 'constant', (TaskRunner pid=2347384) 'lr_warmup_steps': -1, (TaskRunner pid=2347384) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2347384) 'min_lr_ratio': 0.0, (TaskRunner pid=2347384) 'num_cycles': 0.5, (TaskRunner pid=2347384) 'optimizer': 'AdamW', (TaskRunner pid=2347384) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2347384) 'override_optimizer_config': None, (TaskRunner pid=2347384) 'total_training_steps': -1, (TaskRunner pid=2347384) 'warmup_style': None, (TaskRunner pid=2347384) 'weight_decay': 0.01}, (TaskRunner pid=2347384) 'ppo_epochs': 1, (TaskRunner pid=2347384) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2347384) 'ppo_micro_batch_size': None, (TaskRunner pid=2347384) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2347384) 'ppo_mini_batch_size': 8, (TaskRunner pid=2347384) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'all_ranks': False, (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'ranks': [], (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'tool': None, (TaskRunner pid=2347384) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2347384) 'analysis': True, (TaskRunner pid=2347384) 'contents': [], (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'level': 'level0'}, (TaskRunner pid=2347384) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'discrete': False}, (TaskRunner pid=2347384) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2347384) 'step_end': None, (TaskRunner pid=2347384) 'step_start': 0}, (TaskRunner pid=2347384) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2347384) 'rollout_n': 8, (TaskRunner pid=2347384) 'shuffle': False, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_dynamic_bsz': False}, (TaskRunner pid=2347384) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2347384) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2347384) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2347384) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2347384) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2347384) 'dataloader_num_workers': 8, (TaskRunner pid=2347384) 'filter_overlong_prompts': True, (TaskRunner pid=2347384) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2347384) 'image_key': 'images', (TaskRunner pid=2347384) 'image_patch_size': 14, (TaskRunner pid=2347384) 'max_prompt_length': 2048, (TaskRunner pid=2347384) 'max_response_length': 8192, (TaskRunner pid=2347384) 'prompt_key': 'prompt', (TaskRunner pid=2347384) 'return_full_prompt': False, (TaskRunner pid=2347384) 'return_multi_modal_inputs': True, (TaskRunner pid=2347384) 'return_raw_chat': True, (TaskRunner pid=2347384) 'return_raw_input_ids': False, (TaskRunner pid=2347384) 'reward_fn_key': 'data_source', (TaskRunner pid=2347384) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2347384) 'seed': None, (TaskRunner pid=2347384) 'shuffle': True, (TaskRunner pid=2347384) 'tokenizer': None, (TaskRunner pid=2347384) 'tool_config_path': None, (TaskRunner pid=2347384) 'train_batch_size': 32, (TaskRunner pid=2347384) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet'], (TaskRunner pid=2347384) 'train_max_samples': 3200, (TaskRunner pid=2347384) 'truncation': 'error', (TaskRunner pid=2347384) 'trust_remote_code': True, (TaskRunner pid=2347384) 'use_shm': False, (TaskRunner pid=2347384) 'val_batch_size': None, (TaskRunner pid=2347384) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet'], (TaskRunner pid=2347384) 'val_max_samples': -1, (TaskRunner pid=2347384) 'validation_shuffle': False, (TaskRunner pid=2347384) 'video_key': 'videos'}, (TaskRunner pid=2347384) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2347384) 'cuda-memory-usage': 'true', (TaskRunner pid=2347384) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2347384) 'capture-range-end': None, (TaskRunner pid=2347384) 'cuda-graph-trace': 'graph', (TaskRunner pid=2347384) 'cuda-memory-usage': 'true', (TaskRunner pid=2347384) 'kill': 'none', (TaskRunner pid=2347384) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2347384) 'torch_memory': {'context': 'all', (TaskRunner pid=2347384) 'kw_args': {}, (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'stacks': 'all', (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2347384) 'profile_continuous_steps': False, (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'steps': None, (TaskRunner pid=2347384) 'tool': None}, (TaskRunner pid=2347384) 'max_model_len': 10240, (TaskRunner pid=2347384) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_biology_grpo_Qwen3_8B', (TaskRunner pid=2347384) 'include_dashboard': False, (TaskRunner pid=2347384) 'num_cpus': None, (TaskRunner pid=2347384) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2347384) 'TASK': 'datasets/sciknoweval/biology', (TaskRunner pid=2347384) 'USER': 'root'}}}, (TaskRunner pid=2347384) 'timeline_json_file': None}, (TaskRunner pid=2347384) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2347384) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2347384) 'name': 'custom_reward_manager', (TaskRunner pid=2347384) 'path': None}, (TaskRunner pid=2347384) 'name': 'naive', (TaskRunner pid=2347384) 'source': 'register'}, (TaskRunner pid=2347384) 'reward_model': {'enable': False, (TaskRunner pid=2347384) 'enable_resource_pool': False, (TaskRunner pid=2347384) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2347384) 'launch_reward_fn_async': False, (TaskRunner pid=2347384) 'max_length': None, (TaskRunner pid=2347384) 'micro_batch_size': None, (TaskRunner pid=2347384) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2347384) 'model': {'external_lib': None, (TaskRunner pid=2347384) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2347384) 'forward_prefetch': False, (TaskRunner pid=2347384) 'fsdp_size': -1, (TaskRunner pid=2347384) 'param_offload': False, (TaskRunner pid=2347384) 'reshard_after_forward': True, (TaskRunner pid=2347384) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2347384) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2347384) 'override_config': {}, (TaskRunner pid=2347384) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2347384) 'trust_remote_code': False, (TaskRunner pid=2347384) 'use_fused_kernels': False, (TaskRunner pid=2347384) 'use_remove_padding': False, (TaskRunner pid=2347384) 'use_shm': False}, (TaskRunner pid=2347384) 'n_gpus_per_node': 8, (TaskRunner pid=2347384) 'nnodes': 0, (TaskRunner pid=2347384) 'num_workers': 1, (TaskRunner pid=2347384) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2347384) 'all_ranks': False, (TaskRunner pid=2347384) 'enable': False, (TaskRunner pid=2347384) 'ranks': [], (TaskRunner pid=2347384) 'save_path': 'outputs/profile', (TaskRunner pid=2347384) 'tool': None, (TaskRunner pid=2347384) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2347384) 'analysis': True, (TaskRunner pid=2347384) 'contents': [], (TaskRunner pid=2347384) 'discrete': False, (TaskRunner pid=2347384) 'level': 'level0'}, (TaskRunner pid=2347384) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2347384) 'discrete': False}, (TaskRunner pid=2347384) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2347384) 'step_end': None, (TaskRunner pid=2347384) 'step_start': 0}, (TaskRunner pid=2347384) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2347384) 'stack_depth': 32, (TaskRunner pid=2347384) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2347384) 'reward_loop_class_name': None, (TaskRunner pid=2347384) 'reward_loop_module_path': None, (TaskRunner pid=2347384) 'reward_loop_source': 'register', (TaskRunner pid=2347384) 'reward_manager': 'naive', (TaskRunner pid=2347384) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2347384) 'cudagraph_capture_sizes': None, (TaskRunner pid=2347384) 'data_parallel_size': 1, (TaskRunner pid=2347384) 'disable_log_stats': True, (TaskRunner pid=2347384) 'dtype': 'bfloat16', (TaskRunner pid=2347384) 'enable_chunked_prefill': True, (TaskRunner pid=2347384) 'enable_prefix_caching': True, (TaskRunner pid=2347384) 'enforce_eager': True, (TaskRunner pid=2347384) 'engine_kwargs': {}, (TaskRunner pid=2347384) 'expert_parallel_size': 1, (TaskRunner pid=2347384) 'free_cache_engine': True, (TaskRunner pid=2347384) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2347384) 'limit_images': None, (TaskRunner pid=2347384) 'load_format': 'auto', (TaskRunner pid=2347384) 'max_model_len': None, (TaskRunner pid=2347384) 'max_num_batched_tokens': 8192, (TaskRunner pid=2347384) 'max_num_seqs': 1024, (TaskRunner pid=2347384) 'name': '???', (TaskRunner pid=2347384) 'prompt_length': 2048, (TaskRunner pid=2347384) 'response_length': 2048, (TaskRunner pid=2347384) 'skip_tokenizer_init': False, (TaskRunner pid=2347384) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2347384) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2347384) 'memory_limit_mb': 1024, (TaskRunner pid=2347384) 'url': None}, (TaskRunner pid=2347384) 'strategy': 'fsdp', (TaskRunner pid=2347384) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2347384) 'use_dynamic_bsz': False, (TaskRunner pid=2347384) 'use_reward_loop': False}, (TaskRunner pid=2347384) 'trainer': {'balance_batch': True, (TaskRunner pid=2347384) 'critic_warmup': 0, (TaskRunner pid=2347384) 'default_hdfs_dir': None, (TaskRunner pid=2347384) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2347384) 'del_local_ckpt_after_load': False, (TaskRunner pid=2347384) 'device': 'cuda', (TaskRunner pid=2347384) 'esi_redundant_time': 0, (TaskRunner pid=2347384) 'experiment_name': 'qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2347384) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2347384) 'log_val_generations': 0, (TaskRunner pid=2347384) 'logger': ['console', 'wandb'], (TaskRunner pid=2347384) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2347384) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2347384) 'n_gpus_per_node': 8, (TaskRunner pid=2347384) 'nnodes': 1, (TaskRunner pid=2347384) 'project_name': 'SDPO-root', (TaskRunner pid=2347384) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2347384) 'resume_from_path': None, (TaskRunner pid=2347384) 'resume_mode': 'auto', (TaskRunner pid=2347384) 'rollout_data_dir': None, (TaskRunner pid=2347384) 'save_freq': 10, (TaskRunner pid=2347384) 'test_freq': 10, (TaskRunner pid=2347384) 'total_epochs': 30, (TaskRunner pid=2347384) 'total_training_steps': 100, (TaskRunner pid=2347384) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2347384) 'val_before_train': False, (TaskRunner pid=2347384) 'val_only': False, (TaskRunner pid=2347384) 'validation_data_dir': None}, (TaskRunner pid=2347384) 'transfer_queue': {'enable': False}, (TaskRunner pid=2347384) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/biology', (TaskRunner pid=2347384) 'dir': '/users/root/SDPO', (TaskRunner pid=2347384) 'log_dir': '/users/root/output', (TaskRunner pid=2347384) 'task': 'datasets/sciknoweval/biology'}} (TaskRunner pid=2347384) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2347384) use_critic=need_critic(config), (TaskRunner pid=2347384) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2347384) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2347384) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2347384) Using dataset class: RLHFDataset (TaskRunner pid=2347384) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2347384) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2347384) dataset len: 450 (TaskRunner pid=2347384) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2347384) WARNING:2026-07-03 07:28:14,003:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2347384) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/450 [00:00 (TaskRunner pid=2347384) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2347384) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2347384) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2347384) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2347384) bind role actor_rollout method wake_up to class .WorkerDict'> (WorkerDict pid=2347763) [W703 07:28:23.527944856 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:44381 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2347763) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2347766) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2347766) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2347764) [W703 07:28:23.702362713 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:44381 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2347763) Model config after override: Qwen3Config { (WorkerDict pid=2347763) "architectures": [ (WorkerDict pid=2347763) "Qwen3ForCausalLM" (WorkerDict pid=2347763) ], (WorkerDict pid=2347763) "attention_bias": false, (WorkerDict pid=2347763) "attention_dropout": 0.0, (WorkerDict pid=2347763) "dtype": "bfloat16", (WorkerDict pid=2347763) "eos_token_id": 151645, (WorkerDict pid=2347763) "head_dim": 128, (WorkerDict pid=2347763) "hidden_act": "silu", (WorkerDict pid=2347763) "hidden_size": 4096, (WorkerDict pid=2347763) "initializer_range": 0.02, (WorkerDict pid=2347763) "intermediate_size": 12288, (WorkerDict pid=2347763) "layer_types": [ (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention", (WorkerDict pid=2347763) "full_attention" (WorkerDict pid=2347763) ], (WorkerDict pid=2347763) "max_position_embeddings": 40960, (WorkerDict pid=2347763) "max_window_layers": 36, (WorkerDict pid=2347763) "model_type": "qwen3", (WorkerDict pid=2347763) "num_attention_heads": 32, (WorkerDict pid=2347763) "num_hidden_layers": 36, (WorkerDict pid=2347763) "num_key_value_heads": 8, (WorkerDict pid=2347763) "pad_token_id": 151643, (WorkerDict pid=2347763) "rms_norm_eps": 1e-06, (WorkerDict pid=2347763) "rope_scaling": null, (WorkerDict pid=2347763) "rope_theta": 1000000, (WorkerDict pid=2347763) "sliding_window": null, (WorkerDict pid=2347763) "tie_word_embeddings": false, (WorkerDict pid=2347763) "transformers_version": "4.57.1", (WorkerDict pid=2347763) "use_cache": true, (WorkerDict pid=2347763) "use_sliding_window": false, (WorkerDict pid=2347763) "vocab_size": 151936 (WorkerDict pid=2347763) } (WorkerDict pid=2347763) (WorkerDict pid=2347766) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2347766) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2347766) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2347769) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2347763) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2347764) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2347764) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2347770) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.95s/it] [repeated 15x across cluster] (WorkerDict pid=2347770) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.24s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.65s/it] [repeated 7x across cluster] (WorkerDict pid=2347763) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2347763) Actor use_remove_padding=True (WorkerDict pid=2347763) Actor use_fused_kernels=False (WorkerDict pid=2347763) Actor use_prefix_grouper=False (WorkerDict pid=2347770) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2347770) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2347766) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2347766) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2347764) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2347763) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2347763) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2347763) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2347763) warnings.warn( (WorkerDict pid=2347763) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2347763) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2347384) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2347384) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2347769) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2347769) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2348653) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2348653) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2348653) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2348653) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2348652) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2348652) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2348652) ['serve', (vLLMHttpServer pid=2348652) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2348652) '--dtype', (vLLMHttpServer pid=2348652) 'bfloat16', (vLLMHttpServer pid=2348652) '--load_format', (vLLMHttpServer pid=2348652) 'dummy', (vLLMHttpServer pid=2348652) '--trust_remote_code', (vLLMHttpServer pid=2348652) '--max_model_len', (vLLMHttpServer pid=2348652) '40960', (vLLMHttpServer pid=2348652) '--max_num_seqs', (vLLMHttpServer pid=2348652) '1024', (vLLMHttpServer pid=2348652) '--enable_chunked_prefill', (vLLMHttpServer pid=2348652) '--max_num_batched_tokens', (vLLMHttpServer pid=2348652) '10240', (vLLMHttpServer pid=2348652) '--enable_prefix_caching', (vLLMHttpServer pid=2348652) '--enable_sleep_mode', (vLLMHttpServer pid=2348652) '--logprobs_mode', (vLLMHttpServer pid=2348652) 'processed_logprobs', (vLLMHttpServer pid=2348652) '--disable_custom_all_reduce', (vLLMHttpServer pid=2348652) '--gpu_memory_utilization', (vLLMHttpServer pid=2348652) '0.8', (vLLMHttpServer pid=2348652) '--disable_log_stats', (vLLMHttpServer pid=2348652) '--tensor_parallel_size', (vLLMHttpServer pid=2348652) '2', (vLLMHttpServer pid=2348652) '--seed', (vLLMHttpServer pid=2348652) '0', (vLLMHttpServer pid=2348652) '--override_generation_config', (vLLMHttpServer pid=2348652) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2348652) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2348652) '--hf_overrides', (vLLMHttpServer pid=2348652) '{}', (vLLMHttpServer pid=2348652) '--scheduling_policy', (vLLMHttpServer pid=2348652) 'fcfs'] (WorkerDict pid=2347769) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2348653) WARNING 07-03 07:29:13 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2347766) WARNING 07-03 07:29:21 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2348654) WARNING 07-03 07:29:14 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2347767) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2347763) 2026-07-03 07:29:35,888 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2347763) 2026-07-03 07:29:35,906 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2348654) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2348654) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (vLLMHttpServer pid=2348654) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2348654) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (WorkerDict pid=2347763) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) The folding stability score of a protein sequence is typically calculated using computational methods that evaluate the likelihood of the sequence folding into a stable three-dimensional structure. These methods often involve analyzing the energy landscape of the protein, predicting secondary structure elements, and assessing the overall stability based on amino acid interactions and hydrophobicity. (TaskRunner pid=2347384) (TaskRunner pid=2347384) In the absence of direct computation or access to a specific algorithm to calculate the folding stability score for the given protein sequence, we cannot determine the exact value. However, based on the provided options, we can infer that the correct answer is the one that represents a reasonable folding stability score. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Among the options, option B (1.69) and option C (2.49) are both positive values, which could indicate a relatively stable folding structure. Option A (-0.01) is very close to zero, which might suggest a slightly unstable or neutral folding stability. Option D (0.45) is also a positive value but smaller than B and C. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given that folding stability scores are typically reported as positive values, and considering the context of the question, the most plausible answer is B (1.69), as it represents a moderate to stable folding stability score. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) B (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 1.0 (TaskRunner pid=2347384) [acc] 1.0 (TaskRunner pid=2347384) [pred] B (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2347384) step:10 - global_seqlen/min:14026 - global_seqlen/max:26212 - global_seqlen/minmax_diff:12186 - global_seqlen/balanced_min:18467 - global_seqlen/balanced_max:25366 - global_seqlen/mean:19332.375 - actor/entropy:0.45262259244918823 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5830507278442383 - training/rollout_probs_diff_mean:0.004972450900822878 - training/rollout_probs_diff_std:0.011055339127779007 - training/rollout_actor_probs_pearson_corr:0.9992586374282837 - rollout_corr/rollout_is_mean:0.9999414682388306 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1841186278616078e-05 - rollout_corr/rollout_is_max:1.7506085634231567 - rollout_corr/rollout_is_min:0.3154762089252472 - rollout_corr/rollout_is_std:0.03809688985347748 - rollout_corr/rollout_is_eff_sample_size:0.9985504926027746 - rollout_corr/rollout_is_seq_mean:0.999889612197876 - rollout_corr/rollout_is_seq_std:0.0027533581014722586 - rollout_corr/rollout_is_seq_max:1.010258674621582 - rollout_corr/rollout_is_seq_min:0.9903730750083923 - rollout_corr/rollout_is_seq_max_deviation:0.010258674621582031 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.716814047191292) - rollout_corr/training_log_ppl:np.float64(0.5270733479010232) - rollout_corr/kl:np.float64(0.0010048211948259222) - rollout_corr/k3_kl:np.float64(0.0008655035604760997) - rollout_corr/rollout_ppl:np.float64(1.715072752442211) - rollout_corr/rollout_log_ppl:np.float64(0.5260685242756153) - rollout_corr/log_ppl_diff:np.float64(0.0010048236254078802) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021937599194643553) - rollout_corr/log_ppl_diff_max:np.float64(0.00876617431640625) - rollout_corr/log_ppl_diff_min:np.float64(-0.009220004081726074) - rollout_corr/ppl_ratio:np.float64(1.0010089005809277) - rollout_corr/chi2_token:np.float64(0.0014747201930731535) - rollout_corr/chi2_seq:np.float64(0.7448661010712385) - actor/pg_loss:np.float64(-0.00027040555141866207) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006960646828702011) - actor/ppo_kl:np.float64(3.835439729282797e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5604352355003357) - perf/mfu/actor:np.float64(0.07364472637593265) - perf/max_memory_allocated_gb:np.float64(132.02229022979736) - perf/max_memory_reserved_gb:np.float64(144.71875) - perf/cpu_memory_used_gb:np.float64(98.74493789672852) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.305) - val-aux/sciknoweval/reward/std@16:np.float64(0.34964298228177393) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.45586) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.34684612554072713) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.1565) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.24691334720794672) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.30598000000000003) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.350415368201675) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6034799999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.2922414405058621) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.0688) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13262228575634746) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.30114) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.3016085903429573) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7244600000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.20789043475164487) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.025199999999999997) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.057717604537027185) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.29326) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.2483461544296699) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8033999999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.11852761284640682) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.0065000000000000014) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.023749015715872786) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.28244) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1927622182845085) - val-aux/sciknoweval/score/mean@16:np.float64(0.305) - val-aux/sciknoweval/score/std@16:np.float64(0.34964298228177393) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.45586) - val-aux/sciknoweval/score/best@2/std:np.float64(0.34684612554072713) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.1565) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.24691334720794672) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.30598000000000003) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.350415368201675) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6034799999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.2922414405058621) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.0688) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13262228575634746) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.30114) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.3016085903429573) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7244600000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.20789043475164487) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.025199999999999997) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.057717604537027185) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.29326) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.2483461544296699) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8033999999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.11852761284640682) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.0065000000000000014) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.023749015715872786) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.28244) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1927622182845085) - val-core/sciknoweval/acc/mean@16:np.float64(0.305) - val-aux/sciknoweval/acc/std@16:np.float64(0.34964298228177393) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.45586) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.34684612554072713) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.1565) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.24691334720794672) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.30598000000000003) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.350415368201675) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6034799999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.2922414405058621) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.0688) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13262228575634746) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.30114) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.3016085903429573) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7244600000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.20789043475164487) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.025199999999999997) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.057717604537027185) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.29326) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.2483461544296699) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8033999999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.11852761284640682) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.0065000000000000014) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.023749015715872786) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.28244) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1927622182845085) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.97125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.060544496479082686) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9928400000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.02934722149080065) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9498399999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.07393366420051682) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9717999999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.06020345104042302) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.99938) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.005582341464791209) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.91966) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07936022015675398) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.98834) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03981096521967535) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.88452) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07223829094842676) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9972) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.019694784702167224) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8517600000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05263531516347757) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004406536886723831) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.32421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.32421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.029959680512547493 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.029959680512547493 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:329.88671875 - response_length/max:8192.0 - response_length/min:128.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:329.88671875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:274.25 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019344687461853027 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5390802696347237) - timing_s/agent_loop/generate_sequences/max:np.float64(56.44360770471394) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5275733740854776) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(56.44360770471394) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:62.94056263193488 - timing_s/reward:0.05771280638873577 - timing_s/old_log_prob:3.3531495556235313 - timing_s/adv:0.13802666775882244 - timing_s/update_actor:13.871477348729968 - timing_s/step:80.44779045693576 - timing_s/testing:96.19575156643987 - timing_s/save_checkpoint:12.671725941821933 - timing_s/stop_profile:0.00013373419642448425 - timing_per_token_ms/update_actor:0.08969072183791417 - timing_per_token_ms/gen:0.7452909098996445 - timing_per_token_ms/adv:0.0008924580383865307 - perf/total_num_tokens:154659 - perf/time_per_step:80.44779045693576 - perf/throughput:240.30958327374756 (TaskRunner pid=2347384) Training Progress: 10%|█ | 10/100 [10:23<2:12:15, 88.17s/it] (TaskRunner pid=2347384) step:11 - global_seqlen/min:16957 - global_seqlen/max:30433 - global_seqlen/minmax_diff:13476 - global_seqlen/balanced_min:22590 - global_seqlen/balanced_max:25365 - global_seqlen/mean:22965.625 - actor/entropy:0.4155322313308716 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3120696544647217 - training/rollout_probs_diff_mean:0.004567084833979607 - training/rollout_probs_diff_std:0.010465013794600964 - training/rollout_actor_probs_pearson_corr:0.9992883801460266 - rollout_corr/rollout_is_mean:0.9998583197593689 - rollout_corr/rollout_is_ratio_fraction_high:1.0084203495352995e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.016840699070599e-05 - rollout_corr/rollout_is_max:2.0753023624420166 - rollout_corr/rollout_is_min:0.4422110319137573 - rollout_corr/rollout_is_std:0.0358213447034359 - rollout_corr/rollout_is_eff_sample_size:0.9987180000038098 - rollout_corr/rollout_is_seq_mean:0.9998087882995605 - rollout_corr/rollout_is_seq_std:0.002214111154899001 - rollout_corr/rollout_is_seq_max:1.0086524486541748 - rollout_corr/rollout_is_seq_min:0.993055522441864 - rollout_corr/rollout_is_seq_max_deviation:0.008652448654174805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6284825187176466) - rollout_corr/training_log_ppl:np.float64(0.47535870561841875) - rollout_corr/kl:np.float64(0.0009332117763483083) - rollout_corr/k3_kl:np.float64(0.0008248986909222822) - rollout_corr/rollout_ppl:np.float64(1.6268996740691364) - rollout_corr/rollout_log_ppl:np.float64(0.47442549155675806) - rollout_corr/log_ppl_diff:np.float64(0.0009332140616606921) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019218551169615239) - rollout_corr/log_ppl_diff_max:np.float64(0.007362186908721924) - rollout_corr/log_ppl_diff_min:np.float64(-0.007423877716064453) - rollout_corr/ppl_ratio:np.float64(1.0009363759309053) - rollout_corr/chi2_token:np.float64(0.001480589620769024) - rollout_corr/chi2_seq:np.float64(0.6636504703201354) - actor/pg_loss:np.float64(0.0001596701331436634) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00046512034583656714) - actor/ppo_kl:np.float64(-2.2117470098947933e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5356682911515236) - perf/mfu/actor:np.float64(0.08978195743431006) - perf/max_memory_allocated_gb:np.float64(132.02229022979736) - perf/max_memory_reserved_gb:np.float64(144.71875) - perf/cpu_memory_used_gb:np.float64(102.40783309936523) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004459738731384277 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004459738731384277 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:387.36328125 - response_length/max:4649.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:387.36328125 - response_length_non_aborted/max:4649.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.3125 - prompt_length/max:500.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001032371073961258 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7794717792421579) - timing_s/agent_loop/generate_sequences/max:np.float64(28.687389539554715) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.100075885755359) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.687389539554715) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:433 - timing_s/agent_loop/slowest/response_length:4649 - timing_s/gen:35.13612602651119 - timing_s/reward:0.05825859680771828 - timing_s/old_log_prob:3.552225437015295 - timing_s/adv:0.17767742276191711 - timing_s/update_actor:13.233200673013926 - timing_s/step:52.25685699656606 - timing_s/stop_profile:0.00011438876390457153 - timing_per_token_ms/update_actor:0.07202721825017785 - timing_per_token_ms/gen:0.3543198308527322 - timing_per_token_ms/adv:0.0009670835366004469 - perf/total_num_tokens:183725 - perf/time_per_step:52.25685699656606 - perf/throughput:439.4758184846275 (TaskRunner pid=2347384) Training Progress: 11%|█ | 11/100 [11:16<1:54:30, 77.20s/it] (TaskRunner pid=2347384) step:12 - global_seqlen/min:18215 - global_seqlen/max:29458 - global_seqlen/minmax_diff:11243 - global_seqlen/balanced_min:23841 - global_seqlen/balanced_max:27478 - global_seqlen/mean:24339.0 - actor/entropy:0.43766021728515625 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30099600553512573 - training/rollout_probs_diff_mean:0.004334617871791124 - training/rollout_probs_diff_std:0.009989089332520962 - training/rollout_actor_probs_pearson_corr:0.9993811249732971 - rollout_corr/rollout_is_mean:1.000182867050171 - rollout_corr/rollout_is_ratio_fraction_high:9.775553735380527e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.887776594841853e-05 - rollout_corr/rollout_is_max:2.1124563217163086 - rollout_corr/rollout_is_min:0.011348603293299675 - rollout_corr/rollout_is_std:0.034875381737947464 - rollout_corr/rollout_is_eff_sample_size:0.9987856608682638 - rollout_corr/rollout_is_seq_mean:1.000220537185669 - rollout_corr/rollout_is_seq_std:0.0024740397930145264 - rollout_corr/rollout_is_seq_max:1.0120216608047485 - rollout_corr/rollout_is_seq_min:0.9940353035926819 - rollout_corr/rollout_is_seq_max_deviation:0.012021660804748535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6821262426674366) - rollout_corr/training_log_ppl:np.float64(0.5046567447425332) - rollout_corr/kl:np.float64(0.0005845932231274276) - rollout_corr/k3_kl:np.float64(0.0007912340478242186) - rollout_corr/rollout_ppl:np.float64(1.681106602307409) - rollout_corr/rollout_log_ppl:np.float64(0.5040721487021074) - rollout_corr/log_ppl_diff:np.float64(0.0005845960404258221) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019499089976307005) - rollout_corr/log_ppl_diff_max:np.float64(0.013257354497909546) - rollout_corr/log_ppl_diff_min:np.float64(-0.010273128747940063) - rollout_corr/ppl_ratio:np.float64(1.000588194001466) - rollout_corr/chi2_token:np.float64(0.001965596806257963) - rollout_corr/chi2_seq:np.float64(1.858226215466857) - actor/pg_loss:np.float64(0.0003983316710218787) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006193824146976112) - actor/ppo_kl:np.float64(6.872035001781462e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5202421024441719) - perf/mfu/actor:np.float64(0.09425964715479349) - perf/max_memory_allocated_gb:np.float64(132.02229022979736) - perf/max_memory_reserved_gb:np.float64(144.71875) - perf/cpu_memory_used_gb:np.float64(101.28682327270508) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.013849515467882156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.013849515467882156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:399.59375 - response_length/max:5387.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:399.59375 - response_length_non_aborted/max:5387.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:361.0 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.916350245475769e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.688298499211669) - timing_s/agent_loop/generate_sequences/max:np.float64(32.550289833918214) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.273808764126443) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(32.550289833918214) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:5387 - timing_s/gen:39.82046948000789 - timing_s/reward:0.0908251591026783 - timing_s/old_log_prob:3.288546323776245 - timing_s/adv:0.13700509257614613 - timing_s/update_actor:13.3698172532022 - timing_s/step:56.79229616187513 - timing_s/stop_profile:0.0001241397112607956 - timing_per_token_ms/update_actor:0.06866457770040983 - timing_per_token_ms/gen:0.3892671216861645 - timing_per_token_ms/adv:0.0007036294248744101 - perf/total_num_tokens:194712 - perf/time_per_step:56.79229616187513 - perf/throughput:428.5616473513684 (TaskRunner pid=2347384) Training Progress: 12%|█▏ | 12/100 [12:12<1:44:08, 71.01s/it] (TaskRunner pid=2347384) step:13 - global_seqlen/min:16426 - global_seqlen/max:28676 - global_seqlen/minmax_diff:12250 - global_seqlen/balanced_min:20919 - global_seqlen/balanced_max:27689 - global_seqlen/mean:21768.75 - actor/entropy:0.45051950216293335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25071096420288086 - training/rollout_probs_diff_mean:0.004617288708686829 - training/rollout_probs_diff_std:0.010263320989906788 - training/rollout_actor_probs_pearson_corr:0.999343752861023 - rollout_corr/rollout_is_mean:0.9999645948410034 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.044822874973761e-05 - rollout_corr/rollout_is_max:1.7554703950881958 - rollout_corr/rollout_is_min:0.37157756090164185 - rollout_corr/rollout_is_std:0.03532538563013077 - rollout_corr/rollout_is_eff_sample_size:0.9987534346031791 - rollout_corr/rollout_is_seq_mean:0.999995231628418 - rollout_corr/rollout_is_seq_std:0.002299003768712282 - rollout_corr/rollout_is_seq_max:1.0069160461425781 - rollout_corr/rollout_is_seq_min:0.9921134114265442 - rollout_corr/rollout_is_seq_max_deviation:0.00788658857345581 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6914544617757201) - rollout_corr/training_log_ppl:np.float64(0.5109681343019474) - rollout_corr/kl:np.float64(0.0007979853619914934) - rollout_corr/k3_kl:np.float64(0.00077781562168866) - rollout_corr/rollout_ppl:np.float64(1.6901205792091787) - rollout_corr/rollout_log_ppl:np.float64(0.5101701469830004) - rollout_corr/log_ppl_diff:np.float64(0.0007979873189469799) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018314038024982437) - rollout_corr/log_ppl_diff_max:np.float64(0.008499443531036377) - rollout_corr/log_ppl_diff_min:np.float64(-0.005160987377166748) - rollout_corr/ppl_ratio:np.float64(1.0008007904980332) - rollout_corr/chi2_token:np.float64(0.0015173715073615313) - rollout_corr/chi2_seq:np.float64(0.3772704761940986) - actor/pg_loss:np.float64(-0.0003307295264676213) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005128576804054319) - actor/ppo_kl:np.float64(6.100696820965368e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.45558615028858185) - perf/mfu/actor:np.float64(0.0824730901618031) - perf/max_memory_allocated_gb:np.float64(132.02229022979736) - perf/max_memory_reserved_gb:np.float64(144.71875) - perf/cpu_memory_used_gb:np.float64(101.28923034667969) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.3125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06721868366003036 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06721868366003036 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:373.8671875 - response_length/max:8192.0 - response_length/min:126.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:373.8671875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:306.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.137524127960205e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.67017375305295) - timing_s/agent_loop/generate_sequences/max:np.float64(58.25485855713487) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.205507205348113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.25485855713487) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.14460217580199 - timing_s/reward:0.06125919334590435 - timing_s/old_log_prob:3.345760280266404 - timing_s/adv:0.16389351710677147 - timing_s/update_actor:13.742141423746943 - timing_s/step:82.54458156228065 - timing_s/stop_profile:0.0001163724809885025 - timing_per_token_ms/update_actor:0.07890979858597154 - timing_per_token_ms/gen:0.6806457232870337 - timing_per_token_ms/adv:0.0009411054671649237 - perf/total_num_tokens:174150 - perf/time_per_step:82.54458156228065 - perf/throughput:263.72112606295394 (TaskRunner pid=2347384) Training Progress: 13%|█▎ | 13/100 [13:35<1:48:02, 74.52s/it] (TaskRunner pid=2347384) step:14 - global_seqlen/min:14671 - global_seqlen/max:39208 - global_seqlen/minmax_diff:24537 - global_seqlen/balanced_min:21329 - global_seqlen/balanced_max:27996 - global_seqlen/mean:22634.625 - actor/entropy:0.4110657870769501 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016807436943054 - training/rollout_probs_diff_mean:0.00429363502189517 - training/rollout_probs_diff_std:0.010314399376511574 - training/rollout_actor_probs_pearson_corr:0.9993447661399841 - rollout_corr/rollout_is_mean:0.9998406171798706 - rollout_corr/rollout_is_ratio_fraction_high:1.9119544958812185e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.8679316528723575e-05 - rollout_corr/rollout_is_max:2.719468116760254 - rollout_corr/rollout_is_min:0.39786025881767273 - rollout_corr/rollout_is_std:0.03521892428398132 - rollout_corr/rollout_is_eff_sample_size:0.9987608072183694 - rollout_corr/rollout_is_seq_mean:0.9998125433921814 - rollout_corr/rollout_is_seq_std:0.0023194169625639915 - rollout_corr/rollout_is_seq_max:1.006919264793396 - rollout_corr/rollout_is_seq_min:0.9936529397964478 - rollout_corr/rollout_is_seq_max_deviation:0.006919264793395996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.67210001219064) - rollout_corr/training_log_ppl:np.float64(0.4983691323141102) - rollout_corr/kl:np.float64(0.0009185323109477395) - rollout_corr/k3_kl:np.float64(0.0007933990461879148) - rollout_corr/rollout_ppl:np.float64(1.6704831109382212) - rollout_corr/rollout_log_ppl:np.float64(0.49745059617998777) - rollout_corr/log_ppl_diff:np.float64(0.0009185361341224052) - rollout_corr/log_ppl_abs_diff:np.float64(0.001841111145040486) - rollout_corr/log_ppl_diff_max:np.float64(0.008407264947891235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005116939544677734) - rollout_corr/ppl_ratio:np.float64(1.0009215117897838) - rollout_corr/chi2_token:np.float64(0.0013447413221001625) - rollout_corr/chi2_seq:np.float64(0.33762073097750545) - actor/pg_loss:np.float64(0.00017369771376252174) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007912135890819627) - actor/ppo_kl:np.float64(-9.891378601523115e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4692867696285248) - perf/mfu/actor:np.float64(0.08400188820824843) - perf/max_memory_allocated_gb:np.float64(132.02516555786133) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(101.32085037231445) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03370058909058571 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.03370058909058571 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:408.61328125 - response_length/max:8192.0 - response_length/min:119.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:408.61328125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010697729885578156 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5121872909367085) - timing_s/agent_loop/generate_sequences/max:np.float64(58.620669377967715) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.286885734763928) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.620669377967715) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.46320316195488 - timing_s/reward:0.061744675040245056 - timing_s/old_log_prob:3.311249475926161 - timing_s/adv:0.13611549139022827 - timing_s/update_actor:14.122505748644471 - timing_s/step:83.18165424838662 - timing_s/stop_profile:0.00011662021279335022 - timing_per_token_ms/update_actor:0.0779917148431025 - timing_per_token_ms/gen:0.6258133278710853 - timing_per_token_ms/adv:0.0007516995056811648 - perf/total_num_tokens:181077 - perf/time_per_step:83.18165424838662 - perf/throughput:272.1107821733302 (TaskRunner pid=2347384) Training Progress: 14%|█▍ | 14/100 [14:58<1:50:34, 77.15s/it] (TaskRunner pid=2347384) step:15 - global_seqlen/min:16770 - global_seqlen/max:25955 - global_seqlen/minmax_diff:9185 - global_seqlen/balanced_min:19227 - global_seqlen/balanced_max:19234 - global_seqlen/mean:19231.875 - actor/entropy:0.49017786979675293 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772047519683838 - training/rollout_probs_diff_mean:0.005305732134729624 - training/rollout_probs_diff_std:0.011075073853135109 - training/rollout_actor_probs_pearson_corr:0.9992448687553406 - rollout_corr/rollout_is_mean:0.9997841715812683 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2659669664571993e-05 - rollout_corr/rollout_is_max:1.859199047088623 - rollout_corr/rollout_is_min:0.44539427757263184 - rollout_corr/rollout_is_std:0.03849146142601967 - rollout_corr/rollout_is_eff_sample_size:0.9985199455019984 - rollout_corr/rollout_is_seq_mean:0.9997751116752625 - rollout_corr/rollout_is_seq_std:0.0026260388549417257 - rollout_corr/rollout_is_seq_max:1.008506417274475 - rollout_corr/rollout_is_seq_min:0.9886805415153503 - rollout_corr/rollout_is_seq_max_deviation:0.011319458484649658 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6824355935677886) - rollout_corr/training_log_ppl:np.float64(0.5090422219946049) - rollout_corr/kl:np.float64(0.001040822978014333) - rollout_corr/k3_kl:np.float64(0.0008659341230554674) - rollout_corr/rollout_ppl:np.float64(1.680602803826332) - rollout_corr/rollout_log_ppl:np.float64(0.5080013968981802) - rollout_corr/log_ppl_diff:np.float64(0.001040825096424669) - rollout_corr/log_ppl_abs_diff:np.float64(0.002198683505412191) - rollout_corr/log_ppl_diff_max:np.float64(0.009897172451019287) - rollout_corr/log_ppl_diff_min:np.float64(-0.007486939430236816) - rollout_corr/ppl_ratio:np.float64(1.001044871052727) - rollout_corr/chi2_token:np.float64(0.0013809786178171635) - rollout_corr/chi2_seq:np.float64(0.2935559705365449) - actor/pg_loss:np.float64(-0.00017933628987520933) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008264910120487912) - actor/ppo_kl:np.float64(8.757520348012804e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.48963937163352966) - perf/mfu/actor:np.float64(0.07772259665729145) - perf/max_memory_allocated_gb:np.float64(132.02516555786133) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(102.28882598876953) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:1 - critic/score/mean:0.3671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001990733202546835 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001990733202546835 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:308.55859375 - response_length/max:980.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.55859375 - response_length_non_aborted/max:980.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00030867382884025574 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3679221644997597) - timing_s/agent_loop/generate_sequences/max:np.float64(7.741090871393681) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4445935255935183) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.741090871393681) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:980 - timing_s/gen:14.896692911162972 - timing_s/reward:0.05144878849387169 - timing_s/old_log_prob:3.2346764598041773 - timing_s/adv:0.14543371461331844 - timing_s/update_actor:12.904560782015324 - timing_s/step:31.340551301836967 - timing_s/stop_profile:0.00012101233005523682 - timing_per_token_ms/update_actor:0.08387482228081845 - timing_per_token_ms/gen:0.1885872176724307 - timing_per_token_ms/adv:0.0009452647922610148 - perf/total_num_tokens:153855 - perf/time_per_step:31.340551301836967 - perf/throughput:613.6418857083972 (TaskRunner pid=2347384) Training Progress: 15%|█▌ | 15/100 [15:30<1:29:52, 63.44s/it] (TaskRunner pid=2347384) step:16 - global_seqlen/min:18245 - global_seqlen/max:24988 - global_seqlen/minmax_diff:6743 - global_seqlen/balanced_min:21582 - global_seqlen/balanced_max:21592 - global_seqlen/mean:21588.375 - actor/entropy:0.46256548166275024 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2309231162071228 - training/rollout_probs_diff_mean:0.005185081623494625 - training/rollout_probs_diff_std:0.011045632883906364 - training/rollout_actor_probs_pearson_corr:0.9992262721061707 - rollout_corr/rollout_is_mean:0.9999391436576843 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.128833810071228e-05 - rollout_corr/rollout_is_max:1.7464262247085571 - rollout_corr/rollout_is_min:0.4746478796005249 - rollout_corr/rollout_is_std:0.03808280825614929 - rollout_corr/rollout_is_eff_sample_size:0.9985516812436948 - rollout_corr/rollout_is_seq_mean:0.9998874068260193 - rollout_corr/rollout_is_seq_std:0.002428104868158698 - rollout_corr/rollout_is_seq_max:1.01023268699646 - rollout_corr/rollout_is_seq_min:0.9897691607475281 - rollout_corr/rollout_is_seq_max_deviation:0.010232686996459961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.641688687261194) - rollout_corr/training_log_ppl:np.float64(0.4859840482240543) - rollout_corr/kl:np.float64(0.000994573286405398) - rollout_corr/k3_kl:np.float64(0.0008416437668756771) - rollout_corr/rollout_ppl:np.float64(1.6399632166139781) - rollout_corr/rollout_log_ppl:np.float64(0.4849894735380076) - rollout_corr/log_ppl_diff:np.float64(0.0009945746860466897) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020759932813234627) - rollout_corr/log_ppl_diff_max:np.float64(0.008678853511810303) - rollout_corr/log_ppl_diff_min:np.float64(-0.009289979934692383) - rollout_corr/ppl_ratio:np.float64(1.0009982739575207) - rollout_corr/chi2_token:np.float64(0.001368396682664752) - rollout_corr/chi2_seq:np.float64(0.8329015655908734) - actor/pg_loss:np.float64(-0.0002698972821235657) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005803697106330219) - actor/ppo_kl:np.float64(8.798259023556909e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5201293975114822) - perf/mfu/actor:np.float64(0.08792344894975813) - perf/max_memory_allocated_gb:np.float64(132.02516555786133) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(102.23332977294922) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:1 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0021574837155640125 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0021574837155640125 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:346.04296875 - response_length/max:1053.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:346.04296875 - response_length_non_aborted/max:1053.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.59375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001698676496744156 - timing_s/agent_loop/generate_sequences/min:np.float64(1.689747143536806) - timing_s/agent_loop/generate_sequences/max:np.float64(8.236624287441373) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8304059656438767) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.236624287441373) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:1053 - timing_s/gen:16.13315940834582 - timing_s/reward:0.06064635515213013 - timing_s/old_log_prob:3.0629591830074787 - timing_s/adv:0.17271914333105087 - timing_s/update_actor:12.850557547062635 - timing_s/step:32.3675423245877 - timing_s/stop_profile:0.00011733546853065491 - timing_per_token_ms/update_actor:0.07440669774278191 - timing_per_token_ms/gen:0.1821165566995814 - timing_per_token_ms/adv:0.0010000703117479365 - perf/total_num_tokens:172707 - perf/time_per_step:32.3675423245877 - perf/throughput:666.9760336916464 (TaskRunner pid=2347384) Training Progress: 16%|█▌ | 16/100 [16:02<1:15:44, 54.11s/it] (TaskRunner pid=2347384) step:17 - global_seqlen/min:14579 - global_seqlen/max:23993 - global_seqlen/minmax_diff:9414 - global_seqlen/balanced_min:20135 - global_seqlen/balanced_max:20148 - global_seqlen/mean:20144.625 - actor/entropy:0.47733035683631897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2896254360675812 - training/rollout_probs_diff_mean:0.005261630285531282 - training/rollout_probs_diff_std:0.011144670657813549 - training/rollout_actor_probs_pearson_corr:0.9992367029190063 - rollout_corr/rollout_is_mean:0.9999669790267944 - rollout_corr/rollout_is_ratio_fraction_high:1.1454622836026829e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1454622836026829e-05 - rollout_corr/rollout_is_max:2.0969088077545166 - rollout_corr/rollout_is_min:0.48796799778938293 - rollout_corr/rollout_is_std:0.03874843195080757 - rollout_corr/rollout_is_eff_sample_size:0.9985005722393395 - rollout_corr/rollout_is_seq_mean:0.9999296069145203 - rollout_corr/rollout_is_seq_std:0.0025035692378878593 - rollout_corr/rollout_is_seq_max:1.0074434280395508 - rollout_corr/rollout_is_seq_min:0.989840030670166 - rollout_corr/rollout_is_seq_max_deviation:0.010159969329833984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.689788539428264) - rollout_corr/training_log_ppl:np.float64(0.5136786702787504) - rollout_corr/kl:np.float64(0.0010801191343556837) - rollout_corr/k3_kl:np.float64(0.0009097063799572425) - rollout_corr/rollout_ppl:np.float64(1.6878817342221737) - rollout_corr/rollout_log_ppl:np.float64(0.5125985508784652) - rollout_corr/log_ppl_diff:np.float64(0.0010801194002851844) - rollout_corr/log_ppl_abs_diff:np.float64(0.002021018648520112) - rollout_corr/log_ppl_diff_max:np.float64(0.009488582611083984) - rollout_corr/log_ppl_diff_min:np.float64(-0.006097257137298584) - rollout_corr/ppl_ratio:np.float64(1.001083607552573) - rollout_corr/chi2_token:np.float64(0.0014436279889196157) - rollout_corr/chi2_seq:np.float64(0.4424182176589966) - actor/pg_loss:np.float64(-1.9055441953241825e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010638892772476538) - actor/ppo_kl:np.float64(0.00019094234705008262) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4444914981722832) - perf/mfu/actor:np.float64(0.08021553795910458) - perf/max_memory_allocated_gb:np.float64(132.02516555786133) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(102.20829391479492) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:1 - critic/score/mean:0.23046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.23046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00047966232523322105 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00047966232523322105 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:341.01953125 - response_length/max:1183.0 - response_length/min:132.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:341.01953125 - response_length_non_aborted/max:1183.0 - response_length_non_aborted/min:132.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010183639824390411 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5702809430658817) - timing_s/agent_loop/generate_sequences/max:np.float64(9.146460255607963) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.727873096773692) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.146460255607963) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1183 - timing_s/gen:15.822298798710108 - timing_s/reward:0.05862102285027504 - timing_s/old_log_prob:3.125773211941123 - timing_s/adv:0.14922525733709335 - timing_s/update_actor:13.030285563319921 - timing_s/step:32.271964421495795 - timing_s/stop_profile:0.00011430494487285614 - timing_per_token_ms/update_actor:0.08085460490900129 - timing_per_token_ms/gen:0.18123846002577412 - timing_per_token_ms/adv:0.0009259619956756042 - perf/total_num_tokens:161157 - perf/time_per_step:32.271964421495795 - perf/throughput:624.2144028450284 (TaskRunner pid=2347384) Training Progress: 17%|█▋ | 17/100 [16:35<1:05:47, 47.56s/it] (TaskRunner pid=2347384) step:18 - global_seqlen/min:14050 - global_seqlen/max:31007 - global_seqlen/minmax_diff:16957 - global_seqlen/balanced_min:21074 - global_seqlen/balanced_max:27792 - global_seqlen/mean:21943.25 - actor/entropy:0.41611501574516296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30242347717285156 - training/rollout_probs_diff_mean:0.005012628622353077 - training/rollout_probs_diff_std:0.011453614570200443 - training/rollout_actor_probs_pearson_corr:0.999149739742279 - rollout_corr/rollout_is_mean:1.0000622272491455 - rollout_corr/rollout_is_ratio_fraction_high:1.0493399713595863e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0493399713595863e-05 - rollout_corr/rollout_is_max:2.399968147277832 - rollout_corr/rollout_is_min:0.20653177797794342 - rollout_corr/rollout_is_std:0.038318414241075516 - rollout_corr/rollout_is_eff_sample_size:0.9985340896470543 - rollout_corr/rollout_is_seq_mean:1.0000394582748413 - rollout_corr/rollout_is_seq_std:0.002293739467859268 - rollout_corr/rollout_is_seq_max:1.0059384107589722 - rollout_corr/rollout_is_seq_min:0.9915452599525452 - rollout_corr/rollout_is_seq_max_deviation:0.008454740047454834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6364765437319875) - rollout_corr/training_log_ppl:np.float64(0.48095705287414603) - rollout_corr/kl:np.float64(0.0008645531091753256) - rollout_corr/k3_kl:np.float64(0.000911699380878872) - rollout_corr/rollout_ppl:np.float64(1.6350416275672615) - rollout_corr/rollout_log_ppl:np.float64(0.4800924996234244) - rollout_corr/log_ppl_diff:np.float64(0.0008645532507216558) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019391910027479753) - rollout_corr/log_ppl_diff_max:np.float64(0.011366486549377441) - rollout_corr/log_ppl_diff_min:np.float64(-0.005639404058456421) - rollout_corr/ppl_ratio:np.float64(1.0008679183665663) - rollout_corr/chi2_token:np.float64(0.0019431104883551598) - rollout_corr/chi2_seq:np.float64(0.5791504352819175) - actor/pg_loss:np.float64(0.00032479327637702227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010526327953357395) - actor/ppo_kl:np.float64(4.073885254030074e-05) - actor/pg_clipfrac_lower:np.float64(6.4566115725028794e-06) - actor/grad_norm:np.float64(0.48805930465459824) - perf/mfu/actor:np.float64(0.08156170908553631) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(102.15203094482422) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:1 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04970329999923706 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04970329999923706 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:372.2578125 - response_length/max:8192.0 - response_length/min:103.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:372.2578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:313.46875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018807873129844666 - timing_s/agent_loop/generate_sequences/min:np.float64(1.205347741022706) - timing_s/agent_loop/generate_sequences/max:np.float64(58.50628229603171) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.006803513904742) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.50628229603171) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.4495088905096 - timing_s/reward:0.059897202998399734 - timing_s/old_log_prob:3.325123270973563 - timing_s/adv:0.12080579437315464 - timing_s/update_actor:14.093188527971506 - timing_s/step:83.13446616567671 - timing_s/stop_profile:0.00012601912021636963 - timing_per_token_ms/update_actor:0.08028202595314907 - timing_per_token_ms/gen:0.6867878537903167 - timing_per_token_ms/adv:0.0006881717291943686 - perf/total_num_tokens:175546 - perf/time_per_step:83.13446616567671 - perf/throughput:263.9488892160541 (TaskRunner pid=2347384) Training Progress: 18%|█▊ | 18/100 [17:58<1:19:37, 58.26s/it] (TaskRunner pid=2347384) step:19 - global_seqlen/min:14821 - global_seqlen/max:32064 - global_seqlen/minmax_diff:17243 - global_seqlen/balanced_min:21027 - global_seqlen/balanced_max:25572 - global_seqlen/mean:21620.625 - actor/entropy:0.3882409930229187 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27794426679611206 - training/rollout_probs_diff_mean:0.004868786782026291 - training/rollout_probs_diff_std:0.011396627873182297 - training/rollout_actor_probs_pearson_corr:0.9991189241409302 - rollout_corr/rollout_is_mean:1.000097632408142 - rollout_corr/rollout_is_ratio_fraction_high:1.0341154847992584e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.035048723220825 - rollout_corr/rollout_is_min:0.5708956718444824 - rollout_corr/rollout_is_std:0.03782682865858078 - rollout_corr/rollout_is_eff_sample_size:0.9985715319653053 - rollout_corr/rollout_is_seq_mean:1.0002447366714478 - rollout_corr/rollout_is_seq_std:0.0023739675525575876 - rollout_corr/rollout_is_seq_max:1.0081961154937744 - rollout_corr/rollout_is_seq_min:0.9933203458786011 - rollout_corr/rollout_is_seq_max_deviation:0.008196115493774414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5852337004616857) - rollout_corr/training_log_ppl:np.float64(0.4482002856093459) - rollout_corr/kl:np.float64(0.0009470764403403109) - rollout_corr/k3_kl:np.float64(0.0009331986865674935) - rollout_corr/rollout_ppl:np.float64(1.5837227194570005) - rollout_corr/rollout_log_ppl:np.float64(0.4472532053478062) - rollout_corr/log_ppl_diff:np.float64(0.0009470802615396678) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020817756303586066) - rollout_corr/log_ppl_diff_max:np.float64(0.00791981816291809) - rollout_corr/log_ppl_diff_min:np.float64(-0.007333278656005859) - rollout_corr/ppl_ratio:np.float64(1.0009507073555142) - rollout_corr/chi2_token:np.float64(0.0018556120339781046) - rollout_corr/chi2_seq:np.float64(0.8564345519989729) - actor/pg_loss:np.float64(0.0005588408093899488) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011391409620955528) - actor/ppo_kl:np.float64(0.0003408312764214827) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5193510204553604) - perf/mfu/actor:np.float64(0.06523030261134072) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(137.5371389389038) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:1 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01663116179406643 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01663116179406643 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:377.73828125 - response_length/max:6077.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:377.73828125 - response_length_non_aborted/max:6077.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.90625 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.726174175739288e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7443513739854097) - timing_s/agent_loop/generate_sequences/max:np.float64(36.85969475097954) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.116641358414199) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(36.85969475097954) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:6077 - timing_s/gen:44.36828749626875 - timing_s/reward:0.06108302250504494 - timing_s/old_log_prob:3.1765850484371185 - timing_s/adv:0.11546890065073967 - timing_s/update_actor:17.13972050882876 - timing_s/step:64.9494182318449 - timing_s/stop_profile:0.0001245979219675064 - timing_per_token_ms/update_actor:0.09909357678622126 - timing_per_token_ms/gen:0.45881932447719 - timing_per_token_ms/adv:0.000667585353399472 - perf/total_num_tokens:172965 - perf/time_per_step:64.9494182318449 - perf/throughput:332.88404405444453 (TaskRunner pid=2347384) Training Progress: 19%|█▉ | 19/100 [19:03<1:21:23, 60.28s/it] (TaskRunner pid=2347384) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2347384) validation generation end (TaskRunner pid=2347384) [prompt] system (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) To determine the folding stability score of a protein sequence, one would typically use computational tools or algorithms that predict protein folding stability based on factors such as amino acid composition, secondary structure propensity, and energy minimization. However, without access to such tools or specific data for the given sequence, it is not possible to calculate the exact folding stability score. (TaskRunner pid=2347384) (TaskRunner pid=2347384) In the absence of specific computational results, we must infer the most reasonable answer based on typical values for folding stability scores. Folding stability scores can vary widely, but values above 0 generally indicate a more stable structure, while values below 0 suggest instability. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Among the given options, the highest value is 2.49 (Option C), which is a strong indicator of high folding stability. This is a plausible value for a protein sequence that may have favorable interactions and a well-defined structure. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Therefore, based on the reasoning that a higher value indicates greater folding stability, the most reasonable answer is Option C. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) C (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 0.0 (TaskRunner pid=2347384) [acc] 0.0 (TaskRunner pid=2347384) [pred] C (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2347384) step:20 - global_seqlen/min:14273 - global_seqlen/max:36229 - global_seqlen/minmax_diff:21956 - global_seqlen/balanced_min:21364 - global_seqlen/balanced_max:28151 - global_seqlen/mean:22222.625 - actor/entropy:0.4128585159778595 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2921063005924225 - training/rollout_probs_diff_mean:0.00500514218583703 - training/rollout_probs_diff_std:0.01128156017512083 - training/rollout_actor_probs_pearson_corr:0.9991604685783386 - rollout_corr/rollout_is_mean:1.0001616477966309 - rollout_corr/rollout_is_ratio_fraction_high:2.049243266810663e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.636357069015503 - rollout_corr/rollout_is_min:0.5685460567474365 - rollout_corr/rollout_is_std:0.03822029009461403 - rollout_corr/rollout_is_eff_sample_size:0.9985419344716855 - rollout_corr/rollout_is_seq_mean:1.0001330375671387 - rollout_corr/rollout_is_seq_std:0.002333165379241109 - rollout_corr/rollout_is_seq_max:1.0072177648544312 - rollout_corr/rollout_is_seq_min:0.9936760663986206 - rollout_corr/rollout_is_seq_max_deviation:0.007217764854431152 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6222130814567208) - rollout_corr/training_log_ppl:np.float64(0.4735873161116615) - rollout_corr/kl:np.float64(0.0007982173941947224) - rollout_corr/k3_kl:np.float64(0.0009112951114502721) - rollout_corr/rollout_ppl:np.float64(1.620887991040945) - rollout_corr/rollout_log_ppl:np.float64(0.4727890971262241) - rollout_corr/log_ppl_diff:np.float64(0.0007982189854374155) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018730945739662275) - rollout_corr/log_ppl_diff_max:np.float64(0.009568095207214355) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058307647705078125) - rollout_corr/ppl_ratio:np.float64(1.0008012137841433) - rollout_corr/chi2_token:np.float64(0.002104806946590543) - rollout_corr/chi2_seq:np.float64(0.6861413994338363) - actor/pg_loss:np.float64(8.781743235886097e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000662013314467913) - actor/ppo_kl:np.float64(8.268784525711226e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5173042640089989) - perf/mfu/actor:np.float64(0.08140710296600062) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(103.83859252929688) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.29875) - val-aux/sciknoweval/reward/std@16:np.float64(0.3468667593557151) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.44404000000000005) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.3538906206864504) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.153) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2321419100449532) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.30033999999999994) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3456856053262063) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.59438) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.31099638559049564) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.07102) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11713290401712242) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.2959399999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.296988236418229) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.72616) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.22979892018300924) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.035379999999999995) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.045051659239938216) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.28952) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.24205058730129803) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.82064) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.1367142800110298) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.02352) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.014194645288274138) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.28737999999999997) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.18809265792622348) - val-aux/sciknoweval/score/mean@16:np.float64(0.29875) - val-aux/sciknoweval/score/std@16:np.float64(0.3468667593557151) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.44404000000000005) - val-aux/sciknoweval/score/best@2/std:np.float64(0.3538906206864504) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.153) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2321419100449532) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.30033999999999994) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3456856053262063) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.59438) - val-aux/sciknoweval/score/best@4/std:np.float64(0.31099638559049564) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.07102) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11713290401712242) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.2959399999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.296988236418229) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.72616) - val-aux/sciknoweval/score/best@8/std:np.float64(0.22979892018300924) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.035379999999999995) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.045051659239938216) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.28952) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.24205058730129803) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.82064) - val-aux/sciknoweval/score/best@16/std:np.float64(0.1367142800110298) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.02352) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.014194645288274138) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.28737999999999997) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.18809265792622348) - val-core/sciknoweval/acc/mean@16:np.float64(0.29875) - val-aux/sciknoweval/acc/std@16:np.float64(0.3468667593557151) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.44404000000000005) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.3538906206864504) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.153) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2321419100449532) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.30033999999999994) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3456856053262063) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.59438) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.31099638559049564) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.07102) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11713290401712242) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.2959399999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.296988236418229) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.72616) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.22979892018300924) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.035379999999999995) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.045051659239938216) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.28952) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.24205058730129803) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.82064) - val-core/sciknoweval/acc/best@16/std:np.float64(0.1367142800110298) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.02352) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.014194645288274138) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.28737999999999997) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.18809265792622348) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.96875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.06419532354634003) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99274) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.03086109412532124) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.94476) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.07737425766671907) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.96902) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.06375376155268878) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9994000000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.006930407004094272) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.90996) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08132162866379287) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9865200000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.04278061614992783) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.87322) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0690276502853235) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99558) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.02343612776778892) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8452) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04535055696097805) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9996200000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004769012360302181) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:1 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0028561328072100878 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0028561328072100878 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:381.23828125 - response_length/max:8192.0 - response_length/min:141.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:381.23828125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:313.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011074729263782501 - timing_s/agent_loop/generate_sequences/min:np.float64(1.769131125882268) - timing_s/agent_loop/generate_sequences/max:np.float64(61.283475656062365) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.212042792241846) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(61.283475656062365) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:68.50083152763546 - timing_s/reward:0.05965007469058037 - timing_s/old_log_prob:3.427032060921192 - timing_s/adv:0.16247421689331532 - timing_s/update_actor:14.29509094916284 - timing_s/step:86.53164429403841 - timing_s/testing:94.49010137841105 - timing_s/save_checkpoint:12.844500087201595 - timing_s/stop_profile:0.00029869936406612396 - timing_per_token_ms/update_actor:0.08040842918626198 - timing_per_token_ms/gen:0.7018743560522911 - timing_per_token_ms/adv:0.0009139009055709852 - perf/total_num_tokens:177781 - perf/time_per_step:86.53164429403841 - perf/throughput:256.8150089057192 (TaskRunner pid=2347384) Training Progress: 20%|██ | 20/100 [22:17<2:13:53, 100.42s/it] (TaskRunner pid=2347384) step:21 - global_seqlen/min:13775 - global_seqlen/max:27026 - global_seqlen/minmax_diff:13251 - global_seqlen/balanced_min:19708 - global_seqlen/balanced_max:19740 - global_seqlen/mean:19720.0 - actor/entropy:0.43985289335250854 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18748968839645386 - training/rollout_probs_diff_mean:0.005339562427252531 - training/rollout_probs_diff_std:0.01158396527171135 - training/rollout_actor_probs_pearson_corr:0.9991340637207031 - rollout_corr/rollout_is_mean:0.9999651312828064 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9055321216583252 - rollout_corr/rollout_is_min:0.5899193286895752 - rollout_corr/rollout_is_std:0.03874843195080757 - rollout_corr/rollout_is_eff_sample_size:0.9985005722393395 - rollout_corr/rollout_is_seq_mean:1.0000044107437134 - rollout_corr/rollout_is_seq_std:0.002407708903774619 - rollout_corr/rollout_is_seq_max:1.008156418800354 - rollout_corr/rollout_is_seq_min:0.9918157458305359 - rollout_corr/rollout_is_seq_max_deviation:0.008184254169464111 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5938744340091944) - rollout_corr/training_log_ppl:np.float64(0.4572917863843031) - rollout_corr/kl:np.float64(0.0008235171446138168) - rollout_corr/k3_kl:np.float64(0.0009057562956513721) - rollout_corr/rollout_ppl:np.float64(1.5925458329729736) - rollout_corr/rollout_log_ppl:np.float64(0.4564682655618526) - rollout_corr/log_ppl_diff:np.float64(0.0008235208224505186) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020844079554080963) - rollout_corr/log_ppl_diff_max:np.float64(0.009804129600524902) - rollout_corr/log_ppl_diff_min:np.float64(-0.007019698619842529) - rollout_corr/ppl_ratio:np.float64(1.0008272039704025) - rollout_corr/chi2_token:np.float64(0.001998132560402155) - rollout_corr/chi2_seq:np.float64(0.7063760042656213) - actor/pg_loss:np.float64(0.0004940113285556436) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008618902672878903) - actor/ppo_kl:np.float64(2.2110816431109015e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4871308505535126) - perf/mfu/actor:np.float64(0.07992006169026471) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(114.48110771179199) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:1 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01268029399216175 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01268029399216175 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.28125 - response_length/max:1239.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.28125 - response_length_non_aborted/max:1239.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.96875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.81531023979187e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9318560305982828) - timing_s/agent_loop/generate_sequences/max:np.float64(9.249703470617533) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8056380497218925) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.249703470617533) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1239 - timing_s/gen:15.968066735193133 - timing_s/reward:0.05399642139673233 - timing_s/old_log_prob:3.154765773564577 - timing_s/adv:0.13141521997749805 - timing_s/update_actor:12.992189822718501 - timing_s/step:32.40803734585643 - timing_s/stop_profile:0.0001332201063632965 - timing_per_token_ms/update_actor:0.08235414441378361 - timing_per_token_ms/gen:0.18603861887399958 - timing_per_token_ms/adv:0.0008330072260236945 - perf/total_num_tokens:157760 - perf/time_per_step:32.40803734585643 - perf/throughput:608.4910292329482 (TaskRunner pid=2347384) Training Progress: 21%|██ | 21/100 [22:49<1:45:21, 80.02s/it] (TaskRunner pid=2347384) step:22 - global_seqlen/min:18700 - global_seqlen/max:33221 - global_seqlen/minmax_diff:14521 - global_seqlen/balanced_min:23858 - global_seqlen/balanced_max:26558 - global_seqlen/mean:24255.0 - actor/entropy:0.41630059480667114 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15547972917556763 - training/rollout_probs_diff_mean:0.004655183758586645 - training/rollout_probs_diff_std:0.010528627783060074 - training/rollout_actor_probs_pearson_corr:0.9992711544036865 - rollout_corr/rollout_is_mean:1.00006103515625 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.499924090050627e-06 - rollout_corr/rollout_is_max:1.9030859470367432 - rollout_corr/rollout_is_min:0.49738994240760803 - rollout_corr/rollout_is_std:0.036124564707279205 - rollout_corr/rollout_is_eff_sample_size:0.998696835570079 - rollout_corr/rollout_is_seq_mean:1.0000958442687988 - rollout_corr/rollout_is_seq_std:0.002197669818997383 - rollout_corr/rollout_is_seq_max:1.0082387924194336 - rollout_corr/rollout_is_seq_min:0.9935979843139648 - rollout_corr/rollout_is_seq_max_deviation:0.008238792419433594 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6229639295488596) - rollout_corr/training_log_ppl:np.float64(0.47301355128729483) - rollout_corr/kl:np.float64(0.0008646119430739674) - rollout_corr/k3_kl:np.float64(0.0008283676010023555) - rollout_corr/rollout_ppl:np.float64(1.6215176028199494) - rollout_corr/rollout_log_ppl:np.float64(0.4721489406147157) - rollout_corr/log_ppl_diff:np.float64(0.0008646106725791469) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017696021968731657) - rollout_corr/log_ppl_diff_max:np.float64(0.007478058338165283) - rollout_corr/log_ppl_diff_min:np.float64(-0.006179004907608032) - rollout_corr/ppl_ratio:np.float64(1.0008673779666424) - rollout_corr/chi2_token:np.float64(0.0016075125895440578) - rollout_corr/chi2_seq:np.float64(0.4220045474357903) - actor/pg_loss:np.float64(0.00016741419676691294) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008953819645967087) - actor/ppo_kl:np.float64(0.000198875633639517) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5035498887300491) - perf/mfu/actor:np.float64(0.09435049589776996) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(108.81607818603516) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:1 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.018244603648781776 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.018244603648781776 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:411.1875 - response_length/max:4363.0 - response_length/min:158.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:411.1875 - response_length_non_aborted/max:4363.0 - response_length_non_aborted/min:158.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012529082596302032 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9556401576846838) - timing_s/agent_loop/generate_sequences/max:np.float64(27.262505570426583) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.428927275461319) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.262505570426583) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:4363 - timing_s/gen:33.83263132162392 - timing_s/reward:0.06293242238461971 - timing_s/old_log_prob:3.2037085443735123 - timing_s/adv:0.1506052128970623 - timing_s/update_actor:13.474846798926592 - timing_s/step:50.81742474809289 - timing_s/stop_profile:0.00014409422874450684 - timing_per_token_ms/update_actor:0.06944365491098017 - timing_per_token_ms/gen:0.3214074262960169 - timing_per_token_ms/adv:0.0007761554983357158 - perf/total_num_tokens:194040 - perf/time_per_step:50.81742474809289 - perf/throughput:477.29691380927875 (TaskRunner pid=2347384) Training Progress: 22%|██▏ | 22/100 [23:40<1:32:38, 71.27s/it] (TaskRunner pid=2347384) step:23 - global_seqlen/min:18550 - global_seqlen/max:35535 - global_seqlen/minmax_diff:16985 - global_seqlen/balanced_min:23395 - global_seqlen/balanced_max:29703 - global_seqlen/mean:24808.875 - actor/entropy:0.3801974058151245 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553989887237549 - training/rollout_probs_diff_mean:0.004351600538939238 - training/rollout_probs_diff_std:0.010345465503633022 - training/rollout_actor_probs_pearson_corr:0.9992690086364746 - rollout_corr/rollout_is_mean:0.9998360276222229 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.767262578010559 - rollout_corr/rollout_is_min:0.5374431610107422 - rollout_corr/rollout_is_std:0.03489076346158981 - rollout_corr/rollout_is_eff_sample_size:0.9987837581527793 - rollout_corr/rollout_is_seq_mean:0.9997899532318115 - rollout_corr/rollout_is_seq_std:0.0022617734503000975 - rollout_corr/rollout_is_seq_max:1.0066187381744385 - rollout_corr/rollout_is_seq_min:0.9941260814666748 - rollout_corr/rollout_is_seq_max_deviation:0.0066187381744384766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6270498312078416) - rollout_corr/training_log_ppl:np.float64(0.4729785490198992) - rollout_corr/kl:np.float64(0.0009565757269722752) - rollout_corr/k3_kl:np.float64(0.000844647590980685) - rollout_corr/rollout_ppl:np.float64(1.6254838155582547) - rollout_corr/rollout_log_ppl:np.float64(0.47202197147271363) - rollout_corr/log_ppl_diff:np.float64(0.0009565775471855886) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019879597311955877) - rollout_corr/log_ppl_diff_max:np.float64(0.0070632100105285645) - rollout_corr/log_ppl_diff_min:np.float64(-0.007271766662597656) - rollout_corr/ppl_ratio:np.float64(1.0009599297773093) - rollout_corr/chi2_token:np.float64(0.0015118732117116451) - rollout_corr/chi2_seq:np.float64(0.8003104475792497) - actor/pg_loss:np.float64(0.00017989519983530045) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005136319407483825) - actor/ppo_kl:np.float64(-2.9271853385637314e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.41223112493753433) - perf/mfu/actor:np.float64(0.09283550601107665) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(105.54674911499023) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.3515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.05055713653564453 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.05055713653564453 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.30859375 - response_length/max:8192.0 - response_length/min:159.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:440.30859375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:334.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015219859778881073 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0618123151361942) - timing_s/agent_loop/generate_sequences/max:np.float64(59.26402160525322) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.621393935289234) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.26402160525322) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:66.566022856161 - timing_s/reward:0.06328636221587658 - timing_s/old_log_prob:3.5292106363922358 - timing_s/adv:0.1336524598300457 - timing_s/update_actor:14.019571641460061 - timing_s/step:84.3989474978298 - timing_s/stop_profile:0.00012223795056343079 - timing_per_token_ms/update_actor:0.07063788483687825 - timing_per_token_ms/gen:0.5905483800970643 - timing_per_token_ms/adv:0.0006734105225954708 - perf/total_num_tokens:198471 - perf/time_per_step:84.3989474978298 - perf/throughput:293.9476822342829 (TaskRunner pid=2347384) Training Progress: 23%|██▎ | 23/100 [25:05<1:36:32, 75.22s/it] (TaskRunner pid=2347384) step:24 - global_seqlen/min:14381 - global_seqlen/max:26554 - global_seqlen/minmax_diff:12173 - global_seqlen/balanced_min:19808 - global_seqlen/balanced_max:20388 - global_seqlen/mean:19882.75 - actor/entropy:0.4750767946243286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23108524084091187 - training/rollout_probs_diff_mean:0.005285514052957296 - training/rollout_probs_diff_std:0.011154617182910442 - training/rollout_actor_probs_pearson_corr:0.9992302656173706 - rollout_corr/rollout_is_mean:1.0001184940338135 - rollout_corr/rollout_is_ratio_fraction_high:2.3042006432660855e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1521003216330428e-05 - rollout_corr/rollout_is_max:4.353000164031982 - rollout_corr/rollout_is_min:0.41086792945861816 - rollout_corr/rollout_is_std:0.03882833942770958 - rollout_corr/rollout_is_eff_sample_size:0.998494986223482 - rollout_corr/rollout_is_seq_mean:1.000071406364441 - rollout_corr/rollout_is_seq_std:0.0025317457038909197 - rollout_corr/rollout_is_seq_max:1.0152056217193604 - rollout_corr/rollout_is_seq_min:0.9927321672439575 - rollout_corr/rollout_is_seq_max_deviation:0.015205621719360352 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6779361688531935) - rollout_corr/training_log_ppl:np.float64(0.5069089613389224) - rollout_corr/kl:np.float64(0.0009095346787901804) - rollout_corr/k3_kl:np.float64(0.0009307687192858793) - rollout_corr/rollout_ppl:np.float64(1.6764288442209363) - rollout_corr/rollout_log_ppl:np.float64(0.505999423970934) - rollout_corr/log_ppl_diff:np.float64(0.0009095373679883778) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023033165489323437) - rollout_corr/log_ppl_diff_max:np.float64(0.009229183197021484) - rollout_corr/log_ppl_diff_min:np.float64(-0.008145451545715332) - rollout_corr/ppl_ratio:np.float64(1.0009138768073171) - rollout_corr/chi2_token:np.float64(0.002055418211966753) - rollout_corr/chi2_seq:np.float64(1.0026492818724364) - actor/pg_loss:np.float64(-0.00018458650447428226) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005724298384848225) - actor/ppo_kl:np.float64(0.00015153375217558107) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4611205756664276) - perf/mfu/actor:np.float64(0.0801533090041476) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(105.51220703125) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.37890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.020999908447265625 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.020999908447265625 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:339.0546875 - response_length/max:1875.0 - response_length/min:158.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.0546875 - response_length_non_aborted/max:1875.0 - response_length_non_aborted/min:158.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013718008995056152 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0218928940594196) - timing_s/agent_loop/generate_sequences/max:np.float64(12.864371560513973) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8413179576964467) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.864371560513973) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:1875 - timing_s/gen:19.719263887032866 - timing_s/reward:0.059100065380334854 - timing_s/old_log_prob:3.030965318903327 - timing_s/adv:0.12511055171489716 - timing_s/update_actor:12.712991928681731 - timing_s/step:35.73627816140652 - timing_s/stop_profile:0.0001299344003200531 - timing_per_token_ms/update_actor:0.07992475845067792 - timing_per_token_ms/gen:0.22718569422144366 - timing_per_token_ms/adv:0.000786552109962764 - perf/total_num_tokens:159062 - perf/time_per_step:35.73627816140652 - perf/throughput:556.3743910375207 (TaskRunner pid=2347384) Training Progress: 24%|██▍ | 24/100 [25:40<1:20:17, 63.39s/it] (TaskRunner pid=2347384) step:25 - global_seqlen/min:20990 - global_seqlen/max:30984 - global_seqlen/minmax_diff:9994 - global_seqlen/balanced_min:24812 - global_seqlen/balanced_max:27350 - global_seqlen/mean:25437.0 - actor/entropy:0.4363953769207001 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.531131386756897 - training/rollout_probs_diff_mean:0.004387347958981991 - training/rollout_probs_diff_std:0.010115292854607105 - training/rollout_actor_probs_pearson_corr:0.9993716478347778 - rollout_corr/rollout_is_mean:1.0000243186950684 - rollout_corr/rollout_is_ratio_fraction_high:1.8080567315337248e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.712085006351117e-05 - rollout_corr/rollout_is_max:2.0666491985321045 - rollout_corr/rollout_is_min:0.35002005100250244 - rollout_corr/rollout_is_std:0.03474181890487671 - rollout_corr/rollout_is_eff_sample_size:0.9987945799437343 - rollout_corr/rollout_is_seq_mean:0.9999772310256958 - rollout_corr/rollout_is_seq_std:0.00208827736787498 - rollout_corr/rollout_is_seq_max:1.0061976909637451 - rollout_corr/rollout_is_seq_min:0.9945194721221924 - rollout_corr/rollout_is_seq_max_deviation:0.006197690963745117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6769645470194519) - rollout_corr/training_log_ppl:np.float64(0.5054974173108349) - rollout_corr/kl:np.float64(0.0008113924816184337) - rollout_corr/k3_kl:np.float64(0.0007960972822615986) - rollout_corr/rollout_ppl:np.float64(1.6756097814068198) - rollout_corr/rollout_log_ppl:np.float64(0.5046860251750331) - rollout_corr/log_ppl_diff:np.float64(0.0008113921358017251) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017934010393219069) - rollout_corr/log_ppl_diff_max:np.float64(0.00806504487991333) - rollout_corr/log_ppl_diff_min:np.float64(-0.005534529685974121) - rollout_corr/ppl_ratio:np.float64(1.000814215745777) - rollout_corr/chi2_token:np.float64(0.0015990398824214935) - rollout_corr/chi2_seq:np.float64(0.4457212893757969) - actor/pg_loss:np.float64(0.00010927650146186352) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006512334759349869) - actor/ppo_kl:np.float64(8.092571112894298e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.49562444537878036) - perf/mfu/actor:np.float64(0.09939930232459235) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(105.31881713867188) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004961985629051924 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004961985629051924 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:432.09375 - response_length/max:4253.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:432.09375 - response_length_non_aborted/max:4253.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.8125 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010738149285316467 - timing_s/agent_loop/generate_sequences/min:np.float64(1.915298454463482) - timing_s/agent_loop/generate_sequences/max:np.float64(27.074555112048984) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.673549318940786) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.074555112048984) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:4253 - timing_s/gen:33.73908548243344 - timing_s/reward:0.06454445607960224 - timing_s/old_log_prob:3.16552746668458 - timing_s/adv:0.10949888452887535 - timing_s/update_actor:13.26191484555602 - timing_s/step:50.427627546712756 - timing_s/stop_profile:0.00013010576367378235 - timing_per_token_ms/update_actor:0.06517039571075608 - timing_per_token_ms/gen:0.30501089790295655 - timing_per_token_ms/adv:0.0005380886333337037 - perf/total_num_tokens:203496 - perf/time_per_step:50.427627546712756 - perf/throughput:504.4258720368488 (TaskRunner pid=2347384) Training Progress: 25%|██▌ | 25/100 [26:31<1:14:24, 59.52s/it] (TaskRunner pid=2347384) step:26 - global_seqlen/min:14899 - global_seqlen/max:25252 - global_seqlen/minmax_diff:10353 - global_seqlen/balanced_min:21642 - global_seqlen/balanced_max:24852 - global_seqlen/mean:22062.375 - actor/entropy:0.4528347849845886 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28152310848236084 - training/rollout_probs_diff_mean:0.00456203892827034 - training/rollout_probs_diff_std:0.010279463604092598 - training/rollout_actor_probs_pearson_corr:0.999377965927124 - rollout_corr/rollout_is_mean:1.0002533197402954 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6443817615509033 - rollout_corr/rollout_is_min:0.5422587990760803 - rollout_corr/rollout_is_std:0.035841308534145355 - rollout_corr/rollout_is_eff_sample_size:0.9987176432924386 - rollout_corr/rollout_is_seq_mean:1.000220775604248 - rollout_corr/rollout_is_seq_std:0.002359299920499325 - rollout_corr/rollout_is_seq_max:1.0104557275772095 - rollout_corr/rollout_is_seq_min:0.9919562339782715 - rollout_corr/rollout_is_seq_max_deviation:0.010455727577209473 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.678601372987032) - rollout_corr/training_log_ppl:np.float64(0.504833974307985) - rollout_corr/kl:np.float64(0.0005614295998803698) - rollout_corr/k3_kl:np.float64(0.0007734230811990983) - rollout_corr/rollout_ppl:np.float64(1.677660213317722) - rollout_corr/rollout_log_ppl:np.float64(0.5042725394159788) - rollout_corr/log_ppl_diff:np.float64(0.0005614348920062184) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016854319255799055) - rollout_corr/log_ppl_diff_max:np.float64(0.008096307516098022) - rollout_corr/log_ppl_diff_min:np.float64(-0.009202778339385986) - rollout_corr/ppl_ratio:np.float64(1.0005641190800816) - rollout_corr/chi2_token:np.float64(0.0019795263651758432) - rollout_corr/chi2_seq:np.float64(0.8771327515132725) - actor/pg_loss:np.float64(-0.00016296107787638903) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005223186910825461) - actor/ppo_kl:np.float64(4.247915952193537e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3661017343401909) - perf/mfu/actor:np.float64(0.08498147246204509) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(105.42106628417969) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0021807325538247824 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0021807325538247824 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:380.41796875 - response_length/max:4567.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:380.41796875 - response_length_non_aborted/max:4567.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013206899166107178 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4601224940270185) - timing_s/agent_loop/generate_sequences/max:np.float64(28.66743589565158) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.032827958923008) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.66743589565158) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:4567 - timing_s/gen:35.79071536846459 - timing_s/reward:0.0627571102231741 - timing_s/old_log_prob:3.1867961399257183 - timing_s/adv:0.16336078941822052 - timing_s/update_actor:13.53571617230773 - timing_s/step:52.82645891420543 - timing_s/stop_profile:4.873797297477722e-05 - timing_per_token_ms/update_actor:0.07669004454590525 - timing_per_token_ms/gen:0.3675101950821423 - timing_per_token_ms/adv:0.0009255621245345329 - perf/total_num_tokens:176499 - perf/time_per_step:52.82645891420543 - perf/throughput:417.6387259996195 (TaskRunner pid=2347384) Training Progress: 26%|██▌ | 26/100 [27:24<1:10:56, 57.52s/it] (TaskRunner pid=2347384) step:27 - global_seqlen/min:15705 - global_seqlen/max:30289 - global_seqlen/minmax_diff:14584 - global_seqlen/balanced_min:22392 - global_seqlen/balanced_max:25878 - global_seqlen/mean:23112.375 - actor/entropy:0.4682733118534088 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5454909205436707 - training/rollout_probs_diff_mean:0.004588646814227104 - training/rollout_probs_diff_std:0.010231602936983109 - training/rollout_actor_probs_pearson_corr:0.9993829727172852 - rollout_corr/rollout_is_mean:1.0001314878463745 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9136008631903678e-05 - rollout_corr/rollout_is_max:1.7160686254501343 - rollout_corr/rollout_is_min:0.2300301492214203 - rollout_corr/rollout_is_std:0.035254448652267456 - rollout_corr/rollout_is_eff_sample_size:0.9987590235111645 - rollout_corr/rollout_is_seq_mean:1.000084400177002 - rollout_corr/rollout_is_seq_std:0.002220951719209552 - rollout_corr/rollout_is_seq_max:1.0060702562332153 - rollout_corr/rollout_is_seq_min:0.9901464581489563 - rollout_corr/rollout_is_seq_max_deviation:0.009853541851043701 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7234379891306162) - rollout_corr/training_log_ppl:np.float64(0.5305853445461253) - rollout_corr/kl:np.float64(0.0008600184118137122) - rollout_corr/k3_kl:np.float64(0.0007880550206209591) - rollout_corr/rollout_ppl:np.float64(1.721922215539962) - rollout_corr/rollout_log_ppl:np.float64(0.5297253206954338) - rollout_corr/log_ppl_diff:np.float64(0.0008600238506915048) - rollout_corr/log_ppl_abs_diff:np.float64(0.001834972543292679) - rollout_corr/log_ppl_diff_max:np.float64(0.011312544345855713) - rollout_corr/log_ppl_diff_min:np.float64(-0.005065798759460449) - rollout_corr/ppl_ratio:np.float64(1.0008631183300167) - rollout_corr/chi2_token:np.float64(0.0014241689350456) - rollout_corr/chi2_seq:np.float64(0.6947200584691018) - actor/pg_loss:np.float64(-0.0004229054320603609) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007544087525275245) - actor/ppo_kl:np.float64(0.00022471997308315395) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4978668913245201) - perf/mfu/actor:np.float64(0.0900470521876152) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(105.44089126586914) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.019456537440419197 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.019456537440419197 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:408.26171875 - response_length/max:4945.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:408.26171875 - response_length_non_aborted/max:4945.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.0 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.715821266174316e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7510246485471725) - timing_s/agent_loop/generate_sequences/max:np.float64(30.639742696657777) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.416832888658973) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(30.639742696657777) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:4945 - timing_s/gen:37.67792792059481 - timing_s/reward:0.0616065114736557 - timing_s/old_log_prob:3.1845420319586992 - timing_s/adv:0.15087821520864964 - timing_s/update_actor:13.340245062485337 - timing_s/step:54.504173554480076 - timing_s/stop_profile:0.00014214962720870972 - timing_per_token_ms/update_actor:0.0721488221271361 - timing_per_token_ms/gen:0.3605025873854931 - timing_per_token_ms/adv:0.0008160034138023982 - perf/total_num_tokens:184899 - perf/time_per_step:54.504173554480076 - perf/throughput:424.04780208065796 (TaskRunner pid=2347384) Training Progress: 27%|██▋ | 27/100 [28:18<1:08:54, 56.63s/it] (TaskRunner pid=2347384) step:28 - global_seqlen/min:18610 - global_seqlen/max:27723 - global_seqlen/minmax_diff:9113 - global_seqlen/balanced_min:23188 - global_seqlen/balanced_max:24923 - global_seqlen/mean:23408.125 - actor/entropy:0.47311460971832275 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27339667081832886 - training/rollout_probs_diff_mean:0.004760510753840208 - training/rollout_probs_diff_std:0.010186607018113136 - training/rollout_actor_probs_pearson_corr:0.999364972114563 - rollout_corr/rollout_is_mean:1.0001866817474365 - rollout_corr/rollout_is_ratio_fraction_high:1.0103357453772333e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2587881088256836 - rollout_corr/rollout_is_min:0.6093446016311646 - rollout_corr/rollout_is_std:0.03614930436015129 - rollout_corr/rollout_is_eff_sample_size:0.9986954087865968 - rollout_corr/rollout_is_seq_mean:1.0002046823501587 - rollout_corr/rollout_is_seq_std:0.0023463431280106306 - rollout_corr/rollout_is_seq_max:1.007077932357788 - rollout_corr/rollout_is_seq_min:0.9923440217971802 - rollout_corr/rollout_is_seq_max_deviation:0.007655978202819824 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7137529752217233) - rollout_corr/training_log_ppl:np.float64(0.5269968162901932) - rollout_corr/kl:np.float64(0.0008623486197407715) - rollout_corr/k3_kl:np.float64(0.0009063949795518056) - rollout_corr/rollout_ppl:np.float64(1.7122624972835183) - rollout_corr/rollout_log_ppl:np.float64(0.5261344668688253) - rollout_corr/log_ppl_diff:np.float64(0.0008623494213679805) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020449918374652043) - rollout_corr/log_ppl_diff_max:np.float64(0.012380421161651611) - rollout_corr/log_ppl_diff_min:np.float64(-0.005613207817077637) - rollout_corr/ppl_ratio:np.float64(1.0008662319742143) - rollout_corr/chi2_token:np.float64(0.0018957597203552723) - rollout_corr/chi2_seq:np.float64(0.56739765801467) - actor/pg_loss:np.float64(-0.00012453715316951275) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001367721694350621) - actor/ppo_kl:np.float64(0.0003233809929383824) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4850858673453331) - perf/mfu/actor:np.float64(0.09199498753655419) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(105.3936653137207) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.3671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0176707711070776 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0176707711070776 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:386.62890625 - response_length/max:3352.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:386.62890625 - response_length_non_aborted/max:3352.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:344.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001003313809633255 - timing_s/agent_loop/generate_sequences/min:np.float64(1.799130529165268) - timing_s/agent_loop/generate_sequences/max:np.float64(21.598102193325758) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.267342348233797) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.598102193325758) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:3352 - timing_s/gen:28.37084392271936 - timing_s/reward:0.062208110466599464 - timing_s/old_log_prob:3.1338888853788376 - timing_s/adv:0.12042229063808918 - timing_s/update_actor:13.231987729668617 - timing_s/step:45.007485903799534 - timing_s/stop_profile:0.0001107528805732727 - timing_per_token_ms/update_actor:0.07065916070631788 - timing_per_token_ms/gen:0.28664077434878166 - timing_per_token_ms/adv:0.0006430581829924929 - perf/total_num_tokens:187265 - perf/time_per_step:45.007485903799534 - perf/throughput:520.094036135084 (TaskRunner pid=2347384) Training Progress: 28%|██▊ | 28/100 [29:03<1:03:47, 53.16s/it] (TaskRunner pid=2347384) step:29 - global_seqlen/min:13908 - global_seqlen/max:22396 - global_seqlen/minmax_diff:8488 - global_seqlen/balanced_min:19396 - global_seqlen/balanced_max:19400 - global_seqlen/mean:19398.625 - actor/entropy:0.5204939246177673 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18593913316726685 - training/rollout_probs_diff_mean:0.0054834987968206406 - training/rollout_probs_diff_std:0.010920548811554909 - training/rollout_actor_probs_pearson_corr:0.9992918968200684 - rollout_corr/rollout_is_mean:0.9999595284461975 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2546894140541553e-05 - rollout_corr/rollout_is_max:1.8008973598480225 - rollout_corr/rollout_is_min:0.49691760540008545 - rollout_corr/rollout_is_std:0.03862517699599266 - rollout_corr/rollout_is_eff_sample_size:0.9985104370582881 - rollout_corr/rollout_is_seq_mean:0.9999402761459351 - rollout_corr/rollout_is_seq_std:0.002590069780126214 - rollout_corr/rollout_is_seq_max:1.006123661994934 - rollout_corr/rollout_is_seq_min:0.9924188256263733 - rollout_corr/rollout_is_seq_max_deviation:0.007581174373626709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7231960976496339) - rollout_corr/training_log_ppl:np.float64(0.5328172956360504) - rollout_corr/kl:np.float64(0.0008222202101677567) - rollout_corr/k3_kl:np.float64(0.0008401776635764691) - rollout_corr/rollout_ppl:np.float64(1.7217592736706138) - rollout_corr/rollout_log_ppl:np.float64(0.5319950737175532) - rollout_corr/log_ppl_diff:np.float64(0.0008222219184972346) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021424852893687785) - rollout_corr/log_ppl_diff_max:np.float64(0.011124730110168457) - rollout_corr/log_ppl_diff_min:np.float64(-0.008998692035675049) - rollout_corr/ppl_ratio:np.float64(1.0008265001233667) - rollout_corr/chi2_token:np.float64(0.0017317512538284063) - rollout_corr/chi2_seq:np.float64(1.5403655688278377) - actor/pg_loss:np.float64(0.00012683775275945663) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005490041689881764) - actor/ppo_kl:np.float64(-2.5919941826657578e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.539950042963028) - perf/mfu/actor:np.float64(0.0787262959745778) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(105.37881088256836) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:2 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.013511436060070992 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.013511436060070992 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:311.33203125 - response_length/max:797.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:311.33203125 - response_length_non_aborted/max:797.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00022766925394535065 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5709626711905003) - timing_s/agent_loop/generate_sequences/max:np.float64(6.911529300734401) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6116345795235247) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.911529300734401) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:797 - timing_s/gen:13.802695669233799 - timing_s/reward:0.05287378840148449 - timing_s/old_log_prob:3.188429022207856 - timing_s/adv:0.12535426020622253 - timing_s/update_actor:12.729761390015483 - timing_s/step:29.998526748269796 - timing_s/stop_profile:3.0953437089920044e-05 - timing_per_token_ms/update_actor:0.08202747224362218 - timing_per_token_ms/gen:0.17318095970230987 - timing_per_token_ms/adv:0.0008077522260354957 - perf/total_num_tokens:155189 - perf/time_per_step:29.998526748269796 - perf/throughput:646.6525894015392 (TaskRunner pid=2347384) Training Progress: 29%|██▉ | 29/100 [29:34<54:48, 46.32s/it] (TaskRunner pid=2347384) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2347384) validation generation end (TaskRunner pid=2347384) [prompt] system (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) The folding stability score of a protein sequence is typically calculated using computational methods that consider factors such as hydrogen bonding, hydrophobic interactions, and the overall energy landscape of the folded protein. The sequence provided, 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR', contains a mix of hydrophilic and hydrophobic amino acids, and the presence of multiple charged residues (like E, R, K) may suggest a higher tendency for forming stable interactions. However, without specific computational tools or data, it is not possible to determine the exact folding stability score. Based on typical values observed in such sequences, a positive score indicates a more stable structure. Among the options, B (1.69) and C (2.49) are positive values, with C being higher and suggesting greater stability. Given the sequence's complexity and potential for favorable interactions, a score of 2.49 seems reasonable. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) C (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 0.0 (TaskRunner pid=2347384) [acc] 0.0 (TaskRunner pid=2347384) [pred] C (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2347384) step:30 - global_seqlen/min:15921 - global_seqlen/max:29115 - global_seqlen/minmax_diff:13194 - global_seqlen/balanced_min:21015 - global_seqlen/balanced_max:21850 - global_seqlen/mean:21127.0 - actor/entropy:0.5034244656562805 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35961899161338806 - training/rollout_probs_diff_mean:0.004888434894382954 - training/rollout_probs_diff_std:0.010262053459882736 - training/rollout_actor_probs_pearson_corr:0.9993752241134644 - rollout_corr/rollout_is_mean:1.0002319812774658 - rollout_corr/rollout_is_ratio_fraction_high:1.1315289157209918e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2630578314419836e-05 - rollout_corr/rollout_is_max:4.365750789642334 - rollout_corr/rollout_is_min:0.3669763505458832 - rollout_corr/rollout_is_std:0.03633022680878639 - rollout_corr/rollout_is_eff_sample_size:0.9986825679187074 - rollout_corr/rollout_is_seq_mean:1.0001426935195923 - rollout_corr/rollout_is_seq_std:0.002683563157916069 - rollout_corr/rollout_is_seq_max:1.016749620437622 - rollout_corr/rollout_is_seq_min:0.9914579391479492 - rollout_corr/rollout_is_seq_max_deviation:0.01674962043762207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7409726032055914) - rollout_corr/training_log_ppl:np.float64(0.5416285220417194) - rollout_corr/kl:np.float64(0.0008744750540028434) - rollout_corr/k3_kl:np.float64(0.0008884881920039334) - rollout_corr/rollout_ppl:np.float64(1.7394368732348084) - rollout_corr/rollout_log_ppl:np.float64(0.5407540432061069) - rollout_corr/log_ppl_diff:np.float64(0.0008744788356125355) - rollout_corr/log_ppl_abs_diff:np.float64(0.002122406382113695) - rollout_corr/log_ppl_diff_max:np.float64(0.009858965873718262) - rollout_corr/log_ppl_diff_min:np.float64(-0.011270463466644287) - rollout_corr/ppl_ratio:np.float64(1.0008785829413682) - rollout_corr/chi2_token:np.float64(0.002094854135066271) - rollout_corr/chi2_seq:np.float64(1.5728097292594612) - actor/pg_loss:np.float64(0.00017506827134639025) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022806990341450728) - actor/ppo_kl:np.float64(0.0002573708973727662) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4844425171613693) - perf/mfu/actor:np.float64(0.0842390753133567) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(105.5502815246582) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.41125) - val-aux/sciknoweval/reward/std@16:np.float64(0.3835546105023346) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.58116) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.3378056404427167) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.23626000000000003) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.31866308960167566) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.40958) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3834787728134225) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7272599999999998) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.2301793621926673) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.10631999999999998) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.20106062878529374) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.42849999999999994) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.3462837893823021) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.81274) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.11568744149859533) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.037360000000000004) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0940545327678721) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4434) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.2949953521763409) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.84726) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04303278259713324) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.01134) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03089105639405398) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.45254) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.23150986048248337) - val-aux/sciknoweval/score/mean@16:np.float64(0.41125) - val-aux/sciknoweval/score/std@16:np.float64(0.3835546105023346) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.58116) - val-aux/sciknoweval/score/best@2/std:np.float64(0.3378056404427167) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.23626000000000003) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.31866308960167566) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.40958) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3834787728134225) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7272599999999998) - val-aux/sciknoweval/score/best@4/std:np.float64(0.2301793621926673) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.10631999999999998) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.20106062878529374) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.42849999999999994) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.3462837893823021) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.81274) - val-aux/sciknoweval/score/best@8/std:np.float64(0.11568744149859533) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.037360000000000004) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0940545327678721) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4434) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.2949953521763409) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.84726) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04303278259713324) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.01134) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03089105639405398) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.45254) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.23150986048248337) - val-core/sciknoweval/acc/mean@16:np.float64(0.41125) - val-aux/sciknoweval/acc/std@16:np.float64(0.3835546105023346) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.58116) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.3378056404427167) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.23626000000000003) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.31866308960167566) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.40958) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3834787728134225) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7272599999999998) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.2301793621926673) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.10631999999999998) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.20106062878529374) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.42849999999999994) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.3462837893823021) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.81274) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.11568744149859533) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.037360000000000004) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0940545327678721) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4434) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.2949953521763409) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.84726) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04303278259713324) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.01134) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03089105639405398) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.45254) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.23150986048248337) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04534421173973564) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9991199999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.011701162321065644) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9760599999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.060492315441077034) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9878599999999998) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04444985160309866) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9557800000000002) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0766390454435194) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9961) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.02587639537628238) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.92368) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08778503477812429) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9993000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.011019914219333161) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8808) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08336348185294205) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0012642784503423287) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:2 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007349280174821615 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007349280174821615 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:345.21875 - response_length/max:2169.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:345.21875 - response_length_non_aborted/max:2169.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.0 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5538727641105652e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8688378892838955) - timing_s/agent_loop/generate_sequences/max:np.float64(14.761361297219992) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8996219056498376) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.761361297219992) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:412 - timing_s/agent_loop/slowest/response_length:2169 - timing_s/gen:21.558863831683993 - timing_s/reward:0.05828162655234337 - timing_s/old_log_prob:3.0150050278753042 - timing_s/adv:0.13200783170759678 - timing_s/update_actor:12.994205441325903 - timing_s/step:37.84763587638736 - timing_s/testing:88.05157169140875 - timing_s/save_checkpoint:12.999501321464777 - timing_s/stop_profile:0.000311022624373436 - timing_per_token_ms/update_actor:0.07688151087072173 - timing_per_token_ms/gen:0.243944779484068 - timing_per_token_ms/adv:0.000781037485845108 - perf/total_num_tokens:169016 - perf/time_per_step:37.84763587638736 - perf/throughput:558.2118806311191 (TaskRunner pid=2347384) Training Progress: 30%|███ | 30/100 [31:53<1:26:27, 74.11s/it] (TaskRunner pid=2347384) step:31 - global_seqlen/min:16045 - global_seqlen/max:30169 - global_seqlen/minmax_diff:14124 - global_seqlen/balanced_min:21042 - global_seqlen/balanced_max:21044 - global_seqlen/mean:21043.125 - actor/entropy:0.5093265175819397 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2890603542327881 - training/rollout_probs_diff_mean:0.005097019951790571 - training/rollout_probs_diff_std:0.010553320869803429 - training/rollout_actor_probs_pearson_corr:0.9993405938148499 - rollout_corr/rollout_is_mean:1.0000298023223877 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5787277221679688 - rollout_corr/rollout_is_min:0.5416204929351807 - rollout_corr/rollout_is_std:0.037192802876234055 - rollout_corr/rollout_is_eff_sample_size:0.9986186062923174 - rollout_corr/rollout_is_seq_mean:0.9999734163284302 - rollout_corr/rollout_is_seq_std:0.002234366023913026 - rollout_corr/rollout_is_seq_max:1.006969928741455 - rollout_corr/rollout_is_seq_min:0.9920904636383057 - rollout_corr/rollout_is_seq_max_deviation:0.007909536361694336 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7148501025512815) - rollout_corr/training_log_ppl:np.float64(0.527589610486757) - rollout_corr/kl:np.float64(0.0006631820047111603) - rollout_corr/k3_kl:np.float64(0.0008296667777472067) - rollout_corr/rollout_ppl:np.float64(1.713729961309582) - rollout_corr/rollout_log_ppl:np.float64(0.5269264254020527) - rollout_corr/log_ppl_diff:np.float64(0.0006631850847043097) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019215813954360783) - rollout_corr/log_ppl_diff_max:np.float64(0.008836150169372559) - rollout_corr/log_ppl_diff_min:np.float64(-0.006797522306442261) - rollout_corr/ppl_ratio:np.float64(1.0006662704981863) - rollout_corr/chi2_token:np.float64(0.001993033802136779) - rollout_corr/chi2_seq:np.float64(0.5590176100376993) - actor/pg_loss:np.float64(8.000177331268787e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010203329434261832) - actor/ppo_kl:np.float64(-0.00012113135247204809) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5038574263453484) - perf/mfu/actor:np.float64(0.08390962990998627) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(109.54733657836914) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:2 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0038380834739655256 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0038380834739655256 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.09765625 - response_length/max:1008.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.09765625 - response_length_non_aborted/max:1008.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.787401020526886e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.539093492552638) - timing_s/agent_loop/generate_sequences/max:np.float64(7.527396231889725) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.514145264525723) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.527396231889725) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:419 - timing_s/agent_loop/slowest/response_length:1008 - timing_s/gen:14.28430387750268 - timing_s/reward:0.0593858789652586 - timing_s/old_log_prob:3.35727896168828 - timing_s/adv:0.11829328536987305 - timing_s/update_actor:13.268341844901443 - timing_s/step:31.176795164123178 - timing_s/stop_profile:0.00011946819722652435 - timing_per_token_ms/update_actor:0.07881637022128037 - timing_per_token_ms/gen:0.16651283881217788 - timing_per_token_ms/adv:0.0007026836874862517 - perf/total_num_tokens:168345 - perf/time_per_step:31.176795164123178 - perf/throughput:674.9611334078193 (TaskRunner pid=2347384) Training Progress: 31%|███ | 31/100 [32:24<1:10:26, 61.25s/it] (TaskRunner pid=2347384) step:32 - global_seqlen/min:17853 - global_seqlen/max:26307 - global_seqlen/minmax_diff:8454 - global_seqlen/balanced_min:22042 - global_seqlen/balanced_max:22072 - global_seqlen/mean:22061.125 - actor/entropy:0.5101040601730347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102376401424408 - training/rollout_probs_diff_mean:0.004974946845322847 - training/rollout_probs_diff_std:0.010487372055649757 - training/rollout_actor_probs_pearson_corr:0.9993613958358765 - rollout_corr/rollout_is_mean:0.9999424815177917 - rollout_corr/rollout_is_ratio_fraction_high:1.1160589565406553e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1160589565406553e-05 - rollout_corr/rollout_is_max:2.0903308391571045 - rollout_corr/rollout_is_min:0.3959798514842987 - rollout_corr/rollout_is_std:0.0362941138446331 - rollout_corr/rollout_is_eff_sample_size:0.998684232457028 - rollout_corr/rollout_is_seq_mean:0.9999070167541504 - rollout_corr/rollout_is_seq_std:0.002210472244769335 - rollout_corr/rollout_is_seq_max:1.0059552192687988 - rollout_corr/rollout_is_seq_min:0.9879215359687805 - rollout_corr/rollout_is_seq_max_deviation:0.012078464031219482 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.741852265316993) - rollout_corr/training_log_ppl:np.float64(0.5438747547450475) - rollout_corr/kl:np.float64(0.0010154247492444313) - rollout_corr/k3_kl:np.float64(0.0008875310091411848) - rollout_corr/rollout_ppl:np.float64(1.740026846062392) - rollout_corr/rollout_log_ppl:np.float64(0.542859322624281) - rollout_corr/log_ppl_diff:np.float64(0.0010154321207664907) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019701081910170615) - rollout_corr/log_ppl_diff_max:np.float64(0.013227999210357666) - rollout_corr/log_ppl_diff_min:np.float64(-0.0068541765213012695) - rollout_corr/ppl_ratio:np.float64(1.0010191169567406) - rollout_corr/chi2_token:np.float64(0.00150773162022233) - rollout_corr/chi2_seq:np.float64(0.5326227811165154) - actor/pg_loss:np.float64(-0.0004458230687305331) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012204310414745123) - actor/ppo_kl:np.float64(0.00019506082471743014) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5660746768116951) - perf/mfu/actor:np.float64(0.08873230760940706) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(107.13384246826172) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:2 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014278579503297806 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014278579503297806 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:350.00390625 - response_length/max:1380.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:350.00390625 - response_length_non_aborted/max:1380.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:339.40625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001362096518278122 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5374602898955345) - timing_s/agent_loop/generate_sequences/max:np.float64(10.008588448166847) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.020550312859996) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.008588448166847) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1380 - timing_s/gen:16.92846231162548 - timing_s/reward:0.06167461909353733 - timing_s/old_log_prob:3.0529309920966625 - timing_s/adv:0.1211532223969698 - timing_s/update_actor:12.942295119166374 - timing_s/step:33.1945539675653 - timing_s/stop_profile:0.00012456625699996948 - timing_per_token_ms/update_actor:0.07333202136771343 - timing_per_token_ms/gen:0.18893162254467563 - timing_per_token_ms/adv:0.0006864633059112454 - perf/total_num_tokens:176489 - perf/time_per_step:33.1945539675653 - perf/throughput:664.6007360591778 (TaskRunner pid=2347384) Training Progress: 32%|███▏ | 32/100 [32:57<59:53, 52.85s/it] (TaskRunner pid=2347384) step:33 - global_seqlen/min:15850 - global_seqlen/max:34873 - global_seqlen/minmax_diff:19023 - global_seqlen/balanced_min:23762 - global_seqlen/balanced_max:30149 - global_seqlen/mean:24574.75 - actor/entropy:0.43911048769950867 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15237966179847717 - training/rollout_probs_diff_mean:0.00434852996841073 - training/rollout_probs_diff_std:0.00967381801456213 - training/rollout_actor_probs_pearson_corr:0.9994093179702759 - rollout_corr/rollout_is_mean:0.999904453754425 - rollout_corr/rollout_is_ratio_fraction_high:9.356462214782368e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.356462214782368e-06 - rollout_corr/rollout_is_max:2.930999755859375 - rollout_corr/rollout_is_min:0.4452812671661377 - rollout_corr/rollout_is_std:0.03401712328195572 - rollout_corr/rollout_is_eff_sample_size:0.9988439350407945 - rollout_corr/rollout_is_seq_mean:0.9999206066131592 - rollout_corr/rollout_is_seq_std:0.002141724806278944 - rollout_corr/rollout_is_seq_max:1.0081114768981934 - rollout_corr/rollout_is_seq_min:0.9935972690582275 - rollout_corr/rollout_is_seq_max_deviation:0.00811147689819336 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.679862623102963) - rollout_corr/training_log_ppl:np.float64(0.5077659674498136) - rollout_corr/kl:np.float64(0.000853035526844792) - rollout_corr/k3_kl:np.float64(0.0007963023309116579) - rollout_corr/rollout_ppl:np.float64(1.678448023274541) - rollout_corr/rollout_log_ppl:np.float64(0.506912927092344) - rollout_corr/log_ppl_diff:np.float64(0.0008530403574695811) - rollout_corr/log_ppl_abs_diff:np.float64(0.00179992952325847) - rollout_corr/log_ppl_diff_max:np.float64(0.006957203149795532) - rollout_corr/log_ppl_diff_min:np.float64(-0.00761336088180542) - rollout_corr/ppl_ratio:np.float64(1.0008557294495404) - rollout_corr/chi2_token:np.float64(0.0015092436224222183) - rollout_corr/chi2_seq:np.float64(0.5108529871795326) - actor/pg_loss:np.float64(7.511698640882969e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008746653911657631) - actor/ppo_kl:np.float64(7.056684518147449e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4631449654698372) - perf/mfu/actor:np.float64(0.09239895920539236) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(107.0075798034668) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:2 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004488762933760881 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004488762933760881 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:417.4921875 - response_length/max:8192.0 - response_length/min:118.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:417.4921875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:350.46875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.045323193073273e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5268178712576628) - timing_s/agent_loop/generate_sequences/max:np.float64(60.010625906288624) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.565083497051091) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(60.010625906288624) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:180 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:66.80252124741673 - timing_s/reward:0.06485738791525364 - timing_s/old_log_prob:3.320697782561183 - timing_s/adv:0.11981038749217987 - timing_s/update_actor:13.837974788621068 - timing_s/step:84.23281303234398 - timing_s/stop_profile:0.00011246092617511749 - timing_per_token_ms/update_actor:0.07038715952665373 - timing_per_token_ms/gen:0.6250352855350655 - timing_per_token_ms/adv:0.0006094181400226852 - perf/total_num_tokens:196598 - perf/time_per_step:84.23281303234398 - perf/throughput:291.74794376822854 (TaskRunner pid=2347384) Training Progress: 33%|███▎ | 33/100 [34:21<1:09:32, 62.28s/it] (TaskRunner pid=2347384) step:34 - global_seqlen/min:15870 - global_seqlen/max:30220 - global_seqlen/minmax_diff:14350 - global_seqlen/balanced_min:21412 - global_seqlen/balanced_max:28185 - global_seqlen/mean:22574.0 - actor/entropy:0.46322205662727356 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2762543857097626 - training/rollout_probs_diff_mean:0.004577747080475092 - training/rollout_probs_diff_std:0.010074884630739689 - training/rollout_actor_probs_pearson_corr:0.9993917942047119 - rollout_corr/rollout_is_mean:1.0000509023666382 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.806998605199624e-06 - rollout_corr/rollout_is_max:1.8433923721313477 - rollout_corr/rollout_is_min:0.38417690992355347 - rollout_corr/rollout_is_std:0.03546011075377464 - rollout_corr/rollout_is_eff_sample_size:0.9987442784422956 - rollout_corr/rollout_is_seq_mean:1.0000061988830566 - rollout_corr/rollout_is_seq_std:0.0022956987377256155 - rollout_corr/rollout_is_seq_max:1.0057731866836548 - rollout_corr/rollout_is_seq_min:0.9929726123809814 - rollout_corr/rollout_is_seq_max_deviation:0.007027387619018555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7362358481623232) - rollout_corr/training_log_ppl:np.float64(0.540509320795536) - rollout_corr/kl:np.float64(0.0009273811082515238) - rollout_corr/k3_kl:np.float64(0.0008446048440760023) - rollout_corr/rollout_ppl:np.float64(1.7345940633676946) - rollout_corr/rollout_log_ppl:np.float64(0.5395819384139031) - rollout_corr/log_ppl_diff:np.float64(0.0009273823816329241) - rollout_corr/log_ppl_abs_diff:np.float64(0.002035444718785584) - rollout_corr/log_ppl_diff_max:np.float64(0.007113933563232422) - rollout_corr/log_ppl_diff_min:np.float64(-0.006178379058837891) - rollout_corr/ppl_ratio:np.float64(1.000930847832933) - rollout_corr/chi2_token:np.float64(0.0015633984003216028) - rollout_corr/chi2_seq:np.float64(0.7919935090467334) - actor/pg_loss:np.float64(-0.0002476725494489074) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007952609025778656) - actor/ppo_kl:np.float64(0.00020245065443447174) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4739364758133888) - perf/mfu/actor:np.float64(0.08472597953413721) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(107.10835266113281) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:2 - critic/score/mean:0.37890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010730082169175148 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010730082169175148 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.3125 - response_length/max:8192.0 - response_length/min:140.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:398.3125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:307.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.35023820400238e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7196151595562696) - timing_s/agent_loop/generate_sequences/max:np.float64(58.53228139877319) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.315582930379605) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.53228139877319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.91073139943182 - timing_s/reward:0.05947660468518734 - timing_s/old_log_prob:3.3044851403683424 - timing_s/adv:0.13075527548789978 - timing_s/update_actor:13.936455773189664 - timing_s/step:83.42622312903404 - timing_s/stop_profile:0.00011181272566318512 - timing_per_token_ms/update_actor:0.07717094762331479 - timing_per_token_ms/gen:0.6463864290702164 - timing_per_token_ms/adv:0.0007240369201730962 - perf/total_num_tokens:180592 - perf/time_per_step:83.42622312903404 - perf/throughput:270.5863834334816 (TaskRunner pid=2347384) Training Progress: 34%|███▍ | 34/100 [35:45<1:15:30, 68.64s/it] (TaskRunner pid=2347384) step:35 - global_seqlen/min:15371 - global_seqlen/max:29210 - global_seqlen/minmax_diff:13839 - global_seqlen/balanced_min:21381 - global_seqlen/balanced_max:28274 - global_seqlen/mean:22251.75 - actor/entropy:0.48208385705947876 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294640988111496 - training/rollout_probs_diff_mean:0.004745776765048504 - training/rollout_probs_diff_std:0.010112657211720943 - training/rollout_actor_probs_pearson_corr:0.9993820190429688 - rollout_corr/rollout_is_mean:0.9998485445976257 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.073355244647246e-05 - rollout_corr/rollout_is_max:1.5992817878723145 - rollout_corr/rollout_is_min:0.29721227288246155 - rollout_corr/rollout_is_std:0.035781387239694595 - rollout_corr/rollout_is_eff_sample_size:0.9987208537039524 - rollout_corr/rollout_is_seq_mean:0.9997965693473816 - rollout_corr/rollout_is_seq_std:0.002371368696913123 - rollout_corr/rollout_is_seq_max:1.012627363204956 - rollout_corr/rollout_is_seq_min:0.9940608739852905 - rollout_corr/rollout_is_seq_max_deviation:0.012627363204956055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7351841717027128) - rollout_corr/training_log_ppl:np.float64(0.5391048130259151) - rollout_corr/kl:np.float64(0.0009379798822379826) - rollout_corr/k3_kl:np.float64(0.0008646021540812399) - rollout_corr/rollout_ppl:np.float64(1.7336103562265635) - rollout_corr/rollout_log_ppl:np.float64(0.5381668309128145) - rollout_corr/log_ppl_diff:np.float64(0.0009379821131005883) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019617589423432946) - rollout_corr/log_ppl_diff_max:np.float64(0.008763253688812256) - rollout_corr/log_ppl_diff_min:np.float64(-0.01156991720199585) - rollout_corr/ppl_ratio:np.float64(1.0009413054212928) - rollout_corr/chi2_token:np.float64(0.001647185068577528) - rollout_corr/chi2_seq:np.float64(0.5248547857627273) - actor/pg_loss:np.float64(9.360024705529213e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006373033320414834) - actor/ppo_kl:np.float64(6.670383458029505e-06) - actor/pg_clipfrac_lower:np.float64(4.76837158203125e-07) - actor/grad_norm:np.float64(0.4435551390051842) - perf/mfu/actor:np.float64(0.08314078009404126) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(107.08636093139648) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:2 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.05523677542805672 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.05523677542805672 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:376.8046875 - response_length/max:8192.0 - response_length/min:150.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:376.8046875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:318.5625 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001079067587852478 - timing_s/agent_loop/generate_sequences/min:np.float64(1.865217786282301) - timing_s/agent_loop/generate_sequences/max:np.float64(58.86797649227083) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.1633613895537565) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.86797649227083) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:66.6016163174063 - timing_s/reward:0.060171300545334816 - timing_s/old_log_prob:3.3308295011520386 - timing_s/adv:0.12102525122463703 - timing_s/update_actor:13.883810916915536 - timing_s/step:84.08180479519069 - timing_s/stop_profile:3.536045551300049e-05 - timing_per_token_ms/update_actor:0.07799280347003908 - timing_per_token_ms/gen:0.6904440745309687 - timing_per_token_ms/adv:0.0006798636692880169 - perf/total_num_tokens:178014 - perf/time_per_step:84.08180479519069 - perf/throughput:264.64405770310907 (TaskRunner pid=2347384) Training Progress: 35%|███▌ | 35/100 [37:09<1:19:23, 73.28s/it] (TaskRunner pid=2347384) step:36 - global_seqlen/min:17895 - global_seqlen/max:28009 - global_seqlen/minmax_diff:10114 - global_seqlen/balanced_min:22366 - global_seqlen/balanced_max:22392 - global_seqlen/mean:22385.25 - actor/entropy:0.5474426746368408 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3531901240348816 - training/rollout_probs_diff_mean:0.004762253724038601 - training/rollout_probs_diff_std:0.009661497548222542 - training/rollout_actor_probs_pearson_corr:0.9994480013847351 - rollout_corr/rollout_is_mean:0.999980092048645 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.989411410060711e-06 - rollout_corr/rollout_is_max:1.6471166610717773 - rollout_corr/rollout_is_min:0.43200916051864624 - rollout_corr/rollout_is_std:0.03524091839790344 - rollout_corr/rollout_is_eff_sample_size:0.998759618079525 - rollout_corr/rollout_is_seq_mean:1.0000606775283813 - rollout_corr/rollout_is_seq_std:0.002138224197551608 - rollout_corr/rollout_is_seq_max:1.0066604614257812 - rollout_corr/rollout_is_seq_min:0.9929714798927307 - rollout_corr/rollout_is_seq_max_deviation:0.007028520107269287 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7957607205025852) - rollout_corr/training_log_ppl:np.float64(0.5723578177276067) - rollout_corr/kl:np.float64(0.0008410818211324056) - rollout_corr/k3_kl:np.float64(0.0007388105183281368) - rollout_corr/rollout_ppl:np.float64(1.7942767110653222) - rollout_corr/rollout_log_ppl:np.float64(0.5715167365851812) - rollout_corr/log_ppl_diff:np.float64(0.0008410811424255371) - rollout_corr/log_ppl_abs_diff:np.float64(0.00185401423368603) - rollout_corr/log_ppl_diff_max:np.float64(0.007687121629714966) - rollout_corr/log_ppl_diff_min:np.float64(-0.004860281944274902) - rollout_corr/ppl_ratio:np.float64(1.0008438881486654) - rollout_corr/chi2_token:np.float64(0.0013015002477914095) - rollout_corr/chi2_seq:np.float64(0.7014591388870031) - actor/pg_loss:np.float64(-0.0004100718069821596) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045033029300611815) - actor/ppo_kl:np.float64(0.00023396518165341718) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.419260635972023) - perf/mfu/actor:np.float64(0.09084009435068277) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(107.06069564819336) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:2 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004429055377840996 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004429055377840996 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:391.0390625 - response_length/max:1408.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:391.0390625 - response_length_non_aborted/max:1408.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:308.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013133883476257324 - timing_s/agent_loop/generate_sequences/min:np.float64(2.008711403235793) - timing_s/agent_loop/generate_sequences/max:np.float64(10.44145399145782) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.272390514044673) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.44145399145782) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1408 - timing_s/gen:17.639118649065495 - timing_s/reward:0.05866706371307373 - timing_s/old_log_prob:2.9297636710107327 - timing_s/adv:0.11646445654332638 - timing_s/update_actor:12.849967055022717 - timing_s/step:33.6741211079061 - timing_s/stop_profile:0.00012512877583503723 - timing_per_token_ms/update_actor:0.07175465459969577 - timing_per_token_ms/gen:0.1762044098162497 - timing_per_token_ms/adv:0.0006503415002251839 - perf/total_num_tokens:179082 - perf/time_per_step:33.6741211079061 - perf/throughput:664.761225044841 (TaskRunner pid=2347384) Training Progress: 36%|███▌ | 36/100 [37:43<1:05:30, 61.41s/it] (TaskRunner pid=2347384) step:37 - global_seqlen/min:14796 - global_seqlen/max:28597 - global_seqlen/minmax_diff:13801 - global_seqlen/balanced_min:21550 - global_seqlen/balanced_max:21695 - global_seqlen/mean:21581.125 - actor/entropy:0.5615962743759155 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19333729147911072 - training/rollout_probs_diff_mean:0.004837579093873501 - training/rollout_probs_diff_std:0.009564741514623165 - training/rollout_actor_probs_pearson_corr:0.999474823474884 - rollout_corr/rollout_is_mean:1.000237226486206 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5724468231201172 - rollout_corr/rollout_is_min:0.5126587152481079 - rollout_corr/rollout_is_std:0.03539786487817764 - rollout_corr/rollout_is_eff_sample_size:0.9987492726910567 - rollout_corr/rollout_is_seq_mean:1.0002822875976562 - rollout_corr/rollout_is_seq_std:0.002085695043206215 - rollout_corr/rollout_is_seq_max:1.0059657096862793 - rollout_corr/rollout_is_seq_min:0.9942064881324768 - rollout_corr/rollout_is_seq_max_deviation:0.005965709686279297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8264168752357364) - rollout_corr/training_log_ppl:np.float64(0.5892809363431297) - rollout_corr/kl:np.float64(0.0006145548779485921) - rollout_corr/k3_kl:np.float64(0.0007332920498583917) - rollout_corr/rollout_ppl:np.float64(1.8253387045115232) - rollout_corr/rollout_log_ppl:np.float64(0.5886663760757074) - rollout_corr/log_ppl_diff:np.float64(0.0006145602674223483) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017893717740662396) - rollout_corr/log_ppl_diff_max:np.float64(0.0075185298919677734) - rollout_corr/log_ppl_diff_min:np.float64(-0.006856441497802734) - rollout_corr/ppl_ratio:np.float64(1.0006174142472446) - rollout_corr/chi2_token:np.float64(0.0017267197836190462) - rollout_corr/chi2_seq:np.float64(0.9451530280057341) - actor/pg_loss:np.float64(-2.6800204068422318e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035696033910426195) - actor/ppo_kl:np.float64(0.00022737449768328588) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4306887537240982) - perf/mfu/actor:np.float64(0.08714707167339927) - perf/max_memory_allocated_gb:np.float64(132.02660465240479) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(107.01277923583984) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:2 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01045309193432331 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01045309193432331 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:378.69140625 - response_length/max:1760.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:378.69140625 - response_length_non_aborted/max:1760.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015825405716896057 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5638416465371847) - timing_s/agent_loop/generate_sequences/max:np.float64(12.766646282747388) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.241187332525442) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.766646282747388) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1760 - timing_s/gen:19.752604868263006 - timing_s/reward:0.06103142164647579 - timing_s/old_log_prob:3.150852795690298 - timing_s/adv:0.16423144564032555 - timing_s/update_actor:12.863763146102428 - timing_s/step:36.080593310296535 - timing_s/stop_profile:0.00011505931615829468 - timing_per_token_ms/update_actor:0.07450818218525695 - timing_per_token_ms/gen:0.20375063044265312 - timing_per_token_ms/adv:0.0009512446966986519 - perf/total_num_tokens:172649 - perf/time_per_step:36.080593310296535 - perf/throughput:598.1366441067161 (TaskRunner pid=2347384) Training Progress: 37%|███▋ | 37/100 [38:19<56:31, 53.83s/it] (TaskRunner pid=2347384) step:38 - global_seqlen/min:22667 - global_seqlen/max:31748 - global_seqlen/minmax_diff:9081 - global_seqlen/balanced_min:24882 - global_seqlen/balanced_max:31312 - global_seqlen/mean:26002.125 - actor/entropy:0.49343523383140564 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19023588299751282 - training/rollout_probs_diff_mean:0.004170398227870464 - training/rollout_probs_diff_std:0.008934952318668365 - training/rollout_actor_probs_pearson_corr:0.9995138645172119 - rollout_corr/rollout_is_mean:1.000038981437683 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.40950906422222e-06 - rollout_corr/rollout_is_max:1.605200171470642 - rollout_corr/rollout_is_min:0.43745046854019165 - rollout_corr/rollout_is_std:0.03293818235397339 - rollout_corr/rollout_is_eff_sample_size:0.9989163709907513 - rollout_corr/rollout_is_seq_mean:1.0000364780426025 - rollout_corr/rollout_is_seq_std:0.0018815776566043496 - rollout_corr/rollout_is_seq_max:1.006003499031067 - rollout_corr/rollout_is_seq_min:0.9939581155776978 - rollout_corr/rollout_is_seq_max_deviation:0.006041884422302246 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7932175691239536) - rollout_corr/training_log_ppl:np.float64(0.5726296329521574) - rollout_corr/kl:np.float64(0.0007390747027464784) - rollout_corr/k3_kl:np.float64(0.00069970584891621) - rollout_corr/rollout_ppl:np.float64(1.7918073264881968) - rollout_corr/rollout_log_ppl:np.float64(0.5718905562534928) - rollout_corr/log_ppl_diff:np.float64(0.0007390766986645758) - rollout_corr/log_ppl_abs_diff:np.float64(0.00165215675951913) - rollout_corr/log_ppl_diff_max:np.float64(0.006316483020782471) - rollout_corr/log_ppl_diff_min:np.float64(-0.004039406776428223) - rollout_corr/ppl_ratio:np.float64(1.0007413655985147) - rollout_corr/chi2_token:np.float64(0.001315293600782752) - rollout_corr/chi2_seq:np.float64(0.5285406170878559) - actor/pg_loss:np.float64(0.00012777664232999086) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005600684596629435) - actor/ppo_kl:np.float64(0.00014184987492527057) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5138222500681877) - perf/mfu/actor:np.float64(0.09607802312833064) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(107.05743408203125) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:2 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02484484389424324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02484484389424324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:464.50390625 - response_length/max:8192.0 - response_length/min:168.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:464.50390625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:348.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.371340274810791e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1275972314178944) - timing_s/agent_loop/generate_sequences/max:np.float64(58.58451947197318) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.890880755818216) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.58451947197318) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:433 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.2947052065283 - timing_s/reward:0.06526453979313374 - timing_s/old_log_prob:3.377095114439726 - timing_s/adv:0.1566112320870161 - timing_s/update_actor:14.238119848072529 - timing_s/step:83.21680972911417 - timing_s/stop_profile:0.00012373551726341248 - timing_per_token_ms/update_actor:0.06844690505137815 - timing_per_token_ms/gen:0.5490964419914417 - timing_per_token_ms/adv:0.0007528770825798666 - perf/total_num_tokens:208017 - perf/time_per_step:83.21680972911417 - perf/throughput:312.46241095569076 (TaskRunner pid=2347384) Training Progress: 38%|███▊ | 38/100 [39:42<1:04:45, 62.66s/it] (TaskRunner pid=2347384) step:39 - global_seqlen/min:21922 - global_seqlen/max:31119 - global_seqlen/minmax_diff:9197 - global_seqlen/balanced_min:24178 - global_seqlen/balanced_max:30466 - global_seqlen/mean:25818.875 - actor/entropy:0.49117639660835266 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44297343492507935 - training/rollout_probs_diff_mean:0.004097122233361006 - training/rollout_probs_diff_std:0.009035130962729454 - training/rollout_actor_probs_pearson_corr:0.9995251893997192 - rollout_corr/rollout_is_mean:0.9999533891677856 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.3793094442225993e-05 - rollout_corr/rollout_is_max:1.7279109954833984 - rollout_corr/rollout_is_min:0.329568475484848 - rollout_corr/rollout_is_std:0.03281852975487709 - rollout_corr/rollout_is_eff_sample_size:0.998923983917394 - rollout_corr/rollout_is_seq_mean:0.9999627470970154 - rollout_corr/rollout_is_seq_std:0.0019861103501170874 - rollout_corr/rollout_is_seq_max:1.0055146217346191 - rollout_corr/rollout_is_seq_min:0.9945484399795532 - rollout_corr/rollout_is_seq_max_deviation:0.005514621734619141 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.832747123669833) - rollout_corr/training_log_ppl:np.float64(0.5925828650360927) - rollout_corr/kl:np.float64(0.0006982816480949339) - rollout_corr/k3_kl:np.float64(0.0007501204074173984) - rollout_corr/rollout_ppl:np.float64(1.8314804444089532) - rollout_corr/rollout_log_ppl:np.float64(0.5918845816631801) - rollout_corr/log_ppl_diff:np.float64(0.0006982833729125559) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017554989899508655) - rollout_corr/log_ppl_diff_max:np.float64(0.006209671497344971) - rollout_corr/log_ppl_diff_min:np.float64(-0.005997836589813232) - rollout_corr/ppl_ratio:np.float64(1.0007007885724306) - rollout_corr/chi2_token:np.float64(0.0016319723799824715) - rollout_corr/chi2_seq:np.float64(0.898369211005047) - actor/pg_loss:np.float64(-0.00016278354451060295) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040480234133610793) - actor/ppo_kl:np.float64(9.550054748785897e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4561876431107521) - perf/mfu/actor:np.float64(0.09614644721123172) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(107.13670349121094) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:2 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.020068485289812088 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.020068485289812088 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:492.52734375 - response_length/max:8192.0 - response_length/min:150.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:492.52734375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:314.3125 - prompt_length/max:498.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.406358003616333e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9149847514927387) - timing_s/agent_loop/generate_sequences/max:np.float64(58.697369664907455) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.188801962154685) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.697369664907455) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.92476520128548 - timing_s/reward:0.06863936595618725 - timing_s/old_log_prob:3.327646302059293 - timing_s/adv:0.1567628588527441 - timing_s/update_actor:13.910940635949373 - timing_s/step:83.47381004877388 - timing_s/stop_profile:3.9068982005119324e-05 - timing_per_token_ms/update_actor:0.06734869662189664 - timing_per_token_ms/gen:0.5228514057855725 - timing_per_token_ms/adv:0.0007589547320165194 - perf/total_num_tokens:206551 - perf/time_per_step:83.47381004877388 - perf/throughput:309.3050980291182 (TaskRunner pid=2347384) Training Progress: 39%|███▉ | 39/100 [41:06<1:10:03, 68.91s/it] (TaskRunner pid=2347384) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2347384) validation generation end (TaskRunner pid=2347384) [prompt] system (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) To determine the folding stability score of a protein sequence, we need to evaluate the overall energy landscape of the protein, which includes factors such as hydrogen bonding, hydrophobic interactions, and the overall secondary and tertiary structure formation. The stability score is typically a measure of how favorable the folding process is, with higher values indicating greater stability. (TaskRunner pid=2347384) (TaskRunner pid=2347384) The given protein sequence is 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'. The sequence contains a mix of hydrophilic and hydrophobic amino acids, as well as charged residues, which may influence the folding behavior. The presence of multiple charged residues, such as lysine, arginine, and glutamic acid, could contribute to the formation of salt bridges and electrostatic interactions, which can stabilize the folded structure. (TaskRunner pid=2347384) (TaskRunner pid=2347384) However, without performing a detailed energy calculation or using a specific algorithm (such as DSSP or FoldX), we cannot calculate the exact folding stability score. Instead, we can infer the likely score based on the sequence characteristics. The sequence has a relatively balanced distribution of hydrophilic and hydrophobic residues, suggesting moderate stability. Among the given options, a score of 1.69 (Option B) is the most reasonable as it reflects moderate folding stability, which is typical for a sequence with a mix of stabilizing and destabilizing interactions. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) B (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 1.0 (TaskRunner pid=2347384) [acc] 1.0 (TaskRunner pid=2347384) [pred] B (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2347384) step:40 - global_seqlen/min:19098 - global_seqlen/max:28317 - global_seqlen/minmax_diff:9219 - global_seqlen/balanced_min:22424 - global_seqlen/balanced_max:29206 - global_seqlen/mean:23277.75 - actor/entropy:0.5539281964302063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29786157608032227 - training/rollout_probs_diff_mean:0.004488453734666109 - training/rollout_probs_diff_std:0.008966107852756977 - training/rollout_actor_probs_pearson_corr:0.9995365142822266 - rollout_corr/rollout_is_mean:1.0000907182693481 - rollout_corr/rollout_is_ratio_fraction_high:9.498660801909864e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.498660801909864e-06 - rollout_corr/rollout_is_max:3.913763999938965 - rollout_corr/rollout_is_min:0.4965573251247406 - rollout_corr/rollout_is_std:0.03439177945256233 - rollout_corr/rollout_is_eff_sample_size:0.9988188406349303 - rollout_corr/rollout_is_seq_mean:1.0001420974731445 - rollout_corr/rollout_is_seq_std:0.0018958095461130142 - rollout_corr/rollout_is_seq_max:1.0051233768463135 - rollout_corr/rollout_is_seq_min:0.9933481812477112 - rollout_corr/rollout_is_seq_max_deviation:0.006651818752288818 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8776730671525002) - rollout_corr/training_log_ppl:np.float64(0.6196891939471243) - rollout_corr/kl:np.float64(0.000507959112133971) - rollout_corr/k3_kl:np.float64(0.0007296323929608661) - rollout_corr/rollout_ppl:np.float64(1.8767864825204015) - rollout_corr/rollout_log_ppl:np.float64(0.6191812297620345) - rollout_corr/log_ppl_diff:np.float64(0.0005079641850898042) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017655136616667733) - rollout_corr/log_ppl_diff_max:np.float64(0.007503926753997803) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071929097175598145) - rollout_corr/ppl_ratio:np.float64(1.0005105822347105) - rollout_corr/chi2_token:np.float64(0.0019365742336958647) - rollout_corr/chi2_seq:np.float64(1.3874397985637188) - actor/pg_loss:np.float64(0.0001728396164253354) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002644850069373206) - actor/ppo_kl:np.float64(-1.4685543522574562e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4062229320406914) - perf/mfu/actor:np.float64(0.08707724686407023) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(107.21352910995483) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.39125) - val-aux/sciknoweval/reward/std@16:np.float64(0.37253521037910325) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5513800000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.3506419915147702) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.23148) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.286829527822001) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.39202000000000004) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3743125310521978) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.69842) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.2768187174348611) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.1204) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.17445881738042573) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.41034000000000004) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.3181482376669904) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8081400000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.18146952350964213) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.06094) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08641530208328256) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.42536) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.2527305764302417) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.87614) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.09890418393972275) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.0333) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03730281737209264) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.43076000000000003) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.18547242699672178) - val-aux/sciknoweval/score/mean@16:np.float64(0.39125) - val-aux/sciknoweval/score/std@16:np.float64(0.37253521037910325) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5513800000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.3506419915147702) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.23148) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.286829527822001) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.39202000000000004) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3743125310521978) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.69842) - val-aux/sciknoweval/score/best@4/std:np.float64(0.2768187174348611) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.1204) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.17445881738042573) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.41034000000000004) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.3181482376669904) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8081400000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.18146952350964213) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.06094) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08641530208328256) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.42536) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.2527305764302417) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.87614) - val-aux/sciknoweval/score/best@16/std:np.float64(0.09890418393972275) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.0333) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03730281737209264) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.43076000000000003) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.18547242699672178) - val-core/sciknoweval/acc/mean@16:np.float64(0.39125) - val-aux/sciknoweval/acc/std@16:np.float64(0.37253521037910325) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5513800000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.3506419915147702) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.23148) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.286829527822001) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.39202000000000004) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3743125310521978) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.69842) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.2768187174348611) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.1204) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.17445881738042573) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.41034000000000004) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.3181482376669904) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8081400000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.18146952350964213) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.06094) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08641530208328256) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.42536) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.2527305764302417) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.87614) - val-core/sciknoweval/acc/best@16/std:np.float64(0.09890418393972275) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.0333) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03730281737209264) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.43076000000000003) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.18547242699672178) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.95875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.07511446610139584) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.98752) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.040188154411142325) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9291799999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.08671215012675501) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9583200000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.07496689612436125) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9980999999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.01168845214380852) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.89128) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08805667011323962) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9778800000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.05525880779068303) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.99988) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.001544538766104626) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.84994) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07221574751054122) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99162) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.030663840185843803) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8207800000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04400775794183956) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9979800000000002) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.011026911444074304) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:2 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04454397037625313 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04454397037625313 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:411.2421875 - response_length/max:8192.0 - response_length/min:143.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:411.2421875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:316.1875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.996173083782196e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6584660951048136) - timing_s/agent_loop/generate_sequences/max:np.float64(58.39521440118551) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.46966753853485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.39521440118551) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.45467731356621 - timing_s/reward:0.0636279359459877 - timing_s/old_log_prob:3.5946736447513103 - timing_s/adv:0.1700342483818531 - timing_s/update_actor:13.864726072177291 - timing_s/step:83.23722095787525 - timing_s/testing:101.98301873914897 - timing_s/save_checkpoint:12.744284307584167 - timing_s/stop_profile:0.00019394978880882263 - timing_per_token_ms/update_actor:0.0744526751521157 - timing_per_token_ms/gen:0.6217317702992667 - timing_per_token_ms/adv:0.0009130728291064058 - perf/total_num_tokens:186222 - perf/time_per_step:83.23722095787525 - perf/throughput:279.6555402994583 (TaskRunner pid=2347384) Training Progress: 40%|████ | 40/100 [44:24<1:47:39, 107.65s/it] (TaskRunner pid=2347384) step:41 - global_seqlen/min:15547 - global_seqlen/max:35816 - global_seqlen/minmax_diff:20269 - global_seqlen/balanced_min:24560 - global_seqlen/balanced_max:31004 - global_seqlen/mean:25459.625 - actor/entropy:0.5520501136779785 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30264812707901 - training/rollout_probs_diff_mean:0.0042206603102386 - training/rollout_probs_diff_std:0.008700028993189335 - training/rollout_actor_probs_pearson_corr:0.9995612502098083 - rollout_corr/rollout_is_mean:0.9999479055404663 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.706178954918869e-05 - rollout_corr/rollout_is_max:1.6090575456619263 - rollout_corr/rollout_is_min:0.4421665668487549 - rollout_corr/rollout_is_std:0.032896533608436584 - rollout_corr/rollout_is_eff_sample_size:0.9989186310662374 - rollout_corr/rollout_is_seq_mean:0.9999524354934692 - rollout_corr/rollout_is_seq_std:0.0018278578063473105 - rollout_corr/rollout_is_seq_max:1.0051695108413696 - rollout_corr/rollout_is_seq_min:0.9950945973396301 - rollout_corr/rollout_is_seq_max_deviation:0.005169510841369629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9041591510176659) - rollout_corr/training_log_ppl:np.float64(0.628253795846831) - rollout_corr/kl:np.float64(0.0006011622540711414) - rollout_corr/k3_kl:np.float64(0.0006767768304882793) - rollout_corr/rollout_ppl:np.float64(1.9029451552778482) - rollout_corr/rollout_log_ppl:np.float64(0.6276526338187978) - rollout_corr/log_ppl_diff:np.float64(0.0006011620280332863) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015633253497071564) - rollout_corr/log_ppl_diff_max:np.float64(0.006269395351409912) - rollout_corr/log_ppl_diff_min:np.float64(-0.005324721336364746) - rollout_corr/ppl_ratio:np.float64(1.0006031626835465) - rollout_corr/chi2_token:np.float64(0.0015126701910048723) - rollout_corr/chi2_seq:np.float64(0.4702533932868391) - actor/pg_loss:np.float64(0.00018820911645889282) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024871044433893985) - actor/ppo_kl:np.float64(-6.770004552336673e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.43679335713386536) - perf/mfu/actor:np.float64(0.09446160551951877) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(107.07995986938477) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:2 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005686907563358545 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005686907563358545 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:457.89453125 - response_length/max:8192.0 - response_length/min:159.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:457.89453125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:337.71875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.386451423168182e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.994308490306139) - timing_s/agent_loop/generate_sequences/max:np.float64(58.91641258075833) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.007014467555564) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.91641258075833) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.41723501868546 - timing_s/reward:0.0621959175914526 - timing_s/old_log_prob:3.644586319103837 - timing_s/adv:0.16345126181840897 - timing_s/update_actor:14.090331194922328 - timing_s/step:83.48437073267996 - timing_s/stop_profile:0.00014322064816951752 - timing_per_token_ms/update_actor:0.06917978561606038 - timing_per_token_ms/gen:0.5580675392522284 - timing_per_token_ms/adv:0.0008025023042287984 - perf/total_num_tokens:203677 - perf/time_per_step:83.48437073267996 - perf/throughput:304.96277059478183 (TaskRunner pid=2347384) Training Progress: 41%|████ | 41/100 [45:47<1:38:45, 100.42s/it] (TaskRunner pid=2347384) step:42 - global_seqlen/min:14836 - global_seqlen/max:23639 - global_seqlen/minmax_diff:8803 - global_seqlen/balanced_min:18882 - global_seqlen/balanced_max:18891 - global_seqlen/mean:18888.875 - actor/entropy:0.6401734352111816 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22833934426307678 - training/rollout_probs_diff_mean:0.005033794790506363 - training/rollout_probs_diff_std:0.009340723045170307 - training/rollout_actor_probs_pearson_corr:0.9995176792144775 - rollout_corr/rollout_is_mean:0.9999657273292542 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6378281116485596 - rollout_corr/rollout_is_min:0.5725600123405457 - rollout_corr/rollout_is_std:0.03597741574048996 - rollout_corr/rollout_is_eff_sample_size:0.9987071798722467 - rollout_corr/rollout_is_seq_mean:0.9999330043792725 - rollout_corr/rollout_is_seq_std:0.002047613263130188 - rollout_corr/rollout_is_seq_max:1.0051140785217285 - rollout_corr/rollout_is_seq_min:0.9947119951248169 - rollout_corr/rollout_is_seq_max_deviation:0.0052880048751831055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9359472929500043) - rollout_corr/training_log_ppl:np.float64(0.6511492785648443) - rollout_corr/kl:np.float64(0.0007118323650878722) - rollout_corr/k3_kl:np.float64(0.0007069421216101546) - rollout_corr/rollout_ppl:np.float64(1.9345732158981264) - rollout_corr/rollout_log_ppl:np.float64(0.6504374450305477) - rollout_corr/log_ppl_diff:np.float64(0.000711833534296602) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018189054098911583) - rollout_corr/log_ppl_diff_max:np.float64(0.005858421325683594) - rollout_corr/log_ppl_diff_min:np.float64(-0.006035566329956055) - rollout_corr/ppl_ratio:np.float64(1.000714389840141) - rollout_corr/chi2_token:np.float64(0.0014167285989969969) - rollout_corr/chi2_seq:np.float64(0.5743190047796816) - actor/pg_loss:np.float64(4.471093416213989e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019467047195576015) - actor/ppo_kl:np.float64(-1.92434488219817e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.40224236249923706) - perf/mfu/actor:np.float64(0.0758990405772834) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(103.73267364501953) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:2 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018167474772781134 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0018167474772781134 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:339.18359375 - response_length/max:937.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.18359375 - response_length_non_aborted/max:937.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.09375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.158013224601746e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5642009321600199) - timing_s/agent_loop/generate_sequences/max:np.float64(7.6919788755476475) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8862094332580455) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.6919788755476475) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:937 - timing_s/gen:14.409921823069453 - timing_s/reward:0.059355009347200394 - timing_s/old_log_prob:3.0306448582559824 - timing_s/adv:0.12006686069071293 - timing_s/update_actor:12.850445739924908 - timing_s/step:30.512496650218964 - timing_s/stop_profile:0.0001128166913986206 - timing_per_token_ms/update_actor:0.0850397769846332 - timing_per_token_ms/gen:0.16595365506638704 - timing_per_token_ms/adv:0.0007945606917478736 - perf/total_num_tokens:151111 - perf/time_per_step:30.512496650218964 - perf/throughput:619.0537344922395 (TaskRunner pid=2347384) Training Progress: 42%|████▏ | 42/100 [46:18<1:16:49, 79.47s/it] (TaskRunner pid=2347384) step:43 - global_seqlen/min:17035 - global_seqlen/max:32944 - global_seqlen/minmax_diff:15909 - global_seqlen/balanced_min:23168 - global_seqlen/balanced_max:23388 - global_seqlen/mean:23206.0 - actor/entropy:0.61858069896698 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17755085229873657 - training/rollout_probs_diff_mean:0.004656197037547827 - training/rollout_probs_diff_std:0.0089277857914567 - training/rollout_actor_probs_pearson_corr:0.9995635747909546 - rollout_corr/rollout_is_mean:1.0001128911972046 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8120098114013672 - rollout_corr/rollout_is_min:0.6413815021514893 - rollout_corr/rollout_is_std:0.034812089055776596 - rollout_corr/rollout_is_eff_sample_size:0.9987899420046099 - rollout_corr/rollout_is_seq_mean:1.0000479221343994 - rollout_corr/rollout_is_seq_std:0.0017971129855141044 - rollout_corr/rollout_is_seq_max:1.0046108961105347 - rollout_corr/rollout_is_seq_min:0.9942375421524048 - rollout_corr/rollout_is_seq_max_deviation:0.005762457847595215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9328222768381238) - rollout_corr/training_log_ppl:np.float64(0.6469125346047804) - rollout_corr/kl:np.float64(0.0006033569120225124) - rollout_corr/k3_kl:np.float64(0.0006782633599868859) - rollout_corr/rollout_ppl:np.float64(1.9316168171353638) - rollout_corr/rollout_log_ppl:np.float64(0.6463091767509468) - rollout_corr/log_ppl_diff:np.float64(0.000603357853833586) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016255388618446887) - rollout_corr/log_ppl_diff_max:np.float64(0.006385475397109985) - rollout_corr/log_ppl_diff_min:np.float64(-0.0064054131507873535) - rollout_corr/ppl_ratio:np.float64(1.000605583190918) - rollout_corr/chi2_token:np.float64(0.0015161496121436357) - rollout_corr/chi2_seq:np.float64(0.5250671177636832) - actor/pg_loss:np.float64(-6.645242683589458e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00037597838763758773) - actor/ppo_kl:np.float64(1.0658516490735792e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4496193006634712) - perf/mfu/actor:np.float64(0.09248141553247767) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.73828125) - perf/cpu_memory_used_gb:np.float64(103.65077209472656) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:3 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023391157388687134 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023391157388687134 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:405.65625 - response_length/max:1901.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:405.65625 - response_length_non_aborted/max:1901.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:319.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003095921128988266 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7278583254665136) - timing_s/agent_loop/generate_sequences/max:np.float64(13.650023434311152) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.456003722312744) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.650023434311152) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1901 - timing_s/gen:20.256701685488224 - timing_s/reward:0.05929107777774334 - timing_s/old_log_prob:3.2901252806186676 - timing_s/adv:0.12749466486275196 - timing_s/update_actor:13.040550323203206 - timing_s/step:36.859653325751424 - timing_s/stop_profile:0.0001307222992181778 - timing_per_token_ms/update_actor:0.07024341939155394 - timing_per_token_ms/gen:0.1950610669968437 - timing_per_token_ms/adv:0.0006867548525314141 - perf/total_num_tokens:185648 - perf/time_per_step:36.859653325751424 - perf/throughput:629.5772723339069 (TaskRunner pid=2347384) Training Progress: 43%|████▎ | 43/100 [46:55<1:03:28, 66.81s/it] (TaskRunner pid=2347384) step:44 - global_seqlen/min:17897 - global_seqlen/max:43686 - global_seqlen/minmax_diff:25789 - global_seqlen/balanced_min:22762 - global_seqlen/balanced_max:29526 - global_seqlen/mean:24801.625 - actor/entropy:0.5510364770889282 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39974313974380493 - training/rollout_probs_diff_mean:0.004158153664320707 - training/rollout_probs_diff_std:0.008739613927900791 - training/rollout_actor_probs_pearson_corr:0.9995846152305603 - rollout_corr/rollout_is_mean:0.999917209148407 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.5946843379642814e-05 - rollout_corr/rollout_is_max:1.7508643865585327 - rollout_corr/rollout_is_min:0.33229053020477295 - rollout_corr/rollout_is_std:0.03309163823723793 - rollout_corr/rollout_is_eff_sample_size:0.9989060223546776 - rollout_corr/rollout_is_seq_mean:0.9999911785125732 - rollout_corr/rollout_is_seq_std:0.0019545264076441526 - rollout_corr/rollout_is_seq_max:1.0059704780578613 - rollout_corr/rollout_is_seq_min:0.9944885969161987 - rollout_corr/rollout_is_seq_max_deviation:0.005970478057861328 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9735831669531763) - rollout_corr/training_log_ppl:np.float64(0.6637885194068076) - rollout_corr/kl:np.float64(0.0006672312732414198) - rollout_corr/k3_kl:np.float64(0.0006956736225447457) - rollout_corr/rollout_ppl:np.float64(1.9722777991555631) - rollout_corr/rollout_log_ppl:np.float64(0.663121281308122) - rollout_corr/log_ppl_diff:np.float64(0.0006672380986856297) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015487042401218787) - rollout_corr/log_ppl_diff_max:np.float64(0.005845367908477783) - rollout_corr/log_ppl_diff_min:np.float64(-0.003970146179199219) - rollout_corr/ppl_ratio:np.float64(1.0006691010203212) - rollout_corr/chi2_token:np.float64(0.0014661811292171478) - rollout_corr/chi2_seq:np.float64(0.3399782250635326) - actor/pg_loss:np.float64(0.00020380807109177113) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040891994444791635) - actor/ppo_kl:np.float64(4.297608763081939e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4148004576563835) - perf/mfu/actor:np.float64(0.09312832470559576) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(103.71192932128906) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:3 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06937429308891296 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06937429308891296 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:451.64453125 - response_length/max:8192.0 - response_length/min:138.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:451.64453125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:323.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015909038484096527 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7205688413232565) - timing_s/agent_loop/generate_sequences/max:np.float64(58.78740573488176) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.852808053030458) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.78740573488176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.53086483851075 - timing_s/reward:0.06448325142264366 - timing_s/old_log_prob:3.387485284358263 - timing_s/adv:0.1587330959737301 - timing_s/update_actor:14.077303072437644 - timing_s/step:83.3061857149005 - timing_s/stop_profile:0.00012183934450149536 - timing_per_token_ms/update_actor:0.07094949964184627 - timing_per_token_ms/gen:0.5667730329136641 - timing_per_token_ms/adv:0.0008000135876869463 - perf/total_num_tokens:198413 - perf/time_per_step:83.3061857149005 - perf/throughput:297.71648752325336 (TaskRunner pid=2347384) Training Progress: 44%|████▍ | 44/100 [48:19<1:06:59, 71.78s/it] (TaskRunner pid=2347384) step:45 - global_seqlen/min:16930 - global_seqlen/max:27121 - global_seqlen/minmax_diff:10191 - global_seqlen/balanced_min:21969 - global_seqlen/balanced_max:24050 - global_seqlen/mean:22232.375 - actor/entropy:0.631592869758606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17483389377593994 - training/rollout_probs_diff_mean:0.004726639948785305 - training/rollout_probs_diff_std:0.00895187258720398 - training/rollout_actor_probs_pearson_corr:0.9995525479316711 - rollout_corr/rollout_is_mean:0.9998272657394409 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.460617184638977 - rollout_corr/rollout_is_min:0.5448459982872009 - rollout_corr/rollout_is_std:0.034936003386974335 - rollout_corr/rollout_is_eff_sample_size:0.9987805473368377 - rollout_corr/rollout_is_seq_mean:0.9997673630714417 - rollout_corr/rollout_is_seq_std:0.0020082779228687286 - rollout_corr/rollout_is_seq_max:1.005624771118164 - rollout_corr/rollout_is_seq_min:0.9916337132453918 - rollout_corr/rollout_is_seq_max_deviation:0.008366286754608154 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9634366636164486) - rollout_corr/training_log_ppl:np.float64(0.6637803053599782) - rollout_corr/kl:np.float64(0.0008265025999527609) - rollout_corr/k3_kl:np.float64(0.0006917285731447009) - rollout_corr/rollout_ppl:np.float64(1.9618264771997929) - rollout_corr/rollout_log_ppl:np.float64(0.6629537985136267) - rollout_corr/log_ppl_diff:np.float64(0.000826506846351549) - rollout_corr/log_ppl_abs_diff:np.float64(0.001893459091661498) - rollout_corr/log_ppl_diff_max:np.float64(0.008935153484344482) - rollout_corr/log_ppl_diff_min:np.float64(-0.004910647869110107) - rollout_corr/ppl_ratio:np.float64(1.0008292747661471) - rollout_corr/chi2_token:np.float64(0.001116161234676838) - rollout_corr/chi2_seq:np.float64(0.6050010113976896) - actor/pg_loss:np.float64(0.00011260458268225193) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002743983029631636) - actor/ppo_kl:np.float64(-5.155293970204866e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.42793848365545273) - perf/mfu/actor:np.float64(0.08729111832699632) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(103.7159948348999) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:3 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007823261432349682 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007823261432349682 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:377.16796875 - response_length/max:3316.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:377.16796875 - response_length_non_aborted/max:3316.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.59375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.276345372200012e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.33570821210742) - timing_s/agent_loop/generate_sequences/max:np.float64(21.658590575680137) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9980578175236587) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.658590575680137) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:3316 - timing_s/gen:28.738526644185185 - timing_s/reward:0.06110992096364498 - timing_s/old_log_prob:3.1626545432955027 - timing_s/adv:0.22752412594854832 - timing_s/update_actor:13.23834990337491 - timing_s/step:45.51777372136712 - timing_s/stop_profile:0.0002045668661594391 - timing_per_token_ms/update_actor:0.07443171221796428 - timing_per_token_ms/gen:0.2976389274940209 - timing_per_token_ms/adv:0.0012792387562538209 - perf/total_num_tokens:177859 - perf/time_per_step:45.51777372136712 - perf/throughput:488.4328292524464 (TaskRunner pid=2347384) Training Progress: 45%|████▌ | 45/100 [49:04<58:35, 63.92s/it] (TaskRunner pid=2347384) step:46 - global_seqlen/min:16902 - global_seqlen/max:26550 - global_seqlen/minmax_diff:9648 - global_seqlen/balanced_min:20277 - global_seqlen/balanced_max:27022 - global_seqlen/mean:21122.625 - actor/entropy:0.6554891467094421 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3749000132083893 - training/rollout_probs_diff_mean:0.004718408919870853 - training/rollout_probs_diff_std:0.008995809592306614 - training/rollout_actor_probs_pearson_corr:0.9995769262313843 - rollout_corr/rollout_is_mean:0.9999872446060181 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0000299880630337e-05 - rollout_corr/rollout_is_max:1.43068265914917 - rollout_corr/rollout_is_min:0.059669289737939835 - rollout_corr/rollout_is_std:0.034765832126140594 - rollout_corr/rollout_is_eff_sample_size:0.9987927961158968 - rollout_corr/rollout_is_seq_mean:1.0000014305114746 - rollout_corr/rollout_is_seq_std:0.00208776886574924 - rollout_corr/rollout_is_seq_max:1.006962537765503 - rollout_corr/rollout_is_seq_min:0.9941145181655884 - rollout_corr/rollout_is_seq_max_deviation:0.00696253776550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0744992191903293) - rollout_corr/training_log_ppl:np.float64(0.7169467412386439) - rollout_corr/kl:np.float64(0.0007758290953425728) - rollout_corr/k3_kl:np.float64(0.0007976444751136569) - rollout_corr/rollout_ppl:np.float64(2.0729074915871024) - rollout_corr/rollout_log_ppl:np.float64(0.71617091196822) - rollout_corr/log_ppl_diff:np.float64(0.0007758292704238556) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019027811140404083) - rollout_corr/log_ppl_diff_max:np.float64(0.008478999137878418) - rollout_corr/log_ppl_diff_min:np.float64(-0.006452798843383789) - rollout_corr/ppl_ratio:np.float64(1.0007788352668285) - rollout_corr/chi2_token:np.float64(0.001621277304366231) - rollout_corr/chi2_seq:np.float64(0.7940898754168302) - actor/pg_loss:np.float64(-0.0001730915391817689) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00055276202465393) - actor/ppo_kl:np.float64(9.482850173014867e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4456499367952347) - perf/mfu/actor:np.float64(0.07777215994781624) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(103.74797439575195) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:3 - critic/score/mean:0.41796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.029245877638459206 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.029245877638459206 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:390.61328125 - response_length/max:8192.0 - response_length/min:139.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:390.61328125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:269.46875 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014945492148399353 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8304623439908028) - timing_s/agent_loop/generate_sequences/max:np.float64(59.264669789001346) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3520388190299855) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.264669789001346) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:184 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:67.29519834555686 - timing_s/reward:0.052218323573470116 - timing_s/old_log_prob:3.32750260643661 - timing_s/adv:0.11696512624621391 - timing_s/update_actor:14.175002483651042 - timing_s/step:85.08170239627361 - timing_s/stop_profile:0.00011155568063259125 - timing_per_token_ms/update_actor:0.08388518521994213 - timing_per_token_ms/gen:0.6729721726207472 - timing_per_token_ms/adv:0.0006921791576935508 - perf/total_num_tokens:168981 - perf/time_per_step:85.08170239627361 - perf/throughput:248.26283918979416 (TaskRunner pid=2347384) Training Progress: 46%|████▌ | 46/100 [50:29<1:03:15, 70.29s/it] (TaskRunner pid=2347384) step:47 - global_seqlen/min:20725 - global_seqlen/max:34120 - global_seqlen/minmax_diff:13395 - global_seqlen/balanced_min:25094 - global_seqlen/balanced_max:31559 - global_seqlen/mean:26145.5 - actor/entropy:0.546191394329071 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583375573158264 - training/rollout_probs_diff_mean:0.004138430114835501 - training/rollout_probs_diff_std:0.008594999089837074 - training/rollout_actor_probs_pearson_corr:0.9995757937431335 - rollout_corr/rollout_is_mean:0.9999886751174927 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7381517887115479 - rollout_corr/rollout_is_min:0.5752727389335632 - rollout_corr/rollout_is_std:0.03275854140520096 - rollout_corr/rollout_is_eff_sample_size:0.9989279093780362 - rollout_corr/rollout_is_seq_mean:0.9999130368232727 - rollout_corr/rollout_is_seq_std:0.002008902607485652 - rollout_corr/rollout_is_seq_max:1.004290223121643 - rollout_corr/rollout_is_seq_min:0.9934692978858948 - rollout_corr/rollout_is_seq_max_deviation:0.006530702114105225 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8914625435136259) - rollout_corr/training_log_ppl:np.float64(0.6221555507218) - rollout_corr/kl:np.float64(0.0005894855491597184) - rollout_corr/k3_kl:np.float64(0.0006707891554356138) - rollout_corr/rollout_ppl:np.float64(1.890320950653404) - rollout_corr/rollout_log_ppl:np.float64(0.6215660634625237) - rollout_corr/log_ppl_diff:np.float64(0.0005894872592762113) - rollout_corr/log_ppl_abs_diff:np.float64(0.001678252941928804) - rollout_corr/log_ppl_diff_max:np.float64(0.007238566875457764) - rollout_corr/log_ppl_diff_min:np.float64(-0.0041100382804870605) - rollout_corr/ppl_ratio:np.float64(1.0005917930975556) - rollout_corr/chi2_token:np.float64(0.0015165875665843487) - rollout_corr/chi2_seq:np.float64(0.7157283113338053) - actor/pg_loss:np.float64(-0.0003227494889870286) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023992822298168903) - actor/ppo_kl:np.float64(-0.00010562632341559741) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37839525192976) - perf/mfu/actor:np.float64(0.09708267992140404) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(103.68896484375) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:3 - critic/score/mean:0.390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.026096303015947342 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.026096303015947342 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:454.296875 - response_length/max:8192.0 - response_length/min:157.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:454.296875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:362.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.846260607242584e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.933855528011918) - timing_s/agent_loop/generate_sequences/max:np.float64(58.546155823394656) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.940962525368377) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.546155823394656) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.31403193995357 - timing_s/reward:0.06695490144193172 - timing_s/old_log_prob:3.3485095649957657 - timing_s/adv:0.1384153552353382 - timing_s/update_actor:13.936238881200552 - timing_s/step:82.88974814489484 - timing_s/stop_profile:0.00011194124817848206 - timing_per_token_ms/update_actor:0.06662828632652154 - timing_per_token_ms/gen:0.5615995867579843 - timing_per_token_ms/adv:0.0006617551549757042 - perf/total_num_tokens:209164 - perf/time_per_step:82.88974814489484 - perf/throughput:315.4250143732678 (TaskRunner pid=2347384) Training Progress: 47%|████▋ | 47/100 [51:52<1:05:26, 74.08s/it] (TaskRunner pid=2347384) step:48 - global_seqlen/min:18309 - global_seqlen/max:29930 - global_seqlen/minmax_diff:11621 - global_seqlen/balanced_min:23391 - global_seqlen/balanced_max:30028 - global_seqlen/mean:24536.75 - actor/entropy:0.5771048069000244 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9025864601135254 - training/rollout_probs_diff_mean:0.004265796393156052 - training/rollout_probs_diff_std:0.009076961316168308 - training/rollout_actor_probs_pearson_corr:0.9995290040969849 - rollout_corr/rollout_is_mean:1.0000274181365967 - rollout_corr/rollout_is_ratio_fraction_high:8.586492185713723e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.586492185713723e-06 - rollout_corr/rollout_is_max:2.0691685676574707 - rollout_corr/rollout_is_min:0.06241634115576744 - rollout_corr/rollout_is_std:0.033391084522008896 - rollout_corr/rollout_is_eff_sample_size:0.9988863962200344 - rollout_corr/rollout_is_seq_mean:1.0000274181365967 - rollout_corr/rollout_is_seq_std:0.001794546376913786 - rollout_corr/rollout_is_seq_max:1.005397915840149 - rollout_corr/rollout_is_seq_min:0.9948599934577942 - rollout_corr/rollout_is_seq_max_deviation:0.005397915840148926 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9582446995191276) - rollout_corr/training_log_ppl:np.float64(0.658810585460742) - rollout_corr/kl:np.float64(0.000730968798361431) - rollout_corr/k3_kl:np.float64(0.0006957861213834349) - rollout_corr/rollout_ppl:np.float64(1.956826860550791) - rollout_corr/rollout_log_ppl:np.float64(0.6580796112684766) - rollout_corr/log_ppl_diff:np.float64(0.0007309741922654212) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016826559440232813) - rollout_corr/log_ppl_diff_max:np.float64(0.007451772689819336) - rollout_corr/log_ppl_diff_min:np.float64(-0.005151212215423584) - rollout_corr/ppl_ratio:np.float64(1.0007332868408412) - rollout_corr/chi2_token:np.float64(0.001318385824561119) - rollout_corr/chi2_seq:np.float64(0.46904304763302207) - actor/pg_loss:np.float64(-0.0001319999573752284) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004603672068697051) - actor/ppo_kl:np.float64(0.0001262122454654957) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4332354739308357) - perf/mfu/actor:np.float64(0.09280957925379549) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(103.68746185302734) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:3 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.042791854590177536 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.042791854590177536 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:454.9296875 - response_length/max:8192.0 - response_length/min:164.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:454.9296875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:311.84375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012995675206184387 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9755186159163713) - timing_s/agent_loop/generate_sequences/max:np.float64(58.255842162296176) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.836520178461797) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.255842162296176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.0151872895658 - timing_s/reward:0.06391563452780247 - timing_s/old_log_prob:3.24536057934165 - timing_s/adv:0.12374880351126194 - timing_s/update_actor:13.649311082437634 - timing_s/step:82.18338214792311 - timing_s/stop_profile:0.00012098066508769989 - timing_per_token_ms/update_actor:0.06953503969778818 - timing_per_token_ms/gen:0.5582523680648263 - timing_per_token_ms/adv:0.0006304258077743688 - perf/total_num_tokens:196294 - perf/time_per_step:82.18338214792311 - perf/throughput:298.5609664474009 (TaskRunner pid=2347384) Training Progress: 48%|████▊ | 48/100 [53:14<1:06:19, 76.53s/it] (TaskRunner pid=2347384) step:49 - global_seqlen/min:19061 - global_seqlen/max:30005 - global_seqlen/minmax_diff:10944 - global_seqlen/balanced_min:22404 - global_seqlen/balanced_max:24976 - global_seqlen/mean:22761.375 - actor/entropy:0.6405560374259949 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1910906434059143 - training/rollout_probs_diff_mean:0.004676402546465397 - training/rollout_probs_diff_std:0.008850921876728535 - training/rollout_actor_probs_pearson_corr:0.9995667338371277 - rollout_corr/rollout_is_mean:1.0000193119049072 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.8814853319781832e-05 - rollout_corr/rollout_is_max:1.6065211296081543 - rollout_corr/rollout_is_min:0.16068048775196075 - rollout_corr/rollout_is_std:0.03496243804693222 - rollout_corr/rollout_is_eff_sample_size:0.9987792392325578 - rollout_corr/rollout_is_seq_mean:1.0000687837600708 - rollout_corr/rollout_is_seq_std:0.001924062380567193 - rollout_corr/rollout_is_seq_max:1.0058914422988892 - rollout_corr/rollout_is_seq_min:0.9947021007537842 - rollout_corr/rollout_is_seq_max_deviation:0.00589144229888916 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0206294134259224) - rollout_corr/training_log_ppl:np.float64(0.6906947079696693) - rollout_corr/kl:np.float64(0.0009516554765113261) - rollout_corr/k3_kl:np.float64(0.0007131432693086026) - rollout_corr/rollout_ppl:np.float64(2.018631271086633) - rollout_corr/rollout_log_ppl:np.float64(0.6897430497338064) - rollout_corr/log_ppl_diff:np.float64(0.0009516582358628511) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017689871601760387) - rollout_corr/log_ppl_diff_max:np.float64(0.006156444549560547) - rollout_corr/log_ppl_diff_min:np.float64(-0.004617631435394287) - rollout_corr/ppl_ratio:np.float64(1.0009541262406856) - rollout_corr/chi2_token:np.float64(0.0009525960776954889) - rollout_corr/chi2_seq:np.float64(0.27055922290310264) - actor/pg_loss:np.float64(-0.0005947427125647664) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003312197569584896) - actor/ppo_kl:np.float64(0.00035593098959907365) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3832462206482887) - perf/mfu/actor:np.float64(0.09048492943683159) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(103.68398666381836) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:3 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01670641265809536 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01670641265809536 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:415.23046875 - response_length/max:4041.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:415.23046875 - response_length_non_aborted/max:4041.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.8424032330513e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8784635066986084) - timing_s/agent_loop/generate_sequences/max:np.float64(26.004397485405207) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.704731465812074) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.004397485405207) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:4041 - timing_s/gen:32.565655291080475 - timing_s/reward:0.06145591288805008 - timing_s/old_log_prob:3.1043693106621504 - timing_s/adv:0.1213848628103733 - timing_s/update_actor:12.925205308943987 - timing_s/step:48.86529274098575 - timing_s/stop_profile:0.00011293217539787292 - timing_per_token_ms/update_actor:0.07098212052734065 - timing_per_token_ms/gen:0.30635899953038576 - timing_per_token_ms/adv:0.0006666164874176829 - perf/total_num_tokens:182091 - perf/time_per_step:48.86529274098575 - perf/throughput:465.7983964334036 (TaskRunner pid=2347384) Training Progress: 49%|████▉ | 49/100 [54:03<58:00, 68.24s/it] (TaskRunner pid=2347384) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2347384) validation generation end (TaskRunner pid=2347384) [prompt] system (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) To determine the folding stability score of a protein sequence, one must evaluate the balance between stabilizing and destabilizing interactions within the protein. Stabilizing interactions, such as hydrogen bonds, hydrophobic effects, and disulfide bonds, contribute positively to the folding stability score, while destabilizing interactions, such as excessive charge repulsion or lack of hydrophobic core formation, lower the score. (TaskRunner pid=2347384) (TaskRunner pid=2347384) The given sequence is: (TaskRunner pid=2347384) `GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR` (TaskRunner pid=2347384) (TaskRunner pid=2347384) This sequence contains a mix of charged and hydrophobic residues. The presence of multiple charged residues (e.g., D, E, R, K) suggests potential electrostatic interactions, which can either stabilize or destabilize the folded structure depending on their spatial arrangement. However, the sequence also contains hydrophobic residues (e.g., G, S, T, V, W, F), which can contribute to the formation of a hydrophobic core, a key stabilizing factor in protein folding. (TaskRunner pid=2347384) (TaskRunner pid=2347384) To calculate the folding stability score, one would typically use computational methods that simulate the folding process and evaluate the energy landscape of the protein. These simulations take into account factors like the burial of hydrophobic residues, the formation of hydrogen bonds, and the reduction of unfavorable interactions. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Based on the sequence composition, it is reasonable to infer that the folding stability score would not be extremely low (like -0.01) or extremely high (like 2.49). A score of 0.45 suggests a relatively low but still positive stability, while a score of 1.69 indicates moderate stability. Given the presence of both stabilizing and destabilizing residues, the most likely score would be 0.45, indicating a weakly stable structure with some favorable folding interactions. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) D (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 0.0 (TaskRunner pid=2347384) [acc] 0.0 (TaskRunner pid=2347384) [pred] D (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2347384) step:50 - global_seqlen/min:15798 - global_seqlen/max:26595 - global_seqlen/minmax_diff:10797 - global_seqlen/balanced_min:23164 - global_seqlen/balanced_max:23175 - global_seqlen/mean:23172.0 - actor/entropy:0.6356609463691711 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22906708717346191 - training/rollout_probs_diff_mean:0.004797298926860094 - training/rollout_probs_diff_std:0.009071983397006989 - training/rollout_actor_probs_pearson_corr:0.9995721578598022 - rollout_corr/rollout_is_mean:1.0001471042633057 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8442926406860352 - rollout_corr/rollout_is_min:0.6120340824127197 - rollout_corr/rollout_is_std:0.03550882637500763 - rollout_corr/rollout_is_eff_sample_size:0.9987411867895215 - rollout_corr/rollout_is_seq_mean:1.0002213716506958 - rollout_corr/rollout_is_seq_std:0.0020395577885210514 - rollout_corr/rollout_is_seq_max:1.0062882900238037 - rollout_corr/rollout_is_seq_min:0.9926304221153259 - rollout_corr/rollout_is_seq_max_deviation:0.007369577884674072 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.959245269652456) - rollout_corr/training_log_ppl:np.float64(0.660572504857555) - rollout_corr/kl:np.float64(0.0006797749082165794) - rollout_corr/k3_kl:np.float64(0.000713326107870671) - rollout_corr/rollout_ppl:np.float64(1.957892449107021) - rollout_corr/rollout_log_ppl:np.float64(0.6598927302984521) - rollout_corr/log_ppl_diff:np.float64(0.0006797745591029525) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017374403541907668) - rollout_corr/log_ppl_diff_max:np.float64(0.007385313510894775) - rollout_corr/log_ppl_diff_min:np.float64(-0.0051781535148620605) - rollout_corr/ppl_ratio:np.float64(1.0006822519935668) - rollout_corr/chi2_token:np.float64(0.0015025152824819088) - rollout_corr/chi2_seq:np.float64(0.7895834888331592) - actor/pg_loss:np.float64(-0.00022027932573109865) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004089342028237297) - actor/ppo_kl:np.float64(0.00023849693610777933) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.40693123638629913) - perf/mfu/actor:np.float64(0.09358700382629766) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(103.87079238891602) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.37625) - val-aux/sciknoweval/reward/std@16:np.float64(0.3149417869526249) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5122800000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2800374699000979) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.2419) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2551582988533073) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.37561999999999995) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.31500810894282244) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6240400000000002) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.21381166334732124) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.14386) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1710792137209249) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.39626000000000006) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2606780320108342) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7063600000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.14089073297353036) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.08278) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09575880296652768) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.41328000000000004) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.20326424584816394) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7609999999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.08080565001198156) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.0483) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05158976586003544) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.42574000000000006) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1520283086021903) - val-aux/sciknoweval/score/mean@16:np.float64(0.37625) - val-aux/sciknoweval/score/std@16:np.float64(0.3149417869526249) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5122800000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2800374699000979) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.2419) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2551582988533073) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.37561999999999995) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.31500810894282244) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6240400000000002) - val-aux/sciknoweval/score/best@4/std:np.float64(0.21381166334732124) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.14386) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1710792137209249) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.39626000000000006) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2606780320108342) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7063600000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.14089073297353036) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.08278) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09575880296652768) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.41328000000000004) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.20326424584816394) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7609999999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.08080565001198156) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.0483) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05158976586003544) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.42574000000000006) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1520283086021903) - val-core/sciknoweval/acc/mean@16:np.float64(0.37625) - val-aux/sciknoweval/acc/std@16:np.float64(0.3149417869526249) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5122800000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2800374699000979) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.2419) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2551582988533073) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.37561999999999995) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.31500810894282244) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6240400000000002) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.21381166334732124) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.14386) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1710792137209249) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.39626000000000006) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2606780320108342) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7063600000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.14089073297353036) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.08278) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09575880296652768) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.41328000000000004) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.20326424584816394) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7609999999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.08080565001198156) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.0483) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05158976586003544) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.42574000000000006) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1520283086021903) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04997931737725824) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99748) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.018656957662192008) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.96442) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06469300105372805) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9809399999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0505567893098151) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0006321392251711646) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.93878) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07486034891528723) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9938600000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.029236546024091266) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9034800000000002) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07357355563508784) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99866) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.01254092925926912) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.86922) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05557894999568232) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99988) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002526657157387089) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:3 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008166423998773098 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008166423998773098 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:391.8125 - response_length/max:1183.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:391.8125 - response_length_non_aborted/max:1183.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.617581963539124e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9893430918455124) - timing_s/agent_loop/generate_sequences/max:np.float64(9.523660020902753) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.396290250122547) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.523660020902753) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1183 - timing_s/gen:16.292969847097993 - timing_s/reward:0.06562184356153011 - timing_s/old_log_prob:2.9420079346746206 - timing_s/adv:0.1449713110923767 - timing_s/update_actor:12.90351471118629 - timing_s/step:32.449002681300044 - timing_s/testing:90.13463921286166 - timing_s/save_checkpoint:12.586417723447084 - timing_s/stop_profile:0.0001592729240655899 - timing_per_token_ms/update_actor:0.06960725612369612 - timing_per_token_ms/gen:0.1624358933551802 - timing_per_token_ms/adv:0.0007820392666384899 - perf/total_num_tokens:185376 - perf/time_per_step:32.449002681300044 - perf/throughput:714.1051522472133 (TaskRunner pid=2347384) Training Progress: 50%|█████ | 50/100 [56:19<1:13:37, 88.35s/it] (TaskRunner pid=2347384) step:51 - global_seqlen/min:19382 - global_seqlen/max:27616 - global_seqlen/minmax_diff:8234 - global_seqlen/balanced_min:23599 - global_seqlen/balanced_max:23606 - global_seqlen/mean:23604.25 - actor/entropy:0.6392678618431091 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.14446893334388733 - training/rollout_probs_diff_mean:0.004770862404257059 - training/rollout_probs_diff_std:0.008967405185103416 - training/rollout_actor_probs_pearson_corr:0.999556303024292 - rollout_corr/rollout_is_mean:0.999977707862854 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.625535011291504 - rollout_corr/rollout_is_min:0.5792540907859802 - rollout_corr/rollout_is_std:0.03496755287051201 - rollout_corr/rollout_is_eff_sample_size:0.9987787635591238 - rollout_corr/rollout_is_seq_mean:1.0000072717666626 - rollout_corr/rollout_is_seq_std:0.001823716564103961 - rollout_corr/rollout_is_seq_max:1.0050665140151978 - rollout_corr/rollout_is_seq_min:0.994660496711731 - rollout_corr/rollout_is_seq_max_deviation:0.005339503288269043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9575814469717443) - rollout_corr/training_log_ppl:np.float64(0.6596837055403739) - rollout_corr/kl:np.float64(0.0007464144207958157) - rollout_corr/k3_kl:np.float64(0.0007177888666092258) - rollout_corr/rollout_ppl:np.float64(1.9560449752025306) - rollout_corr/rollout_log_ppl:np.float64(0.6589372934540734) - rollout_corr/log_ppl_diff:np.float64(0.0007464120863005519) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015787462471053004) - rollout_corr/log_ppl_diff_max:np.float64(0.00665968656539917) - rollout_corr/log_ppl_diff_min:np.float64(-0.003551959991455078) - rollout_corr/ppl_ratio:np.float64(1.0007483852095902) - rollout_corr/chi2_token:np.float64(0.0013662567362189293) - rollout_corr/chi2_seq:np.float64(0.3143873498775065) - actor/pg_loss:np.float64(-5.5046286433935165e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005373947194584616) - actor/ppo_kl:np.float64(0.00011551689045319335) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.42167578637599945) - perf/mfu/actor:np.float64(0.09487652336910979) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(107.66786479949951) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:3 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003804941661655903 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003804941661655903 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:407.5703125 - response_length/max:1151.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:407.5703125 - response_length_non_aborted/max:1151.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.0625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017059221863746643 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1205009147524834) - timing_s/agent_loop/generate_sequences/max:np.float64(9.291227325797081) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.643093930448231) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.291227325797081) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1151 - timing_s/gen:15.901783429086208 - timing_s/reward:0.056893499568104744 - timing_s/old_log_prob:3.2239135056734085 - timing_s/adv:0.13362528383731842 - timing_s/update_actor:12.872088333591819 - timing_s/step:32.28387093171477 - timing_s/stop_profile:0.00015329942107200623 - timing_per_token_ms/update_actor:0.06816615828501127 - timing_per_token_ms/gen:0.1524064428021067 - timing_per_token_ms/adv:0.0007076336032563967 - perf/total_num_tokens:188834 - perf/time_per_step:32.28387093171477 - perf/throughput:731.1468333498956 (TaskRunner pid=2347384) Training Progress: 51%|█████ | 51/100 [56:51<58:25, 71.54s/it] (TaskRunner pid=2347384) step:52 - global_seqlen/min:18215 - global_seqlen/max:24612 - global_seqlen/minmax_diff:6397 - global_seqlen/balanced_min:21227 - global_seqlen/balanced_max:21336 - global_seqlen/mean:21249.25 - actor/entropy:0.6677834391593933 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17890137434005737 - training/rollout_probs_diff_mean:0.005169688258320093 - training/rollout_probs_diff_std:0.00939327385276556 - training/rollout_actor_probs_pearson_corr:0.9995176792144775 - rollout_corr/rollout_is_mean:0.9997798800468445 - rollout_corr/rollout_is_ratio_fraction_high:1.0146721251658164e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.256162643432617 - rollout_corr/rollout_is_min:0.6178552508354187 - rollout_corr/rollout_is_std:0.03671131283044815 - rollout_corr/rollout_is_eff_sample_size:0.9986535582847357 - rollout_corr/rollout_is_seq_mean:0.999818742275238 - rollout_corr/rollout_is_seq_std:0.0019563911482691765 - rollout_corr/rollout_is_seq_max:1.0066373348236084 - rollout_corr/rollout_is_seq_min:0.9947150945663452 - rollout_corr/rollout_is_seq_max_deviation:0.0066373348236083984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9809529050253332) - rollout_corr/training_log_ppl:np.float64(0.6739747864776291) - rollout_corr/kl:np.float64(0.0008196673196678717) - rollout_corr/k3_kl:np.float64(0.0007884356174372442) - rollout_corr/rollout_ppl:np.float64(1.979280007071793) - rollout_corr/rollout_log_ppl:np.float64(0.6731551115517505) - rollout_corr/log_ppl_diff:np.float64(0.000819674925878644) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018322267569601536) - rollout_corr/log_ppl_diff_max:np.float64(0.00656890869140625) - rollout_corr/log_ppl_diff_min:np.float64(-0.005955934524536133) - rollout_corr/ppl_ratio:np.float64(1.0008223119657487) - rollout_corr/chi2_token:np.float64(0.0014989387709647417) - rollout_corr/chi2_seq:np.float64(0.612495215376839) - actor/pg_loss:np.float64(0.00021331990137696266) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003496147446639952) - actor/ppo_kl:np.float64(-4.643269364512648e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3523213863372803) - perf/mfu/actor:np.float64(0.08601763801628917) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.16965675354004) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:3 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009626702405512333 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009626702405512333 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:384.9765625 - response_length/max:1171.0 - response_length/min:182.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:384.9765625 - response_length_non_aborted/max:1171.0 - response_length_non_aborted/min:182.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.0625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017726048827171326 - timing_s/agent_loop/generate_sequences/min:np.float64(2.171152727678418) - timing_s/agent_loop/generate_sequences/max:np.float64(9.38106920197606) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.427015884917637) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.38106920197606) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1171 - timing_s/gen:16.18165148049593 - timing_s/reward:0.0594619233161211 - timing_s/old_log_prob:3.101227018982172 - timing_s/adv:0.12002318911254406 - timing_s/update_actor:12.550954319536686 - timing_s/step:32.100309908390045 - timing_s/stop_profile:0.00010958872735500336 - timing_per_token_ms/update_actor:0.0738317488825293 - timing_per_token_ms/gen:0.1641907125078224 - timing_per_token_ms/adv:0.0007060436786742124 - perf/total_num_tokens:169994 - perf/time_per_step:32.100309908390045 - perf/throughput:661.9640140747081 (TaskRunner pid=2347384) Training Progress: 52%|█████▏ | 52/100 [57:23<47:46, 59.73s/it] (TaskRunner pid=2347384) step:53 - global_seqlen/min:19153 - global_seqlen/max:28485 - global_seqlen/minmax_diff:9332 - global_seqlen/balanced_min:24316 - global_seqlen/balanced_max:24343 - global_seqlen/mean:24336.125 - actor/entropy:0.5912443995475769 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5450356006622314 - training/rollout_probs_diff_mean:0.004726866725832224 - training/rollout_probs_diff_std:0.009441981092095375 - training/rollout_actor_probs_pearson_corr:0.9994943737983704 - rollout_corr/rollout_is_mean:1.0000923871994019 - rollout_corr/rollout_is_ratio_fraction_high:9.124670214077923e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.124670214077923e-06 - rollout_corr/rollout_is_max:2.0695583820343018 - rollout_corr/rollout_is_min:0.3888539969921112 - rollout_corr/rollout_is_std:0.03535068407654762 - rollout_corr/rollout_is_eff_sample_size:0.9987521265699173 - rollout_corr/rollout_is_seq_mean:1.0000860691070557 - rollout_corr/rollout_is_seq_std:0.001946592121385038 - rollout_corr/rollout_is_seq_max:1.0063363313674927 - rollout_corr/rollout_is_seq_min:0.9949162602424622 - rollout_corr/rollout_is_seq_max_deviation:0.006336331367492676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8581669991835952) - rollout_corr/training_log_ppl:np.float64(0.6067303767194971) - rollout_corr/kl:np.float64(0.0005786082886274357) - rollout_corr/k3_kl:np.float64(0.0007370528182377711) - rollout_corr/rollout_ppl:np.float64(1.857081515248865) - rollout_corr/rollout_log_ppl:np.float64(0.6061517667258158) - rollout_corr/log_ppl_diff:np.float64(0.0005786099936813116) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015172131825238466) - rollout_corr/log_ppl_diff_max:np.float64(0.006741583347320557) - rollout_corr/log_ppl_diff_min:np.float64(-0.005707681179046631) - rollout_corr/ppl_ratio:np.float64(1.0005806684494019) - rollout_corr/chi2_token:np.float64(0.0017955824732780457) - rollout_corr/chi2_seq:np.float64(0.8056872091256082) - actor/pg_loss:np.float64(4.05055470764637e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005777111191491713) - actor/ppo_kl:np.float64(4.964845899735337e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.338901124894619) - perf/mfu/actor:np.float64(0.09806070279864) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.09315490722656) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:3 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008918224833905697 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008918224833905697 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:428.09765625 - response_length/max:1420.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:428.09765625 - response_length_non_aborted/max:1420.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.40625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012840516865253448 - timing_s/agent_loop/generate_sequences/min:np.float64(1.90303741581738) - timing_s/agent_loop/generate_sequences/max:np.float64(11.140737239271402) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.837989689585811) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.140737239271402) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1420 - timing_s/gen:17.92714038863778 - timing_s/reward:0.06386916898190975 - timing_s/old_log_prob:3.0421290788799524 - timing_s/adv:0.12108439579606056 - timing_s/update_actor:12.841582082211971 - timing_s/step:34.08444775082171 - timing_s/stop_profile:0.00011683255434036255 - timing_per_token_ms/update_actor:0.06595946397696825 - timing_per_token_ms/gen:0.16357924674603105 - timing_per_token_ms/adv:0.000621937530091893 - perf/total_num_tokens:194689 - perf/time_per_step:34.08444775082171 - perf/throughput:713.9949920242819 (TaskRunner pid=2347384) Training Progress: 53%|█████▎ | 53/100 [57:57<40:46, 52.06s/it] (TaskRunner pid=2347384) step:54 - global_seqlen/min:17561 - global_seqlen/max:32252 - global_seqlen/minmax_diff:14691 - global_seqlen/balanced_min:23155 - global_seqlen/balanced_max:29915 - global_seqlen/mean:24014.625 - actor/entropy:0.5520281195640564 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34774380922317505 - training/rollout_probs_diff_mean:0.004567669704556465 - training/rollout_probs_diff_std:0.009347373619675636 - training/rollout_actor_probs_pearson_corr:0.999484121799469 - rollout_corr/rollout_is_mean:1.0000128746032715 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.610223630967084e-06 - rollout_corr/rollout_is_max:1.6269819736480713 - rollout_corr/rollout_is_min:0.37064963579177856 - rollout_corr/rollout_is_std:0.03476754575967789 - rollout_corr/rollout_is_eff_sample_size:0.9987926771942676 - rollout_corr/rollout_is_seq_mean:1.000026822090149 - rollout_corr/rollout_is_seq_std:0.0019094093004241586 - rollout_corr/rollout_is_seq_max:1.0044440031051636 - rollout_corr/rollout_is_seq_min:0.9933951497077942 - rollout_corr/rollout_is_seq_max_deviation:0.0066048502922058105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8538824222050607) - rollout_corr/training_log_ppl:np.float64(0.6052331507526105) - rollout_corr/kl:np.float64(0.0007334417485118649) - rollout_corr/k3_kl:np.float64(0.0007262715936349196) - rollout_corr/rollout_ppl:np.float64(1.8525602836161852) - rollout_corr/rollout_log_ppl:np.float64(0.6044997070275713) - rollout_corr/log_ppl_diff:np.float64(0.0007334437250392511) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017532818455947563) - rollout_corr/log_ppl_diff_max:np.float64(0.007780998945236206) - rollout_corr/log_ppl_diff_min:np.float64(-0.0056771934032440186) - rollout_corr/ppl_ratio:np.float64(1.0007360137533396) - rollout_corr/chi2_token:np.float64(0.0014576194807887077) - rollout_corr/chi2_seq:np.float64(1.1083380549680442) - actor/pg_loss:np.float64(1.718697603791952e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027073693217971595) - actor/ppo_kl:np.float64(7.857253805276798e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3225150853395462) - perf/mfu/actor:np.float64(0.09037306999521552) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(103.98453140258789) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:3 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.05943207070231438 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.05943207070231438 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:453.67578125 - response_length/max:8192.0 - response_length/min:170.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:453.67578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:170.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:296.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001210235059261322 - timing_s/agent_loop/generate_sequences/min:np.float64(2.048864269629121) - timing_s/agent_loop/generate_sequences/max:np.float64(59.003117674961686) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.114247565550613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.003117674961686) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.83943621441722 - timing_s/reward:0.06302537955343723 - timing_s/old_log_prob:3.3005123753100634 - timing_s/adv:0.12873866967856884 - timing_s/update_actor:13.856510968878865 - timing_s/step:83.27658375538886 - timing_s/stop_profile:0.00014846399426460266 - timing_per_token_ms/update_actor:0.07212537656156855 - timing_per_token_ms/gen:0.5668922793364722 - timing_per_token_ms/adv:0.0006701055590008632 - perf/total_num_tokens:192117 - perf/time_per_step:83.27658375538886 - perf/throughput:288.3718797896294 (TaskRunner pid=2347384) Training Progress: 54%|█████▍ | 54/100 [59:21<47:06, 61.44s/it] (TaskRunner pid=2347384) step:55 - global_seqlen/min:18325 - global_seqlen/max:29759 - global_seqlen/minmax_diff:11434 - global_seqlen/balanced_min:24643 - global_seqlen/balanced_max:24656 - global_seqlen/mean:24652.25 - actor/entropy:0.6070672273635864 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2475433647632599 - training/rollout_probs_diff_mean:0.004747605882585049 - training/rollout_probs_diff_std:0.009215133264660835 - training/rollout_actor_probs_pearson_corr:0.999550998210907 - rollout_corr/rollout_is_mean:0.9999242424964905 - rollout_corr/rollout_is_ratio_fraction_high:8.880836503522005e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.880836503522005e-06 - rollout_corr/rollout_is_max:2.243964195251465 - rollout_corr/rollout_is_min:0.43895408511161804 - rollout_corr/rollout_is_std:0.03562445938587189 - rollout_corr/rollout_is_eff_sample_size:0.9987323875748504 - rollout_corr/rollout_is_seq_mean:0.9999244213104248 - rollout_corr/rollout_is_seq_std:0.0019231056794524193 - rollout_corr/rollout_is_seq_max:1.0114800930023193 - rollout_corr/rollout_is_seq_min:0.9941225647926331 - rollout_corr/rollout_is_seq_max_deviation:0.011480093002319336 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9015162643045187) - rollout_corr/training_log_ppl:np.float64(0.6316752466373146) - rollout_corr/kl:np.float64(0.0009169320378272783) - rollout_corr/k3_kl:np.float64(0.000735976497367119) - rollout_corr/rollout_ppl:np.float64(1.8997333198785782) - rollout_corr/rollout_log_ppl:np.float64(0.6307583111338317) - rollout_corr/log_ppl_diff:np.float64(0.0009169355034828186) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016291378997266293) - rollout_corr/log_ppl_diff_max:np.float64(0.007391035556793213) - rollout_corr/log_ppl_diff_min:np.float64(-0.008635938167572021) - rollout_corr/ppl_ratio:np.float64(1.0009192482102662) - rollout_corr/chi2_token:np.float64(0.0011360023636370897) - rollout_corr/chi2_seq:np.float64(0.9624351807869971) - actor/pg_loss:np.float64(-8.239713497459888e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022346916739479639) - actor/ppo_kl:np.float64(0.00015720206336777842) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3877424970269203) - perf/mfu/actor:np.float64(0.09980069748066714) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.0140380859375) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:3 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014694455079734325 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014694455079734325 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.8515625 - response_length/max:1282.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.8515625 - response_length_non_aborted/max:1282.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:498.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010054372251033783 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3944042921066284) - timing_s/agent_loop/generate_sequences/max:np.float64(10.38448597677052) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.044274065250647) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.38448597677052) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1282 - timing_s/gen:17.20891674607992 - timing_s/reward:0.07135780528187752 - timing_s/old_log_prob:3.054533002898097 - timing_s/adv:0.11768579855561256 - timing_s/update_actor:12.718747563660145 - timing_s/step:33.26628116518259 - timing_s/stop_profile:0.00011440552771091461 - timing_per_token_ms/update_actor:0.06449080491466369 - timing_per_token_ms/gen:0.15282958336512603 - timing_per_token_ms/adv:0.0005967295001247987 - perf/total_num_tokens:197218 - perf/time_per_step:33.26628116518259 - perf/throughput:741.0581867444122 (TaskRunner pid=2347384) Training Progress: 55%|█████▌ | 55/100 [59:54<39:45, 53.00s/it] (TaskRunner pid=2347384) step:56 - global_seqlen/min:18313 - global_seqlen/max:31738 - global_seqlen/minmax_diff:13425 - global_seqlen/balanced_min:24983 - global_seqlen/balanced_max:27049 - global_seqlen/mean:25287.375 - actor/entropy:0.6156255602836609 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2897600531578064 - training/rollout_probs_diff_mean:0.004696155432611704 - training/rollout_probs_diff_std:0.00904381275177002 - training/rollout_actor_probs_pearson_corr:0.9995206594467163 - rollout_corr/rollout_is_mean:0.9999057054519653 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.324245754920412e-06 - rollout_corr/rollout_is_max:1.6316847801208496 - rollout_corr/rollout_is_min:0.447075217962265 - rollout_corr/rollout_is_std:0.03502035513520241 - rollout_corr/rollout_is_eff_sample_size:0.998774958187962 - rollout_corr/rollout_is_seq_mean:0.9998769164085388 - rollout_corr/rollout_is_seq_std:0.001785187516361475 - rollout_corr/rollout_is_seq_max:1.0063698291778564 - rollout_corr/rollout_is_seq_min:0.995042622089386 - rollout_corr/rollout_is_seq_max_deviation:0.006369829177856445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9300051955506206) - rollout_corr/training_log_ppl:np.float64(0.6460484475246631) - rollout_corr/kl:np.float64(0.0007186015388320754) - rollout_corr/k3_kl:np.float64(0.0007027769499359238) - rollout_corr/rollout_ppl:np.float64(1.9286025040782988) - rollout_corr/rollout_log_ppl:np.float64(0.6453298433916643) - rollout_corr/log_ppl_diff:np.float64(0.0007186041329987347) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015268662828020751) - rollout_corr/log_ppl_diff_max:np.float64(0.005717098712921143) - rollout_corr/log_ppl_diff_min:np.float64(-0.005181849002838135) - rollout_corr/ppl_ratio:np.float64(1.0007205524016172) - rollout_corr/chi2_token:np.float64(0.0013860114850103855) - rollout_corr/chi2_seq:np.float64(0.8416184603702277) - actor/pg_loss:np.float64(3.6787823773920536e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017812409623729764) - actor/ppo_kl:np.float64(-6.579846164633807e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37728193402290344) - perf/mfu/actor:np.float64(0.10006187263665729) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.04159164428711) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:3 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0035013859160244465 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0035013859160244465 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:469.26171875 - response_length/max:3821.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:469.26171875 - response_length_non_aborted/max:3821.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:320.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.054131805896759e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9092184081673622) - timing_s/agent_loop/generate_sequences/max:np.float64(24.566567834466696) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.309498217960936) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.566567834466696) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:3821 - timing_s/gen:31.656500335782766 - timing_s/reward:0.06610700115561485 - timing_s/old_log_prob:3.1339143197983503 - timing_s/adv:0.11929569207131863 - timing_s/update_actor:12.975913612172008 - timing_s/step:48.03798942826688 - timing_s/stop_profile:0.00010988861322402954 - timing_per_token_ms/update_actor:0.06414225286418622 - timing_per_token_ms/gen:0.2635164972886496 - timing_per_token_ms/adv:0.0005896998604605986 - perf/total_num_tokens:202299 - perf/time_per_step:48.03798942826688 - perf/throughput:526.4036921811763 (TaskRunner pid=2347384) Training Progress: 56%|█████▌ | 56/100 [1:00:42<37:47, 51.53s/it] (TaskRunner pid=2347384) step:57 - global_seqlen/min:22638 - global_seqlen/max:28865 - global_seqlen/minmax_diff:6227 - global_seqlen/balanced_min:25125 - global_seqlen/balanced_max:25447 - global_seqlen/mean:25174.25 - actor/entropy:0.6356367468833923 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26699820160865784 - training/rollout_probs_diff_mean:0.004728567320853472 - training/rollout_probs_diff_std:0.009011255577206612 - training/rollout_actor_probs_pearson_corr:0.9995681047439575 - rollout_corr/rollout_is_mean:1.0001047849655151 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7189937829971313 - rollout_corr/rollout_is_min:0.5848475098609924 - rollout_corr/rollout_is_std:0.03533213213086128 - rollout_corr/rollout_is_eff_sample_size:0.9987534346031791 - rollout_corr/rollout_is_seq_mean:1.000051498413086 - rollout_corr/rollout_is_seq_std:0.001815226161852479 - rollout_corr/rollout_is_seq_max:1.0048424005508423 - rollout_corr/rollout_is_seq_min:0.99420166015625 - rollout_corr/rollout_is_seq_max_deviation:0.00579833984375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9671564442105591) - rollout_corr/training_log_ppl:np.float64(0.6644838369684294) - rollout_corr/kl:np.float64(0.0007726970619188478) - rollout_corr/k3_kl:np.float64(0.0007176388960488111) - rollout_corr/rollout_ppl:np.float64(1.965628553647548) - rollout_corr/rollout_log_ppl:np.float64(0.663711137836799) - rollout_corr/log_ppl_diff:np.float64(0.0007726991316303611) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016954998718574643) - rollout_corr/log_ppl_diff_max:np.float64(0.006702065467834473) - rollout_corr/log_ppl_diff_min:np.float64(-0.004583835601806641) - rollout_corr/ppl_ratio:np.float64(1.0007751293014735) - rollout_corr/chi2_token:np.float64(0.0013350171502679586) - rollout_corr/chi2_seq:np.float64(0.8401855137199163) - actor/pg_loss:np.float64(1.752958633005619e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036715206033477443) - actor/ppo_kl:np.float64(0.00016723765993731377) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.39439400285482407) - perf/mfu/actor:np.float64(0.10120472404547595) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.08139419555664) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:4 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0065002902410924435 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0065002902410924435 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:471.1328125 - response_length/max:1989.0 - response_length/min:179.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:471.1328125 - response_length_non_aborted/max:1989.0 - response_length_non_aborted/min:179.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.5625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00023532472550868988 - timing_s/agent_loop/generate_sequences/min:np.float64(2.268577501177788) - timing_s/agent_loop/generate_sequences/max:np.float64(14.693121699616313) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.465190599577909) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.693121699616313) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1989 - timing_s/gen:21.458615578711033 - timing_s/reward:0.059584351256489754 - timing_s/old_log_prob:3.274692514911294 - timing_s/adv:0.15362177230417728 - timing_s/update_actor:12.915594087913632 - timing_s/step:37.963578660041094 - timing_s/stop_profile:0.00012173876166343689 - timing_per_token_ms/update_actor:0.06413097752621047 - timing_per_token_ms/gen:0.17791738312503966 - timing_per_token_ms/adv:0.0007627921998876693 - perf/total_num_tokens:201394 - perf/time_per_step:37.963578660041094 - perf/throughput:663.1158307132247 (TaskRunner pid=2347384) Training Progress: 57%|█████▋ | 57/100 [1:01:20<34:06, 47.60s/it] (TaskRunner pid=2347384) step:58 - global_seqlen/min:18787 - global_seqlen/max:30113 - global_seqlen/minmax_diff:11326 - global_seqlen/balanced_min:25213 - global_seqlen/balanced_max:25223 - global_seqlen/mean:25218.25 - actor/entropy:0.6536850929260254 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18439441919326782 - training/rollout_probs_diff_mean:0.004884521476924419 - training/rollout_probs_diff_std:0.009191151708364487 - training/rollout_actor_probs_pearson_corr:0.9995525479316711 - rollout_corr/rollout_is_mean:1.0000460147857666 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6425801515579224 - rollout_corr/rollout_is_min:0.6226503849029541 - rollout_corr/rollout_is_std:0.0359061062335968 - rollout_corr/rollout_is_eff_sample_size:0.9987124115549366 - rollout_corr/rollout_is_seq_mean:1.0001271963119507 - rollout_corr/rollout_is_seq_std:0.0018013427034020424 - rollout_corr/rollout_is_seq_max:1.006547451019287 - rollout_corr/rollout_is_seq_min:0.9956664443016052 - rollout_corr/rollout_is_seq_max_deviation:0.006547451019287109 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9582122191786766) - rollout_corr/training_log_ppl:np.float64(0.6608518530847505) - rollout_corr/kl:np.float64(0.0005865323429432578) - rollout_corr/k3_kl:np.float64(0.0007301697299908483) - rollout_corr/rollout_ppl:np.float64(1.957027964759618) - rollout_corr/rollout_log_ppl:np.float64(0.6602653161389753) - rollout_corr/log_ppl_diff:np.float64(0.0005865369457751513) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016829436644911766) - rollout_corr/log_ppl_diff_max:np.float64(0.006009936332702637) - rollout_corr/log_ppl_diff_min:np.float64(-0.006467998027801514) - rollout_corr/ppl_ratio:np.float64(1.0005888703744859) - rollout_corr/chi2_token:np.float64(0.0017614297103136778) - rollout_corr/chi2_seq:np.float64(1.7256589946337044) - actor/pg_loss:np.float64(0.00024261465296149254) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00030057721528464754) - actor/ppo_kl:np.float64(3.630212307470515e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.34353265166282654) - perf/mfu/actor:np.float64(0.10136190630720349) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.04206848144531) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:4 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:475.5703125 - response_length/max:1138.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:475.5703125 - response_length_non_aborted/max:1138.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.758397936820984e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6111684869974852) - timing_s/agent_loop/generate_sequences/max:np.float64(9.551169227808714) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.126937979643117) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.551169227808714) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1112 - timing_s/gen:16.2585074249655 - timing_s/reward:0.06692283600568771 - timing_s/old_log_prob:3.0540560856461525 - timing_s/adv:0.12516210600733757 - timing_s/update_actor:12.771576290950179 - timing_s/step:32.364589558914304 - timing_s/stop_profile:6.457418203353882e-05 - timing_per_token_ms/update_actor:0.06330522682457237 - timing_per_token_ms/gen:0.13354448955173476 - timing_per_token_ms/adv:0.0006203944861724028 - perf/total_num_tokens:201746 - perf/time_per_step:32.364589558914304 - perf/throughput:779.1926405893208 (TaskRunner pid=2347384) Training Progress: 58%|█████▊ | 58/100 [1:01:53<30:07, 43.04s/it] (TaskRunner pid=2347384) step:59 - global_seqlen/min:23052 - global_seqlen/max:37407 - global_seqlen/minmax_diff:14355 - global_seqlen/balanced_min:26442 - global_seqlen/balanced_max:28701 - global_seqlen/mean:27068.75 - actor/entropy:0.5959640145301819 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2920418381690979 - training/rollout_probs_diff_mean:0.004463896621018648 - training/rollout_probs_diff_std:0.008969206362962723 - training/rollout_actor_probs_pearson_corr:0.999552845954895 - rollout_corr/rollout_is_mean:1.0000066757202148 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5120586795092095e-05 - rollout_corr/rollout_is_max:1.860863208770752 - rollout_corr/rollout_is_min:0.3381444215774536 - rollout_corr/rollout_is_std:0.03402763232588768 - rollout_corr/rollout_is_eff_sample_size:0.998843459305735 - rollout_corr/rollout_is_seq_mean:1.000009298324585 - rollout_corr/rollout_is_seq_std:0.001725348294712603 - rollout_corr/rollout_is_seq_max:1.0050756931304932 - rollout_corr/rollout_is_seq_min:0.9946471452713013 - rollout_corr/rollout_is_seq_max_deviation:0.0053528547286987305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9399731773883104) - rollout_corr/training_log_ppl:np.float64(0.649218359583756) - rollout_corr/kl:np.float64(0.000773638378980479) - rollout_corr/k3_kl:np.float64(0.0006697148989474044) - rollout_corr/rollout_ppl:np.float64(1.9384577758610249) - rollout_corr/rollout_log_ppl:np.float64(0.6484447162947617) - rollout_corr/log_ppl_diff:np.float64(0.000773643288994208) - rollout_corr/log_ppl_abs_diff:np.float64(0.001426163042197004) - rollout_corr/log_ppl_diff_max:np.float64(0.007445991039276123) - rollout_corr/log_ppl_diff_min:np.float64(-0.004075765609741211) - rollout_corr/ppl_ratio:np.float64(1.0007753937970847) - rollout_corr/chi2_token:np.float64(0.001131066819652915) - rollout_corr/chi2_seq:np.float64(0.42264019534923136) - actor/pg_loss:np.float64(4.913005977869034e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023999804159302585) - actor/ppo_kl:np.float64(0.00015063014610561254) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29279158264398575) - perf/mfu/actor:np.float64(0.10514635279833859) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.06935119628906) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:4 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004941596649587154 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004941596649587154 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:516.6796875 - response_length/max:4028.0 - response_length/min:204.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:516.6796875 - response_length_non_aborted/max:4028.0 - response_length_non_aborted/min:204.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:329.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.350261926651001e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3985396195203066) - timing_s/agent_loop/generate_sequences/max:np.float64(26.399093862622976) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.683440201057238) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.399093862622976) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:4028 - timing_s/gen:33.1715356875211 - timing_s/reward:0.06764680333435535 - timing_s/old_log_prob:3.159480446949601 - timing_s/adv:0.12089948542416096 - timing_s/update_actor:13.217570504173636 - timing_s/step:49.82458985969424 - timing_s/stop_profile:0.00011782906949520111 - timing_per_token_ms/update_actor:0.06103703765492328 - timing_per_token_ms/gen:0.25078654031542374 - timing_per_token_ms/adv:0.0005582982471676794 - perf/total_num_tokens:216550 - perf/time_per_step:49.82458985969424 - perf/throughput:543.2809397172249 (TaskRunner pid=2347384) Training Progress: 59%|█████▉ | 59/100 [1:02:43<30:48, 45.09s/it] (TaskRunner pid=2347384) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2347384) validation generation end (TaskRunner pid=2347384) [prompt] system (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) To determine the folding stability score of a protein sequence, we need to evaluate the balance between hydrophobic and hydrophilic interactions, as well as the overall structural potential of the protein. The folding stability score typically reflects how likely the protein is to fold into a stable, functional 3D structure. This can be calculated using computational methods that analyze the amino acid composition, secondary structure propensity, and energy contributions from interactions. (TaskRunner pid=2347384) (TaskRunner pid=2347384) The given sequence is: (TaskRunner pid=2347384) **GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR** (TaskRunner pid=2347384) (TaskRunner pid=2347384) Analyzing this sequence, we observe the following: (TaskRunner pid=2347384) - It contains a mix of hydrophobic (e.g., G, S, T, R, A, V, W) and hydrophilic (e.g., D, E, R, K, Q, N) amino acids. (TaskRunner pid=2347384) - The presence of multiple charged residues (D, E, R, K) suggests potential for salt bridges and electrostatic interactions, which can stabilize the folded structure. (TaskRunner pid=2347384) - The sequence also includes several repeated residues (e.g., R, A, E), which may contribute to structural rigidity or flexibility depending on their context. (TaskRunner pid=2347384) - The presence of tryptophan (W) and tyrosine (Y) may enhance hydrophobic core formation and contribute to stability. (TaskRunner pid=2347384) (TaskRunner pid=2347384) However, the sequence also contains a high number of charged residues and polar residues, which may introduce destabilizing interactions if not properly balanced. Computational folding algorithms (like DSSP, FoldX, or Rosetta) can estimate the folding stability score by calculating the energy of the folded state relative to the unfolded state. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Based on the general trend that sequences with a balanced mix of hydrophobic and hydrophilic residues, along with favorable secondary structure propensities, tend to have higher folding stability scores, and given the presence of both stabilizing and destabilizing elements in this sequence, the folding stability score is expected to be moderate but positive. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Among the given options, the most reasonable value that aligns with this expectation is **B: 1.69**, as it reflects a moderate to high folding stability score, consistent with the sequence's structural potential. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) B (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 1.0 (TaskRunner pid=2347384) [acc] 1.0 (TaskRunner pid=2347384) [pred] B (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2347384) step:60 - global_seqlen/min:17684 - global_seqlen/max:40237 - global_seqlen/minmax_diff:22553 - global_seqlen/balanced_min:23459 - global_seqlen/balanced_max:29905 - global_seqlen/mean:24542.625 - actor/entropy:0.5872594118118286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1548393964767456 - training/rollout_probs_diff_mean:0.004558309447020292 - training/rollout_probs_diff_std:0.009059520438313484 - training/rollout_actor_probs_pearson_corr:0.9995567202568054 - rollout_corr/rollout_is_mean:1.0000841617584229 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.981291673786473e-06 - rollout_corr/rollout_is_max:1.6430926322937012 - rollout_corr/rollout_is_min:0.4997630715370178 - rollout_corr/rollout_is_std:0.03440564125776291 - rollout_corr/rollout_is_eff_sample_size:0.9988178892130724 - rollout_corr/rollout_is_seq_mean:1.0000742673873901 - rollout_corr/rollout_is_seq_std:0.0018727714195847511 - rollout_corr/rollout_is_seq_max:1.0050119161605835 - rollout_corr/rollout_is_seq_min:0.9923495650291443 - rollout_corr/rollout_is_seq_max_deviation:0.007650434970855713 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.939800176769495) - rollout_corr/training_log_ppl:np.float64(0.6510654075245839) - rollout_corr/kl:np.float64(0.0005472015655350049) - rollout_corr/k3_kl:np.float64(0.0007254680065500452) - rollout_corr/rollout_ppl:np.float64(1.9387605665251613) - rollout_corr/rollout_log_ppl:np.float64(0.6505182061591768) - rollout_corr/log_ppl_diff:np.float64(0.0005472013654070906) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015570658797514625) - rollout_corr/log_ppl_diff_max:np.float64(0.008733868598937988) - rollout_corr/log_ppl_diff_min:np.float64(-0.004288792610168457) - rollout_corr/ppl_ratio:np.float64(1.0005491839256138) - rollout_corr/chi2_token:np.float64(0.0018134494312107563) - rollout_corr/chi2_seq:np.float64(0.8434074395336211) - actor/pg_loss:np.float64(0.0002843305701389909) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002580215148100251) - actor/ppo_kl:np.float64(-5.161208666493167e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3583648204803467) - perf/mfu/actor:np.float64(0.09145036782866803) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.17703628540039) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.475) - val-aux/sciknoweval/reward/std@16:np.float64(0.29967586650697225) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6009000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.23900647793838267) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.35223999999999994) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2710608944671888) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.47751999999999994) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.300212750362816) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6911000000000002) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16058718266186794) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.24136) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.2124433538938892) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.51696) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.23947344703230816) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.75082) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09953551005391528) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.15498) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.14800330439408715) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.54078) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.17034718155898534) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.78774) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.057118418254788444) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.09644) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.08436996829072994) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.55278) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11716805132589453) - val-aux/sciknoweval/score/mean@16:np.float64(0.475) - val-aux/sciknoweval/score/std@16:np.float64(0.29967586650697225) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6009000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.23900647793838267) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.35223999999999994) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2710608944671888) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.47751999999999994) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.300212750362816) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6911000000000002) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16058718266186794) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.24136) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.2124433538938892) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.51696) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.23947344703230816) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.75082) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09953551005391528) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.15498) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.14800330439408715) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.54078) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.17034718155898534) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.78774) - val-aux/sciknoweval/score/best@16/std:np.float64(0.057118418254788444) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.09644) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.08436996829072994) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.55278) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11716805132589453) - val-core/sciknoweval/acc/mean@16:np.float64(0.475) - val-aux/sciknoweval/acc/std@16:np.float64(0.29967586650697225) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6009000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.23900647793838267) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.35223999999999994) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2710608944671888) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.47751999999999994) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.300212750362816) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6911000000000002) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16058718266186794) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.24136) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.2124433538938892) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.51696) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.23947344703230816) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.75082) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09953551005391528) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.15498) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.14800330439408715) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.54078) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.17034718155898534) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.78774) - val-core/sciknoweval/acc/best@16/std:np.float64(0.057118418254788444) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.09644) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.08436996829072994) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.55278) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11716805132589453) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.95375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.07760067103994159) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9849600000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.04443669920492302) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9238599999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.08824016898146585) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9537599999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.07830257322381366) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9979199999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.012872981833386461) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8837800000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08593337831141738) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.97552) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.056384204403371815) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0006321392251711644) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8454199999999998) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06563651232462947) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9898600000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.03374521623261899) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8198399999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.040520072455439135) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9976) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.014302659069178597) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:4 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01849963702261448 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01849963702261448 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:489.42578125 - response_length/max:8192.0 - response_length/min:178.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:489.42578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.34483653306961e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3690809793770313) - timing_s/agent_loop/generate_sequences/max:np.float64(59.658367624506354) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.051393561006989) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.658367624506354) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:288 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:67.2401324082166 - timing_s/reward:0.0644314605742693 - timing_s/old_log_prob:3.3287812639027834 - timing_s/adv:0.12955297529697418 - timing_s/update_actor:13.879926843568683 - timing_s/step:84.73182376660407 - timing_s/testing:112.50635466352105 - timing_s/save_checkpoint:12.7056083381176 - timing_s/stop_profile:0.000159531831741333 - timing_per_token_ms/update_actor:0.07069296195684387 - timing_per_token_ms/gen:0.536663120910319 - timing_per_token_ms/adv:0.0006598365868411294 - perf/total_num_tokens:196341 - perf/time_per_step:84.73182376660407 - perf/throughput:289.65061660425573 (TaskRunner pid=2347384) Training Progress: 60%|██████ | 60/100 [1:06:13<1:03:02, 94.57s/it] (TaskRunner pid=2347384) step:61 - global_seqlen/min:20361 - global_seqlen/max:30675 - global_seqlen/minmax_diff:10314 - global_seqlen/balanced_min:24501 - global_seqlen/balanced_max:24516 - global_seqlen/mean:24509.75 - actor/entropy:0.6352754831314087 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1791786551475525 - training/rollout_probs_diff_mean:0.004841999616473913 - training/rollout_probs_diff_std:0.00912234652787447 - training/rollout_actor_probs_pearson_corr:0.9995156526565552 - rollout_corr/rollout_is_mean:1.000041127204895 - rollout_corr/rollout_is_ratio_fraction_high:8.36078470456414e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.5300827026367188 - rollout_corr/rollout_is_min:0.630948543548584 - rollout_corr/rollout_is_std:0.03550546616315842 - rollout_corr/rollout_is_eff_sample_size:0.9987411867895215 - rollout_corr/rollout_is_seq_mean:1.0000251531600952 - rollout_corr/rollout_is_seq_std:0.0017330985283479095 - rollout_corr/rollout_is_seq_max:1.0049047470092773 - rollout_corr/rollout_is_seq_min:0.9955405592918396 - rollout_corr/rollout_is_seq_max_deviation:0.004904747009277344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9562030648812652) - rollout_corr/training_log_ppl:np.float64(0.6602855395758525) - rollout_corr/kl:np.float64(0.0007152016177514042) - rollout_corr/k3_kl:np.float64(0.0007574325115342617) - rollout_corr/rollout_ppl:np.float64(1.9546900023706257) - rollout_corr/rollout_log_ppl:np.float64(0.6595703373895958) - rollout_corr/log_ppl_diff:np.float64(0.0007152021862566471) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016494286246597767) - rollout_corr/log_ppl_diff_max:np.float64(0.005584716796875) - rollout_corr/log_ppl_diff_min:np.float64(-0.0048583149909973145) - rollout_corr/ppl_ratio:np.float64(1.0007172890473157) - rollout_corr/chi2_token:np.float64(0.001656613778322935) - rollout_corr/chi2_seq:np.float64(0.8631992822047323) - actor/pg_loss:np.float64(8.75210389494896e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000287665474388632) - actor/ppo_kl:np.float64(7.040267049474025e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3369012773036957) - perf/mfu/actor:np.float64(0.0976842071003502) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(107.49849319458008) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:4 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012956080958247185 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012956080958247185 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:467.2109375 - response_length/max:1677.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:467.2109375 - response_length_non_aborted/max:1677.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.038827657699585e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.349672941491008) - timing_s/agent_loop/generate_sequences/max:np.float64(12.728003274649382) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.392123898680438) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.728003274649382) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:1677 - timing_s/gen:19.317255599424243 - timing_s/reward:0.06245894730091095 - timing_s/old_log_prob:3.1906673051416874 - timing_s/adv:0.1575964167714119 - timing_s/update_actor:13.019807241857052 - timing_s/step:35.8414760529995 - timing_s/stop_profile:0.00012412667274475098 - timing_per_token_ms/update_actor:0.0664011630160296 - timing_per_token_ms/gen:0.16150741266679133 - timing_per_token_ms/adv:0.000803743493769887 - perf/total_num_tokens:196078 - perf/time_per_step:35.8414760529995 - perf/throughput:683.8376288899751 (TaskRunner pid=2347384) Training Progress: 61%|██████ | 61/100 [1:06:49<50:01, 76.97s/it] (TaskRunner pid=2347384) step:62 - global_seqlen/min:21025 - global_seqlen/max:34946 - global_seqlen/minmax_diff:13921 - global_seqlen/balanced_min:26106 - global_seqlen/balanced_max:26541 - global_seqlen/mean:26172.375 - actor/entropy:0.5981945395469666 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18470343947410583 - training/rollout_probs_diff_mean:0.004628941882401705 - training/rollout_probs_diff_std:0.009126180782914162 - training/rollout_actor_probs_pearson_corr:0.9994977712631226 - rollout_corr/rollout_is_mean:1.0000007152557373 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.59934663772583 - rollout_corr/rollout_is_min:0.6257065534591675 - rollout_corr/rollout_is_std:0.03485829010605812 - rollout_corr/rollout_is_eff_sample_size:0.9987863743884395 - rollout_corr/rollout_is_seq_mean:1.0000046491622925 - rollout_corr/rollout_is_seq_std:0.0016272451030090451 - rollout_corr/rollout_is_seq_max:1.004730463027954 - rollout_corr/rollout_is_seq_min:0.9942823648452759 - rollout_corr/rollout_is_seq_max_deviation:0.005717635154724121 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.885653248988092) - rollout_corr/training_log_ppl:np.float64(0.6225170233519748) - rollout_corr/kl:np.float64(0.0006676197259634087) - rollout_corr/k3_kl:np.float64(0.0006973658590823106) - rollout_corr/rollout_ppl:np.float64(1.8844418218359351) - rollout_corr/rollout_log_ppl:np.float64(0.6218493984779343) - rollout_corr/log_ppl_diff:np.float64(0.0006676248740404844) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013843933120369911) - rollout_corr/log_ppl_diff_max:np.float64(0.00526583194732666) - rollout_corr/log_ppl_diff_min:np.float64(-0.003930270671844482) - rollout_corr/ppl_ratio:np.float64(1.000669184140861) - rollout_corr/chi2_token:np.float64(0.0014710784889757633) - rollout_corr/chi2_seq:np.float64(0.8179175723344088) - actor/pg_loss:np.float64(2.6686815544962883e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002690604562758381) - actor/ppo_kl:np.float64(1.4355201699345344e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.32805465161800385) - perf/mfu/actor:np.float64(0.1028074892283904) - perf/max_memory_allocated_gb:np.float64(132.02947998046875) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.34636688232422) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:4 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027322403620928526 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027322403620928526 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:523.98046875 - response_length/max:2378.0 - response_length/min:203.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:523.98046875 - response_length_non_aborted/max:2378.0 - response_length_non_aborted/min:203.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.90625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010083988308906555 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5903304368257523) - timing_s/agent_loop/generate_sequences/max:np.float64(16.81711182370782) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.918771314056357) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.81711182370782) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:2378 - timing_s/gen:24.048296004533768 - timing_s/reward:0.07129916176199913 - timing_s/old_log_prob:3.1615519113838673 - timing_s/adv:0.1441831737756729 - timing_s/update_actor:13.251855365931988 - timing_s/step:40.76489446312189 - timing_s/stop_profile:0.00011922791600227356 - timing_per_token_ms/update_actor:0.06329123439280915 - timing_per_token_ms/gen:0.17927892711689938 - timing_per_token_ms/adv:0.0006886228980732209 - perf/total_num_tokens:209379 - perf/time_per_step:40.76489446312189 - perf/throughput:642.03220306818 (TaskRunner pid=2347384) Training Progress: 62%|██████▏ | 62/100 [1:07:29<41:52, 66.12s/it] (TaskRunner pid=2347384) step:63 - global_seqlen/min:23101 - global_seqlen/max:34980 - global_seqlen/minmax_diff:11879 - global_seqlen/balanced_min:27924 - global_seqlen/balanced_max:34011 - global_seqlen/mean:29010.875 - actor/entropy:0.5455924272537231 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6205249428749084 - training/rollout_probs_diff_mean:0.004216565750539303 - training/rollout_probs_diff_std:0.008970578201115131 - training/rollout_actor_probs_pearson_corr:0.9995482563972473 - rollout_corr/rollout_is_mean:0.9999991655349731 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.8351296249602456e-06 - rollout_corr/rollout_is_max:1.7473889589309692 - rollout_corr/rollout_is_min:0.003095390973612666 - rollout_corr/rollout_is_std:0.03316900134086609 - rollout_corr/rollout_is_eff_sample_size:0.9989009075907395 - rollout_corr/rollout_is_seq_mean:1.000017762184143 - rollout_corr/rollout_is_seq_std:0.001850213622674346 - rollout_corr/rollout_is_seq_max:1.0064337253570557 - rollout_corr/rollout_is_seq_min:0.9946577548980713 - rollout_corr/rollout_is_seq_max_deviation:0.006433725357055664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8669897536747158) - rollout_corr/training_log_ppl:np.float64(0.6120052234909963) - rollout_corr/kl:np.float64(0.0007567557882453002) - rollout_corr/k3_kl:np.float64(0.0006992573034665384) - rollout_corr/rollout_ppl:np.float64(1.8655388155020773) - rollout_corr/rollout_log_ppl:np.float64(0.6112484632030828) - rollout_corr/log_ppl_diff:np.float64(0.0007567602879134938) - rollout_corr/log_ppl_abs_diff:np.float64(0.001599654569872655) - rollout_corr/log_ppl_diff_max:np.float64(0.006732255220413208) - rollout_corr/log_ppl_diff_min:np.float64(-0.006649374961853027) - rollout_corr/ppl_ratio:np.float64(1.0007588441949338) - rollout_corr/chi2_token:np.float64(0.0012939136940985918) - rollout_corr/chi2_seq:np.float64(1.0524738836102188) - actor/pg_loss:np.float64(0.0002161843003705144) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022222603729460388) - actor/ppo_kl:np.float64(0.000107501104544383) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2938495799899101) - perf/mfu/actor:np.float64(0.10630582669558855) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.32476997375488) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:4 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02725508064031601 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02725508064031601 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:571.49609375 - response_length/max:8192.0 - response_length/min:177.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:571.49609375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:177.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:335.09375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011601671576499939 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2523946966975927) - timing_s/agent_loop/generate_sequences/max:np.float64(59.683830101042986) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.351266218240198) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.683830101042986) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:66.53031222894788 - timing_s/reward:0.07077518291771412 - timing_s/old_log_prob:3.451879436150193 - timing_s/adv:0.17366058193147182 - timing_s/update_actor:14.32019299827516 - timing_s/step:84.63406862691045 - timing_s/stop_profile:0.00011523626744747162 - timing_per_token_ms/update_actor:0.06170183163328907 - timing_per_token_ms/gen:0.45474332193425887 - timing_per_token_ms/adv:0.0007482563949358294 - perf/total_num_tokens:232087 - perf/time_per_step:84.63406862691045 - perf/throughput:342.78010582106924 (TaskRunner pid=2347384) Training Progress: 63%|██████▎ | 63/100 [1:08:54<44:12, 71.69s/it] (TaskRunner pid=2347384) step:64 - global_seqlen/min:25406 - global_seqlen/max:40662 - global_seqlen/minmax_diff:15256 - global_seqlen/balanced_min:31631 - global_seqlen/balanced_max:37496 - global_seqlen/mean:32842.375 - actor/entropy:0.5346024036407471 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27727752923965454 - training/rollout_probs_diff_mean:0.003883012570440769 - training/rollout_probs_diff_std:0.008337817154824734 - training/rollout_actor_probs_pearson_corr:0.9996206164360046 - rollout_corr/rollout_is_mean:0.9999945163726807 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2086417882528622e-05 - rollout_corr/rollout_is_max:1.8514951467514038 - rollout_corr/rollout_is_min:0.26193851232528687 - rollout_corr/rollout_is_std:0.031942326575517654 - rollout_corr/rollout_is_eff_sample_size:0.998980846633257 - rollout_corr/rollout_is_seq_mean:0.999992847442627 - rollout_corr/rollout_is_seq_std:0.001399002387188375 - rollout_corr/rollout_is_seq_max:1.004547119140625 - rollout_corr/rollout_is_seq_min:0.9960532188415527 - rollout_corr/rollout_is_seq_max_deviation:0.004547119140625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8697756584733725) - rollout_corr/training_log_ppl:np.float64(0.6125503201401443) - rollout_corr/kl:np.float64(0.0006201905730827129) - rollout_corr/k3_kl:np.float64(0.0006303364185669125) - rollout_corr/rollout_ppl:np.float64(1.8685544421896338) - rollout_corr/rollout_log_ppl:np.float64(0.6119301268918207) - rollout_corr/log_ppl_diff:np.float64(0.0006201932483236305) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012861887298640795) - rollout_corr/log_ppl_diff_max:np.float64(0.007578432559967041) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036764144897460938) - rollout_corr/ppl_ratio:np.float64(1.000621582614258) - rollout_corr/chi2_token:np.float64(0.0012918002903461456) - rollout_corr/chi2_seq:np.float64(0.4350412057247013) - actor/pg_loss:np.float64(-6.110407412052155e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002444168826514215) - actor/ppo_kl:np.float64(2.1309184818463223e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29246145486831665) - perf/mfu/actor:np.float64(0.1163736099323967) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.28197860717773) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:4 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.026083245873451233 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.026083245873451233 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:646.38671875 - response_length/max:8192.0 - response_length/min:213.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:646.38671875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:213.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:379.9375 - prompt_length/max:499.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.551644325256348e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7468412443995476) - timing_s/agent_loop/generate_sequences/max:np.float64(59.15042114444077) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.085374305446749) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(59.15042114444077) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:66.60914850793779 - timing_s/reward:0.08201035298407078 - timing_s/old_log_prob:3.4901863280683756 - timing_s/adv:0.18269767984747887 - timing_s/update_actor:14.948789360001683 - timing_s/step:85.39833105355501 - timing_s/stop_profile:0.00014822743833065033 - timing_per_token_ms/update_actor:0.056895966567588684 - timing_per_token_ms/gen:0.40253300201201264 - timing_per_token_ms/adv:0.0006953580543713681 - perf/total_num_tokens:262739 - perf/time_per_step:85.39833105355501 - perf/throughput:384.5786515359871 (TaskRunner pid=2347384) Training Progress: 64%|██████▍ | 64/100 [1:10:20<45:29, 75.82s/it] (TaskRunner pid=2347384) step:65 - global_seqlen/min:21295 - global_seqlen/max:32107 - global_seqlen/minmax_diff:10812 - global_seqlen/balanced_min:26803 - global_seqlen/balanced_max:26845 - global_seqlen/mean:26832.5 - actor/entropy:0.6207481622695923 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19513005018234253 - training/rollout_probs_diff_mean:0.004546179436147213 - training/rollout_probs_diff_std:0.008902329951524734 - training/rollout_actor_probs_pearson_corr:0.9995855093002319 - rollout_corr/rollout_is_mean:0.9999449253082275 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4900909263815265e-05 - rollout_corr/rollout_is_max:1.6073755025863647 - rollout_corr/rollout_is_min:0.43231064081192017 - rollout_corr/rollout_is_std:0.03524599224328995 - rollout_corr/rollout_is_eff_sample_size:0.99875914242478 - rollout_corr/rollout_is_seq_mean:0.999968409538269 - rollout_corr/rollout_is_seq_std:0.0018359578680247068 - rollout_corr/rollout_is_seq_max:1.0063998699188232 - rollout_corr/rollout_is_seq_min:0.9933979511260986 - rollout_corr/rollout_is_seq_max_deviation:0.006602048873901367 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.925604076590389) - rollout_corr/training_log_ppl:np.float64(0.6440437774290331) - rollout_corr/kl:np.float64(0.0008413790875252403) - rollout_corr/k3_kl:np.float64(0.0007516867610206646) - rollout_corr/rollout_ppl:np.float64(1.9239637488499284) - rollout_corr/rollout_log_ppl:np.float64(0.6432023975066841) - rollout_corr/log_ppl_diff:np.float64(0.0008413799223490059) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015998666058294475) - rollout_corr/log_ppl_diff_max:np.float64(0.007391870021820068) - rollout_corr/log_ppl_diff_min:np.float64(-0.00554269552230835) - rollout_corr/ppl_ratio:np.float64(1.0008436078205705) - rollout_corr/chi2_token:np.float64(0.0013334299437701702) - rollout_corr/chi2_seq:np.float64(1.0197422185447067) - actor/pg_loss:np.float64(6.12802105024457e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045300728334041196) - actor/ppo_kl:np.float64(0.0001409739878859284) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3358636423945427) - perf/mfu/actor:np.float64(0.10631855529977301) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.38580703735352) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:4 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006580614019185305 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006580614019185305 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:524.296875 - response_length/max:1686.0 - response_length/min:185.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:524.296875 - response_length_non_aborted/max:1686.0 - response_length_non_aborted/min:185.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.21875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001552291214466095 - timing_s/agent_loop/generate_sequences/min:np.float64(2.352688204497099) - timing_s/agent_loop/generate_sequences/max:np.float64(13.756486231461167) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.0002494652726455) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.756486231461167) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1686 - timing_s/gen:20.46908939629793 - timing_s/reward:0.06825436651706696 - timing_s/old_log_prob:3.132166314870119 - timing_s/adv:0.13421783968806267 - timing_s/update_actor:13.157137896865606 - timing_s/step:37.04886046051979 - timing_s/stop_profile:0.00011565163731575012 - timing_per_token_ms/update_actor:0.06129291855429799 - timing_per_token_ms/gen:0.1525040187475632 - timing_per_token_ms/adv:0.0006252578015841921 - perf/total_num_tokens:214660 - perf/time_per_step:37.04886046051979 - perf/throughput:724.246297091739 (TaskRunner pid=2347384) Training Progress: 65%|██████▌ | 65/100 [1:10:57<37:27, 64.21s/it] (TaskRunner pid=2347384) step:66 - global_seqlen/min:22274 - global_seqlen/max:32623 - global_seqlen/minmax_diff:10349 - global_seqlen/balanced_min:26816 - global_seqlen/balanced_max:27261 - global_seqlen/mean:26891.375 - actor/entropy:0.6257913112640381 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351458191871643 - training/rollout_probs_diff_mean:0.0045900181867182255 - training/rollout_probs_diff_std:0.008917701430618763 - training/rollout_actor_probs_pearson_corr:0.9995673894882202 - rollout_corr/rollout_is_mean:0.9999930262565613 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9363694190979004 - rollout_corr/rollout_is_min:0.5830235481262207 - rollout_corr/rollout_is_std:0.034758973866701126 - rollout_corr/rollout_is_eff_sample_size:0.9987932718026972 - rollout_corr/rollout_is_seq_mean:0.9999854564666748 - rollout_corr/rollout_is_seq_std:0.0016893023857846856 - rollout_corr/rollout_is_seq_max:1.005061388015747 - rollout_corr/rollout_is_seq_min:0.9951789379119873 - rollout_corr/rollout_is_seq_max_deviation:0.00506138801574707 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9523011581040919) - rollout_corr/training_log_ppl:np.float64(0.6582004217198119) - rollout_corr/kl:np.float64(0.0008687902869892739) - rollout_corr/k3_kl:np.float64(0.000780955700861341) - rollout_corr/rollout_ppl:np.float64(1.9505894747562706) - rollout_corr/rollout_log_ppl:np.float64(0.6573316331487149) - rollout_corr/log_ppl_diff:np.float64(0.0008687885710969567) - rollout_corr/log_ppl_abs_diff:np.float64(0.001621669507585466) - rollout_corr/log_ppl_diff_max:np.float64(0.006796002388000488) - rollout_corr/log_ppl_diff_min:np.float64(-0.004694759845733643) - rollout_corr/ppl_ratio:np.float64(1.0008709919638932) - rollout_corr/chi2_token:np.float64(0.0014324793592095375) - rollout_corr/chi2_seq:np.float64(2.069492584094405) - actor/pg_loss:np.float64(3.935385029762983e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027896277674699377) - actor/ppo_kl:np.float64(0.00021445795575480986) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2940399497747421) - perf/mfu/actor:np.float64(0.10609425143395275) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.2559814453125) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:4 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005820830352604389 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005820830352604389 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:544.91796875 - response_length/max:2301.0 - response_length/min:180.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:544.91796875 - response_length_non_aborted/max:2301.0 - response_length_non_aborted/min:180.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.4375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.931383490562439e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3505223356187344) - timing_s/agent_loop/generate_sequences/max:np.float64(16.464555468410254) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.176658551463333) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.464555468410254) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:2301 - timing_s/gen:23.52073303423822 - timing_s/reward:0.06874074228107929 - timing_s/old_log_prob:3.1002825889736414 - timing_s/adv:0.12155797518789768 - timing_s/update_actor:13.144755797460675 - timing_s/step:40.042431050911546 - timing_s/stop_profile:0.00012300163507461548 - timing_per_token_ms/update_actor:0.06110116997299634 - timing_per_token_ms/gen:0.16860861392725554 - timing_per_token_ms/adv:0.0005650416499151572 - perf/total_num_tokens:215131 - perf/time_per_step:40.042431050911546 - perf/throughput:671.5719873703281 (TaskRunner pid=2347384) Training Progress: 66%|██████▌ | 66/100 [1:11:37<32:17, 56.97s/it] (TaskRunner pid=2347384) step:67 - global_seqlen/min:24880 - global_seqlen/max:34447 - global_seqlen/minmax_diff:9567 - global_seqlen/balanced_min:29948 - global_seqlen/balanced_max:31331 - global_seqlen/mean:30130.0 - actor/entropy:0.589844286441803 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22166013717651367 - training/rollout_probs_diff_mean:0.004247101955115795 - training/rollout_probs_diff_std:0.00861254334449768 - training/rollout_actor_probs_pearson_corr:0.9995855093002319 - rollout_corr/rollout_is_mean:1.0001040697097778 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9481531126075424e-05 - rollout_corr/rollout_is_max:1.6337190866470337 - rollout_corr/rollout_is_min:0.3651112914085388 - rollout_corr/rollout_is_std:0.03365423530340195 - rollout_corr/rollout_is_eff_sample_size:0.9988689117678543 - rollout_corr/rollout_is_seq_mean:1.0000989437103271 - rollout_corr/rollout_is_seq_std:0.0015551011310890317 - rollout_corr/rollout_is_seq_max:1.004523754119873 - rollout_corr/rollout_is_seq_min:0.9955280423164368 - rollout_corr/rollout_is_seq_max_deviation:0.004523754119873047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8956821579486132) - rollout_corr/training_log_ppl:np.float64(0.6294339749729261) - rollout_corr/kl:np.float64(0.0006292968909278329) - rollout_corr/k3_kl:np.float64(0.0006888913353577664) - rollout_corr/rollout_ppl:np.float64(1.8944640522822738) - rollout_corr/rollout_log_ppl:np.float64(0.6288046729168855) - rollout_corr/log_ppl_diff:np.float64(0.0006293020560406148) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013327563065104187) - rollout_corr/log_ppl_diff_max:np.float64(0.0057659149169921875) - rollout_corr/log_ppl_diff_min:np.float64(-0.005035281181335449) - rollout_corr/ppl_ratio:np.float64(1.0006307614967227) - rollout_corr/chi2_token:np.float64(0.001525928732007742) - rollout_corr/chi2_seq:np.float64(1.6525246889796108) - actor/pg_loss:np.float64(7.881573401391506e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020492133899097098) - actor/ppo_kl:np.float64(0.00011573680059484559) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29331646859645844) - perf/mfu/actor:np.float64(0.11493123702509477) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.14929580688477) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:4 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.02250116877257824 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.02250116877257824 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:601.53125 - response_length/max:3336.0 - response_length/min:239.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:601.53125 - response_length_non_aborted/max:3336.0 - response_length_non_aborted/min:239.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:340.03125 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.856132626533508e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8754393495619297) - timing_s/agent_loop/generate_sequences/max:np.float64(23.423784406855702) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.6740256431512535) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.423784406855702) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:3336 - timing_s/gen:30.97574547305703 - timing_s/reward:0.07137090712785721 - timing_s/old_log_prob:3.2540190517902374 - timing_s/adv:0.12272687815129757 - timing_s/update_actor:13.755379335954785 - timing_s/step:48.266349263489246 - timing_s/stop_profile:0.00011717714369297028 - timing_per_token_ms/update_actor:0.05706679113821268 - timing_per_token_ms/gen:0.2011516538070616 - timing_per_token_ms/adv:0.000509155651142124 - perf/total_num_tokens:241040 - perf/time_per_step:48.266349263489246 - perf/throughput:624.2444365435285 (TaskRunner pid=2347384) Training Progress: 67%|██████▋ | 67/100 [1:12:25<29:54, 54.37s/it] (TaskRunner pid=2347384) step:68 - global_seqlen/min:17421 - global_seqlen/max:34085 - global_seqlen/minmax_diff:16664 - global_seqlen/balanced_min:26909 - global_seqlen/balanced_max:27165 - global_seqlen/mean:26947.375 - actor/entropy:0.6444172263145447 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18439927697181702 - training/rollout_probs_diff_mean:0.004641249775886536 - training/rollout_probs_diff_std:0.008944928646087646 - training/rollout_actor_probs_pearson_corr:0.9995980858802795 - rollout_corr/rollout_is_mean:1.0000133514404297 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.867990578030003e-06 - rollout_corr/rollout_is_max:1.6314746141433716 - rollout_corr/rollout_is_min:0.42825600504875183 - rollout_corr/rollout_is_std:0.03462151810526848 - rollout_corr/rollout_is_eff_sample_size:0.9988027856338563 - rollout_corr/rollout_is_seq_mean:1.000099539756775 - rollout_corr/rollout_is_seq_std:0.0016578711802139878 - rollout_corr/rollout_is_seq_max:1.0061999559402466 - rollout_corr/rollout_is_seq_min:0.9961308240890503 - rollout_corr/rollout_is_seq_max_deviation:0.006199955940246582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9535676850937307) - rollout_corr/training_log_ppl:np.float64(0.6591222073184326) - rollout_corr/kl:np.float64(0.0004966206334939827) - rollout_corr/k3_kl:np.float64(0.0006638419986302324) - rollout_corr/rollout_ppl:np.float64(1.9525953237898648) - rollout_corr/rollout_log_ppl:np.float64(0.6586255870060995) - rollout_corr/log_ppl_diff:np.float64(0.000496620312333107) - rollout_corr/log_ppl_abs_diff:np.float64(0.001403916859999299) - rollout_corr/log_ppl_diff_max:np.float64(0.0046253204345703125) - rollout_corr/log_ppl_diff_min:np.float64(-0.005441248416900635) - rollout_corr/ppl_ratio:np.float64(1.0004981856327504) - rollout_corr/chi2_token:np.float64(0.001669891644269228) - rollout_corr/chi2_seq:np.float64(0.9913946138694882) - actor/pg_loss:np.float64(9.917933493852615e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002313076165592065) - actor/ppo_kl:np.float64(-3.844997100088676e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2564214989542961) - perf/mfu/actor:np.float64(0.10539461159894445) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.2226333618164) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:4 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008263050578534603 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008263050578534603 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.76171875 - response_length/max:2239.0 - response_length/min:213.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.76171875 - response_length_non_aborted/max:2239.0 - response_length_non_aborted/min:213.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.34375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001437310129404068 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7150635942816734) - timing_s/agent_loop/generate_sequences/max:np.float64(16.899478439241648) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.49198151560995) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.899478439241648) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:2239 - timing_s/gen:23.6356644872576 - timing_s/reward:0.0673914086073637 - timing_s/old_log_prob:3.09491696767509 - timing_s/adv:0.1251463834196329 - timing_s/update_actor:13.352374255657196 - timing_s/step:40.3640451002866 - timing_s/stop_profile:0.00014880113303661346 - timing_per_token_ms/update_actor:0.06193726780278782 - timing_per_token_ms/gen:0.16232951578784502 - timing_per_token_ms/adv:0.0005805128672998433 - perf/total_num_tokens:215579 - perf/time_per_step:40.3640451002866 - perf/throughput:667.6083859545747 (TaskRunner pid=2347384) Training Progress: 68%|██████▊ | 68/100 [1:13:06<26:45, 50.19s/it] (TaskRunner pid=2347384) step:69 - global_seqlen/min:26099 - global_seqlen/max:48042 - global_seqlen/minmax_diff:21943 - global_seqlen/balanced_min:31376 - global_seqlen/balanced_max:37337 - global_seqlen/mean:32333.0 - actor/entropy:0.5698190927505493 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5148013234138489 - training/rollout_probs_diff_mean:0.003968259319663048 - training/rollout_probs_diff_std:0.008453773334622383 - training/rollout_actor_probs_pearson_corr:0.9995850920677185 - rollout_corr/rollout_is_mean:1.0000203847885132 - rollout_corr/rollout_is_ratio_fraction_high:5.7056781770370435e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.1411356354074087e-05 - rollout_corr/rollout_is_max:2.177001714706421 - rollout_corr/rollout_is_min:0.2908869981765747 - rollout_corr/rollout_is_std:0.03237602859735489 - rollout_corr/rollout_is_eff_sample_size:0.9989528903049885 - rollout_corr/rollout_is_seq_mean:0.999998927116394 - rollout_corr/rollout_is_seq_std:0.0014363601803779602 - rollout_corr/rollout_is_seq_max:1.0048489570617676 - rollout_corr/rollout_is_seq_min:0.9959189295768738 - rollout_corr/rollout_is_seq_max_deviation:0.004848957061767578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8955526044592261) - rollout_corr/training_log_ppl:np.float64(0.6262829734914703) - rollout_corr/kl:np.float64(0.0006352785842267394) - rollout_corr/k3_kl:np.float64(0.0006139969935361478) - rollout_corr/rollout_ppl:np.float64(1.8942792634479702) - rollout_corr/rollout_log_ppl:np.float64(0.625647687484161) - rollout_corr/log_ppl_diff:np.float64(0.0006352860073093325) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012822118878830224) - rollout_corr/log_ppl_diff_max:np.float64(0.005681276321411133) - rollout_corr/log_ppl_diff_min:np.float64(-0.005220234394073486) - rollout_corr/ppl_ratio:np.float64(1.0006366996094584) - rollout_corr/chi2_token:np.float64(0.0011912821792066097) - rollout_corr/chi2_seq:np.float64(0.8885124691296369) - actor/pg_loss:np.float64(3.740633837878704e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000206039456088547) - actor/ppo_kl:np.float64(5.142674557312432e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25310875102877617) - perf/mfu/actor:np.float64(0.11655053927614872) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.29055118560791) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:4 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008751084096729755 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008751084096729755 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:684.625 - response_length/max:8192.0 - response_length/min:238.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:684.625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:238.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:325.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.796954691410065e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.0695799440145493) - timing_s/agent_loop/generate_sequences/max:np.float64(60.719977186992764) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.716403425481985) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(60.719977186992764) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:67.55595040693879 - timing_s/reward:0.07401405461132526 - timing_s/old_log_prob:3.473916418850422 - timing_s/adv:0.11979939602315426 - timing_s/update_actor:14.49752671085298 - timing_s/step:85.80659639462829 - timing_s/stop_profile:0.00012106448411941528 - timing_per_token_ms/update_actor:0.0560477171575982 - timing_per_token_ms/gen:0.38545251966712385 - timing_per_token_ms/adv:0.0004631467696438401 - perf/total_num_tokens:258664 - perf/time_per_step:85.80659639462829 - perf/throughput:376.81252209677587 (TaskRunner pid=2347384) Training Progress: 69%|██████▉ | 69/100 [1:14:31<31:27, 60.89s/it] (TaskRunner pid=2347384) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2347384) validation generation end (TaskRunner pid=2347384) [prompt] system (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) To determine the folding stability score of a protein sequence, we analyze the balance of hydrophobic and hydrophilic interactions, hydrogen bonding, and the overall tendency of the protein to fold into a stable 3D structure. Stability scores are typically calculated using computational models that consider the energy of interactions between amino acids, such as hydrogen bonds, van der Waals forces, and electrostatic interactions. (TaskRunner pid=2347384) (TaskRunner pid=2347384) The given protein sequence is: (TaskRunner pid=2347384) **GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR** (TaskRunner pid=2347384) (TaskRunner pid=2347384) Let's break it down: (TaskRunner pid=2347384) (TaskRunner pid=2347384) 1. **Hydrophobic vs. Hydrophilic Balance**: (TaskRunner pid=2347384) - Hydrophobic residues (e.g., G, S, T, R, Y, F, L, W, V, A, K, E, D, Q, N, G) are present in the sequence. (TaskRunner pid=2347384) - The sequence includes a mix of polar and non-polar residues, which is important for proper folding. (TaskRunner pid=2347384) - However, the sequence contains several charged residues (e.g., R, E, D, K, Q, N), which can contribute to stability through salt bridges and hydrogen bonding. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 2. **Secondary Structure Potential**: (TaskRunner pid=2347384) - The sequence includes segments that may form alpha-helices and beta-sheets, such as the presence of multiple charged residues and repetitive sequences (e.g., "EEE", "ARRAAK", "ARRG", "QVHVTQNG"). (TaskRunner pid=2347384) - These segments suggest a potential for forming stable secondary structures, which increases folding stability. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 3. **Energy Calculation**: (TaskRunner pid=2347384) - Stability scores are often derived from energy calculations. A positive score indicates a more stable structure, while a negative score suggests instability. (TaskRunner pid=2347384) - The presence of multiple charged residues and favorable interactions increases the likelihood of a positive score. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 4. **Empirical and Computational Models**: (TaskRunner pid=2347384) - Using standard folding prediction tools or energy-based models, the stability score for this sequence is estimated to be **positive**, as the sequence has a good balance of interactions and potential for stable folding. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Based on the composition and energy considerations, the folding stability score is expected to be a **positive value**, and among the given options, **1.69** and **2.49** are the most likely candidates. Given the complexity of the sequence and the presence of multiple favorable interactions, the score is more likely to be **2.49**, which indicates a strong tendency for stable folding. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) C (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 0.0 (TaskRunner pid=2347384) [acc] 0.0 (TaskRunner pid=2347384) [pred] C (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2347384) step:70 - global_seqlen/min:22892 - global_seqlen/max:45274 - global_seqlen/minmax_diff:22382 - global_seqlen/balanced_min:29894 - global_seqlen/balanced_max:32846 - global_seqlen/mean:30273.875 - actor/entropy:0.6131719946861267 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15366443991661072 - training/rollout_probs_diff_mean:0.00431956397369504 - training/rollout_probs_diff_std:0.008580975234508514 - training/rollout_actor_probs_pearson_corr:0.9995657205581665 - rollout_corr/rollout_is_mean:0.999946117401123 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6171174049377441 - rollout_corr/rollout_is_min:0.5561785697937012 - rollout_corr/rollout_is_std:0.033606380224227905 - rollout_corr/rollout_is_eff_sample_size:0.9988717663304828 - rollout_corr/rollout_is_seq_mean:0.9999675750732422 - rollout_corr/rollout_is_seq_std:0.0015103485202416778 - rollout_corr/rollout_is_seq_max:1.0039453506469727 - rollout_corr/rollout_is_seq_min:0.9966089725494385 - rollout_corr/rollout_is_seq_max_deviation:0.003945350646972656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9447635901160538) - rollout_corr/training_log_ppl:np.float64(0.6509870570152998) - rollout_corr/kl:np.float64(0.0007401770297512655) - rollout_corr/k3_kl:np.float64(0.0006394408320602452) - rollout_corr/rollout_ppl:np.float64(1.9432618194259703) - rollout_corr/rollout_log_ppl:np.float64(0.6502468808030244) - rollout_corr/log_ppl_diff:np.float64(0.0007401762122754008) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014322019706014544) - rollout_corr/log_ppl_diff_max:np.float64(0.0058574676513671875) - rollout_corr/log_ppl_diff_min:np.float64(-0.00428539514541626) - rollout_corr/ppl_ratio:np.float64(1.0007417981978506) - rollout_corr/chi2_token:np.float64(0.001070341793820262) - rollout_corr/chi2_seq:np.float64(0.7282506925985217) - actor/pg_loss:np.float64(5.149014759808779e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00034245533919374793) - actor/ppo_kl:np.float64(9.44623739158601e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2602544203400612) - perf/mfu/actor:np.float64(0.11409519055455762) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.31228256225586) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.46) - val-aux/sciknoweval/reward/std@16:np.float64(0.2861467664393313) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5721) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22144790929251737) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.34534000000000004) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.26577659783570406) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4579200000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.289007680712865) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6529399999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15183231396340532) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.23908000000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.21850489251907942) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5016999999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22016362267535464) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.70884) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10756847878326091) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.15309999999999999) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.1624652573430824) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.53166) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14702402596896638) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.75438) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07530998121437738) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.08869999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.110746156790834) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5504199999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1029355380272631) - val-aux/sciknoweval/score/mean@16:np.float64(0.46) - val-aux/sciknoweval/score/std@16:np.float64(0.2861467664393313) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5721) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22144790929251737) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.34534000000000004) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.26577659783570406) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4579200000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.289007680712865) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6529399999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15183231396340532) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.23908000000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.21850489251907942) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5016999999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.22016362267535464) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.70884) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10756847878326091) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.15309999999999999) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.1624652573430824) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.53166) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14702402596896638) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.75438) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07530998121437738) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.08869999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.110746156790834) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5504199999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1029355380272631) - val-core/sciknoweval/acc/mean@16:np.float64(0.46) - val-aux/sciknoweval/acc/std@16:np.float64(0.2861467664393313) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5721) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22144790929251737) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.34534000000000004) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.26577659783570406) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4579200000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.289007680712865) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6529399999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15183231396340532) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.23908000000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.21850489251907942) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5016999999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.22016362267535464) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.70884) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10756847878326091) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.15309999999999999) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.1624652573430824) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.53166) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14702402596896638) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.75438) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07530998121437738) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.08869999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.110746156790834) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5504199999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1029355380272631) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.93875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0925526696176096) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.97748) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.05575465994908158) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9023800000000002) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.09992117498196824) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9401799999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.09202214906465964) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9958800000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.02104885738557665) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8588000000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.09333229945051996) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.96532) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.06986118323539842) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.99976) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0037004497892969336) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8199799999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07392789335026084) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9852199999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.042138960397667125) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.78848) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04993190373494974) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99626) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.01849264755196333) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:4 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011390896514058113 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011390896514058113 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:622.02734375 - response_length/max:4884.0 - response_length/min:266.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:622.02734375 - response_length_non_aborted/max:4884.0 - response_length_non_aborted/min:266.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:324.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.258277714252472e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.4557352848351) - timing_s/agent_loop/generate_sequences/max:np.float64(32.4846227299422) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.985455781745259) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(32.4846227299422) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:4884 - timing_s/gen:39.87588879093528 - timing_s/reward:0.07234281301498413 - timing_s/old_log_prob:3.3542037308216095 - timing_s/adv:0.11924778297543526 - timing_s/update_actor:13.857784865424037 - timing_s/step:57.36728532798588 - timing_s/testing:114.77782279998064 - timing_s/save_checkpoint:12.545790920034051 - timing_s/stop_profile:0.00016822479665279388 - timing_per_token_ms/update_actor:0.05721841383628639 - timing_per_token_ms/gen:0.2504153429180997 - timing_per_token_ms/adv:0.0004923708270556514 - perf/total_num_tokens:242191 - perf/time_per_step:57.36728532798588 - perf/throughput:527.7201950016499 (TaskRunner pid=2347384) Training Progress: 70%|███████ | 70/100 [1:17:36<49:01, 98.06s/it] (TaskRunner pid=2347384) step:71 - global_seqlen/min:21235 - global_seqlen/max:37753 - global_seqlen/minmax_diff:16518 - global_seqlen/balanced_min:30701 - global_seqlen/balanced_max:33643 - global_seqlen/mean:31073.625 - actor/entropy:0.6121453642845154 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2395903766155243 - training/rollout_probs_diff_mean:0.004203904420137405 - training/rollout_probs_diff_std:0.008488737046718597 - training/rollout_actor_probs_pearson_corr:0.9996594190597534 - rollout_corr/rollout_is_mean:0.9998877048492432 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.20478298413218e-05 - rollout_corr/rollout_is_max:1.6326193809509277 - rollout_corr/rollout_is_min:0.32221582531929016 - rollout_corr/rollout_is_std:0.033426765352487564 - rollout_corr/rollout_is_eff_sample_size:0.9988836605170551 - rollout_corr/rollout_is_seq_mean:0.9998750686645508 - rollout_corr/rollout_is_seq_std:0.00159530492965132 - rollout_corr/rollout_is_seq_max:1.00459623336792 - rollout_corr/rollout_is_seq_min:0.9939055442810059 - rollout_corr/rollout_is_seq_max_deviation:0.006094455718994141 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9578393707051873) - rollout_corr/training_log_ppl:np.float64(0.6591245254967362) - rollout_corr/kl:np.float64(0.0007705473704504584) - rollout_corr/k3_kl:np.float64(0.0006441831602614911) - rollout_corr/rollout_ppl:np.float64(1.956278124358505) - rollout_corr/rollout_log_ppl:np.float64(0.6583539699786343) - rollout_corr/log_ppl_diff:np.float64(0.0007705555181019008) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013020281330682337) - rollout_corr/log_ppl_diff_max:np.float64(0.007874637842178345) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038064122200012207) - rollout_corr/ppl_ratio:np.float64(1.0007721513975412) - rollout_corr/chi2_token:np.float64(0.0010341168381273746) - rollout_corr/chi2_seq:np.float64(0.3578775206115097) - actor/pg_loss:np.float64(0.00021309871226549149) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002649730504344916) - actor/ppo_kl:np.float64(2.861905230844286e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3365911841392517) - perf/mfu/actor:np.float64(0.11590899664866659) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(108.04169845581055) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:5 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009181952103972435 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009181952103972435 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:648.45703125 - response_length/max:5210.0 - response_length/min:232.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:648.45703125 - response_length_non_aborted/max:5210.0 - response_length_non_aborted/min:232.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.59375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00022904016077518463 - timing_s/agent_loop/generate_sequences/min:np.float64(3.0012999195605516) - timing_s/agent_loop/generate_sequences/max:np.float64(34.256529461592436) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.292897454295598) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(34.256529461592436) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:5210 - timing_s/gen:41.30897274427116 - timing_s/reward:0.07003145292401314 - timing_s/old_log_prob:3.4965239875018597 - timing_s/adv:0.13453359715640545 - timing_s/update_actor:14.05078205280006 - timing_s/step:59.180613439530134 - timing_s/stop_profile:0.00011994130909442902 - timing_per_token_ms/update_actor:0.0565221391646455 - timing_per_token_ms/gen:0.24884173816614655 - timing_per_token_ms/adv:0.0005411888585432398 - perf/total_num_tokens:248589 - perf/time_per_step:59.180613439530134 - perf/throughput:525.0642599666623 (TaskRunner pid=2347384) Training Progress: 71%|███████ | 71/100 [1:18:36<41:49, 86.53s/it] (TaskRunner pid=2347384) step:72 - global_seqlen/min:26234 - global_seqlen/max:43642 - global_seqlen/minmax_diff:17408 - global_seqlen/balanced_min:35514 - global_seqlen/balanced_max:36709 - global_seqlen/mean:35738.5 - actor/entropy:0.5381041765213013 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8288860321044922 - training/rollout_probs_diff_mean:0.003765193512663245 - training/rollout_probs_diff_std:0.008447541855275631 - training/rollout_actor_probs_pearson_corr:0.9995538592338562 - rollout_corr/rollout_is_mean:0.999891459941864 - rollout_corr/rollout_is_ratio_fraction_high:4.945305136061506e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.472652522556018e-05 - rollout_corr/rollout_is_max:2.261007785797119 - rollout_corr/rollout_is_min:0.026971934363245964 - rollout_corr/rollout_is_std:0.031901247799396515 - rollout_corr/rollout_is_eff_sample_size:0.9989831070005086 - rollout_corr/rollout_is_seq_mean:0.9999176263809204 - rollout_corr/rollout_is_seq_std:0.001497675315476954 - rollout_corr/rollout_is_seq_max:1.0063440799713135 - rollout_corr/rollout_is_seq_min:0.9956987500190735 - rollout_corr/rollout_is_seq_max_deviation:0.0063440799713134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.8888928033411503) - rollout_corr/training_log_ppl:np.float64(0.6195288861927111) - rollout_corr/kl:np.float64(0.0006622874627879582) - rollout_corr/k3_kl:np.float64(0.0006225923072804562) - rollout_corr/rollout_ppl:np.float64(1.8875912888906896) - rollout_corr/rollout_log_ppl:np.float64(0.6188665961381048) - rollout_corr/log_ppl_diff:np.float64(0.0006622900546062738) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012200255587231368) - rollout_corr/log_ppl_diff_max:np.float64(0.004466533660888672) - rollout_corr/log_ppl_diff_min:np.float64(-0.0036079883575439453) - rollout_corr/ppl_ratio:np.float64(1.0006635207682848) - rollout_corr/chi2_token:np.float64(0.0011803614906966686) - rollout_corr/chi2_seq:np.float64(0.9953480577096343) - actor/pg_loss:np.float64(-1.6446225345134735e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016102354027225374) - actor/ppo_kl:np.float64(-1.5812976214668595e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25563713535666466) - perf/mfu/actor:np.float64(0.12827517036615993) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.44058990478516) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:5 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02554064616560936 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02554064616560936 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:789.890625 - response_length/max:5425.0 - response_length/min:240.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:789.890625 - response_length_non_aborted/max:5425.0 - response_length_non_aborted/min:240.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:326.9375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015281513333320618 - timing_s/agent_loop/generate_sequences/min:np.float64(3.174361787736416) - timing_s/agent_loop/generate_sequences/max:np.float64(36.92909706197679) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.576639924409392) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(36.92909706197679) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:416 - timing_s/agent_loop/slowest/response_length:5425 - timing_s/gen:44.73805496841669 - timing_s/reward:0.07870646938681602 - timing_s/old_log_prob:3.50969978235662 - timing_s/adv:0.129770303145051 - timing_s/update_actor:14.501738341525197 - timing_s/step:63.0446462277323 - timing_s/stop_profile:2.95247882604599e-05 - timing_per_token_ms/update_actor:0.05072169488620534 - timing_per_token_ms/gen:0.22124332368215877 - timing_per_token_ms/adv:0.000453888324723516 - perf/total_num_tokens:285908 - perf/time_per_step:63.0446462277323 - perf/throughput:566.8760495681745 (TaskRunner pid=2347384) Training Progress: 72%|███████▏ | 72/100 [1:19:39<37:05, 79.50s/it] (TaskRunner pid=2347384) step:73 - global_seqlen/min:18456 - global_seqlen/max:34185 - global_seqlen/minmax_diff:15729 - global_seqlen/balanced_min:26807 - global_seqlen/balanced_max:26822 - global_seqlen/mean:26816.125 - actor/entropy:0.6565453410148621 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3378452658653259 - training/rollout_probs_diff_mean:0.004737126640975475 - training/rollout_probs_diff_std:0.009036988951265812 - training/rollout_actor_probs_pearson_corr:0.9995699524879456 - rollout_corr/rollout_is_mean:1.0000122785568237 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.862052032374777e-06 - rollout_corr/rollout_is_max:1.7784450054168701 - rollout_corr/rollout_is_min:0.3966788649559021 - rollout_corr/rollout_is_std:0.03564620390534401 - rollout_corr/rollout_is_eff_sample_size:0.9987310795967171 - rollout_corr/rollout_is_seq_mean:0.9999911785125732 - rollout_corr/rollout_is_seq_std:0.0017538155661895871 - rollout_corr/rollout_is_seq_max:1.0043984651565552 - rollout_corr/rollout_is_seq_min:0.9940966367721558 - rollout_corr/rollout_is_seq_max_deviation:0.005903363227844238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.975407786667347) - rollout_corr/training_log_ppl:np.float64(0.6694524426129647) - rollout_corr/kl:np.float64(0.0007055007173235595) - rollout_corr/k3_kl:np.float64(0.000711057839851037) - rollout_corr/rollout_ppl:np.float64(1.9739703219383955) - rollout_corr/rollout_log_ppl:np.float64(0.668746936309617) - rollout_corr/log_ppl_diff:np.float64(0.0007055063033476472) - rollout_corr/log_ppl_abs_diff:np.float64(0.001451704534702003) - rollout_corr/log_ppl_diff_max:np.float64(0.006269991397857666) - rollout_corr/log_ppl_diff_min:np.float64(-0.004275083541870117) - rollout_corr/ppl_ratio:np.float64(1.0007072330918163) - rollout_corr/chi2_token:np.float64(0.001449948875233531) - rollout_corr/chi2_seq:np.float64(0.9193006653804332) - actor/pg_loss:np.float64(-2.8471346013247967e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001824800635858992) - actor/ppo_kl:np.float64(2.208415196447433e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2451118528842926) - perf/mfu/actor:np.float64(0.10596420532794248) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.42919921875) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:5 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001449608476832509 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001449608476832509 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:569.25390625 - response_length/max:1432.0 - response_length/min:216.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:569.25390625 - response_length_non_aborted/max:1432.0 - response_length_non_aborted/min:216.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.316385507583618e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6824790202081203) - timing_s/agent_loop/generate_sequences/max:np.float64(12.198815386742353) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.428953409311362) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.198815386742353) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:289 - timing_s/agent_loop/slowest/response_length:1432 - timing_s/gen:19.612039322033525 - timing_s/reward:0.06841935031116009 - timing_s/old_log_prob:3.175808610394597 - timing_s/adv:0.17335210368037224 - timing_s/update_actor:13.279872816056013 - timing_s/step:36.40009694546461 - timing_s/stop_profile:0.0001198798418045044 - timing_per_token_ms/update_actor:0.061902459882141866 - timing_per_token_ms/gen:0.13457883689611214 - timing_per_token_ms/adv:0.0008080590674471621 - perf/total_num_tokens:214529 - perf/time_per_step:36.40009694546461 - perf/throughput:736.7047686762065 (TaskRunner pid=2347384) Training Progress: 73%|███████▎ | 73/100 [1:20:15<29:57, 66.59s/it] (TaskRunner pid=2347384) step:74 - global_seqlen/min:23559 - global_seqlen/max:37332 - global_seqlen/minmax_diff:13773 - global_seqlen/balanced_min:29294 - global_seqlen/balanced_max:29301 - global_seqlen/mean:29297.125 - actor/entropy:0.6140167713165283 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22837792336940765 - training/rollout_probs_diff_mean:0.0043827081099152565 - training/rollout_probs_diff_std:0.008729599416255951 - training/rollout_actor_probs_pearson_corr:0.9995607733726501 - rollout_corr/rollout_is_mean:0.9999913573265076 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2848763617512304e-05 - rollout_corr/rollout_is_max:1.7442296743392944 - rollout_corr/rollout_is_min:0.39243054389953613 - rollout_corr/rollout_is_std:0.0341971218585968 - rollout_corr/rollout_is_eff_sample_size:0.9988320418002583 - rollout_corr/rollout_is_seq_mean:1.0000061988830566 - rollout_corr/rollout_is_seq_std:0.0016586340498179197 - rollout_corr/rollout_is_seq_max:1.005176305770874 - rollout_corr/rollout_is_seq_min:0.9943239688873291 - rollout_corr/rollout_is_seq_max_deviation:0.0056760311126708984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.9385907775722444) - rollout_corr/training_log_ppl:np.float64(0.6473882601130754) - rollout_corr/kl:np.float64(0.00046914799348485303) - rollout_corr/k3_kl:np.float64(0.0006984650416939076) - rollout_corr/rollout_ppl:np.float64(1.9377238838933408) - rollout_corr/rollout_log_ppl:np.float64(0.6469191099167801) - rollout_corr/log_ppl_diff:np.float64(0.0004691501962952316) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013067128020338714) - rollout_corr/log_ppl_diff_max:np.float64(0.006288111209869385) - rollout_corr/log_ppl_diff_min:np.float64(-0.005218505859375) - rollout_corr/ppl_ratio:np.float64(1.0004705695901066) - rollout_corr/chi2_token:np.float64(0.0018577906303107738) - rollout_corr/chi2_seq:np.float64(1.442874014377594) - actor/pg_loss:np.float64(-2.4739885702729225e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025258869686695107) - actor/ppo_kl:np.float64(-0.00015895344180405502) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26296986266970634) - perf/mfu/actor:np.float64(0.11032628531317712) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.27821350097656) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:5 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005717699881643057 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.005717699881643057 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:608.03515625 - response_length/max:1934.0 - response_length/min:235.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:608.03515625 - response_length_non_aborted/max:1934.0 - response_length_non_aborted/min:235.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.5 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.83881002664566e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(3.0531131979078054) - timing_s/agent_loop/generate_sequences/max:np.float64(14.95792481303215) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.960708833503304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.95792481303215) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1934 - timing_s/gen:21.812017796561122 - timing_s/reward:0.07214215584099293 - timing_s/old_log_prob:3.187674755230546 - timing_s/adv:0.12273179553449154 - timing_s/update_actor:13.860707404091954 - timing_s/step:39.14221609942615 - timing_s/stop_profile:0.00012460164725780487 - timing_per_token_ms/update_actor:0.05913851360880954 - timing_per_token_ms/gen:0.14012873045581709 - timing_per_token_ms/adv:0.0005236511924569883 - perf/total_num_tokens:234377 - perf/time_per_step:39.14221609942615 - perf/throughput:748.4789549365734 (TaskRunner pid=2347384) Training Progress: 74%|███████▍ | 74/100 [1:20:55<25:17, 58.37s/it] (TaskRunner pid=2347384) step:75 - global_seqlen/min:19667 - global_seqlen/max:37303 - global_seqlen/minmax_diff:17636 - global_seqlen/balanced_min:31292 - global_seqlen/balanced_max:31535 - global_seqlen/mean:31373.875 - actor/entropy:0.5995069146156311 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17969393730163574 - training/rollout_probs_diff_mean:0.004113918170332909 - training/rollout_probs_diff_std:0.008396285586059093 - training/rollout_actor_probs_pearson_corr:0.9995931386947632 - rollout_corr/rollout_is_mean:1.0000536441802979 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.111908987804782e-06 - rollout_corr/rollout_is_max:1.6734154224395752 - rollout_corr/rollout_is_min:0.39688947796821594 - rollout_corr/rollout_is_std:0.03306100517511368 - rollout_corr/rollout_is_eff_sample_size:0.9989082823833357 - rollout_corr/rollout_is_seq_mean:1.0000866651535034 - rollout_corr/rollout_is_seq_std:0.001442281762138009 - rollout_corr/rollout_is_seq_max:1.0049080848693848 - rollout_corr/rollout_is_seq_min:0.9960072040557861 - rollout_corr/rollout_is_seq_max_deviation:0.004908084869384766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.938878451474011) - rollout_corr/training_log_ppl:np.float64(0.6489311760815326) - rollout_corr/kl:np.float64(0.0006695286699649472) - rollout_corr/k3_kl:np.float64(0.000658467178908495) - rollout_corr/rollout_ppl:np.float64(1.9376000561751425) - rollout_corr/rollout_log_ppl:np.float64(0.6482616436260287) - rollout_corr/log_ppl_diff:np.float64(0.0006695324555039406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013180868700146675) - rollout_corr/log_ppl_diff_max:np.float64(0.0052838921546936035) - rollout_corr/log_ppl_diff_min:np.float64(-0.006177842617034912) - rollout_corr/ppl_ratio:np.float64(1.000671031884849) - rollout_corr/chi2_token:np.float64(0.001291059423238039) - rollout_corr/chi2_seq:np.float64(1.7275320449844003) - actor/pg_loss:np.float64(2.64681875705719e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002035862344200723) - actor/ppo_kl:np.float64(0.00014959122469004882) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24130704253911972) - perf/mfu/actor:np.float64(0.11917468439495588) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.44599914550781) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:5 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018550407141447067 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018550407141447067 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:639.12109375 - response_length/max:2413.0 - response_length/min:219.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:639.12109375 - response_length_non_aborted/max:2413.0 - response_length_non_aborted/min:219.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:341.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010386854410171509 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8121387287974358) - timing_s/agent_loop/generate_sequences/max:np.float64(18.46451293863356) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.216221448288707) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.46451293863356) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:2369 - timing_s/gen:25.22961967252195 - timing_s/reward:0.07359615340828896 - timing_s/old_log_prob:3.3025001361966133 - timing_s/adv:0.1650307159870863 - timing_s/update_actor:13.809780960902572 - timing_s/step:42.66800866276026 - timing_s/stop_profile:0.00012409500777721405 - timing_per_token_ms/update_actor:0.05502102051827584 - timing_per_token_ms/gen:0.15420114092547718 - timing_per_token_ms/adv:0.000657516468666551 - perf/total_num_tokens:250991 - perf/time_per_step:42.66800866276026 - perf/throughput:735.3020678319693 (TaskRunner pid=2347384) Training Progress: 75%|███████▌ | 75/100 [1:21:37<22:21, 53.67s/it] (TaskRunner pid=2347384) step:76 - global_seqlen/min:20283 - global_seqlen/max:32811 - global_seqlen/minmax_diff:12528 - global_seqlen/balanced_min:26504 - global_seqlen/balanced_max:26517 - global_seqlen/mean:26512.5 - actor/entropy:0.6350308656692505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917877435684204 - training/rollout_probs_diff_mean:0.004586298018693924 - training/rollout_probs_diff_std:0.008971956558525562 - training/rollout_actor_probs_pearson_corr:0.9995372295379639 - rollout_corr/rollout_is_mean:1.0000380277633667 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.827222466468811 - rollout_corr/rollout_is_min:0.5686995387077332 - rollout_corr/rollout_is_std:0.03533213213086128 - rollout_corr/rollout_is_eff_sample_size:0.9987533156909227 - rollout_corr/rollout_is_seq_mean:1.0000674724578857 - rollout_corr/rollout_is_seq_std:0.0018894363893195987 - rollout_corr/rollout_is_seq_max:1.005513310432434 - rollout_corr/rollout_is_seq_min:0.9936065673828125 - rollout_corr/rollout_is_seq_max_deviation:0.0063934326171875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.968894435558468) - rollout_corr/training_log_ppl:np.float64(0.6658430882962421) - rollout_corr/kl:np.float64(0.0007195233954240265) - rollout_corr/k3_kl:np.float64(0.0007665455403866872) - rollout_corr/rollout_ppl:np.float64(1.9675318174995482) - rollout_corr/rollout_log_ppl:np.float64(0.665123556391336) - rollout_corr/log_ppl_diff:np.float64(0.0007195319049060345) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016117857303470373) - rollout_corr/log_ppl_diff_max:np.float64(0.009828507900238037) - rollout_corr/log_ppl_diff_min:np.float64(-0.005744516849517822) - rollout_corr/ppl_ratio:np.float64(1.0007219149265438) - rollout_corr/chi2_token:np.float64(0.0016625274438410997) - rollout_corr/chi2_seq:np.float64(0.8674180095549673) - actor/pg_loss:np.float64(-0.0004104750696569681) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002753232561190089) - actor/ppo_kl:np.float64(0.00010094181278219594) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2881191372871399) - perf/mfu/actor:np.float64(0.10367918366765841) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.43100261688232) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:5 - critic/score/mean:0.3671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004805878852494061 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004805878852494061 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:538.484375 - response_length/max:1629.0 - response_length/min:209.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:538.484375 - response_length_non_aborted/max:1629.0 - response_length_non_aborted/min:209.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.03125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.703341543674469e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.552292490378022) - timing_s/agent_loop/generate_sequences/max:np.float64(12.575598703697324) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.731568812974729) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.575598703697324) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:1543 - timing_s/gen:19.247506875544786 - timing_s/reward:0.06717930547893047 - timing_s/old_log_prob:3.1278603598475456 - timing_s/adv:0.11961442232131958 - timing_s/update_actor:13.31668702699244 - timing_s/step:35.96712216921151 - timing_s/stop_profile:0.00011586397886276245 - timing_per_token_ms/update_actor:0.06278494590755512 - timing_per_token_ms/gen:0.13962442964588678 - timing_per_token_ms/adv:0.0005639529576677019 - perf/total_num_tokens:212100 - perf/time_per_step:35.96712216921151 - perf/throughput:737.1315357194513 (TaskRunner pid=2347384) Training Progress: 76%|███████▌ | 76/100 [1:22:13<19:20, 48.37s/it] (TaskRunner pid=2347384) step:77 - global_seqlen/min:25157 - global_seqlen/max:34201 - global_seqlen/minmax_diff:9044 - global_seqlen/balanced_min:27673 - global_seqlen/balanced_max:27682 - global_seqlen/mean:27679.75 - actor/entropy:0.6796004176139832 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4517993927001953 - training/rollout_probs_diff_mean:0.004683211445808411 - training/rollout_probs_diff_std:0.008955772966146469 - training/rollout_actor_probs_pearson_corr:0.9995827674865723 - rollout_corr/rollout_is_mean:1.0000066757202148 - rollout_corr/rollout_is_ratio_fraction_high:1.4305537661130074e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4305537661130074e-05 - rollout_corr/rollout_is_max:2.7140040397644043 - rollout_corr/rollout_is_min:0.32336246967315674 - rollout_corr/rollout_is_std:0.03546515479683876 - rollout_corr/rollout_is_eff_sample_size:0.9987438028021615 - rollout_corr/rollout_is_seq_mean:0.9999802112579346 - rollout_corr/rollout_is_seq_std:0.0016934500308707356 - rollout_corr/rollout_is_seq_max:1.006002426147461 - rollout_corr/rollout_is_seq_min:0.9955582022666931 - rollout_corr/rollout_is_seq_max_deviation:0.0060024261474609375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.043808192014694) - rollout_corr/training_log_ppl:np.float64(0.7021688789827749) - rollout_corr/kl:np.float64(0.0007098385119235218) - rollout_corr/k3_kl:np.float64(0.0007099981705778191) - rollout_corr/rollout_ppl:np.float64(2.042325856164098) - rollout_corr/rollout_log_ppl:np.float64(0.701459038304165) - rollout_corr/log_ppl_diff:np.float64(0.0007098406786099076) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014463755069300532) - rollout_corr/log_ppl_diff_max:np.float64(0.005881637334823608) - rollout_corr/log_ppl_diff_min:np.float64(-0.00529932975769043) - rollout_corr/ppl_ratio:np.float64(1.0007116561755538) - rollout_corr/chi2_token:np.float64(0.0014412272721529007) - rollout_corr/chi2_seq:np.float64(1.0193054659757763) - actor/pg_loss:np.float64(-0.00013291975483298302) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021383253101703303) - actor/ppo_kl:np.float64(2.8176918711153576e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27433451265096664) - perf/mfu/actor:np.float64(0.10918538885073378) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.56311416625977) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:5 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004157546907663345 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004157546907663345 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:546.1171875 - response_length/max:1262.0 - response_length/min:197.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:546.1171875 - response_length_non_aborted/max:1262.0 - response_length_non_aborted/min:197.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010059215128421783 - timing_s/agent_loop/generate_sequences/min:np.float64(2.604365859180689) - timing_s/agent_loop/generate_sequences/max:np.float64(11.20194417051971) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.375331809722411) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.20194417051971) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1262 - timing_s/gen:17.961976988241076 - timing_s/reward:0.06827071122825146 - timing_s/old_log_prob:3.1079613491892815 - timing_s/adv:0.1182383131235838 - timing_s/update_actor:13.173749316483736 - timing_s/step:34.48337167315185 - timing_s/stop_profile:0.0001250244677066803 - timing_per_token_ms/update_actor:0.05949181855184628 - timing_per_token_ms/gen:0.12847786924911003 - timing_per_token_ms/adv:0.0005339567424000568 - perf/total_num_tokens:221438 - perf/time_per_step:34.48337167315185 - perf/throughput:802.6984792079068 (TaskRunner pid=2347384) Training Progress: 77%|███████▋ | 77/100 [1:22:48<16:57, 44.23s/it] (TaskRunner pid=2347384) step:78 - global_seqlen/min:23644 - global_seqlen/max:33932 - global_seqlen/minmax_diff:10288 - global_seqlen/balanced_min:28426 - global_seqlen/balanced_max:28449 - global_seqlen/mean:28444.0 - actor/entropy:0.6485849618911743 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16888165473937988 - training/rollout_probs_diff_mean:0.004333537071943283 - training/rollout_probs_diff_std:0.00853008683770895 - training/rollout_actor_probs_pearson_corr:0.9996119141578674 - rollout_corr/rollout_is_mean:1.0000039339065552 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8161580562591553 - rollout_corr/rollout_is_min:0.5402593612670898 - rollout_corr/rollout_is_std:0.034211061894893646 - rollout_corr/rollout_is_eff_sample_size:0.9988309714225022 - rollout_corr/rollout_is_seq_mean:0.999974250793457 - rollout_corr/rollout_is_seq_std:0.0015947482315823436 - rollout_corr/rollout_is_seq_max:1.0047028064727783 - rollout_corr/rollout_is_seq_min:0.9950341582298279 - rollout_corr/rollout_is_seq_max_deviation:0.004965841770172119 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0210465653799474) - rollout_corr/training_log_ppl:np.float64(0.6858960817335173) - rollout_corr/kl:np.float64(0.0007213819349995987) - rollout_corr/k3_kl:np.float64(0.000684070233944567) - rollout_corr/rollout_ppl:np.float64(2.0195200140587986) - rollout_corr/rollout_log_ppl:np.float64(0.6851746974280104) - rollout_corr/log_ppl_diff:np.float64(0.0007213843055069447) - rollout_corr/log_ppl_abs_diff:np.float64(0.001465151784941554) - rollout_corr/log_ppl_diff_max:np.float64(0.008537650108337402) - rollout_corr/log_ppl_diff_min:np.float64(-0.004275679588317871) - rollout_corr/ppl_ratio:np.float64(1.0007232842035592) - rollout_corr/chi2_token:np.float64(0.0013091855216771364) - rollout_corr/chi2_seq:np.float64(1.0697790174745023) - actor/pg_loss:np.float64(0.0002961248392239213) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014098366227699444) - actor/ppo_kl:np.float64(6.694363881720733e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2639455832540989) - perf/mfu/actor:np.float64(0.11271615823649146) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.49803924560547) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:5 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0016177310608327389 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0016177310608327389 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:560.5 - response_length/max:1547.0 - response_length/min:195.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:560.5 - response_length_non_aborted/max:1547.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001051872968673706 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3753585014492273) - timing_s/agent_loop/generate_sequences/max:np.float64(12.783233623951674) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.357404050890182) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.783233623951674) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1547 - timing_s/gen:19.534673934802413 - timing_s/reward:0.06886883825063705 - timing_s/old_log_prob:3.174489678815007 - timing_s/adv:0.11739128082990646 - timing_s/update_actor:13.177136279642582 - timing_s/step:36.16048224456608 - timing_s/stop_profile:0.00011270865797996521 - timing_per_token_ms/update_actor:0.05790824198267904 - timing_per_token_ms/gen:0.13614151660628354 - timing_per_token_ms/adv:0.0005158877128300629 - perf/total_num_tokens:227552 - perf/time_per_step:36.16048224456608 - perf/throughput:786.6045537673753 (TaskRunner pid=2347384) Training Progress: 78%|███████▊ | 78/100 [1:23:24<15:20, 41.82s/it] (TaskRunner pid=2347384) step:79 - global_seqlen/min:19696 - global_seqlen/max:31968 - global_seqlen/minmax_diff:12272 - global_seqlen/balanced_min:26870 - global_seqlen/balanced_max:26875 - global_seqlen/mean:26873.5 - actor/entropy:0.6964158415794373 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33973264694213867 - training/rollout_probs_diff_mean:0.004594298545271158 - training/rollout_probs_diff_std:0.008812104351818562 - training/rollout_actor_probs_pearson_corr:0.9996107816696167 - rollout_corr/rollout_is_mean:0.9997699856758118 - rollout_corr/rollout_is_ratio_fraction_high:7.363987151620677e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.363987151620677e-06 - rollout_corr/rollout_is_max:2.002164602279663 - rollout_corr/rollout_is_min:0.009415961802005768 - rollout_corr/rollout_is_std:0.03540375828742981 - rollout_corr/rollout_is_eff_sample_size:0.99874760793592 - rollout_corr/rollout_is_seq_mean:0.9997334480285645 - rollout_corr/rollout_is_seq_std:0.00164352310821414 - rollout_corr/rollout_is_seq_max:1.0043056011199951 - rollout_corr/rollout_is_seq_min:0.9954257607460022 - rollout_corr/rollout_is_seq_max_deviation:0.004574239253997803 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.086203448474407) - rollout_corr/training_log_ppl:np.float64(0.7215332236373797) - rollout_corr/kl:np.float64(0.0009883559448091717) - rollout_corr/k3_kl:np.float64(0.0007297017808696182) - rollout_corr/rollout_ppl:np.float64(2.08414251729846) - rollout_corr/rollout_log_ppl:np.float64(0.7205448638414964) - rollout_corr/log_ppl_diff:np.float64(0.000988359795883298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016334704123437405) - rollout_corr/log_ppl_diff_max:np.float64(0.0070310235023498535) - rollout_corr/log_ppl_diff_min:np.float64(-0.004558265209197998) - rollout_corr/ppl_ratio:np.float64(1.0009905940387398) - rollout_corr/chi2_token:np.float64(0.0009604699444025755) - rollout_corr/chi2_seq:np.float64(0.667339360807091) - actor/pg_loss:np.float64(-0.0001147101866081357) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016125454158100183) - actor/ppo_kl:np.float64(4.487651064266629e-05) - actor/pg_clipfrac_lower:np.float64(4.876716502622003e-06) - actor/grad_norm:np.float64(0.29671041294932365) - perf/mfu/actor:np.float64(0.10439448959828303) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.48786544799805) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:5 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009948746301233768 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009948746301233768 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:530.453125 - response_length/max:1364.0 - response_length/min:188.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:530.453125 - response_length_non_aborted/max:1364.0 - response_length_non_aborted/min:188.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.34375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014105066657066345 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3973023276776075) - timing_s/agent_loop/generate_sequences/max:np.float64(11.570676302537322) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.052781823455007) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.570676302537322) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:287 - timing_s/agent_loop/slowest/response_length:1364 - timing_s/gen:18.360605031251907 - timing_s/reward:0.06794219091534615 - timing_s/old_log_prob:3.106847492977977 - timing_s/adv:0.1279370728880167 - timing_s/update_actor:13.429241504520178 - timing_s/step:35.17842346616089 - timing_s/stop_profile:3.9618462324142456e-05 - timing_per_token_ms/update_actor:0.062465074815897526 - timing_per_token_ms/gen:0.13520725964867822 - timing_per_token_ms/adv:0.0005950893672577851 - perf/total_num_tokens:214988 - perf/time_per_step:35.17842346616089 - perf/throughput:763.9199643454849 (TaskRunner pid=2347384) Training Progress: 79%|███████▉ | 79/100 [1:23:59<13:56, 39.84s/it] (TaskRunner pid=2347384) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2347384) validation generation end (TaskRunner pid=2347384) [prompt] system (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) To determine the folding stability score of a protein sequence, we need to evaluate the overall balance of stabilizing and destabilizing interactions within the folded structure. These interactions include hydrogen bonds, hydrophobic effects, electrostatic interactions, and van der Waals forces. Stability scores are often calculated using computational tools that analyze the sequence and predict the likelihood of proper folding and the energetic stability of the resulting structure. (TaskRunner pid=2347384) (TaskRunner pid=2347384) The given sequence is: (TaskRunner pid=2347384) **GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR** (TaskRunner pid=2347384) (TaskRunner pid=2347384) Analyzing this sequence: (TaskRunner pid=2347384) (TaskRunner pid=2347384) 1. **Hydrophobic interactions**: Amino acids such as Gly, Ser, Thr, and Val are relatively hydrophilic, while residues like Trp, Tyr, and Arg are hydrophobic or charged. The presence of charged residues (e.g., Lys, Arg, Glu, Asp) suggests potential for electrostatic interactions, which can stabilize the folded structure. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 2. **Charge distribution**: The sequence has several positively charged residues (Arg, Lys) and negatively charged residues (Glu, Asp), which may lead to salt bridge formation, a stabilizing factor in protein folding. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 3. **Hydrogen bonding**: The sequence contains multiple polar residues (e.g., Ser, Thr, Tyr, Asn, Gln), which are likely to form hydrogen bonds, contributing to folding stability. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 4. **Secondary structure potential**: The presence of repeating charged and hydrophobic residues may suggest a propensity for alpha-helical or beta-sheet formation, which can enhance stability. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 5. **Computational prediction**: Using a known algorithm or tool (e.g., DSSP, FoldX, or similar), the sequence would be analyzed to calculate a folding stability score. Stability scores typically range from negative (unstable) to positive (stable), with higher values indicating better folding stability. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Based on the above considerations, the sequence appears to have a relatively high potential for folding stability, as it contains a good balance of hydrophobic, hydrophilic, and charged residues, which are essential for forming stable secondary and tertiary structures. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Among the given options: (TaskRunner pid=2347384) - **A: -0.01** — Suggests a very low or slightly negative stability, which is unlikely for a sequence with multiple stabilizing interactions. (TaskRunner pid=2347384) - **B: 1.69** — Indicates moderate stability. (TaskRunner pid=2347384) - **C: 2.49** — Indicates a high level of folding stability, which is plausible given the sequence's composition. (TaskRunner pid=2347384) - **D: 0.45** — Indicates a moderate level of stability, which is less likely than option C. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Therefore, the most reasonable answer is **C: 2.49**, as it reflects a high folding stability score consistent with the sequence's characteristics. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) C (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 0.0 (TaskRunner pid=2347384) [acc] 0.0 (TaskRunner pid=2347384) [pred] C (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2347384) step:80 - global_seqlen/min:21635 - global_seqlen/max:35240 - global_seqlen/minmax_diff:13605 - global_seqlen/balanced_min:27813 - global_seqlen/balanced_max:27832 - global_seqlen/mean:27824.375 - actor/entropy:0.6880682110786438 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21059486269950867 - training/rollout_probs_diff_mean:0.004430724307894707 - training/rollout_probs_diff_std:0.008513307198882103 - training/rollout_actor_probs_pearson_corr:0.9996291995048523 - rollout_corr/rollout_is_mean:1.0001370906829834 - rollout_corr/rollout_is_ratio_fraction_high:6.905552709213225e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:9.692143440246582 - rollout_corr/rollout_is_min:0.5781880617141724 - rollout_corr/rollout_is_std:0.034712642431259155 - rollout_corr/rollout_is_eff_sample_size:0.9987967205455502 - rollout_corr/rollout_is_seq_mean:1.0000431537628174 - rollout_corr/rollout_is_seq_std:0.001761785359121859 - rollout_corr/rollout_is_seq_max:1.0066933631896973 - rollout_corr/rollout_is_seq_min:0.995124876499176 - rollout_corr/rollout_is_seq_max_deviation:0.006693363189697266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.105774308554828) - rollout_corr/training_log_ppl:np.float64(0.7272331639542244) - rollout_corr/kl:np.float64(0.0007196503398532172) - rollout_corr/k3_kl:np.float64(0.0006930520933678963) - rollout_corr/rollout_ppl:np.float64(2.104170592967421) - rollout_corr/rollout_log_ppl:np.float64(0.7265135105117224) - rollout_corr/log_ppl_diff:np.float64(0.0007196534425020218) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014593529049307108) - rollout_corr/log_ppl_diff_max:np.float64(0.005758881568908691) - rollout_corr/log_ppl_diff_min:np.float64(-0.005846261978149414) - rollout_corr/ppl_ratio:np.float64(1.000721426680684) - rollout_corr/chi2_token:np.float64(0.0015012603253126144) - rollout_corr/chi2_seq:np.float64(1.1589177006389946) - actor/pg_loss:np.float64(0.00018891145009547472) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001872696202553925) - actor/ppo_kl:np.float64(0.0001051936815343879) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2599523887038231) - perf/mfu/actor:np.float64(0.10929031349885825) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.53002166748047) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4875) - val-aux/sciknoweval/reward/std@16:np.float64(0.2717185290898952) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.59048) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2290722979375459) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.38099999999999995) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.22787553358834262) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4851) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2703209420327715) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6778) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.18127860154420353) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.29338) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1792149680833091) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5096) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20285759046433108) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.75142) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.13853854484060268) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.21716000000000002) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.13436844700805703) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5245599999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13391965760219748) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.80976) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.09433928040011606) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.16302) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.08422725041535316) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5365000000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0899571876545685) - val-aux/sciknoweval/score/mean@16:np.float64(0.4875) - val-aux/sciknoweval/score/std@16:np.float64(0.2717185290898952) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.59048) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2290722979375459) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.38099999999999995) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.22787553358834262) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4851) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2703209420327715) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6778) - val-aux/sciknoweval/score/best@4/std:np.float64(0.18127860154420353) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.29338) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1792149680833091) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5096) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20285759046433108) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.75142) - val-aux/sciknoweval/score/best@8/std:np.float64(0.13853854484060268) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.21716000000000002) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.13436844700805703) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5245599999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13391965760219748) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.80976) - val-aux/sciknoweval/score/best@16/std:np.float64(0.09433928040011606) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.16302) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.08422725041535316) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5365000000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0899571876545685) - val-core/sciknoweval/acc/mean@16:np.float64(0.4875) - val-aux/sciknoweval/acc/std@16:np.float64(0.2717185290898952) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.59048) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2290722979375459) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.38099999999999995) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.22787553358834262) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4851) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2703209420327715) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6778) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.18127860154420353) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.29338) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1792149680833091) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5096) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20285759046433108) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.75142) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.13853854484060268) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.21716000000000002) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.13436844700805703) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5245599999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13391965760219748) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.80976) - val-core/sciknoweval/acc/best@16/std:np.float64(0.09433928040011606) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.16302) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.08422725041535316) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5365000000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0899571876545685) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9525) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.07647884125266201) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9841799999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.04383783176593101) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.92134) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.08427239562793237) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9528200000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.07567872160412564) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9972) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.01633083091034486) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.88412) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07964320072050732) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9732000000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.05679947624864387) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.99988) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0020428128231377535) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8509599999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0627767641464096) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.98854) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.034399142648868716) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8256) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.043489639321060224) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9971000000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.014843508581270601) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:5 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012404099106788635 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012404099106788635 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:565.66796875 - response_length/max:1617.0 - response_length/min:197.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:565.66796875 - response_length_non_aborted/max:1617.0 - response_length_non_aborted/min:197.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.84375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.647143840789795e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5439559053629637) - timing_s/agent_loop/generate_sequences/max:np.float64(13.19933489523828) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.313844419870293) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.19933489523828) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:1606 - timing_s/gen:19.880537405610085 - timing_s/reward:0.07015124894678593 - timing_s/old_log_prob:3.2011679485440254 - timing_s/adv:0.12775819189846516 - timing_s/update_actor:13.358908515423536 - timing_s/step:36.72310545668006 - timing_s/testing:113.02173636108637 - timing_s/save_checkpoint:12.659695487469435 - timing_s/stop_profile:5.769170820713043e-05 - timing_per_token_ms/update_actor:0.06001441413968659 - timing_per_token_ms/gen:0.13728609985160026 - timing_per_token_ms/adv:0.0005739490639882529 - perf/total_num_tokens:222595 - perf/time_per_step:36.72310545668006 - perf/throughput:757.6803392300977 (TaskRunner pid=2347384) Training Progress: 80%|████████ | 80/100 [1:26:42<25:32, 76.62s/it] (TaskRunner pid=2347384) step:81 - global_seqlen/min:25146 - global_seqlen/max:35521 - global_seqlen/minmax_diff:10375 - global_seqlen/balanced_min:30316 - global_seqlen/balanced_max:30334 - global_seqlen/mean:30330.125 - actor/entropy:0.6891587972640991 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19548368453979492 - training/rollout_probs_diff_mean:0.004188473802059889 - training/rollout_probs_diff_std:0.008300716988742352 - training/rollout_actor_probs_pearson_corr:0.9996699094772339 - rollout_corr/rollout_is_mean:1.0000306367874146 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.5681874730216805e-06 - rollout_corr/rollout_is_max:1.659632682800293 - rollout_corr/rollout_is_min:0.31874755024909973 - rollout_corr/rollout_is_std:0.034013617783784866 - rollout_corr/rollout_is_eff_sample_size:0.9988445297102561 - rollout_corr/rollout_is_seq_mean:1.0000059604644775 - rollout_corr/rollout_is_seq_std:0.0014868933940306306 - rollout_corr/rollout_is_seq_max:1.004580020904541 - rollout_corr/rollout_is_seq_min:0.9953084588050842 - rollout_corr/rollout_is_seq_max_deviation:0.0046915411949157715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.118280502036214) - rollout_corr/training_log_ppl:np.float64(0.7280188084114343) - rollout_corr/kl:np.float64(0.0006297170498879012) - rollout_corr/k3_kl:np.float64(0.0006460118133304604) - rollout_corr/rollout_ppl:np.float64(2.116966078057885) - rollout_corr/rollout_log_ppl:np.float64(0.7273890938377008) - rollout_corr/log_ppl_diff:np.float64(0.0006297145737335086) - rollout_corr/log_ppl_abs_diff:np.float64(0.001367017743177712) - rollout_corr/log_ppl_diff_max:np.float64(0.005467712879180908) - rollout_corr/log_ppl_diff_min:np.float64(-0.004240751266479492) - rollout_corr/ppl_ratio:np.float64(1.0006312557961792) - rollout_corr/chi2_token:np.float64(0.0013324737083166838) - rollout_corr/chi2_seq:np.float64(0.8296553255058825) - actor/pg_loss:np.float64(2.6099151000380516e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001224870741225459) - actor/ppo_kl:np.float64(1.4704056052750047e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26757555082440376) - perf/mfu/actor:np.float64(0.11731088274491351) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(107.79968643188477) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:5 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01134572271257639 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01134572271257639 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:594.72265625 - response_length/max:1715.0 - response_length/min:195.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:594.72265625 - response_length_non_aborted/max:1715.0 - response_length_non_aborted/min:195.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:353.09375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.851950168609619e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4953945614397526) - timing_s/agent_loop/generate_sequences/max:np.float64(13.916848164051771) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.67386929008353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.916848164051771) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:1715 - timing_s/gen:20.836444603279233 - timing_s/reward:0.06726226210594177 - timing_s/old_log_prob:3.2936479300260544 - timing_s/adv:0.1266807373613119 - timing_s/update_actor:13.500305961817503 - timing_s/step:37.92319720052183 - timing_s/stop_profile:0.00011635757982730865 - timing_per_token_ms/update_actor:0.05563901385923031 - timing_per_token_ms/gen:0.1368576779044804 - timing_per_token_ms/adv:0.0005220912267972516 - perf/total_num_tokens:242641 - perf/time_per_step:37.92319720052183 - perf/throughput:799.7776358260916 (TaskRunner pid=2347384) Training Progress: 81%|████████ | 81/100 [1:27:20<20:35, 65.03s/it] (TaskRunner pid=2347384) step:82 - global_seqlen/min:19610 - global_seqlen/max:40283 - global_seqlen/minmax_diff:20673 - global_seqlen/balanced_min:28404 - global_seqlen/balanced_max:34937 - global_seqlen/mean:29226.25 - actor/entropy:0.686008095741272 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16780120134353638 - training/rollout_probs_diff_mean:0.00410358514636755 - training/rollout_probs_diff_std:0.00808172021061182 - training/rollout_actor_probs_pearson_corr:0.9996803402900696 - rollout_corr/rollout_is_mean:0.9998846054077148 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6293654441833496 - rollout_corr/rollout_is_min:0.5711241364479065 - rollout_corr/rollout_is_std:0.03366131708025932 - rollout_corr/rollout_is_eff_sample_size:0.9988680791901611 - rollout_corr/rollout_is_seq_mean:0.9998651146888733 - rollout_corr/rollout_is_seq_std:0.0017086053267121315 - rollout_corr/rollout_is_seq_max:1.0089727640151978 - rollout_corr/rollout_is_seq_min:0.9950680732727051 - rollout_corr/rollout_is_seq_max_deviation:0.008972764015197754 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1433401028625667) - rollout_corr/training_log_ppl:np.float64(0.7446303129618173) - rollout_corr/kl:np.float64(0.0006822592909143665) - rollout_corr/k3_kl:np.float64(0.0006462996776974705) - rollout_corr/rollout_ppl:np.float64(2.1418757284991443) - rollout_corr/rollout_log_ppl:np.float64(0.7439480532339076) - rollout_corr/log_ppl_diff:np.float64(0.0006822597279096954) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013924166050856002) - rollout_corr/log_ppl_diff_max:np.float64(0.005618095397949219) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035376548767089844) - rollout_corr/ppl_ratio:np.float64(1.000683878082782) - rollout_corr/chi2_token:np.float64(0.001227846136316657) - rollout_corr/chi2_seq:np.float64(0.5843987429980189) - actor/pg_loss:np.float64(9.488489013165236e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025142339791273116) - actor/ppo_kl:np.float64(-7.640201083702891e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29759738594293594) - perf/mfu/actor:np.float64(0.10630120295716569) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(105.10164022445679) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:5 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010403398424386978 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010403398424386978 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:607.1953125 - response_length/max:8192.0 - response_length/min:211.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:607.1953125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:211.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:306.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010200776159763336 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6243486031889915) - timing_s/agent_loop/generate_sequences/max:np.float64(60.595920572057366) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.673203655576799) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(60.595920572057366) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:67.26469878293574 - timing_s/reward:0.06670200265944004 - timing_s/old_log_prob:3.6930709164589643 - timing_s/adv:0.1629076600074768 - timing_s/update_actor:14.20418213121593 - timing_s/step:85.47302486374974 - timing_s/stop_profile:0.00011339783668518066 - timing_per_token_ms/update_actor:0.060750960742551346 - timing_per_token_ms/gen:0.43273181497237384 - timing_per_token_ms/adv:0.0006967523202920183 - perf/total_num_tokens:233810 - perf/time_per_step:85.47302486374974 - perf/throughput:341.93536553302965 (TaskRunner pid=2347384) Training Progress: 82%|████████▏ | 82/100 [1:28:45<21:21, 71.18s/it] (TaskRunner pid=2347384) step:83 - global_seqlen/min:21423 - global_seqlen/max:33669 - global_seqlen/minmax_diff:12246 - global_seqlen/balanced_min:26443 - global_seqlen/balanced_max:26801 - global_seqlen/mean:26493.875 - actor/entropy:0.7207464575767517 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19547975063323975 - training/rollout_probs_diff_mean:0.0044199395924806595 - training/rollout_probs_diff_std:0.008346516638994217 - training/rollout_actor_probs_pearson_corr:0.999643862247467 - rollout_corr/rollout_is_mean:1.0003044605255127 - rollout_corr/rollout_is_ratio_fraction_high:7.452508725691587e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.270595073699951 - rollout_corr/rollout_is_min:0.5528151988983154 - rollout_corr/rollout_is_std:0.03488905355334282 - rollout_corr/rollout_is_eff_sample_size:0.9987848284293472 - rollout_corr/rollout_is_seq_mean:1.0003567934036255 - rollout_corr/rollout_is_seq_std:0.0018399786204099655 - rollout_corr/rollout_is_seq_max:1.0066711902618408 - rollout_corr/rollout_is_seq_min:0.995099663734436 - rollout_corr/rollout_is_seq_max_deviation:0.00667119026184082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1568660335615277) - rollout_corr/training_log_ppl:np.float64(0.7514473445480689) - rollout_corr/kl:np.float64(0.0004171081800505849) - rollout_corr/k3_kl:np.float64(0.0006982041508081238) - rollout_corr/rollout_ppl:np.float64(2.1558911218307912) - rollout_corr/rollout_log_ppl:np.float64(0.7510302303126082) - rollout_corr/log_ppl_diff:np.float64(0.0004171142354607582) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013870112597942352) - rollout_corr/log_ppl_diff_max:np.float64(0.006706118583679199) - rollout_corr/log_ppl_diff_min:np.float64(-0.004748940467834473) - rollout_corr/ppl_ratio:np.float64(1.0004187426529825) - rollout_corr/chi2_token:np.float64(0.0019722378347069025) - rollout_corr/chi2_seq:np.float64(1.3311789503786713) - actor/pg_loss:np.float64(3.966991789638996e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003556140113687434) - actor/ppo_kl:np.float64(0.00012838464786035786) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3239940069615841) - perf/mfu/actor:np.float64(0.10340301139466113) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.96197891235352) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:5 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004886982496827841 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004886982496827841 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:524.15234375 - response_length/max:1899.0 - response_length/min:221.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:524.15234375 - response_length_non_aborted/max:1899.0 - response_length_non_aborted/min:221.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011915154755115509 - timing_s/agent_loop/generate_sequences/min:np.float64(2.735219283029437) - timing_s/agent_loop/generate_sequences/max:np.float64(14.01869953982532) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.942329752062506) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.01869953982532) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1899 - timing_s/gen:20.68921371549368 - timing_s/reward:0.06755134277045727 - timing_s/old_log_prob:3.0983404833823442 - timing_s/adv:0.1196957379579544 - timing_s/update_actor:13.09600518271327 - timing_s/step:37.157608503475785 - timing_s/stop_profile:0.00011878088116645813 - timing_per_token_ms/update_actor:0.06178789051579502 - timing_per_token_ms/gen:0.1541865490821764 - timing_per_token_ms/adv:0.0005647330654630288 - perf/total_num_tokens:211951 - perf/time_per_step:37.157608503475785 - perf/throughput:713.0134598818898 (TaskRunner pid=2347384) Training Progress: 83%|████████▎ | 83/100 [1:29:22<17:16, 60.99s/it] (TaskRunner pid=2347384) step:84 - global_seqlen/min:21267 - global_seqlen/max:37146 - global_seqlen/minmax_diff:15879 - global_seqlen/balanced_min:27960 - global_seqlen/balanced_max:27970 - global_seqlen/mean:27967.0 - actor/entropy:0.6945655345916748 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29340335726737976 - training/rollout_probs_diff_mean:0.004248195793479681 - training/rollout_probs_diff_std:0.008358235470950603 - training/rollout_actor_probs_pearson_corr:0.9996412396430969 - rollout_corr/rollout_is_mean:1.0000907182693481 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.200460913736606e-06 - rollout_corr/rollout_is_max:1.8420875072479248 - rollout_corr/rollout_is_min:0.4248071014881134 - rollout_corr/rollout_is_std:0.034256331622600555 - rollout_corr/rollout_is_eff_sample_size:0.998828117093035 - rollout_corr/rollout_is_seq_mean:1.0001178979873657 - rollout_corr/rollout_is_seq_std:0.001721128704957664 - rollout_corr/rollout_is_seq_max:1.0046639442443848 - rollout_corr/rollout_is_seq_min:0.9950989484786987 - rollout_corr/rollout_is_seq_max_deviation:0.0049010515213012695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.126299354247749) - rollout_corr/training_log_ppl:np.float64(0.7351681315340102) - rollout_corr/kl:np.float64(0.0005097331647085923) - rollout_corr/k3_kl:np.float64(0.0006819142585072768) - rollout_corr/rollout_ppl:np.float64(2.1252669882960618) - rollout_corr/rollout_log_ppl:np.float64(0.7346583993639797) - rollout_corr/log_ppl_diff:np.float64(0.0005097321700304747) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014854713808745146) - rollout_corr/log_ppl_diff_max:np.float64(0.006580650806427002) - rollout_corr/log_ppl_diff_min:np.float64(-0.004427611827850342) - rollout_corr/ppl_ratio:np.float64(1.0005115061067045) - rollout_corr/chi2_token:np.float64(0.0017158279661089182) - rollout_corr/chi2_seq:np.float64(0.9766051645856351) - actor/pg_loss:np.float64(5.382811650633812e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019292376032353786) - actor/ppo_kl:np.float64(-7.4285403606211275e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28584229201078415) - perf/mfu/actor:np.float64(0.10935386559195037) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.89813232421875) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:5 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014970658347010612 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014970658347010612 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:542.5 - response_length/max:1579.0 - response_length/min:208.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:542.5 - response_length_non_aborted/max:1579.0 - response_length_non_aborted/min:208.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:331.46875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.490867912769318e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6602510679513216) - timing_s/agent_loop/generate_sequences/max:np.float64(13.035030463710427) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.069774381707248) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.035030463710427) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:1579 - timing_s/gen:19.53294676914811 - timing_s/reward:0.06954569928348064 - timing_s/old_log_prob:3.147978825494647 - timing_s/adv:0.1222816277295351 - timing_s/update_actor:13.136210618540645 - timing_s/step:36.09382627531886 - timing_s/stop_profile:7.005035877227783e-05 - timing_per_token_ms/update_actor:0.05871299486243003 - timing_per_token_ms/gen:0.14064621809582453 - timing_per_token_ms/adv:0.0005465442652480383 - perf/total_num_tokens:223736 - perf/time_per_step:36.09382627531886 - perf/throughput:774.8416526048383 (TaskRunner pid=2347384) Training Progress: 84%|████████▍ | 84/100 [1:29:58<14:16, 53.53s/it] (TaskRunner pid=2347384) step:85 - global_seqlen/min:21405 - global_seqlen/max:32074 - global_seqlen/minmax_diff:10669 - global_seqlen/balanced_min:26167 - global_seqlen/balanced_max:26187 - global_seqlen/mean:26182.0 - actor/entropy:0.7858491539955139 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4392957091331482 - training/rollout_probs_diff_mean:0.004719891119748354 - training/rollout_probs_diff_std:0.008597408421337605 - training/rollout_actor_probs_pearson_corr:0.9996190071105957 - rollout_corr/rollout_is_mean:1.000009298324585 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.526040008087875e-06 - rollout_corr/rollout_is_max:1.6576542854309082 - rollout_corr/rollout_is_min:0.12027852237224579 - rollout_corr/rollout_is_std:0.035881198942661285 - rollout_corr/rollout_is_eff_sample_size:0.9987141950956544 - rollout_corr/rollout_is_seq_mean:1.000095248222351 - rollout_corr/rollout_is_seq_std:0.0016835088608786464 - rollout_corr/rollout_is_seq_max:1.005692481994629 - rollout_corr/rollout_is_seq_min:0.994841992855072 - rollout_corr/rollout_is_seq_max_deviation:0.005692481994628906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.269857001490891) - rollout_corr/training_log_ppl:np.float64(0.806094512459822) - rollout_corr/kl:np.float64(0.0005885758834391908) - rollout_corr/k3_kl:np.float64(0.0007050533647543489) - rollout_corr/rollout_ppl:np.float64(2.2685238495469093) - rollout_corr/rollout_log_ppl:np.float64(0.8055059344042093) - rollout_corr/log_ppl_diff:np.float64(0.0005885780556127429) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014217320131137967) - rollout_corr/log_ppl_diff_max:np.float64(0.005595147609710693) - rollout_corr/log_ppl_diff_min:np.float64(-0.005057334899902344) - rollout_corr/ppl_ratio:np.float64(1.0005902617704123) - rollout_corr/chi2_token:np.float64(0.0016473988071084023) - rollout_corr/chi2_seq:np.float64(0.7892564854118973) - actor/pg_loss:np.float64(6.9422647356987e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012136728582845535) - actor/ppo_kl:np.float64(9.103724437409255e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.275136798620224) - perf/mfu/actor:np.float64(0.1049194029127459) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.56171798706055) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:6 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0074517205357551575 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0074517205357551575 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:519.03125 - response_length/max:1381.0 - response_length/min:206.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:519.03125 - response_length_non_aborted/max:1381.0 - response_length_non_aborted/min:206.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.15625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00036164745688438416 - timing_s/agent_loop/generate_sequences/min:np.float64(2.641048202291131) - timing_s/agent_loop/generate_sequences/max:np.float64(11.569497916847467) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.01978995812533) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.569497916847467) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1381 - timing_s/gen:18.682460702955723 - timing_s/reward:0.06173485144972801 - timing_s/old_log_prob:3.406264690682292 - timing_s/adv:0.165722344070673 - timing_s/update_actor:12.83843670412898 - timing_s/step:35.27484982833266 - timing_s/stop_profile:0.00011977925896644592 - timing_per_token_ms/update_actor:0.06129419402704616 - timing_per_token_ms/gen:0.14060494839360982 - timing_per_token_ms/adv:0.0007912036135067651 - perf/total_num_tokens:209456 - perf/time_per_step:35.27484982833266 - perf/throughput:742.2285318694876 (TaskRunner pid=2347384) Training Progress: 85%|████████▌ | 85/100 [1:30:34<12:02, 48.20s/it] (TaskRunner pid=2347384) step:86 - global_seqlen/min:19682 - global_seqlen/max:37294 - global_seqlen/minmax_diff:17612 - global_seqlen/balanced_min:27964 - global_seqlen/balanced_max:29745 - global_seqlen/mean:28198.125 - actor/entropy:0.70988929271698 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1860121190547943 - training/rollout_probs_diff_mean:0.004345105029642582 - training/rollout_probs_diff_std:0.008342367596924305 - training/rollout_actor_probs_pearson_corr:0.9996600151062012 - rollout_corr/rollout_is_mean:0.9999462366104126 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6360780000686646 - rollout_corr/rollout_is_min:0.6581078767776489 - rollout_corr/rollout_is_std:0.034631844609975815 - rollout_corr/rollout_is_eff_sample_size:0.9988019531663942 - rollout_corr/rollout_is_seq_mean:0.9998987317085266 - rollout_corr/rollout_is_seq_std:0.0016613727202638984 - rollout_corr/rollout_is_seq_max:1.0039769411087036 - rollout_corr/rollout_is_seq_min:0.993069589138031 - rollout_corr/rollout_is_seq_max_deviation:0.006930410861968994 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1687073512002826) - rollout_corr/training_log_ppl:np.float64(0.7549141421914101) - rollout_corr/kl:np.float64(0.0008395029327878945) - rollout_corr/k3_kl:np.float64(0.0006729341305913294) - rollout_corr/rollout_ppl:np.float64(2.1668638051487505) - rollout_corr/rollout_log_ppl:np.float64(0.7540746349841356) - rollout_corr/log_ppl_diff:np.float64(0.000839507207274437) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014746382366865873) - rollout_corr/log_ppl_diff_max:np.float64(0.0075005292892456055) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031682252883911133) - rollout_corr/ppl_ratio:np.float64(1.0008413784671575) - rollout_corr/chi2_token:np.float64(0.0010109413415193558) - rollout_corr/chi2_seq:np.float64(0.45675797667354345) - actor/pg_loss:np.float64(-0.0001677896361798048) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025052251385204727) - actor/ppo_kl:np.float64(8.880921075160586e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29671191424131393) - perf/mfu/actor:np.float64(0.10864218597143618) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.52780151367188) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:6 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004235010594129562 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004235010594129562 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:563.22265625 - response_length/max:3676.0 - response_length/min:178.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:563.22265625 - response_length_non_aborted/max:3676.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.96875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010552443563938141 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3972385227680206) - timing_s/agent_loop/generate_sequences/max:np.float64(24.382686145603657) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.4161440088792006) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.382686145603657) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:3676 - timing_s/gen:31.087301252409816 - timing_s/reward:0.07022450864315033 - timing_s/old_log_prob:3.2779465466737747 - timing_s/adv:0.12053791247308254 - timing_s/update_actor:13.33815480209887 - timing_s/step:47.980936938896775 - timing_s/stop_profile:0.00011267699301242828 - timing_per_token_ms/update_actor:0.05912695791873959 - timing_per_token_ms/gen:0.21560704131781958 - timing_per_token_ms/adv:0.0005343347849949356 - perf/total_num_tokens:225585 - perf/time_per_step:47.980936938896775 - perf/throughput:587.6943386059763 (TaskRunner pid=2347384) Training Progress: 86%|████████▌ | 86/100 [1:31:22<11:14, 48.15s/it] (TaskRunner pid=2347384) step:87 - global_seqlen/min:21340 - global_seqlen/max:37517 - global_seqlen/minmax_diff:16177 - global_seqlen/balanced_min:28688 - global_seqlen/balanced_max:29862 - global_seqlen/mean:28898.5 - actor/entropy:0.7026273012161255 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2268109917640686 - training/rollout_probs_diff_mean:0.004347937181591988 - training/rollout_probs_diff_std:0.008354222401976585 - training/rollout_actor_probs_pearson_corr:0.9996702075004578 - rollout_corr/rollout_is_mean:0.9998772740364075 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.528608537337277e-06 - rollout_corr/rollout_is_max:1.4650890827178955 - rollout_corr/rollout_is_min:0.3679155707359314 - rollout_corr/rollout_is_std:0.034628402441740036 - rollout_corr/rollout_is_eff_sample_size:0.9988020720902324 - rollout_corr/rollout_is_seq_mean:0.9999104738235474 - rollout_corr/rollout_is_seq_std:0.0016025210497900844 - rollout_corr/rollout_is_seq_max:1.004460096359253 - rollout_corr/rollout_is_seq_min:0.9954553842544556 - rollout_corr/rollout_is_seq_max_deviation:0.004544615745544434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1807974809780717) - rollout_corr/training_log_ppl:np.float64(0.7608980417135172) - rollout_corr/kl:np.float64(0.0007492550087313532) - rollout_corr/k3_kl:np.float64(0.0006785160367712706) - rollout_corr/rollout_ppl:np.float64(2.1790433721616864) - rollout_corr/rollout_log_ppl:np.float64(0.7601487888605334) - rollout_corr/log_ppl_diff:np.float64(0.0007492528529837728) - rollout_corr/log_ppl_abs_diff:np.float64(0.00137826066929847) - rollout_corr/log_ppl_diff_max:np.float64(0.005486965179443359) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035854578018188477) - rollout_corr/ppl_ratio:np.float64(1.0007508331909776) - rollout_corr/chi2_token:np.float64(0.001214949181303382) - rollout_corr/chi2_seq:np.float64(0.8461277757305652) - actor/pg_loss:np.float64(-6.593985017389059e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016964398423624516) - actor/ppo_kl:np.float64(7.072242368622028e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24131453037261963) - perf/mfu/actor:np.float64(0.11180273417491973) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.57994842529297) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:6 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01241006888449192 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01241006888449192 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:598.328125 - response_length/max:3403.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:598.328125 - response_length_non_aborted/max:3403.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.75 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012595579028129578 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4072420690208673) - timing_s/agent_loop/generate_sequences/max:np.float64(23.30582166276872) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.752518724722904) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.30582166276872) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:3403 - timing_s/gen:30.597409987822175 - timing_s/reward:0.07339327968657017 - timing_s/old_log_prob:3.302472412586212 - timing_s/adv:0.13628184236586094 - timing_s/update_actor:13.274278812110424 - timing_s/step:47.47132308781147 - timing_s/stop_profile:0.00011784769594669342 - timing_per_token_ms/update_actor:0.05741768090087039 - timing_per_token_ms/gen:0.19975850669719122 - timing_per_token_ms/adv:0.0005894849315961942 - perf/total_num_tokens:231188 - perf/time_per_step:47.47132308781147 - perf/throughput:608.7569951767334 (TaskRunner pid=2347384) Training Progress: 87%|████████▋ | 87/100 [1:32:10<10:23, 47.96s/it] (TaskRunner pid=2347384) step:88 - global_seqlen/min:22129 - global_seqlen/max:32159 - global_seqlen/minmax_diff:10030 - global_seqlen/balanced_min:27987 - global_seqlen/balanced_max:28019 - global_seqlen/mean:28012.625 - actor/entropy:0.710129976272583 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22817057371139526 - training/rollout_probs_diff_mean:0.004395557567477226 - training/rollout_probs_diff_std:0.008365591987967491 - training/rollout_actor_probs_pearson_corr:0.9996364712715149 - rollout_corr/rollout_is_mean:0.9999300241470337 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.644992709159851 - rollout_corr/rollout_is_min:0.5691877007484436 - rollout_corr/rollout_is_std:0.034205835312604904 - rollout_corr/rollout_is_eff_sample_size:0.9988312092840276 - rollout_corr/rollout_is_seq_mean:0.9998932480812073 - rollout_corr/rollout_is_seq_std:0.0017496892251074314 - rollout_corr/rollout_is_seq_max:1.0070726871490479 - rollout_corr/rollout_is_seq_min:0.9952826499938965 - rollout_corr/rollout_is_seq_max_deviation:0.0070726871490478516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1347923753783107) - rollout_corr/training_log_ppl:np.float64(0.7425751016126014) - rollout_corr/kl:np.float64(0.0008565335552534556) - rollout_corr/k3_kl:np.float64(0.0006832548394868354) - rollout_corr/rollout_ppl:np.float64(2.1328414655290544) - rollout_corr/rollout_log_ppl:np.float64(0.741718566743657) - rollout_corr/log_ppl_diff:np.float64(0.0008565348689444363) - rollout_corr/log_ppl_abs_diff:np.float64(0.001574763038661331) - rollout_corr/log_ppl_diff_max:np.float64(0.007050633430480957) - rollout_corr/log_ppl_diff_min:np.float64(-0.0063100457191467285) - rollout_corr/ppl_ratio:np.float64(1.0008587399497628) - rollout_corr/chi2_token:np.float64(0.0010143548715859652) - rollout_corr/chi2_seq:np.float64(2.729879302671179) - actor/pg_loss:np.float64(6.0663907788693905e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029990614757480216) - actor/ppo_kl:np.float64(0.00012599919414668648) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2936526760458946) - perf/mfu/actor:np.float64(0.10966507870859521) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.56151580810547) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:6 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0022913627326488495 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0022913627326488495 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:554.05078125 - response_length/max:1673.0 - response_length/min:208.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:554.05078125 - response_length_non_aborted/max:1673.0 - response_length_non_aborted/min:208.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:321.34375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012337416410446167 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6710923090577126) - timing_s/agent_loop/generate_sequences/max:np.float64(13.50397158227861) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.358491666447662) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.50397158227861) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:1673 - timing_s/gen:20.717307522892952 - timing_s/reward:0.06772028654813766 - timing_s/old_log_prob:3.1851572655141354 - timing_s/adv:0.12023868039250374 - timing_s/update_actor:13.030956732109189 - timing_s/step:37.21506952494383 - timing_s/stop_profile:0.00012156739830970764 - timing_per_token_ms/update_actor:0.05814769560202404 - timing_per_token_ms/gen:0.14606419709168236 - timing_per_token_ms/adv:0.0005365379020731891 - perf/total_num_tokens:224101 - perf/time_per_step:37.21506952494383 - perf/throughput:752.7226297729799 (TaskRunner pid=2347384) Training Progress: 88%|████████▊ | 88/100 [1:32:47<08:56, 44.75s/it] (TaskRunner pid=2347384) step:89 - global_seqlen/min:24263 - global_seqlen/max:33571 - global_seqlen/minmax_diff:9308 - global_seqlen/balanced_min:29234 - global_seqlen/balanced_max:29274 - global_seqlen/mean:29260.5 - actor/entropy:0.7037457823753357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21477144956588745 - training/rollout_probs_diff_mean:0.0044005149975419044 - training/rollout_probs_diff_std:0.008454520255327225 - training/rollout_actor_probs_pearson_corr:0.9996297359466553 - rollout_corr/rollout_is_mean:1.0000073909759521 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.918117378518218e-06 - rollout_corr/rollout_is_max:1.4661989212036133 - rollout_corr/rollout_is_min:0.35283762216567993 - rollout_corr/rollout_is_std:0.03454740718007088 - rollout_corr/rollout_is_eff_sample_size:0.9988077804677696 - rollout_corr/rollout_is_seq_mean:0.9999973773956299 - rollout_corr/rollout_is_seq_std:0.0017847850685939193 - rollout_corr/rollout_is_seq_max:1.0056325197219849 - rollout_corr/rollout_is_seq_min:0.9943992495536804 - rollout_corr/rollout_is_seq_max_deviation:0.005632519721984863 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.126528596971184) - rollout_corr/training_log_ppl:np.float64(0.7374557201983407) - rollout_corr/kl:np.float64(0.0007500734154106681) - rollout_corr/k3_kl:np.float64(0.0006861104094468828) - rollout_corr/rollout_ppl:np.float64(2.1248417971655726) - rollout_corr/rollout_log_ppl:np.float64(0.7367056490620598) - rollout_corr/log_ppl_diff:np.float64(0.0007500711362808943) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015250509604811668) - rollout_corr/log_ppl_diff_max:np.float64(0.006453990936279297) - rollout_corr/log_ppl_diff_min:np.float64(-0.005076348781585693) - rollout_corr/ppl_ratio:np.float64(1.0007519533392042) - rollout_corr/chi2_token:np.float64(0.0012383947614580393) - rollout_corr/chi2_seq:np.float64(0.5420775599777699) - actor/pg_loss:np.float64(-6.23153755441308e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000183974101446438) - actor/ppo_kl:np.float64(0.00010195988847883086) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2833341397345066) - perf/mfu/actor:np.float64(0.11457874682299031) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.51513290405273) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:6 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004804632160812616 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004804632160812616 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:564.640625 - response_length/max:1703.0 - response_length/min:206.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:564.640625 - response_length_non_aborted/max:1703.0 - response_length_non_aborted/min:206.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012629106640815735 - timing_s/agent_loop/generate_sequences/min:np.float64(2.444547191262245) - timing_s/agent_loop/generate_sequences/max:np.float64(14.016501100733876) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.474856704888225) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.016501100733876) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1703 - timing_s/gen:20.999775690957904 - timing_s/reward:0.07121119648218155 - timing_s/old_log_prob:3.164371369406581 - timing_s/adv:0.12098673731088638 - timing_s/update_actor:13.180567843839526 - timing_s/step:37.624169800430536 - timing_s/stop_profile:0.00012341514229774475 - timing_per_token_ms/update_actor:0.056307000238544826 - timing_per_token_ms/gen:0.14527890867364407 - timing_per_token_ms/adv:0.0005168518023909638 - perf/total_num_tokens:234084 - perf/time_per_step:37.624169800430536 - perf/throughput:777.7048677806353 (TaskRunner pid=2347384) Training Progress: 89%|████████▉ | 89/100 [1:33:25<07:48, 42.63s/it] (TaskRunner pid=2347384) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2347384) validation generation end (TaskRunner pid=2347384) [prompt] system (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) To determine the folding stability score of a protein sequence, we need to evaluate the balance of hydrophobic and hydrophilic interactions, hydrogen bonding potential, and the overall structural tendencies of the amino acid residues. The folding stability score typically reflects how likely the protein is to fold into a stable, functional structure based on these factors. (TaskRunner pid=2347384) (TaskRunner pid=2347384) The given protein sequence is: (TaskRunner pid=2347384) **GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR** (TaskRunner pid=2347384) (TaskRunner pid=2347384) First, we identify key features: (TaskRunner pid=2347384) - **Hydrophobic residues**: These tend to cluster in the core of the folded protein, contributing to stability. Examples include **S, T, R, F, Y, L, V, W, G, A, K, E**. (TaskRunner pid=2347384) - **Hydrophilic residues**: These often form the surface and are involved in interactions with water or other molecules. Examples include **E, D, Q, N, R, Y, H, V, T, G, W**. (TaskRunner pid=2347384) (TaskRunner pid=2347384) The sequence contains a mix of both hydrophobic and hydrophilic residues. Notably, there are several charged residues (E, D, R, K, Q, N) that can form hydrogen bonds, which are essential for stabilizing the folded structure. Additionally, the presence of **hydrophobic clusters** (e.g., F, L, V, W) suggests potential for core formation, which is a key factor in stability. (TaskRunner pid=2347384) (TaskRunner pid=2347384) However, the stability score also depends on the **balance** between hydrophobic and hydrophilic interactions and the **distribution** of charged residues. If the sequence has a high number of charged residues that could lead to repulsive forces or disrupt hydrophobic interactions, the stability score may decrease. In this sequence, the number of charged residues is relatively high, but the presence of **hydrophobic clusters** and **hydrogen-bonding potential** suggests that the sequence is likely to fold into a stable structure. (TaskRunner pid=2347384) (TaskRunner pid=2347384) Based on the balance of these factors, the folding stability score is expected to be **positive**, indicating a stable fold. Among the given options, **B: 1.69** and **C: 2.49** are both positive values, but **1.69** is more representative of a stable fold without extreme hydrophobic clustering or overcharge, which is consistent with the given sequence. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) B (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 1.0 (TaskRunner pid=2347384) [acc] 1.0 (TaskRunner pid=2347384) [pred] B (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2347384) step:90 - global_seqlen/min:21849 - global_seqlen/max:34497 - global_seqlen/minmax_diff:12648 - global_seqlen/balanced_min:28097 - global_seqlen/balanced_max:28100 - global_seqlen/mean:28098.0 - actor/entropy:0.7137050032615662 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22357679903507233 - training/rollout_probs_diff_mean:0.004411489702761173 - training/rollout_probs_diff_std:0.008546157740056515 - training/rollout_actor_probs_pearson_corr:0.9996163845062256 - rollout_corr/rollout_is_mean:1.000036358833313 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6476123332977295 - rollout_corr/rollout_is_min:0.5123901963233948 - rollout_corr/rollout_is_std:0.034772686660289764 - rollout_corr/rollout_is_eff_sample_size:0.998792439351094 - rollout_corr/rollout_is_seq_mean:1.0000545978546143 - rollout_corr/rollout_is_seq_std:0.001701094675809145 - rollout_corr/rollout_is_seq_max:1.00467050075531 - rollout_corr/rollout_is_seq_min:0.9945640563964844 - rollout_corr/rollout_is_seq_max_deviation:0.005435943603515625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.121528161689639) - rollout_corr/training_log_ppl:np.float64(0.7370664237532765) - rollout_corr/kl:np.float64(0.0006172493178056015) - rollout_corr/k3_kl:np.float64(0.0006593498709435153) - rollout_corr/rollout_ppl:np.float64(2.12016929872334) - rollout_corr/rollout_log_ppl:np.float64(0.7364491681801155) - rollout_corr/log_ppl_diff:np.float64(0.0006172555731609464) - rollout_corr/log_ppl_abs_diff:np.float64(0.001292749191634357) - rollout_corr/log_ppl_diff_max:np.float64(0.006107687950134277) - rollout_corr/log_ppl_diff_min:np.float64(-0.00531315803527832) - rollout_corr/ppl_ratio:np.float64(1.0006187898106873) - rollout_corr/chi2_token:np.float64(0.0014007261488586664) - rollout_corr/chi2_seq:np.float64(0.4249414587393403) - actor/pg_loss:np.float64(3.0110590159893036e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002521567566873273) - actor/ppo_kl:np.float64(3.702425574481083e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3106554299592972) - perf/mfu/actor:np.float64(0.11035315526207326) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.51908874511719) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.50375) - val-aux/sciknoweval/reward/std@16:np.float64(0.2812551333413633) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.62002) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.21320744865826927) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.38696) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.260062459746992) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5031) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2813695693880417) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6996199999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13875989230757715) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.28044) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.21434077725964976) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.53596) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22171055727144304) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.75186) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08165739178554601) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.19372) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.15500562921804206) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5532) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15283869458101149) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7835399999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.042499662452586476) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.13176) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.09534463379805076) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5650600000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09728077850315583) - val-aux/sciknoweval/score/mean@16:np.float64(0.50375) - val-aux/sciknoweval/score/std@16:np.float64(0.2812551333413633) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.62002) - val-aux/sciknoweval/score/best@2/std:np.float64(0.21320744865826927) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.38696) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.260062459746992) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5031) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2813695693880417) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6996199999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13875989230757715) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.28044) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.21434077725964976) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.53596) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.22171055727144304) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.75186) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08165739178554601) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.19372) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.15500562921804206) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5532) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15283869458101149) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7835399999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.042499662452586476) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.13176) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.09534463379805076) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5650600000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09728077850315583) - val-core/sciknoweval/acc/mean@16:np.float64(0.50375) - val-aux/sciknoweval/acc/std@16:np.float64(0.2812551333413633) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.62002) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.21320744865826927) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.38696) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.260062459746992) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5031) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2813695693880417) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6996199999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13875989230757715) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.28044) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.21434077725964976) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.53596) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.22171055727144304) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.75186) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08165739178554601) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.19372) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.15500562921804206) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5532) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15283869458101149) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7835399999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.042499662452586476) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.13176) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.09534463379805076) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5650600000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09728077850315583) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.975) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.06256165804280608) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99546) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.025370056864576846) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.95502) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.07796051048390831) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.97508) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.06184326284266497) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998200000000002) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0031366135362477787) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9241400000000002) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08943092701860544) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99128) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03616298702899823) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8820399999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08830310262373967) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99784) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.016452495442419442) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8408599999999998) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06833170165452436) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998200000000002) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.003939230498651246) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:6 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0001023623117362149 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0001023623117362149 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:562.875 - response_length/max:1204.0 - response_length/min:218.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:562.875 - response_length_non_aborted/max:1204.0 - response_length_non_aborted/min:218.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.1875 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013893656432628632 - timing_s/agent_loop/generate_sequences/min:np.float64(2.7718864362686872) - timing_s/agent_loop/generate_sequences/max:np.float64(11.125447349622846) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.426409698018688) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.125447349622846) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1204 - timing_s/gen:17.74707208201289 - timing_s/reward:0.06988384947180748 - timing_s/old_log_prob:3.156799355521798 - timing_s/adv:0.12230040319263935 - timing_s/update_actor:13.047853155061603 - timing_s/step:34.23183760419488 - timing_s/testing:101.45159381069243 - timing_s/save_checkpoint:12.717426024377346 - timing_s/stop_profile:0.00021499581634998322 - timing_per_token_ms/update_actor:0.05804618280243079 - timing_per_token_ms/gen:0.12316144849276102 - timing_per_token_ms/adv:0.0005440796640002818 - perf/total_num_tokens:224784 - perf/time_per_step:34.23183760419488 - perf/throughput:820.8148310611517 (TaskRunner pid=2347384) Training Progress: 90%|█████████ | 90/100 [1:35:53<12:23, 74.38s/it] (TaskRunner pid=2347384) step:91 - global_seqlen/min:20973 - global_seqlen/max:34102 - global_seqlen/minmax_diff:13129 - global_seqlen/balanced_min:27404 - global_seqlen/balanced_max:27437 - global_seqlen/mean:27425.625 - actor/entropy:0.6958094239234924 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21857798099517822 - training/rollout_probs_diff_mean:0.004454716574400663 - training/rollout_probs_diff_std:0.008537987247109413 - training/rollout_actor_probs_pearson_corr:0.9996330142021179 - rollout_corr/rollout_is_mean:1.000101089477539 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8720309734344482 - rollout_corr/rollout_is_min:0.6129111051559448 - rollout_corr/rollout_is_std:0.03470920771360397 - rollout_corr/rollout_is_eff_sample_size:0.9987970773134115 - rollout_corr/rollout_is_seq_mean:1.0001044273376465 - rollout_corr/rollout_is_seq_std:0.0017643962055444717 - rollout_corr/rollout_is_seq_max:1.005553960800171 - rollout_corr/rollout_is_seq_min:0.9953451752662659 - rollout_corr/rollout_is_seq_max_deviation:0.0055539608001708984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0903072808869183) - rollout_corr/training_log_ppl:np.float64(0.7229948686435819) - rollout_corr/kl:np.float64(0.0006692291897181235) - rollout_corr/k3_kl:np.float64(0.0006798961969138873) - rollout_corr/rollout_ppl:np.float64(2.088830603286624) - rollout_corr/rollout_log_ppl:np.float64(0.7223256411962211) - rollout_corr/log_ppl_diff:np.float64(0.000669227447360754) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015533484984189272) - rollout_corr/log_ppl_diff_max:np.float64(0.007445096969604492) - rollout_corr/log_ppl_diff_min:np.float64(-0.0048781633377075195) - rollout_corr/ppl_ratio:np.float64(1.0006712889298797) - rollout_corr/chi2_token:np.float64(0.0013731613289564848) - rollout_corr/chi2_seq:np.float64(1.5056298803538084) - actor/pg_loss:np.float64(-0.00013465248048305511) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018048949277726933) - actor/ppo_kl:np.float64(0.0001323158714434669) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2975430339574814) - perf/mfu/actor:np.float64(0.1073125876728456) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(107.8349838256836) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:6 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0063367560505867004 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0063367560505867004 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:526.51953125 - response_length/max:1596.0 - response_length/min:207.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:526.51953125 - response_length_non_aborted/max:1596.0 - response_length_non_aborted/min:207.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.178075611591339e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6735423505306244) - timing_s/agent_loop/generate_sequences/max:np.float64(12.761884409934282) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.794378179460182) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.761884409934282) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1596 - timing_s/gen:19.508875962346792 - timing_s/reward:0.06818237528204918 - timing_s/old_log_prob:3.443881580606103 - timing_s/adv:0.12259230576455593 - timing_s/update_actor:13.179625660181046 - timing_s/step:36.414061430841684 - timing_s/stop_profile:0.00011343881487846375 - timing_per_token_ms/update_actor:0.06006985100695538 - timing_per_token_ms/gen:0.14473640996184253 - timing_per_token_ms/adv:0.0005587489153144 - perf/total_num_tokens:219405 - perf/time_per_step:36.414061430841684 - perf/throughput:753.1602881509742 (TaskRunner pid=2347384) Training Progress: 91%|█████████ | 91/100 [1:36:30<09:27, 63.01s/it] (TaskRunner pid=2347384) step:92 - global_seqlen/min:24882 - global_seqlen/max:37140 - global_seqlen/minmax_diff:12258 - global_seqlen/balanced_min:29909 - global_seqlen/balanced_max:29917 - global_seqlen/mean:29913.875 - actor/entropy:0.6660850048065186 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2649591565132141 - training/rollout_probs_diff_mean:0.004180658608675003 - training/rollout_probs_diff_std:0.008352335542440414 - training/rollout_actor_probs_pearson_corr:0.9996336102485657 - rollout_corr/rollout_is_mean:0.9999892115592957 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.839243724243715e-06 - rollout_corr/rollout_is_max:1.6460182666778564 - rollout_corr/rollout_is_min:0.4513525664806366 - rollout_corr/rollout_is_std:0.03382206708192825 - rollout_corr/rollout_is_eff_sample_size:0.9988574936804934 - rollout_corr/rollout_is_seq_mean:1.0000672340393066 - rollout_corr/rollout_is_seq_std:0.0017553241923451424 - rollout_corr/rollout_is_seq_max:1.0054320096969604 - rollout_corr/rollout_is_seq_min:0.9943798780441284 - rollout_corr/rollout_is_seq_max_deviation:0.005620121955871582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.046488129068166) - rollout_corr/training_log_ppl:np.float64(0.6976048549986444) - rollout_corr/kl:np.float64(0.0004347642772373206) - rollout_corr/k3_kl:np.float64(0.0006588202651869324) - rollout_corr/rollout_ppl:np.float64(2.0456050471402705) - rollout_corr/rollout_log_ppl:np.float64(0.6971700830035843) - rollout_corr/log_ppl_diff:np.float64(0.00043477199506014585) - rollout_corr/log_ppl_abs_diff:np.float64(0.001337943016551435) - rollout_corr/log_ppl_diff_max:np.float64(0.006161212921142578) - rollout_corr/log_ppl_diff_min:np.float64(-0.005505204200744629) - rollout_corr/ppl_ratio:np.float64(1.0004362612962723) - rollout_corr/chi2_token:np.float64(0.0017820540815591812) - rollout_corr/chi2_seq:np.float64(1.6084863217547536) - actor/pg_loss:np.float64(0.00012694147881120443) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016383558727284253) - actor/ppo_kl:np.float64(-0.00011150972658313663) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26658932492136955) - perf/mfu/actor:np.float64(0.11617136707963896) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.64786911010742) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:6 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001590979052707553 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001590979052707553 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:571.15234375 - response_length/max:1261.0 - response_length/min:190.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.15234375 - response_length_non_aborted/max:1261.0 - response_length_non_aborted/min:190.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:363.65625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.707175195217133e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4817494228482246) - timing_s/agent_loop/generate_sequences/max:np.float64(11.177728792652488) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.464044597872999) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.177728792652488) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1261 - timing_s/gen:17.71172520145774 - timing_s/reward:0.07110476866364479 - timing_s/old_log_prob:3.1798686385154724 - timing_s/adv:0.11830317229032516 - timing_s/update_actor:13.35173504985869 - timing_s/step:34.52292972803116 - timing_s/stop_profile:0.00011513568460941315 - timing_per_token_ms/update_actor:0.055792400056239325 - timing_per_token_ms/gen:0.12113480286877365 - timing_per_token_ms/adv:0.000494349078355467 - perf/total_num_tokens:239311 - perf/time_per_step:34.52292972803116 - perf/throughput:866.4929435496663 (TaskRunner pid=2347384) Training Progress: 92%|█████████▏| 92/100 [1:37:04<07:15, 54.47s/it] (TaskRunner pid=2347384) step:93 - global_seqlen/min:25331 - global_seqlen/max:35661 - global_seqlen/minmax_diff:10330 - global_seqlen/balanced_min:28672 - global_seqlen/balanced_max:28698 - global_seqlen/mean:28690.0 - actor/entropy:0.6885712146759033 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.13872385025024414 - training/rollout_probs_diff_mean:0.004355665761977434 - training/rollout_probs_diff_std:0.008415261283516884 - training/rollout_actor_probs_pearson_corr:0.9996607303619385 - rollout_corr/rollout_is_mean:0.9999585151672363 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5950614213943481 - rollout_corr/rollout_is_min:0.6058589220046997 - rollout_corr/rollout_is_std:0.03460601717233658 - rollout_corr/rollout_is_eff_sample_size:0.9988036181027059 - rollout_corr/rollout_is_seq_mean:0.9999375939369202 - rollout_corr/rollout_is_seq_std:0.0015914164250716567 - rollout_corr/rollout_is_seq_max:1.0040444135665894 - rollout_corr/rollout_is_seq_min:0.9935377836227417 - rollout_corr/rollout_is_seq_max_deviation:0.006462216377258301 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.090018341317773) - rollout_corr/training_log_ppl:np.float64(0.7194376521511003) - rollout_corr/kl:np.float64(0.00079889160743285) - rollout_corr/k3_kl:np.float64(0.0006610654560859075) - rollout_corr/rollout_ppl:np.float64(2.088347557466477) - rollout_corr/rollout_log_ppl:np.float64(0.7186387614347041) - rollout_corr/log_ppl_diff:np.float64(0.0007988907163962722) - rollout_corr/log_ppl_abs_diff:np.float64(0.001344712800346315) - rollout_corr/log_ppl_diff_max:np.float64(0.005600929260253906) - rollout_corr/log_ppl_diff_min:np.float64(-0.004459023475646973) - rollout_corr/ppl_ratio:np.float64(1.0008004421833903) - rollout_corr/chi2_token:np.float64(0.0010499106720089912) - rollout_corr/chi2_seq:np.float64(0.41712698247283697) - actor/pg_loss:np.float64(-2.4716602638363838e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015107757553778356) - actor/ppo_kl:np.float64(9.72518970875913e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23958631977438927) - perf/mfu/actor:np.float64(0.11208078605854563) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.56364822387695) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:6 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002769256941974163 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002769256941974163 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:550.125 - response_length/max:1703.0 - response_length/min:234.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:550.125 - response_length_non_aborted/max:1703.0 - response_length_non_aborted/min:234.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001787487417459488 - timing_s/agent_loop/generate_sequences/min:np.float64(3.0153187550604343) - timing_s/agent_loop/generate_sequences/max:np.float64(13.88548337854445) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.346155295141216) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.88548337854445) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:289 - timing_s/agent_loop/slowest/response_length:1703 - timing_s/gen:20.733097605407238 - timing_s/reward:0.06744629144668579 - timing_s/old_log_prob:3.3500890657305717 - timing_s/adv:0.12276385724544525 - timing_s/update_actor:13.170130914077163 - timing_s/step:37.53802668303251 - timing_s/stop_profile:0.00011480040848255157 - timing_per_token_ms/update_actor:0.05738119080723755 - timing_per_token_ms/gen:0.14721865488956515 - timing_per_token_ms/adv:0.0005348721560014171 - perf/total_num_tokens:229520 - perf/time_per_step:37.53802668303251 - perf/throughput:764.2916406409852 (TaskRunner pid=2347384) Training Progress: 93%|█████████▎| 93/100 [1:37:42<05:45, 49.41s/it] (TaskRunner pid=2347384) step:94 - global_seqlen/min:22044 - global_seqlen/max:33717 - global_seqlen/minmax_diff:11673 - global_seqlen/balanced_min:25721 - global_seqlen/balanced_max:25743 - global_seqlen/mean:25736.25 - actor/entropy:0.7494757771492004 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17908385396003723 - training/rollout_probs_diff_mean:0.004814704414457083 - training/rollout_probs_diff_std:0.008782243356108665 - training/rollout_actor_probs_pearson_corr:0.9995895624160767 - rollout_corr/rollout_is_mean:1.0000247955322266 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.683864951133728 - rollout_corr/rollout_is_min:0.6347923874855042 - rollout_corr/rollout_is_std:0.03598569706082344 - rollout_corr/rollout_is_eff_sample_size:0.9987068231686047 - rollout_corr/rollout_is_seq_mean:1.0000247955322266 - rollout_corr/rollout_is_seq_std:0.001746542751789093 - rollout_corr/rollout_is_seq_max:1.0052481889724731 - rollout_corr/rollout_is_seq_min:0.9954923987388611 - rollout_corr/rollout_is_seq_max_deviation:0.0052481889724731445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.1610376546159387) - rollout_corr/training_log_ppl:np.float64(0.7593257842818275) - rollout_corr/kl:np.float64(0.0007937323189421974) - rollout_corr/k3_kl:np.float64(0.0007018388494088867) - rollout_corr/rollout_ppl:np.float64(2.159263103734702) - rollout_corr/rollout_log_ppl:np.float64(0.7585320442449301) - rollout_corr/log_ppl_diff:np.float64(0.0007937400368973613) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015924718463793397) - rollout_corr/log_ppl_diff_max:np.float64(0.00515371561050415) - rollout_corr/log_ppl_diff_min:np.float64(-0.003914773464202881) - rollout_corr/ppl_ratio:np.float64(1.000795733416453) - rollout_corr/chi2_token:np.float64(0.0012172725982964039) - rollout_corr/chi2_seq:np.float64(1.9561607241630554) - actor/pg_loss:np.float64(0.00015463947784155607) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002284840575157432) - actor/ppo_kl:np.float64(0.0001500120649460257) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3215147852897644) - perf/mfu/actor:np.float64(0.10280180963219644) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.62215423583984) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:6 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002445860533043742 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002445860533043742 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:519.8515625 - response_length/max:1290.0 - response_length/min:227.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:519.8515625 - response_length_non_aborted/max:1290.0 - response_length_non_aborted/min:227.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:284.40625 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.894371032714844e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8935541324317455) - timing_s/agent_loop/generate_sequences/max:np.float64(11.20853840559721) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.054502220184077) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.20853840559721) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1267 - timing_s/gen:18.01648428104818 - timing_s/reward:0.06695256195962429 - timing_s/old_log_prob:3.1328779999166727 - timing_s/adv:0.12063981033861637 - timing_s/update_actor:12.919747779145837 - timing_s/step:34.34514455497265 - timing_s/stop_profile:0.00011434033513069153 - timing_per_token_ms/update_actor:0.0627507299001692 - timing_per_token_ms/gen:0.13537882118579656 - timing_per_token_ms/adv:0.0005859430294750419 - perf/total_num_tokens:205890 - perf/time_per_step:34.34514455497265 - perf/throughput:749.3417288958182 (TaskRunner pid=2347384) Training Progress: 94%|█████████▍| 94/100 [1:38:16<04:29, 44.91s/it] (TaskRunner pid=2347384) step:95 - global_seqlen/min:22132 - global_seqlen/max:35004 - global_seqlen/minmax_diff:12872 - global_seqlen/balanced_min:25749 - global_seqlen/balanced_max:26008 - global_seqlen/mean:25786.875 - actor/entropy:0.7113838791847229 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2521419823169708 - training/rollout_probs_diff_mean:0.004754967521876097 - training/rollout_probs_diff_std:0.00897925440222025 - training/rollout_actor_probs_pearson_corr:0.9995635747909546 - rollout_corr/rollout_is_mean:0.9998835921287537 - rollout_corr/rollout_is_ratio_fraction_high:7.594686849188292e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.594686849188292e-06 - rollout_corr/rollout_is_max:2.095022201538086 - rollout_corr/rollout_is_min:0.4158303737640381 - rollout_corr/rollout_is_std:0.03595089912414551 - rollout_corr/rollout_is_eff_sample_size:0.9987089633942786 - rollout_corr/rollout_is_seq_mean:0.9998339414596558 - rollout_corr/rollout_is_seq_std:0.0019074503798037767 - rollout_corr/rollout_is_seq_max:1.0060638189315796 - rollout_corr/rollout_is_seq_min:0.9946165084838867 - rollout_corr/rollout_is_seq_max_deviation:0.00606381893157959 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.098421197384596) - rollout_corr/training_log_ppl:np.float64(0.7261631052242592) - rollout_corr/kl:np.float64(0.0008981620562418247) - rollout_corr/k3_kl:np.float64(0.0007242549703505574) - rollout_corr/rollout_ppl:np.float64(2.09659134875983) - rollout_corr/rollout_log_ppl:np.float64(0.7252649461152032) - rollout_corr/log_ppl_diff:np.float64(0.0008981591090559959) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015511228702962399) - rollout_corr/log_ppl_diff_max:np.float64(0.00717771053314209) - rollout_corr/log_ppl_diff_min:np.float64(-0.00464785099029541) - rollout_corr/ppl_ratio:np.float64(1.0009002273436636) - rollout_corr/chi2_token:np.float64(0.0011242812033742666) - rollout_corr/chi2_seq:np.float64(0.9126021836418658) - actor/pg_loss:np.float64(5.6369928643107414e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019206729507459386) - actor/ppo_kl:np.float64(4.751628545118791e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28937822207808495) - perf/mfu/actor:np.float64(0.10144171053381716) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.62709426879883) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:6 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0062304912135005 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0062304912135005 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:514.33984375 - response_length/max:1917.0 - response_length/min:227.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:514.33984375 - response_length_non_aborted/max:1917.0 - response_length_non_aborted/min:227.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.5 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.57524573802948e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.8952717129141092) - timing_s/agent_loop/generate_sequences/max:np.float64(14.242058008909225) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.883536388988432) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.242058008909225) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1917 - timing_s/gen:20.822622876614332 - timing_s/reward:0.06796691566705704 - timing_s/old_log_prob:3.166369017213583 - timing_s/adv:0.12020312435925007 - timing_s/update_actor:13.03413606621325 - timing_s/step:37.30040380731225 - timing_s/stop_profile:0.00011594966053962708 - timing_per_token_ms/update_actor:0.06318202606080249 - timing_per_token_ms/gen:0.1581412982100412 - timing_per_token_ms/adv:0.0005826758979095473 - perf/total_num_tokens:206295 - perf/time_per_step:37.30040380731225 - perf/throughput:691.3296470786416 (TaskRunner pid=2347384) Training Progress: 95%|█████████▌| 95/100 [1:38:54<03:33, 42.64s/it] (TaskRunner pid=2347384) step:96 - global_seqlen/min:17038 - global_seqlen/max:35857 - global_seqlen/minmax_diff:18819 - global_seqlen/balanced_min:26788 - global_seqlen/balanced_max:29249 - global_seqlen/mean:27103.875 - actor/entropy:0.6755845546722412 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2782573103904724 - training/rollout_probs_diff_mean:0.004448726773262024 - training/rollout_probs_diff_std:0.008604924194514751 - training/rollout_actor_probs_pearson_corr:0.9996158480644226 - rollout_corr/rollout_is_mean:0.9999901652336121 - rollout_corr/rollout_is_ratio_fraction_high:7.160092536651064e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.160092536651064e-06 - rollout_corr/rollout_is_max:2.921051025390625 - rollout_corr/rollout_is_min:0.4102022647857666 - rollout_corr/rollout_is_std:0.034630123525857925 - rollout_corr/rollout_is_eff_sample_size:0.998802191014099 - rollout_corr/rollout_is_seq_mean:0.9999725222587585 - rollout_corr/rollout_is_seq_std:0.0018183635547757149 - rollout_corr/rollout_is_seq_max:1.0058923959732056 - rollout_corr/rollout_is_seq_min:0.9942228198051453 - rollout_corr/rollout_is_seq_max_deviation:0.005892395973205566 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.069570147898048) - rollout_corr/training_log_ppl:np.float64(0.7117336596129462) - rollout_corr/kl:np.float64(0.000528029880543443) - rollout_corr/k3_kl:np.float64(0.0006776321753250159) - rollout_corr/rollout_ppl:np.float64(2.068463566713035) - rollout_corr/rollout_log_ppl:np.float64(0.7112056275946088) - rollout_corr/log_ppl_diff:np.float64(0.0005280320183373988) - rollout_corr/log_ppl_abs_diff:np.float64(0.001428587653208524) - rollout_corr/log_ppl_diff_max:np.float64(0.006919503211975098) - rollout_corr/log_ppl_diff_min:np.float64(-0.0063474178314208984) - rollout_corr/ppl_ratio:np.float64(1.000529950717464) - rollout_corr/chi2_token:np.float64(0.001661372371017933) - rollout_corr/chi2_seq:np.float64(1.0614761812612414) - actor/pg_loss:np.float64(-5.9843179769814014e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018992950958818255) - actor/ppo_kl:np.float64(-0.00015546518476128313) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29364553466439247) - perf/mfu/actor:np.float64(0.10426150507634732) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.73516464233398) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:6 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003478909842669964 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003478909842669964 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:545.55859375 - response_length/max:4293.0 - response_length/min:185.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:545.55859375 - response_length_non_aborted/max:4293.0 - response_length_non_aborted/min:185.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:301.4375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001343023031949997 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4207872468978167) - timing_s/agent_loop/generate_sequences/max:np.float64(25.339606577530503) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.618198693111481) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.339606577530503) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:4293 - timing_s/gen:32.00834040902555 - timing_s/reward:0.06888510473072529 - timing_s/old_log_prob:3.170083498582244 - timing_s/adv:0.12208253517746925 - timing_s/update_actor:13.468160927295685 - timing_s/step:48.92416075617075 - timing_s/stop_profile:4.196353256702423e-05 - timing_per_token_ms/update_actor:0.062113631940523654 - timing_per_token_ms/gen:0.22918267836882747 - timing_per_token_ms/adv:0.000563030817445242 - perf/total_num_tokens:216831 - perf/time_per_step:48.92416075617075 - perf/throughput:553.997750417853 (TaskRunner pid=2347384) Training Progress: 96%|█████████▌| 96/100 [1:39:43<02:58, 44.53s/it] (TaskRunner pid=2347384) step:97 - global_seqlen/min:17718 - global_seqlen/max:33203 - global_seqlen/minmax_diff:15485 - global_seqlen/balanced_min:26500 - global_seqlen/balanced_max:26512 - global_seqlen/mean:26508.625 - actor/entropy:0.6954312324523926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016299605369568 - training/rollout_probs_diff_mean:0.004451677203178406 - training/rollout_probs_diff_std:0.008618551306426525 - training/rollout_actor_probs_pearson_corr:0.9996195435523987 - rollout_corr/rollout_is_mean:0.9999681711196899 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4766141248401254e-05 - rollout_corr/rollout_is_max:1.9401957988739014 - rollout_corr/rollout_is_min:0.3894535005092621 - rollout_corr/rollout_is_std:0.03504587337374687 - rollout_corr/rollout_is_eff_sample_size:0.998773293347197 - rollout_corr/rollout_is_seq_mean:0.9999765157699585 - rollout_corr/rollout_is_seq_std:0.0017547223251312971 - rollout_corr/rollout_is_seq_max:1.0046765804290771 - rollout_corr/rollout_is_seq_min:0.9950421452522278 - rollout_corr/rollout_is_seq_max_deviation:0.004957854747772217 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0819673286750913) - rollout_corr/training_log_ppl:np.float64(0.7189342251513153) - rollout_corr/kl:np.float64(0.0007908260792977018) - rollout_corr/k3_kl:np.float64(0.0007246574010650875) - rollout_corr/rollout_ppl:np.float64(2.0803378019481897) - rollout_corr/rollout_log_ppl:np.float64(0.7181433973601088) - rollout_corr/log_ppl_diff:np.float64(0.0007908277912065387) - rollout_corr/log_ppl_abs_diff:np.float64(0.001576655195094645) - rollout_corr/log_ppl_diff_max:np.float64(0.010724842548370361) - rollout_corr/log_ppl_diff_min:np.float64(-0.003208339214324951) - rollout_corr/ppl_ratio:np.float64(1.000792994396761) - rollout_corr/chi2_token:np.float64(0.00133550725877285) - rollout_corr/chi2_seq:np.float64(0.3194058546796441) - actor/pg_loss:np.float64(3.89214837923646e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016333569760718092) - actor/ppo_kl:np.float64(0.00011111655820128519) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19514655321836472) - perf/mfu/actor:np.float64(0.10557414618095853) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.6673698425293) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:6 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005504079163074493 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.005504079163074493 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:529.08203125 - response_length/max:1413.0 - response_length/min:192.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:529.08203125 - response_length_non_aborted/max:1413.0 - response_length_non_aborted/min:192.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.3125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.032813012599945e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4806480705738068) - timing_s/agent_loop/generate_sequences/max:np.float64(11.77598080970347) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.949599402942113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.77598080970347) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:1413 - timing_s/gen:18.839728977531195 - timing_s/reward:0.06856903247535229 - timing_s/old_log_prob:3.186891995370388 - timing_s/adv:0.12463601492345333 - timing_s/update_actor:13.099989926442504 - timing_s/step:35.40725245513022 - timing_s/stop_profile:0.00011800415813922882 - timing_per_token_ms/update_actor:0.06177230017797276 - timing_per_token_ms/gen:0.13909504948526114 - timing_per_token_ms/adv:0.0005877144463521464 - perf/total_num_tokens:212069 - perf/time_per_step:35.40725245513022 - perf/throughput:748.6778318536014 (TaskRunner pid=2347384) Training Progress: 97%|█████████▋| 97/100 [1:40:18<02:05, 41.81s/it] (TaskRunner pid=2347384) step:98 - global_seqlen/min:17094 - global_seqlen/max:34276 - global_seqlen/minmax_diff:17182 - global_seqlen/balanced_min:27897 - global_seqlen/balanced_max:32984 - global_seqlen/mean:28545.625 - actor/entropy:0.6646912693977356 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20037442445755005 - training/rollout_probs_diff_mean:0.004278605338186026 - training/rollout_probs_diff_std:0.00845154095441103 - training/rollout_actor_probs_pearson_corr:0.9996199011802673 - rollout_corr/rollout_is_mean:1.0000585317611694 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.661636471748352 - rollout_corr/rollout_is_min:0.6088563799858093 - rollout_corr/rollout_is_std:0.033866096287965775 - rollout_corr/rollout_is_eff_sample_size:0.9988545202630841 - rollout_corr/rollout_is_seq_mean:1.0000534057617188 - rollout_corr/rollout_is_seq_std:0.00176704628393054 - rollout_corr/rollout_is_seq_max:1.0056651830673218 - rollout_corr/rollout_is_seq_min:0.993948757648468 - rollout_corr/rollout_is_seq_max_deviation:0.006051242351531982 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.0891312742605805) - rollout_corr/training_log_ppl:np.float64(0.7216394667339046) - rollout_corr/kl:np.float64(0.0006897106773937622) - rollout_corr/k3_kl:np.float64(0.0007059377225857588) - rollout_corr/rollout_ppl:np.float64(2.0876432391814888) - rollout_corr/rollout_log_ppl:np.float64(0.7209497544099577) - rollout_corr/log_ppl_diff:np.float64(0.0006897123239468783) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014765246014576405) - rollout_corr/log_ppl_diff_max:np.float64(0.005883216857910156) - rollout_corr/log_ppl_diff_min:np.float64(-0.004796862602233887) - rollout_corr/ppl_ratio:np.float64(1.0006916315760463) - rollout_corr/chi2_token:np.float64(0.0014585081953555346) - rollout_corr/chi2_seq:np.float64(0.7151871314272285) - actor/pg_loss:np.float64(0.00012189231347292662) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010599532777177956) - actor/ppo_kl:np.float64(0.00010292452467197677) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2680858075618744) - perf/mfu/actor:np.float64(0.10484056707281225) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.66368865966797) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:6 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024818500969558954 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024818500969558954 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:594.55078125 - response_length/max:6876.0 - response_length/min:185.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:594.55078125 - response_length_non_aborted/max:6876.0 - response_length_non_aborted/min:185.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014608167111873627 - timing_s/agent_loop/generate_sequences/min:np.float64(2.333287037909031) - timing_s/agent_loop/generate_sequences/max:np.float64(43.170285960659385) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.6546497269082465) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(43.170285960659385) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:6876 - timing_s/gen:50.08524839952588 - timing_s/reward:0.0718032456934452 - timing_s/old_log_prob:3.381595876067877 - timing_s/adv:0.13589257560670376 - timing_s/update_actor:14.206524349749088 - timing_s/step:67.97185587324202 - timing_s/stop_profile:0.00011174194514751434 - timing_per_token_ms/update_actor:0.06220972719002075 - timing_per_token_ms/gen:0.3290644091818658 - timing_per_token_ms/adv:0.0005950674385597782 - perf/total_num_tokens:228365 - perf/time_per_step:67.97185587324202 - perf/throughput:419.96241875804577 (TaskRunner pid=2347384) Training Progress: 98%|█████████▊| 98/100 [1:41:26<01:39, 49.67s/it] (TaskRunner pid=2347384) step:99 - global_seqlen/min:21386 - global_seqlen/max:35082 - global_seqlen/minmax_diff:13696 - global_seqlen/balanced_min:28123 - global_seqlen/balanced_max:28141 - global_seqlen/mean:28135.875 - actor/entropy:0.7309073805809021 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016211688518524 - training/rollout_probs_diff_mean:0.004499722272157669 - training/rollout_probs_diff_std:0.008437901735305786 - training/rollout_actor_probs_pearson_corr:0.9996675252914429 - rollout_corr/rollout_is_mean:1.0000288486480713 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.609737515449524 - rollout_corr/rollout_is_min:0.5558425188064575 - rollout_corr/rollout_is_std:0.03454740718007088 - rollout_corr/rollout_is_eff_sample_size:0.9988080183182497 - rollout_corr/rollout_is_seq_mean:1.0000360012054443 - rollout_corr/rollout_is_seq_std:0.0017323825741186738 - rollout_corr/rollout_is_seq_max:1.0061935186386108 - rollout_corr/rollout_is_seq_min:0.9949625134468079 - rollout_corr/rollout_is_seq_max_deviation:0.00619351863861084 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.149438609369099) - rollout_corr/training_log_ppl:np.float64(0.7540047036018223) - rollout_corr/kl:np.float64(0.0007572453385922273) - rollout_corr/k3_kl:np.float64(0.0006864361699285837) - rollout_corr/rollout_ppl:np.float64(2.1477363551966846) - rollout_corr/rollout_log_ppl:np.float64(0.7532474518520758) - rollout_corr/log_ppl_diff:np.float64(0.0007572517497465014) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015343393897637725) - rollout_corr/log_ppl_diff_max:np.float64(0.005730688571929932) - rollout_corr/log_ppl_diff_min:np.float64(-0.004656791687011719) - rollout_corr/ppl_ratio:np.float64(1.0007591738831252) - rollout_corr/chi2_token:np.float64(0.0012387807946652174) - rollout_corr/chi2_seq:np.float64(1.0537742541637272) - actor/pg_loss:np.float64(-0.00016253837384283543) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015073742224558373) - actor/ppo_kl:np.float64(0.00016457487916454738) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24583709239959717) - perf/mfu/actor:np.float64(0.11041626514217825) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.51335525512695) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:7 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003921212162822485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003921212162822485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:596.58984375 - response_length/max:1495.0 - response_length/min:219.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:596.58984375 - response_length_non_aborted/max:1495.0 - response_length_non_aborted/min:219.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.65625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00023109838366508484 - timing_s/agent_loop/generate_sequences/min:np.float64(2.727230118587613) - timing_s/agent_loop/generate_sequences/max:np.float64(12.507632702589035) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.730043012139504) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.507632702589035) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:1495 - timing_s/gen:19.345353173092008 - timing_s/reward:0.06204149313271046 - timing_s/old_log_prob:3.282617036253214 - timing_s/adv:0.1314301285892725 - timing_s/update_actor:13.092846116051078 - timing_s/step:36.034051816910505 - timing_s/stop_profile:0.0001142602413892746 - timing_per_token_ms/update_actor:0.05816793558069137 - timing_per_token_ms/gen:0.12666622910874964 - timing_per_token_ms/adv:0.000583908127032092 - perf/total_num_tokens:225087 - perf/time_per_step:36.034051816910505 - perf/throughput:780.813524467322 (TaskRunner pid=2347384) Training Progress: 99%|█████████▉| 99/100 [1:42:03<00:45, 45.73s/it] (TaskRunner pid=2347384) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2347384) validation generation end (TaskRunner pid=2347384) [prompt] system (TaskRunner pid=2347384) (TaskRunner pid=2347384) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) ... (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2347384) (TaskRunner pid=2347384) A (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) user (TaskRunner pid=2347384) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2347384) (TaskRunner pid=2347384) A: -0.01 (TaskRunner pid=2347384) B: 1.69 (TaskRunner pid=2347384) C: 2.49 (TaskRunner pid=2347384) D: 0.45 (TaskRunner pid=2347384) Please reason step by step. (TaskRunner pid=2347384) assistant (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) [response] (TaskRunner pid=2347384) To determine the folding stability score of a protein sequence, we need to evaluate the balance of hydrophobic and hydrophilic interactions, as well as the potential for forming stable secondary structures like alpha-helices and beta-sheets. The folding stability score is often calculated using computational methods that analyze the propensity of amino acid residues to form such interactions. (TaskRunner pid=2347384) (TaskRunner pid=2347384) The given sequence is: (TaskRunner pid=2347384) **GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR** (TaskRunner pid=2347384) (TaskRunner pid=2347384) Let's break it down step by step: (TaskRunner pid=2347384) (TaskRunner pid=2347384) 1. **Hydrophobic vs. Hydrophilic Balance**: (TaskRunner pid=2347384) - Hydrophobic amino acids (e.g., G, S, T, R, Y, F, L, E, D, A, K, W, Q, V, H, N, G, T, Y, E, V, R) are present in varying amounts. (TaskRunner pid=2347384) - Hydrophobic residues tend to cluster in the core of the protein, while hydrophilic residues are often on the surface. A well-balanced distribution of these residues contributes to folding stability. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 2. **Secondary Structure Propensity**: (TaskRunner pid=2347384) - Amino acids like **R, K, D, E** are known to participate in salt bridges, which are important for stabilizing folded structures. (TaskRunner pid=2347384) - Residues like **F, Y, W** are aromatic and may form pi-stacking interactions. (TaskRunner pid=2347384) - The presence of multiple charged residues (e.g., EEE, RRAA, etc.) suggests the potential for hydrogen bonding and salt bridge formation, which can stabilize the folded structure. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 3. **Hydrogen Bonding Potential**: (TaskRunner pid=2347384) - The sequence contains several polar residues (e.g., E, D, R, K, Q, N, H, Y, etc.) that can form hydrogen bonds, contributing to the stability of the folded structure. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 4. **Potential for Misfolding**: (TaskRunner pid=2347384) - The sequence has a high number of charged residues and some repetitive patterns (e.g., multiple E's, R's, etc.), which could lead to electrostatic repulsion or aggregation if not properly folded. However, the presence of hydrophobic residues (e.g., G, S, T, V, etc.) suggests the possibility of forming a hydrophobic core. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 5. **Computational Tools**: (TaskRunner pid=2347384) - Folding stability scores can be estimated using tools like DSSP, FoldX, or other protein folding prediction algorithms. These tools consider the energy of hydrogen bonds, salt bridges, and van der Waals interactions. (TaskRunner pid=2347384) (TaskRunner pid=2347384) 6. **Estimation of Stability**: (TaskRunner pid=2347384) - Based on the balance of hydrophobic/hydrophilic residues and the potential for forming hydrogen bonds and salt bridges, the stability score is likely to be **positive**. (TaskRunner pid=2347384) - Among the given options (A: -0.01, B: 1.69, C: 2.49, D: 0.45), the most reasonable value would be **B: 1.69**, as it represents a moderate to high stability score, consistent with the presence of favorable interactions. (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) (TaskRunner pid=2347384) B (TaskRunner pid=2347384) (TaskRunner pid=2347384) [ground_truth] B (TaskRunner pid=2347384) [score] 1.0 (TaskRunner pid=2347384) [acc] 1.0 (TaskRunner pid=2347384) [pred] B (TaskRunner pid=2347384) [incorrect_format] 1 (TaskRunner pid=2347384) [feedback] (TaskRunner pid=2347384) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2347384) step:100 - global_seqlen/min:25660 - global_seqlen/max:43400 - global_seqlen/minmax_diff:17740 - global_seqlen/balanced_min:33498 - global_seqlen/balanced_max:37603 - global_seqlen/mean:34031.0 - actor/entropy:0.6494476199150085 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27054858207702637 - training/rollout_probs_diff_mean:0.003967562224715948 - training/rollout_probs_diff_std:0.008115967735648155 - training/rollout_actor_probs_pearson_corr:0.9996527433395386 - rollout_corr/rollout_is_mean:0.999800443649292 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.474292720464291e-06 - rollout_corr/rollout_is_max:1.875651478767395 - rollout_corr/rollout_is_min:0.3743396997451782 - rollout_corr/rollout_is_std:0.0328901931643486 - rollout_corr/rollout_is_eff_sample_size:0.9989191068729061 - rollout_corr/rollout_is_seq_mean:0.9997690320014954 - rollout_corr/rollout_is_seq_std:0.001492936396971345 - rollout_corr/rollout_is_seq_max:1.0050616264343262 - rollout_corr/rollout_is_seq_min:0.9940590262413025 - rollout_corr/rollout_is_seq_max_deviation:0.00594097375869751 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(2.040964142885059) - rollout_corr/training_log_ppl:np.float64(0.6961746725428384) - rollout_corr/kl:np.float64(0.0009347146087534952) - rollout_corr/k3_kl:np.float64(0.0006231129761431475) - rollout_corr/rollout_ppl:np.float64(2.0389733486808836) - rollout_corr/rollout_log_ppl:np.float64(0.6952399488363881) - rollout_corr/log_ppl_diff:np.float64(0.0009347237064503133) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013740459107793868) - rollout_corr/log_ppl_diff_max:np.float64(0.007616400718688965) - rollout_corr/log_ppl_diff_min:np.float64(-0.0043163299560546875) - rollout_corr/ppl_ratio:np.float64(1.0009363645222038) - rollout_corr/chi2_token:np.float64(0.000629746587947011) - rollout_corr/chi2_seq:np.float64(0.7056945031508803) - actor/pg_loss:np.float64(0.00012731854803860188) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012224168438024208) - actor/ppo_kl:np.float64(0.00011320239463330495) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2531181573867798) - perf/mfu/actor:np.float64(0.12333177659241132) - perf/max_memory_allocated_gb:np.float64(132.03091621398926) - perf/max_memory_reserved_gb:np.float64(144.787109375) - perf/cpu_memory_used_gb:np.float64(104.6416015625) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.49875) - val-aux/sciknoweval/reward/std@16:np.float64(0.2541486398580183) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.60202) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2054002064937382) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.39715999999999996) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.22536220172213436) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5014) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.25438537975653214) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6779) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14785794921891468) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.303) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1768647807422225) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.52822) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.19553605651535125) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.73414) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10570202036547473) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.2291) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11648076304197817) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5476200000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1254995217888188) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7777600000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07256344316810731) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.18696) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05929977285001639) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5538599999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0756443320805554) - val-aux/sciknoweval/score/mean@16:np.float64(0.49875) - val-aux/sciknoweval/score/std@16:np.float64(0.2541486398580183) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.60202) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2054002064937382) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.39715999999999996) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.22536220172213436) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5014) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.25438537975653214) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6779) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14785794921891468) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.303) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1768647807422225) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.52822) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.19553605651535125) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.73414) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10570202036547473) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.2291) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11648076304197817) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5476200000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1254995217888188) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7777600000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07256344316810731) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.18696) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05929977285001639) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5538599999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0756443320805554) - val-core/sciknoweval/acc/mean@16:np.float64(0.49875) - val-aux/sciknoweval/acc/std@16:np.float64(0.2541486398580183) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.60202) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2054002064937382) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.39715999999999996) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.22536220172213436) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5014) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.25438537975653214) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6779) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14785794921891468) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.303) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1768647807422225) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.52822) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.19553605651535125) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.73414) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10570202036547473) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.2291) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11648076304197817) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5476200000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1254995217888188) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7777600000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07256344316810731) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.18696) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05929977285001639) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5538599999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0756443320805554) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.95375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.10381127722390864) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9902200000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.04408740320835782) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.91962) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.12679376175372256) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.95504) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.10303116829198576) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9988800000000002) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.009215556459094006) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8675400000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.1409918624624078) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.98158) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.06365340394879623) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0012623787070447602) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8033600000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.13449797212902684) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9949799999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0280523469618475) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.74356) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.1034399395499081) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99924) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.006510218388623519) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:7 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02030278369784355 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02030278369784355 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:713.5625 - response_length/max:6262.0 - response_length/min:235.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:713.5625 - response_length_non_aborted/max:6262.0 - response_length_non_aborted/min:235.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.90625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017099827527999878 - timing_s/agent_loop/generate_sequences/min:np.float64(3.021278627216816) - timing_s/agent_loop/generate_sequences/max:np.float64(40.42630261927843) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.260560911876382) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(40.42630261927843) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:6262 - timing_s/gen:47.96704002469778 - timing_s/reward:0.0772107895463705 - timing_s/old_log_prob:3.4488426838070154 - timing_s/adv:0.12742269970476627 - timing_s/update_actor:14.441238490864635 - timing_s/step:66.15220173634589 - timing_s/testing:115.95291896350682 - timing_s/save_checkpoint:12.46778230369091 - timing_s/stop_profile:3.900192677974701e-05 - timing_per_token_ms/update_actor:0.05304442453522022 - timing_per_token_ms/gen:0.2625856180733653 - timing_per_token_ms/adv:0.0004680390662365427 - perf/total_num_tokens:272248 - perf/time_per_step:66.15220173634589 - perf/throughput:514.4348805748427 (TaskRunner pid=2347384) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2347384) "np.float64(0.49875), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2347384) "np.float64(0.2541486398580183), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2347384) "np.float64(0.60202), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2347384) "np.float64(0.2054002064937382), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2347384) "np.float64(0.39715999999999996), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2347384) "np.float64(0.22536220172213436), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2347384) "np.float64(0.5014), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2347384) "np.float64(0.25438537975653214), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2347384) "np.float64(0.6779), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2347384) "np.float64(0.14785794921891468), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2347384) "np.float64(0.303), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2347384) "np.float64(0.1768647807422225), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2347384) "np.float64(0.52822), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2347384) "np.float64(0.19553605651535125), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2347384) "np.float64(0.73414), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2347384) "np.float64(0.10570202036547473), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2347384) "np.float64(0.2291), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2347384) "np.float64(0.11648076304197817), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2347384) "np.float64(0.5476200000000001), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2347384) "np.float64(0.1254995217888188), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2347384) "np.float64(0.7777600000000001), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2347384) "np.float64(0.07256344316810731), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2347384) "np.float64(0.18696), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2347384) "np.float64(0.05929977285001639), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2347384) "np.float64(0.5538599999999999), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2347384) "np.float64(0.0756443320805554), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2347384) "np.float64(0.49875), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2347384) "np.float64(0.2541486398580183), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2347384) "np.float64(0.60202), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2347384) "np.float64(0.2054002064937382), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2347384) "np.float64(0.39715999999999996), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2347384) "np.float64(0.22536220172213436), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2347384) "np.float64(0.5014), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2347384) "np.float64(0.25438537975653214), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2347384) "np.float64(0.6779), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2347384) "np.float64(0.14785794921891468), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2347384) "np.float64(0.303), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2347384) "np.float64(0.1768647807422225), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2347384) "np.float64(0.52822), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2347384) "np.float64(0.19553605651535125), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2347384) "np.float64(0.73414), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2347384) "np.float64(0.10570202036547473), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2347384) "np.float64(0.2291), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2347384) "np.float64(0.11648076304197817), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2347384) "np.float64(0.5476200000000001), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2347384) "np.float64(0.1254995217888188), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2347384) "np.float64(0.7777600000000001), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2347384) "np.float64(0.07256344316810731), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2347384) "np.float64(0.18696), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2347384) "np.float64(0.05929977285001639), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2347384) "np.float64(0.5538599999999999), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2347384) "np.float64(0.0756443320805554), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2347384) "np.float64(0.49875), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2347384) "np.float64(0.2541486398580183), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2347384) "np.float64(0.60202), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2347384) "np.float64(0.2054002064937382), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2347384) "np.float64(0.39715999999999996), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2347384) "np.float64(0.22536220172213436), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2347384) "np.float64(0.5014), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2347384) "np.float64(0.25438537975653214), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2347384) "np.float64(0.6779), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2347384) "np.float64(0.14785794921891468), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2347384) "np.float64(0.303), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2347384) "np.float64(0.1768647807422225), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2347384) "np.float64(0.52822), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2347384) "np.float64(0.19553605651535125), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2347384) "np.float64(0.73414), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2347384) "np.float64(0.10570202036547473), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2347384) "np.float64(0.2291), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2347384) "np.float64(0.11648076304197817), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2347384) "np.float64(0.5476200000000001), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2347384) "np.float64(0.1254995217888188), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2347384) "np.float64(0.7777600000000001), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2347384) "np.float64(0.07256344316810731), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2347384) "np.float64(0.18696), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2347384) "np.float64(0.05929977285001639), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2347384) "np.float64(0.5538599999999999), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2347384) 'np.float64(0.0756443320805554), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.95375), " (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2347384) 'np.float64(0.10381127722390864), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2347384) 'np.float64(0.9902200000000001), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2347384) 'np.float64(0.04408740320835782), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': np.float64(0.91962), " (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2347384) 'np.float64(0.12679376175372256), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.95504), " (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2347384) 'np.float64(0.10303116829198576), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/best@4/mean': " (TaskRunner pid=2347384) 'np.float64(0.9988800000000002), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2347384) 'np.float64(0.009215556459094006), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2347384) 'np.float64(0.8675400000000001), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2347384) 'np.float64(0.1409918624624078), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(0.98158), " (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2347384) 'np.float64(0.06365340394879623), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(0.99992), " (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/best@8/std': " (TaskRunner pid=2347384) 'np.float64(0.0012623787070447602), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2347384) 'np.float64(0.8033600000000001), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2347384) 'np.float64(0.13449797212902684), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2347384) 'np.float64(0.9949799999999999), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2347384) 'np.float64(0.0280523469618475), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(0.74356), " (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2347384) 'np.float64(0.1034399395499081), ' (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(0.99924), " (TaskRunner pid=2347384) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2347384) "np.float64(0.006510218388623519), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2347384) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2347384) 'np.float64(2.0)}') (TaskRunner pid=2347384) Training Progress: 100%|██████████| 100/100 [1:45:17<00:00, 90.40s/it] Training Progress: 100%|██████████| 100/100 [1:45:17<00:00, 63.18s/it] (TaskRunner pid=2347384) wandb: updating run metadata; uploading output.log; uploading wandb-summary.json (TaskRunner pid=2347384) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2347384) wandb: uploading history steps 98-99, summary, console lines 674-798 (TaskRunner pid=2347384) wandb: (TaskRunner pid=2347384) wandb: Run history: (TaskRunner pid=2347384) wandb: actor/entropy ▂▂▁▂▂▂▂▂▂▁▂▂▂▃▄▅▄▅▆▅▄▅▅▆▅▅▅▅▅▅▆▆▇█▇▆▆▆▆▆ (TaskRunner pid=2347384) wandb: actor/grad_norm █▇▇▇▇▇█▇▆▄█▇▇▆▆▆▆▅▅▅▃▄▄▄▄▂▂▂▃▂▂▃▄▃▂▃▂▃▁▂ (TaskRunner pid=2347384) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2347384) wandb: actor/lr ▁▃▆▇████████████████████████████████████ (TaskRunner pid=2347384) wandb: actor/pg_clipfrac ▃▄▃▄▄▅▄▆▇▄█▃▂▅▅▃▃▂▂▁▃▃▃▂▂▂▂▃▁▂▂▂▂▁▂▁▂▁▂▁ (TaskRunner pid=2347384) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2347384) wandb: actor/pg_loss ▅▇▃▂█▆▇▆▆▃▁▅▁▃▅▃▄▃▅▅▆▅▅▆▅▅▄▄▅▄▅▅▅▃▄▄▆▄▅▆ (TaskRunner pid=2347384) wandb: actor/ppo_kl ▂▃▄▄▃▄█▃▅▄▃▇▁▆▃▃▂▃▃▃█▅▃▄▂▂▃▃▆▅▄▄▃▄▃▃▂▄▁▅ (TaskRunner pid=2347384) wandb: critic/advantages/max ████████████████████▁████████▁██████████ (TaskRunner pid=2347384) wandb: critic/advantages/mean ▂▆▆▃▁▅▅▇▂▄█▇▄▄▂▄▅▁▆▅▅▄▃▂▄█▅▇▂▅▅▄▅▄▃▄▅▅▅▅ (TaskRunner pid=2347384) wandb: +204 ... (TaskRunner pid=2347384) wandb: (TaskRunner pid=2347384) wandb: Run summary: (TaskRunner pid=2347384) wandb: actor/entropy 0.64945 (TaskRunner pid=2347384) wandb: actor/grad_norm 0.25312 (TaskRunner pid=2347384) wandb: actor/kl_loss 0 (TaskRunner pid=2347384) wandb: actor/lr 0.0 (TaskRunner pid=2347384) wandb: actor/pg_clipfrac 0.00012 (TaskRunner pid=2347384) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2347384) wandb: actor/pg_loss 0.00013 (TaskRunner pid=2347384) wandb: actor/ppo_kl 0.00011 (TaskRunner pid=2347384) wandb: critic/advantages/max 0.875 (TaskRunner pid=2347384) wandb: critic/advantages/mean -0.0203 (TaskRunner pid=2347384) wandb: +204 ... (TaskRunner pid=2347384) wandb: (TaskRunner pid=2347384) wandb: 🚀 View run qwen3gen-biology-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/j4sb7trr (TaskRunner pid=2347384) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2347384) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2347384) wandb: Find logs at: ./wandb/run-20260703_072949-j4sb7trr/logs (TaskRunner pid=2347384) Exception ignored in atexit callback: .teardown_atexit at 0x78cacdaab9c0> (TaskRunner pid=2347384) Traceback (most recent call last): (TaskRunner pid=2347384) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2347384) conn.teardown(hooks.exit_code) (TaskRunner pid=2347384) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2347384) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2347384) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2347384) return future.result() (TaskRunner pid=2347384) ^^^^^^^^^^^^^^^ (TaskRunner pid=2347384) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2347384) return self.__get_result() (TaskRunner pid=2347384) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2347384) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2347384) raise self._exception (TaskRunner pid=2347384) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2347384) return await fn() (TaskRunner pid=2347384) ^^^^^^^^^^ (TaskRunner pid=2347384) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2347384) await self._client.close() (TaskRunner pid=2347384) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2347384) await self._writer.wait_closed() (TaskRunner pid=2347384) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2347384) await self._protocol._get_close_waiter(self) (TaskRunner pid=2347384) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-03 09:15:13] Finished Qwen/Qwen3-8B biology grpo. [2026-07-03 09:15:13] Starting Qwen/Qwen3-8B biology rlsd_tr. Experiment: qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: biology Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet Ray tmp: /tmp/ray_q3g_biology_rlsd_tr_Qwen3_8B 2026-07-03 09:15:15,570 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/biology Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_biology_rlsd_tr_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/biology +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_biology_rlsd_tr_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/biology', 'EXPERIMENT': 'qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 09:15:25,314 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2377588) TaskRunner hostname: mole-workspace-rw, PID: 2377588 (TaskRunner pid=2377588) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2377588) 'calculate_entropy': False, (TaskRunner pid=2377588) 'calculate_sum_pi_squared': False, (TaskRunner pid=2377588) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2377588) 'async_save': False, (TaskRunner pid=2377588) 'load_contents': ['model', (TaskRunner pid=2377588) 'optimizer', (TaskRunner pid=2377588) 'extra'], (TaskRunner pid=2377588) 'save_contents': ['model', (TaskRunner pid=2377588) 'optimizer', (TaskRunner pid=2377588) 'extra']}, (TaskRunner pid=2377588) 'clip_ratio': 0.2, (TaskRunner pid=2377588) 'clip_ratio_c': 3.0, (TaskRunner pid=2377588) 'clip_ratio_high': 0.28, (TaskRunner pid=2377588) 'clip_ratio_low': 0.2, (TaskRunner pid=2377588) 'data_loader_seed': 42, (TaskRunner pid=2377588) 'entropy_checkpointing': False, (TaskRunner pid=2377588) 'entropy_coeff': 0, (TaskRunner pid=2377588) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2377588) 'freeze_vision_tower': False, (TaskRunner pid=2377588) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2377588) 'dtype': 'bfloat16', (TaskRunner pid=2377588) 'entropy_checkpointing': False, (TaskRunner pid=2377588) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2377588) 'forward_only': False, (TaskRunner pid=2377588) 'forward_prefetch': False, (TaskRunner pid=2377588) 'fsdp_size': -1, (TaskRunner pid=2377588) 'full_determinism': False, (TaskRunner pid=2377588) 'model_dtype': 'fp32', (TaskRunner pid=2377588) 'offload_policy': False, (TaskRunner pid=2377588) 'optimizer_offload': False, (TaskRunner pid=2377588) 'param_offload': False, (TaskRunner pid=2377588) 'reshard_after_forward': True, (TaskRunner pid=2377588) 'seed': 42, (TaskRunner pid=2377588) 'strategy': 'fsdp', (TaskRunner pid=2377588) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2377588) 'use_orig_params': False, (TaskRunner pid=2377588) 'use_torch_compile': True, (TaskRunner pid=2377588) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2377588) 'grad_clip': 1.0, (TaskRunner pid=2377588) 'kl_loss_coef': 0.001, (TaskRunner pid=2377588) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2377588) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2377588) 'loss_scale_factor': None, (TaskRunner pid=2377588) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2377588) 'betas': [0.9, 0.999], (TaskRunner pid=2377588) 'clip_grad': 1.0, (TaskRunner pid=2377588) 'lr': 1e-06, (TaskRunner pid=2377588) 'lr_scheduler_type': 'constant', (TaskRunner pid=2377588) 'lr_warmup_steps': 10, (TaskRunner pid=2377588) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2377588) 'min_lr_ratio': 0.0, (TaskRunner pid=2377588) 'num_cycles': 0.5, (TaskRunner pid=2377588) 'optimizer': 'AdamW', (TaskRunner pid=2377588) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2377588) 'override_optimizer_config': None, (TaskRunner pid=2377588) 'total_training_steps': -1, (TaskRunner pid=2377588) 'warmup_style': None, (TaskRunner pid=2377588) 'weight_decay': 0.01}, (TaskRunner pid=2377588) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2377588) 'clip_cov_lb': 1.0, (TaskRunner pid=2377588) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2377588) 'clip_cov_ub': 5.0, (TaskRunner pid=2377588) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2377588) 'loss_mode': 'rlsd', (TaskRunner pid=2377588) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2377588) 'ppo_epochs': 1, (TaskRunner pid=2377588) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2377588) 'ppo_micro_batch_size': None, (TaskRunner pid=2377588) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2377588) 'ppo_mini_batch_size': 8, (TaskRunner pid=2377588) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2377588) 'all_ranks': False, (TaskRunner pid=2377588) 'enable': False, (TaskRunner pid=2377588) 'ranks': [], (TaskRunner pid=2377588) 'save_path': 'outputs/profile', (TaskRunner pid=2377588) 'tool': None, (TaskRunner pid=2377588) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2377588) 'analysis': True, (TaskRunner pid=2377588) 'contents': [], (TaskRunner pid=2377588) 'discrete': False, (TaskRunner pid=2377588) 'level': 'level0'}, (TaskRunner pid=2377588) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2377588) 'discrete': False}, (TaskRunner pid=2377588) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2377588) 'step_end': None, (TaskRunner pid=2377588) 'step_start': 0}, (TaskRunner pid=2377588) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2377588) 'stack_depth': 32, (TaskRunner pid=2377588) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2377588) 'rollout_n': 8, (TaskRunner pid=2377588) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2377588) 'mode': 'disabled', (TaskRunner pid=2377588) 'record_file': None, (TaskRunner pid=2377588) 'replay_file': None}, (TaskRunner pid=2377588) 'self_distillation': (TaskRunner pid=2377588) {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2377588) 'alpha': 0.5, (TaskRunner pid=2377588) 'distillation_add_tail': True, (TaskRunner pid=2377588) 'distillation_topk': 100, (TaskRunner pid=2377588) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2377588) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2377588) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2377588) 'enable': False, (TaskRunner pid=2377588) 'loss_weight': 0.0, (TaskRunner pid=2377588) 'mask': 'all'}, (TaskRunner pid=2377588) 'feedback_template': '\n' (TaskRunner pid=2377588) 'The ' (TaskRunner pid=2377588) 'following ' (TaskRunner pid=2377588) 'is ' (TaskRunner pid=2377588) 'feedback ' (TaskRunner pid=2377588) 'from ' (TaskRunner pid=2377588) 'your ' (TaskRunner pid=2377588) 'unsuccessful ' (TaskRunner pid=2377588) 'earlier ' (TaskRunner pid=2377588) 'attempt:\n' (TaskRunner pid=2377588) '\n' (TaskRunner pid=2377588) '{feedback_raw}', (TaskRunner pid=2377588) 'full_logit_distillation': True, (TaskRunner pid=2377588) 'include_environment_feedback': True, (TaskRunner pid=2377588) 'is_clip': 2, (TaskRunner pid=2377588) 'max_reprompt_len': 10240, (TaskRunner pid=2377588) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2377588) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2377588) '\n' (TaskRunner pid=2377588) 'Correctly ' (TaskRunner pid=2377588) 'solve ' (TaskRunner pid=2377588) 'the ' (TaskRunner pid=2377588) 'original ' (TaskRunner pid=2377588) 'question.', (TaskRunner pid=2377588) 'reprompt_truncation': 'right', (TaskRunner pid=2377588) 'solution_template': '\n' (TaskRunner pid=2377588) 'Correct ' (TaskRunner pid=2377588) 'solution:\n' (TaskRunner pid=2377588) '\n' (TaskRunner pid=2377588) '{successful_previous_attempt}', (TaskRunner pid=2377588) 'srpo_dynamic_weighting': False, (TaskRunner pid=2377588) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2377588) 'success_reward_threshold': 0.5, (TaskRunner pid=2377588) 'teacher_regularization': 'trust-region', (TaskRunner pid=2377588) 'teacher_update_rate': 0.1, (TaskRunner pid=2377588) 'token_reweight_decay_steps': 0, (TaskRunner pid=2377588) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2377588) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2377588) 'shuffle': False, (TaskRunner pid=2377588) 'strategy': 'fsdp', (TaskRunner pid=2377588) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2377588) 'tau_neg': 1.05, (TaskRunner pid=2377588) 'tau_pos': 1.0, (TaskRunner pid=2377588) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2377588) 'use_dynamic_bsz': False, (TaskRunner pid=2377588) 'use_fused_kernels': False, (TaskRunner pid=2377588) 'use_kl_loss': False, (TaskRunner pid=2377588) 'use_prefix_grouper': False, (TaskRunner pid=2377588) 'use_remove_padding': True, (TaskRunner pid=2377588) 'use_torch_compile': True}, (TaskRunner pid=2377588) 'hybrid_engine': True, (TaskRunner pid=2377588) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2377588) 'custom_chat_template': None, (TaskRunner pid=2377588) 'enable_activation_offload': False, (TaskRunner pid=2377588) 'enable_gradient_checkpointing': True, (TaskRunner pid=2377588) 'exclude_modules': None, (TaskRunner pid=2377588) 'external_lib': None, (TaskRunner pid=2377588) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2377588) 'hf_config_path': None, (TaskRunner pid=2377588) 'lora_adapter_path': None, (TaskRunner pid=2377588) 'lora_alpha': 16, (TaskRunner pid=2377588) 'lora_rank': 0, (TaskRunner pid=2377588) 'override_config': {}, (TaskRunner pid=2377588) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2377588) 'target_modules': 'all-linear', (TaskRunner pid=2377588) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2377588) 'num_shards': 4}, (TaskRunner pid=2377588) 'tokenizer_path': None, (TaskRunner pid=2377588) 'trust_remote_code': True, (TaskRunner pid=2377588) 'use_fused_kernels': False, (TaskRunner pid=2377588) 'use_liger': False, (TaskRunner pid=2377588) 'use_remove_padding': True, (TaskRunner pid=2377588) 'use_shm': False}, (TaskRunner pid=2377588) 'nccl_timeout': 600, (TaskRunner pid=2377588) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2377588) 'entropy_checkpointing': False, (TaskRunner pid=2377588) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2377588) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2377588) 'dtype': 'bfloat16', (TaskRunner pid=2377588) 'entropy_checkpointing': False, (TaskRunner pid=2377588) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2377588) 'forward_only': True, (TaskRunner pid=2377588) 'forward_prefetch': False, (TaskRunner pid=2377588) 'fsdp_size': -1, (TaskRunner pid=2377588) 'full_determinism': False, (TaskRunner pid=2377588) 'model_dtype': 'fp32', (TaskRunner pid=2377588) 'offload_policy': False, (TaskRunner pid=2377588) 'optimizer_offload': False, (TaskRunner pid=2377588) 'param_offload': False, (TaskRunner pid=2377588) 'reshard_after_forward': True, (TaskRunner pid=2377588) 'seed': 42, (TaskRunner pid=2377588) 'strategy': 'fsdp', (TaskRunner pid=2377588) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2377588) 'use_orig_params': False, (TaskRunner pid=2377588) 'use_torch_compile': True, (TaskRunner pid=2377588) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2377588) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2377588) 'log_prob_micro_batch_size': None, (TaskRunner pid=2377588) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2377588) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2377588) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2377588) 'all_ranks': False, (TaskRunner pid=2377588) 'enable': False, (TaskRunner pid=2377588) 'ranks': [], (TaskRunner pid=2377588) 'save_path': 'outputs/profile', (TaskRunner pid=2377588) 'tool': None, (TaskRunner pid=2377588) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2377588) 'analysis': True, (TaskRunner pid=2377588) 'contents': [], (TaskRunner pid=2377588) 'discrete': False, (TaskRunner pid=2377588) 'level': 'level0'}, (TaskRunner pid=2377588) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2377588) 'discrete': False}, (TaskRunner pid=2377588) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2377588) 'step_end': None, (TaskRunner pid=2377588) 'step_start': 0}, (TaskRunner pid=2377588) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2377588) 'stack_depth': 32, (TaskRunner pid=2377588) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2377588) 'rollout_n': 8, (TaskRunner pid=2377588) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2377588) 'mode': 'disabled', (TaskRunner pid=2377588) 'record_file': None, (TaskRunner pid=2377588) 'replay_file': None}, (TaskRunner pid=2377588) 'strategy': 'fsdp', (TaskRunner pid=2377588) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2377588) 'use_torch_compile': True}, (TaskRunner pid=2377588) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2377588) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2377588) 'agent_loop_config_path': None, (TaskRunner pid=2377588) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2377588) 'name': None, (TaskRunner pid=2377588) 'path': None}, (TaskRunner pid=2377588) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2377588) 'num_workers': 8}, (TaskRunner pid=2377588) 'calculate_log_probs': True, (TaskRunner pid=2377588) 'cudagraph_capture_sizes': None, (TaskRunner pid=2377588) 'data_parallel_size': 1, (TaskRunner pid=2377588) 'disable_log_stats': True, (TaskRunner pid=2377588) 'do_sample': True, (TaskRunner pid=2377588) 'dtype': 'bfloat16', (TaskRunner pid=2377588) 'enable_chunked_prefill': True, (TaskRunner pid=2377588) 'enable_prefix_caching': True, (TaskRunner pid=2377588) 'enable_rollout_routing_replay': False, (TaskRunner pid=2377588) 'enforce_eager': False, (TaskRunner pid=2377588) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2377588) 'expert_parallel_size': 1, (TaskRunner pid=2377588) 'free_cache_engine': True, (TaskRunner pid=2377588) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2377588) 'ignore_eos': False, (TaskRunner pid=2377588) 'layered_summon': False, (TaskRunner pid=2377588) 'load_format': 'dummy', (TaskRunner pid=2377588) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2377588) 'log_prob_micro_batch_size': None, (TaskRunner pid=2377588) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2377588) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2377588) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2377588) 'max_model_len': 10240, (TaskRunner pid=2377588) 'max_num_batched_tokens': 10240, (TaskRunner pid=2377588) 'max_num_seqs': 1024, (TaskRunner pid=2377588) 'mode': 'async', (TaskRunner pid=2377588) 'multi_stage_wake_up': False, (TaskRunner pid=2377588) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2377588) 'enable': False, (TaskRunner pid=2377588) 'format': 'hermes', (TaskRunner pid=2377588) 'interaction_config_path': None, (TaskRunner pid=2377588) 'max_assistant_turns': None, (TaskRunner pid=2377588) 'max_parallel_calls': 1, (TaskRunner pid=2377588) 'max_tool_response_length': 256, (TaskRunner pid=2377588) 'max_user_turns': None, (TaskRunner pid=2377588) 'num_repeat_rollouts': None, (TaskRunner pid=2377588) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2377588) 'tool_config_path': None, (TaskRunner pid=2377588) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2377588) 'use_inference_chat_template': False}, (TaskRunner pid=2377588) 'n': 8, (TaskRunner pid=2377588) 'name': 'vllm', (TaskRunner pid=2377588) 'over_sample_rate': 0, (TaskRunner pid=2377588) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2377588) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2377588) 'all_ranks': False, (TaskRunner pid=2377588) 'enable': False, (TaskRunner pid=2377588) 'ranks': [], (TaskRunner pid=2377588) 'save_path': 'outputs/profile', (TaskRunner pid=2377588) 'tool': None, (TaskRunner pid=2377588) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2377588) 'analysis': True, (TaskRunner pid=2377588) 'contents': [], (TaskRunner pid=2377588) 'discrete': False, (TaskRunner pid=2377588) 'level': 'level0'}, (TaskRunner pid=2377588) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2377588) 'discrete': False}, (TaskRunner pid=2377588) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2377588) 'step_end': None, (TaskRunner pid=2377588) 'step_start': 0}, (TaskRunner pid=2377588) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2377588) 'stack_depth': 32, (TaskRunner pid=2377588) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2377588) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2377588) 'enable': False, (TaskRunner pid=2377588) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2377588) 'port': 9090, (TaskRunner pid=2377588) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2377588) 'prompt_length': 2048, (TaskRunner pid=2377588) 'quantization': None, (TaskRunner pid=2377588) 'quantization_config_file': None, (TaskRunner pid=2377588) 'response_length': 8192, (TaskRunner pid=2377588) 'scheduling_policy': 'fcfs', (TaskRunner pid=2377588) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2377588) 'skip_rollout': False, (TaskRunner pid=2377588) 'skip_tokenizer_init': True, (TaskRunner pid=2377588) 'temperature': 1.0, (TaskRunner pid=2377588) 'tensor_model_parallel_size': 2, (TaskRunner pid=2377588) 'top_k': -1, (TaskRunner pid=2377588) 'top_p': 1.0, (TaskRunner pid=2377588) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2377588) 'backend': None, (TaskRunner pid=2377588) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2377588) 'token2text': False}, (TaskRunner pid=2377588) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2377588) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2377588) 'do_sample': True, (TaskRunner pid=2377588) 'n': 16, (TaskRunner pid=2377588) 'temperature': 0.6, (TaskRunner pid=2377588) 'top_k': -1, (TaskRunner pid=2377588) 'top_p': 0.95}}}, (TaskRunner pid=2377588) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2377588) 'adv_estimator': 'grpo', (TaskRunner pid=2377588) 'gamma': 1.0, (TaskRunner pid=2377588) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2377588) 'horizon': 10000, (TaskRunner pid=2377588) 'kl_coef': 0.001, (TaskRunner pid=2377588) 'target_kl': 0.1, (TaskRunner pid=2377588) 'type': 'fixed'}, (TaskRunner pid=2377588) 'kl_penalty': 'kl', (TaskRunner pid=2377588) 'lam': 1.0, (TaskRunner pid=2377588) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2377588) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2377588) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2377588) 'loss_type': 'ppo_clip', (TaskRunner pid=2377588) 'rollout_is': 'token', (TaskRunner pid=2377588) 'rollout_is_batch_normalize': False, (TaskRunner pid=2377588) 'rollout_is_threshold': 2.0, (TaskRunner pid=2377588) 'rollout_rs': None, (TaskRunner pid=2377588) 'rollout_rs_threshold': None}, (TaskRunner pid=2377588) 'use_kl_in_reward': False, (TaskRunner pid=2377588) 'use_pf_ppo': False}, (TaskRunner pid=2377588) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2377588) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2377588) 'async_save': False, (TaskRunner pid=2377588) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2377588) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2377588) 'cliprange_value': 0.5, (TaskRunner pid=2377588) 'data_loader_seed': 42, (TaskRunner pid=2377588) 'enable': None, (TaskRunner pid=2377588) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2377588) 'forward_micro_batch_size': None, (TaskRunner pid=2377588) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2377588) 'grad_clip': 1.0, (TaskRunner pid=2377588) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2377588) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2377588) 'enable_activation_offload': False, (TaskRunner pid=2377588) 'enable_gradient_checkpointing': True, (TaskRunner pid=2377588) 'external_lib': None, (TaskRunner pid=2377588) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2377588) 'dtype': 'bfloat16', (TaskRunner pid=2377588) 'entropy_checkpointing': False, (TaskRunner pid=2377588) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2377588) 'forward_only': False, (TaskRunner pid=2377588) 'forward_prefetch': False, (TaskRunner pid=2377588) 'fsdp_size': -1, (TaskRunner pid=2377588) 'full_determinism': False, (TaskRunner pid=2377588) 'model_dtype': 'fp32', (TaskRunner pid=2377588) 'offload_policy': False, (TaskRunner pid=2377588) 'optimizer_offload': False, (TaskRunner pid=2377588) 'param_offload': False, (TaskRunner pid=2377588) 'reshard_after_forward': True, (TaskRunner pid=2377588) 'seed': 42, (TaskRunner pid=2377588) 'strategy': 'fsdp', (TaskRunner pid=2377588) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2377588) 'use_orig_params': False, (TaskRunner pid=2377588) 'use_torch_compile': True, (TaskRunner pid=2377588) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2377588) 'lora_alpha': 16, (TaskRunner pid=2377588) 'lora_rank': 0, (TaskRunner pid=2377588) 'override_config': {}, (TaskRunner pid=2377588) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2377588) 'target_modules': 'all-linear', (TaskRunner pid=2377588) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2377588) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2377588) 'trust_remote_code': True, (TaskRunner pid=2377588) 'use_remove_padding': False, (TaskRunner pid=2377588) 'use_shm': False}, (TaskRunner pid=2377588) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2377588) 'betas': [0.9, 0.999], (TaskRunner pid=2377588) 'clip_grad': 1.0, (TaskRunner pid=2377588) 'lr': 1e-05, (TaskRunner pid=2377588) 'lr_scheduler_type': 'constant', (TaskRunner pid=2377588) 'lr_warmup_steps': -1, (TaskRunner pid=2377588) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2377588) 'min_lr_ratio': 0.0, (TaskRunner pid=2377588) 'num_cycles': 0.5, (TaskRunner pid=2377588) 'optimizer': 'AdamW', (TaskRunner pid=2377588) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2377588) 'override_optimizer_config': None, (TaskRunner pid=2377588) 'total_training_steps': -1, (TaskRunner pid=2377588) 'warmup_style': None, (TaskRunner pid=2377588) 'weight_decay': 0.01}, (TaskRunner pid=2377588) 'ppo_epochs': 1, (TaskRunner pid=2377588) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2377588) 'ppo_micro_batch_size': None, (TaskRunner pid=2377588) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2377588) 'ppo_mini_batch_size': 8, (TaskRunner pid=2377588) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2377588) 'all_ranks': False, (TaskRunner pid=2377588) 'enable': False, (TaskRunner pid=2377588) 'ranks': [], (TaskRunner pid=2377588) 'save_path': 'outputs/profile', (TaskRunner pid=2377588) 'tool': None, (TaskRunner pid=2377588) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2377588) 'analysis': True, (TaskRunner pid=2377588) 'contents': [], (TaskRunner pid=2377588) 'discrete': False, (TaskRunner pid=2377588) 'level': 'level0'}, (TaskRunner pid=2377588) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2377588) 'discrete': False}, (TaskRunner pid=2377588) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2377588) 'step_end': None, (TaskRunner pid=2377588) 'step_start': 0}, (TaskRunner pid=2377588) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2377588) 'stack_depth': 32, (TaskRunner pid=2377588) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2377588) 'rollout_n': 8, (TaskRunner pid=2377588) 'shuffle': False, (TaskRunner pid=2377588) 'strategy': 'fsdp', (TaskRunner pid=2377588) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2377588) 'use_dynamic_bsz': False}, (TaskRunner pid=2377588) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2377588) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2377588) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2377588) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2377588) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2377588) 'dataloader_num_workers': 8, (TaskRunner pid=2377588) 'filter_overlong_prompts': True, (TaskRunner pid=2377588) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2377588) 'image_key': 'images', (TaskRunner pid=2377588) 'image_patch_size': 14, (TaskRunner pid=2377588) 'max_prompt_length': 2048, (TaskRunner pid=2377588) 'max_response_length': 8192, (TaskRunner pid=2377588) 'prompt_key': 'prompt', (TaskRunner pid=2377588) 'return_full_prompt': False, (TaskRunner pid=2377588) 'return_multi_modal_inputs': True, (TaskRunner pid=2377588) 'return_raw_chat': True, (TaskRunner pid=2377588) 'return_raw_input_ids': False, (TaskRunner pid=2377588) 'reward_fn_key': 'data_source', (TaskRunner pid=2377588) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2377588) 'seed': None, (TaskRunner pid=2377588) 'shuffle': True, (TaskRunner pid=2377588) 'tokenizer': None, (TaskRunner pid=2377588) 'tool_config_path': None, (TaskRunner pid=2377588) 'train_batch_size': 32, (TaskRunner pid=2377588) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet'], (TaskRunner pid=2377588) 'train_max_samples': 3200, (TaskRunner pid=2377588) 'truncation': 'error', (TaskRunner pid=2377588) 'trust_remote_code': True, (TaskRunner pid=2377588) 'use_shm': False, (TaskRunner pid=2377588) 'val_batch_size': None, (TaskRunner pid=2377588) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet'], (TaskRunner pid=2377588) 'val_max_samples': -1, (TaskRunner pid=2377588) 'validation_shuffle': False, (TaskRunner pid=2377588) 'video_key': 'videos'}, (TaskRunner pid=2377588) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2377588) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2377588) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2377588) 'cuda-memory-usage': 'true', (TaskRunner pid=2377588) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2377588) 'discrete': False, (TaskRunner pid=2377588) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2377588) 'capture-range-end': None, (TaskRunner pid=2377588) 'cuda-graph-trace': 'graph', (TaskRunner pid=2377588) 'cuda-memory-usage': 'true', (TaskRunner pid=2377588) 'kill': 'none', (TaskRunner pid=2377588) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2377588) 'torch_memory': (TaskRunner pid=2377588) {'context': 'all', (TaskRunner pid=2377588) 'kw_args': {}, (TaskRunner pid=2377588) 'stack_depth': 32, (TaskRunner pid=2377588) 'stacks': 'all', (TaskRunner pid=2377588) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2377588) 'profile_continuous_steps': False, (TaskRunner pid=2377588) 'save_path': 'outputs/profile', (TaskRunner pid=2377588) 'steps': None, (TaskRunner pid=2377588) 'tool': None}, (TaskRunner pid=2377588) 'max_model_len': 10240, (TaskRunner pid=2377588) 'ray_kwargs': (TaskRunner pid=2377588) {'ray_init': {'_temp_dir': '/tmp/ray_q3g_biology_rlsd_tr_Qwen3_8B', (TaskRunner pid=2377588) 'include_dashboard': False, (TaskRunner pid=2377588) 'num_cpus': None, (TaskRunner pid=2377588) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2377588) 'TASK': 'datasets/sciknoweval/biology', (TaskRunner pid=2377588) 'USER': 'root'}}}, (TaskRunner pid=2377588) 'timeline_json_file': None}, (TaskRunner pid=2377588) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2377588) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2377588) 'name': 'custom_reward_manager', (TaskRunner pid=2377588) 'path': None}, (TaskRunner pid=2377588) 'name': 'naive', (TaskRunner pid=2377588) 'source': 'register'}, (TaskRunner pid=2377588) 'reward_model': {'enable': False, (TaskRunner pid=2377588) 'enable_resource_pool': False, (TaskRunner pid=2377588) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2377588) 'launch_reward_fn_async': False, (TaskRunner pid=2377588) 'max_length': None, (TaskRunner pid=2377588) 'micro_batch_size': None, (TaskRunner pid=2377588) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2377588) 'model': {'external_lib': None, (TaskRunner pid=2377588) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2377588) 'forward_prefetch': False, (TaskRunner pid=2377588) 'fsdp_size': -1, (TaskRunner pid=2377588) 'param_offload': False, (TaskRunner pid=2377588) 'reshard_after_forward': True, (TaskRunner pid=2377588) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2377588) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2377588) 'override_config': {}, (TaskRunner pid=2377588) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2377588) 'trust_remote_code': False, (TaskRunner pid=2377588) 'use_fused_kernels': False, (TaskRunner pid=2377588) 'use_remove_padding': False, (TaskRunner pid=2377588) 'use_shm': False}, (TaskRunner pid=2377588) 'n_gpus_per_node': 8, (TaskRunner pid=2377588) 'nnodes': 0, (TaskRunner pid=2377588) 'num_workers': 1, (TaskRunner pid=2377588) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2377588) 'all_ranks': False, (TaskRunner pid=2377588) 'enable': False, (TaskRunner pid=2377588) 'ranks': [], (TaskRunner pid=2377588) 'save_path': 'outputs/profile', (TaskRunner pid=2377588) 'tool': None, (TaskRunner pid=2377588) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2377588) 'analysis': True, (TaskRunner pid=2377588) 'contents': [], (TaskRunner pid=2377588) 'discrete': False, (TaskRunner pid=2377588) 'level': 'level0'}, (TaskRunner pid=2377588) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2377588) 'discrete': False}, (TaskRunner pid=2377588) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2377588) 'step_end': None, (TaskRunner pid=2377588) 'step_start': 0}, (TaskRunner pid=2377588) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2377588) 'stack_depth': 32, (TaskRunner pid=2377588) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2377588) 'reward_loop_class_name': None, (TaskRunner pid=2377588) 'reward_loop_module_path': None, (TaskRunner pid=2377588) 'reward_loop_source': 'register', (TaskRunner pid=2377588) 'reward_manager': 'naive', (TaskRunner pid=2377588) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2377588) 'cudagraph_capture_sizes': None, (TaskRunner pid=2377588) 'data_parallel_size': 1, (TaskRunner pid=2377588) 'disable_log_stats': True, (TaskRunner pid=2377588) 'dtype': 'bfloat16', (TaskRunner pid=2377588) 'enable_chunked_prefill': True, (TaskRunner pid=2377588) 'enable_prefix_caching': True, (TaskRunner pid=2377588) 'enforce_eager': True, (TaskRunner pid=2377588) 'engine_kwargs': {}, (TaskRunner pid=2377588) 'expert_parallel_size': 1, (TaskRunner pid=2377588) 'free_cache_engine': True, (TaskRunner pid=2377588) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2377588) 'limit_images': None, (TaskRunner pid=2377588) 'load_format': 'auto', (TaskRunner pid=2377588) 'max_model_len': None, (TaskRunner pid=2377588) 'max_num_batched_tokens': 8192, (TaskRunner pid=2377588) 'max_num_seqs': 1024, (TaskRunner pid=2377588) 'name': '???', (TaskRunner pid=2377588) 'prompt_length': 2048, (TaskRunner pid=2377588) 'response_length': 2048, (TaskRunner pid=2377588) 'skip_tokenizer_init': False, (TaskRunner pid=2377588) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2377588) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2377588) 'memory_limit_mb': 1024, (TaskRunner pid=2377588) 'url': None}, (TaskRunner pid=2377588) 'strategy': 'fsdp', (TaskRunner pid=2377588) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2377588) 'use_dynamic_bsz': False, (TaskRunner pid=2377588) 'use_reward_loop': False}, (TaskRunner pid=2377588) 'trainer': {'balance_batch': True, (TaskRunner pid=2377588) 'critic_warmup': 0, (TaskRunner pid=2377588) 'default_hdfs_dir': None, (TaskRunner pid=2377588) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2377588) 'del_local_ckpt_after_load': False, (TaskRunner pid=2377588) 'device': 'cuda', (TaskRunner pid=2377588) 'esi_redundant_time': 0, (TaskRunner pid=2377588) 'experiment_name': 'qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2377588) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2377588) 'log_val_generations': 0, (TaskRunner pid=2377588) 'logger': ['console', 'wandb'], (TaskRunner pid=2377588) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2377588) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2377588) 'n_gpus_per_node': 8, (TaskRunner pid=2377588) 'nnodes': 1, (TaskRunner pid=2377588) 'project_name': 'SDPO-root', (TaskRunner pid=2377588) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2377588) 'resume_from_path': None, (TaskRunner pid=2377588) 'resume_mode': 'auto', (TaskRunner pid=2377588) 'rollout_data_dir': None, (TaskRunner pid=2377588) 'save_freq': 10, (TaskRunner pid=2377588) 'test_freq': 10, (TaskRunner pid=2377588) 'total_epochs': 30, (TaskRunner pid=2377588) 'total_training_steps': 100, (TaskRunner pid=2377588) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2377588) 'val_before_train': False, (TaskRunner pid=2377588) 'val_only': False, (TaskRunner pid=2377588) 'validation_data_dir': None}, (TaskRunner pid=2377588) 'transfer_queue': {'enable': False}, (TaskRunner pid=2377588) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/biology', (TaskRunner pid=2377588) 'dir': '/users/root/SDPO', (TaskRunner pid=2377588) 'log_dir': '/users/root/output', (TaskRunner pid=2377588) 'task': 'datasets/sciknoweval/biology'}} (TaskRunner pid=2377588) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2377588) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2377588) use_critic=need_critic(config), (TaskRunner pid=2377588) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2377588) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2377588) Using dataset class: RLHFDataset (TaskRunner pid=2377588) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2377588) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2377588) dataset len: 450 (TaskRunner pid=2377588) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2377588) WARNING:2026-07-03 09:15:34,341:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2377588) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/450 [00:00 (TaskRunner pid=2377588) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2377588) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2377588) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2377588) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2377588) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2377588) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 50/50 [00:00<00:00, 81.68 examples/s] (TaskRunner pid=2377588) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2377588) self.use_critic = need_critic(self.config) (WorkerDict pid=2377965) [W703 09:15:42.218884410 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53309 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2377966) [Gloo] Rank 1 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2377965) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2377965) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2377972) [W703 09:15:43.975535048 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53309 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2377965) Model config after override: Qwen3Config { (WorkerDict pid=2377965) "architectures": [ (WorkerDict pid=2377965) "Qwen3ForCausalLM" (WorkerDict pid=2377965) ], (WorkerDict pid=2377965) "attention_bias": false, (WorkerDict pid=2377965) "attention_dropout": 0.0, (WorkerDict pid=2377965) "dtype": "bfloat16", (WorkerDict pid=2377965) "eos_token_id": 151645, (WorkerDict pid=2377965) "head_dim": 128, (WorkerDict pid=2377965) "hidden_act": "silu", (WorkerDict pid=2377965) "hidden_size": 4096, (WorkerDict pid=2377965) "initializer_range": 0.02, (WorkerDict pid=2377965) "intermediate_size": 12288, (WorkerDict pid=2377965) "layer_types": [ (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention", (WorkerDict pid=2377965) "full_attention" (WorkerDict pid=2377965) ], (WorkerDict pid=2377965) "max_position_embeddings": 40960, (WorkerDict pid=2377965) "max_window_layers": 36, (WorkerDict pid=2377965) "model_type": "qwen3", (WorkerDict pid=2377965) "num_attention_heads": 32, (WorkerDict pid=2377965) "num_hidden_layers": 36, (WorkerDict pid=2377965) "num_key_value_heads": 8, (WorkerDict pid=2377965) "pad_token_id": 151643, (WorkerDict pid=2377965) "rms_norm_eps": 1e-06, (WorkerDict pid=2377965) "rope_scaling": null, (WorkerDict pid=2377965) "rope_theta": 1000000, (WorkerDict pid=2377965) "sliding_window": null, (WorkerDict pid=2377965) "tie_word_embeddings": false, (WorkerDict pid=2377965) "transformers_version": "4.57.1", (WorkerDict pid=2377965) "use_cache": true, (WorkerDict pid=2377965) "use_sliding_window": false, (WorkerDict pid=2377965) "vocab_size": 151936 (WorkerDict pid=2377965) } (WorkerDict pid=2377965) (WorkerDict pid=2377965) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2377965) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2377965) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2377965) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2377965) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2377966) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2377966) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2377972) Loading checkpoint shards: 80%|████████ | 4/5 [00:11<00:02, 2.87s/it] [repeated 15x across cluster] (WorkerDict pid=2377971) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.20s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.60s/it] [repeated 7x across cluster] (WorkerDict pid=2377965) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2377965) Actor use_remove_padding=True (WorkerDict pid=2377965) Actor use_fused_kernels=False (WorkerDict pid=2377965) Actor use_prefix_grouper=False (WorkerDict pid=2377971) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2377971) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2377969) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2377969) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2377966) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2377965) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2377965) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2377968) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2377968) warnings.warn( (WorkerDict pid=2377965) reference model: Qwen/Qwen3-8B (WorkerDict pid=2377968) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2377968) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2377968) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2377965) Ref use_remove_padding=True (WorkerDict pid=2377965) Ref use_fused_kernels=False (WorkerDict pid=2377965) Ref use_prefix_grouper=False (WorkerDict pid=2377967) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2377967) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2377588) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2377588) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2377967) Loading checkpoint shards: 80%|████████ | 4/5 [00:11<00:02, 2.88s/it] [repeated 15x across cluster] (WorkerDict pid=2377967) Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00, 2.19s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:12<00:00, 2.59s/it] [repeated 7x across cluster] (vLLMHttpServer pid=2378895) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2378895) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2378895) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2378895) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2378893) ['serve', (vLLMHttpServer pid=2378893) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2378893) '--dtype', (vLLMHttpServer pid=2378893) 'bfloat16', (vLLMHttpServer pid=2378893) '--load_format', (vLLMHttpServer pid=2378893) 'dummy', (vLLMHttpServer pid=2378893) '--trust_remote_code', (vLLMHttpServer pid=2378893) '--max_model_len', (vLLMHttpServer pid=2378893) '40960', (vLLMHttpServer pid=2378893) '--max_num_seqs', (vLLMHttpServer pid=2378893) '1024', (vLLMHttpServer pid=2378893) '--enable_chunked_prefill', (vLLMHttpServer pid=2378893) '--max_num_batched_tokens', (vLLMHttpServer pid=2378893) '10240', (vLLMHttpServer pid=2378893) '--enable_prefix_caching', (vLLMHttpServer pid=2378893) '--enable_sleep_mode', (vLLMHttpServer pid=2378893) '--logprobs_mode', (vLLMHttpServer pid=2378893) 'processed_logprobs', (vLLMHttpServer pid=2378893) '--disable_custom_all_reduce', (vLLMHttpServer pid=2378893) '--gpu_memory_utilization', (vLLMHttpServer pid=2378893) '0.8', (vLLMHttpServer pid=2378893) '--disable_log_stats', (vLLMHttpServer pid=2378893) '--tensor_parallel_size', (vLLMHttpServer pid=2378893) '2', (vLLMHttpServer pid=2378893) '--seed', (vLLMHttpServer pid=2378893) '0', (vLLMHttpServer pid=2378893) '--override_generation_config', (vLLMHttpServer pid=2378893) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2378893) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2378893) '--hf_overrides', (vLLMHttpServer pid=2378893) '{}', (vLLMHttpServer pid=2378893) '--scheduling_policy', (vLLMHttpServer pid=2378893) 'fcfs'] (vLLMHttpServer pid=2378895) WARNING 07-03 09:16:55 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2377969) WARNING 07-03 09:17:03 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2378896) WARNING 07-03 09:16:56 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2377969) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2377966) 2026-07-03 09:17:17,609 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2377966) 2026-07-03 09:17:17,623 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2378896) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2378896) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2378896) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2378896) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2377965) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2377588) (TaskRunner pid=2377588) A (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) user (TaskRunner pid=2377588) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2377588) (TaskRunner pid=2377588) A: -0.01 (TaskRunner pid=2377588) B: 1.69 (TaskRunner pid=2377588) C: 2.49 (TaskRunner pid=2377588) D: 0.45 (TaskRunner pid=2377588) Please reason step by step. (TaskRunner pid=2377588) assistant (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) [response] (TaskRunner pid=2377588) The folding stability score of a protein sequence is a measure of how stable the protein is when folded into its native conformation. This score is typically calculated using computational methods that consider factors such as hydrogen bonding, hydrophobic interactions, and the overall energy of the folded structure. (TaskRunner pid=2377588) (TaskRunner pid=2377588) However, the question does not provide any computational method or context for calculating the folding stability score. Therefore, it is not possible to calculate the folding stability score directly from the given sequence without additional information. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given the lack of specific computational context or method, the most reasonable assumption is that the question is testing knowledge of protein folding stability scores and their typical ranges. Folding stability scores are often reported as positive values, where higher values indicate greater stability. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Among the given options, B (1.69) and C (2.49) are positive values, and C is higher than B. Since the question is likely testing the understanding of typical ranges, and higher values represent greater stability, the most reasonable answer is C (2.49). (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) C (TaskRunner pid=2377588) (TaskRunner pid=2377588) [ground_truth] B (TaskRunner pid=2377588) [score] 0.0 (TaskRunner pid=2377588) [acc] 0.0 (TaskRunner pid=2377588) [pred] C (TaskRunner pid=2377588) [incorrect_format] 1 (TaskRunner pid=2377588) [feedback] (TaskRunner pid=2377588) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2377588) step:10 - global_seqlen/min:14163 - global_seqlen/max:26060 - global_seqlen/minmax_diff:11897 - global_seqlen/balanced_min:18712 - global_seqlen/balanced_max:19099 - global_seqlen/mean:18761.375 - actor/entropy:0.47597649693489075 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7415763139724731 - training/rollout_probs_diff_mean:0.005607969127595425 - training/rollout_probs_diff_std:0.012069297954440117 - training/rollout_actor_probs_pearson_corr:0.9991002678871155 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.999903678894043 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.5036615625140257e-05 - rollout_corr/rollout_is_max:1.965978980064392 - rollout_corr/rollout_is_min:0.19742274284362793 - rollout_corr/rollout_is_std:0.03998841717839241 - rollout_corr/rollout_is_eff_sample_size:0.9984030042277898 - rollout_corr/rollout_is_seq_mean:0.9999511241912842 - rollout_corr/rollout_is_seq_std:0.0025766082108020782 - rollout_corr/rollout_is_seq_max:1.008123755455017 - rollout_corr/rollout_is_seq_min:0.9903047680854797 - rollout_corr/rollout_is_seq_max_deviation:0.009695231914520264 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.67899210145697) - rollout_corr/training_log_ppl:np.float64(0.5075996979721822) - rollout_corr/kl:np.float64(0.0008633161575428705) - rollout_corr/k3_kl:np.float64(0.0009114048918377193) - rollout_corr/rollout_ppl:np.float64(1.6775279380381107) - rollout_corr/rollout_log_ppl:np.float64(0.5067363798152655) - rollout_corr/log_ppl_diff:np.float64(0.0008633181569166481) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021968886139802635) - rollout_corr/log_ppl_diff_max:np.float64(0.010740458965301514) - rollout_corr/log_ppl_diff_min:np.float64(-0.008011102676391602) - rollout_corr/ppl_ratio:np.float64(1.000867237802595) - rollout_corr/chi2_token:np.float64(0.001955550629645586) - rollout_corr/chi2_seq:np.float64(0.6110475764144212) - actor/pg_loss:np.float64(0.008315118844620883) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006162434947327711) - actor/ppo_kl:np.float64(-7.371878186646086e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.296875) - self_distillation/token_reweight_w_mean:np.float64(61154.90416962118) - self_distillation/token_reweight_w_std:np.float64(1086673.487870453) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0078934647608548) - self_distillation/token_reweight_w_clip_frac:np.float64(0.2088699692103546) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.5064633563160896) - perf/mfu/actor:np.float64(0.04507108621137307) - perf/max_memory_allocated_gb:np.float64(134.80358123779297) - perf/max_memory_reserved_gb:np.float64(149.30859375) - perf/cpu_memory_used_gb:np.float64(197.69725799560547) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3025) - val-aux/sciknoweval/reward/std@16:np.float64(0.3142083590269627) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.43629999999999997) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.32482317222761753) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.17256) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.20891699967002722) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.30282) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.31435611298503363) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.57296) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.28749051524060554) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.10133999999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10171041584431649) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.28906000000000004) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.26570889420220517) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6938000000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.21517373219431118) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.06924000000000001) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.04829965212448794) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.27432) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.20693061592326475) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7772) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.12898781295420383) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.053) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.02569837977166126) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.26713999999999993) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1582230540434245) - val-aux/sciknoweval/score/mean@16:np.float64(0.3025) - val-aux/sciknoweval/score/std@16:np.float64(0.3142083590269627) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.43629999999999997) - val-aux/sciknoweval/score/best@2/std:np.float64(0.32482317222761753) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.17256) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.20891699967002722) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.30282) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.31435611298503363) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.57296) - val-aux/sciknoweval/score/best@4/std:np.float64(0.28749051524060554) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.10133999999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10171041584431649) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.28906000000000004) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.26570889420220517) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6938000000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.21517373219431118) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.06924000000000001) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.04829965212448794) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.27432) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.20693061592326475) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7772) - val-aux/sciknoweval/score/best@16/std:np.float64(0.12898781295420383) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.053) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.02569837977166126) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.26713999999999993) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1582230540434245) - val-core/sciknoweval/acc/mean@16:np.float64(0.3025) - val-aux/sciknoweval/acc/std@16:np.float64(0.3142083590269627) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.43629999999999997) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.32482317222761753) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.17256) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.20891699967002722) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.30282) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.31435611298503363) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.57296) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.28749051524060554) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.10133999999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10171041584431649) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.28906000000000004) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.26570889420220517) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6938000000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.21517373219431118) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.06924000000000001) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.04829965212448794) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.27432) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.20693061592326475) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7772) - val-core/sciknoweval/acc/best@16/std:np.float64(0.12898781295420383) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.053) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.02569837977166126) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.26713999999999993) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1582230540434245) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.985) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.037604567361781666) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9978) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.013365533374506466) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9729800000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.047065364855350754) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9852199999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03745930750812268) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998800000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0021559110231923074) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.95318) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05405391126243009) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9936400000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.023680801924393105) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9286) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.052901573697869415) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9985) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009538039123408256) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.90492) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04089420171406851) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998600000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0023042059141141357) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021421953570097685 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021421953570097685 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:312.04296875 - response_length/max:1640.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:312.04296875 - response_length_non_aborted/max:1640.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.25 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014005787670612335 - timing_s/agent_loop/generate_sequences/min:np.float64(1.50009729526937) - timing_s/agent_loop/generate_sequences/max:np.float64(11.204129315912724) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.45635859331378) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.204129315912724) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1640 - timing_s/gen:18.32468436099589 - timing_s/reward:0.0471637137234211 - timing_s/old_log_prob:3.045778840780258 - timing_s/adv:0.6256122794002295 - timing_s/update_actor:21.578345684334636 - timing_s/step:43.657363602891564 - timing_s/testing:91.4358235578984 - timing_s/save_checkpoint:12.777142945677042 - timing_s/stop_profile:0.0001415107399225235 - timing_per_token_ms/update_actor:0.1437684183884086 - timing_per_token_ms/gen:0.229394043300776 - timing_per_token_ms/adv:0.00416821980931721 - perf/total_num_tokens:150091 - perf/time_per_step:43.657363602891564 - perf/throughput:429.74136438136577 (TaskRunner pid=2377588) Training Progress: 10%|█ | 10/100 [13:01<2:21:58, 94.65s/it] (TaskRunner pid=2377588) step:11 - global_seqlen/min:19878 - global_seqlen/max:29749 - global_seqlen/minmax_diff:9871 - global_seqlen/balanced_min:22634 - global_seqlen/balanced_max:29409 - global_seqlen/mean:23573.125 - actor/entropy:0.4132035970687866 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3519445061683655 - training/rollout_probs_diff_mean:0.0046004424802958965 - training/rollout_probs_diff_std:0.010705293156206608 - training/rollout_actor_probs_pearson_corr:0.9992732405662537 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.42578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000061988830566 - rollout_corr/rollout_is_ratio_fraction_high:9.613073416403495e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.922614683280699e-05 - rollout_corr/rollout_is_max:2.170545816421509 - rollout_corr/rollout_is_min:0.29609596729278564 - rollout_corr/rollout_is_std:0.03640725463628769 - rollout_corr/rollout_is_eff_sample_size:0.9986762665024709 - rollout_corr/rollout_is_seq_mean:1.0000648498535156 - rollout_corr/rollout_is_seq_std:0.002408467000350356 - rollout_corr/rollout_is_seq_max:1.0085158348083496 - rollout_corr/rollout_is_seq_min:0.9925718903541565 - rollout_corr/rollout_is_seq_max_deviation:0.00851583480834961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6400495143607259) - rollout_corr/training_log_ppl:np.float64(0.48171610439021606) - rollout_corr/kl:np.float64(0.0008449217568031742) - rollout_corr/k3_kl:np.float64(0.0008658878602432196) - rollout_corr/rollout_ppl:np.float64(1.6386342416517437) - rollout_corr/rollout_log_ppl:np.float64(0.4808711838850286) - rollout_corr/log_ppl_diff:np.float64(0.0008449205051874742) - rollout_corr/log_ppl_abs_diff:np.float64(0.001999393556616269) - rollout_corr/log_ppl_diff_max:np.float64(0.008769512176513672) - rollout_corr/log_ppl_diff_min:np.float64(-0.007282614707946777) - rollout_corr/ppl_ratio:np.float64(1.0008485354483128) - rollout_corr/chi2_token:np.float64(0.001798983896151185) - rollout_corr/chi2_seq:np.float64(1.4591931896284223) - actor/pg_loss:np.float64(0.009221029351465404) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006929661744834448) - actor/ppo_kl:np.float64(0.00010937049111703345) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.42578125) - self_distillation/token_reweight_w_mean:np.float64(4804.514788687695) - self_distillation/token_reweight_w_std:np.float64(100706.80984627793) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012406844180077) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19181303342338651) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.4714777320623398) - perf/mfu/actor:np.float64(0.05300926662791261) - perf/max_memory_allocated_gb:np.float64(134.80358123779297) - perf/max_memory_reserved_gb:np.float64(149.30859375) - perf/cpu_memory_used_gb:np.float64(201.26231002807617) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.05954578146338463 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.05954578146338463 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.34765625 - response_length/max:8192.0 - response_length/min:123.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:406.34765625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:330.3125 - prompt_length/max:500.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014281272888183594 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4909711610525846) - timing_s/agent_loop/generate_sequences/max:np.float64(57.758349468931556) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.46330490327091) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(57.758349468931556) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:64.09137350693345 - timing_s/reward:0.056338438764214516 - timing_s/old_log_prob:3.4729202669113874 - timing_s/adv:0.570438215509057 - timing_s/update_actor:23.2205776181072 - timing_s/step:91.51654797978699 - timing_s/stop_profile:2.8973445296287537e-05 - timing_per_token_ms/update_actor:0.12313056509323222 - timing_per_token_ms/gen:0.6161151022055606 - timing_per_token_ms/adv:0.0030248334465045315 - perf/total_num_tokens:188585 - perf/time_per_step:91.51654797978699 - perf/throughput:257.58319692310215 (TaskRunner pid=2377588) Training Progress: 11%|█ | 11/100 [14:33<2:18:58, 93.70s/it] (TaskRunner pid=2377588) step:12 - global_seqlen/min:18385 - global_seqlen/max:28317 - global_seqlen/minmax_diff:9932 - global_seqlen/balanced_min:23826 - global_seqlen/balanced_max:24194 - global_seqlen/mean:23876.125 - actor/entropy:0.46063926815986633 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.32935598492622375 - training/rollout_probs_diff_mean:0.004924152046442032 - training/rollout_probs_diff_std:0.010741683654487133 - training/rollout_actor_probs_pearson_corr:0.9992657899856567 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9998393058776855 - rollout_corr/rollout_is_ratio_fraction_high:1.014270765153924e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0428123864112422e-05 - rollout_corr/rollout_is_max:2.115323066711426 - rollout_corr/rollout_is_min:0.31808769702911377 - rollout_corr/rollout_is_std:0.03753577917814255 - rollout_corr/rollout_is_eff_sample_size:0.9985926910884324 - rollout_corr/rollout_is_seq_mean:0.9997738003730774 - rollout_corr/rollout_is_seq_std:0.002363481791689992 - rollout_corr/rollout_is_seq_max:1.0073082447052002 - rollout_corr/rollout_is_seq_min:0.9929303526878357 - rollout_corr/rollout_is_seq_max_deviation:0.007308244705200195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.667306321207434) - rollout_corr/training_log_ppl:np.float64(0.4998956560739316) - rollout_corr/kl:np.float64(0.0008017755705127527) - rollout_corr/k3_kl:np.float64(0.0007993293274353164) - rollout_corr/rollout_ppl:np.float64(1.6659347931854427) - rollout_corr/rollout_log_ppl:np.float64(0.4990938797709532) - rollout_corr/log_ppl_diff:np.float64(0.0008017763029783964) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019694457296282053) - rollout_corr/log_ppl_diff_max:np.float64(0.007415413856506348) - rollout_corr/log_ppl_diff_min:np.float64(-0.006106704473495483) - rollout_corr/ppl_ratio:np.float64(1.0008050745818764) - rollout_corr/chi2_token:np.float64(0.0016002452466636896) - rollout_corr/chi2_seq:np.float64(1.5330298303160816) - actor/pg_loss:np.float64(0.01018837618175894) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005670921141245344) - actor/ppo_kl:np.float64(-0.0002125074469123689) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(41117.01144917286) - self_distillation/token_reweight_w_std:np.float64(729604.6371962249) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022824390325695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.20069699187297374) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.49005672335624695) - perf/mfu/actor:np.float64(0.05639098111809849) - perf/max_memory_allocated_gb:np.float64(134.80358123779297) - perf/max_memory_reserved_gb:np.float64(149.30859375) - perf/cpu_memory_used_gb:np.float64(201.36243438720703) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004437434836290777 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004437434836290777 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:385.12890625 - response_length/max:1905.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:385.12890625 - response_length_non_aborted/max:1905.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:361.0 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7008354663848877e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9910570811480284) - timing_s/agent_loop/generate_sequences/max:np.float64(13.448075471445918) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.258809796177957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.448075471445918) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:1905 - timing_s/gen:20.14148236066103 - timing_s/reward:0.061461759731173515 - timing_s/old_log_prob:3.1489084251224995 - timing_s/adv:0.6428592801094055 - timing_s/update_actor:21.851633308455348 - timing_s/step:45.93478007800877 - timing_s/stop_profile:0.00011536665260791779 - timing_per_token_ms/update_actor:0.11440106648616216 - timing_per_token_ms/gen:0.2042891722603129 - timing_per_token_ms/adv:0.003365596804911839 - perf/total_num_tokens:191009 - perf/time_per_step:45.93478007800877 - perf/throughput:519.7831568901028 (TaskRunner pid=2377588) Training Progress: 12%|█▏ | 12/100 [15:19<1:56:07, 79.18s/it] (TaskRunner pid=2377588) step:13 - global_seqlen/min:16421 - global_seqlen/max:25900 - global_seqlen/minmax_diff:9479 - global_seqlen/balanced_min:20704 - global_seqlen/balanced_max:20711 - global_seqlen/mean:20708.875 - actor/entropy:0.4776025414466858 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135509133338928 - training/rollout_probs_diff_mean:0.005327445454895496 - training/rollout_probs_diff_std:0.011302385479211807 - training/rollout_actor_probs_pearson_corr:0.9992084503173828 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.3203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:0.9999509453773499 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1463814189482946e-05 - rollout_corr/rollout_is_max:1.5783871412277222 - rollout_corr/rollout_is_min:0.3131815791130066 - rollout_corr/rollout_is_std:0.03857112675905228 - rollout_corr/rollout_is_eff_sample_size:0.9985144781247381 - rollout_corr/rollout_is_seq_mean:0.9999363422393799 - rollout_corr/rollout_is_seq_std:0.002374144969508052 - rollout_corr/rollout_is_seq_max:1.0061818361282349 - rollout_corr/rollout_is_seq_min:0.9920371770858765 - rollout_corr/rollout_is_seq_max_deviation:0.007962822914123535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6632360299117863) - rollout_corr/training_log_ppl:np.float64(0.4970624519046396) - rollout_corr/kl:np.float64(0.0008047310386665174) - rollout_corr/k3_kl:np.float64(0.0008303326969780755) - rollout_corr/rollout_ppl:np.float64(1.6619074558839202) - rollout_corr/rollout_log_ppl:np.float64(0.4962577171390876) - rollout_corr/log_ppl_diff:np.float64(0.0008047347655519843) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018828000174835324) - rollout_corr/log_ppl_diff_max:np.float64(0.008822381496429443) - rollout_corr/log_ppl_diff_min:np.float64(-0.005275368690490723) - rollout_corr/ppl_ratio:np.float64(1.000807830831036) - rollout_corr/chi2_token:np.float64(0.0017302141059190035) - rollout_corr/chi2_seq:np.float64(0.6168705336749554) - actor/pg_loss:np.float64(0.00849684129934758) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007973982387738943) - actor/ppo_kl:np.float64(-6.118477535199673e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3203125) - self_distillation/token_reweight_w_mean:np.float64(35265.86888537393) - self_distillation/token_reweight_w_std:np.float64(622361.6682741044) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.006797220557928) - self_distillation/token_reweight_w_clip_frac:np.float64(0.21815393256838433) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.5196256712079048) - perf/mfu/actor:np.float64(0.05029604712876614) - perf/max_memory_allocated_gb:np.float64(134.80358123779297) - perf/max_memory_reserved_gb:np.float64(149.30859375) - perf/cpu_memory_used_gb:np.float64(201.29859161376953) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.3203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011776203289628029 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011776203289628029 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:340.74609375 - response_length/max:963.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:340.74609375 - response_length_non_aborted/max:963.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010397285223007202 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6881712302565575) - timing_s/agent_loop/generate_sequences/max:np.float64(7.84628788754344) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.976413582735404) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.84628788754344) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:963 - timing_s/gen:14.7631844766438 - timing_s/reward:0.057788727805018425 - timing_s/old_log_prob:3.224893044680357 - timing_s/adv:0.5577129703015089 - timing_s/update_actor:21.260457810014486 - timing_s/step:39.951109854504466 - timing_s/stop_profile:0.000125763937830925 - timing_per_token_ms/update_actor:0.1283293866157293 - timing_per_token_ms/gen:0.16924240782111635 - timing_per_token_ms/adv:0.0033663886274695565 - perf/total_num_tokens:165671 - perf/time_per_step:39.951109854504466 - perf/throughput:518.355436818111 (TaskRunner pid=2377588) Training Progress: 13%|█▎ | 13/100 [15:59<1:37:35, 67.31s/it] (TaskRunner pid=2377588) step:14 - global_seqlen/min:15490 - global_seqlen/max:33772 - global_seqlen/minmax_diff:18282 - global_seqlen/balanced_min:21903 - global_seqlen/balanced_max:21936 - global_seqlen/mean:21923.375 - actor/entropy:0.44397908449172974 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23033642768859863 - training/rollout_probs_diff_mean:0.004825624171644449 - training/rollout_probs_diff_std:0.010725203901529312 - training/rollout_actor_probs_pearson_corr:0.9992879629135132 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.29296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.000025987625122 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0109690265380777e-05 - rollout_corr/rollout_is_max:1.7586177587509155 - rollout_corr/rollout_is_min:0.14063116908073425 - rollout_corr/rollout_is_std:0.03678187355399132 - rollout_corr/rollout_is_eff_sample_size:0.9986489216551941 - rollout_corr/rollout_is_seq_mean:0.9999697804450989 - rollout_corr/rollout_is_seq_std:0.0023032512981444597 - rollout_corr/rollout_is_seq_max:1.0063151121139526 - rollout_corr/rollout_is_seq_min:0.9938376545906067 - rollout_corr/rollout_is_seq_max_deviation:0.006315112113952637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6474613510072231) - rollout_corr/training_log_ppl:np.float64(0.4857261731522158) - rollout_corr/kl:np.float64(0.0007369806717463234) - rollout_corr/k3_kl:np.float64(0.000849582680984895) - rollout_corr/rollout_ppl:np.float64(1.6462675756774843) - rollout_corr/rollout_log_ppl:np.float64(0.484989188000327) - rollout_corr/log_ppl_diff:np.float64(0.0007369851518888026) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020617938425857574) - rollout_corr/log_ppl_diff_max:np.float64(0.008909136056900024) - rollout_corr/log_ppl_diff_min:np.float64(-0.0077390968799591064) - rollout_corr/ppl_ratio:np.float64(1.0007406633812934) - rollout_corr/chi2_token:np.float64(0.0019566912669688463) - rollout_corr/chi2_seq:np.float64(1.2719929860904813) - actor/pg_loss:np.float64(0.008388595771975815) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00056596449621793) - actor/ppo_kl:np.float64(-7.312906656409268e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.29296875) - self_distillation/token_reweight_w_mean:np.float64(34162.489921058295) - self_distillation/token_reweight_w_std:np.float64(530391.0706189828) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.006006008014083) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19466896940139122) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.43929925560951233) - perf/mfu/actor:np.float64(0.052392346982771475) - perf/max_memory_allocated_gb:np.float64(134.80358123779297) - perf/max_memory_reserved_gb:np.float64(149.30859375) - perf/cpu_memory_used_gb:np.float64(201.10105514526367) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.29296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.29296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008782793302088976 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008782793302088976 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:386.38671875 - response_length/max:1619.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:386.38671875 - response_length_non_aborted/max:1619.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.07406210899353e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2059249598532915) - timing_s/agent_loop/generate_sequences/max:np.float64(11.64221527427435) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.356408068240853) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.64221527427435) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1619 - timing_s/gen:18.54988630861044 - timing_s/reward:0.062196334823966026 - timing_s/old_log_prob:3.0879969354718924 - timing_s/adv:0.633884409442544 - timing_s/update_actor:21.664242211729288 - timing_s/step:44.087370770052075 - timing_s/stop_profile:0.00010524503886699677 - timing_per_token_ms/update_actor:0.12352250857662933 - timing_per_token_ms/gen:0.18753360267512956 - timing_per_token_ms/adv:0.0036142040712398523 - perf/total_num_tokens:175387 - perf/time_per_step:44.087370770052075 - perf/throughput:497.271091858629 (TaskRunner pid=2377588) Training Progress: 14%|█▍ | 14/100 [16:43<1:26:27, 60.32s/it] (TaskRunner pid=2377588) step:15 - global_seqlen/min:17084 - global_seqlen/max:25266 - global_seqlen/minmax_diff:8182 - global_seqlen/balanced_min:19670 - global_seqlen/balanced_max:19687 - global_seqlen/mean:19681.5 - actor/entropy:0.5048806071281433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2852863371372223 - training/rollout_probs_diff_mean:0.005556733813136816 - training/rollout_probs_diff_std:0.0113645875826478 - training/rollout_actor_probs_pearson_corr:0.9991916418075562 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.75390625 - self_distillation/correct_sample_fraction:0.3671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75390625 - rollout_corr/rollout_is_mean:1.0000077486038208 - rollout_corr/rollout_is_ratio_fraction_high:1.2108296687074471e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.285754442214966 - rollout_corr/rollout_is_min:0.5329305529594421 - rollout_corr/rollout_is_std:0.03878533095121384 - rollout_corr/rollout_is_eff_sample_size:0.9984979575007318 - rollout_corr/rollout_is_seq_mean:1.0000890493392944 - rollout_corr/rollout_is_seq_std:0.0022774795070290565 - rollout_corr/rollout_is_seq_max:1.007437825202942 - rollout_corr/rollout_is_seq_min:0.9928779602050781 - rollout_corr/rollout_is_seq_max_deviation:0.0074378252029418945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7272440576925874) - rollout_corr/training_log_ppl:np.float64(0.5358740043593571) - rollout_corr/kl:np.float64(0.0008740120214412173) - rollout_corr/k3_kl:np.float64(0.0008694112475495785) - rollout_corr/rollout_ppl:np.float64(1.7256188867613673) - rollout_corr/rollout_log_ppl:np.float64(0.5349999887403101) - rollout_corr/log_ppl_diff:np.float64(0.0008740156190469861) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019992778543382883) - rollout_corr/log_ppl_diff_max:np.float64(0.008453011512756348) - rollout_corr/log_ppl_diff_min:np.float64(-0.00622972846031189) - rollout_corr/ppl_ratio:np.float64(1.000877300510183) - rollout_corr/chi2_token:np.float64(0.0017548007890582085) - rollout_corr/chi2_seq:np.float64(0.40972969960421324) - actor/pg_loss:np.float64(0.007485751644708216) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006225892097972974) - actor/ppo_kl:np.float64(0.000151493788763446) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75390625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3671875) - self_distillation/token_reweight_w_mean:np.float64(30712.322506404947) - self_distillation/token_reweight_w_std:np.float64(526423.1995366343) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.004842618945986) - self_distillation/token_reweight_w_clip_frac:np.float64(0.187502094428055) - self_distillation/empty_target_batch:np.float64(0.24609375) - actor/grad_norm:np.float64(0.46229737251996994) - perf/mfu/actor:np.float64(0.04804320165645968) - perf/max_memory_allocated_gb:np.float64(134.80358123779297) - perf/max_memory_reserved_gb:np.float64(149.30859375) - perf/cpu_memory_used_gb:np.float64(202.12303161621094) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:1 - critic/score/mean:0.3671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0032677266281098127 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0032677266281098127 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:322.609375 - response_length/max:1081.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.609375 - response_length_non_aborted/max:1081.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005458574742078781 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9119639489799738) - timing_s/agent_loop/generate_sequences/max:np.float64(8.773982778191566) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8003880018222844) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.773982778191566) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1081 - timing_s/gen:15.666535932570696 - timing_s/reward:0.05560224689543247 - timing_s/old_log_prob:3.091739386320114 - timing_s/adv:0.6229269485920668 - timing_s/update_actor:20.891464350745082 - timing_s/step:40.4376270826906 - timing_s/stop_profile:8.308514952659607e-05 - timing_per_token_ms/update_actor:0.1326846553282593 - timing_per_token_ms/gen:0.18969506384184986 - timing_per_token_ms/adv:0.003956297465843982 - perf/total_num_tokens:157452 - perf/time_per_step:40.4376270826906 - perf/throughput:486.71253532640407 (TaskRunner pid=2377588) Training Progress: 15%|█▌ | 15/100 [17:24<1:17:07, 54.44s/it] (TaskRunner pid=2377588) step:16 - global_seqlen/min:18817 - global_seqlen/max:25215 - global_seqlen/minmax_diff:6398 - global_seqlen/balanced_min:21509 - global_seqlen/balanced_max:22269 - global_seqlen/mean:21608.25 - actor/entropy:0.48058828711509705 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23161542415618896 - training/rollout_probs_diff_mean:0.005330477841198444 - training/rollout_probs_diff_std:0.011212863959372044 - training/rollout_actor_probs_pearson_corr:0.999249279499054 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.40234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000536441802979 - rollout_corr/rollout_is_ratio_fraction_high:1.1268113667028956e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1268113667028956e-05 - rollout_corr/rollout_is_max:2.335620880126953 - rollout_corr/rollout_is_min:0.49919837713241577 - rollout_corr/rollout_is_std:0.03880991414189339 - rollout_corr/rollout_is_eff_sample_size:0.9984961747322598 - rollout_corr/rollout_is_seq_mean:1.000025749206543 - rollout_corr/rollout_is_seq_std:0.0025925845839083195 - rollout_corr/rollout_is_seq_max:1.0084913969039917 - rollout_corr/rollout_is_seq_min:0.9928736686706543 - rollout_corr/rollout_is_seq_max_deviation:0.0084913969039917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6833268832415342) - rollout_corr/training_log_ppl:np.float64(0.5120809821528383) - rollout_corr/kl:np.float64(0.0009129491295354342) - rollout_corr/k3_kl:np.float64(0.0009237226916525287) - rollout_corr/rollout_ppl:np.float64(1.6817396972328424) - rollout_corr/rollout_log_ppl:np.float64(0.5111680260451976) - rollout_corr/log_ppl_diff:np.float64(0.000912956107640639) - rollout_corr/log_ppl_abs_diff:np.float64(0.002133499045157805) - rollout_corr/log_ppl_diff_max:np.float64(0.009822428226470947) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058751702308654785) - rollout_corr/ppl_ratio:np.float64(1.0009168873075396) - rollout_corr/chi2_token:np.float64(0.001953057013452053) - rollout_corr/chi2_seq:np.float64(0.9934253571555018) - actor/pg_loss:np.float64(0.009466712712310255) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008731100817840343) - actor/ppo_kl:np.float64(0.00011994022706218743) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40234375) - self_distillation/token_reweight_w_mean:np.float64(42795.93652941706) - self_distillation/token_reweight_w_std:np.float64(710707.7772537603) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0035682215820998) - self_distillation/token_reweight_w_clip_frac:np.float64(0.2260787099803565) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5287577584385872) - perf/mfu/actor:np.float64(0.05175481424415243) - perf/max_memory_allocated_gb:np.float64(134.80358123779297) - perf/max_memory_reserved_gb:np.float64(149.30859375) - perf/cpu_memory_used_gb:np.float64(201.87517929077148) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:1 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.019586797803640366 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.019586797803640366 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:346.6640625 - response_length/max:2098.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:346.6640625 - response_length_non_aborted/max:2098.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.59375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.221715688705444e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.881571490317583) - timing_s/agent_loop/generate_sequences/max:np.float64(14.497364845126867) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.013453907864459) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.497364845126867) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:2098 - timing_s/gen:21.813503231853247 - timing_s/reward:0.06134501099586487 - timing_s/old_log_prob:3.068139998242259 - timing_s/adv:0.5708373822271824 - timing_s/update_actor:21.570074083283544 - timing_s/step:47.17207078076899 - timing_s/stop_profile:2.8120353817939758e-05 - timing_per_token_ms/update_actor:0.12477915890506834 - timing_per_token_ms/gen:0.2457970300842094 - timing_per_token_ms/adv:0.003302195817726924 - perf/total_num_tokens:172866 - perf/time_per_step:47.17207078076899 - perf/throughput:458.0729580523144 (TaskRunner pid=2377588) Training Progress: 16%|█▌ | 16/100 [18:11<1:13:09, 52.26s/it] (TaskRunner pid=2377588) step:17 - global_seqlen/min:14575 - global_seqlen/max:23493 - global_seqlen/minmax_diff:8918 - global_seqlen/balanced_min:20065 - global_seqlen/balanced_max:20289 - global_seqlen/mean:20094.625 - actor/entropy:0.5044421553611755 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17137330770492554 - training/rollout_probs_diff_mean:0.0054691084660589695 - training/rollout_probs_diff_std:0.011116547510027885 - training/rollout_actor_probs_pearson_corr:0.9992220997810364 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.75390625 - self_distillation/correct_sample_fraction:0.30078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75390625 - rollout_corr/rollout_is_mean:1.0001300573349 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.571394443511963 - rollout_corr/rollout_is_min:0.5040745139122009 - rollout_corr/rollout_is_std:0.03839610889554024 - rollout_corr/rollout_is_eff_sample_size:0.9985283843974703 - rollout_corr/rollout_is_seq_mean:1.000121831893921 - rollout_corr/rollout_is_seq_std:0.0024755955673754215 - rollout_corr/rollout_is_seq_max:1.0072437524795532 - rollout_corr/rollout_is_seq_min:0.993012011051178 - rollout_corr/rollout_is_seq_max_deviation:0.007243752479553223 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.7106747217476368) - rollout_corr/training_log_ppl:np.float64(0.5281793175381608) - rollout_corr/kl:np.float64(0.0008961270885556871) - rollout_corr/k3_kl:np.float64(0.0009057738034243812) - rollout_corr/rollout_ppl:np.float64(1.7091461159288883) - rollout_corr/rollout_log_ppl:np.float64(0.5272831902257167) - rollout_corr/log_ppl_diff:np.float64(0.0008961273124441504) - rollout_corr/log_ppl_abs_diff:np.float64(0.002192425075918436) - rollout_corr/log_ppl_diff_max:np.float64(0.010523557662963867) - rollout_corr/log_ppl_diff_min:np.float64(-0.005999863147735596) - rollout_corr/ppl_ratio:np.float64(1.0009001134894788) - rollout_corr/chi2_token:np.float64(0.001813844544813037) - rollout_corr/chi2_seq:np.float64(1.0275559448637068) - actor/pg_loss:np.float64(0.008227652637287974) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009567388847244729) - actor/ppo_kl:np.float64(0.00021358127866477616) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75390625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.30078125) - self_distillation/token_reweight_w_mean:np.float64(20752.148633872624) - self_distillation/token_reweight_w_std:np.float64(356833.07592836407) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0058617040049285) - self_distillation/token_reweight_w_clip_frac:np.float64(0.2114278933731839) - self_distillation/empty_target_batch:np.float64(0.24609375) - actor/grad_norm:np.float64(0.48358816653490067) - perf/mfu/actor:np.float64(0.047982434727945195) - perf/max_memory_allocated_gb:np.float64(134.80358123779297) - perf/max_memory_reserved_gb:np.float64(149.30859375) - perf/cpu_memory_used_gb:np.float64(202.05701446533203) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:1 - critic/score/mean:0.30078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0031041069887578487 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0031041069887578487 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:339.45703125 - response_length/max:1479.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.45703125 - response_length_non_aborted/max:1479.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.717924356460571e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7589580323547125) - timing_s/agent_loop/generate_sequences/max:np.float64(10.715008419007063) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8807519262409187) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.715008419007063) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1479 - timing_s/gen:18.08533625677228 - timing_s/reward:0.05921829678118229 - timing_s/old_log_prob:3.1184179466217756 - timing_s/adv:0.616282070055604 - timing_s/update_actor:21.76741582341492 - timing_s/step:43.69610506482422 - timing_s/stop_profile:3.9730221033096313e-05 - timing_per_token_ms/update_actor:0.13540571062793486 - timing_per_token_ms/gen:0.20811424790016547 - timing_per_token_ms/adv:0.00383362509909742 - perf/total_num_tokens:160757 - perf/time_per_step:43.69610506482422 - perf/throughput:459.8722236270061 (TaskRunner pid=2377588) Training Progress: 17%|█▋ | 17/100 [18:55<1:08:44, 49.69s/it] (TaskRunner pid=2377588) step:18 - global_seqlen/min:14736 - global_seqlen/max:25306 - global_seqlen/minmax_diff:10570 - global_seqlen/balanced_min:20384 - global_seqlen/balanced_max:20958 - global_seqlen/mean:20460.25 - actor/entropy:0.5144548416137695 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3014741837978363 - training/rollout_probs_diff_mean:0.0057943398132920265 - training/rollout_probs_diff_std:0.011700835078954697 - training/rollout_actor_probs_pearson_corr:0.999168336391449 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.4296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:0.9999101758003235 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9397753477096558 - rollout_corr/rollout_is_min:0.5967299342155457 - rollout_corr/rollout_is_std:0.03979267552495003 - rollout_corr/rollout_is_eff_sample_size:0.9984186898811149 - rollout_corr/rollout_is_seq_mean:0.9998713731765747 - rollout_corr/rollout_is_seq_std:0.0022981511428952217 - rollout_corr/rollout_is_seq_max:1.0065547227859497 - rollout_corr/rollout_is_seq_min:0.994442343711853 - rollout_corr/rollout_is_seq_max_deviation:0.006554722785949707 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.712197933346033) - rollout_corr/training_log_ppl:np.float64(0.5284221491892822) - rollout_corr/kl:np.float64(0.0008549049711259471) - rollout_corr/k3_kl:np.float64(0.0009556110655921657) - rollout_corr/rollout_ppl:np.float64(1.7106376010924578) - rollout_corr/rollout_log_ppl:np.float64(0.5275672430871055) - rollout_corr/log_ppl_diff:np.float64(0.0008549061021767557) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021137886797077954) - rollout_corr/log_ppl_diff_max:np.float64(0.008665680885314941) - rollout_corr/log_ppl_diff_min:np.float64(-0.007487297058105469) - rollout_corr/ppl_ratio:np.float64(1.0008588274940848) - rollout_corr/chi2_token:np.float64(0.0020940450485795736) - rollout_corr/chi2_seq:np.float64(1.2327839243225753) - actor/pg_loss:np.float64(0.009695283486507833) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009595171841283445) - actor/ppo_kl:np.float64(-0.00010468710715372254) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4296875) - self_distillation/token_reweight_w_mean:np.float64(66621.74735869002) - self_distillation/token_reweight_w_std:np.float64(1074925.2043313854) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0039839667733759) - self_distillation/token_reweight_w_clip_frac:np.float64(0.2095157642615959) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.4879278242588043) - perf/mfu/actor:np.float64(0.049070913235495076) - perf/max_memory_allocated_gb:np.float64(134.80358123779297) - perf/max_memory_reserved_gb:np.float64(149.30859375) - perf/cpu_memory_used_gb:np.float64(201.9274444580078) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:1 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003212119685485959 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003212119685485959 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.9140625 - response_length/max:1660.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:325.9140625 - response_length_non_aborted/max:1660.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.46875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012042000889778137 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9431864209473133) - timing_s/agent_loop/generate_sequences/max:np.float64(11.884437868371606) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.755999008419167) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.884437868371606) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1660 - timing_s/gen:18.753878876566887 - timing_s/reward:0.057517144829034805 - timing_s/old_log_prob:3.012032635509968 - timing_s/adv:0.5344807077199221 - timing_s/update_actor:21.54125608690083 - timing_s/step:43.98590915463865 - timing_s/stop_profile:0.00011321529746055603 - timing_per_token_ms/update_actor:0.13160430644115315 - timing_per_token_ms/gen:0.22477501829670024 - timing_per_token_ms/adv:0.003265360318910583 - perf/total_num_tokens:163682 - perf/time_per_step:43.98590915463865 - perf/throughput:465.15464595875727 (TaskRunner pid=2377588) Training Progress: 18%|█▊ | 18/100 [19:39<1:05:35, 47.99s/it] (TaskRunner pid=2377588) step:19 - global_seqlen/min:15635 - global_seqlen/max:25097 - global_seqlen/minmax_diff:9462 - global_seqlen/balanced_min:20058 - global_seqlen/balanced_max:20486 - global_seqlen/mean:20113.5 - actor/entropy:0.47090667486190796 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21561288833618164 - training/rollout_probs_diff_mean:0.0057164705358445644 - training/rollout_probs_diff_std:0.011928950436413288 - training/rollout_actor_probs_pearson_corr:0.9991148710250854 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.3828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:0.9999209642410278 - rollout_corr/rollout_is_ratio_fraction_high:1.1814186109404545e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.091331720352173 - rollout_corr/rollout_is_min:0.6094146966934204 - rollout_corr/rollout_is_std:0.039958592504262924 - rollout_corr/rollout_is_eff_sample_size:0.9984056184691176 - rollout_corr/rollout_is_seq_mean:0.9998630285263062 - rollout_corr/rollout_is_seq_std:0.002485917881131172 - rollout_corr/rollout_is_seq_max:1.0072880983352661 - rollout_corr/rollout_is_seq_min:0.9917079210281372 - rollout_corr/rollout_is_seq_max_deviation:0.008292078971862793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.644785768352449) - rollout_corr/training_log_ppl:np.float64(0.48851912398822606) - rollout_corr/kl:np.float64(0.0009973215278713532) - rollout_corr/k3_kl:np.float64(0.0009430778742398616) - rollout_corr/rollout_ppl:np.float64(1.6430969191715121) - rollout_corr/rollout_log_ppl:np.float64(0.4875218014931306) - rollout_corr/log_ppl_diff:np.float64(0.0009973224950954318) - rollout_corr/log_ppl_abs_diff:np.float64(0.002121600671671331) - rollout_corr/log_ppl_diff_max:np.float64(0.009614884853363037) - rollout_corr/log_ppl_diff_min:np.float64(-0.00788038969039917) - rollout_corr/ppl_ratio:np.float64(1.0010012907441705) - rollout_corr/chi2_token:np.float64(0.001794034382328391) - rollout_corr/chi2_seq:np.float64(0.8907242745626718) - actor/pg_loss:np.float64(0.009201137232594192) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008604836850736319) - actor/ppo_kl:np.float64(-4.0184713014923545e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3828125) - self_distillation/token_reweight_w_mean:np.float64(32568.39766512066) - self_distillation/token_reweight_w_std:np.float64(623905.8605569799) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0034110613632947) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1873939734941814) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.5041789636015892) - perf/mfu/actor:np.float64(0.04822362737420109) - perf/max_memory_allocated_gb:np.float64(134.80358123779297) - perf/max_memory_reserved_gb:np.float64(149.30859375) - perf/cpu_memory_used_gb:np.float64(201.6483497619629) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:1 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.02114739455282688 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.02114739455282688 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:330.640625 - response_length/max:1616.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:330.640625 - response_length_non_aborted/max:1616.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.90625 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.167698979377747e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2575477976351976) - timing_s/agent_loop/generate_sequences/max:np.float64(11.33365480788052) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.440710900191334) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.33365480788052) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1616 - timing_s/gen:18.463607147336006 - timing_s/reward:0.06573698297142982 - timing_s/old_log_prob:3.1804314330220222 - timing_s/adv:0.6017686352133751 - timing_s/update_actor:21.50344476290047 - timing_s/step:43.9161685295403 - timing_s/stop_profile:0.00010671652853488922 - timing_per_token_ms/update_actor:0.13363813336130256 - timing_per_token_ms/gen:0.21813249784197353 - timing_per_token_ms/adv:0.003739830432379839 - perf/total_num_tokens:160908 - perf/time_per_step:43.9161685295403 - perf/throughput:457.99760483364145 (TaskRunner pid=2377588) Training Progress: 19%|█▉ | 19/100 [20:23<1:03:09, 46.78s/it] (TaskRunner pid=2377588) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2377588) (TaskRunner pid=2377588) validation generation end (TaskRunner pid=2377588) [prompt] system (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2377588) (TaskRunner pid=2377588) A (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) user (TaskRunner pid=2377588) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2377588) (TaskRunner pid=2377588) A: -0.01 (TaskRunner pid=2377588) B: 1.69 (TaskRunner pid=2377588) C: 2.49 (TaskRunner pid=2377588) D: 0.45 (TaskRunner pid=2377588) Please reason step by step. (TaskRunner pid=2377588) assistant (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) [response] (TaskRunner pid=2377588) The folding stability score of a protein sequence is typically determined through computational methods such as energy minimization, molecular dynamics simulations, or machine learning models trained on known protein structures. These methods evaluate the likelihood of a protein folding into a stable, functional conformation based on factors like hydrogen bonding, hydrophobic interactions, and secondary structure formation. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given that the question provides a specific protein sequence and asks for a folding stability score, we must infer the correct answer based on typical values for such scores. Folding stability scores are often reported on a scale where negative values indicate instability and positive values indicate greater stability. Among the given options, the most reasonable and commonly observed value for a protein sequence with a moderate level of stability would be **1.69**. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Therefore, the correct answer is **B**. (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) B (TaskRunner pid=2377588) (TaskRunner pid=2377588) [ground_truth] B (TaskRunner pid=2377588) [score] 1.0 (TaskRunner pid=2377588) [acc] 1.0 (TaskRunner pid=2377588) [pred] B (TaskRunner pid=2377588) [incorrect_format] 1 (TaskRunner pid=2377588) [feedback] (TaskRunner pid=2377588) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2377588) step:20 - global_seqlen/min:14412 - global_seqlen/max:30620 - global_seqlen/minmax_diff:16208 - global_seqlen/balanced_min:21182 - global_seqlen/balanced_max:27867 - global_seqlen/mean:22595.875 - actor/entropy:0.42691105604171753 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21057669818401337 - training/rollout_probs_diff_mean:0.00484515680000186 - training/rollout_probs_diff_std:0.010859712958335876 - training/rollout_actor_probs_pearson_corr:0.999250054359436 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.37890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9997731447219849 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9884075300069526e-05 - rollout_corr/rollout_is_max:1.6617965698242188 - rollout_corr/rollout_is_min:0.2997890114784241 - rollout_corr/rollout_is_std:0.036396611481904984 - rollout_corr/rollout_is_eff_sample_size:0.9986763853963808 - rollout_corr/rollout_is_seq_mean:0.9996659755706787 - rollout_corr/rollout_is_seq_std:0.0021870434284210205 - rollout_corr/rollout_is_seq_max:1.0057438611984253 - rollout_corr/rollout_is_seq_min:0.9915841221809387 - rollout_corr/rollout_is_seq_max_deviation:0.00841587781906128 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6964317760430276) - rollout_corr/training_log_ppl:np.float64(0.5197410888431477) - rollout_corr/kl:np.float64(0.0009075633603430333) - rollout_corr/k3_kl:np.float64(0.0008967669461696914) - rollout_corr/rollout_ppl:np.float64(1.6948677045293152) - rollout_corr/rollout_log_ppl:np.float64(0.5188335263665067) - rollout_corr/log_ppl_diff:np.float64(0.0009075624766410328) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020453127726796083) - rollout_corr/log_ppl_diff_max:np.float64(0.010220468044281006) - rollout_corr/log_ppl_diff_min:np.float64(-0.005238473415374756) - rollout_corr/ppl_ratio:np.float64(1.0009110185783356) - rollout_corr/chi2_token:np.float64(0.0018275347538292408) - rollout_corr/chi2_seq:np.float64(0.5847274619154632) - actor/pg_loss:np.float64(0.010421653045341372) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007143971079131006) - actor/ppo_kl:np.float64(-0.00023351565544871278) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37890625) - self_distillation/token_reweight_w_mean:np.float64(22940.386835711543) - self_distillation/token_reweight_w_std:np.float64(348380.7681167661) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0041218136902899) - self_distillation/token_reweight_w_clip_frac:np.float64(0.22477550763142062) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5363280698657036) - perf/mfu/actor:np.float64(0.050816591898341784) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(201.65056610107422) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3675) - val-aux/sciknoweval/reward/std@16:np.float64(0.34553995156018635) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.51764) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.3286702215225457) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.21756) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.25970357156636786) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.36713999999999997) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.34526391655636607) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.65952) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.25721954586378204) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.1179) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14900443185185872) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.37262000000000006) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.3054085656915063) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7614799999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.16519717706692746) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.0701) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06481774221488329) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.37366) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.252794235449463) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8232) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.08455870021905877) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.04985999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.025416520357219453) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.3764) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.20104623246709427) - val-aux/sciknoweval/score/mean@16:np.float64(0.3675) - val-aux/sciknoweval/score/std@16:np.float64(0.34553995156018635) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.51764) - val-aux/sciknoweval/score/best@2/std:np.float64(0.3286702215225457) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.21756) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.25970357156636786) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.36713999999999997) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.34526391655636607) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.65952) - val-aux/sciknoweval/score/best@4/std:np.float64(0.25721954586378204) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.1179) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14900443185185872) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.37262000000000006) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.3054085656915063) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7614799999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.16519717706692746) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.0701) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06481774221488329) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.37366) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.252794235449463) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8232) - val-aux/sciknoweval/score/best@16/std:np.float64(0.08455870021905877) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.04985999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.025416520357219453) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.3764) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.20104623246709427) - val-core/sciknoweval/acc/mean@16:np.float64(0.3675) - val-aux/sciknoweval/acc/std@16:np.float64(0.34553995156018635) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.51764) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.3286702215225457) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.21756) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.25970357156636786) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.36713999999999997) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.34526391655636607) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.65952) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.25721954586378204) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.1179) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14900443185185872) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.37262000000000006) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.3054085656915063) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7614799999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.16519717706692746) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.0701) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06481774221488329) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.37366) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.252794235449463) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8232) - val-core/sciknoweval/acc/best@16/std:np.float64(0.08455870021905877) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.04985999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.025416520357219453) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.3764) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.20104623246709427) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.028025170768881474) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99824) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010963856324053802) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.98212) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03559319728466575) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9905799999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.026909865509666977) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.99994) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0010938007131100255) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.96926) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.042336219101766) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99714) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.014954189754174105) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9505600000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0452524973135319) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99944) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.006943139599575737) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9287799999999998) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03976527224638606) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711646) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:1 - critic/score/mean:0.37890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06899774074554443 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06899774074554443 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:392.90234375 - response_length/max:8192.0 - response_length/min:153.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:392.90234375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:313.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.606033563613892e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9240788705646992) - timing_s/agent_loop/generate_sequences/max:np.float64(58.68176590651274) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.320921234764683) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.68176590651274) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:65.48318416252732 - timing_s/reward:0.06254185922443867 - timing_s/old_log_prob:3.3231370579451323 - timing_s/adv:0.5750339459627867 - timing_s/update_actor:23.25154311209917 - timing_s/step:92.78286455199122 - timing_s/testing:85.74846938438714 - timing_s/save_checkpoint:12.991443185135722 - timing_s/stop_profile:0.000163935124874115 - timing_per_token_ms/update_actor:0.12862714495510336 - timing_per_token_ms/gen:0.6510362999962949 - timing_per_token_ms/adv:0.0031810781058643814 - perf/total_num_tokens:180767 - perf/time_per_step:92.78286455199122 - perf/throughput:243.53500087657153 (TaskRunner pid=2377588) Training Progress: 20%|██ | 20/100 [23:34<2:00:20, 90.25s/it] (TaskRunner pid=2377588) step:21 - global_seqlen/min:14483 - global_seqlen/max:22990 - global_seqlen/minmax_diff:8507 - global_seqlen/balanced_min:19046 - global_seqlen/balanced_max:19055 - global_seqlen/mean:19052.875 - actor/entropy:0.5120877623558044 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1840621531009674 - training/rollout_probs_diff_mean:0.005876380018889904 - training/rollout_probs_diff_std:0.01167724933475256 - training/rollout_actor_probs_pearson_corr:0.9991423487663269 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.79296875 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.79296875 - rollout_corr/rollout_is_mean:1.0001429319381714 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2423131920513697e-05 - rollout_corr/rollout_is_max:1.5040106773376465 - rollout_corr/rollout_is_min:0.40624791383743286 - rollout_corr/rollout_is_std:0.04042573645710945 - rollout_corr/rollout_is_eff_sample_size:0.9983687826948624 - rollout_corr/rollout_is_seq_mean:1.0001145601272583 - rollout_corr/rollout_is_seq_std:0.002321508713066578 - rollout_corr/rollout_is_seq_max:1.0061085224151611 - rollout_corr/rollout_is_seq_min:0.9920489192008972 - rollout_corr/rollout_is_seq_max_deviation:0.007951080799102783 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6984184235334396) - rollout_corr/training_log_ppl:np.float64(0.5222156696254387) - rollout_corr/kl:np.float64(0.0009260044083276853) - rollout_corr/k3_kl:np.float64(0.0009265095248451871) - rollout_corr/rollout_ppl:np.float64(1.696846168488264) - rollout_corr/rollout_log_ppl:np.float64(0.5212896637967788) - rollout_corr/log_ppl_diff:np.float64(0.0009260058286599815) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021170968539081514) - rollout_corr/log_ppl_diff_max:np.float64(0.008951246738433838) - rollout_corr/log_ppl_diff_min:np.float64(-0.005579054355621338) - rollout_corr/ppl_ratio:np.float64(1.0009295963682234) - rollout_corr/chi2_token:np.float64(0.0018469667993485928) - rollout_corr/chi2_seq:np.float64(0.6196231204085052) - actor/pg_loss:np.float64(0.007685429649427533) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000788077480137872) - actor/ppo_kl:np.float64(0.00019138877772206797) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.79296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(15467.604913280811) - self_distillation/token_reweight_w_std:np.float64(268551.2451789037) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0037785940803587) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19150046835420653) - self_distillation/empty_target_batch:np.float64(0.20703125) - actor/grad_norm:np.float64(0.4531072974205017) - perf/mfu/actor:np.float64(0.046303095723849996) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.82554244995117) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:1 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00038356418372131884 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00038356418372131884 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:314.43359375 - response_length/max:932.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.43359375 - response_length_non_aborted/max:932.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.96875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.072145283222198e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2106927819550037) - timing_s/agent_loop/generate_sequences/max:np.float64(7.464233869686723) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.750309351438773) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.464233869686723) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:932 - timing_s/gen:13.852330228313804 - timing_s/reward:0.058679722249507904 - timing_s/old_log_prob:3.020858768373728 - timing_s/adv:0.5515983290970325 - timing_s/update_actor:21.368308167904615 - timing_s/step:38.950380796566606 - timing_s/stop_profile:0.00011771172285079956 - timing_per_token_ms/update_actor:0.14019083844239133 - timing_per_token_ms/gen:0.1720893251545289 - timing_per_token_ms/adv:0.0036188654540130595 - perf/total_num_tokens:152423 - perf/time_per_step:38.950380796566606 - perf/throughput:489.1576054034232 (TaskRunner pid=2377588) Training Progress: 21%|██ | 21/100 [24:13<1:38:34, 74.87s/it] (TaskRunner pid=2377588) step:22 - global_seqlen/min:18594 - global_seqlen/max:28185 - global_seqlen/minmax_diff:9591 - global_seqlen/balanced_min:22499 - global_seqlen/balanced_max:22520 - global_seqlen/mean:22505.625 - actor/entropy:0.4973858594894409 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5542013049125671 - training/rollout_probs_diff_mean:0.005480976775288582 - training/rollout_probs_diff_std:0.011348037049174309 - training/rollout_actor_probs_pearson_corr:0.9992117285728455 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.4296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0000638961791992 - rollout_corr/rollout_is_ratio_fraction_high:1.095662264560815e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.095662264560815e-05 - rollout_corr/rollout_is_max:3.4903404712677 - rollout_corr/rollout_is_min:0.29839155077934265 - rollout_corr/rollout_is_std:0.03946327418088913 - rollout_corr/rollout_is_eff_sample_size:0.9984451902484637 - rollout_corr/rollout_is_seq_mean:1.0000715255737305 - rollout_corr/rollout_is_seq_std:0.0023582447320222855 - rollout_corr/rollout_is_seq_max:1.006727933883667 - rollout_corr/rollout_is_seq_min:0.9927475452423096 - rollout_corr/rollout_is_seq_max_deviation:0.00725245475769043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6967427376657724) - rollout_corr/training_log_ppl:np.float64(0.5204585177707486) - rollout_corr/kl:np.float64(0.0009738926087301536) - rollout_corr/k3_kl:np.float64(0.0009308744723739437) - rollout_corr/rollout_ppl:np.float64(1.6951308120042086) - rollout_corr/rollout_log_ppl:np.float64(0.5194846225786023) - rollout_corr/log_ppl_diff:np.float64(0.0009738951921463013) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020332636777311563) - rollout_corr/log_ppl_diff_max:np.float64(0.008866548538208008) - rollout_corr/log_ppl_diff_min:np.float64(-0.005231022834777832) - rollout_corr/ppl_ratio:np.float64(1.0009775443468243) - rollout_corr/chi2_token:np.float64(0.001786800567060709) - rollout_corr/chi2_seq:np.float64(0.5120338518172503) - actor/pg_loss:np.float64(0.010984312975779176) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007735706917628704) - actor/ppo_kl:np.float64(0.00020683866064885592) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4296875) - self_distillation/token_reweight_w_mean:np.float64(8344.419405999593) - self_distillation/token_reweight_w_std:np.float64(158612.5904527451) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0037203771062195) - self_distillation/token_reweight_w_clip_frac:np.float64(0.2241530851169955) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.5577966943383217) - perf/mfu/actor:np.float64(0.05420531182612104) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.53710174560547) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:1 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012328939512372017 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012328939512372017 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:356.51953125 - response_length/max:1224.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:356.51953125 - response_length_non_aborted/max:1224.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001379065215587616 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6555746141821146) - timing_s/agent_loop/generate_sequences/max:np.float64(9.639305803924799) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.903418488713214) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.639305803924799) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1224 - timing_s/gen:16.33606146275997 - timing_s/reward:0.08744135685265064 - timing_s/old_log_prob:3.0897953268140554 - timing_s/adv:0.6122857052832842 - timing_s/update_actor:21.522057734429836 - timing_s/step:41.73598306812346 - timing_s/stop_profile:0.0001183096319437027 - timing_per_token_ms/update_actor:0.11953710313771466 - timing_per_token_ms/gen:0.17898806235151005 - timing_per_token_ms/adv:0.0034007370673069743 - perf/total_num_tokens:180045 - perf/time_per_step:41.73598306812346 - perf/throughput:539.2379272165518 (TaskRunner pid=2377588) Training Progress: 22%|██▏ | 22/100 [24:55<1:24:25, 64.94s/it] (TaskRunner pid=2377588) step:23 - global_seqlen/min:18388 - global_seqlen/max:29877 - global_seqlen/minmax_diff:11489 - global_seqlen/balanced_min:22176 - global_seqlen/balanced_max:28830 - global_seqlen/mean:23014.375 - actor/entropy:0.44951191544532776 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.505251944065094 - training/rollout_probs_diff_mean:0.005099051166325808 - training/rollout_probs_diff_std:0.011282109655439854 - training/rollout_actor_probs_pearson_corr:0.9992111325263977 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.33984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.999826192855835 - rollout_corr/rollout_is_ratio_fraction_high:1.0166424544877373e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.0332849089754745e-05 - rollout_corr/rollout_is_max:2.010528087615967 - rollout_corr/rollout_is_min:0.35777750611305237 - rollout_corr/rollout_is_std:0.037814222276210785 - rollout_corr/rollout_is_eff_sample_size:0.9985717697032809 - rollout_corr/rollout_is_seq_mean:0.9998192191123962 - rollout_corr/rollout_is_seq_std:0.0022390682715922594 - rollout_corr/rollout_is_seq_max:1.007559895515442 - rollout_corr/rollout_is_seq_min:0.9938457608222961 - rollout_corr/rollout_is_seq_max_deviation:0.0075598955154418945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6801493726670742) - rollout_corr/training_log_ppl:np.float64(0.5111657797133375) - rollout_corr/kl:np.float64(0.0012937965818515806) - rollout_corr/k3_kl:np.float64(0.0009380410455719357) - rollout_corr/rollout_ppl:np.float64(1.6778879221528769) - rollout_corr/rollout_log_ppl:np.float64(0.5098719815941877) - rollout_corr/log_ppl_diff:np.float64(0.0012937981191498693) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021350611023081) - rollout_corr/log_ppl_diff_max:np.float64(0.009157359600067139) - rollout_corr/log_ppl_diff_min:np.float64(-0.006571829319000244) - rollout_corr/ppl_ratio:np.float64(1.0012975032441318) - rollout_corr/chi2_token:np.float64(0.0011514842044562101) - rollout_corr/chi2_seq:np.float64(0.5593507506418973) - actor/pg_loss:np.float64(0.007840548758395016) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008452609945379663) - actor/ppo_kl:np.float64(0.00028594181592467294) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.33984375) - self_distillation/token_reweight_w_mean:np.float64(7624.458969130879) - self_distillation/token_reweight_w_std:np.float64(171420.9914759735) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002567195566371) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17762680785381235) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.44826125353574753) - perf/mfu/actor:np.float64(0.05091760317287464) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(201.82965469360352) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.33984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.019520806148648262 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.019520806148648262 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:384.23046875 - response_length/max:8192.0 - response_length/min:145.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:384.23046875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:334.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012360885739326477 - timing_s/agent_loop/generate_sequences/min:np.float64(1.862102184444666) - timing_s/agent_loop/generate_sequences/max:np.float64(57.67887479625642) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.265522913708992) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(57.67887479625642) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:64.70091735385358 - timing_s/reward:0.06458873860538006 - timing_s/old_log_prob:3.269435051828623 - timing_s/adv:0.61101308837533 - timing_s/update_actor:23.46258319914341 - timing_s/step:92.19749046117067 - timing_s/stop_profile:0.00011623650789260864 - timing_per_token_ms/update_actor:0.12743439263038542 - timing_per_token_ms/gen:0.6577769827460893 - timing_per_token_ms/adv:0.003318649150668495 - perf/total_num_tokens:184115 - perf/time_per_step:92.19749046117067 - perf/throughput:249.6204059880848 (TaskRunner pid=2377588) Training Progress: 23%|██▎ | 23/100 [26:27<1:33:51, 73.14s/it] (TaskRunner pid=2377588) step:24 - global_seqlen/min:15054 - global_seqlen/max:26746 - global_seqlen/minmax_diff:11692 - global_seqlen/balanced_min:19319 - global_seqlen/balanced_max:20121 - global_seqlen/mean:19421.875 - actor/entropy:0.48785677552223206 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.13491398096084595 - training/rollout_probs_diff_mean:0.005895652342587709 - training/rollout_probs_diff_std:0.011872273869812489 - training/rollout_actor_probs_pearson_corr:0.9990942478179932 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.87890625 - self_distillation/correct_sample_fraction:0.453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87890625 - rollout_corr/rollout_is_mean:1.000136375427246 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.609630584833212e-05 - rollout_corr/rollout_is_max:1.7274718284606934 - rollout_corr/rollout_is_min:0.260029137134552 - rollout_corr/rollout_is_std:0.04023804888129234 - rollout_corr/rollout_is_eff_sample_size:0.9983838731511212 - rollout_corr/rollout_is_seq_mean:1.0001271963119507 - rollout_corr/rollout_is_seq_std:0.002382654929533601 - rollout_corr/rollout_is_seq_max:1.0068795680999756 - rollout_corr/rollout_is_seq_min:0.9931517243385315 - rollout_corr/rollout_is_seq_max_deviation:0.006879568099975586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6724183834157884) - rollout_corr/training_log_ppl:np.float64(0.5071819148724899) - rollout_corr/kl:np.float64(0.000800754967855255) - rollout_corr/k3_kl:np.float64(0.0010584685109051861) - rollout_corr/rollout_ppl:np.float64(1.6710083363577724) - rollout_corr/rollout_log_ppl:np.float64(0.5063811567379162) - rollout_corr/log_ppl_diff:np.float64(0.000800758134573698) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021024892339482903) - rollout_corr/log_ppl_diff_max:np.float64(0.011236429214477539) - rollout_corr/log_ppl_diff_min:np.float64(-0.006066709756851196) - rollout_corr/ppl_ratio:np.float64(1.0008045218419284) - rollout_corr/chi2_token:np.float64(0.002573532285168767) - rollout_corr/chi2_seq:np.float64(0.9754134530667216) - actor/pg_loss:np.float64(0.01049113622866571) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016053650110734452) - actor/ppo_kl:np.float64(6.589988715788309e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.453125) - self_distillation/token_reweight_w_mean:np.float64(50844.10125133139) - self_distillation/token_reweight_w_std:np.float64(845871.179314584) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0049168036784977) - self_distillation/token_reweight_w_clip_frac:np.float64(0.22834623540984467) - self_distillation/empty_target_batch:np.float64(0.12109375) - actor/grad_norm:np.float64(0.5027361214160919) - perf/mfu/actor:np.float64(0.04647768502684861) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(201.8885383605957) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004921129439026117 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004921129439026117 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:324.65234375 - response_length/max:1946.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.65234375 - response_length_non_aborted/max:1946.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013499706983566284 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2722700219601393) - timing_s/agent_loop/generate_sequences/max:np.float64(13.309977117925882) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9156080200409633) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.309977117925882) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1946 - timing_s/gen:20.28083667345345 - timing_s/reward:0.06008563190698624 - timing_s/old_log_prob:3.1650710944086313 - timing_s/adv:0.6510030515491962 - timing_s/update_actor:21.854908676818013 - timing_s/step:46.099592266604304 - timing_s/stop_profile:0.00012127310037612915 - timing_per_token_ms/update_actor:0.14065910652819316 - timing_per_token_ms/gen:0.24402108834514624 - timing_per_token_ms/adv:0.004189882874009308 - perf/total_num_tokens:155375 - perf/time_per_step:46.099592266604304 - perf/throughput:421.3025331694678 (TaskRunner pid=2377588) Training Progress: 24%|██▍ | 24/100 [27:13<1:22:22, 65.04s/it] (TaskRunner pid=2377588) step:25 - global_seqlen/min:19839 - global_seqlen/max:25827 - global_seqlen/minmax_diff:5988 - global_seqlen/balanced_min:23176 - global_seqlen/balanced_max:24566 - global_seqlen/mean:23357.75 - actor/entropy:0.4787044823169708 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27728891372680664 - training/rollout_probs_diff_mean:0.005640818737447262 - training/rollout_probs_diff_std:0.011776676401495934 - training/rollout_actor_probs_pearson_corr:0.9991179704666138 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.4296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:1.0000944137573242 - rollout_corr/rollout_is_ratio_fraction_high:2.1280669898260385e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0640334949130192e-05 - rollout_corr/rollout_is_max:2.632779836654663 - rollout_corr/rollout_is_min:0.33126774430274963 - rollout_corr/rollout_is_std:0.039915312081575394 - rollout_corr/rollout_is_eff_sample_size:0.9984097775176272 - rollout_corr/rollout_is_seq_mean:1.0001171827316284 - rollout_corr/rollout_is_seq_std:0.002497070701792836 - rollout_corr/rollout_is_seq_max:1.0080758333206177 - rollout_corr/rollout_is_seq_min:0.9934976100921631 - rollout_corr/rollout_is_seq_max_deviation:0.008075833320617676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.679585395846516) - rollout_corr/training_log_ppl:np.float64(0.5109057316149119) - rollout_corr/kl:np.float64(0.0008244847170508773) - rollout_corr/k3_kl:np.float64(0.001034108960766389) - rollout_corr/rollout_ppl:np.float64(1.67820990877226) - rollout_corr/rollout_log_ppl:np.float64(0.5100812441960443) - rollout_corr/log_ppl_diff:np.float64(0.0008244874188676476) - rollout_corr/log_ppl_abs_diff:np.float64(0.002083872794173658) - rollout_corr/log_ppl_diff_max:np.float64(0.008001565933227539) - rollout_corr/log_ppl_diff_min:np.float64(-0.0066817402839660645) - rollout_corr/ppl_ratio:np.float64(1.0008281350601465) - rollout_corr/chi2_token:np.float64(0.0024740875232964754) - rollout_corr/chi2_seq:np.float64(1.2950782587286085) - actor/pg_loss:np.float64(0.013218701118603349) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012952951287843462) - actor/ppo_kl:np.float64(7.08249014544915e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4296875) - self_distillation/token_reweight_w_mean:np.float64(11869.051639601123) - self_distillation/token_reweight_w_std:np.float64(222322.9643033404) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0048303119838238) - self_distillation/token_reweight_w_clip_frac:np.float64(0.24168428985285573) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.5900527536869049) - perf/mfu/actor:np.float64(0.055767033339097286) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(201.84992218017578) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015102625824511051 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.015102625824511051 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:367.1171875 - response_length/max:2705.0 - response_length/min:199.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:367.1171875 - response_length_non_aborted/max:2705.0 - response_length_non_aborted/min:199.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.8125 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018621794879436493 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6178430411964655) - timing_s/agent_loop/generate_sequences/max:np.float64(17.92142884247005) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.261537622631295) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.92142884247005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:2705 - timing_s/gen:25.04952882602811 - timing_s/reward:0.07000832818448544 - timing_s/old_log_prob:3.052505671977997 - timing_s/adv:0.5657446440309286 - timing_s/update_actor:21.892791898921132 - timing_s/step:50.72720322571695 - timing_s/stop_profile:2.9752030968666077e-05 - timing_per_token_ms/update_actor:0.11716021394890953 - timing_per_token_ms/gen:0.2665353879043658 - timing_per_token_ms/adv:0.003027606704578398 - perf/total_num_tokens:186862 - perf/time_per_step:50.72720322571695 - perf/throughput:460.4580681506688 (TaskRunner pid=2377588) Training Progress: 25%|██▌ | 25/100 [28:04<1:15:56, 60.75s/it] (TaskRunner pid=2377588) step:26 - global_seqlen/min:16214 - global_seqlen/max:22537 - global_seqlen/minmax_diff:6323 - global_seqlen/balanced_min:20610 - global_seqlen/balanced_max:20702 - global_seqlen/mean:20623.0 - actor/entropy:0.47131651639938354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38784223794937134 - training/rollout_probs_diff_mean:0.005906547885388136 - training/rollout_probs_diff_std:0.012139247730374336 - training/rollout_actor_probs_pearson_corr:0.9990562796592712 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.38671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000663995742798 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1645239283097908e-05 - rollout_corr/rollout_is_max:1.6350750923156738 - rollout_corr/rollout_is_min:0.41726866364479065 - rollout_corr/rollout_is_std:0.04128937050700188 - rollout_corr/rollout_is_eff_sample_size:0.9982982081917355 - rollout_corr/rollout_is_seq_mean:1.0001332759857178 - rollout_corr/rollout_is_seq_std:0.002408140804618597 - rollout_corr/rollout_is_seq_max:1.0065250396728516 - rollout_corr/rollout_is_seq_min:0.99333256483078 - rollout_corr/rollout_is_seq_max_deviation:0.006667435169219971 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6376840164884925) - rollout_corr/training_log_ppl:np.float64(0.48659548844443634) - rollout_corr/kl:np.float64(0.0008468073168614865) - rollout_corr/k3_kl:np.float64(0.0009677614357315178) - rollout_corr/rollout_ppl:np.float64(1.6362743326462805) - rollout_corr/rollout_log_ppl:np.float64(0.4857486785040237) - rollout_corr/log_ppl_diff:np.float64(0.0008468099404126406) - rollout_corr/log_ppl_abs_diff:np.float64(0.002121247351169586) - rollout_corr/log_ppl_diff_max:np.float64(0.009415626525878906) - rollout_corr/log_ppl_diff_min:np.float64(-0.007289290428161621) - rollout_corr/ppl_ratio:np.float64(1.0008504705037922) - rollout_corr/chi2_token:np.float64(0.002211208688095212) - rollout_corr/chi2_seq:np.float64(0.7919924787711352) - actor/pg_loss:np.float64(0.009229068295098841) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009685162540336023) - actor/ppo_kl:np.float64(7.747204033403676e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.38671875) - self_distillation/token_reweight_w_mean:np.float64(9200.740631787106) - self_distillation/token_reweight_w_std:np.float64(142808.09028592636) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.006582108559087) - self_distillation/token_reweight_w_clip_frac:np.float64(0.20154389954404905) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.45165353268384933) - perf/mfu/actor:np.float64(0.049917817511633766) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(201.8992156982422) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005173397716134787 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005173397716134787 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.4375 - response_length/max:1265.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.4375 - response_length_non_aborted/max:1265.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.2283907532691956e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1422349009662867) - timing_s/agent_loop/generate_sequences/max:np.float64(9.300913831219077) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.855257952622196) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.300913831219077) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1265 - timing_s/gen:16.89883165806532 - timing_s/reward:0.06087753735482693 - timing_s/old_log_prob:2.9687695261090994 - timing_s/adv:0.5324470903724432 - timing_s/update_actor:21.44449758902192 - timing_s/step:41.99549892731011 - timing_s/stop_profile:0.00010871142148971558 - timing_per_token_ms/update_actor:0.12997925610375505 - timing_per_token_ms/gen:0.19679094067991101 - timing_per_token_ms/adv:0.0032272650097733306 - perf/total_num_tokens:164984 - perf/time_per_step:41.99549892731011 - perf/throughput:491.07643739859583 (TaskRunner pid=2377588) Training Progress: 26%|██▌ | 26/100 [28:46<1:08:00, 55.14s/it] (TaskRunner pid=2377588) step:27 - global_seqlen/min:15658 - global_seqlen/max:29662 - global_seqlen/minmax_diff:14004 - global_seqlen/balanced_min:20372 - global_seqlen/balanced_max:27480 - global_seqlen/mean:21262.75 - actor/entropy:0.4531696140766144 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35207241773605347 - training/rollout_probs_diff_mean:0.005608410108834505 - training/rollout_probs_diff_std:0.01199356559664011 - training/rollout_actor_probs_pearson_corr:0.9990782737731934 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8515625 - self_distillation/correct_sample_fraction:0.359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8515625 - rollout_corr/rollout_is_mean:0.9999279975891113 - rollout_corr/rollout_is_ratio_fraction_high:1.1146034921694081e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.4584139686776325e-05 - rollout_corr/rollout_is_max:2.2843728065490723 - rollout_corr/rollout_is_min:0.35500991344451904 - rollout_corr/rollout_is_std:0.04010748118162155 - rollout_corr/rollout_is_eff_sample_size:0.998393735664305 - rollout_corr/rollout_is_seq_mean:0.999920129776001 - rollout_corr/rollout_is_seq_std:0.0026268567889928818 - rollout_corr/rollout_is_seq_max:1.0078831911087036 - rollout_corr/rollout_is_seq_min:0.9926331043243408 - rollout_corr/rollout_is_seq_max_deviation:0.007883191108703613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6703167296946049) - rollout_corr/training_log_ppl:np.float64(0.5051685213620658) - rollout_corr/kl:np.float64(0.0011934659439702955) - rollout_corr/k3_kl:np.float64(0.001111360418235563) - rollout_corr/rollout_ppl:np.float64(1.668345833197236) - rollout_corr/rollout_log_ppl:np.float64(0.5039750541182002) - rollout_corr/log_ppl_diff:np.float64(0.001193467243865598) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022786261470173486) - rollout_corr/log_ppl_diff_max:np.float64(0.00844663381576538) - rollout_corr/log_ppl_diff_min:np.float64(-0.006630361080169678) - rollout_corr/ppl_ratio:np.float64(1.0011976696550846) - rollout_corr/chi2_token:np.float64(0.002105120336636901) - rollout_corr/chi2_seq:np.float64(0.4242347127292305) - actor/pg_loss:np.float64(0.01180989935528487) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016225343665610126) - actor/ppo_kl:np.float64(0.00022113704509862941) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.359375) - self_distillation/token_reweight_w_mean:np.float64(49236.24129780242) - self_distillation/token_reweight_w_std:np.float64(829986.4696171954) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0058862876612693) - self_distillation/token_reweight_w_clip_frac:np.float64(0.23669213621178642) - self_distillation/empty_target_batch:np.float64(0.1484375) - actor/grad_norm:np.float64(0.5475868284702301) - perf/mfu/actor:np.float64(0.048284592743189) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(201.92653274536133) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.038016341626644135 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.038016341626644135 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:350.4609375 - response_length/max:8192.0 - response_length/min:163.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:350.4609375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:314.0 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.47117805480957e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0815379098057747) - timing_s/agent_loop/generate_sequences/max:np.float64(57.935925507918) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9501595712208655) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(57.935925507918) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:64.9002950899303 - timing_s/reward:0.06266527436673641 - timing_s/old_log_prob:3.3962056171149015 - timing_s/adv:0.5389443319290876 - timing_s/update_actor:22.851768266409636 - timing_s/step:91.83943105489016 - timing_s/stop_profile:0.00012227147817611694 - timing_per_token_ms/update_actor:0.13434156133619612 - timing_per_token_ms/gen:0.7233809836368432 - timing_per_token_ms/adv:0.003168359760197338 - perf/total_num_tokens:170102 - perf/time_per_step:91.83943105489016 - perf/throughput:231.5209246809443 (TaskRunner pid=2377588) Training Progress: 27%|██▋ | 27/100 [30:18<1:20:30, 66.17s/it] (TaskRunner pid=2377588) step:28 - global_seqlen/min:18169 - global_seqlen/max:23902 - global_seqlen/minmax_diff:5733 - global_seqlen/balanced_min:21033 - global_seqlen/balanced_max:21043 - global_seqlen/mean:21040.625 - actor/entropy:0.4889027774333954 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21022874116897583 - training/rollout_probs_diff_mean:0.006093648727983236 - training/rollout_probs_diff_std:0.012167380191385746 - training/rollout_actor_probs_pearson_corr:0.9990872740745544 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001447200775146 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4988443328766152e-05 - rollout_corr/rollout_is_max:1.7780239582061768 - rollout_corr/rollout_is_min:0.3028954267501831 - rollout_corr/rollout_is_std:0.04164011403918266 - rollout_corr/rollout_is_eff_sample_size:0.9982696960710439 - rollout_corr/rollout_is_seq_mean:1.0000829696655273 - rollout_corr/rollout_is_seq_std:0.002573413075879216 - rollout_corr/rollout_is_seq_max:1.0086705684661865 - rollout_corr/rollout_is_seq_min:0.992500364780426 - rollout_corr/rollout_is_seq_max_deviation:0.008670568466186523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.666687823832035) - rollout_corr/training_log_ppl:np.float64(0.5036763361422345) - rollout_corr/kl:np.float64(0.0008290242327049224) - rollout_corr/k3_kl:np.float64(0.0010337613970818893) - rollout_corr/rollout_ppl:np.float64(1.6652166354469955) - rollout_corr/rollout_log_ppl:np.float64(0.5028473130078055) - rollout_corr/log_ppl_diff:np.float64(0.0008290231344290078) - rollout_corr/log_ppl_abs_diff:np.float64(0.002348267997149378) - rollout_corr/log_ppl_diff_max:np.float64(0.011133730411529541) - rollout_corr/log_ppl_diff_min:np.float64(-0.00680994987487793) - rollout_corr/ppl_ratio:np.float64(1.000833887141198) - rollout_corr/chi2_token:np.float64(0.002451273612678051) - rollout_corr/chi2_seq:np.float64(1.2610166266094893) - actor/pg_loss:np.float64(0.00974678189959377) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001906346491068689) - actor/ppo_kl:np.float64(1.3276782340199134e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(22649.079057591967) - self_distillation/token_reweight_w_std:np.float64(383394.1029969334) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0064352522604167) - self_distillation/token_reweight_w_clip_frac:np.float64(0.2239057349797804) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.4965593293309212) - perf/mfu/actor:np.float64(0.05102766963542814) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(201.8553237915039) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009323812555521727 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009323812555521727 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:312.64453125 - response_length/max:897.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:312.64453125 - response_length_non_aborted/max:897.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:344.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.8436766266822815e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2576064094901085) - timing_s/agent_loop/generate_sequences/max:np.float64(7.265696134418249) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8134725530981086) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.265696134418249) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:897 - timing_s/gen:14.313099836930633 - timing_s/reward:0.06055969372391701 - timing_s/old_log_prob:3.0706457532942295 - timing_s/adv:0.5548284370452166 - timing_s/update_actor:21.259054869413376 - timing_s/step:39.34340473264456 - timing_s/stop_profile:0.00011925958096981049 - timing_per_token_ms/update_actor:0.1262976674255956 - timing_per_token_ms/gen:0.1788310386062775 - timing_per_token_ms/adv:0.0032961736940158417 - perf/total_num_tokens:168325 - perf/time_per_step:39.34340473264456 - perf/throughput:534.7942086603878 (TaskRunner pid=2377588) Training Progress: 28%|██▊ | 28/100 [30:58<1:09:45, 58.13s/it] (TaskRunner pid=2377588) step:29 - global_seqlen/min:15203 - global_seqlen/max:22647 - global_seqlen/minmax_diff:7444 - global_seqlen/balanced_min:19201 - global_seqlen/balanced_max:19468 - global_seqlen/mean:19238.125 - actor/entropy:0.48344773054122925 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.353477418422699 - training/rollout_probs_diff_mean:0.006384729873389006 - training/rollout_probs_diff_std:0.01290085818618536 - training/rollout_actor_probs_pearson_corr:0.998953640460968 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.4375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0000580549240112 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.100934868096374e-05 - rollout_corr/rollout_is_max:1.8041176795959473 - rollout_corr/rollout_is_min:0.3592689335346222 - rollout_corr/rollout_is_std:0.04326458275318146 - rollout_corr/rollout_is_eff_sample_size:0.9981319104440246 - rollout_corr/rollout_is_seq_mean:1.0001035928726196 - rollout_corr/rollout_is_seq_std:0.0025487877428531647 - rollout_corr/rollout_is_seq_max:1.0069139003753662 - rollout_corr/rollout_is_seq_min:0.9937771558761597 - rollout_corr/rollout_is_seq_max_deviation:0.006913900375366211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6430399301461875) - rollout_corr/training_log_ppl:np.float64(0.4894179511175025) - rollout_corr/kl:np.float64(0.001021881831787752) - rollout_corr/k3_kl:np.float64(0.0010562381066279158) - rollout_corr/rollout_ppl:np.float64(1.641366011928767) - rollout_corr/rollout_log_ppl:np.float64(0.48839606769615784) - rollout_corr/log_ppl_diff:np.float64(0.0010218834213446826) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022649977181572467) - rollout_corr/log_ppl_diff_max:np.float64(0.011546403169631958) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058746337890625) - rollout_corr/ppl_ratio:np.float64(1.0010259815026075) - rollout_corr/chi2_token:np.float64(0.002216867869719863) - rollout_corr/chi2_seq:np.float64(0.5971095731947571) - actor/pg_loss:np.float64(0.0148409673711285) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000948047128986218) - actor/ppo_kl:np.float64(0.0001435367357309758) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4375) - self_distillation/token_reweight_w_mean:np.float64(16616.868872509338) - self_distillation/token_reweight_w_std:np.float64(288701.20495704864) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.005621918477118) - self_distillation/token_reweight_w_clip_frac:np.float64(0.24011366692138836) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.5514515414834023) - perf/mfu/actor:np.float64(0.04641049529807512) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(201.83461380004883) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:2 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015844778390601277 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0015844778390601277 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:306.31640625 - response_length/max:1234.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:306.31640625 - response_length_non_aborted/max:1234.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00048289261758327484 - timing_s/agent_loop/generate_sequences/min:np.float64(2.137800371274352) - timing_s/agent_loop/generate_sequences/max:np.float64(9.177594237029552) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.658530576947669) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.177594237029552) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1234 - timing_s/gen:16.37922285683453 - timing_s/reward:0.05153236724436283 - timing_s/old_log_prob:3.102977057918906 - timing_s/adv:0.6394347418099642 - timing_s/update_actor:21.543278941884637 - timing_s/step:41.81104242056608 - timing_s/stop_profile:0.00012219324707984924 - timing_per_token_ms/update_actor:0.13997777162460373 - timing_per_token_ms/gen:0.208873367469229 - timing_per_token_ms/adv:0.004154736635001879 - perf/total_num_tokens:153905 - perf/time_per_step:41.81104242056608 - perf/throughput:460.1206735409477 (TaskRunner pid=2377588) Training Progress: 29%|██▉ | 29/100 [31:40<1:03:09, 53.37s/it] (TaskRunner pid=2377588) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2377588) validation generation end (TaskRunner pid=2377588) [prompt] system (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2377588) (TaskRunner pid=2377588) A (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) user (TaskRunner pid=2377588) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2377588) (TaskRunner pid=2377588) A: -0.01 (TaskRunner pid=2377588) B: 1.69 (TaskRunner pid=2377588) C: 2.49 (TaskRunner pid=2377588) D: 0.45 (TaskRunner pid=2377588) Please reason step by step. (TaskRunner pid=2377588) assistant (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) [response] (TaskRunner pid=2377588) To determine the folding stability score of the given protein sequence, we need to evaluate the overall structural and energetic properties of the protein. Folding stability scores are typically derived from computational methods that assess the likelihood of a protein folding into its native conformation based on amino acid composition, secondary structure propensity, and other physicochemical properties. (TaskRunner pid=2377588) (TaskRunner pid=2377588) The sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR' contains a mix of hydrophobic and hydrophilic amino acids, which can influence folding stability. However, without direct computation or access to a specific scoring algorithm, we can infer based on the sequence's characteristics. (TaskRunner pid=2377588) (TaskRunner pid=2377588) The presence of multiple charged residues (e.g., lysine, arginine, glutamic acid) and a relatively high proportion of polar amino acids may suggest a less stable folding environment, as these residues can lead to electrostatic repulsion or reduced hydrophobic core formation. However, the presence of some hydrophobic residues (e.g., glycine, serine, tryptophan) may contribute to some degree of stability. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given the options, the most reasonable estimate for a folding stability score would be a moderate value, neither extremely negative nor extremely positive. Among the provided choices, option B (1.69) represents a moderate positive value, suggesting a relatively stable folding environment, which is consistent with the sequence's composition. (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) B (TaskRunner pid=2377588) (TaskRunner pid=2377588) [ground_truth] B (TaskRunner pid=2377588) [score] 1.0 (TaskRunner pid=2377588) [acc] 1.0 (TaskRunner pid=2377588) [pred] B (TaskRunner pid=2377588) [incorrect_format] 1 (TaskRunner pid=2377588) [feedback] (TaskRunner pid=2377588) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2377588) step:30 - global_seqlen/min:16427 - global_seqlen/max:25363 - global_seqlen/minmax_diff:8936 - global_seqlen/balanced_min:20071 - global_seqlen/balanced_max:20179 - global_seqlen/mean:20092.5 - actor/entropy:0.493918776512146 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1742207407951355 - training/rollout_probs_diff_mean:0.0060681733302772045 - training/rollout_probs_diff_std:0.012186025269329548 - training/rollout_actor_probs_pearson_corr:0.9990466237068176 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.42578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0000113248825073 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.768505573272705 - rollout_corr/rollout_is_min:0.5041154026985168 - rollout_corr/rollout_is_std:0.04171591252088547 - rollout_corr/rollout_is_eff_sample_size:0.9982628058862296 - rollout_corr/rollout_is_seq_mean:1.0000231266021729 - rollout_corr/rollout_is_seq_std:0.002335590310394764 - rollout_corr/rollout_is_seq_max:1.0094752311706543 - rollout_corr/rollout_is_seq_min:0.9928362965583801 - rollout_corr/rollout_is_seq_max_deviation:0.009475231170654297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.677387180738151) - rollout_corr/training_log_ppl:np.float64(0.5095777458627708) - rollout_corr/kl:np.float64(0.0010656633592986964) - rollout_corr/k3_kl:np.float64(0.0010183820149904932) - rollout_corr/rollout_ppl:np.float64(1.6755933612585068) - rollout_corr/rollout_log_ppl:np.float64(0.5085120819858275) - rollout_corr/log_ppl_diff:np.float64(0.0010656638769432902) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020819585770368576) - rollout_corr/log_ppl_diff_max:np.float64(0.008295953273773193) - rollout_corr/log_ppl_diff_min:np.float64(-0.008558869361877441) - rollout_corr/ppl_ratio:np.float64(1.0010692360810935) - rollout_corr/chi2_token:np.float64(0.0019709025509655476) - rollout_corr/chi2_seq:np.float64(0.5447194008156657) - actor/pg_loss:np.float64(0.01119235239457339) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007738353790500696) - actor/ppo_kl:np.float64(0.0001763532315237626) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.42578125) - self_distillation/token_reweight_w_mean:np.float64(12927.568906920962) - self_distillation/token_reweight_w_std:np.float64(248859.6214018159) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012700511142612) - self_distillation/token_reweight_w_clip_frac:np.float64(0.21296004866599105) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.5157803595066071) - perf/mfu/actor:np.float64(0.048999916451758774) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(201.89230728149414) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.36375) - val-aux/sciknoweval/reward/std@16:np.float64(0.3439600412634843) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5123) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.3215447298393228) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.21533999999999998) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.26134794614231943) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.36513999999999996) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3449867327077535) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.64628) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.25266009317469307) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.11406000000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15847589739667162) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.37432000000000004) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.29176674648986617) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7456400000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.16914281385742286) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.05906000000000001) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07827069204398616) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.38305999999999996) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.23326021185603288) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8088200000000002) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.09880062672697684) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.034140000000000004) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03703547833675639) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.3891800000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.17777605729385754) - val-aux/sciknoweval/score/mean@16:np.float64(0.36375) - val-aux/sciknoweval/score/std@16:np.float64(0.3439600412634843) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5123) - val-aux/sciknoweval/score/best@2/std:np.float64(0.3215447298393228) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.21533999999999998) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.26134794614231943) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.36513999999999996) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3449867327077535) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.64628) - val-aux/sciknoweval/score/best@4/std:np.float64(0.25266009317469307) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.11406000000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15847589739667162) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.37432000000000004) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.29176674648986617) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7456400000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.16914281385742286) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.05906000000000001) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07827069204398616) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.38305999999999996) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.23326021185603288) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8088200000000002) - val-aux/sciknoweval/score/best@16/std:np.float64(0.09880062672697684) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.034140000000000004) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03703547833675639) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.3891800000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.17777605729385754) - val-core/sciknoweval/acc/mean@16:np.float64(0.36375) - val-aux/sciknoweval/acc/std@16:np.float64(0.3439600412634843) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5123) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.3215447298393228) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.21533999999999998) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.26134794614231943) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.36513999999999996) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3449867327077535) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.64628) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.25266009317469307) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.11406000000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15847589739667162) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.37432000000000004) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.29176674648986617) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7456400000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.16914281385742286) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.05906000000000001) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07827069204398616) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.38305999999999996) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.23326021185603288) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8088200000000002) - val-core/sciknoweval/acc/best@16/std:np.float64(0.09880062672697684) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.034140000000000004) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03703547833675639) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.3891800000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.17777605729385754) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9925) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.02232993504627581) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99924) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006492742258451383) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.98612) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.02863510383595038) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9928200000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.02183086532127425) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999799999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0006321392251711644) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.97572) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03465045185629395) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99806) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.011435006795088137) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9597399999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.037024556387939826) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99968) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.004244988568798449) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.94188) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03230423252800871) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999799999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711644) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:2 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006329588126391172 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006329588126391172 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:312.890625 - response_length/max:1224.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:312.890625 - response_length_non_aborted/max:1224.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.0 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.872097194194794e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4409107211977243) - timing_s/agent_loop/generate_sequences/max:np.float64(9.400854682549834) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6527277899804176) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.400854682549834) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:1224 - timing_s/gen:16.121729787439108 - timing_s/reward:0.058806924149394035 - timing_s/old_log_prob:3.022511411458254 - timing_s/adv:0.5684543661773205 - timing_s/update_actor:21.17614277265966 - timing_s/step:41.03685702010989 - timing_s/testing:82.65723511017859 - timing_s/save_checkpoint:13.18932064063847 - timing_s/stop_profile:0.0001387670636177063 - timing_per_token_ms/update_actor:0.13174158748699552 - timing_per_token_ms/gen:0.2012700347994895 - timing_per_token_ms/adv:0.0035364835521794233 - perf/total_num_tokens:160740 - perf/time_per_step:41.03685702010989 - perf/throughput:489.6208301272629 (TaskRunner pid=2377588) Training Progress: 30%|███ | 30/100 [33:57<1:31:30, 78.44s/it] (TaskRunner pid=2377588) step:31 - global_seqlen/min:16509 - global_seqlen/max:25877 - global_seqlen/minmax_diff:9368 - global_seqlen/balanced_min:19842 - global_seqlen/balanced_max:19850 - global_seqlen/mean:19847.875 - actor/entropy:0.46630626916885376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18102285265922546 - training/rollout_probs_diff_mean:0.006234156899154186 - training/rollout_probs_diff_std:0.01267819944769144 - training/rollout_actor_probs_pearson_corr:0.998958170413971 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.60546875 - self_distillation/correct_sample_fraction:0.31640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.60546875 - rollout_corr/rollout_is_mean:1.000218152999878 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.778808832168579 - rollout_corr/rollout_is_min:0.5478630661964417 - rollout_corr/rollout_is_std:0.042811714112758636 - rollout_corr/rollout_is_eff_sample_size:0.9981713417553787 - rollout_corr/rollout_is_seq_mean:1.000227451324463 - rollout_corr/rollout_is_seq_std:0.0026660594157874584 - rollout_corr/rollout_is_seq_max:1.0059154033660889 - rollout_corr/rollout_is_seq_min:0.9907479882240295 - rollout_corr/rollout_is_seq_max_deviation:0.009252011775970459 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6235858551226556) - rollout_corr/training_log_ppl:np.float64(0.47780306957429275) - rollout_corr/kl:np.float64(0.0009481541051883369) - rollout_corr/k3_kl:np.float64(0.0011065828289247293) - rollout_corr/rollout_ppl:np.float64(1.6220748396590352) - rollout_corr/rollout_log_ppl:np.float64(0.4768549154396169) - rollout_corr/log_ppl_diff:np.float64(0.0009481541346758604) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023901162203401327) - rollout_corr/log_ppl_diff_max:np.float64(0.008728623390197754) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058653950691223145) - rollout_corr/ppl_ratio:np.float64(1.0009525464847684) - rollout_corr/chi2_token:np.float64(0.0025139120407402515) - rollout_corr/chi2_seq:np.float64(0.7535167471505702) - actor/pg_loss:np.float64(0.005722168716602027) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012057437961630058) - actor/ppo_kl:np.float64(0.0002287827413596233) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.60546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.31640625) - self_distillation/token_reweight_w_mean:np.float64(35464.82859730418) - self_distillation/token_reweight_w_std:np.float64(573582.2694413096) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0045762308873236) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13131537244771607) - self_distillation/empty_target_batch:np.float64(0.39453125) - actor/grad_norm:np.float64(0.37259600311517715) - perf/mfu/actor:np.float64(0.04827067544683741) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.56623077392578) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:2 - critic/score/mean:0.31640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.31640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00029190664645284414 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.00029190664645284414 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:297.74609375 - response_length/max:893.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:297.74609375 - response_length_non_aborted/max:893.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013593211770057678 - timing_s/agent_loop/generate_sequences/min:np.float64(2.189517730847001) - timing_s/agent_loop/generate_sequences/max:np.float64(7.393782524392009) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.630496432131622) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.393782524392009) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:893 - timing_s/gen:13.749396739527583 - timing_s/reward:0.05902630276978016 - timing_s/old_log_prob:3.0275142565369606 - timing_s/adv:0.5265827756375074 - timing_s/update_actor:21.22019432298839 - timing_s/step:38.677121015265584 - timing_s/stop_profile:0.00011759437620639801 - timing_per_token_ms/update_actor:0.13364273456848902 - timing_per_token_ms/gen:0.1803838308585018 - timing_per_token_ms/adv:0.003316367467786271 - perf/total_num_tokens:158783 - perf/time_per_step:38.677121015265584 - perf/throughput:513.1683661812932 (TaskRunner pid=2377588) Training Progress: 31%|███ | 31/100 [34:35<1:16:30, 66.53s/it] (TaskRunner pid=2377588) step:32 - global_seqlen/min:17909 - global_seqlen/max:24421 - global_seqlen/minmax_diff:6512 - global_seqlen/balanced_min:20791 - global_seqlen/balanced_max:20802 - global_seqlen/mean:20799.25 - actor/entropy:0.47777989506721497 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3979852795600891 - training/rollout_probs_diff_mean:0.0061486284248530865 - training/rollout_probs_diff_std:0.012513584457337856 - training/rollout_actor_probs_pearson_corr:0.9989986419677734 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000030994415283 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.515533378755208e-05 - rollout_corr/rollout_is_max:1.6426279544830322 - rollout_corr/rollout_is_min:0.39760929346084595 - rollout_corr/rollout_is_std:0.04181153327226639 - rollout_corr/rollout_is_eff_sample_size:0.9982547278596414 - rollout_corr/rollout_is_seq_mean:1.0000025033950806 - rollout_corr/rollout_is_seq_std:0.002330186776816845 - rollout_corr/rollout_is_seq_max:1.0070310831069946 - rollout_corr/rollout_is_seq_min:0.9937487840652466 - rollout_corr/rollout_is_seq_max_deviation:0.007031083106994629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6390082482248545) - rollout_corr/training_log_ppl:np.float64(0.48696012602886185) - rollout_corr/kl:np.float64(0.0009611047022932695) - rollout_corr/k3_kl:np.float64(0.0010630507269979717) - rollout_corr/rollout_ppl:np.float64(1.6373524283990264) - rollout_corr/rollout_log_ppl:np.float64(0.48599902080604807) - rollout_corr/log_ppl_diff:np.float64(0.0009611052228137851) - rollout_corr/log_ppl_abs_diff:np.float64(0.002148654661141336) - rollout_corr/log_ppl_diff_max:np.float64(0.007834315299987793) - rollout_corr/log_ppl_diff_min:np.float64(-0.005520284175872803) - rollout_corr/ppl_ratio:np.float64(1.0009647351689637) - rollout_corr/chi2_token:np.float64(0.0023397107142955065) - rollout_corr/chi2_seq:np.float64(0.426089143846184) - actor/pg_loss:np.float64(0.01020576199516654) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011351422472216655) - actor/ppo_kl:np.float64(5.787937134016374e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(9832.988094384782) - self_distillation/token_reweight_w_std:np.float64(155216.80877948168) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005309956613928) - self_distillation/token_reweight_w_clip_frac:np.float64(0.20914308965438977) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5270844399929047) - perf/mfu/actor:np.float64(0.050776773519896054) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.42731094360352) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:2 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0035610520280897617 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0035610520280897617 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:310.5703125 - response_length/max:914.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:310.5703125 - response_length_non_aborted/max:914.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:339.40625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011726096272468567 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0971967596560717) - timing_s/agent_loop/generate_sequences/max:np.float64(7.502464842051268) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7191463554481743) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.502464842051268) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:914 - timing_s/gen:14.310144474729896 - timing_s/reward:0.05996168404817581 - timing_s/old_log_prob:3.0337655879557133 - timing_s/adv:0.5110094286501408 - timing_s/update_actor:21.098169365897775 - timing_s/step:39.10354966111481 - timing_s/stop_profile:0.00012037903070449829 - timing_per_token_ms/update_actor:0.12679645519608745 - timing_per_token_ms/gen:0.17998823327459432 - timing_per_token_ms/adv:0.0030710808601881125 - perf/total_num_tokens:166394 - perf/time_per_step:39.10354966111481 - perf/throughput:531.9018396092849 (TaskRunner pid=2377588) Training Progress: 32%|███▏ | 32/100 [35:15<1:06:05, 58.32s/it] (TaskRunner pid=2377588) step:33 - global_seqlen/min:15937 - global_seqlen/max:24955 - global_seqlen/minmax_diff:9018 - global_seqlen/balanced_min:20774 - global_seqlen/balanced_max:20798 - global_seqlen/mean:20781.625 - actor/entropy:0.4674495458602905 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294977903366089 - training/rollout_probs_diff_mean:0.00613911310210824 - training/rollout_probs_diff_std:0.012502591125667095 - training/rollout_actor_probs_pearson_corr:0.9989820122718811 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0001457929611206 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.919877781299874e-05 - rollout_corr/rollout_is_max:1.6822738647460938 - rollout_corr/rollout_is_min:0.4011400043964386 - rollout_corr/rollout_is_std:0.04206451401114464 - rollout_corr/rollout_is_eff_sample_size:0.9982341769990338 - rollout_corr/rollout_is_seq_mean:1.000181794166565 - rollout_corr/rollout_is_seq_std:0.002640316728502512 - rollout_corr/rollout_is_seq_max:1.0097932815551758 - rollout_corr/rollout_is_seq_min:0.9917042255401611 - rollout_corr/rollout_is_seq_max_deviation:0.009793281555175781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6236442402005196) - rollout_corr/training_log_ppl:np.float64(0.4787052171304822) - rollout_corr/kl:np.float64(0.0010754358174693834) - rollout_corr/k3_kl:np.float64(0.0010445474838434166) - rollout_corr/rollout_ppl:np.float64(1.6218907786533237) - rollout_corr/rollout_log_ppl:np.float64(0.4776297784410417) - rollout_corr/log_ppl_diff:np.float64(0.0010754386894404888) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025129715213552117) - rollout_corr/log_ppl_diff_max:np.float64(0.009329020977020264) - rollout_corr/log_ppl_diff_min:np.float64(-0.007838189601898193) - rollout_corr/ppl_ratio:np.float64(1.0010803188197315) - rollout_corr/chi2_token:np.float64(0.0020940976683050394) - rollout_corr/chi2_seq:np.float64(1.0270451633259654) - actor/pg_loss:np.float64(0.01178671873640269) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011148202829645015) - actor/ppo_kl:np.float64(0.0003355119419090613) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(40092.434928422095) - self_distillation/token_reweight_w_std:np.float64(645629.2541055046) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0035637321416289) - self_distillation/token_reweight_w_clip_frac:np.float64(0.24692582045099698) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.5470886528491974) - perf/mfu/actor:np.float64(0.050417892074124684) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.36059188842773) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:2 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00022539297060575336 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00022539297060575336 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:298.95703125 - response_length/max:960.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:298.95703125 - response_length_non_aborted/max:960.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:350.46875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.933324038982391e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8425326123833656) - timing_s/agent_loop/generate_sequences/max:np.float64(7.561475658789277) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5510697506106226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.561475658789277) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:960 - timing_s/gen:15.50871255621314 - timing_s/reward:0.06243013218045235 - timing_s/old_log_prob:2.9950323551893234 - timing_s/adv:0.5154443625360727 - timing_s/update_actor:21.01259404234588 - timing_s/step:40.18192465044558 - timing_s/stop_profile:0.0001189354807138443 - timing_per_token_ms/update_actor:0.12638926240336043 - timing_per_token_ms/gen:0.20264085500650883 - timing_per_token_ms/adv:0.0031003612718932755 - perf/total_num_tokens:166253 - perf/time_per_step:40.18192465044558 - perf/throughput:517.1883920639812 (TaskRunner pid=2377588) Training Progress: 33%|███▎ | 33/100 [35:55<59:03, 52.89s/it] (TaskRunner pid=2377588) step:34 - global_seqlen/min:14906 - global_seqlen/max:22772 - global_seqlen/minmax_diff:7866 - global_seqlen/balanced_min:18875 - global_seqlen/balanced_max:19145 - global_seqlen/mean:18913.125 - actor/entropy:0.4958954453468323 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25869929790496826 - training/rollout_probs_diff_mean:0.006314093247056007 - training/rollout_probs_diff_std:0.012487724423408508 - training/rollout_actor_probs_pearson_corr:0.9989966154098511 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:1.0000925064086914 - rollout_corr/rollout_is_ratio_fraction_high:1.3758753993897699e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.237330913543701 - rollout_corr/rollout_is_min:0.5961244106292725 - rollout_corr/rollout_is_std:0.04214661940932274 - rollout_corr/rollout_is_eff_sample_size:0.9982269309436012 - rollout_corr/rollout_is_seq_mean:1.0001099109649658 - rollout_corr/rollout_is_seq_std:0.0026173973456025124 - rollout_corr/rollout_is_seq_max:1.008721947669983 - rollout_corr/rollout_is_seq_min:0.9934532642364502 - rollout_corr/rollout_is_seq_max_deviation:0.00872194766998291 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6644066097214818) - rollout_corr/training_log_ppl:np.float64(0.5033825403661467) - rollout_corr/kl:np.float64(0.0008166040661237162) - rollout_corr/k3_kl:np.float64(0.0009966837123442929) - rollout_corr/rollout_ppl:np.float64(1.663013834040612) - rollout_corr/rollout_log_ppl:np.float64(0.5025659375824034) - rollout_corr/log_ppl_diff:np.float64(0.0008166027837432921) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022712033824063838) - rollout_corr/log_ppl_diff_max:np.float64(0.007540076971054077) - rollout_corr/log_ppl_diff_min:np.float64(-0.009322822093963623) - rollout_corr/ppl_ratio:np.float64(1.000820868415758) - rollout_corr/chi2_token:np.float64(0.002364296466112137) - rollout_corr/chi2_seq:np.float64(0.8672483554109931) - actor/pg_loss:np.float64(0.009502434404566884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000947456233916455) - actor/ppo_kl:np.float64(2.172364201769028e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.390625) - self_distillation/token_reweight_w_mean:np.float64(17952.3283205505) - self_distillation/token_reweight_w_std:np.float64(298466.17212537775) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0044964824337512) - self_distillation/token_reweight_w_clip_frac:np.float64(0.21122038035537116) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.512259304523468) - perf/mfu/actor:np.float64(0.046688926438285976) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.45004653930664) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:2 - critic/score/mean:0.390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009245882742106915 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009245882742106915 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:283.91015625 - response_length/max:1070.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:283.91015625 - response_length_non_aborted/max:1070.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.798258543014526e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8690441809594631) - timing_s/agent_loop/generate_sequences/max:np.float64(8.087268119677901) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.444383415451739) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.087268119677901) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1070 - timing_s/gen:14.945382874459028 - timing_s/reward:0.0586177296936512 - timing_s/old_log_prob:2.9456582106649876 - timing_s/adv:0.5022626649588346 - timing_s/update_actor:21.097876647487283 - timing_s/step:39.637251829728484 - timing_s/stop_profile:0.00011603906750679016 - timing_per_token_ms/update_actor:0.13943938830499508 - timing_per_token_ms/gen:0.20562984651365596 - timing_per_token_ms/adv:0.003319537787639765 - perf/total_num_tokens:151305 - perf/time_per_step:39.637251829728484 - perf/throughput:477.1553053487653 (TaskRunner pid=2377588) Training Progress: 34%|███▍ | 34/100 [36:35<53:49, 48.93s/it] (TaskRunner pid=2377588) step:35 - global_seqlen/min:14866 - global_seqlen/max:24849 - global_seqlen/minmax_diff:9983 - global_seqlen/balanced_min:19413 - global_seqlen/balanced_max:19435 - global_seqlen/mean:19421.25 - actor/entropy:0.483135849237442 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15418004989624023 - training/rollout_probs_diff_mean:0.006350232753902674 - training/rollout_probs_diff_std:0.012629092670977116 - training/rollout_actor_probs_pearson_corr:0.9989728331565857 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999948143959045 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6271798610687256 - rollout_corr/rollout_is_min:0.5956010818481445 - rollout_corr/rollout_is_std:0.042342741042375565 - rollout_corr/rollout_is_eff_sample_size:0.9982103010502106 - rollout_corr/rollout_is_seq_mean:0.9999474883079529 - rollout_corr/rollout_is_seq_std:0.0026951427571475506 - rollout_corr/rollout_is_seq_max:1.006748080253601 - rollout_corr/rollout_is_seq_min:0.9912431836128235 - rollout_corr/rollout_is_seq_max_deviation:0.008756816387176514 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6359317428432405) - rollout_corr/training_log_ppl:np.float64(0.4859962440095842) - rollout_corr/kl:np.float64(0.00122511935049463) - rollout_corr/k3_kl:np.float64(0.000987188733347466) - rollout_corr/rollout_ppl:np.float64(1.6338720610365272) - rollout_corr/rollout_log_ppl:np.float64(0.48477112455293536) - rollout_corr/log_ppl_diff:np.float64(0.0012251194566488266) - rollout_corr/log_ppl_abs_diff:np.float64(0.002485145116224885) - rollout_corr/log_ppl_diff_max:np.float64(0.008322000503540039) - rollout_corr/log_ppl_diff_min:np.float64(-0.005573272705078125) - rollout_corr/ppl_ratio:np.float64(1.0012299029622227) - rollout_corr/chi2_token:np.float64(0.001508611487224698) - rollout_corr/chi2_seq:np.float64(0.5689334736671299) - actor/pg_loss:np.float64(0.00848642096389085) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011056569219363155) - actor/ppo_kl:np.float64(0.00026043441236822673) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(29381.75686307368) - self_distillation/token_reweight_w_std:np.float64(481742.71656613064) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014087620656937) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19030485482653603) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.4803352430462837) - perf/mfu/actor:np.float64(0.04726025487320032) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.48976516723633) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:2 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015392587520182133 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015392587520182133 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:288.3515625 - response_length/max:890.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:288.3515625 - response_length_non_aborted/max:890.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.5625 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.720772504806519e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9109692052006721) - timing_s/agent_loop/generate_sequences/max:np.float64(7.1837674379348755) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.433367057084979) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.1837674379348755) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:890 - timing_s/gen:14.348391022533178 - timing_s/reward:0.058897338807582855 - timing_s/old_log_prob:2.978777900338173 - timing_s/adv:0.5391388908028603 - timing_s/update_actor:21.11523146368563 - timing_s/step:39.12836896441877 - timing_s/stop_profile:0.00011306256055831909 - timing_per_token_ms/update_actor:0.13590288642392762 - timing_per_token_ms/gen:0.19437523398809475 - timing_per_token_ms/adv:0.0034700321220496894 - perf/total_num_tokens:155370 - perf/time_per_step:39.12836896441877 - perf/throughput:496.34703704773995 (TaskRunner pid=2377588) Training Progress: 35%|███▌ | 35/100 [37:14<49:50, 46.00s/it] (TaskRunner pid=2377588) step:36 - global_seqlen/min:16456 - global_seqlen/max:23654 - global_seqlen/minmax_diff:7198 - global_seqlen/balanced_min:19423 - global_seqlen/balanced_max:22349 - global_seqlen/mean:19824.0 - actor/entropy:0.46020257472991943 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2821692228317261 - training/rollout_probs_diff_mean:0.005832870025187731 - training/rollout_probs_diff_std:0.012065492570400238 - training/rollout_actor_probs_pearson_corr:0.999054491519928 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:1.0000755786895752 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2560289178509265e-05 - rollout_corr/rollout_is_max:1.6291385889053345 - rollout_corr/rollout_is_min:0.177614226937294 - rollout_corr/rollout_is_std:0.040680013597011566 - rollout_corr/rollout_is_eff_sample_size:0.9983479895064427 - rollout_corr/rollout_is_seq_mean:1.0001189708709717 - rollout_corr/rollout_is_seq_std:0.0023579420521855354 - rollout_corr/rollout_is_seq_max:1.0070582628250122 - rollout_corr/rollout_is_seq_min:0.9934430718421936 - rollout_corr/rollout_is_seq_max_deviation:0.007058262825012207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6329674059525132) - rollout_corr/training_log_ppl:np.float64(0.48337619690573774) - rollout_corr/kl:np.float64(0.0008564960014898482) - rollout_corr/k3_kl:np.float64(0.0010357112777512612) - rollout_corr/rollout_ppl:np.float64(1.6315131476148963) - rollout_corr/rollout_log_ppl:np.float64(0.48251969991542865) - rollout_corr/log_ppl_diff:np.float64(0.0008564969903090969) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022688616445520893) - rollout_corr/log_ppl_diff_max:np.float64(0.011876553297042847) - rollout_corr/log_ppl_diff_min:np.float64(-0.010160982608795166) - rollout_corr/ppl_ratio:np.float64(1.0008609551005065) - rollout_corr/chi2_token:np.float64(0.0024111124221235514) - rollout_corr/chi2_seq:np.float64(0.8620115192607045) - actor/pg_loss:np.float64(0.010058529442176223) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011524870928951714) - actor/ppo_kl:np.float64(9.863854258895799e-05) - actor/pg_clipfrac_lower:np.float64(1.0018594593930175e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.390625) - self_distillation/token_reweight_w_mean:np.float64(23227.686136340257) - self_distillation/token_reweight_w_std:np.float64(402868.9956974654) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00154262711294) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19054086284450023) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.5297933220863342) - perf/mfu/actor:np.float64(0.04711285479899424) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.57667541503906) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:2 - critic/score/mean:0.390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034289590548723936 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034289590548723936 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:311.0 - response_length/max:3899.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:311.0 - response_length_non_aborted/max:3899.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:308.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.546242654323578e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9752626102417707) - timing_s/agent_loop/generate_sequences/max:np.float64(23.855083597823977) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.721207518821757) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.855083597823977) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:3899 - timing_s/gen:30.444381792098284 - timing_s/reward:0.059937192127108574 - timing_s/old_log_prob:3.1676184479147196 - timing_s/adv:0.5165410302579403 - timing_s/update_actor:21.752410136163235 - timing_s/step:56.02927944436669 - timing_s/stop_profile:0.00011079385876655579 - timing_per_token_ms/update_actor:0.1371595675454199 - timing_per_token_ms/gen:0.3823902455800126 - timing_per_token_ms/adv:0.0032570434212188526 - perf/total_num_tokens:158592 - perf/time_per_step:56.02927944436669 - perf/throughput:353.8150088059565 (TaskRunner pid=2377588) Training Progress: 36%|███▌ | 36/100 [38:10<52:17, 49.02s/it] (TaskRunner pid=2377588) step:37 - global_seqlen/min:15067 - global_seqlen/max:24455 - global_seqlen/minmax_diff:9388 - global_seqlen/balanced_min:19234 - global_seqlen/balanced_max:19247 - global_seqlen/mean:19241.0 - actor/entropy:0.4648912250995636 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31236985325813293 - training/rollout_probs_diff_mean:0.006110828835517168 - training/rollout_probs_diff_std:0.01255817711353302 - training/rollout_actor_probs_pearson_corr:0.9989880919456482 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.79296875 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.79296875 - rollout_corr/rollout_is_mean:1.000205159187317 - rollout_corr/rollout_is_ratio_fraction_high:1.2783800229954068e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2783800229954068e-05 - rollout_corr/rollout_is_max:2.195718288421631 - rollout_corr/rollout_is_min:0.4073386490345001 - rollout_corr/rollout_is_std:0.04194674268364906 - rollout_corr/rollout_is_eff_sample_size:0.9982441553459014 - rollout_corr/rollout_is_seq_mean:1.0002442598342896 - rollout_corr/rollout_is_seq_std:0.0025654735509306192 - rollout_corr/rollout_is_seq_max:1.0082298517227173 - rollout_corr/rollout_is_seq_min:0.9938198924064636 - rollout_corr/rollout_is_seq_max_deviation:0.008229851722717285 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6210425379686058) - rollout_corr/training_log_ppl:np.float64(0.47613325994461775) - rollout_corr/kl:np.float64(0.0008513373754652775) - rollout_corr/k3_kl:np.float64(0.001013167244309443) - rollout_corr/rollout_ppl:np.float64(1.6196617097593844) - rollout_corr/rollout_log_ppl:np.float64(0.4752819243003614) - rollout_corr/log_ppl_diff:np.float64(0.0008513356442563236) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021160157048143446) - rollout_corr/log_ppl_diff_max:np.float64(0.008521497249603271) - rollout_corr/log_ppl_diff_min:np.float64(-0.006072044372558594) - rollout_corr/ppl_ratio:np.float64(1.000855072401464) - rollout_corr/chi2_token:np.float64(0.0023777401074767113) - rollout_corr/chi2_seq:np.float64(0.6142277545295656) - actor/pg_loss:np.float64(0.011070344713516533) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009973425271709857) - actor/ppo_kl:np.float64(0.00018685973462595484) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.79296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(16930.707895939704) - self_distillation/token_reweight_w_std:np.float64(287963.5889257451) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0042877539526671) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1920611820241902) - self_distillation/empty_target_batch:np.float64(0.20703125) - actor/grad_norm:np.float64(0.47131136804819107) - perf/mfu/actor:np.float64(0.04664556260582101) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.45281219482422) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:2 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011570937000215054 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011570937000215054 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:305.5625 - response_length/max:1061.0 - response_length/min:170.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.5625 - response_length_non_aborted/max:1061.0 - response_length_non_aborted/min:170.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014707818627357483 - timing_s/agent_loop/generate_sequences/min:np.float64(2.10099833086133) - timing_s/agent_loop/generate_sequences/max:np.float64(8.151288989931345) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6658011424078722) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.151288989931345) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1061 - timing_s/gen:14.92355296574533 - timing_s/reward:0.05888610519468784 - timing_s/old_log_prob:2.943532507866621 - timing_s/adv:0.5621297135949135 - timing_s/update_actor:21.135435147210956 - timing_s/step:39.70921685732901 - timing_s/stop_profile:0.0001109689474105835 - timing_per_token_ms/update_actor:0.13730728098338804 - timing_per_token_ms/gen:0.19077972189795114 - timing_per_token_ms/adv:0.003651900327392765 - perf/total_num_tokens:153928 - perf/time_per_step:39.70921685732901 - perf/throughput:484.54745579926356 (TaskRunner pid=2377588) Training Progress: 37%|███▋ | 37/100 [38:50<48:33, 46.25s/it] (TaskRunner pid=2377588) step:38 - global_seqlen/min:19970 - global_seqlen/max:23426 - global_seqlen/minmax_diff:3456 - global_seqlen/balanced_min:20959 - global_seqlen/balanced_max:21378 - global_seqlen/mean:21016.875 - actor/entropy:0.44162091612815857 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3749816119670868 - training/rollout_probs_diff_mean:0.006124292034655809 - training/rollout_probs_diff_std:0.012841424904763699 - training/rollout_actor_probs_pearson_corr:0.9989215731620789 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.44140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999114274978638 - rollout_corr/rollout_is_ratio_fraction_high:2.5306524548796006e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.3016817569732666 - rollout_corr/rollout_is_min:0.5254566669464111 - rollout_corr/rollout_is_std:0.04207726567983627 - rollout_corr/rollout_is_eff_sample_size:0.9982323951723706 - rollout_corr/rollout_is_seq_mean:0.9999388456344604 - rollout_corr/rollout_is_seq_std:0.0027120537124574184 - rollout_corr/rollout_is_seq_max:1.007246971130371 - rollout_corr/rollout_is_seq_min:0.991539716720581 - rollout_corr/rollout_is_seq_max_deviation:0.008460283279418945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.578642808366567) - rollout_corr/training_log_ppl:np.float64(0.45133330079261214) - rollout_corr/kl:np.float64(0.0012311862048299815) - rollout_corr/k3_kl:np.float64(0.0010565676156488735) - rollout_corr/rollout_ppl:np.float64(1.5766425835900009) - rollout_corr/rollout_log_ppl:np.float64(0.4501021119649522) - rollout_corr/log_ppl_diff:np.float64(0.0012311888276599348) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026498145307414234) - rollout_corr/log_ppl_diff_max:np.float64(0.009859323501586914) - rollout_corr/log_ppl_diff_min:np.float64(-0.007890194654464722) - rollout_corr/ppl_ratio:np.float64(1.0012366939336061) - rollout_corr/chi2_token:np.float64(0.0017297896556556225) - rollout_corr/chi2_seq:np.float64(0.9841942663770169) - actor/pg_loss:np.float64(0.01014754525385797) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015553931343674776) - actor/ppo_kl:np.float64(0.00025599947714916027) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44140625) - self_distillation/token_reweight_w_mean:np.float64(28873.09791884478) - self_distillation/token_reweight_w_std:np.float64(456959.85658787086) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009312713518739) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19558428326854482) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.4880059137940407) - perf/mfu/actor:np.float64(0.050717615398884225) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.5504493713379) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:2 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015644177794456482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015644177794456482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:308.71484375 - response_length/max:1441.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.71484375 - response_length_non_aborted/max:1441.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:348.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.9254230260849e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0638767201453447) - timing_s/agent_loop/generate_sequences/max:np.float64(10.672797536477447) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.788226707125432) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.672797536477447) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:1441 - timing_s/gen:17.639124050736427 - timing_s/reward:0.06129726581275463 - timing_s/old_log_prob:3.0924447868019342 - timing_s/adv:0.5303551685065031 - timing_s/update_actor:21.261099139228463 - timing_s/step:42.67122959904373 - timing_s/stop_profile:0.00011066533625125885 - timing_per_token_ms/update_actor:0.12645254788847332 - timing_per_token_ms/gen:0.22319246942005577 - timing_per_token_ms/adv:0.003154341264498784 - perf/total_num_tokens:168135 - perf/time_per_step:42.67122959904373 - perf/throughput:492.5303347825485 (TaskRunner pid=2377588) Training Progress: 38%|███▊ | 38/100 [39:32<46:41, 45.19s/it] (TaskRunner pid=2377588) step:39 - global_seqlen/min:18443 - global_seqlen/max:22812 - global_seqlen/minmax_diff:4369 - global_seqlen/balanced_min:19977 - global_seqlen/balanced_max:19987 - global_seqlen/mean:19983.375 - actor/entropy:0.4740733802318573 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34833580255508423 - training/rollout_probs_diff_mean:0.006365400739014149 - training/rollout_probs_diff_std:0.012884997762739658 - training/rollout_actor_probs_pearson_corr:0.9989339709281921 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.9997560977935791 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2593982319231145e-05 - rollout_corr/rollout_is_max:1.5982438325881958 - rollout_corr/rollout_is_min:0.3015452027320862 - rollout_corr/rollout_is_std:0.042863890528678894 - rollout_corr/rollout_is_eff_sample_size:0.9981651655610857 - rollout_corr/rollout_is_seq_mean:0.9997711181640625 - rollout_corr/rollout_is_seq_std:0.002585791517049074 - rollout_corr/rollout_is_seq_max:1.005784273147583 - rollout_corr/rollout_is_seq_min:0.9936142563819885 - rollout_corr/rollout_is_seq_max_deviation:0.006385743618011475 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6230566296726465) - rollout_corr/training_log_ppl:np.float64(0.4783219291130081) - rollout_corr/kl:np.float64(0.0016783186468458666) - rollout_corr/k3_kl:np.float64(0.0013566379532221617) - rollout_corr/rollout_ppl:np.float64(1.6202314072288573) - rollout_corr/rollout_log_ppl:np.float64(0.4766436066129245) - rollout_corr/log_ppl_diff:np.float64(0.0016783225000835955) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026711929240264) - rollout_corr/log_ppl_diff_max:np.float64(0.011705935001373291) - rollout_corr/log_ppl_diff_min:np.float64(-0.004385650157928467) - rollout_corr/ppl_ratio:np.float64(1.0016840817406774) - rollout_corr/chi2_token:np.float64(0.0019611280877143145) - rollout_corr/chi2_seq:np.float64(0.3429665556177497) - actor/pg_loss:np.float64(0.016169345937669277) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0033310484145658847) - actor/ppo_kl:np.float64(0.0004999858873553364) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(22027.251874742797) - self_distillation/token_reweight_w_std:np.float64(370863.1111371557) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0041118303779513) - self_distillation/token_reweight_w_clip_frac:np.float64(0.25385374887264334) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.6407449543476105) - perf/mfu/actor:np.float64(0.0482990317839901) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.48219299316406) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:2 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034932559356093407 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034932559356093407 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:310.16796875 - response_length/max:833.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:310.16796875 - response_length_non_aborted/max:833.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.3125 - prompt_length/max:498.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015905685722827911 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2486149854958057) - timing_s/agent_loop/generate_sequences/max:np.float64(6.902357917279005) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7572810447891243) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.902357917279005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:833 - timing_s/gen:13.872650580480695 - timing_s/reward:0.06029580160975456 - timing_s/old_log_prob:3.0033159237354994 - timing_s/adv:0.5526164472103119 - timing_s/update_actor:21.469840431585908 - timing_s/step:39.046306617558 - timing_s/stop_profile:0.00011818297207355499 - timing_per_token_ms/update_actor:0.1342981380246449 - timing_per_token_ms/gen:0.17471191995870047 - timing_per_token_ms/adv:0.003456726198717133 - perf/total_num_tokens:159867 - perf/time_per_step:39.046306617558 - perf/throughput:511.7865614212549 (TaskRunner pid=2377588) Training Progress: 39%|███▉ | 39/100 [40:11<44:05, 43.36s/it] (TaskRunner pid=2377588) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2377588) validation generation end (TaskRunner pid=2377588) [prompt] system (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2377588) (TaskRunner pid=2377588) A (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) user (TaskRunner pid=2377588) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2377588) (TaskRunner pid=2377588) A: -0.01 (TaskRunner pid=2377588) B: 1.69 (TaskRunner pid=2377588) C: 2.49 (TaskRunner pid=2377588) D: 0.45 (TaskRunner pid=2377588) Please reason step by step. (TaskRunner pid=2377588) assistant (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) [response] (TaskRunner pid=2377588) To determine the folding stability score of the given protein sequence, we need to evaluate the likelihood of the protein forming a stable, functional structure. This involves assessing the amino acid composition, secondary structure potential, and overall hydrophobicity/hydrophilicity balance. (TaskRunner pid=2377588) (TaskRunner pid=2377588) The sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR' contains a mix of hydrophilic and hydrophobic residues. However, it also includes several charged residues (like lysine, arginine, and glutamic acid) and repetitive patterns, which may indicate regions of high flexibility or potential for forming structured domains. (TaskRunner pid=2377588) (TaskRunner pid=2377588) A folding stability score typically ranges from negative (unstable) to positive (stable). A score of -0.01 suggests minimal instability, which is not very informative. A score of 1.69 indicates moderate stability, while 2.49 suggests a high degree of stability. A score of 0.45 is relatively low but still positive, suggesting some stability. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given the presence of multiple charged residues and a relatively long sequence, the score is more likely to be higher than 0.45 but not extremely high. A score of 1.69 is reasonable for a sequence that may form some stable secondary structures but is not fully folded or highly stable. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Therefore, the most reasonable answer is B: 1.69. (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) B (TaskRunner pid=2377588) (TaskRunner pid=2377588) [ground_truth] B (TaskRunner pid=2377588) [score] 1.0 (TaskRunner pid=2377588) [acc] 1.0 (TaskRunner pid=2377588) [pred] B (TaskRunner pid=2377588) [incorrect_format] 1 (TaskRunner pid=2377588) [feedback] (TaskRunner pid=2377588) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2377588) step:40 - global_seqlen/min:16924 - global_seqlen/max:22815 - global_seqlen/minmax_diff:5891 - global_seqlen/balanced_min:19431 - global_seqlen/balanced_max:19444 - global_seqlen/mean:19439.125 - actor/entropy:0.44513219594955444 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19883915781974792 - training/rollout_probs_diff_mean:0.006402170285582542 - training/rollout_probs_diff_std:0.013032508082687855 - training/rollout_actor_probs_pearson_corr:0.9988617300987244 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8203125 - self_distillation/correct_sample_fraction:0.4453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8203125 - rollout_corr/rollout_is_mean:0.9999405145645142 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.341039842372993e-05 - rollout_corr/rollout_is_max:1.5872390270233154 - rollout_corr/rollout_is_min:0.37882742285728455 - rollout_corr/rollout_is_std:0.04299648478627205 - rollout_corr/rollout_is_eff_sample_size:0.9981545949443588 - rollout_corr/rollout_is_seq_mean:0.9998748898506165 - rollout_corr/rollout_is_seq_std:0.002650556853041053 - rollout_corr/rollout_is_seq_max:1.0072165727615356 - rollout_corr/rollout_is_seq_min:0.9915069341659546 - rollout_corr/rollout_is_seq_max_deviation:0.00849306583404541 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5746450014412403) - rollout_corr/training_log_ppl:np.float64(0.44897763611515984) - rollout_corr/kl:np.float64(0.0011384600795034672) - rollout_corr/k3_kl:np.float64(0.0011895181490899631) - rollout_corr/rollout_ppl:np.float64(1.5728909838944674) - rollout_corr/rollout_log_ppl:np.float64(0.44783917599124834) - rollout_corr/log_ppl_diff:np.float64(0.0011384601239115) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024658453185111284) - rollout_corr/log_ppl_diff_max:np.float64(0.009177625179290771) - rollout_corr/log_ppl_diff_min:np.float64(-0.009199351072311401) - rollout_corr/ppl_ratio:np.float64(1.0011434056796134) - rollout_corr/chi2_token:np.float64(0.002606436377391219) - rollout_corr/chi2_seq:np.float64(2.0256223927717656) - actor/pg_loss:np.float64(0.01009189069736749) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011032083666577819) - actor/ppo_kl:np.float64(6.541757507250168e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4453125) - self_distillation/token_reweight_w_mean:np.float64(38846.50039524678) - self_distillation/token_reweight_w_std:np.float64(629609.1893131913) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0045572421513498) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19737870804965496) - self_distillation/empty_target_batch:np.float64(0.1796875) - actor/grad_norm:np.float64(0.4980562701821327) - perf/mfu/actor:np.float64(0.04722880308717044) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.4748306274414) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4125) - val-aux/sciknoweval/reward/std@16:np.float64(0.34117903969277913) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.55796) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.30197247322095855) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.26738) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2766511209522595) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.41253999999999996) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3418744138405596) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6773000000000002) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.22835773262501724) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.15608) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.18600637725950755) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.4293) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2836007701626556) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.76506) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.15556186702441197) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.08439999999999999) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10199377009945527) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4406400000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.21453439034116106) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8266000000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.09814987871221652) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.04977999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04041596183299681) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.44758000000000003) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.15560076058651195) - val-aux/sciknoweval/score/mean@16:np.float64(0.4125) - val-aux/sciknoweval/score/std@16:np.float64(0.34117903969277913) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.55796) - val-aux/sciknoweval/score/best@2/std:np.float64(0.30197247322095855) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.26738) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2766511209522595) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.41253999999999996) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3418744138405596) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6773000000000002) - val-aux/sciknoweval/score/best@4/std:np.float64(0.22835773262501724) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.15608) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.18600637725950755) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.4293) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2836007701626556) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.76506) - val-aux/sciknoweval/score/best@8/std:np.float64(0.15556186702441197) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.08439999999999999) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10199377009945527) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4406400000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.21453439034116106) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8266000000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.09814987871221652) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.04977999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04041596183299681) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.44758000000000003) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.15560076058651195) - val-core/sciknoweval/acc/mean@16:np.float64(0.4125) - val-aux/sciknoweval/acc/std@16:np.float64(0.34117903969277913) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.55796) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.30197247322095855) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.26738) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2766511209522595) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.41253999999999996) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3418744138405596) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6773000000000002) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.22835773262501724) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.15608) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.18600637725950755) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.4293) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2836007701626556) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.76506) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.15556186702441197) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.08439999999999999) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10199377009945527) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4406400000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.21453439034116106) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8266000000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.09814987871221652) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.04977999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04041596183299681) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.44758000000000003) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.15560076058651195) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.027752444103600768) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99908) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.009010070513757872) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.98342) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03645057038222041) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99118) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.027923193032685978) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.96968) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.044900412420776144) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9971599999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.015987199308620758) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.94904) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04841025821716694) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99944) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.006463864121392813) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9264) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.041825247908282515) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99996) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0008935323161475471) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:2 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-3.855489558191039e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-3.855489558191039e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.28515625 - response_length/max:874.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.28515625 - response_length_non_aborted/max:874.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:316.1875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.173100650310516e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.118532342836261) - timing_s/agent_loop/generate_sequences/max:np.float64(7.155706750229001) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.614957957812294) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.155706750229001) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:874 - timing_s/gen:13.993909772485495 - timing_s/reward:0.058549365028738976 - timing_s/old_log_prob:3.071204725652933 - timing_s/adv:0.5231851823627949 - timing_s/update_actor:21.199190955609083 - timing_s/step:38.93401595763862 - timing_s/testing:85.03915292955935 - timing_s/save_checkpoint:13.287784896790981 - timing_s/stop_profile:0.00015090405941009521 - timing_per_token_ms/update_actor:0.13631780594296994 - timing_per_token_ms/gen:0.1876639055436642 - timing_per_token_ms/adv:0.0033642536788744018 - perf/total_num_tokens:155513 - perf/time_per_step:38.93401595763862 - perf/throughput:499.2838401553632 (TaskRunner pid=2377588) Training Progress: 40%|████ | 40/100 [42:29<1:11:33, 71.56s/it] (TaskRunner pid=2377588) step:41 - global_seqlen/min:15408 - global_seqlen/max:22983 - global_seqlen/minmax_diff:7575 - global_seqlen/balanced_min:20422 - global_seqlen/balanced_max:20494 - global_seqlen/mean:20432.625 - actor/entropy:0.45965611934661865 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413528561592102 - training/rollout_probs_diff_mean:0.006295996252447367 - training/rollout_probs_diff_std:0.012785677798092365 - training/rollout_actor_probs_pearson_corr:0.9989537000656128 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.45703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:1.0001287460327148 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9511066675186157 - rollout_corr/rollout_is_min:0.5370932221412659 - rollout_corr/rollout_is_std:0.042504314333200455 - rollout_corr/rollout_is_eff_sample_size:0.9981972350941091 - rollout_corr/rollout_is_seq_mean:1.0001194477081299 - rollout_corr/rollout_is_seq_std:0.002400510711595416 - rollout_corr/rollout_is_seq_max:1.0069642066955566 - rollout_corr/rollout_is_seq_min:0.9941142201423645 - rollout_corr/rollout_is_seq_max_deviation:0.006964206695556641 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6045056390576065) - rollout_corr/training_log_ppl:np.float64(0.4663193255546503) - rollout_corr/kl:np.float64(0.0011170693861970449) - rollout_corr/k3_kl:np.float64(0.0010359886136939167) - rollout_corr/rollout_ppl:np.float64(1.6027684807777405) - rollout_corr/rollout_log_ppl:np.float64(0.46520225348649547) - rollout_corr/log_ppl_diff:np.float64(0.0011170720681548119) - rollout_corr/log_ppl_abs_diff:np.float64(0.002356308395974338) - rollout_corr/log_ppl_diff_max:np.float64(0.008682578802108765) - rollout_corr/log_ppl_diff_min:np.float64(-0.00811341404914856) - rollout_corr/ppl_ratio:np.float64(1.0011216211132705) - rollout_corr/chi2_token:np.float64(0.0019299194682389498) - rollout_corr/chi2_seq:np.float64(1.121945014456287) - actor/pg_loss:np.float64(0.011288162786513567) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012192110252726707) - actor/ppo_kl:np.float64(0.00031442064742748244) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.45703125) - self_distillation/token_reweight_w_mean:np.float64(41760.65258256742) - self_distillation/token_reweight_w_std:np.float64(727182.3370472678) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0036190431565046) - self_distillation/token_reweight_w_clip_frac:np.float64(0.21599252949818037) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.5489989519119263) - perf/mfu/actor:np.float64(0.050270106983118526) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(203.2335548400879) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:2 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00123530940618366 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.00123530940618366 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:300.80078125 - response_length/max:914.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:300.80078125 - response_length_non_aborted/max:914.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:337.71875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.553290247917175e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.364460725337267) - timing_s/agent_loop/generate_sequences/max:np.float64(7.804874313995242) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8609984184149653) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.804874313995242) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:914 - timing_s/gen:14.103510553017259 - timing_s/reward:0.05593244358897209 - timing_s/old_log_prob:3.0655716191977262 - timing_s/adv:0.5167239084839821 - timing_s/update_actor:20.978015588596463 - timing_s/step:38.789996990934014 - timing_s/stop_profile:0.00010593608021736145 - timing_per_token_ms/update_actor:0.1283365181211204 - timing_per_token_ms/gen:0.18315058181958652 - timing_per_token_ms/adv:0.0031611449121440715 - perf/total_num_tokens:163461 - perf/time_per_step:38.789996990934014 - perf/throughput:526.7498475128912 (TaskRunner pid=2377588) Training Progress: 41%|████ | 41/100 [43:08<1:00:42, 61.74s/it] (TaskRunner pid=2377588) step:42 - global_seqlen/min:15134 - global_seqlen/max:20305 - global_seqlen/minmax_diff:5171 - global_seqlen/balanced_min:17959 - global_seqlen/balanced_max:18003 - global_seqlen/mean:17968.25 - actor/entropy:0.4570571482181549 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7748275399208069 - training/rollout_probs_diff_mean:0.006258582696318626 - training/rollout_probs_diff_std:0.01304932776838541 - training/rollout_actor_probs_pearson_corr:0.998887836933136 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.69921875 - self_distillation/correct_sample_fraction:0.3515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.69921875 - rollout_corr/rollout_is_mean:0.9998893737792969 - rollout_corr/rollout_is_ratio_fraction_high:1.2583997886395082e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2583997886395082e-05 - rollout_corr/rollout_is_max:2.284717082977295 - rollout_corr/rollout_is_min:0.22325187921524048 - rollout_corr/rollout_is_std:0.042170654982328415 - rollout_corr/rollout_is_eff_sample_size:0.998224436424269 - rollout_corr/rollout_is_seq_mean:0.9999182224273682 - rollout_corr/rollout_is_seq_std:0.0025441586039960384 - rollout_corr/rollout_is_seq_max:1.00813627243042 - rollout_corr/rollout_is_seq_min:0.9901077151298523 - rollout_corr/rollout_is_seq_max_deviation:0.009892284870147705 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5881843524985015) - rollout_corr/training_log_ppl:np.float64(0.457741612684913) - rollout_corr/kl:np.float64(0.001088180524718041) - rollout_corr/k3_kl:np.float64(0.001009936439345438) - rollout_corr/rollout_ppl:np.float64(1.5864198198541999) - rollout_corr/rollout_log_ppl:np.float64(0.45665342919528484) - rollout_corr/log_ppl_diff:np.float64(0.0010881834896281362) - rollout_corr/log_ppl_abs_diff:np.float64(0.002267185249365866) - rollout_corr/log_ppl_diff_max:np.float64(0.011401891708374023) - rollout_corr/log_ppl_diff_min:np.float64(-0.007148087024688721) - rollout_corr/ppl_ratio:np.float64(1.0010924919042736) - rollout_corr/chi2_token:np.float64(0.0018666633404791355) - rollout_corr/chi2_seq:np.float64(0.835359200835228) - actor/pg_loss:np.float64(0.008744741440750659) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012483641817198077) - actor/ppo_kl:np.float64(9.679444900534406e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.69921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3515625) - self_distillation/token_reweight_w_mean:np.float64(34454.47747365851) - self_distillation/token_reweight_w_std:np.float64(616857.0199022216) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0046089186798781) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17166461775195785) - self_distillation/empty_target_batch:np.float64(0.30078125) - actor/grad_norm:np.float64(0.43421047180891037) - perf/mfu/actor:np.float64(0.04456099608373405) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.6717643737793) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:2 - critic/score/mean:0.3515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007298719137907028 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007298719137907028 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:310.4140625 - response_length/max:805.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:310.4140625 - response_length_non_aborted/max:805.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.09375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.766779839992523e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.295503880828619) - timing_s/agent_loop/generate_sequences/max:np.float64(6.7998224049806595) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7119227322618826) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.7998224049806595) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:805 - timing_s/gen:13.797013998031616 - timing_s/reward:0.057562943547964096 - timing_s/old_log_prob:2.9283091723918915 - timing_s/adv:0.4927501007914543 - timing_s/update_actor:20.904700068756938 - timing_s/step:38.26865663006902 - timing_s/stop_profile:2.847425639629364e-05 - timing_per_token_ms/update_actor:0.14542804717179567 - timing_per_token_ms/gen:0.17362159914971959 - timing_per_token_ms/adv:0.0034279221737749522 - perf/total_num_tokens:143746 - perf/time_per_step:38.26865663006902 - perf/throughput:469.52915472558607 (TaskRunner pid=2377588) Training Progress: 42%|████▏ | 42/100 [43:46<52:52, 54.70s/it] (TaskRunner pid=2377588) step:43 - global_seqlen/min:16542 - global_seqlen/max:24995 - global_seqlen/minmax_diff:8453 - global_seqlen/balanced_min:20349 - global_seqlen/balanced_max:24018 - global_seqlen/mean:20827.25 - actor/entropy:0.44059035181999207 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5086237192153931 - training/rollout_probs_diff_mean:0.006040246691554785 - training/rollout_probs_diff_std:0.012639202177524567 - training/rollout_actor_probs_pearson_corr:0.9989492297172546 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.999949038028717 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.35798997891834e-05 - rollout_corr/rollout_is_max:1.6296545267105103 - rollout_corr/rollout_is_min:0.39326563477516174 - rollout_corr/rollout_is_std:0.04117661714553833 - rollout_corr/rollout_is_eff_sample_size:0.9983069997574627 - rollout_corr/rollout_is_seq_mean:0.9999187588691711 - rollout_corr/rollout_is_seq_std:0.002482477342709899 - rollout_corr/rollout_is_seq_max:1.0058101415634155 - rollout_corr/rollout_is_seq_min:0.991828203201294 - rollout_corr/rollout_is_seq_max_deviation:0.008171796798706055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.590677146334201) - rollout_corr/training_log_ppl:np.float64(0.4580551938561257) - rollout_corr/kl:np.float64(0.0012790096782708815) - rollout_corr/k3_kl:np.float64(0.000981491270749757) - rollout_corr/rollout_ppl:np.float64(1.588688905350864) - rollout_corr/rollout_log_ppl:np.float64(0.45677618618356064) - rollout_corr/log_ppl_diff:np.float64(0.0012790076725650579) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022130336437840015) - rollout_corr/log_ppl_diff_max:np.float64(0.009368181228637695) - rollout_corr/log_ppl_diff_min:np.float64(-0.006417810916900635) - rollout_corr/ppl_ratio:np.float64(1.0012832230422646) - rollout_corr/chi2_token:np.float64(0.001390925608575344) - rollout_corr/chi2_seq:np.float64(0.2726682838983834) - actor/pg_loss:np.float64(0.01187965925782919) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010726483542384813) - actor/ppo_kl:np.float64(0.00030101987926367535) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(38002.45210678852) - self_distillation/token_reweight_w_std:np.float64(666991.4729211024) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008108539041132) - self_distillation/token_reweight_w_clip_frac:np.float64(0.20847218368726317) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.48353035748004913) - perf/mfu/actor:np.float64(0.049177262371158326) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.6334376335144) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:3 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02291671559214592 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02291671559214592 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:331.3203125 - response_length/max:4610.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.3203125 - response_length_non_aborted/max:4610.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:319.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002294108271598816 - timing_s/agent_loop/generate_sequences/min:np.float64(2.300520248711109) - timing_s/agent_loop/generate_sequences/max:np.float64(28.413594864308834) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8569238531708834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.413594864308834) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:4610 - timing_s/gen:35.49145572446287 - timing_s/reward:0.05434819869697094 - timing_s/old_log_prob:3.323965821415186 - timing_s/adv:0.6106252055615187 - timing_s/update_actor:21.689782194793224 - timing_s/step:61.283970488235354 - timing_s/stop_profile:0.00012102536857128143 - timing_per_token_ms/update_actor:0.1301767047665512 - timing_per_token_ms/gen:0.41844249716407916 - timing_per_token_ms/adv:0.003664821361206584 - perf/total_num_tokens:166618 - perf/time_per_step:61.283970488235354 - perf/throughput:339.8482479851431 (TaskRunner pid=2377588) Training Progress: 43%|████▎ | 43/100 [44:48<53:57, 56.80s/it] (TaskRunner pid=2377588) step:44 - global_seqlen/min:17595 - global_seqlen/max:23070 - global_seqlen/minmax_diff:5475 - global_seqlen/balanced_min:20450 - global_seqlen/balanced_max:20464 - global_seqlen/mean:20460.0 - actor/entropy:0.474331796169281 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4704689383506775 - training/rollout_probs_diff_mean:0.0063726408407092094 - training/rollout_probs_diff_std:0.01279825996607542 - training/rollout_actor_probs_pearson_corr:0.9989205598831177 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.000135064125061 - rollout_corr/rollout_is_ratio_fraction_high:1.2362773304630537e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2362773304630537e-05 - rollout_corr/rollout_is_max:2.6121466159820557 - rollout_corr/rollout_is_min:0.24590198695659637 - rollout_corr/rollout_is_std:0.04267924651503563 - rollout_corr/rollout_is_eff_sample_size:0.9981822690555938 - rollout_corr/rollout_is_seq_mean:1.0001096725463867 - rollout_corr/rollout_is_seq_std:0.002360835438594222 - rollout_corr/rollout_is_seq_max:1.0070704221725464 - rollout_corr/rollout_is_seq_min:0.9907374978065491 - rollout_corr/rollout_is_seq_max_deviation:0.009262502193450928 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6223690561018884) - rollout_corr/training_log_ppl:np.float64(0.4782852207426913) - rollout_corr/kl:np.float64(0.0009240632728122478) - rollout_corr/k3_kl:np.float64(0.0010457007117565809) - rollout_corr/rollout_ppl:np.float64(1.6208787923678756) - rollout_corr/rollout_log_ppl:np.float64(0.4773611561395228) - rollout_corr/log_ppl_diff:np.float64(0.0009240646031685174) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021032296936027706) - rollout_corr/log_ppl_diff_max:np.float64(0.011524856090545654) - rollout_corr/log_ppl_diff_min:np.float64(-0.005751490592956543) - rollout_corr/ppl_ratio:np.float64(1.0009276950731874) - rollout_corr/chi2_token:np.float64(0.00238722562789917) - rollout_corr/chi2_seq:np.float64(0.8587617790326476) - actor/pg_loss:np.float64(0.01242549135349691) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011448220052443503) - actor/ppo_kl:np.float64(0.00010319172345418792) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(17835.628145632334) - self_distillation/token_reweight_w_std:np.float64(306406.5038258827) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0023843697272241) - self_distillation/token_reweight_w_clip_frac:np.float64(0.22484046139288694) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.5263424441218376) - perf/mfu/actor:np.float64(0.049945100206549506) - perf/max_memory_allocated_gb:np.float64(134.84235525131226) - perf/max_memory_reserved_gb:np.float64(149.34765625) - perf/cpu_memory_used_gb:np.float64(202.70220565795898) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:3 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012749109417200089 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012749109417200089 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:315.96875 - response_length/max:915.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.96875 - response_length_non_aborted/max:915.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:323.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000141192227602005 - timing_s/agent_loop/generate_sequences/min:np.float64(2.088733084499836) - timing_s/agent_loop/generate_sequences/max:np.float64(7.521113304421306) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8606676526833326) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.521113304421306) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:915 - timing_s/gen:14.371255427598953 - timing_s/reward:0.06165381893515587 - timing_s/old_log_prob:2.9580648466944695 - timing_s/adv:0.5170920882374048 - timing_s/update_actor:21.126081401482224 - timing_s/step:39.123630264773965 - timing_s/stop_profile:0.00012073665857315063 - timing_per_token_ms/update_actor:0.1290694122768953 - timing_per_token_ms/gen:0.17766857169912662 - timing_per_token_ms/adv:0.0031591647619587295 - perf/total_num_tokens:163680 - perf/time_per_step:39.123630264773965 - perf/throughput:522.9576054556911 (TaskRunner pid=2377588) Training Progress: 44%|████▍ | 44/100 [45:27<48:04, 51.51s/it] (TaskRunner pid=2377588) step:45 - global_seqlen/min:15296 - global_seqlen/max:31819 - global_seqlen/minmax_diff:16523 - global_seqlen/balanced_min:20142 - global_seqlen/balanced_max:27358 - global_seqlen/mean:21067.25 - actor/entropy:0.4163608253002167 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46165400743484497 - training/rollout_probs_diff_mean:0.005653797648847103 - training/rollout_probs_diff_std:0.012383944354951382 - training/rollout_actor_probs_pearson_corr:0.998989462852478 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.81640625 - self_distillation/correct_sample_fraction:0.40234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.81640625 - rollout_corr/rollout_is_mean:1.0001009702682495 - rollout_corr/rollout_is_ratio_fraction_high:1.1463420378277078e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2926840756554157e-05 - rollout_corr/rollout_is_max:2.334789752960205 - rollout_corr/rollout_is_min:0.3680537939071655 - rollout_corr/rollout_is_std:0.04027210548520088 - rollout_corr/rollout_is_eff_sample_size:0.9983810213763973 - rollout_corr/rollout_is_seq_mean:1.000077486038208 - rollout_corr/rollout_is_seq_std:0.002576942555606365 - rollout_corr/rollout_is_seq_max:1.0098541975021362 - rollout_corr/rollout_is_seq_min:0.9936208128929138 - rollout_corr/rollout_is_seq_max_deviation:0.00985419750213623 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5872353138402104) - rollout_corr/training_log_ppl:np.float64(0.4567617337452248) - rollout_corr/kl:np.float64(0.000973212498668552) - rollout_corr/k3_kl:np.float64(0.0009783451907594554) - rollout_corr/rollout_ppl:np.float64(1.5856868028640747) - rollout_corr/rollout_log_ppl:np.float64(0.45578852190374164) - rollout_corr/log_ppl_diff:np.float64(0.0009732118414831348) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021274683604133315) - rollout_corr/log_ppl_diff_max:np.float64(0.007531642913818359) - rollout_corr/log_ppl_diff_min:np.float64(-0.00891140103340149) - rollout_corr/ppl_ratio:np.float64(1.0009770693723112) - rollout_corr/chi2_token:np.float64(0.00200288207270205) - rollout_corr/chi2_seq:np.float64(0.6401831244584173) - actor/pg_loss:np.float64(0.008764656144194305) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008251814861068851) - actor/ppo_kl:np.float64(0.00015411172130441742) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.81640625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40234375) - self_distillation/token_reweight_w_mean:np.float64(29602.74064092664) - self_distillation/token_reweight_w_std:np.float64(509085.9035597397) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0053935272153467) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18625332941883244) - self_distillation/empty_target_batch:np.float64(0.18359375) - actor/grad_norm:np.float64(0.438546784222126) - perf/mfu/actor:np.float64(0.04704109183172239) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.59466934204102) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:3 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06530137360095978 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06530137360095978 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:340.7578125 - response_length/max:8192.0 - response_length/min:188.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:340.7578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:188.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:317.59375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.974969387054443e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3864134326577187) - timing_s/agent_loop/generate_sequences/max:np.float64(58.6604364477098) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.00799150961393) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(58.6604364477098) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:66.65385578013957 - timing_s/reward:0.061994725838303566 - timing_s/old_log_prob:3.199101507663727 - timing_s/adv:0.5954687744379044 - timing_s/update_actor:23.232400882989168 - timing_s/step:93.83052004128695 - timing_s/stop_profile:0.00010902434587478638 - timing_per_token_ms/update_actor:0.13784666296615106 - timing_per_token_ms/gen:0.7640811584948479 - timing_per_token_ms/adv:0.0035331425223860753 - perf/total_num_tokens:168538 - perf/time_per_step:93.83052004128695 - perf/throughput:224.52449363735883 (TaskRunner pid=2377588) Training Progress: 45%|████▌ | 45/100 [47:01<58:52, 64.22s/it] (TaskRunner pid=2377588) step:46 - global_seqlen/min:16735 - global_seqlen/max:24435 - global_seqlen/minmax_diff:7700 - global_seqlen/balanced_min:18509 - global_seqlen/balanced_max:19928 - global_seqlen/mean:18692.5 - actor/entropy:0.5046643614768982 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23075830936431885 - training/rollout_probs_diff_mean:0.006242220290005207 - training/rollout_probs_diff_std:0.012211744673550129 - training/rollout_actor_probs_pearson_corr:0.9990357160568237 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:1.0001463890075684 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.482745003362652e-05 - rollout_corr/rollout_is_max:1.6443771123886108 - rollout_corr/rollout_is_min:0.42285895347595215 - rollout_corr/rollout_is_std:0.04131534695625305 - rollout_corr/rollout_is_eff_sample_size:0.9982965449399626 - rollout_corr/rollout_is_seq_mean:1.0000933408737183 - rollout_corr/rollout_is_seq_std:0.0024182708002626896 - rollout_corr/rollout_is_seq_max:1.006613850593567 - rollout_corr/rollout_is_seq_min:0.9920908212661743 - rollout_corr/rollout_is_seq_max_deviation:0.007909178733825684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6816683495417237) - rollout_corr/training_log_ppl:np.float64(0.5142650831548963) - rollout_corr/kl:np.float64(0.0009203854278680978) - rollout_corr/k3_kl:np.float64(0.0009512752532145896) - rollout_corr/rollout_ppl:np.float64(1.6801149183884263) - rollout_corr/rollout_log_ppl:np.float64(0.5133446983527392) - rollout_corr/log_ppl_diff:np.float64(0.0009203848021570593) - rollout_corr/log_ppl_abs_diff:np.float64(0.002121724042808637) - rollout_corr/log_ppl_diff_max:np.float64(0.011334240436553955) - rollout_corr/log_ppl_diff_min:np.float64(-0.006246447563171387) - rollout_corr/ppl_ratio:np.float64(1.000924016116187) - rollout_corr/chi2_token:np.float64(0.001985518028959632) - rollout_corr/chi2_seq:np.float64(0.9741074193734676) - actor/pg_loss:np.float64(0.00946681818459183) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007681857287025196) - actor/ppo_kl:np.float64(0.00013474482491915296) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(18555.689070786582) - self_distillation/token_reweight_w_std:np.float64(225338.53030294098) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0038297504652292) - self_distillation/token_reweight_w_clip_frac:np.float64(0.20741881692083552) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.4815898835659027) - perf/mfu/actor:np.float64(0.04551584721805092) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.6351318359375) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:3 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0066397907212376595 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0066397907212376595 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:314.671875 - response_length/max:2246.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.671875 - response_length_non_aborted/max:2246.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.46875 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.988939225673676e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8591262754052877) - timing_s/agent_loop/generate_sequences/max:np.float64(14.905399655923247) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7275639467770816) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.905399655923247) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:2246 - timing_s/gen:21.791888078674674 - timing_s/reward:0.056276820600032806 - timing_s/old_log_prob:3.048815766349435 - timing_s/adv:0.5306549984961748 - timing_s/update_actor:21.164391972124577 - timing_s/step:46.683281594887376 - timing_s/stop_profile:0.00012224726378917694 - timing_per_token_ms/update_actor:0.1415299717274614 - timing_per_token_ms/gen:0.2705184974263205 - timing_per_token_ms/adv:0.0035485823090556025 - perf/total_num_tokens:149540 - perf/time_per_step:46.683281594887376 - perf/throughput:400.41101142399447 (TaskRunner pid=2377588) Training Progress: 46%|████▌ | 46/100 [47:47<53:04, 58.98s/it] (TaskRunner pid=2377588) step:47 - global_seqlen/min:17819 - global_seqlen/max:25075 - global_seqlen/minmax_diff:7256 - global_seqlen/balanced_min:21257 - global_seqlen/balanced_max:21265 - global_seqlen/mean:21261.625 - actor/entropy:0.4444273114204407 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4400259554386139 - training/rollout_probs_diff_mean:0.006007350981235504 - training/rollout_probs_diff_std:0.012589584104716778 - training/rollout_actor_probs_pearson_corr:0.9989708662033081 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.40625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.9999087452888489 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.5897006707964465e-05 - rollout_corr/rollout_is_max:1.5053962469100952 - rollout_corr/rollout_is_min:0.29472655057907104 - rollout_corr/rollout_is_std:0.04111432656645775 - rollout_corr/rollout_is_eff_sample_size:0.9983122272483306 - rollout_corr/rollout_is_seq_mean:0.9998648166656494 - rollout_corr/rollout_is_seq_std:0.0023827117402106524 - rollout_corr/rollout_is_seq_max:1.0055263042449951 - rollout_corr/rollout_is_seq_min:0.9899473786354065 - rollout_corr/rollout_is_seq_max_deviation:0.010052621364593506 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5705607468262315) - rollout_corr/training_log_ppl:np.float64(0.4469556713011116) - rollout_corr/kl:np.float64(0.0010853679043982822) - rollout_corr/k3_kl:np.float64(0.0010037241863756208) - rollout_corr/rollout_ppl:np.float64(1.568854795768857) - rollout_corr/rollout_log_ppl:np.float64(0.44587030314141884) - rollout_corr/log_ppl_diff:np.float64(0.0010853681596927345) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020706650684587657) - rollout_corr/log_ppl_diff_max:np.float64(0.008695393800735474) - rollout_corr/log_ppl_diff_min:np.float64(-0.006803631782531738) - rollout_corr/ppl_ratio:np.float64(1.0010890441481024) - rollout_corr/chi2_token:np.float64(0.0018414370715618134) - rollout_corr/chi2_seq:np.float64(0.4363660938106477) - actor/pg_loss:np.float64(0.008401424973271787) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008565550738239835) - actor/ppo_kl:np.float64(8.28054695958258e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40625) - self_distillation/token_reweight_w_mean:np.float64(35985.00870908215) - self_distillation/token_reweight_w_std:np.float64(607269.1316539588) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0023986937012523) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18423875700682402) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.44970042258501053) - perf/mfu/actor:np.float64(0.05236695723408108) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.62533569335938) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:3 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008636652491986752 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008636652491986752 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.67578125 - response_length/max:709.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.67578125 - response_length_non_aborted/max:709.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.827634155750275e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3688517697155476) - timing_s/agent_loop/generate_sequences/max:np.float64(6.532411988824606) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7759869190340396) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.532411988824606) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:709 - timing_s/gen:13.272230938076973 - timing_s/reward:0.059456592425704 - timing_s/old_log_prob:2.9300238769501448 - timing_s/adv:0.5104846581816673 - timing_s/update_actor:20.837091894820333 - timing_s/step:37.68995941616595 - timing_s/stop_profile:0.00011603906750679016 - timing_per_token_ms/update_actor:0.1225041118377613 - timing_per_token_ms/gen:0.17185553274128854 - timing_per_token_ms/adv:0.0030012090925650515 - perf/total_num_tokens:170093 - perf/time_per_step:37.68995941616595 - perf/throughput:564.1190738687949 (TaskRunner pid=2377588) Training Progress: 47%|████▋ | 47/100 [48:25<46:27, 52.60s/it] (TaskRunner pid=2377588) step:48 - global_seqlen/min:17483 - global_seqlen/max:22895 - global_seqlen/minmax_diff:5412 - global_seqlen/balanced_min:19181 - global_seqlen/balanced_max:19186 - global_seqlen/mean:19184.625 - actor/entropy:0.4989648461341858 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18431976437568665 - training/rollout_probs_diff_mean:0.0062462324276566505 - training/rollout_probs_diff_std:0.012339205481112003 - training/rollout_actor_probs_pearson_corr:0.9990322589874268 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.000258445739746 - rollout_corr/rollout_is_ratio_fraction_high:1.357865403406322e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.357865403406322e-05 - rollout_corr/rollout_is_max:2.057563066482544 - rollout_corr/rollout_is_min:0.470103919506073 - rollout_corr/rollout_is_std:0.04147230088710785 - rollout_corr/rollout_is_eff_sample_size:0.9982839519278051 - rollout_corr/rollout_is_seq_mean:1.000274658203125 - rollout_corr/rollout_is_seq_std:0.0025153234601020813 - rollout_corr/rollout_is_seq_max:1.00737726688385 - rollout_corr/rollout_is_seq_min:0.993608295917511 - rollout_corr/rollout_is_seq_max_deviation:0.007377266883850098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6499635931104422) - rollout_corr/training_log_ppl:np.float64(0.49548498587682843) - rollout_corr/kl:np.float64(0.0008148331013586585) - rollout_corr/k3_kl:np.float64(0.0009374350726147895) - rollout_corr/rollout_ppl:np.float64(1.648575996980071) - rollout_corr/rollout_log_ppl:np.float64(0.4946701503940858) - rollout_corr/log_ppl_diff:np.float64(0.0008148354827426374) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023317066370509565) - rollout_corr/log_ppl_diff_max:np.float64(0.009462296962738037) - rollout_corr/log_ppl_diff_min:np.float64(-0.00883948802947998) - rollout_corr/ppl_ratio:np.float64(1.0008193084504455) - rollout_corr/chi2_token:np.float64(0.002115017268806696) - rollout_corr/chi2_seq:np.float64(1.4036534039769322) - actor/pg_loss:np.float64(0.00924574350938201) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007358052316703834) - actor/ppo_kl:np.float64(0.00022666109313185245) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(13798.467892235843) - self_distillation/token_reweight_w_std:np.float64(218088.21206535352) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0028005323838443) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19191132939886302) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.49331025034189224) - perf/mfu/actor:np.float64(0.047835606836249434) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.66539001464844) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:3 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0010574377374723554 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0010574377374723554 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:287.67578125 - response_length/max:592.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.67578125 - response_length_non_aborted/max:592.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:311.84375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017088651657104492 - timing_s/agent_loop/generate_sequences/min:np.float64(2.119000919163227) - timing_s/agent_loop/generate_sequences/max:np.float64(5.597619671374559) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5647109401979833) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.597619671374559) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:421 - timing_s/agent_loop/slowest/response_length:592 - timing_s/gen:12.353877300396562 - timing_s/reward:0.05825226008892059 - timing_s/old_log_prob:2.8633284363895655 - timing_s/adv:0.49232253059744835 - timing_s/update_actor:20.665002543479204 - timing_s/step:36.52122236415744 - timing_s/stop_profile:0.00012777559459209442 - timing_per_token_ms/update_actor:0.13464559864656728 - timing_per_token_ms/gen:0.16774902981053108 - timing_per_token_ms/adv:0.0032077935495054524 - perf/total_num_tokens:153477 - perf/time_per_step:36.52122236415744 - perf/throughput:525.3007363419501 (TaskRunner pid=2377588) Training Progress: 48%|████▊ | 48/100 [49:02<41:25, 47.79s/it] (TaskRunner pid=2377588) step:49 - global_seqlen/min:16987 - global_seqlen/max:21266 - global_seqlen/minmax_diff:4279 - global_seqlen/balanced_min:19138 - global_seqlen/balanced_max:19146 - global_seqlen/mean:19143.625 - actor/entropy:0.5215277075767517 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22537505626678467 - training/rollout_probs_diff_mean:0.006179735995829105 - training/rollout_probs_diff_std:0.011852401308715343 - training/rollout_actor_probs_pearson_corr:0.9991031289100647 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.81640625 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.81640625 - rollout_corr/rollout_is_mean:0.9997779130935669 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7723264694213867 - rollout_corr/rollout_is_min:0.5217617750167847 - rollout_corr/rollout_is_std:0.040510427206754684 - rollout_corr/rollout_is_eff_sample_size:0.9983608217718287 - rollout_corr/rollout_is_seq_mean:0.999786376953125 - rollout_corr/rollout_is_seq_std:0.0023819187190383673 - rollout_corr/rollout_is_seq_max:1.0075643062591553 - rollout_corr/rollout_is_seq_min:0.9942116737365723 - rollout_corr/rollout_is_seq_max_deviation:0.0075643062591552734 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6935842824168503) - rollout_corr/training_log_ppl:np.float64(0.5215203997213393) - rollout_corr/kl:np.float64(0.001009607866770068) - rollout_corr/k3_kl:np.float64(0.0009367430029669777) - rollout_corr/rollout_ppl:np.float64(1.6918844282627106) - rollout_corr/rollout_log_ppl:np.float64(0.5205107883084565) - rollout_corr/log_ppl_diff:np.float64(0.0010096114128828049) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022241247352212667) - rollout_corr/log_ppl_diff_max:np.float64(0.009637236595153809) - rollout_corr/log_ppl_diff_min:np.float64(-0.008005857467651367) - rollout_corr/ppl_ratio:np.float64(1.001013510627672) - rollout_corr/chi2_token:np.float64(0.0017460049130022526) - rollout_corr/chi2_seq:np.float64(0.633887531934306) - actor/pg_loss:np.float64(0.009288873756304383) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009839653034759976) - actor/ppo_kl:np.float64(-3.1075492399779137e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.81640625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(2881.341757737333) - self_distillation/token_reweight_w_std:np.float64(43805.62359804881) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0042963603045791) - self_distillation/token_reweight_w_clip_frac:np.float64(0.22476474451832473) - self_distillation/empty_target_batch:np.float64(0.18359375) - actor/grad_norm:np.float64(0.45831435173749924) - perf/mfu/actor:np.float64(0.047177114873946914) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.7123794555664) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:3 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005017322488129139 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005017322488129139 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:302.17578125 - response_length/max:772.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:302.17578125 - response_length_non_aborted/max:772.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011060759425163269 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3090773448348045) - timing_s/agent_loop/generate_sequences/max:np.float64(6.693190542981029) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7540069321985357) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.693190542981029) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:772 - timing_s/gen:13.364387858659029 - timing_s/reward:0.058825232088565826 - timing_s/old_log_prob:2.9428266622126102 - timing_s/adv:0.5102043468505144 - timing_s/update_actor:20.955557495355606 - timing_s/step:37.91879368387163 - timing_s/stop_profile:2.7874484658241272e-05 - timing_per_token_ms/update_actor:0.13683117418563365 - timing_per_token_ms/gen:0.17276248896233087 - timing_per_token_ms/adv:0.0033314246051264745 - perf/total_num_tokens:153149 - perf/time_per_step:37.91879368387163 - perf/throughput:504.85849206069406 (TaskRunner pid=2377588) Training Progress: 49%|████▉ | 49/100 [49:40<38:06, 44.84s/it] (TaskRunner pid=2377588) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2377588) (TaskRunner pid=2377588) validation generation end (TaskRunner pid=2377588) [prompt] system (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2377588) (TaskRunner pid=2377588) A (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) user (TaskRunner pid=2377588) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2377588) (TaskRunner pid=2377588) A: -0.01 (TaskRunner pid=2377588) B: 1.69 (TaskRunner pid=2377588) C: 2.49 (TaskRunner pid=2377588) D: 0.45 (TaskRunner pid=2377588) Please reason step by step. (TaskRunner pid=2377588) assistant (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) [response] (TaskRunner pid=2377588) To determine the folding stability score of the given protein sequence, we need to assess the overall balance of hydrophobic and hydrophilic residues, as well as the potential for secondary structure formation. The sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR' contains a mix of hydrophilic and hydrophobic residues, which can influence the protein's ability to fold into a stable conformation. (TaskRunner pid=2377588) (TaskRunner pid=2377588) The presence of charged residues (like lysine, arginine, glutamic acid, and aspartic acid) indicates potential for forming hydrogen bonds and ionic interactions, which can contribute to stability. However, an excess of charged residues can also lead to destabilization due to electrostatic repulsion. The sequence also includes several repeated amino acids (e.g., 'RR' and 'AA'), which can promote local stability. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Among the given options, the score that best reflects a moderate level of stability, neither strongly favorable nor strongly unfavorable, is 0.45. This score suggests that the protein has a reasonable potential for folding into a stable structure, but not an extremely high level of stability. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Therefore, the most appropriate answer is D. (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) D (TaskRunner pid=2377588) (TaskRunner pid=2377588) [ground_truth] B (TaskRunner pid=2377588) [score] 0.0 (TaskRunner pid=2377588) [acc] 0.0 (TaskRunner pid=2377588) [pred] D (TaskRunner pid=2377588) [incorrect_format] 1 (TaskRunner pid=2377588) [feedback] (TaskRunner pid=2377588) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2377588) step:50 - global_seqlen/min:14605 - global_seqlen/max:21580 - global_seqlen/minmax_diff:6975 - global_seqlen/balanced_min:19390 - global_seqlen/balanced_max:19392 - global_seqlen/mean:19391.5 - actor/entropy:0.47946396470069885 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3996677100658417 - training/rollout_probs_diff_mean:0.006240477319806814 - training/rollout_probs_diff_std:0.012605458498001099 - training/rollout_actor_probs_pearson_corr:0.9989849328994751 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000673532485962 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.136740168789402e-05 - rollout_corr/rollout_is_max:1.6150994300842285 - rollout_corr/rollout_is_min:0.25283050537109375 - rollout_corr/rollout_is_std:0.04149097949266434 - rollout_corr/rollout_is_eff_sample_size:0.9982818135233342 - rollout_corr/rollout_is_seq_mean:1.000045895576477 - rollout_corr/rollout_is_seq_std:0.0027281756047159433 - rollout_corr/rollout_is_seq_max:1.008570671081543 - rollout_corr/rollout_is_seq_min:0.9895012378692627 - rollout_corr/rollout_is_seq_max_deviation:0.010498762130737305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6148534356616437) - rollout_corr/training_log_ppl:np.float64(0.4745740353828296) - rollout_corr/kl:np.float64(0.0008616547988395951) - rollout_corr/k3_kl:np.float64(0.0009946930891828742) - rollout_corr/rollout_ppl:np.float64(1.6134692886844277) - rollout_corr/rollout_log_ppl:np.float64(0.47371237888000906) - rollout_corr/log_ppl_diff:np.float64(0.0008616565028205514) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024016679963096976) - rollout_corr/log_ppl_diff_max:np.float64(0.01107800006866455) - rollout_corr/log_ppl_diff_min:np.float64(-0.007686227560043335) - rollout_corr/ppl_ratio:np.float64(1.000866251764819) - rollout_corr/chi2_token:np.float64(0.0022321681026369333) - rollout_corr/chi2_seq:np.float64(0.9324172651395202) - actor/pg_loss:np.float64(0.007270396803505719) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010783002867356117) - actor/ppo_kl:np.float64(3.474096706668206e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(8663.78590165032) - self_distillation/token_reweight_w_std:np.float64(143810.25243300595) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007838597521186) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16617618181044236) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.4546274393796921) - perf/mfu/actor:np.float64(0.04808585897670077) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.63968658447266) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.44625) - val-aux/sciknoweval/reward/std@16:np.float64(0.29480083625642206) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5642) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.25330119370356924) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.32504000000000005) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.24230366164914596) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.44444) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.29497408806997927) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.66376) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.19480495069379736) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.23332000000000006) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.18179379521844805) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.45910000000000006) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22956531167842603) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7399800000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.13930519034890534) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.16241999999999998) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.1311525055997694) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.46721999999999997) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.16076544683553418) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.79632) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.08636227070470426) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.10996) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.08708786467201995) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.4734999999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11035782553174701) - val-aux/sciknoweval/score/mean@16:np.float64(0.44625) - val-aux/sciknoweval/score/std@16:np.float64(0.29480083625642206) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5642) - val-aux/sciknoweval/score/best@2/std:np.float64(0.25330119370356924) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.32504000000000005) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.24230366164914596) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.44444) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.29497408806997927) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.66376) - val-aux/sciknoweval/score/best@4/std:np.float64(0.19480495069379736) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.23332000000000006) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.18179379521844805) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.45910000000000006) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.22956531167842603) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7399800000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.13930519034890534) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.16241999999999998) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.1311525055997694) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.46721999999999997) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.16076544683553418) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.79632) - val-aux/sciknoweval/score/best@16/std:np.float64(0.08636227070470426) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.10996) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.08708786467201995) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.4734999999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11035782553174701) - val-core/sciknoweval/acc/mean@16:np.float64(0.44625) - val-aux/sciknoweval/acc/std@16:np.float64(0.29480083625642206) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5642) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.25330119370356924) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.32504000000000005) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.24230366164914596) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.44444) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.29497408806997927) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.66376) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.19480495069379736) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.23332000000000006) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.18179379521844805) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.45910000000000006) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.22956531167842603) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7399800000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.13930519034890534) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.16241999999999998) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.1311525055997694) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.46721999999999997) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.16076544683553418) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.79632) - val-core/sciknoweval/acc/best@16/std:np.float64(0.08636227070470426) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.10996) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.08708786467201995) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.4734999999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11035782553174701) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.014762796599251487) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9958800000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.008706171674944946) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.98356) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.015516336872433008) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9895400000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.014686540352152026) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.99928) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003725801927102405) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9775) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01410794665435406) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9932) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01179854615382624) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9722599999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.011558698455152753) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9968600000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.007276015393056834) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.96698) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.009533079250693346) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99912) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004101901997854166) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:3 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00023551241611130536 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00023551241611130536 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:273.671875 - response_length/max:546.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:273.671875 - response_length_non_aborted/max:546.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.528252899646759e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9725825749337673) - timing_s/agent_loop/generate_sequences/max:np.float64(4.856273099780083) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.286718484283483) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.856273099780083) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:417 - timing_s/agent_loop/slowest/response_length:546 - timing_s/gen:11.50307503156364 - timing_s/reward:0.05687335692346096 - timing_s/old_log_prob:2.917559811845422 - timing_s/adv:0.49597372859716415 - timing_s/update_actor:20.57628460228443 - timing_s/step:35.63206495717168 - timing_s/testing:84.0180158931762 - timing_s/save_checkpoint:13.367561668157578 - timing_s/stop_profile:0.0001475401222705841 - timing_per_token_ms/update_actor:0.13263726763198072 - timing_per_token_ms/gen:0.16418890995666058 - timing_per_token_ms/adv:0.003197107808815487 - perf/total_num_tokens:155132 - perf/time_per_step:35.63206495717168 - perf/throughput:544.2148812679762 (TaskRunner pid=2377588) Training Progress: 50%|█████ | 50/100 [51:53<59:25, 71.32s/it] (TaskRunner pid=2377588) step:51 - global_seqlen/min:17188 - global_seqlen/max:26045 - global_seqlen/minmax_diff:8857 - global_seqlen/balanced_min:19228 - global_seqlen/balanced_max:26697 - global_seqlen/mean:20192.125 - actor/entropy:0.4293040931224823 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2549576759338379 - training/rollout_probs_diff_mean:0.005704172886908054 - training/rollout_probs_diff_std:0.012154080905020237 - training/rollout_actor_probs_pearson_corr:0.9990142583847046 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.4375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:1.0001018047332764 - rollout_corr/rollout_is_ratio_fraction_high:1.298010192840593e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.394634485244751 - rollout_corr/rollout_is_min:0.5615415573120117 - rollout_corr/rollout_is_std:0.040227677673101425 - rollout_corr/rollout_is_eff_sample_size:0.9983845860973478 - rollout_corr/rollout_is_seq_mean:1.0000746250152588 - rollout_corr/rollout_is_seq_std:0.0027263914234936237 - rollout_corr/rollout_is_seq_max:1.0084176063537598 - rollout_corr/rollout_is_seq_min:0.9908918142318726 - rollout_corr/rollout_is_seq_max_deviation:0.009108185768127441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6170092625543475) - rollout_corr/training_log_ppl:np.float64(0.47497447705609375) - rollout_corr/kl:np.float64(0.0012497635771566706) - rollout_corr/k3_kl:np.float64(0.00102243886846054) - rollout_corr/rollout_ppl:np.float64(1.6149390735663474) - rollout_corr/rollout_log_ppl:np.float64(0.4737247100347304) - rollout_corr/log_ppl_diff:np.float64(0.0012497670213633683) - rollout_corr/log_ppl_abs_diff:np.float64(0.002456330697896192) - rollout_corr/log_ppl_diff_max:np.float64(0.011276543140411377) - rollout_corr/log_ppl_diff_min:np.float64(-0.006423234939575195) - rollout_corr/ppl_ratio:np.float64(1.001254602568224) - rollout_corr/chi2_token:np.float64(0.0016376497223973274) - rollout_corr/chi2_seq:np.float64(0.4780591963790357) - actor/pg_loss:np.float64(0.0067009651102125645) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007522285332015599) - actor/ppo_kl:np.float64(0.0004095833642168145) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4375) - self_distillation/token_reweight_w_mean:np.float64(13076.554133867845) - self_distillation/token_reweight_w_std:np.float64(211910.2565514514) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022720373235643) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1773863446433097) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.4671621173620224) - perf/mfu/actor:np.float64(0.04654881126382293) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.96300601959229) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:3 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.05634662136435509 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.05634662136435509 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:300.94140625 - response_length/max:8192.0 - response_length/min:134.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:300.94140625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:330.0625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.477588951587677e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5246296878904104) - timing_s/agent_loop/generate_sequences/max:np.float64(57.90095570497215) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.419511581028928) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(57.90095570497215) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:64.51415247097611 - timing_s/reward:0.05432269722223282 - timing_s/old_log_prob:3.2740233596414328 - timing_s/adv:0.533617552369833 - timing_s/update_actor:22.38153512775898 - timing_s/step:90.81331651471555 - timing_s/stop_profile:0.00011249631643295288 - timing_per_token_ms/update_actor:0.13855361389501464 - timing_per_token_ms/gen:0.8374002475432057 - timing_per_token_ms/adv:0.0033033766404590467 - perf/total_num_tokens:161537 - perf/time_per_step:90.81331651471555 - perf/throughput:222.34762229752982 (TaskRunner pid=2377588) Training Progress: 51%|█████ | 51/100 [53:24<1:03:01, 77.18s/it] (TaskRunner pid=2377588) step:52 - global_seqlen/min:15114 - global_seqlen/max:19678 - global_seqlen/minmax_diff:4564 - global_seqlen/balanced_min:17391 - global_seqlen/balanced_max:17393 - global_seqlen/mean:17392.0 - actor/entropy:0.5117276906967163 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3672093152999878 - training/rollout_probs_diff_mean:0.006458327639847994 - training/rollout_probs_diff_std:0.012513774447143078 - training/rollout_actor_probs_pearson_corr:0.9990215301513672 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999927282333374 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6279516220092773 - rollout_corr/rollout_is_min:0.5086525082588196 - rollout_corr/rollout_is_std:0.042578574270009995 - rollout_corr/rollout_is_eff_sample_size:0.9981903459095249 - rollout_corr/rollout_is_seq_mean:0.999974250793457 - rollout_corr/rollout_is_seq_std:0.002831320744007826 - rollout_corr/rollout_is_seq_max:1.0063813924789429 - rollout_corr/rollout_is_seq_min:0.9922206997871399 - rollout_corr/rollout_is_seq_max_deviation:0.007779300212860107 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6746168392710388) - rollout_corr/training_log_ppl:np.float64(0.5098408633493818) - rollout_corr/kl:np.float64(0.0014690806421100433) - rollout_corr/k3_kl:np.float64(0.001028424924356841) - rollout_corr/rollout_ppl:np.float64(1.6721903183497488) - rollout_corr/rollout_log_ppl:np.float64(0.5083717810339294) - rollout_corr/log_ppl_diff:np.float64(0.0014690823154523969) - rollout_corr/log_ppl_abs_diff:np.float64(0.002573086996562779) - rollout_corr/log_ppl_diff_max:np.float64(0.009477674961090088) - rollout_corr/log_ppl_diff_min:np.float64(-0.006135165691375732) - rollout_corr/ppl_ratio:np.float64(1.0014744414947927) - rollout_corr/chi2_token:np.float64(0.0011512499768286943) - rollout_corr/chi2_seq:np.float64(0.4030457080807537) - actor/pg_loss:np.float64(0.006522214272990823) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001122471877351927) - actor/ppo_kl:np.float64(0.0005382808814404783) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(27136.17666967702) - self_distillation/token_reweight_w_std:np.float64(425517.41254893073) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014785304665565) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14723654714180157) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.40594881772994995) - perf/mfu/actor:np.float64(0.04314178159835623) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.96533203125) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:3 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003247976303100586 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003247976303100586 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:264.4375 - response_length/max:394.0 - response_length/min:158.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:264.4375 - response_length_non_aborted/max:394.0 - response_length_non_aborted/min:158.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.0625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014764443039894104 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9593238420784473) - timing_s/agent_loop/generate_sequences/max:np.float64(4.324066650122404) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.299596995500906) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.324066650122404) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:184 - timing_s/agent_loop/slowest/response_length:394 - timing_s/gen:11.446590397506952 - timing_s/reward:0.05542705953121185 - timing_s/old_log_prob:2.9832492768764496 - timing_s/adv:0.48082349821925163 - timing_s/update_actor:20.904979698359966 - timing_s/step:35.96184862777591 - timing_s/stop_profile:0.00010491535067558289 - timing_per_token_ms/update_actor:0.15024853164069663 - timing_per_token_ms/gen:0.16908813515579876 - timing_per_token_ms/adv:0.003455780662224382 - perf/total_num_tokens:139136 - perf/time_per_step:35.96184862777591 - perf/throughput:483.62363626009244 (TaskRunner pid=2377588) Training Progress: 52%|█████▏ | 52/100 [54:00<51:51, 64.83s/it] (TaskRunner pid=2377588) step:53 - global_seqlen/min:17059 - global_seqlen/max:22874 - global_seqlen/minmax_diff:5815 - global_seqlen/balanced_min:19281 - global_seqlen/balanced_max:19292 - global_seqlen/mean:19289.0 - actor/entropy:0.4607000946998596 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29402709007263184 - training/rollout_probs_diff_mean:0.006315375212579966 - training/rollout_probs_diff_std:0.01282504852861166 - training/rollout_actor_probs_pearson_corr:0.9988994598388672 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.000185251235962 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.889505230996292e-05 - rollout_corr/rollout_is_max:1.780316710472107 - rollout_corr/rollout_is_min:0.4562900960445404 - rollout_corr/rollout_is_std:0.04276713728904724 - rollout_corr/rollout_is_eff_sample_size:0.9981750237537251 - rollout_corr/rollout_is_seq_mean:1.0001364946365356 - rollout_corr/rollout_is_seq_std:0.002618843223899603 - rollout_corr/rollout_is_seq_max:1.009338140487671 - rollout_corr/rollout_is_seq_min:0.9941945672035217 - rollout_corr/rollout_is_seq_max_deviation:0.009338140487670898 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.592559686396271) - rollout_corr/training_log_ppl:np.float64(0.4603707636706531) - rollout_corr/kl:np.float64(0.0007604588569023463) - rollout_corr/k3_kl:np.float64(0.001011009657759132) - rollout_corr/rollout_ppl:np.float64(1.5913150617852807) - rollout_corr/rollout_log_ppl:np.float64(0.45961030514445156) - rollout_corr/log_ppl_diff:np.float64(0.0007604585262015462) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022676836233586073) - rollout_corr/log_ppl_diff_max:np.float64(0.01063990592956543) - rollout_corr/log_ppl_diff_min:np.float64(-0.00778353214263916) - rollout_corr/ppl_ratio:np.float64(1.000764571595937) - rollout_corr/chi2_token:np.float64(0.0025336912367492914) - rollout_corr/chi2_seq:np.float64(1.063055372564122) - actor/pg_loss:np.float64(0.007254078867845237) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012323748633207288) - actor/ppo_kl:np.float64(-2.4577615384657747e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(67757.99035247811) - self_distillation/token_reweight_w_std:np.float64(1011191.0651432538) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005109936464578) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15712974785128608) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4638129398226738) - perf/mfu/actor:np.float64(0.047956530479009274) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(203.01011657714844) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:3 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0025554061867296696 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0025554061867296696 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:270.375 - response_length/max:650.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:270.375 - response_length_non_aborted/max:650.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.40625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.921697735786438e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.752891294658184) - timing_s/agent_loop/generate_sequences/max:np.float64(5.864818444475532) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.371109402840375) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.864818444475532) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:650 - timing_s/gen:13.027901455760002 - timing_s/reward:0.05936954915523529 - timing_s/old_log_prob:2.949233453720808 - timing_s/adv:0.48596382699906826 - timing_s/update_actor:20.439435811713338 - timing_s/step:37.05334206856787 - timing_s/stop_profile:0.0001137610524892807 - timing_per_token_ms/update_actor:0.13245525825414314 - timing_per_token_ms/gen:0.18822095260864544 - timing_per_token_ms/adv:0.0031492290100515077 - perf/total_num_tokens:154312 - perf/time_per_step:37.05334206856787 - perf/throughput:520.5738247390846 (TaskRunner pid=2377588) Training Progress: 53%|█████▎ | 53/100 [54:37<44:16, 56.51s/it] (TaskRunner pid=2377588) step:54 - global_seqlen/min:15141 - global_seqlen/max:20649 - global_seqlen/minmax_diff:5508 - global_seqlen/balanced_min:18262 - global_seqlen/balanced_max:18574 - global_seqlen/mean:18304.125 - actor/entropy:0.46299999952316284 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.502325713634491 - training/rollout_probs_diff_mean:0.006158506963402033 - training/rollout_probs_diff_std:0.012747365981340408 - training/rollout_actor_probs_pearson_corr:0.998969554901123 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.000012993812561 - rollout_corr/rollout_is_ratio_fraction_high:1.4193054084898904e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:5.215323448181152 - rollout_corr/rollout_is_min:0.539194643497467 - rollout_corr/rollout_is_std:0.042119741439819336 - rollout_corr/rollout_is_eff_sample_size:0.9982288315381272 - rollout_corr/rollout_is_seq_mean:1.0000141859054565 - rollout_corr/rollout_is_seq_std:0.0029386195819824934 - rollout_corr/rollout_is_seq_max:1.0114915370941162 - rollout_corr/rollout_is_seq_min:0.9910533428192139 - rollout_corr/rollout_is_seq_max_deviation:0.011491537094116211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6074997074902058) - rollout_corr/training_log_ppl:np.float64(0.4689776684390381) - rollout_corr/kl:np.float64(0.0005251422747960532) - rollout_corr/k3_kl:np.float64(0.001105516353618441) - rollout_corr/rollout_ppl:np.float64(1.606539920438081) - rollout_corr/rollout_log_ppl:np.float64(0.46845252421917394) - rollout_corr/log_ppl_diff:np.float64(0.0005251442198641598) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024390119942836463) - rollout_corr/log_ppl_diff_max:np.float64(0.011168062686920166) - rollout_corr/log_ppl_diff_min:np.float64(-0.008471250534057617) - rollout_corr/ppl_ratio:np.float64(1.0005299369804561) - rollout_corr/chi2_token:np.float64(0.003591886954382062) - rollout_corr/chi2_seq:np.float64(1.352256109006703) - actor/pg_loss:np.float64(0.0086047756485641) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001323506013250153) - actor/ppo_kl:np.float64(-0.00038972256250424664) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(60794.549280457664) - self_distillation/token_reweight_w_std:np.float64(959671.1243281721) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003515179269016) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1663027488393709) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.4908316507935524) - perf/mfu/actor:np.float64(0.04533160368418794) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.9367322921753) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:3 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004701449070125818 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004701449070125818 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.22265625 - response_length/max:1063.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.22265625 - response_length_non_aborted/max:1063.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.123181760311127e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9818878639489412) - timing_s/agent_loop/generate_sequences/max:np.float64(7.9810455068945885) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.282189201861911) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.9810455068945885) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1063 - timing_s/gen:16.092292612418532 - timing_s/reward:0.05770587921142578 - timing_s/old_log_prob:2.9810782838612795 - timing_s/adv:0.4884880259633064 - timing_s/update_actor:20.733204890042543 - timing_s/step:40.44154738076031 - timing_s/stop_profile:6.708130240440369e-05 - timing_per_token_ms/update_actor:0.1415883365774282 - timing_per_token_ms/gen:0.22839877673500905 - timing_per_token_ms/adv:0.003335914895981824 - perf/total_num_tokens:146433 - perf/time_per_step:40.44154738076031 - perf/throughput:452.60693978064785 (TaskRunner pid=2377588) Training Progress: 54%|█████▍ | 54/100 [55:17<39:38, 51.70s/it] (TaskRunner pid=2377588) step:55 - global_seqlen/min:16307 - global_seqlen/max:21708 - global_seqlen/minmax_diff:5401 - global_seqlen/balanced_min:19121 - global_seqlen/balanced_max:19128 - global_seqlen/mean:19125.5 - actor/entropy:0.459578275680542 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17766082286834717 - training/rollout_probs_diff_mean:0.006273999810218811 - training/rollout_probs_diff_std:0.01284012570977211 - training/rollout_actor_probs_pearson_corr:0.9989087581634521 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.9998308420181274 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9069620370864868 - rollout_corr/rollout_is_min:0.6123274564743042 - rollout_corr/rollout_is_std:0.04280196502804756 - rollout_corr/rollout_is_eff_sample_size:0.9981706291136749 - rollout_corr/rollout_is_seq_mean:0.999870240688324 - rollout_corr/rollout_is_seq_std:0.0028112498112022877 - rollout_corr/rollout_is_seq_max:1.0069541931152344 - rollout_corr/rollout_is_seq_min:0.9923368692398071 - rollout_corr/rollout_is_seq_max_deviation:0.007663130760192871 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5917990752495825) - rollout_corr/training_log_ppl:np.float64(0.4598227452370338) - rollout_corr/kl:np.float64(0.0010948574003286637) - rollout_corr/k3_kl:np.float64(0.0010583536472950072) - rollout_corr/rollout_ppl:np.float64(1.5900167520157993) - rollout_corr/rollout_log_ppl:np.float64(0.4587278845719993) - rollout_corr/log_ppl_diff:np.float64(0.0010948606650345027) - rollout_corr/log_ppl_abs_diff:np.float64(0.002278648840729147) - rollout_corr/log_ppl_diff_max:np.float64(0.009043872356414795) - rollout_corr/log_ppl_diff_min:np.float64(-0.006281405687332153) - rollout_corr/ppl_ratio:np.float64(1.0010990139562637) - rollout_corr/chi2_token:np.float64(0.0020766062662005424) - rollout_corr/chi2_seq:np.float64(0.5712041023653001) - actor/pg_loss:np.float64(0.011084495927207172) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001200795573367941) - actor/ppo_kl:np.float64(4.1436044675435824e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(39001.96408048854) - self_distillation/token_reweight_w_std:np.float64(629925.1150107854) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011648854706436) - self_distillation/token_reweight_w_clip_frac:np.float64(0.21475123713025823) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.5816158875823021) - perf/mfu/actor:np.float64(0.047126229302569836) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.79755020141602) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:3 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005410306039266288 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005410306039266288 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:267.140625 - response_length/max:627.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:267.140625 - response_length_non_aborted/max:627.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:498.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.651211202144623e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9998407810926437) - timing_s/agent_loop/generate_sequences/max:np.float64(5.572395589202642) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4080838294685236) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.572395589202642) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:627 - timing_s/gen:12.271607633680105 - timing_s/reward:0.05737173371016979 - timing_s/old_log_prob:3.06848481297493 - timing_s/adv:0.5002676099538803 - timing_s/update_actor:21.015477251261473 - timing_s/step:36.99724875576794 - timing_s/stop_profile:0.00012267008423805237 - timing_per_token_ms/update_actor:0.1373524695515246 - timing_per_token_ms/gen:0.17944094919693668 - timing_per_token_ms/adv:0.003269637460157122 - perf/total_num_tokens:153004 - perf/time_per_step:36.99724875576794 - perf/throughput:516.9438442910784 (TaskRunner pid=2377588) Training Progress: 55%|█████▌ | 55/100 [55:54<35:28, 47.31s/it] (TaskRunner pid=2377588) step:56 - global_seqlen/min:15453 - global_seqlen/max:23313 - global_seqlen/minmax_diff:7860 - global_seqlen/balanced_min:18890 - global_seqlen/balanced_max:18892 - global_seqlen/mean:18890.875 - actor/entropy:0.4716559648513794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18383854627609253 - training/rollout_probs_diff_mean:0.006478022783994675 - training/rollout_probs_diff_std:0.013011007569730282 - training/rollout_actor_probs_pearson_corr:0.9988781213760376 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:0.9998638033866882 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4501370060315821e-05 - rollout_corr/rollout_is_max:1.7269703149795532 - rollout_corr/rollout_is_min:0.4058607816696167 - rollout_corr/rollout_is_std:0.04286736622452736 - rollout_corr/rollout_is_eff_sample_size:0.9981651655610857 - rollout_corr/rollout_is_seq_mean:0.9998332262039185 - rollout_corr/rollout_is_seq_std:0.0027711554430425167 - rollout_corr/rollout_is_seq_max:1.0098469257354736 - rollout_corr/rollout_is_seq_min:0.9920184016227722 - rollout_corr/rollout_is_seq_max_deviation:0.009846925735473633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6096358560025692) - rollout_corr/training_log_ppl:np.float64(0.47044293012004346) - rollout_corr/kl:np.float64(0.001213938884703225) - rollout_corr/k3_kl:np.float64(0.0011389106959995843) - rollout_corr/rollout_ppl:np.float64(1.6076381281018257) - rollout_corr/rollout_log_ppl:np.float64(0.4692289916565642) - rollout_corr/log_ppl_diff:np.float64(0.0012139384634792805) - rollout_corr/log_ppl_abs_diff:np.float64(0.002665675710886717) - rollout_corr/log_ppl_diff_max:np.float64(0.014403939247131348) - rollout_corr/log_ppl_diff_min:np.float64(-0.008231699466705322) - rollout_corr/ppl_ratio:np.float64(1.0012199901975691) - rollout_corr/chi2_token:np.float64(0.00209705694578588) - rollout_corr/chi2_seq:np.float64(1.1846230770461261) - actor/pg_loss:np.float64(0.011043414357118309) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016035224407460191) - actor/ppo_kl:np.float64(0.00012595200854548239) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(98463.75261447299) - self_distillation/token_reweight_w_std:np.float64(1580140.9493958978) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00297643779777) - self_distillation/token_reweight_w_clip_frac:np.float64(0.20217352296458557) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.515902653336525) - perf/mfu/actor:np.float64(0.046606891226913305) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.85066986083984) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:3 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002015690552070737 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002015690552070737 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:269.37109375 - response_length/max:443.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.37109375 - response_length_non_aborted/max:443.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:320.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013717636466026306 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0151682309806347) - timing_s/agent_loop/generate_sequences/max:np.float64(4.7197112292051315) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3755541434366023) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.7197112292051315) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:443 - timing_s/gen:11.962028758600354 - timing_s/reward:0.06106633320450783 - timing_s/old_log_prob:2.9284057077020407 - timing_s/adv:0.4981265049427748 - timing_s/update_actor:20.84898098744452 - timing_s/step:36.386329786852 - timing_s/stop_profile:2.5799497961997986e-05 - timing_per_token_ms/update_actor:0.13795669197062418 - timing_per_token_ms/gen:0.17346580951870466 - timing_per_token_ms/adv:0.00329607882736225 - perf/total_num_tokens:151127 - perf/time_per_step:36.386329786852 - perf/throughput:519.1750613667584 (TaskRunner pid=2377588) Training Progress: 56%|█████▌ | 56/100 [56:31<32:17, 44.04s/it] (TaskRunner pid=2377588) step:57 - global_seqlen/min:16024 - global_seqlen/max:21468 - global_seqlen/minmax_diff:5444 - global_seqlen/balanced_min:18614 - global_seqlen/balanced_max:18621 - global_seqlen/mean:18618.25 - actor/entropy:0.454791396856308 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2935653626918793 - training/rollout_probs_diff_mean:0.006426603067666292 - training/rollout_probs_diff_std:0.013245771639049053 - training/rollout_actor_probs_pearson_corr:0.9988606572151184 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9998227953910828 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9654592275619507 - rollout_corr/rollout_is_min:0.5022943019866943 - rollout_corr/rollout_is_std:0.04381151124835014 - rollout_corr/rollout_is_eff_sample_size:0.9980835757072072 - rollout_corr/rollout_is_seq_mean:0.9998459815979004 - rollout_corr/rollout_is_seq_std:0.0027751659508794546 - rollout_corr/rollout_is_seq_max:1.007636308670044 - rollout_corr/rollout_is_seq_min:0.9912930727005005 - rollout_corr/rollout_is_seq_max_deviation:0.008706927299499512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.576864798553288) - rollout_corr/training_log_ppl:np.float64(0.4503620285540819) - rollout_corr/kl:np.float64(0.0013839330839289232) - rollout_corr/k3_kl:np.float64(0.001118682132300819) - rollout_corr/rollout_ppl:np.float64(1.5746726654469967) - rollout_corr/rollout_log_ppl:np.float64(0.4489780950243585) - rollout_corr/log_ppl_diff:np.float64(0.0013839335297234356) - rollout_corr/log_ppl_abs_diff:np.float64(0.002587239316198975) - rollout_corr/log_ppl_diff_max:np.float64(0.015008151531219482) - rollout_corr/log_ppl_diff_min:np.float64(-0.005801945924758911) - rollout_corr/ppl_ratio:np.float64(1.0013894692528993) - rollout_corr/chi2_token:np.float64(0.0017305072396993637) - rollout_corr/chi2_seq:np.float64(0.3892146178986877) - actor/pg_loss:np.float64(0.011214006342925131) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010815030755111366) - actor/ppo_kl:np.float64(0.0002617336079708821) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(27540.106545773335) - self_distillation/token_reweight_w_std:np.float64(458129.18024963874) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992596958763897) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19985835871193558) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.5223179087042809) - perf/mfu/actor:np.float64(0.04600042528600244) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.85821533203125) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:4 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033486399333924055 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033486399333924055 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:266.2578125 - response_length/max:616.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.2578125 - response_length_non_aborted/max:616.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.5625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003427322953939438 - timing_s/agent_loop/generate_sequences/min:np.float64(1.774012854322791) - timing_s/agent_loop/generate_sequences/max:np.float64(5.524538477882743) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.251167176043964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.524538477882743) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:616 - timing_s/gen:12.881953652948141 - timing_s/reward:0.058807652443647385 - timing_s/old_log_prob:3.071711640805006 - timing_s/adv:0.5218311659991741 - timing_s/update_actor:20.813452599570155 - timing_s/step:37.464955519884825 - timing_s/stop_profile:3.0338764190673828e-05 - timing_per_token_ms/update_actor:0.13973824473010457 - timing_per_token_ms/gen:0.18899025341023065 - timing_per_token_ms/adv:0.0035034923126446775 - perf/total_num_tokens:148946 - perf/time_per_step:37.464955519884825 - perf/throughput:496.9510771237461 (TaskRunner pid=2377588) Training Progress: 57%|█████▋ | 57/100 [57:08<30:14, 42.20s/it] (TaskRunner pid=2377588) step:58 - global_seqlen/min:14459 - global_seqlen/max:21156 - global_seqlen/minmax_diff:6697 - global_seqlen/balanced_min:18220 - global_seqlen/balanced_max:18229 - global_seqlen/mean:18226.875 - actor/entropy:0.44357579946517944 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1497378945350647 - training/rollout_probs_diff_mean:0.0065992739982903 - training/rollout_probs_diff_std:0.013419775292277336 - training/rollout_actor_probs_pearson_corr:0.9987605214118958 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9999858736991882 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5194104889815208e-05 - rollout_corr/rollout_is_max:1.5986714363098145 - rollout_corr/rollout_is_min:0.477508008480072 - rollout_corr/rollout_is_std:0.04383395239710808 - rollout_corr/rollout_is_eff_sample_size:0.9980822694279385 - rollout_corr/rollout_is_seq_mean:1.0000407695770264 - rollout_corr/rollout_is_seq_std:0.002992832101881504 - rollout_corr/rollout_is_seq_max:1.0077518224716187 - rollout_corr/rollout_is_seq_min:0.9927381873130798 - rollout_corr/rollout_is_seq_max_deviation:0.007751822471618652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5614326829090714) - rollout_corr/training_log_ppl:np.float64(0.44203145906794816) - rollout_corr/kl:np.float64(0.0012513224959427305) - rollout_corr/k3_kl:np.float64(0.0011372438857506495) - rollout_corr/rollout_ppl:np.float64(1.559436835348606) - rollout_corr/rollout_log_ppl:np.float64(0.44078013475518674) - rollout_corr/log_ppl_diff:np.float64(0.001251324312761426) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027763640973716974) - rollout_corr/log_ppl_diff_max:np.float64(0.009593278169631958) - rollout_corr/log_ppl_diff_min:np.float64(-0.008433282375335693) - rollout_corr/ppl_ratio:np.float64(1.0012572219129652) - rollout_corr/chi2_token:np.float64(0.002024699468165636) - rollout_corr/chi2_seq:np.float64(0.763364716200158) - actor/pg_loss:np.float64(0.00859343248885125) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011575627140700817) - actor/ppo_kl:np.float64(0.00031340952021707835) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(25006.7489922191) - self_distillation/token_reweight_w_std:np.float64(407317.1976042362) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994073531124741) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17494874715339392) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.44489244371652603) - perf/mfu/actor:np.float64(0.04525287704795743) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.87546920776367) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:4 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008907543495297432 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008907543495297432 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:257.08984375 - response_length/max:683.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:257.08984375 - response_length_non_aborted/max:683.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.3908913135528564e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9305750355124474) - timing_s/agent_loop/generate_sequences/max:np.float64(5.828761048614979) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.161130092186795) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.828761048614979) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:683 - timing_s/gen:13.053296227008104 - timing_s/reward:0.055634522810578346 - timing_s/old_log_prob:3.0284343361854553 - timing_s/adv:0.5005805157124996 - timing_s/update_actor:20.837197083979845 - timing_s/step:37.56855076365173 - timing_s/stop_profile:0.00011104531586170197 - timing_per_token_ms/update_actor:0.14290160192010318 - timing_per_token_ms/gen:0.19833314938856042 - timing_per_token_ms/adv:0.0034329836828344107 - perf/total_num_tokens:145815 - perf/time_per_step:37.56855076365173 - perf/throughput:485.16311195146875 (TaskRunner pid=2377588) Training Progress: 58%|█████▊ | 58/100 [57:46<28:34, 40.82s/it] (TaskRunner pid=2377588) step:59 - global_seqlen/min:16468 - global_seqlen/max:20733 - global_seqlen/minmax_diff:4265 - global_seqlen/balanced_min:18784 - global_seqlen/balanced_max:18871 - global_seqlen/mean:18802.875 - actor/entropy:0.4275815784931183 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19503915309906006 - training/rollout_probs_diff_mean:0.006501158233731985 - training/rollout_probs_diff_std:0.0134871955960989 - training/rollout_actor_probs_pearson_corr:0.9987592101097107 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.000274419784546 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5828957557678223 - rollout_corr/rollout_is_min:0.6114084124565125 - rollout_corr/rollout_is_std:0.04412263631820679 - rollout_corr/rollout_is_eff_sample_size:0.9980578070114071 - rollout_corr/rollout_is_seq_mean:1.000355839729309 - rollout_corr/rollout_is_seq_std:0.002742155222222209 - rollout_corr/rollout_is_seq_max:1.009993314743042 - rollout_corr/rollout_is_seq_min:0.994229257106781 - rollout_corr/rollout_is_seq_max_deviation:0.009993314743041992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5369335850700736) - rollout_corr/training_log_ppl:np.float64(0.42503680812660605) - rollout_corr/kl:np.float64(0.0007365336702038761) - rollout_corr/k3_kl:np.float64(0.0011426141516039934) - rollout_corr/rollout_ppl:np.float64(1.5357845509424806) - rollout_corr/rollout_log_ppl:np.float64(0.42430027161026374) - rollout_corr/log_ppl_diff:np.float64(0.0007365365163423121) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024664205848239362) - rollout_corr/log_ppl_diff_max:np.float64(0.01307147741317749) - rollout_corr/log_ppl_diff_min:np.float64(-0.008815675973892212) - rollout_corr/ppl_ratio:np.float64(1.000741511117667) - rollout_corr/chi2_token:np.float64(0.003107692813500762) - rollout_corr/chi2_seq:np.float64(0.866616667015478) - actor/pg_loss:np.float64(0.008501346106640995) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012260132853043615) - actor/ppo_kl:np.float64(0.00011159085599388163) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(42511.26000588038) - self_distillation/token_reweight_w_std:np.float64(654044.6374392689) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000549259129912) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1569639132649172) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.471733033657074) - perf/mfu/actor:np.float64(0.0459583741860117) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.8160171508789) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:4 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0039762333035469055 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0039762333035469055 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:258.37109375 - response_length/max:751.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:258.37109375 - response_length_non_aborted/max:751.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:329.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010284408926963806 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1431463696062565) - timing_s/agent_loop/generate_sequences/max:np.float64(6.1859929747879505) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2042612918958184) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.1859929747879505) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:751 - timing_s/gen:13.274558758363128 - timing_s/reward:0.058606766164302826 - timing_s/old_log_prob:3.0173478983342648 - timing_s/adv:0.5164175685495138 - timing_s/update_actor:20.998920468613505 - timing_s/step:37.95342009328306 - timing_s/stop_profile:0.00011136196553707123 - timing_per_token_ms/update_actor:0.1395991335674299 - timing_per_token_ms/gen:0.20069483933845045 - timing_per_token_ms/adv:0.0034331024414452167 - perf/total_num_tokens:150423 - perf/time_per_step:37.95342009328306 - perf/throughput:495.4197791341526 (TaskRunner pid=2377588) Training Progress: 59%|█████▉ | 59/100 [58:24<27:18, 39.97s/it] (TaskRunner pid=2377588) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2377588) validation generation end (TaskRunner pid=2377588) [prompt] system (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2377588) (TaskRunner pid=2377588) A (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) user (TaskRunner pid=2377588) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2377588) (TaskRunner pid=2377588) A: -0.01 (TaskRunner pid=2377588) B: 1.69 (TaskRunner pid=2377588) C: 2.49 (TaskRunner pid=2377588) D: 0.45 (TaskRunner pid=2377588) Please reason step by step. (TaskRunner pid=2377588) assistant (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) [response] (TaskRunner pid=2377588) To determine the folding stability score of the given protein sequence, we need to evaluate the balance of hydrophobic and hydrophilic amino acids, as well as the overall secondary structure potential. The sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR' contains a mix of hydrophobic and hydrophilic residues, with several charged amino acids that may contribute to stability through electrostatic interactions. (TaskRunner pid=2377588) (TaskRunner pid=2377588) However, without specific computational tools or databases to calculate the exact folding stability score, we can estimate based on known patterns. A score of 1.69 (Option B) is reasonable as it suggests moderate stability, which aligns with the presence of both stabilizing and destabilizing elements in the sequence. Other options either suggest extreme instability (A: -0.01) or high stability (C: 2.49 or D: 0.45), which may not be supported by the sequence's composition. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Therefore, the most plausible answer is B. (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) B (TaskRunner pid=2377588) (TaskRunner pid=2377588) [ground_truth] B (TaskRunner pid=2377588) [score] 1.0 (TaskRunner pid=2377588) [acc] 1.0 (TaskRunner pid=2377588) [pred] B (TaskRunner pid=2377588) [incorrect_format] 1 (TaskRunner pid=2377588) [feedback] (TaskRunner pid=2377588) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2377588) step:60 - global_seqlen/min:13576 - global_seqlen/max:20947 - global_seqlen/minmax_diff:7371 - global_seqlen/balanced_min:16645 - global_seqlen/balanced_max:16703 - global_seqlen/mean:16656.375 - actor/entropy:0.4254399538040161 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3288995623588562 - training/rollout_probs_diff_mean:0.006534686777740717 - training/rollout_probs_diff_std:0.013477168045938015 - training/rollout_actor_probs_pearson_corr:0.99874347448349 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9996057748794556 - rollout_corr/rollout_is_ratio_fraction_high:1.6076395695563406e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.6076395695563406e-05 - rollout_corr/rollout_is_max:2.032266616821289 - rollout_corr/rollout_is_min:0.433442622423172 - rollout_corr/rollout_is_std:0.04407735541462898 - rollout_corr/rollout_is_eff_sample_size:0.9980593507206911 - rollout_corr/rollout_is_seq_mean:0.9996085166931152 - rollout_corr/rollout_is_seq_std:0.003007634310051799 - rollout_corr/rollout_is_seq_max:1.008218765258789 - rollout_corr/rollout_is_seq_min:0.9911052584648132 - rollout_corr/rollout_is_seq_max_deviation:0.008894741535186768 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5300847575999796) - rollout_corr/training_log_ppl:np.float64(0.4217581414268352) - rollout_corr/kl:np.float64(0.001792657749192017) - rollout_corr/k3_kl:np.float64(0.0012050122458049373) - rollout_corr/rollout_ppl:np.float64(1.5273630735464394) - rollout_corr/rollout_log_ppl:np.float64(0.4199654840049334) - rollout_corr/log_ppl_diff:np.float64(0.001792657421901822) - rollout_corr/log_ppl_abs_diff:np.float64(0.002946705324575305) - rollout_corr/log_ppl_diff_max:np.float64(0.014287739992141724) - rollout_corr/log_ppl_diff_min:np.float64(-0.009233087301254272) - rollout_corr/ppl_ratio:np.float64(1.0018000043928623) - rollout_corr/chi2_token:np.float64(0.001288227504119277) - rollout_corr/chi2_seq:np.float64(0.31369242863729596) - actor/pg_loss:np.float64(0.0078679786529392) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000945317102377885) - actor/ppo_kl:np.float64(0.000415737068987454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(12415.185876371106) - self_distillation/token_reweight_w_std:np.float64(160777.9885632039) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999912645900622) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1443075709976256) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.4619360789656639) - perf/mfu/actor:np.float64(0.04120263510682176) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.9427032470703) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.45875) - val-aux/sciknoweval/reward/std@16:np.float64(0.28589994738248686) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.57374) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.23503150750091606) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.33942) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.25065505727072734) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.45498) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2861518531609652) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6671999999999998) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1677803697765788) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.24321999999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1933932257367078) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.46828) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.23023309858834207) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7315999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10691196248757866) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.16668) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.1404344995572873) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.47262) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1717538736958792) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.77122) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0576150284914792) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.10891999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.09256814392545262) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.4702) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.124096498377995) - val-aux/sciknoweval/score/mean@16:np.float64(0.45875) - val-aux/sciknoweval/score/std@16:np.float64(0.28589994738248686) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.57374) - val-aux/sciknoweval/score/best@2/std:np.float64(0.23503150750091606) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.33942) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.25065505727072734) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.45498) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2861518531609652) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6671999999999998) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1677803697765788) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.24321999999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1933932257367078) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.46828) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.23023309858834207) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7315999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10691196248757866) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.16668) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.1404344995572873) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.47262) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1717538736958792) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.77122) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0576150284914792) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.10891999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.09256814392545262) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.4702) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.124096498377995) - val-core/sciknoweval/acc/mean@16:np.float64(0.45875) - val-aux/sciknoweval/acc/std@16:np.float64(0.28589994738248686) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.57374) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.23503150750091606) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.33942) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.25065505727072734) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.45498) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2861518531609652) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6671999999999998) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1677803697765788) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.24321999999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1933932257367078) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.46828) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.23023309858834207) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7315999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10691196248757866) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.16668) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.1404344995572873) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.47262) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1717538736958792) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.77122) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0576150284914792) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.10891999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.09256814392545262) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.4702) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.124096498377995) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.004841229182759271) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9998600000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0016674531477675766) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.99748) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.00663698726833192) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99884) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.004674868982121317) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.99548) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.008364783320564857) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9996600000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0025854206620973697) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9921000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.009777013859047147) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0014106735979665889) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9868600000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.009494229826584146) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:4 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005952285137027502 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005952285137027502 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:242.98046875 - response_length/max:723.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:242.98046875 - response_length_non_aborted/max:723.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015892833471298218 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6263678707182407) - timing_s/agent_loop/generate_sequences/max:np.float64(5.923971053212881) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9335935315393726) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.923971053212881) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:723 - timing_s/gen:12.829169150441885 - timing_s/reward:0.055075451731681824 - timing_s/old_log_prob:2.9527630638331175 - timing_s/adv:0.5322051215916872 - timing_s/update_actor:20.8302882630378 - timing_s/step:37.292767314240336 - timing_s/testing:78.69150371104479 - timing_s/save_checkpoint:13.373769173398614 - timing_s/stop_profile:0.0001485925167798996 - timing_per_token_ms/update_actor:0.15632369185250242 - timing_per_token_ms/gen:0.2062467911586561 - timing_per_token_ms/adv:0.003994004709845984 - perf/total_num_tokens:133251 - perf/time_per_step:37.292767314240336 - perf/throughput:446.6382143123962 (TaskRunner pid=2377588) Training Progress: 60%|██████ | 60/100 [1:00:34<44:32, 66.81s/it] (TaskRunner pid=2377588) step:61 - global_seqlen/min:15215 - global_seqlen/max:21636 - global_seqlen/minmax_diff:6421 - global_seqlen/balanced_min:17781 - global_seqlen/balanced_max:17863 - global_seqlen/mean:17796.125 - actor/entropy:0.4386352598667145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20178675651550293 - training/rollout_probs_diff_mean:0.006503935903310776 - training/rollout_probs_diff_std:0.013286919333040714 - training/rollout_actor_probs_pearson_corr:0.9987898468971252 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.46875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9996527433395386 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5713914632797241 - rollout_corr/rollout_is_min:0.5322938561439514 - rollout_corr/rollout_is_std:0.042463626712560654 - rollout_corr/rollout_is_eff_sample_size:0.99819889801498 - rollout_corr/rollout_is_seq_mean:0.9995660781860352 - rollout_corr/rollout_is_seq_std:0.002834318671375513 - rollout_corr/rollout_is_seq_max:1.0070080757141113 - rollout_corr/rollout_is_seq_min:0.989690899848938 - rollout_corr/rollout_is_seq_max_deviation:0.010309100151062012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5459735784679651) - rollout_corr/training_log_ppl:np.float64(0.4307885488960892) - rollout_corr/kl:np.float64(0.0011336080857589081) - rollout_corr/k3_kl:np.float64(0.0010721262671040677) - rollout_corr/rollout_ppl:np.float64(1.5442254738882184) - rollout_corr/rollout_log_ppl:np.float64(0.42965493840165436) - rollout_corr/log_ppl_diff:np.float64(0.0011336104944348335) - rollout_corr/log_ppl_abs_diff:np.float64(0.002438421710394323) - rollout_corr/log_ppl_diff_max:np.float64(0.011657178401947021) - rollout_corr/log_ppl_diff_min:np.float64(-0.006434887647628784) - rollout_corr/ppl_ratio:np.float64(1.0011385180987418) - rollout_corr/chi2_token:np.float64(0.0020339724142104387) - rollout_corr/chi2_seq:np.float64(0.4345152941532433) - actor/pg_loss:np.float64(0.00790927221532911) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008774361849646084) - actor/ppo_kl:np.float64(-0.00021857498688859778) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46875) - self_distillation/token_reweight_w_mean:np.float64(90567.05320201884) - self_distillation/token_reweight_w_std:np.float64(1318699.3814123706) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014657031279057) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1606671340414323) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4679563641548157) - perf/mfu/actor:np.float64(0.043273108008049774) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(204.20936584472656) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:4 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008226853795349598 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008226853795349598 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:257.41015625 - response_length/max:743.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:257.41015625 - response_length_non_aborted/max:743.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.660182356834412e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7865658551454544) - timing_s/agent_loop/generate_sequences/max:np.float64(6.220908848568797) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1459032708153245) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.220908848568797) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:743 - timing_s/gen:12.280947236344218 - timing_s/reward:0.057083407416939735 - timing_s/old_log_prob:2.8869006168097258 - timing_s/adv:0.531508132815361 - timing_s/update_actor:21.12980868294835 - timing_s/step:36.95119444653392 - timing_s/stop_profile:0.00011005997657775879 - timing_per_token_ms/update_actor:0.14841579756090406 - timing_per_token_ms/gen:0.186365801726091 - timing_per_token_ms/adv:0.003733313662492263 - perf/total_num_tokens:142369 - perf/time_per_step:36.95119444653392 - perf/throughput:481.6116303290246 (TaskRunner pid=2377588) Training Progress: 61%|██████ | 61/100 [1:01:11<37:36, 57.87s/it] (TaskRunner pid=2377588) step:62 - global_seqlen/min:14910 - global_seqlen/max:19567 - global_seqlen/minmax_diff:4657 - global_seqlen/balanced_min:17778 - global_seqlen/balanced_max:18022 - global_seqlen/mean:17817.375 - actor/entropy:0.43082961440086365 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1324964165687561 - training/rollout_probs_diff_mean:0.006294483318924904 - training/rollout_probs_diff_std:0.013027667067945004 - training/rollout_actor_probs_pearson_corr:0.9988439083099365 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000845193862915 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5962506532669067 - rollout_corr/rollout_is_min:0.6203190088272095 - rollout_corr/rollout_is_std:0.04276295751333237 - rollout_corr/rollout_is_eff_sample_size:0.9981751425283176 - rollout_corr/rollout_is_seq_mean:1.0000911951065063 - rollout_corr/rollout_is_seq_std:0.002576856641098857 - rollout_corr/rollout_is_seq_max:1.0064663887023926 - rollout_corr/rollout_is_seq_min:0.9925443530082703 - rollout_corr/rollout_is_seq_max_deviation:0.007455646991729736 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5557348728179932) - rollout_corr/training_log_ppl:np.float64(0.43825247848872095) - rollout_corr/kl:np.float64(0.0009370925864118362) - rollout_corr/k3_kl:np.float64(0.001064462290969459) - rollout_corr/rollout_ppl:np.float64(1.55428398353979) - rollout_corr/rollout_log_ppl:np.float64(0.4373153829947114) - rollout_corr/log_ppl_diff:np.float64(0.0009370954940095544) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024938444839790463) - rollout_corr/log_ppl_diff_max:np.float64(0.009828656911849976) - rollout_corr/log_ppl_diff_min:np.float64(-0.00672990083694458) - rollout_corr/ppl_ratio:np.float64(1.0009421091526747) - rollout_corr/chi2_token:np.float64(0.0023952664341777563) - rollout_corr/chi2_seq:np.float64(0.695401132106781) - actor/pg_loss:np.float64(0.010420616716146469) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013428862312139245) - actor/ppo_kl:np.float64(9.546323143716506e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(50855.47481812653) - self_distillation/token_reweight_w_std:np.float64(799014.3118602036) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004145544953644) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19556896801805124) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5019177198410034) - perf/mfu/actor:np.float64(0.0431819971259509) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(203.61127090454102) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:4 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001342887757346034 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001342887757346034 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:262.88671875 - response_length/max:980.0 - response_length/min:158.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.88671875 - response_length_non_aborted/max:980.0 - response_length_non_aborted/min:158.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.90625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.987689554691315e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9663371127098799) - timing_s/agent_loop/generate_sequences/max:np.float64(7.428887825459242) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.14778972339991) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.428887825459242) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:980 - timing_s/gen:14.978658078238368 - timing_s/reward:0.05798129178583622 - timing_s/old_log_prob:2.9473987072706223 - timing_s/adv:0.5583777241408825 - timing_s/update_actor:21.287055471912026 - timing_s/step:39.917185328900814 - timing_s/stop_profile:0.00011189654469490051 - timing_per_token_ms/update_actor:0.14934197287698123 - timing_per_token_ms/gen:0.22256880604820828 - timing_per_token_ms/adv:0.003917368047628245 - perf/total_num_tokens:142539 - perf/time_per_step:39.917185328900814 - perf/throughput:446.35850081092457 (TaskRunner pid=2377588) Training Progress: 62%|██████▏ | 62/100 [1:01:50<33:14, 52.50s/it] (TaskRunner pid=2377588) step:63 - global_seqlen/min:16663 - global_seqlen/max:22170 - global_seqlen/minmax_diff:5507 - global_seqlen/balanced_min:19347 - global_seqlen/balanced_max:19703 - global_seqlen/mean:19397.875 - actor/entropy:0.4101067781448364 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20771706104278564 - training/rollout_probs_diff_mean:0.0061399140395224094 - training/rollout_probs_diff_std:0.01295802928507328 - training/rollout_actor_probs_pearson_corr:0.9988389611244202 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001003742218018 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9436557292938232 - rollout_corr/rollout_is_min:0.617319643497467 - rollout_corr/rollout_is_std:0.04209284484386444 - rollout_corr/rollout_is_eff_sample_size:0.9982316824434865 - rollout_corr/rollout_is_seq_mean:1.000100016593933 - rollout_corr/rollout_is_seq_std:0.00288019934669137 - rollout_corr/rollout_is_seq_max:1.0085450410842896 - rollout_corr/rollout_is_seq_min:0.992731511592865 - rollout_corr/rollout_is_seq_max_deviation:0.00854504108428955 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5121602197177708) - rollout_corr/training_log_ppl:np.float64(0.40878716617589816) - rollout_corr/kl:np.float64(0.001358278656141465) - rollout_corr/k3_kl:np.float64(0.0010200752675473268) - rollout_corr/rollout_ppl:np.float64(1.5101502728648484) - rollout_corr/rollout_log_ppl:np.float64(0.4074288872652687) - rollout_corr/log_ppl_diff:np.float64(0.0013582789106294513) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025853109546005726) - rollout_corr/log_ppl_diff_max:np.float64(0.010260730981826782) - rollout_corr/log_ppl_diff_min:np.float64(-0.006247401237487793) - rollout_corr/ppl_ratio:np.float64(1.0013637458905578) - rollout_corr/chi2_token:np.float64(0.0013771248050034046) - rollout_corr/chi2_seq:np.float64(0.26601731847040355) - actor/pg_loss:np.float64(0.008053267258219421) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008274137194348441) - actor/ppo_kl:np.float64(0.000555983587961606) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(39562.41957426979) - self_distillation/token_reweight_w_std:np.float64(604850.7347932355) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992628088220954) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15747668471885845) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.4357759356498718) - perf/mfu/actor:np.float64(0.047575568819104554) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(203.223388671875) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:4 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002204642631113529 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002204642631113529 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:271.08984375 - response_length/max:1172.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.08984375 - response_length_non_aborted/max:1172.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:335.09375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000141095370054245 - timing_s/agent_loop/generate_sequences/min:np.float64(1.462497103959322) - timing_s/agent_loop/generate_sequences/max:np.float64(8.425911355763674) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.266941771929851) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.425911355763674) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1172 - timing_s/gen:15.19361407123506 - timing_s/reward:0.06109987199306488 - timing_s/old_log_prob:2.9197529926896095 - timing_s/adv:0.5066203605383635 - timing_s/update_actor:20.741629377007484 - timing_s/step:39.50878845714033 - timing_s/stop_profile:3.056228160858154e-05 - timing_per_token_ms/update_actor:0.13365915968248768 - timing_per_token_ms/gen:0.2189313112758838 - timing_per_token_ms/adv:0.00326466404527792 - perf/total_num_tokens:155183 - perf/time_per_step:39.50878845714033 - perf/throughput:490.9762044726601 (TaskRunner pid=2377588) Training Progress: 63%|██████▎ | 63/100 [1:02:30<29:58, 48.61s/it] (TaskRunner pid=2377588) step:64 - global_seqlen/min:17063 - global_seqlen/max:26653 - global_seqlen/minmax_diff:9590 - global_seqlen/balanced_min:20419 - global_seqlen/balanced_max:27869 - global_seqlen/mean:21396.125 - actor/entropy:0.36596664786338806 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44310861825942993 - training/rollout_probs_diff_mean:0.005594620946794748 - training/rollout_probs_diff_std:0.012887604534626007 - training/rollout_actor_probs_pearson_corr:0.9988188147544861 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000910758972168 - rollout_corr/rollout_is_ratio_fraction_high:4.0592651203041896e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.7061769287684e-05 - rollout_corr/rollout_is_max:2.850736379623413 - rollout_corr/rollout_is_min:0.018279142677783966 - rollout_corr/rollout_is_std:0.04076636582612991 - rollout_corr/rollout_is_eff_sample_size:0.9983410982408072 - rollout_corr/rollout_is_seq_mean:1.0000557899475098 - rollout_corr/rollout_is_seq_std:0.002877295482903719 - rollout_corr/rollout_is_seq_max:1.0079630613327026 - rollout_corr/rollout_is_seq_min:0.9925675392150879 - rollout_corr/rollout_is_seq_max_deviation:0.007963061332702637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5159107027575374) - rollout_corr/training_log_ppl:np.float64(0.41184405865351437) - rollout_corr/kl:np.float64(0.001103285293467593) - rollout_corr/k3_kl:np.float64(0.0011687450491422169) - rollout_corr/rollout_ppl:np.float64(1.5142215262167156) - rollout_corr/rollout_log_ppl:np.float64(0.41074077133089304) - rollout_corr/log_ppl_diff:np.float64(0.001103287322621327) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026119705216842704) - rollout_corr/log_ppl_diff_max:np.float64(0.014486104249954224) - rollout_corr/log_ppl_diff_min:np.float64(-0.008795589208602905) - rollout_corr/ppl_ratio:np.float64(1.0011089278850704) - rollout_corr/chi2_token:np.float64(0.0024646627716720104) - rollout_corr/chi2_seq:np.float64(0.7182152746245265) - actor/pg_loss:np.float64(0.007432444952428341) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011442802533565555) - actor/ppo_kl:np.float64(0.00014206777123604297) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(38419.34444196313) - self_distillation/token_reweight_w_std:np.float64(600834.2552816451) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000104485079646) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14910062527633272) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.448931559920311) - perf/mfu/actor:np.float64(0.04900363977019724) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.85402297973633) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:4 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01630302332341671 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01630302332341671 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:288.69140625 - response_length/max:8192.0 - response_length/min:141.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:288.69140625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:379.9375 - prompt_length/max:499.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.457946538925171e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7391462977975607) - timing_s/agent_loop/generate_sequences/max:np.float64(57.91169963590801) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3293268794732285) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(57.91169963590801) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:64.946254177019 - timing_s/reward:0.06121533922851086 - timing_s/old_log_prob:3.228845652192831 - timing_s/adv:0.5319570433348417 - timing_s/update_actor:22.536590956151485 - timing_s/step:91.39213843457401 - timing_s/stop_profile:0.00011085532605648041 - timing_per_token_ms/update_actor:0.13166280667732758 - timing_per_token_ms/gen:0.8787802473042284 - timing_per_token_ms/adv:0.0031077884624835204 - perf/total_num_tokens:171169 - perf/time_per_step:91.39213843457401 - perf/throughput:234.11340807302702 (TaskRunner pid=2377588) Training Progress: 64%|██████▍ | 64/100 [1:04:01<36:52, 61.46s/it] (TaskRunner pid=2377588) step:65 - global_seqlen/min:15299 - global_seqlen/max:20473 - global_seqlen/minmax_diff:5174 - global_seqlen/balanced_min:18434 - global_seqlen/balanced_max:18530 - global_seqlen/mean:18470.75 - actor/entropy:0.4395657777786255 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24154376983642578 - training/rollout_probs_diff_mean:0.0062695834785699844 - training/rollout_probs_diff_std:0.01293132919818163 - training/rollout_actor_probs_pearson_corr:0.9988812804222107 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0001733303070068 - rollout_corr/rollout_is_ratio_fraction_high:1.4853102584311273e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0482370853424072 - rollout_corr/rollout_is_min:0.6052082180976868 - rollout_corr/rollout_is_std:0.042885441333055496 - rollout_corr/rollout_is_eff_sample_size:0.9981649280166777 - rollout_corr/rollout_is_seq_mean:1.000065565109253 - rollout_corr/rollout_is_seq_std:0.0031271062325686216 - rollout_corr/rollout_is_seq_max:1.00895357131958 - rollout_corr/rollout_is_seq_min:0.9906403422355652 - rollout_corr/rollout_is_seq_max_deviation:0.009359657764434814 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5767603036947548) - rollout_corr/training_log_ppl:np.float64(0.45017470204038545) - rollout_corr/kl:np.float64(0.0012370372338139646) - rollout_corr/k3_kl:np.float64(0.001115720087113914) - rollout_corr/rollout_ppl:np.float64(1.5748289627954364) - rollout_corr/rollout_log_ppl:np.float64(0.44893766182940453) - rollout_corr/log_ppl_diff:np.float64(0.001237040210980922) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028583357925526798) - rollout_corr/log_ppl_diff_max:np.float64(0.010612547397613525) - rollout_corr/log_ppl_diff_min:np.float64(-0.01109766960144043) - rollout_corr/ppl_ratio:np.float64(1.0012436737306416) - rollout_corr/chi2_token:np.float64(0.002034065080806613) - rollout_corr/chi2_seq:np.float64(1.0242163685616106) - actor/pg_loss:np.float64(0.009213516605086625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008021764306249679) - actor/ppo_kl:np.float64(0.0003486028146646447) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(31378.26998622995) - self_distillation/token_reweight_w_std:np.float64(469320.71432315547) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997176113538444) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17037129099480808) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.5074385404586792) - perf/mfu/actor:np.float64(0.04532778714377611) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.79337692260742) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:4 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01046029757708311 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01046029757708311 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:262.9921875 - response_length/max:913.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.9921875 - response_length_non_aborted/max:913.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.21875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015012361109256744 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6141398791223764) - timing_s/agent_loop/generate_sequences/max:np.float64(7.079806221649051) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.127278054780618) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.079806221649051) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:913 - timing_s/gen:13.761610999703407 - timing_s/reward:0.05926308408379555 - timing_s/old_log_prob:2.9339145570993423 - timing_s/adv:0.5037177950143814 - timing_s/update_actor:20.945132406428456 - timing_s/step:38.29164301045239 - timing_s/stop_profile:0.00011120177805423737 - timing_per_token_ms/update_actor:0.14174527568201384 - timing_per_token_ms/gen:0.20440262305355147 - timing_per_token_ms/adv:0.003408888343829984 - perf/total_num_tokens:147766 - perf/time_per_step:38.29164301045239 - perf/throughput:482.37026535941743 (TaskRunner pid=2377588) Training Progress: 65%|██████▌ | 65/100 [1:04:40<31:48, 54.52s/it] (TaskRunner pid=2377588) step:66 - global_seqlen/min:14882 - global_seqlen/max:23515 - global_seqlen/minmax_diff:8633 - global_seqlen/balanced_min:17826 - global_seqlen/balanced_max:21638 - global_seqlen/mean:18303.125 - actor/entropy:0.4220793545246124 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2584120035171509 - training/rollout_probs_diff_mean:0.00598661694675684 - training/rollout_probs_diff_std:0.012764804065227509 - training/rollout_actor_probs_pearson_corr:0.9989262819290161 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.45703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:1.0000392198562622 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4125690540822688e-05 - rollout_corr/rollout_is_max:1.6294711828231812 - rollout_corr/rollout_is_min:0.340629518032074 - rollout_corr/rollout_is_std:0.04139605537056923 - rollout_corr/rollout_is_eff_sample_size:0.9982894167808524 - rollout_corr/rollout_is_seq_mean:1.0000745058059692 - rollout_corr/rollout_is_seq_std:0.0028324732556939125 - rollout_corr/rollout_is_seq_max:1.0090967416763306 - rollout_corr/rollout_is_seq_min:0.9921995401382446 - rollout_corr/rollout_is_seq_max_deviation:0.009096741676330566 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5753986253403127) - rollout_corr/training_log_ppl:np.float64(0.4505469045252539) - rollout_corr/kl:np.float64(0.0010772378078214295) - rollout_corr/k3_kl:np.float64(0.0011395804199310078) - rollout_corr/rollout_ppl:np.float64(1.573721437714994) - rollout_corr/rollout_log_ppl:np.float64(0.44946966556017287) - rollout_corr/log_ppl_diff:np.float64(0.001077238965081051) - rollout_corr/log_ppl_abs_diff:np.float64(0.002535409206757322) - rollout_corr/log_ppl_diff_max:np.float64(0.01022881269454956) - rollout_corr/log_ppl_diff_min:np.float64(-0.008596599102020264) - rollout_corr/ppl_ratio:np.float64(1.001082690199837) - rollout_corr/chi2_token:np.float64(0.00247390940785408) - rollout_corr/chi2_seq:np.float64(0.5947125048842281) - actor/pg_loss:np.float64(0.006117348559200764) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007169425653046346) - actor/ppo_kl:np.float64(0.00022494086654800682) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.45703125) - self_distillation/token_reweight_w_mean:np.float64(25265.76355998614) - self_distillation/token_reweight_w_std:np.float64(426066.7344984135) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000108608044684) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1348198783234693) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.38147570937871933) - perf/mfu/actor:np.float64(0.04382241574195507) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.73599243164062) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:4 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006466034799814224 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006466034799814224 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:276.53515625 - response_length/max:4576.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:276.53515625 - response_length_non_aborted/max:4576.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.4375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.59150493144989e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5943390280008316) - timing_s/agent_loop/generate_sequences/max:np.float64(27.593950107693672) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1746117096336093) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.593950107693672) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:4576 - timing_s/gen:34.400041012093425 - timing_s/reward:0.05819697305560112 - timing_s/old_log_prob:3.125191170722246 - timing_s/adv:0.4835810400545597 - timing_s/update_actor:21.517225416377187 - timing_s/step:59.67036030627787 - timing_s/stop_profile:2.8932467103004456e-05 - timing_per_token_ms/update_actor:0.14695048944085495 - timing_per_token_ms/gen:0.48592432884739206 - timing_per_token_ms/adv:0.003302585214646131 - perf/total_num_tokens:146425 - perf/time_per_step:59.67036030627787 - perf/throughput:306.73729647438284 (TaskRunner pid=2377588) Training Progress: 66%|██████▌ | 66/100 [1:05:39<31:46, 56.07s/it] (TaskRunner pid=2377588) step:67 - global_seqlen/min:17283 - global_seqlen/max:22148 - global_seqlen/minmax_diff:4865 - global_seqlen/balanced_min:19303 - global_seqlen/balanced_max:19311 - global_seqlen/mean:19309.75 - actor/entropy:0.44037064909935 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2278791069984436 - training/rollout_probs_diff_mean:0.006115133408457041 - training/rollout_probs_diff_std:0.012787416577339172 - training/rollout_actor_probs_pearson_corr:0.9989049434661865 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.85546875 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.85546875 - rollout_corr/rollout_is_mean:1.0001541376113892 - rollout_corr/rollout_is_ratio_fraction_high:1.4830194231763016e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.899169921875 - rollout_corr/rollout_is_min:0.546549916267395 - rollout_corr/rollout_is_std:0.0416916124522686 - rollout_corr/rollout_is_eff_sample_size:0.998265300597333 - rollout_corr/rollout_is_seq_mean:1.0001991987228394 - rollout_corr/rollout_is_seq_std:0.002904656808823347 - rollout_corr/rollout_is_seq_max:1.0123586654663086 - rollout_corr/rollout_is_seq_min:0.9931641817092896 - rollout_corr/rollout_is_seq_max_deviation:0.012358665466308594 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5527842263691127) - rollout_corr/training_log_ppl:np.float64(0.4356874483637512) - rollout_corr/kl:np.float64(0.0006533445317984388) - rollout_corr/k3_kl:np.float64(0.0010051454808035487) - rollout_corr/rollout_ppl:np.float64(1.5517510646022856) - rollout_corr/rollout_log_ppl:np.float64(0.4350341086392291) - rollout_corr/log_ppl_diff:np.float64(0.000653339724522084) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023757576127536595) - rollout_corr/log_ppl_diff_max:np.float64(0.007972121238708496) - rollout_corr/log_ppl_diff_min:np.float64(-0.008267045021057129) - rollout_corr/ppl_ratio:np.float64(1.0006576082669199) - rollout_corr/chi2_token:np.float64(0.002711253007873893) - rollout_corr/chi2_seq:np.float64(0.7541021520737559) - actor/pg_loss:np.float64(0.008652084856294096) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013011539704166353) - actor/ppo_kl:np.float64(-6.02354254652937e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.85546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.85546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(54575.59217771585) - self_distillation/token_reweight_w_std:np.float64(881558.6362724619) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001286598853767) - self_distillation/token_reweight_w_clip_frac:np.float64(0.19273610709933564) - self_distillation/empty_target_batch:np.float64(0.14453125) - actor/grad_norm:np.float64(0.4626174494624138) - perf/mfu/actor:np.float64(0.04740252829823064) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.80462646484375) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:4 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00918359775096178 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00918359775096178 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:263.3984375 - response_length/max:645.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:263.3984375 - response_length_non_aborted/max:645.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:340.03125 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.337619662284851e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8226409200578928) - timing_s/agent_loop/generate_sequences/max:np.float64(5.8391985315829515) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3380439583270345) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.8391985315829515) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:645 - timing_s/gen:12.905634524300694 - timing_s/reward:0.05754808336496353 - timing_s/old_log_prob:3.0211477875709534 - timing_s/adv:0.524041635915637 - timing_s/update_actor:20.95294176414609 - timing_s/step:37.54925003647804 - timing_s/stop_profile:0.00010993145406246185 - timing_per_token_ms/update_actor:0.1356370600612779 - timing_per_token_ms/gen:0.1913930672445602 - timing_per_token_ms/adv:0.0033923383000533215 - perf/total_num_tokens:154478 - perf/time_per_step:37.54925003647804 - perf/throughput:514.2512828149995 (TaskRunner pid=2377588) Training Progress: 67%|██████▋ | 67/100 [1:06:17<27:47, 50.53s/it] (TaskRunner pid=2377588) step:68 - global_seqlen/min:13278 - global_seqlen/max:19936 - global_seqlen/minmax_diff:6658 - global_seqlen/balanced_min:16776 - global_seqlen/balanced_max:17013 - global_seqlen/mean:16870.5 - actor/entropy:0.4389343559741974 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18747451901435852 - training/rollout_probs_diff_mean:0.006103357300162315 - training/rollout_probs_diff_std:0.012646200135350227 - training/rollout_actor_probs_pearson_corr:0.9989157915115356 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0002444982528687 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6292892694473267 - rollout_corr/rollout_is_min:0.621447741985321 - rollout_corr/rollout_is_std:0.04205884784460068 - rollout_corr/rollout_is_eff_sample_size:0.9982348897314802 - rollout_corr/rollout_is_seq_mean:1.0002204179763794 - rollout_corr/rollout_is_seq_std:0.0030804111156612635 - rollout_corr/rollout_is_seq_max:1.0113606452941895 - rollout_corr/rollout_is_seq_min:0.9885759949684143 - rollout_corr/rollout_is_seq_max_deviation:0.011424005031585693 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.562988564837724) - rollout_corr/training_log_ppl:np.float64(0.4414620564202778) - rollout_corr/kl:np.float64(0.0010434942897985522) - rollout_corr/k3_kl:np.float64(0.001034999100170353) - rollout_corr/rollout_ppl:np.float64(1.5613151947036386) - rollout_corr/rollout_log_ppl:np.float64(0.4404185648309067) - rollout_corr/log_ppl_diff:np.float64(0.001043491589371115) - rollout_corr/log_ppl_abs_diff:np.float64(0.002567836258094758) - rollout_corr/log_ppl_diff_max:np.float64(0.010526984930038452) - rollout_corr/log_ppl_diff_min:np.float64(-0.006782621145248413) - rollout_corr/ppl_ratio:np.float64(1.0010488389525563) - rollout_corr/chi2_token:np.float64(0.0020339246839284897) - rollout_corr/chi2_seq:np.float64(0.5519926382694393) - actor/pg_loss:np.float64(0.008678577025420964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012098356446585967) - actor/ppo_kl:np.float64(0.00034817022320510205) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(13878.405282065505) - self_distillation/token_reweight_w_std:np.float64(224388.73338573094) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000999762909487) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1488224413478747) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.39500317722558975) - perf/mfu/actor:np.float64(0.040801433028378) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.92280197143555) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:4 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006101126316934824 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006101126316934824 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:253.859375 - response_length/max:991.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:253.859375 - response_length_non_aborted/max:991.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.34375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001471303403377533 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3576139584183693) - timing_s/agent_loop/generate_sequences/max:np.float64(7.458230992779136) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0870083675545175) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.458230992779136) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:433 - timing_s/agent_loop/slowest/response_length:991 - timing_s/gen:14.033693075180054 - timing_s/reward:0.056325457990169525 - timing_s/old_log_prob:2.91792219504714 - timing_s/adv:0.5644538719207048 - timing_s/update_actor:21.42757635936141 - timing_s/step:39.08609275892377 - timing_s/stop_profile:0.00010946951806545258 - timing_per_token_ms/update_actor:0.15876512521384525 - timing_per_token_ms/gen:0.21594283675724832 - timing_per_token_ms/adv:0.004182255060021227 - perf/total_num_tokens:134964 - perf/time_per_step:39.08609275892377 - perf/throughput:431.6241099885403 (TaskRunner pid=2377588) Training Progress: 68%|██████▊ | 68/100 [1:06:56<25:07, 47.11s/it] (TaskRunner pid=2377588) step:69 - global_seqlen/min:16346 - global_seqlen/max:21101 - global_seqlen/minmax_diff:4755 - global_seqlen/balanced_min:18555 - global_seqlen/balanced_max:18565 - global_seqlen/mean:18561.875 - actor/entropy:0.4276966154575348 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30139583349227905 - training/rollout_probs_diff_mean:0.006168495863676071 - training/rollout_probs_diff_std:0.012948459945619106 - training/rollout_actor_probs_pearson_corr:0.9988653659820557 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0002561807632446 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6267973184585571 - rollout_corr/rollout_is_min:0.5154621005058289 - rollout_corr/rollout_is_std:0.042331479489803314 - rollout_corr/rollout_is_eff_sample_size:0.9982120827979993 - rollout_corr/rollout_is_seq_mean:1.0002408027648926 - rollout_corr/rollout_is_seq_std:0.002673950046300888 - rollout_corr/rollout_is_seq_max:1.0079408884048462 - rollout_corr/rollout_is_seq_min:0.9941331148147583 - rollout_corr/rollout_is_seq_max_deviation:0.007940888404846191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5416952958330512) - rollout_corr/training_log_ppl:np.float64(0.4282347278785892) - rollout_corr/kl:np.float64(0.0010731439164111123) - rollout_corr/k3_kl:np.float64(0.0010075719527549154) - rollout_corr/rollout_ppl:np.float64(1.5399859268218279) - rollout_corr/rollout_log_ppl:np.float64(0.4271615821053274) - rollout_corr/log_ppl_diff:np.float64(0.0010731457732617855) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025948359398171306) - rollout_corr/log_ppl_diff_max:np.float64(0.012785613536834717) - rollout_corr/log_ppl_diff_min:np.float64(-0.006662338972091675) - rollout_corr/ppl_ratio:np.float64(1.0010785202030092) - rollout_corr/chi2_token:np.float64(0.001873696455731988) - rollout_corr/chi2_seq:np.float64(0.7073195539414883) - actor/pg_loss:np.float64(0.008328666328452528) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008509040389981237) - actor/ppo_kl:np.float64(0.00041133684663918757) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(61008.057487468235) - self_distillation/token_reweight_w_std:np.float64(997786.081312076) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000671020243317) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14984558726428077) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.45351843535900116) - perf/mfu/actor:np.float64(0.0454792876437419) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(202.90647888183594) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:4 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00904255360364914 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00904255360364914 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:254.27734375 - response_length/max:709.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:254.27734375 - response_length_non_aborted/max:709.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:325.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001326967030763626 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6677718758583069) - timing_s/agent_loop/generate_sequences/max:np.float64(5.8000955786556005) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0447018001941615) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.8000955786556005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:709 - timing_s/gen:12.798793723806739 - timing_s/reward:0.053392503410577774 - timing_s/old_log_prob:2.957289256155491 - timing_s/adv:0.4778474997729063 - timing_s/update_actor:20.849623154848814 - timing_s/step:37.21205896139145 - timing_s/stop_profile:0.00010747835040092468 - timing_per_token_ms/update_actor:0.14040623020875326 - timing_per_token_ms/gen:0.19661715529313678 - timing_per_token_ms/adv:0.00321793662933369 - perf/total_num_tokens:148495 - perf/time_per_step:37.21205896139145 - perf/throughput:498.81343623738917 (TaskRunner pid=2377588) Training Progress: 69%|██████▉ | 69/100 [1:07:33<22:48, 44.16s/it] (TaskRunner pid=2377588) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2377588) validation generation end (TaskRunner pid=2377588) [prompt] system (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2377588) (TaskRunner pid=2377588) A (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) user (TaskRunner pid=2377588) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2377588) (TaskRunner pid=2377588) A: -0.01 (TaskRunner pid=2377588) B: 1.69 (TaskRunner pid=2377588) C: 2.49 (TaskRunner pid=2377588) D: 0.45 (TaskRunner pid=2377588) Please reason step by step. (TaskRunner pid=2377588) assistant (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) [response] (TaskRunner pid=2377588) To determine the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR', we must consider the balance of hydrophobic and hydrophilic residues, as well as the overall secondary structure potential. (TaskRunner pid=2377588) (TaskRunner pid=2377588) This sequence contains a mix of hydrophilic and hydrophobic residues. The presence of charged residues (like lysine, arginine, and glutamic acid) suggests a potential for forming hydrogen bonds and stabilizing secondary structures such as alpha-helices or beta-sheets. However, the sequence also includes a number of glycine and serine residues, which are flexible and may reduce overall stability. (TaskRunner pid=2377588) (TaskRunner pid=2377588) The folding stability score is a measure of how likely the sequence is to fold into a stable, functional structure. A score of 0.45 (option D) indicates moderate stability, which is consistent with the observed composition of the sequence. A higher score (like 1.69 or 2.49) would suggest a much stronger tendency to fold, which is not strongly supported by the sequence's composition. A score of -0.01 (option A) would indicate very low stability or even instability, which is not the case here. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Therefore, the most reasonable choice is option D. (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) D (TaskRunner pid=2377588) (TaskRunner pid=2377588) [ground_truth] B (TaskRunner pid=2377588) [score] 0.0 (TaskRunner pid=2377588) [acc] 0.0 (TaskRunner pid=2377588) [pred] D (TaskRunner pid=2377588) [incorrect_format] 1 (TaskRunner pid=2377588) [feedback] (TaskRunner pid=2377588) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2377588) step:70 - global_seqlen/min:14195 - global_seqlen/max:22916 - global_seqlen/minmax_diff:8721 - global_seqlen/balanced_min:18575 - global_seqlen/balanced_max:19069 - global_seqlen/mean:18647.5 - actor/entropy:0.40223971009254456 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21410562098026276 - training/rollout_probs_diff_mean:0.005981245543807745 - training/rollout_probs_diff_std:0.012858964502811432 - training/rollout_actor_probs_pearson_corr:0.9988651871681213 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9999248385429382 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5099353731784504e-05 - rollout_corr/rollout_is_max:1.637394666671753 - rollout_corr/rollout_is_min:0.4432382881641388 - rollout_corr/rollout_is_std:0.04244397208094597 - rollout_corr/rollout_is_eff_sample_size:0.9982015111875429 - rollout_corr/rollout_is_seq_mean:0.9998952746391296 - rollout_corr/rollout_is_seq_std:0.002894037403166294 - rollout_corr/rollout_is_seq_max:1.007933497428894 - rollout_corr/rollout_is_seq_min:0.9927491545677185 - rollout_corr/rollout_is_seq_max_deviation:0.007933497428894043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5124935517087579) - rollout_corr/training_log_ppl:np.float64(0.408629662822932) - rollout_corr/kl:np.float64(0.001113242896618516) - rollout_corr/k3_kl:np.float64(0.0010684058212859782) - rollout_corr/rollout_ppl:np.float64(1.5107644419185817) - rollout_corr/rollout_log_ppl:np.float64(0.40751642000395805) - rollout_corr/log_ppl_diff:np.float64(0.0011132428189739585) - rollout_corr/log_ppl_abs_diff:np.float64(0.002559314714744687) - rollout_corr/log_ppl_diff_max:np.float64(0.011656224727630615) - rollout_corr/log_ppl_diff_min:np.float64(-0.00806976854801178) - rollout_corr/ppl_ratio:np.float64(1.0011183749884367) - rollout_corr/chi2_token:np.float64(0.002033733995631337) - rollout_corr/chi2_seq:np.float64(0.45794546813704073) - actor/pg_loss:np.float64(0.009028565953485668) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012773160842698417) - actor/ppo_kl:np.float64(8.440654408481407e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(116265.03865508409) - self_distillation/token_reweight_w_std:np.float64(1752052.7730618762) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003117620944977) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17489009501878172) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.5032764300704002) - perf/mfu/actor:np.float64(0.04566123341865101) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(203.0324592590332) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.47) - val-aux/sciknoweval/reward/std@16:np.float64(0.2837672993079357) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5904) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2402826094689352) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.35114000000000006) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.24436774579815057) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.47056000000000003) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.28327999814077637) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.69132) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16729443275316552) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.25560000000000005) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.17568616332291914) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.47498000000000007) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.23885320678927968) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7524399999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09521451174706015) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.19168000000000002) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11550097103987941) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.47686) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.19049197326410838) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.78116) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0418313700168501) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.14496) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07387739109850737) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.47272000000000003) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1473585898790373) - val-aux/sciknoweval/score/mean@16:np.float64(0.47) - val-aux/sciknoweval/score/std@16:np.float64(0.2837672993079357) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5904) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2402826094689352) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.35114000000000006) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.24436774579815057) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.47056000000000003) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.28327999814077637) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.69132) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16729443275316552) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.25560000000000005) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.17568616332291914) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.47498000000000007) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.23885320678927968) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7524399999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09521451174706015) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.19168000000000002) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11550097103987941) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.47686) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.19049197326410838) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.78116) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0418313700168501) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.14496) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07387739109850737) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.47272000000000003) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1473585898790373) - val-core/sciknoweval/acc/mean@16:np.float64(0.47) - val-aux/sciknoweval/acc/std@16:np.float64(0.2837672993079357) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5904) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2402826094689352) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.35114000000000006) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.24436774579815057) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.47056000000000003) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.28327999814077637) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.69132) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16729443275316552) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.25560000000000005) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.17568616332291914) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.47498000000000007) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.23885320678927968) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7524399999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09521451174706015) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.19168000000000002) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11550097103987941) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.47686) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.19049197326410838) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.78116) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0418313700168501) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.14496) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07387739109850737) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.47272000000000003) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1473585898790373) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9925) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.014111477291628849) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9978) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0077466307237230615) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.98694) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0166223966239868) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9926599999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.013916492216164134) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998400000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0017816845961056067) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9797800000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.016581230241223443) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99618) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.010108844019402631) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9731799999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.014297367827488115) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99866) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005000439980641703) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9669200000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.010588030539694171) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998400000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.001781684596105607) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:4 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006638053338974714 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006638053338974714 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:258.703125 - response_length/max:1181.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:258.703125 - response_length_non_aborted/max:1181.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:324.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.406676352024078e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6390096675604582) - timing_s/agent_loop/generate_sequences/max:np.float64(8.397283358499408) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1178281594693544) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.397283358499408) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1181 - timing_s/gen:15.031767670065165 - timing_s/reward:0.05361991561949253 - timing_s/old_log_prob:3.0793004278093576 - timing_s/adv:0.572328319773078 - timing_s/update_actor:21.104915216565132 - timing_s/step:39.93690488487482 - timing_s/testing:81.71543750353158 - timing_s/save_checkpoint:13.095031322911382 - timing_s/stop_profile:0.00013700127601623535 - timing_per_token_ms/update_actor:0.14147281952383114 - timing_per_token_ms/gen:0.22696997750294687 - timing_per_token_ms/adv:0.003836494970995294 - perf/total_num_tokens:149180 - perf/time_per_step:39.93690488487482 - perf/throughput:466.92401561299533 (TaskRunner pid=2377588) Training Progress: 70%|███████ | 70/100 [1:09:48<35:40, 71.35s/it] (TaskRunner pid=2377588) step:71 - global_seqlen/min:14212 - global_seqlen/max:20949 - global_seqlen/minmax_diff:6737 - global_seqlen/balanced_min:18319 - global_seqlen/balanced_max:18448 - global_seqlen/mean:18337.5 - actor/entropy:0.40672412514686584 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24095088243484497 - training/rollout_probs_diff_mean:0.006252078805118799 - training/rollout_probs_diff_std:0.013184416107833385 - training/rollout_actor_probs_pearson_corr:0.9987838268280029 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000264644622803 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6697510480880737 - rollout_corr/rollout_is_min:0.5687577128410339 - rollout_corr/rollout_is_std:0.042977072298526764 - rollout_corr/rollout_is_eff_sample_size:0.9981564952634434 - rollout_corr/rollout_is_seq_mean:1.0000219345092773 - rollout_corr/rollout_is_seq_std:0.0029963974375277758 - rollout_corr/rollout_is_seq_max:1.0112249851226807 - rollout_corr/rollout_is_seq_min:0.9927828907966614 - rollout_corr/rollout_is_seq_max_deviation:0.011224985122680664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.509089607745409) - rollout_corr/training_log_ppl:np.float64(0.4068939457065426) - rollout_corr/kl:np.float64(0.0009392168688009406) - rollout_corr/k3_kl:np.float64(0.0011883202386115954) - rollout_corr/rollout_ppl:np.float64(1.507674849126488) - rollout_corr/rollout_log_ppl:np.float64(0.4059547278448008) - rollout_corr/log_ppl_diff:np.float64(0.0009392178617417812) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028873439878225327) - rollout_corr/log_ppl_diff_max:np.float64(0.014798790216445923) - rollout_corr/log_ppl_diff_min:np.float64(-0.009581387042999268) - rollout_corr/ppl_ratio:np.float64(1.0009459911379963) - rollout_corr/chi2_token:np.float64(0.0028433329425752163) - rollout_corr/chi2_seq:np.float64(1.480576429516077) - actor/pg_loss:np.float64(0.009686199831776321) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001594028693943983) - actor/ppo_kl:np.float64(3.150309135069307e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(82136.91651543835) - self_distillation/token_reweight_w_std:np.float64(1049356.8156914301) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018565838690847) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16540322510991246) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.4736870229244232) - perf/mfu/actor:np.float64(0.045126120652042294) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(210.3404884338379) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:5 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006460945587605238 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006460945587605238 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:250.453125 - response_length/max:748.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:250.453125 - response_length_non_aborted/max:748.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.59375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00035458803176879883 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3324593473225832) - timing_s/agent_loop/generate_sequences/max:np.float64(5.340911906212568) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8549201381974854) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.340911906212568) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:748 - timing_s/gen:11.884012870490551 - timing_s/reward:0.05087257735431194 - timing_s/old_log_prob:3.062375236302614 - timing_s/adv:0.5302242767065763 - timing_s/update_actor:21.16096432507038 - timing_s/step:36.80331253632903 - timing_s/stop_profile:2.7626752853393555e-05 - timing_per_token_ms/update_actor:0.14424651891663517 - timing_per_token_ms/gen:0.18535175105263196 - timing_per_token_ms/adv:0.0036143440811627564 - perf/total_num_tokens:146700 - perf/time_per_step:36.80331253632903 - perf/throughput:498.25678006290366 (TaskRunner pid=2377588) Training Progress: 71%|███████ | 71/100 [1:10:26<29:32, 61.11s/it] (TaskRunner pid=2377588) step:72 - global_seqlen/min:16697 - global_seqlen/max:21467 - global_seqlen/minmax_diff:4770 - global_seqlen/balanced_min:18983 - global_seqlen/balanced_max:18991 - global_seqlen/mean:18988.875 - actor/entropy:0.41097402572631836 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20542964339256287 - training/rollout_probs_diff_mean:0.006336473859846592 - training/rollout_probs_diff_std:0.013432835228741169 - training/rollout_actor_probs_pearson_corr:0.9987441897392273 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9997965097427368 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4659532098448835e-05 - rollout_corr/rollout_is_max:1.6137325763702393 - rollout_corr/rollout_is_min:0.4197999835014343 - rollout_corr/rollout_is_std:0.04281798005104065 - rollout_corr/rollout_is_eff_sample_size:0.9981692038333202 - rollout_corr/rollout_is_seq_mean:0.9997175931930542 - rollout_corr/rollout_is_seq_std:0.0026586030144244432 - rollout_corr/rollout_is_seq_max:1.006733775138855 - rollout_corr/rollout_is_seq_min:0.9914191961288452 - rollout_corr/rollout_is_seq_max_deviation:0.008580803871154785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5070015247911215) - rollout_corr/training_log_ppl:np.float64(0.4065636890591122) - rollout_corr/kl:np.float64(0.00140102456239255) - rollout_corr/k3_kl:np.float64(0.0011578064955983791) - rollout_corr/rollout_ppl:np.float64(1.5049114520661533) - rollout_corr/rollout_log_ppl:np.float64(0.40516266162740067) - rollout_corr/log_ppl_diff:np.float64(0.0014010274317115545) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025750676868483424) - rollout_corr/log_ppl_diff_max:np.float64(0.012268513441085815) - rollout_corr/log_ppl_diff_min:np.float64(-0.006662309169769287) - rollout_corr/ppl_ratio:np.float64(1.0014065490104258) - rollout_corr/chi2_token:np.float64(0.0018608283717185259) - rollout_corr/chi2_seq:np.float64(0.5882609779946506) - actor/pg_loss:np.float64(0.01144371135160327) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001373824377424171) - actor/ppo_kl:np.float64(0.00017792050590514918) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(66504.55616225093) - self_distillation/token_reweight_w_std:np.float64(920569.7068980994) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00057301716879) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1952171921147965) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.5355398431420326) - perf/mfu/actor:np.float64(0.04648076246703062) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.50950050354004) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:5 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006309096235781908 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006309096235781908 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:266.46484375 - response_length/max:651.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.46484375 - response_length_non_aborted/max:651.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:326.9375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.487484693527222e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6174178533256054) - timing_s/agent_loop/generate_sequences/max:np.float64(5.815540179610252) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2066296639095526) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.815540179610252) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:651 - timing_s/gen:12.6222914326936 - timing_s/reward:0.05736719258129597 - timing_s/old_log_prob:2.8969801031053066 - timing_s/adv:0.5163711216300726 - timing_s/update_actor:20.891388142481446 - timing_s/step:37.074501655995846 - timing_s/stop_profile:2.874433994293213e-05 - timing_per_token_ms/update_actor:0.13752386688575183 - timing_per_token_ms/gen:0.18503688972650592 - timing_per_token_ms/adv:0.003399168734522665 - perf/total_num_tokens:151911 - perf/time_per_step:37.074501655995846 - perf/throughput:512.181530481315 (TaskRunner pid=2377588) Training Progress: 72%|███████▏ | 72/100 [1:11:03<25:09, 53.91s/it] (TaskRunner pid=2377588) step:73 - global_seqlen/min:14215 - global_seqlen/max:19696 - global_seqlen/minmax_diff:5481 - global_seqlen/balanced_min:16555 - global_seqlen/balanced_max:16562 - global_seqlen/mean:16559.875 - actor/entropy:0.41698652505874634 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1791788935661316 - training/rollout_probs_diff_mean:0.006307457108050585 - training/rollout_probs_diff_std:0.01325792446732521 - training/rollout_actor_probs_pearson_corr:0.9987829923629761 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.000073790550232 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.563696026802063 - rollout_corr/rollout_is_min:0.6180529594421387 - rollout_corr/rollout_is_std:0.04264990612864494 - rollout_corr/rollout_is_eff_sample_size:0.9981846445872994 - rollout_corr/rollout_is_seq_mean:1.000035047531128 - rollout_corr/rollout_is_seq_std:0.002873131539672613 - rollout_corr/rollout_is_seq_max:1.0085272789001465 - rollout_corr/rollout_is_seq_min:0.9900835156440735 - rollout_corr/rollout_is_seq_max_deviation:0.009916484355926514 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5194862117059529) - rollout_corr/training_log_ppl:np.float64(0.4137849659891799) - rollout_corr/kl:np.float64(0.0009323785048565014) - rollout_corr/k3_kl:np.float64(0.0010895648143787184) - rollout_corr/rollout_ppl:np.float64(1.5180072654038668) - rollout_corr/rollout_log_ppl:np.float64(0.41285258351126686) - rollout_corr/log_ppl_diff:np.float64(0.0009323824779130518) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025225929566659033) - rollout_corr/log_ppl_diff_max:np.float64(0.012218058109283447) - rollout_corr/log_ppl_diff_min:np.float64(-0.010018676519393921) - rollout_corr/ppl_ratio:np.float64(1.0009376609232277) - rollout_corr/chi2_token:np.float64(0.002501161303371191) - rollout_corr/chi2_seq:np.float64(1.0584821668453515) - actor/pg_loss:np.float64(0.0050578610971570015) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006173019578454841) - actor/ppo_kl:np.float64(5.3568359978939384e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(5927.540609702701) - self_distillation/token_reweight_w_std:np.float64(97029.73679834942) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011405171826482) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09390432483633049) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.3838684782385826) - perf/mfu/actor:np.float64(0.040822386302397096) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.97954177856445) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:5 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00441079493612051 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00441079493612051 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:248.74609375 - response_length/max:586.0 - response_length/min:137.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:248.74609375 - response_length_non_aborted/max:586.0 - response_length_non_aborted/min:137.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010230764746665955 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6381701808422804) - timing_s/agent_loop/generate_sequences/max:np.float64(4.995817299932241) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.006518998030515) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.995817299932241) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:586 - timing_s/gen:12.085897158831358 - timing_s/reward:0.048337629064917564 - timing_s/old_log_prob:2.985896199941635 - timing_s/adv:0.4870305508375168 - timing_s/update_actor:20.904740303754807 - timing_s/step:36.61216112039983 - timing_s/stop_profile:2.9053539037704468e-05 - timing_per_token_ms/update_actor:0.15779663421187362 - timing_per_token_ms/gen:0.18979407903439688 - timing_per_token_ms/adv:0.003676284926950813 - perf/total_num_tokens:132479 - perf/time_per_step:36.61216112039983 - perf/throughput:452.30531313195405 (TaskRunner pid=2377588) Training Progress: 73%|███████▎ | 73/100 [1:11:39<21:55, 48.72s/it] (TaskRunner pid=2377588) step:74 - global_seqlen/min:14815 - global_seqlen/max:21513 - global_seqlen/minmax_diff:6698 - global_seqlen/balanced_min:18169 - global_seqlen/balanced_max:18303 - global_seqlen/mean:18194.0 - actor/entropy:0.3989974558353424 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6189537048339844 - training/rollout_probs_diff_mean:0.006225051824003458 - training/rollout_probs_diff_std:0.013746981509029865 - training/rollout_actor_probs_pearson_corr:0.9986744523048401 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.45703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:0.9999681115150452 - rollout_corr/rollout_is_ratio_fraction_high:1.4962892237235792e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.0699892044067383 - rollout_corr/rollout_is_min:0.5333900451660156 - rollout_corr/rollout_is_std:0.04285763204097748 - rollout_corr/rollout_is_eff_sample_size:0.9981664720573507 - rollout_corr/rollout_is_seq_mean:0.9999874830245972 - rollout_corr/rollout_is_seq_std:0.0030997011344879866 - rollout_corr/rollout_is_seq_max:1.0085495710372925 - rollout_corr/rollout_is_seq_min:0.9903734922409058 - rollout_corr/rollout_is_seq_max_deviation:0.009626507759094238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4952381704933941) - rollout_corr/training_log_ppl:np.float64(0.39897476590704173) - rollout_corr/kl:np.float64(0.000997198554259171) - rollout_corr/k3_kl:np.float64(0.0011406938412505951) - rollout_corr/rollout_ppl:np.float64(1.4937100345268846) - rollout_corr/rollout_log_ppl:np.float64(0.39797756366897374) - rollout_corr/log_ppl_diff:np.float64(0.0009972022380679846) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024678701302036643) - rollout_corr/log_ppl_diff_max:np.float64(0.009318262338638306) - rollout_corr/log_ppl_diff_min:np.float64(-0.006970793008804321) - rollout_corr/ppl_ratio:np.float64(1.0010020148474723) - rollout_corr/chi2_token:np.float64(0.0026073616463690996) - rollout_corr/chi2_seq:np.float64(0.5572159530129284) - actor/pg_loss:np.float64(0.0092886823695153) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017479008729424095) - actor/ppo_kl:np.float64(3.734449150272212e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.45703125) - self_distillation/token_reweight_w_mean:np.float64(57163.51866311091) - self_distillation/token_reweight_w_std:np.float64(861802.6223666023) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017101152334362) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16699737549060956) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.469961017370224) - perf/mfu/actor:np.float64(0.04412343533159409) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.8795623779297) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:5 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002545562107115984 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002545562107115984 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.0625 - response_length/max:820.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.0625 - response_length_non_aborted/max:820.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.5 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.434477210044861e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5130960755050182) - timing_s/agent_loop/generate_sequences/max:np.float64(6.610897928476334) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2031659333370044) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.610897928476334) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:820 - timing_s/gen:13.32564501836896 - timing_s/reward:0.057171694934368134 - timing_s/old_log_prob:2.861457584425807 - timing_s/adv:0.5242922082543373 - timing_s/update_actor:21.231569780036807 - timing_s/step:38.08603209629655 - timing_s/stop_profile:5.274079740047455e-05 - timing_per_token_ms/update_actor:0.14586930980018692 - timing_per_token_ms/gen:0.19939018761026095 - timing_per_token_ms/adv:0.0036020955277449797 - perf/total_num_tokens:145552 - perf/time_per_step:38.08603209629655 - perf/throughput:477.707941693647 (TaskRunner pid=2377588) Training Progress: 74%|███████▍ | 74/100 [1:12:17<19:44, 45.54s/it] (TaskRunner pid=2377588) step:75 - global_seqlen/min:14572 - global_seqlen/max:21312 - global_seqlen/minmax_diff:6740 - global_seqlen/balanced_min:18967 - global_seqlen/balanced_max:18970 - global_seqlen/mean:18968.875 - actor/entropy:0.39935535192489624 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23124143481254578 - training/rollout_probs_diff_mean:0.006188345607370138 - training/rollout_probs_diff_std:0.013194930739700794 - training/rollout_actor_probs_pearson_corr:0.9987638592720032 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.0000230073928833 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5533980331383646e-05 - rollout_corr/rollout_is_max:1.6297835111618042 - rollout_corr/rollout_is_min:0.4850001335144043 - rollout_corr/rollout_is_std:0.043572090566158295 - rollout_corr/rollout_is_eff_sample_size:0.9981051891877871 - rollout_corr/rollout_is_seq_mean:0.9999933242797852 - rollout_corr/rollout_is_seq_std:0.0030989055521786213 - rollout_corr/rollout_is_seq_max:1.0090453624725342 - rollout_corr/rollout_is_seq_min:0.9898204207420349 - rollout_corr/rollout_is_seq_max_deviation:0.010179579257965088 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5011848765425384) - rollout_corr/training_log_ppl:np.float64(0.40088130172807723) - rollout_corr/kl:np.float64(0.0009357360472144194) - rollout_corr/k3_kl:np.float64(0.0010998197089975292) - rollout_corr/rollout_ppl:np.float64(1.4998769955709577) - rollout_corr/rollout_log_ppl:np.float64(0.39994556666351855) - rollout_corr/log_ppl_diff:np.float64(0.0009357350645586848) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026455705519765615) - rollout_corr/log_ppl_diff_max:np.float64(0.010016322135925293) - rollout_corr/log_ppl_diff_min:np.float64(-0.009813278913497925) - rollout_corr/ppl_ratio:np.float64(1.0009412812069058) - rollout_corr/chi2_token:np.float64(0.0025881784968078136) - rollout_corr/chi2_seq:np.float64(1.5566936507821083) - actor/pg_loss:np.float64(0.004741978365927935) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009998526556955767) - actor/ppo_kl:np.float64(-4.0160996810811866e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(41332.166137804044) - self_distillation/token_reweight_w_std:np.float64(635102.5428741215) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003706857096404) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11741762660676613) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.36183500476181507) - perf/mfu/actor:np.float64(0.046820113573180513) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.2332305908203) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:5 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004097087308764458 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004097087308764458 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:251.46484375 - response_length/max:495.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:251.46484375 - response_length_non_aborted/max:495.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:341.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.731227040290833e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5393935684114695) - timing_s/agent_loop/generate_sequences/max:np.float64(4.907172447070479) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1264512085472234) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.907172447070479) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:495 - timing_s/gen:11.543874807655811 - timing_s/reward:0.0575887206941843 - timing_s/old_log_prob:2.9839358050376177 - timing_s/adv:0.9345375504344702 - timing_s/update_actor:20.586461452767253 - timing_s/step:36.196603674441576 - timing_s/stop_profile:3.847293555736542e-05 - timing_per_token_ms/update_actor:0.13565947804473943 - timing_per_token_ms/gen:0.17932232710921647 - timing_per_token_ms/adv:0.0061583617270032495 - perf/total_num_tokens:151751 - perf/time_per_step:36.196603674441576 - perf/throughput:524.0512389120619 (TaskRunner pid=2377588) Training Progress: 75%|███████▌ | 75/100 [1:12:54<17:48, 42.74s/it] (TaskRunner pid=2377588) step:76 - global_seqlen/min:14103 - global_seqlen/max:20745 - global_seqlen/minmax_diff:6642 - global_seqlen/balanced_min:17166 - global_seqlen/balanced_max:17169 - global_seqlen/mean:17168.125 - actor/entropy:0.42019733786582947 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23893426358699799 - training/rollout_probs_diff_mean:0.006229669786989689 - training/rollout_probs_diff_std:0.013057836331427097 - training/rollout_actor_probs_pearson_corr:0.9988378882408142 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:0.9999080300331116 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7706623077392578 - rollout_corr/rollout_is_min:0.5054554343223572 - rollout_corr/rollout_is_std:0.04271693527698517 - rollout_corr/rollout_is_eff_sample_size:0.9981782306776434 - rollout_corr/rollout_is_seq_mean:0.9998851418495178 - rollout_corr/rollout_is_seq_std:0.0030740597285330296 - rollout_corr/rollout_is_seq_max:1.0123316049575806 - rollout_corr/rollout_is_seq_min:0.9912843108177185 - rollout_corr/rollout_is_seq_max_deviation:0.012331604957580566 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5248130233958364) - rollout_corr/training_log_ppl:np.float64(0.4181599976727739) - rollout_corr/kl:np.float64(0.0012952181327534618) - rollout_corr/k3_kl:np.float64(0.001179108729274958) - rollout_corr/rollout_ppl:np.float64(1.5228950078599155) - rollout_corr/rollout_log_ppl:np.float64(0.41686478327028453) - rollout_corr/log_ppl_diff:np.float64(0.0012952144024893641) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027736180927604437) - rollout_corr/log_ppl_diff_max:np.float64(0.010423421859741211) - rollout_corr/log_ppl_diff_min:np.float64(-0.01137995719909668) - rollout_corr/ppl_ratio:np.float64(1.0013016629964113) - rollout_corr/chi2_token:np.float64(0.002266357885673642) - rollout_corr/chi2_seq:np.float64(0.8077209983021021) - actor/pg_loss:np.float64(0.005860538454726338) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011645920758383) - actor/ppo_kl:np.float64(0.00026321933361828087) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(27174.76727405307) - self_distillation/token_reweight_w_std:np.float64(450165.1545943306) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003046297933906) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13639010477345437) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.4141102731227875) - perf/mfu/actor:np.float64(0.04274691369157613) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.19745254516602) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:5 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007379510905593634 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007379510905593634 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:246.47265625 - response_length/max:483.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:246.47265625 - response_length_non_aborted/max:483.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.03125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.950429916381836e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5690273344516754) - timing_s/agent_loop/generate_sequences/max:np.float64(4.661921402439475) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.989754891314078) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.661921402439475) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:483 - timing_s/gen:11.778079733252525 - timing_s/reward:0.05512377433478832 - timing_s/old_log_prob:2.946628851816058 - timing_s/adv:0.46686897426843643 - timing_s/update_actor:20.41484851948917 - timing_s/step:35.751363379880786 - timing_s/stop_profile:0.00012178346514701843 - timing_per_token_ms/update_actor:0.14863918249291325 - timing_per_token_ms/gen:0.18666623980938118 - timing_per_token_ms/adv:0.0033992425954234694 - perf/total_num_tokens:137345 - perf/time_per_step:35.751363379880786 - perf/throughput:480.20895923822104 (TaskRunner pid=2377588) Training Progress: 76%|███████▌ | 76/100 [1:13:29<16:15, 40.66s/it] (TaskRunner pid=2377588) step:77 - global_seqlen/min:16104 - global_seqlen/max:21019 - global_seqlen/minmax_diff:4915 - global_seqlen/balanced_min:17778 - global_seqlen/balanced_max:17945 - global_seqlen/mean:17799.75 - actor/entropy:0.43006381392478943 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43951284885406494 - training/rollout_probs_diff_mean:0.0062884679064154625 - training/rollout_probs_diff_std:0.013276039622724056 - training/rollout_actor_probs_pearson_corr:0.9988093972206116 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.921875 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.921875 - rollout_corr/rollout_is_mean:1.0000693798065186 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.291314351372421e-05 - rollout_corr/rollout_is_max:1.6985567808151245 - rollout_corr/rollout_is_min:0.2780825197696686 - rollout_corr/rollout_is_std:0.04305189847946167 - rollout_corr/rollout_is_eff_sample_size:0.9981503192528741 - rollout_corr/rollout_is_seq_mean:1.0000379085540771 - rollout_corr/rollout_is_seq_std:0.002884289249777794 - rollout_corr/rollout_is_seq_max:1.0086594820022583 - rollout_corr/rollout_is_seq_min:0.9893074631690979 - rollout_corr/rollout_is_seq_max_deviation:0.0106925368309021 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5411567692644894) - rollout_corr/training_log_ppl:np.float64(0.4282152239466086) - rollout_corr/kl:np.float64(0.0012112704294651166) - rollout_corr/k3_kl:np.float64(0.0011300616266680663) - rollout_corr/rollout_ppl:np.float64(1.5393052310682833) - rollout_corr/rollout_log_ppl:np.float64(0.4270039537223056) - rollout_corr/log_ppl_diff:np.float64(0.0012112702243030071) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027090799994766712) - rollout_corr/log_ppl_diff_max:np.float64(0.009529411792755127) - rollout_corr/log_ppl_diff_min:np.float64(-0.007061272859573364) - rollout_corr/ppl_ratio:np.float64(1.001217087265104) - rollout_corr/chi2_token:np.float64(0.002177898772060871) - rollout_corr/chi2_seq:np.float64(1.0026340745389462) - actor/pg_loss:np.float64(0.008232398540712893) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010347240558985504) - actor/ppo_kl:np.float64(0.0003149729090914377) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(57160.41103560012) - self_distillation/token_reweight_w_std:np.float64(864967.8420132976) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007096098270267) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18555074220057577) - self_distillation/empty_target_batch:np.float64(0.078125) - actor/grad_norm:np.float64(0.48504846543073654) - perf/mfu/actor:np.float64(0.04386250769908005) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.19208145141602) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:5 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011542227119207382 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011542227119207382 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:237.3671875 - response_length/max:742.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:237.3671875 - response_length_non_aborted/max:742.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.496882557868958e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5573131944984198) - timing_s/agent_loop/generate_sequences/max:np.float64(5.703901177272201) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.818815386330243) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.703901177272201) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:742 - timing_s/gen:12.793293848633766 - timing_s/reward:0.057360172271728516 - timing_s/old_log_prob:2.92247729934752 - timing_s/adv:0.5316014718264341 - timing_s/update_actor:20.818322157487273 - timing_s/step:37.210345689207315 - timing_s/stop_profile:0.00010646134614944458 - timing_per_token_ms/update_actor:0.1461981359112296 - timing_per_token_ms/gen:0.21053374993637505 - timing_per_token_ms/adv:0.003733208835983891 - perf/total_num_tokens:142398 - perf/time_per_step:37.210345689207315 - perf/throughput:478.35486798938103 (TaskRunner pid=2377588) Training Progress: 77%|███████▋ | 77/100 [1:14:07<15:11, 39.64s/it] (TaskRunner pid=2377588) step:78 - global_seqlen/min:16263 - global_seqlen/max:22150 - global_seqlen/minmax_diff:5887 - global_seqlen/balanced_min:18563 - global_seqlen/balanced_max:18913 - global_seqlen/mean:18630.875 - actor/entropy:0.44512832164764404 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24057871103286743 - training/rollout_probs_diff_mean:0.005958838388323784 - training/rollout_probs_diff_std:0.012366652488708496 - training/rollout_actor_probs_pearson_corr:0.998964250087738 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0002938508987427 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7588984966278076 - rollout_corr/rollout_is_min:0.5031510591506958 - rollout_corr/rollout_is_std:0.04129081219434738 - rollout_corr/rollout_is_eff_sample_size:0.9982988022115691 - rollout_corr/rollout_is_seq_mean:1.0003224611282349 - rollout_corr/rollout_is_seq_std:0.002923476044088602 - rollout_corr/rollout_is_seq_max:1.0111260414123535 - rollout_corr/rollout_is_seq_min:0.9919884204864502 - rollout_corr/rollout_is_seq_max_deviation:0.011126041412353516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5644961800426245) - rollout_corr/training_log_ppl:np.float64(0.44120982865570113) - rollout_corr/kl:np.float64(0.0007225766881724383) - rollout_corr/k3_kl:np.float64(0.0009593505167799776) - rollout_corr/rollout_ppl:np.float64(1.563377182930708) - rollout_corr/rollout_log_ppl:np.float64(0.4404872533050366) - rollout_corr/log_ppl_diff:np.float64(0.000722575350664556) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022972217993810773) - rollout_corr/log_ppl_diff_max:np.float64(0.008512794971466064) - rollout_corr/log_ppl_diff_min:np.float64(-0.009968310594558716) - rollout_corr/ppl_ratio:np.float64(1.0007267771288753) - rollout_corr/chi2_token:np.float64(0.002432366833090782) - rollout_corr/chi2_seq:np.float64(0.7096504950895905) - actor/pg_loss:np.float64(0.006571984151378274) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009601590272723115) - actor/ppo_kl:np.float64(0.00018090755597732766) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(66216.84291877202) - self_distillation/token_reweight_w_std:np.float64(938285.9228849028) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9980305195786059) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15120434225536883) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.4234946370124817) - perf/mfu/actor:np.float64(0.045782685585364344) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.2100067138672) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:5 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0177642609924078 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0177642609924078 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:253.83984375 - response_length/max:996.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:253.83984375 - response_length_non_aborted/max:996.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.831918239593506e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8019796553999186) - timing_s/agent_loop/generate_sequences/max:np.float64(7.383664488792419) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.008693139134266) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.383664488792419) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:996 - timing_s/gen:14.282923229038715 - timing_s/reward:0.05803539976477623 - timing_s/old_log_prob:2.9167770594358444 - timing_s/adv:0.48767990060150623 - timing_s/update_actor:20.92854812927544 - timing_s/step:38.76219773106277 - timing_s/stop_profile:0.00011512823402881622 - timing_per_token_ms/update_actor:0.1404157623385606 - timing_per_token_ms/gen:0.21979476523150232 - timing_per_token_ms/adv:0.0032719873637275907 - perf/total_num_tokens:149047 - perf/time_per_step:38.76219773106277 - perf/throughput:480.6454765352435 (TaskRunner pid=2377588) Training Progress: 78%|███████▊ | 78/100 [1:14:45<14:26, 39.39s/it] (TaskRunner pid=2377588) step:79 - global_seqlen/min:15101 - global_seqlen/max:20604 - global_seqlen/minmax_diff:5503 - global_seqlen/balanced_min:17877 - global_seqlen/balanced_max:17884 - global_seqlen/mean:17882.0 - actor/entropy:0.4710043668746948 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15804535150527954 - training/rollout_probs_diff_mean:0.006219003815203905 - training/rollout_probs_diff_std:0.012460929341614246 - training/rollout_actor_probs_pearson_corr:0.9989616870880127 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.703125 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.703125 - rollout_corr/rollout_is_mean:0.9999518990516663 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.571388840675354 - rollout_corr/rollout_is_min:0.6191216111183167 - rollout_corr/rollout_is_std:0.04181438311934471 - rollout_corr/rollout_is_eff_sample_size:0.9982544902726032 - rollout_corr/rollout_is_seq_mean:0.9999338388442993 - rollout_corr/rollout_is_seq_std:0.0027393146883696318 - rollout_corr/rollout_is_seq_max:1.0075207948684692 - rollout_corr/rollout_is_seq_min:0.9918217658996582 - rollout_corr/rollout_is_seq_max_deviation:0.008178234100341797 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6004510917700827) - rollout_corr/training_log_ppl:np.float64(0.46530697878915817) - rollout_corr/kl:np.float64(0.0011387883292712786) - rollout_corr/k3_kl:np.float64(0.0010031971287389752) - rollout_corr/rollout_ppl:np.float64(1.5985692567192018) - rollout_corr/rollout_log_ppl:np.float64(0.464168187987525) - rollout_corr/log_ppl_diff:np.float64(0.0011387908016331494) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023946556611917913) - rollout_corr/log_ppl_diff_max:np.float64(0.010749489068984985) - rollout_corr/log_ppl_diff_min:np.float64(-0.005749821662902832) - rollout_corr/ppl_ratio:np.float64(1.0011434799525887) - rollout_corr/chi2_token:np.float64(0.0017312881536781788) - rollout_corr/chi2_seq:np.float64(0.3756079205777496) - actor/pg_loss:np.float64(0.005288981716148555) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012618467799256905) - actor/ppo_kl:np.float64(0.00020063513726498172) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(55742.86985031888) - self_distillation/token_reweight_w_std:np.float64(800661.5002698549) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002181438729167) - self_distillation/token_reweight_w_clip_frac:np.float64(0.136719107395038) - self_distillation/empty_target_batch:np.float64(0.296875) - actor/grad_norm:np.float64(0.3855058550834656) - perf/mfu/actor:np.float64(0.044050629168505356) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.16245651245117) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:5 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002076678676530719 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002076678676530719 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:249.46875 - response_length/max:583.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:249.46875 - response_length_non_aborted/max:583.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.34375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010392628610134125 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6932244282215834) - timing_s/agent_loop/generate_sequences/max:np.float64(5.08374061435461) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.030889444577042) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.08374061435461) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:583 - timing_s/gen:12.399973466992378 - timing_s/reward:0.05726361274719238 - timing_s/old_log_prob:2.955918923020363 - timing_s/adv:0.4804356414824724 - timing_s/update_actor:20.80588993616402 - timing_s/step:36.78768317960203 - timing_s/stop_profile:0.00011498667299747467 - timing_per_token_ms/update_actor:0.145438778773096 - timing_per_token_ms/gen:0.19416218005437144 - timing_per_token_ms/adv:0.0033583746328883263 - perf/total_num_tokens:143056 - perf/time_per_step:36.78768317960203 - perf/throughput:486.08660438598054 (TaskRunner pid=2377588) Training Progress: 79%|███████▉ | 79/100 [1:15:22<13:31, 38.62s/it] [36m(TaskRunner pid=2377588) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2377588) validation generation end (TaskRunner pid=2377588) [prompt] system (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2377588) (TaskRunner pid=2377588) A (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) user (TaskRunner pid=2377588) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2377588) (TaskRunner pid=2377588) A: -0.01 (TaskRunner pid=2377588) B: 1.69 (TaskRunner pid=2377588) C: 2.49 (TaskRunner pid=2377588) D: 0.45 (TaskRunner pid=2377588) Please reason step by step. (TaskRunner pid=2377588) assistant (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) [response] (TaskRunner pid=2377588) To determine the folding stability score of the given protein sequence, we must consider the balance of hydrophobic and hydrophilic residues, as well as the potential for forming stable secondary structures such as alpha-helices and beta-sheets. The sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR' contains a mix of both hydrophilic and hydrophobic residues, which is typical for proteins with moderate stability. (TaskRunner pid=2377588) (TaskRunner pid=2377588) However, the presence of multiple charged residues (e.g., lysine, arginine, glutamic acid, aspartic acid) and some hydrophobic residues (e.g., glycine, serine, tryptophan) suggests a moderate tendency for folding. The overall structure appears to have a reasonable balance between stabilizing and destabilizing interactions, which would result in a moderate stability score. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Among the given options, 0.45 (D) represents a moderate stability score, which aligns with the observed composition of the sequence. This score is neither too low (indicating instability) nor too high (indicating strong stability), making it the most reasonable choice. (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) D (TaskRunner pid=2377588) (TaskRunner pid=2377588) [ground_truth] B (TaskRunner pid=2377588) [score] 0.0 (TaskRunner pid=2377588) [acc] 0.0 (TaskRunner pid=2377588) [pred] D (TaskRunner pid=2377588) [incorrect_format] 1 (TaskRunner pid=2377588) [feedback] (TaskRunner pid=2377588) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2377588) step:80 - global_seqlen/min:14745 - global_seqlen/max:21707 - global_seqlen/minmax_diff:6962 - global_seqlen/balanced_min:17639 - global_seqlen/balanced_max:17642 - global_seqlen/mean:17640.875 - actor/entropy:0.46099135279655457 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18779629468917847 - training/rollout_probs_diff_mean:0.006202006246894598 - training/rollout_probs_diff_std:0.012649651616811752 - training/rollout_actor_probs_pearson_corr:0.9989561438560486 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.92578125 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.92578125 - rollout_corr/rollout_is_mean:1.0001035928726196 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5461302995681763 - rollout_corr/rollout_is_min:0.5686233043670654 - rollout_corr/rollout_is_std:0.04139605537056923 - rollout_corr/rollout_is_eff_sample_size:0.9982897731863902 - rollout_corr/rollout_is_seq_mean:1.000148057937622 - rollout_corr/rollout_is_seq_std:0.002706470899283886 - rollout_corr/rollout_is_seq_max:1.0066161155700684 - rollout_corr/rollout_is_seq_min:0.9909796714782715 - rollout_corr/rollout_is_seq_max_deviation:0.009020328521728516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5861644055694342) - rollout_corr/training_log_ppl:np.float64(0.45581070496700704) - rollout_corr/kl:np.float64(0.0007898373794752445) - rollout_corr/k3_kl:np.float64(0.0010849451176682123) - rollout_corr/rollout_ppl:np.float64(1.5848036441020668) - rollout_corr/rollout_log_ppl:np.float64(0.4550208683940582) - rollout_corr/log_ppl_diff:np.float64(0.0007898365729488432) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026570780319161713) - rollout_corr/log_ppl_diff_max:np.float64(0.010708749294281006) - rollout_corr/log_ppl_diff_min:np.float64(-0.00996658205986023) - rollout_corr/ppl_ratio:np.float64(1.0007959597278386) - rollout_corr/chi2_token:np.float64(0.0027055551763623953) - rollout_corr/chi2_seq:np.float64(0.9344324474222958) - actor/pg_loss:np.float64(0.0073711306322366) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011368925797796692) - actor/ppo_kl:np.float64(6.432518136278986e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.92578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.92578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(38544.862787333084) - self_distillation/token_reweight_w_std:np.float64(604984.8078821932) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000675871502608) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1685500395251438) - self_distillation/empty_target_batch:np.float64(0.07421875) - actor/grad_norm:np.float64(0.45776859670877457) - perf/mfu/actor:np.float64(0.04392033948415637) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.0953483581543) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.50875) - val-aux/sciknoweval/reward/std@16:np.float64(0.2565734750979684) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6181199999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.20899415772258045) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.40186) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.22353402018391097) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5111) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2547400139704645) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7021200000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14144751909060901) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.3121) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16563369466627514) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5203599999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2129672899008944) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.755) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07804881779500938) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.25171999999999994) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10872840325106338) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5241600000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1643937540364137) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7843000000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04020183310407499) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.20888) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07474767892062296) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.52946) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13132965602650037) - val-aux/sciknoweval/score/mean@16:np.float64(0.50875) - val-aux/sciknoweval/score/std@16:np.float64(0.2565734750979684) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6181199999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.20899415772258045) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.40186) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.22353402018391097) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5111) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2547400139704645) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7021200000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14144751909060901) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.3121) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.16563369466627514) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5203599999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2129672899008944) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.755) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07804881779500938) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.25171999999999994) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10872840325106338) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5241600000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1643937540364137) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7843000000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04020183310407499) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.20888) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07474767892062296) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.52946) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13132965602650037) - val-core/sciknoweval/acc/mean@16:np.float64(0.50875) - val-aux/sciknoweval/acc/std@16:np.float64(0.2565734750979684) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6181199999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.20899415772258045) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.40186) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.22353402018391097) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5111) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2547400139704645) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7021200000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14144751909060901) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.3121) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.16563369466627514) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5203599999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2129672899008944) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.755) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07804881779500938) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.25171999999999994) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10872840325106338) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5241600000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1643937540364137) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7843000000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04020183310407499) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.20888) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07474767892062296) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.52946) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13132965602650037) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9925) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.014111477291628849) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99812) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.007105548486846941) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.98748) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.016227002618158883) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9929000000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.013754831187404181) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999000000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0014106735979665889) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9803200000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01653821939206084) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9967) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.009572477584771494) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9731000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.013901081206931334) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99914) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0052345982834173885) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9671599999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.010469836404886586) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0008935323161475471) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:5 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0055116587318480015 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0055116587318480015 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:247.43359375 - response_length/max:426.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:247.43359375 - response_length_non_aborted/max:426.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.84375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001433156430721283 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5652617756277323) - timing_s/agent_loop/generate_sequences/max:np.float64(4.414252460002899) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0865544960543048) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.414252460002899) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:426 - timing_s/gen:11.286029623821378 - timing_s/reward:0.05782262608408928 - timing_s/old_log_prob:2.9133325330913067 - timing_s/adv:0.5020781513303518 - timing_s/update_actor:20.370424734428525 - timing_s/step:35.21862106397748 - timing_s/testing:81.87943293899298 - timing_s/save_checkpoint:12.798194015398622 - timing_s/stop_profile:0.0001523047685623169 - timing_per_token_ms/update_actor:0.14434108805847587 - timing_per_token_ms/gen:0.1781732728765827 - timing_per_token_ms/adv:0.0035576335593497477 - perf/total_num_tokens:141127 - perf/time_per_step:35.21862106397748 - perf/throughput:500.896243721579 (TaskRunner pid=2377588) Training Progress: 80%|████████ | 80/100 [1:17:32<22:00, 66.03s/it] (TaskRunner pid=2377588) step:81 - global_seqlen/min:16852 - global_seqlen/max:21893 - global_seqlen/minmax_diff:5041 - global_seqlen/balanced_min:19224 - global_seqlen/balanced_max:19229 - global_seqlen/mean:19226.875 - actor/entropy:0.41335776448249817 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40274474024772644 - training/rollout_probs_diff_mean:0.006201711483299732 - training/rollout_probs_diff_std:0.013162323273718357 - training/rollout_actor_probs_pearson_corr:0.9987941384315491 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.0000382661819458 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.1534302252111956e-05 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:0.3979531228542328 - rollout_corr/rollout_is_std:0.04318460449576378 - rollout_corr/rollout_is_eff_sample_size:0.9981386800560812 - rollout_corr/rollout_is_seq_mean:1.0000500679016113 - rollout_corr/rollout_is_seq_std:0.0025935645680874586 - rollout_corr/rollout_is_seq_max:1.00809907913208 - rollout_corr/rollout_is_seq_min:0.9929088354110718 - rollout_corr/rollout_is_seq_max_deviation:0.008099079132080078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5252565988339484) - rollout_corr/training_log_ppl:np.float64(0.41658992163138464) - rollout_corr/kl:np.float64(0.0015415017732038905) - rollout_corr/k3_kl:np.float64(0.00124868572061132) - rollout_corr/rollout_ppl:np.float64(1.5228456882759929) - rollout_corr/rollout_log_ppl:np.float64(0.4150484151323326) - rollout_corr/log_ppl_diff:np.float64(0.0015415064990520477) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027573411352932453) - rollout_corr/log_ppl_diff_max:np.float64(0.014636814594268799) - rollout_corr/log_ppl_diff_min:np.float64(-0.008184581995010376) - rollout_corr/ppl_ratio:np.float64(1.0015475151594728) - rollout_corr/chi2_token:np.float64(0.0018155982252210379) - rollout_corr/chi2_seq:np.float64(0.41389535763300955) - actor/pg_loss:np.float64(0.006863915361464024) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012353447327768663) - actor/ppo_kl:np.float64(0.0006445885232295723) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(71704.61414219439) - self_distillation/token_reweight_w_std:np.float64(1043082.9806143515) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9978634407743812) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13482154754456133) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.4011821821331978) - perf/mfu/actor:np.float64(0.04735533434874736) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(210.30975341796875) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:5 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00473802862688899 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00473802862688899 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:247.74609375 - response_length/max:512.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:247.74609375 - response_length_non_aborted/max:512.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:353.09375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.013339877128601e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6743445005267859) - timing_s/agent_loop/generate_sequences/max:np.float64(4.776466842740774) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9594508275549742) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.776466842740774) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:512 - timing_s/gen:11.101008331403136 - timing_s/reward:0.05520625598728657 - timing_s/old_log_prob:3.050413876771927 - timing_s/adv:0.5274984035640955 - timing_s/update_actor:20.789001619443297 - timing_s/step:35.629312144592404 - timing_s/stop_profile:0.00012319348752498627 - timing_per_token_ms/update_actor:0.13515587959199882 - timing_per_token_ms/gen:0.17503127148515737 - timing_per_token_ms/adv:0.0034294340835685434 - perf/total_num_tokens:153815 - perf/time_per_step:35.629312144592404 - perf/throughput:539.6364353589727 (TaskRunner pid=2377588) Training Progress: 81%|████████ | 81/100 [1:18:08<18:01, 56.92s/it] (TaskRunner pid=2377588) step:82 - global_seqlen/min:14382 - global_seqlen/max:20771 - global_seqlen/minmax_diff:6389 - global_seqlen/balanced_min:17736 - global_seqlen/balanced_max:17740 - global_seqlen/mean:17738.875 - actor/entropy:0.41357848048210144 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2099715769290924 - training/rollout_probs_diff_mean:0.006365329958498478 - training/rollout_probs_diff_std:0.013332873582839966 - training/rollout_actor_probs_pearson_corr:0.9987560510635376 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.88671875 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.88671875 - rollout_corr/rollout_is_mean:0.9998486638069153 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.594619631767273 - rollout_corr/rollout_is_min:0.5670798420906067 - rollout_corr/rollout_is_std:0.04371821880340576 - rollout_corr/rollout_is_eff_sample_size:0.998091769718796 - rollout_corr/rollout_is_seq_mean:0.9998258352279663 - rollout_corr/rollout_is_seq_std:0.0027360182721167803 - rollout_corr/rollout_is_seq_max:1.0073590278625488 - rollout_corr/rollout_is_seq_min:0.991263210773468 - rollout_corr/rollout_is_seq_max_deviation:0.008736789226531982 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.511741106864065) - rollout_corr/training_log_ppl:np.float64(0.410055655403994) - rollout_corr/kl:np.float64(0.0012775991272766518) - rollout_corr/k3_kl:np.float64(0.0011102865864813793) - rollout_corr/rollout_ppl:np.float64(1.5097454357892275) - rollout_corr/rollout_log_ppl:np.float64(0.4087780554546043) - rollout_corr/log_ppl_diff:np.float64(0.0012775999493896961) - rollout_corr/log_ppl_abs_diff:np.float64(0.002522424212656915) - rollout_corr/log_ppl_diff_max:np.float64(0.01004818081855774) - rollout_corr/log_ppl_diff_min:np.float64(-0.007290631532669067) - rollout_corr/ppl_ratio:np.float64(1.001282709185034) - rollout_corr/chi2_token:np.float64(0.0018994854763150215) - rollout_corr/chi2_seq:np.float64(0.5338654462248087) - actor/pg_loss:np.float64(0.008290401194244623) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015421995217366202) - actor/ppo_kl:np.float64(0.00013838649487496468) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.88671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.88671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(89599.56174836867) - self_distillation/token_reweight_w_std:np.float64(1222119.362034512) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005774809978902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18252522725379094) - self_distillation/empty_target_batch:np.float64(0.11328125) - actor/grad_norm:np.float64(0.45962898433208466) - perf/mfu/actor:np.float64(0.044088099096139206) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(209.18164443969727) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:5 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00388713157735765 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00388713157735765 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:248.21484375 - response_length/max:549.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:248.21484375 - response_length_non_aborted/max:549.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.785352110862732e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7626511324197054) - timing_s/agent_loop/generate_sequences/max:np.float64(4.8650829102844) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0402145619373186) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.8650829102844) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:549 - timing_s/gen:12.229782352223992 - timing_s/reward:0.05644015967845917 - timing_s/old_log_prob:2.86006579361856 - timing_s/adv:0.4905165284872055 - timing_s/update_actor:20.46021844446659 - timing_s/step:36.18464014865458 - timing_s/stop_profile:0.00011810846626758575 - timing_per_token_ms/update_actor:0.14417640947119384 - timing_per_token_ms/gen:0.19246466726821196 - timing_per_token_ms/adv:0.0034565081529071427 - perf/total_num_tokens:141911 - perf/time_per_step:36.18464014865458 - perf/throughput:490.23217937568927 (TaskRunner pid=2377588) Training Progress: 82%|████████▏ | 82/100 [1:18:44<15:12, 50.72s/it] (TaskRunner pid=2377588) step:83 - global_seqlen/min:15040 - global_seqlen/max:22149 - global_seqlen/minmax_diff:7109 - global_seqlen/balanced_min:17649 - global_seqlen/balanced_max:17653 - global_seqlen/mean:17651.875 - actor/entropy:0.40954309701919556 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12594819068908691 - training/rollout_probs_diff_mean:0.006285984069108963 - training/rollout_probs_diff_std:0.01330568641424179 - training/rollout_actor_probs_pearson_corr:0.9987616539001465 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9998940825462341 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6415700912475586 - rollout_corr/rollout_is_min:0.6093568205833435 - rollout_corr/rollout_is_std:0.04359670728445053 - rollout_corr/rollout_is_eff_sample_size:0.9981025765195183 - rollout_corr/rollout_is_seq_mean:0.9998738169670105 - rollout_corr/rollout_is_seq_std:0.002960145240649581 - rollout_corr/rollout_is_seq_max:1.0115435123443604 - rollout_corr/rollout_is_seq_min:0.9898817539215088 - rollout_corr/rollout_is_seq_max_deviation:0.011543512344360352 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.506910104304552) - rollout_corr/training_log_ppl:np.float64(0.406360135297291) - rollout_corr/kl:np.float64(0.0011971348145678462) - rollout_corr/k3_kl:np.float64(0.0011521209726197412) - rollout_corr/rollout_ppl:np.float64(1.5050894767045975) - rollout_corr/rollout_log_ppl:np.float64(0.40516299882438034) - rollout_corr/log_ppl_diff:np.float64(0.0011971364729106426) - rollout_corr/log_ppl_abs_diff:np.float64(0.002693758229725063) - rollout_corr/log_ppl_diff_max:np.float64(0.01060420274734497) - rollout_corr/log_ppl_diff_min:np.float64(-0.008981883525848389) - rollout_corr/ppl_ratio:np.float64(1.0012030606158078) - rollout_corr/chi2_token:np.float64(0.002231443300843239) - rollout_corr/chi2_seq:np.float64(0.8852758023422211) - actor/pg_loss:np.float64(0.008934200275689363) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014839655086689163) - actor/ppo_kl:np.float64(0.00011584551671717236) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(191018.74291138584) - self_distillation/token_reweight_w_std:np.float64(2791527.210146834) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009621342178434) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16119878937024623) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.48575257509946823) - perf/mfu/actor:np.float64(0.043700713892673854) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(209.21949005126953) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:5 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004728355852421373 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004728355852421373 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:247.83984375 - response_length/max:459.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:247.83984375 - response_length_non_aborted/max:459.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010555796325206757 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4683254640549421) - timing_s/agent_loop/generate_sequences/max:np.float64(4.470308784395456) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9836103544948855) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.470308784395456) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:459 - timing_s/gen:11.009428676217794 - timing_s/reward:0.056838855147361755 - timing_s/old_log_prob:2.881275899708271 - timing_s/adv:0.4762078393250704 - timing_s/update_actor:20.635815916582942 - timing_s/step:35.14652156829834 - timing_s/stop_profile:0.00011363066732883453 - timing_per_token_ms/update_actor:0.1461304812986081 - timing_per_token_ms/gen:0.17352165864765542 - timing_per_token_ms/adv:0.0033722185272461874 - perf/total_num_tokens:141215 - perf/time_per_step:35.14652156829834 - perf/throughput:502.2367566502438 (TaskRunner pid=2377588) Training Progress: 83%|████████▎ | 83/100 [1:19:19<13:03, 46.06s/it] (TaskRunner pid=2377588) step:84 - global_seqlen/min:16505 - global_seqlen/max:21553 - global_seqlen/minmax_diff:5048 - global_seqlen/balanced_min:18748 - global_seqlen/balanced_max:18751 - global_seqlen/mean:18750.625 - actor/entropy:0.39950791001319885 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23138868808746338 - training/rollout_probs_diff_mean:0.006192854605615139 - training/rollout_probs_diff_std:0.01324312575161457 - training/rollout_actor_probs_pearson_corr:0.9987532496452332 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:0.9998591542243958 - rollout_corr/rollout_is_ratio_fraction_high:1.534942930447869e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.1552069187164307 - rollout_corr/rollout_is_min:0.5596271753311157 - rollout_corr/rollout_is_std:0.04256037250161171 - rollout_corr/rollout_is_eff_sample_size:0.9981914149146929 - rollout_corr/rollout_is_seq_mean:0.9999157190322876 - rollout_corr/rollout_is_seq_std:0.002676321193575859 - rollout_corr/rollout_is_seq_max:1.0095338821411133 - rollout_corr/rollout_is_seq_min:0.9931747913360596 - rollout_corr/rollout_is_seq_max_deviation:0.009533882141113281 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4860362340696156) - rollout_corr/training_log_ppl:np.float64(0.39177870168350637) - rollout_corr/kl:np.float64(0.0010889215032499067) - rollout_corr/k3_kl:np.float64(0.001161380046596605) - rollout_corr/rollout_ppl:np.float64(1.4844627580605447) - rollout_corr/rollout_log_ppl:np.float64(0.39068978233262897) - rollout_corr/log_ppl_diff:np.float64(0.0010889193508774042) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026460569351911545) - rollout_corr/log_ppl_diff_max:np.float64(0.009106308221817017) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076333582401275635) - rollout_corr/ppl_ratio:np.float64(1.0010943377856165) - rollout_corr/chi2_token:np.float64(0.0026107076555490494) - rollout_corr/chi2_seq:np.float64(1.0454260597471148) - actor/pg_loss:np.float64(0.007366153644397855) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001352531637166976) - actor/ppo_kl:np.float64(9.04767738116874e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(132255.0453516245) - self_distillation/token_reweight_w_std:np.float64(1858316.723701529) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020043211989105) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16166845557745546) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.4519927576184273) - perf/mfu/actor:np.float64(0.0460866952011076) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(209.2583999633789) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:5 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00515165226534009 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00515165226534009 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:254.48828125 - response_length/max:497.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:254.48828125 - response_length_non_aborted/max:497.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:331.46875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015706755220890045 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8207670748233795) - timing_s/agent_loop/generate_sequences/max:np.float64(4.801904888823628) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1772542234757566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.801904888823628) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:497 - timing_s/gen:12.124206624925137 - timing_s/reward:0.05690300837159157 - timing_s/old_log_prob:2.8971258122473955 - timing_s/adv:0.4967538956552744 - timing_s/update_actor:20.677273163571954 - timing_s/step:36.33724758774042 - timing_s/stop_profile:0.00011592544615268707 - timing_per_token_ms/update_actor:0.13784389296071434 - timing_per_token_ms/gen:0.1860996580903028 - timing_per_token_ms/adv:0.003311582251626775 - perf/total_num_tokens:150005 - perf/time_per_step:36.33724758774042 - perf/throughput:516.0166563173086 (TaskRunner pid=2377588) Training Progress: 84%|████████▍ | 84/100 [1:19:56<11:30, 43.16s/it] (TaskRunner pid=2377588) step:85 - global_seqlen/min:14708 - global_seqlen/max:20790 - global_seqlen/minmax_diff:6082 - global_seqlen/balanced_min:17658 - global_seqlen/balanced_max:17661 - global_seqlen/mean:17660.25 - actor/entropy:0.4187212586402893 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4400407671928406 - training/rollout_probs_diff_mean:0.0063665155321359634 - training/rollout_probs_diff_std:0.013367985375225544 - training/rollout_actor_probs_pearson_corr:0.9987665414810181 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:1.0002628564834595 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5456427718163468e-05 - rollout_corr/rollout_is_max:1.6332260370254517 - rollout_corr/rollout_is_min:0.2928842604160309 - rollout_corr/rollout_is_std:0.04358850419521332 - rollout_corr/rollout_is_eff_sample_size:0.9981045953983432 - rollout_corr/rollout_is_seq_mean:1.0002788305282593 - rollout_corr/rollout_is_seq_std:0.002922722604125738 - rollout_corr/rollout_is_seq_max:1.0136947631835938 - rollout_corr/rollout_is_seq_min:0.993243396282196 - rollout_corr/rollout_is_seq_max_deviation:0.01369476318359375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5174573599360883) - rollout_corr/training_log_ppl:np.float64(0.4137719272985123) - rollout_corr/kl:np.float64(0.0009125653955486257) - rollout_corr/k3_kl:np.float64(0.0011271049722836324) - rollout_corr/rollout_ppl:np.float64(1.5161005188710988) - rollout_corr/rollout_log_ppl:np.float64(0.4128593599307351) - rollout_corr/log_ppl_diff:np.float64(0.0009125673677772284) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026199169224128127) - rollout_corr/log_ppl_diff_max:np.float64(0.008989512920379639) - rollout_corr/log_ppl_diff_min:np.float64(-0.01166573166847229) - rollout_corr/ppl_ratio:np.float64(1.0009181203786284) - rollout_corr/chi2_token:np.float64(0.0027237553149461746) - rollout_corr/chi2_seq:np.float64(1.7386555296834558) - actor/pg_loss:np.float64(0.004787841928191483) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008143002551150857) - actor/ppo_kl:np.float64(0.00022756134655610438) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(97371.92777730012) - self_distillation/token_reweight_w_std:np.float64(1470084.7890086106) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015548197552562) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09095226519275457) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.34718380123376846) - perf/mfu/actor:np.float64(0.043120701861785245) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(209.07668685913086) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:6 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006182571523822844 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006182571523822844 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:252.7265625 - response_length/max:455.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:252.7265625 - response_length_non_aborted/max:455.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.15625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0006507784128189087 - timing_s/agent_loop/generate_sequences/min:np.float64(1.658626077696681) - timing_s/agent_loop/generate_sequences/max:np.float64(4.421250740066171) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.201938153251831) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.421250740066171) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:455 - timing_s/gen:11.382569320499897 - timing_s/reward:0.051156461238861084 - timing_s/old_log_prob:3.0369218178093433 - timing_s/adv:0.5200641322880983 - timing_s/update_actor:21.079791894182563 - timing_s/step:36.14501167088747 - timing_s/stop_profile:0.00012112967669963837 - timing_per_token_ms/update_actor:0.1492036628458159 - timing_per_token_ms/gen:0.17593386689696586 - timing_per_token_ms/adv:0.00368103602927548 - perf/total_num_tokens:141282 - perf/time_per_step:36.14501167088747 - perf/throughput:488.59439196762577 (TaskRunner pid=2377588) Training Progress: 85%|████████▌ | 85/100 [1:20:32<10:18, 41.20s/it] (TaskRunner pid=2377588) step:86 - global_seqlen/min:15357 - global_seqlen/max:23273 - global_seqlen/minmax_diff:7916 - global_seqlen/balanced_min:18603 - global_seqlen/balanced_max:18608 - global_seqlen/mean:18605.625 - actor/entropy:0.40425196290016174 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29973217844963074 - training/rollout_probs_diff_mean:0.006176567170768976 - training/rollout_probs_diff_std:0.013260661624372005 - training/rollout_actor_probs_pearson_corr:0.9987645149230957 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0001630783081055 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4826896403974388e-05 - rollout_corr/rollout_is_max:1.6070524454116821 - rollout_corr/rollout_is_min:0.4801696240901947 - rollout_corr/rollout_is_std:0.04250992462038994 - rollout_corr/rollout_is_eff_sample_size:0.9981968787546434 - rollout_corr/rollout_is_seq_mean:1.000205159187317 - rollout_corr/rollout_is_seq_std:0.0028894173447042704 - rollout_corr/rollout_is_seq_max:1.0114176273345947 - rollout_corr/rollout_is_seq_min:0.9924694299697876 - rollout_corr/rollout_is_seq_max_deviation:0.011417627334594727 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5025393795222044) - rollout_corr/training_log_ppl:np.float64(0.4034408396692015) - rollout_corr/kl:np.float64(0.0011210763117617262) - rollout_corr/k3_kl:np.float64(0.0011998043844414497) - rollout_corr/rollout_ppl:np.float64(1.5009187310934067) - rollout_corr/rollout_log_ppl:np.float64(0.40231975860660896) - rollout_corr/log_ppl_diff:np.float64(0.001121081062592566) - rollout_corr/log_ppl_abs_diff:np.float64(0.002800571033731103) - rollout_corr/log_ppl_diff_max:np.float64(0.02209135890007019) - rollout_corr/log_ppl_diff_min:np.float64(-0.009959697723388672) - rollout_corr/ppl_ratio:np.float64(1.0011277555022389) - rollout_corr/chi2_token:np.float64(0.0024239143822342157) - rollout_corr/chi2_seq:np.float64(1.5417849123477936) - actor/pg_loss:np.float64(0.007156474865041673) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001297508112656942) - actor/ppo_kl:np.float64(0.00040998099919420383) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(41711.685749731725) - self_distillation/token_reweight_w_std:np.float64(631638.4512759785) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00046082935296) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15897649846738204) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.4512309655547142) - perf/mfu/actor:np.float64(0.04546269223013959) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.77055740356445) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:6 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018440951826050878 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0018440951826050878 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:263.45703125 - response_length/max:572.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:263.45703125 - response_length_non_aborted/max:572.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.96875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.509548544883728e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2952319346368313) - timing_s/agent_loop/generate_sequences/max:np.float64(5.248734317719936) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9454209651594283) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.248734317719936) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:572 - timing_s/gen:11.893905472010374 - timing_s/reward:0.0586589016020298 - timing_s/old_log_prob:3.1866820082068443 - timing_s/adv:0.5154107548296452 - timing_s/update_actor:21.005756571888924 - timing_s/step:36.750438107177615 - timing_s/stop_profile:6.218068301677704e-05 - timing_per_token_ms/update_actor:0.14112503995356865 - timing_per_token_ms/gen:0.17634969934035694 - timing_per_token_ms/adv:0.0034627347564892682 - perf/total_num_tokens:148845 - perf/time_per_step:36.750438107177615 - perf/throughput:506.26947482202104 (TaskRunner pid=2377588) Training Progress: 86%|████████▌ | 86/100 [1:21:09<09:18, 39.88s/it] (TaskRunner pid=2377588) step:87 - global_seqlen/min:16879 - global_seqlen/max:21457 - global_seqlen/minmax_diff:4578 - global_seqlen/balanced_min:18268 - global_seqlen/balanced_max:18272 - global_seqlen/mean:18271.0 - actor/entropy:0.4323495030403137 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12435343861579895 - training/rollout_probs_diff_mean:0.006164859514683485 - training/rollout_probs_diff_std:0.01268210168927908 - training/rollout_actor_probs_pearson_corr:0.9989056587219238 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9999585151672363 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4673083569505252e-05 - rollout_corr/rollout_is_max:1.738991379737854 - rollout_corr/rollout_is_min:0.3748246431350708 - rollout_corr/rollout_is_std:0.04182293638586998 - rollout_corr/rollout_is_eff_sample_size:0.9982537775121673 - rollout_corr/rollout_is_seq_mean:0.9999431371688843 - rollout_corr/rollout_is_seq_std:0.0026495656929910183 - rollout_corr/rollout_is_seq_max:1.0077472925186157 - rollout_corr/rollout_is_seq_min:0.993108332157135 - rollout_corr/rollout_is_seq_max_deviation:0.007747292518615723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.54904730245471) - rollout_corr/training_log_ppl:np.float64(0.43405604670988396) - rollout_corr/kl:np.float64(0.001005690916798585) - rollout_corr/k3_kl:np.float64(0.0011798760538113129) - rollout_corr/rollout_ppl:np.float64(1.547543877735734) - rollout_corr/rollout_log_ppl:np.float64(0.43305035441881046) - rollout_corr/log_ppl_diff:np.float64(0.0010056922910735011) - rollout_corr/log_ppl_abs_diff:np.float64(0.002353238989599049) - rollout_corr/log_ppl_diff_max:np.float64(0.008197516202926636) - rollout_corr/log_ppl_diff_min:np.float64(-0.006219804286956787) - rollout_corr/ppl_ratio:np.float64(1.0010099858045578) - rollout_corr/chi2_token:np.float64(0.0028422956820577383) - rollout_corr/chi2_seq:np.float64(0.5020803986117244) - actor/pg_loss:np.float64(0.007512948010116816) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013173465995350853) - actor/ppo_kl:np.float64(6.759537333778098e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(90274.61319815903) - self_distillation/token_reweight_w_std:np.float64(1297023.9905954716) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018559605814517) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16146713972557336) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.43995580822229385) - perf/mfu/actor:np.float64(0.04479223033881329) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.84112930297852) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:6 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005045706871896982 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005045706871896982 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:266.21875 - response_length/max:452.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.21875 - response_length_non_aborted/max:452.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.75 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.369609713554382e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.785462087020278) - timing_s/agent_loop/generate_sequences/max:np.float64(4.465826880186796) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2866897606363636) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.465826880186796) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:418 - timing_s/gen:11.313983261585236 - timing_s/reward:0.05831300839781761 - timing_s/old_log_prob:2.9593340177088976 - timing_s/adv:0.4926954545080662 - timing_s/update_actor:20.812357945367694 - timing_s/step:35.724904054775834 - timing_s/stop_profile:0.00010803714394569397 - timing_per_token_ms/update_actor:0.1423865548229961 - timing_per_token_ms/gen:0.1660110233241172 - timing_per_token_ms/adv:0.0033707477321169217 - perf/total_num_tokens:146168 - perf/time_per_step:35.724904054775834 - perf/throughput:511.43594317246226 (TaskRunner pid=2377588) Training Progress: 87%|████████▋ | 87/100 [1:21:45<08:22, 38.64s/it] (TaskRunner pid=2377588) step:88 - global_seqlen/min:15235 - global_seqlen/max:21402 - global_seqlen/minmax_diff:6167 - global_seqlen/balanced_min:18923 - global_seqlen/balanced_max:18930 - global_seqlen/mean:18927.625 - actor/entropy:0.4436388313770294 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22554942965507507 - training/rollout_probs_diff_mean:0.005867298226803541 - training/rollout_probs_diff_std:0.012208770029246807 - training/rollout_actor_probs_pearson_corr:0.9990015625953674 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.44140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.9999916553497314 - rollout_corr/rollout_is_ratio_fraction_high:1.4459852536674589e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4459852536674589e-05 - rollout_corr/rollout_is_max:2.0804450511932373 - rollout_corr/rollout_is_min:0.47980237007141113 - rollout_corr/rollout_is_std:0.040220268070697784 - rollout_corr/rollout_is_eff_sample_size:0.9983848237463163 - rollout_corr/rollout_is_seq_mean:0.9999247193336487 - rollout_corr/rollout_is_seq_std:0.0023860151413828135 - rollout_corr/rollout_is_seq_max:1.0071147680282593 - rollout_corr/rollout_is_seq_min:0.9892575144767761 - rollout_corr/rollout_is_seq_max_deviation:0.010742485523223877 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.570551808923483) - rollout_corr/training_log_ppl:np.float64(0.44664490775903687) - rollout_corr/kl:np.float64(0.0009267657680709362) - rollout_corr/k3_kl:np.float64(0.0010634470957029407) - rollout_corr/rollout_ppl:np.float64(1.569164866115898) - rollout_corr/rollout_log_ppl:np.float64(0.4457181433099322) - rollout_corr/log_ppl_diff:np.float64(0.0009267644491046667) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024069948121905327) - rollout_corr/log_ppl_diff_max:np.float64(0.014932721853256226) - rollout_corr/log_ppl_diff_min:np.float64(-0.006985485553741455) - rollout_corr/ppl_ratio:np.float64(1.0009314508643001) - rollout_corr/chi2_token:np.float64(0.002498921938240528) - rollout_corr/chi2_seq:np.float64(0.8382939335424453) - actor/pg_loss:np.float64(0.00679064029827714) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010806328709804802) - actor/ppo_kl:np.float64(2.8641112038485517e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44140625) - self_distillation/token_reweight_w_mean:np.float64(94666.42452141363) - self_distillation/token_reweight_w_std:np.float64(1455669.5412942192) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011420766822994) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1597920703352429) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.40984785556793213) - perf/mfu/actor:np.float64(0.04645287028376921) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.78445434570312) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:6 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0016990326112136245 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0016990326112136245 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:270.14453125 - response_length/max:656.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:270.14453125 - response_length_non_aborted/max:656.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:321.34375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015264376997947693 - timing_s/agent_loop/generate_sequences/min:np.float64(1.899068521335721) - timing_s/agent_loop/generate_sequences/max:np.float64(5.8207135386765) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.301025057255174) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.8207135386765) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:656 - timing_s/gen:12.621729547157884 - timing_s/reward:0.05856317840516567 - timing_s/old_log_prob:2.8722730986773968 - timing_s/adv:0.5098744314163923 - timing_s/update_actor:21.149825371801853 - timing_s/step:37.30133054032922 - timing_s/stop_profile:0.00013031810522079468 - timing_per_token_ms/update_actor:0.13967564189776752 - timing_per_token_ms/gen:0.18250834401662716 - timing_per_token_ms/adv:0.003367263664989614 - perf/total_num_tokens:151421 - perf/time_per_step:37.30133054032922 - perf/throughput:507.42492897233114 (TaskRunner pid=2377588) Training Progress: 88%|████████▊ | 88/100 [1:22:22<07:39, 38.26s/it] (TaskRunner pid=2377588) step:89 - global_seqlen/min:17209 - global_seqlen/max:24390 - global_seqlen/minmax_diff:7181 - global_seqlen/balanced_min:19635 - global_seqlen/balanced_max:19641 - global_seqlen/mean:19638.875 - actor/entropy:0.4684907793998718 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45078346133232117 - training/rollout_probs_diff_mean:0.005763007793575525 - training/rollout_probs_diff_std:0.011655439622700214 - training/rollout_actor_probs_pearson_corr:0.99910968542099 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.6328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9998112320899963 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4798371921642683e-05 - rollout_corr/rollout_is_max:1.8588063716888428 - rollout_corr/rollout_is_min:0.36560797691345215 - rollout_corr/rollout_is_std:0.03893028944730759 - rollout_corr/rollout_is_eff_sample_size:0.998486310195118 - rollout_corr/rollout_is_seq_mean:0.9998524188995361 - rollout_corr/rollout_is_seq_std:0.0026035348419100046 - rollout_corr/rollout_is_seq_max:1.0062050819396973 - rollout_corr/rollout_is_seq_min:0.9912956953048706 - rollout_corr/rollout_is_seq_max_deviation:0.008704304695129395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6001862306147814) - rollout_corr/training_log_ppl:np.float64(0.46479875105433166) - rollout_corr/kl:np.float64(0.0007109100087667741) - rollout_corr/k3_kl:np.float64(0.0009364084531853223) - rollout_corr/rollout_ppl:np.float64(1.5989437764510512) - rollout_corr/rollout_log_ppl:np.float64(0.46408783580409363) - rollout_corr/log_ppl_diff:np.float64(0.0007109152502380311) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024779607192613184) - rollout_corr/log_ppl_diff_max:np.float64(0.009819626808166504) - rollout_corr/log_ppl_diff_min:np.float64(-0.010041922330856323) - rollout_corr/ppl_ratio:np.float64(1.0007158666849136) - rollout_corr/chi2_token:np.float64(0.0023472344037145376) - rollout_corr/chi2_seq:np.float64(0.9535690350458026) - actor/pg_loss:np.float64(0.00891009229235351) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010016704081863281) - actor/ppo_kl:np.float64(-0.00020780769594352932) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_w_mean:np.float64(165896.30784601858) - self_distillation/token_reweight_w_std:np.float64(2476184.0363599276) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994898571167141) - self_distillation/token_reweight_w_clip_frac:np.float64(0.18939776910701767) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.5010746195912361) - perf/mfu/actor:np.float64(0.04829463977924141) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.78932571411133) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:6 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021254161838442087 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021254161838442087 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:263.96484375 - response_length/max:628.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:263.96484375 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.48198139667511e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8352245278656483) - timing_s/agent_loop/generate_sequences/max:np.float64(5.515040375292301) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2829254266471253) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.515040375292301) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:628 - timing_s/gen:12.670431287959218 - timing_s/reward:0.059604087844491005 - timing_s/old_log_prob:3.0231986604630947 - timing_s/adv:0.5249879453331232 - timing_s/update_actor:21.095048863440752 - timing_s/step:37.46025801450014 - timing_s/stop_profile:3.3913180232048035e-05 - timing_per_token_ms/update_actor:0.13426843991471477 - timing_per_token_ms/gen:0.1875017578684309 - timing_per_token_ms/adv:0.003341509794560045 - perf/total_num_tokens:157111 - perf/time_per_step:37.46025801450014 - perf/throughput:524.2589357606179 (TaskRunner pid=2377588) Training Progress: 89%|████████▉ | 89/100 [1:23:00<06:58, 38.02s/it] (TaskRunner pid=2377588) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2377588) validation generation end (TaskRunner pid=2377588) [prompt] system (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2377588) (TaskRunner pid=2377588) A (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) user (TaskRunner pid=2377588) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2377588) (TaskRunner pid=2377588) A: -0.01 (TaskRunner pid=2377588) B: 1.69 (TaskRunner pid=2377588) C: 2.49 (TaskRunner pid=2377588) D: 0.45 (TaskRunner pid=2377588) Please reason step by step. (TaskRunner pid=2377588) assistant (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) [response] (TaskRunner pid=2377588) To determine the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR', we need to assess the balance of hydrophobic and hydrophilic residues, as well as the overall secondary structure potential. (TaskRunner pid=2377588) (TaskRunner pid=2377588) The sequence contains a mix of hydrophilic (e.g., lysine, arginine, glutamic acid, aspartic acid) and hydrophobic (e.g., glycine, serine, tryptophan, tyrosine) residues. It also includes several charged residues, which can contribute to stabilizing the protein through electrostatic interactions. (TaskRunner pid=2377588) (TaskRunner pid=2377588) While the sequence has a moderate number of charged residues, it also contains enough hydrophobic residues to potentially form a stable core. The presence of repeated charged residues (like 'EEE' and 'ARR') may suggest a region of high local stability, but the overall balance of hydrophobic and hydrophilic residues suggests a moderate to slightly positive folding stability score. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Among the given options, 0.45 (Option D) represents a moderate positive score, indicating reasonable folding stability without being overly favorable. This makes it the most reasonable choice based on the sequence's composition. (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) D (TaskRunner pid=2377588) (TaskRunner pid=2377588) [ground_truth] B (TaskRunner pid=2377588) [score] 0.0 (TaskRunner pid=2377588) [acc] 0.0 (TaskRunner pid=2377588) [pred] D (TaskRunner pid=2377588) [incorrect_format] 1 (TaskRunner pid=2377588) [feedback] (TaskRunner pid=2377588) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2377588) step:90 - global_seqlen/min:16814 - global_seqlen/max:20943 - global_seqlen/minmax_diff:4129 - global_seqlen/balanced_min:18478 - global_seqlen/balanced_max:18644 - global_seqlen/mean:18500.5 - actor/entropy:0.4683268964290619 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.320637583732605 - training/rollout_probs_diff_mean:0.005687726195901632 - training/rollout_probs_diff_std:0.011656074784696102 - training/rollout_actor_probs_pearson_corr:0.9991191029548645 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.000186562538147 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5799123048782349 - rollout_corr/rollout_is_min:0.5087235569953918 - rollout_corr/rollout_is_std:0.03929678350687027 - rollout_corr/rollout_is_eff_sample_size:0.9984587380648611 - rollout_corr/rollout_is_seq_mean:1.000199317932129 - rollout_corr/rollout_is_seq_std:0.0024723911192268133 - rollout_corr/rollout_is_seq_max:1.007213830947876 - rollout_corr/rollout_is_seq_min:0.9930728673934937 - rollout_corr/rollout_is_seq_max_deviation:0.0072138309478759766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.6108632390387356) - rollout_corr/training_log_ppl:np.float64(0.47256311553064734) - rollout_corr/kl:np.float64(0.000714641950771977) - rollout_corr/k3_kl:np.float64(0.0009688343454854476) - rollout_corr/rollout_ppl:np.float64(1.6096989433281124) - rollout_corr/rollout_log_ppl:np.float64(0.47184847004245967) - rollout_corr/log_ppl_diff:np.float64(0.0007146454881876707) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021513516549021006) - rollout_corr/log_ppl_diff_max:np.float64(0.01130560040473938) - rollout_corr/log_ppl_diff_min:np.float64(-0.008196890354156494) - rollout_corr/ppl_ratio:np.float64(1.0007186259608716) - rollout_corr/chi2_token:np.float64(0.0024517192505300045) - rollout_corr/chi2_seq:np.float64(0.719752499833703) - actor/pg_loss:np.float64(0.007385115721262991) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000877700943419768) - actor/ppo_kl:np.float64(0.00013229936251946128) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(66729.47507510171) - self_distillation/token_reweight_w_std:np.float64(1026056.4565310498) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9977818739134818) - self_distillation/token_reweight_w_clip_frac:np.float64(0.20293948473408818) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.4713021218776703) - perf/mfu/actor:np.float64(0.04471748717061138) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.81289672851562) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.50625) - val-aux/sciknoweval/reward/std@16:np.float64(0.22577964247520138) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5990000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.17089905726363366) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.41684) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21220172007634652) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5065200000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.22422321161119194) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6645599999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1074062828641595) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.32814) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1749686074359798) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.52238) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1809414932030317) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.70292) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06118172621270312) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.25682) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.1321700716285853) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.53044) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.12852891587060702) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7241) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.029920735002457146) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.20144) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.08319799392301455) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.52906) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09205978107172769) - val-aux/sciknoweval/score/mean@16:np.float64(0.50625) - val-aux/sciknoweval/score/std@16:np.float64(0.22577964247520138) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5990000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.17089905726363366) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.41684) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21220172007634652) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5065200000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.22422321161119194) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6645599999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1074062828641595) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.32814) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1749686074359798) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.52238) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1809414932030317) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.70292) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06118172621270312) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.25682) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.1321700716285853) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.53044) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.12852891587060702) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7241) - val-aux/sciknoweval/score/best@16/std:np.float64(0.029920735002457146) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.20144) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.08319799392301455) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.52906) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09205978107172769) - val-core/sciknoweval/acc/mean@16:np.float64(0.50625) - val-aux/sciknoweval/acc/std@16:np.float64(0.22577964247520138) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5990000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.17089905726363366) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.41684) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21220172007634652) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5065200000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.22422321161119194) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6645599999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1074062828641595) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.32814) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1749686074359798) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.52238) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1809414932030317) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.70292) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06118172621270312) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.25682) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.1321700716285853) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.53044) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.12852891587060702) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7241) - val-core/sciknoweval/acc/best@16/std:np.float64(0.029920735002457146) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.20144) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.08319799392301455) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.52906) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09205978107172769) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:6 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026925248093903065 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026925248093903065 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:262.953125 - response_length/max:842.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.953125 - response_length_non_aborted/max:842.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.1875 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6298686861991882e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6294401232153177) - timing_s/agent_loop/generate_sequences/max:np.float64(6.709865568205714) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.200628135120496) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.709865568205714) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:842 - timing_s/gen:13.594293916597962 - timing_s/reward:0.05815776064991951 - timing_s/old_log_prob:2.9110132567584515 - timing_s/adv:0.5323890559375286 - timing_s/update_actor:21.538211164996028 - timing_s/step:38.71996928565204 - timing_s/testing:22.932722564786673 - timing_s/save_checkpoint:13.284607119858265 - timing_s/stop_profile:0.00013570301234722137 - timing_per_token_ms/update_actor:0.14552452072238606 - timing_per_token_ms/gen:0.2019474406767776 - timing_per_token_ms/adv:0.003597126131304077 - perf/total_num_tokens:148004 - perf/time_per_step:38.71996928565204 - perf/throughput:477.80254843475535 (TaskRunner pid=2377588) Training Progress: 90%|█████████ | 90/100 [1:24:15<08:11, 49.11s/it] (TaskRunner pid=2377588) step:91 - global_seqlen/min:14829 - global_seqlen/max:22085 - global_seqlen/minmax_diff:7256 - global_seqlen/balanced_min:18719 - global_seqlen/balanced_max:18724 - global_seqlen/mean:18721.625 - actor/entropy:0.4527682065963745 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1532047688961029 - training/rollout_probs_diff_mean:0.005749888252466917 - training/rollout_probs_diff_std:0.01179367583245039 - training/rollout_actor_probs_pearson_corr:0.999067485332489 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001296997070312 - rollout_corr/rollout_is_ratio_fraction_high:1.5347544831456617e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.8861513137817383 - rollout_corr/rollout_is_min:0.5353028774261475 - rollout_corr/rollout_is_std:0.0394270084798336 - rollout_corr/rollout_is_eff_sample_size:0.9984483989085946 - rollout_corr/rollout_is_seq_mean:1.0000967979431152 - rollout_corr/rollout_is_seq_std:0.0025646896101534367 - rollout_corr/rollout_is_seq_max:1.011281132698059 - rollout_corr/rollout_is_seq_min:0.9922240972518921 - rollout_corr/rollout_is_seq_max_deviation:0.011281132698059082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5778993875719607) - rollout_corr/training_log_ppl:np.float64(0.4507169911521487) - rollout_corr/kl:np.float64(0.001065915030999065) - rollout_corr/k3_kl:np.float64(0.0010728293018473778) - rollout_corr/rollout_ppl:np.float64(1.576178366318345) - rollout_corr/rollout_log_ppl:np.float64(0.44965107180178165) - rollout_corr/log_ppl_diff:np.float64(0.0010659193503670394) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023591112694703043) - rollout_corr/log_ppl_diff_max:np.float64(0.010366499423980713) - rollout_corr/log_ppl_diff_min:np.float64(-0.006875216960906982) - rollout_corr/ppl_ratio:np.float64(1.0010704810265452) - rollout_corr/chi2_token:np.float64(0.002170758554711938) - rollout_corr/chi2_seq:np.float64(0.4985712869092822) - actor/pg_loss:np.float64(0.007485660724341869) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014421896648855181) - actor/ppo_kl:np.float64(0.0003714298503076918) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(180692.6870438566) - self_distillation/token_reweight_w_std:np.float64(2815404.9901619083) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010530590079725) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1801842920249328) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.4528066962957382) - perf/mfu/actor:np.float64(0.04583373585461569) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(207.2258071899414) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:6 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006607118062674999 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006607118062674999 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:254.51953125 - response_length/max:552.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:254.51953125 - response_length_non_aborted/max:552.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.244067430496216e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5741483327001333) - timing_s/agent_loop/generate_sequences/max:np.float64(4.969326000660658) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9455529939295957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.969326000660658) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:187 - timing_s/agent_loop/slowest/response_length:552 - timing_s/gen:11.461571337655187 - timing_s/reward:0.055148838087916374 - timing_s/old_log_prob:2.9471467938274145 - timing_s/adv:0.5155574083328247 - timing_s/update_actor:21.09470148012042 - timing_s/step:36.17619802802801 - timing_s/stop_profile:0.00011040456593036652 - timing_per_token_ms/update_actor:0.14084448785909623 - timing_per_token_ms/gen:0.17590698371096256 - timing_per_token_ms/adv:0.003442258673678331 - perf/total_num_tokens:149773 - perf/time_per_step:36.17619802802801 - perf/throughput:517.5122323660204 (TaskRunner pid=2377588) Training Progress: 91%|█████████ | 91/100 [1:24:51<06:47, 45.25s/it] (TaskRunner pid=2377588) step:92 - global_seqlen/min:16456 - global_seqlen/max:22601 - global_seqlen/minmax_diff:6145 - global_seqlen/balanced_min:19261 - global_seqlen/balanced_max:19269 - global_seqlen/mean:19266.25 - actor/entropy:0.4293704032897949 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2622944116592407 - training/rollout_probs_diff_mean:0.005648909602314234 - training/rollout_probs_diff_std:0.011866849847137928 - training/rollout_actor_probs_pearson_corr:0.9990501403808594 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.6328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:1.0001208782196045 - rollout_corr/rollout_is_ratio_fraction_high:1.638431058381684e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.115023374557495 - rollout_corr/rollout_is_min:0.6458032727241516 - rollout_corr/rollout_is_std:0.039033498615026474 - rollout_corr/rollout_is_eff_sample_size:0.9984791793257898 - rollout_corr/rollout_is_seq_mean:1.0000373125076294 - rollout_corr/rollout_is_seq_std:0.0024098134599626064 - rollout_corr/rollout_is_seq_max:1.0092675685882568 - rollout_corr/rollout_is_seq_min:0.9915754199028015 - rollout_corr/rollout_is_seq_max_deviation:0.009267568588256836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5421580448746681) - rollout_corr/training_log_ppl:np.float64(0.42730043368646875) - rollout_corr/kl:np.float64(0.0011324355015993604) - rollout_corr/k3_kl:np.float64(0.0011073531579768314) - rollout_corr/rollout_ppl:np.float64(1.5403385567478836) - rollout_corr/rollout_log_ppl:np.float64(0.4261679949122481) - rollout_corr/log_ppl_diff:np.float64(0.0011324387742206454) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025128433480858803) - rollout_corr/log_ppl_diff_max:np.float64(0.011763691902160645) - rollout_corr/log_ppl_diff_min:np.float64(-0.006782412528991699) - rollout_corr/ppl_ratio:np.float64(1.001137555576861) - rollout_corr/chi2_token:np.float64(0.0021336679346859455) - rollout_corr/chi2_seq:np.float64(0.4580773157067597) - actor/pg_loss:np.float64(0.008646672824397683) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015386051263703848) - actor/ppo_kl:np.float64(0.0003990091671894902) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_w_mean:np.float64(80366.37934077159) - self_distillation/token_reweight_w_std:np.float64(1160360.0781654853) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003282432444394) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16889019947848283) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.5020161867141724) - perf/mfu/actor:np.float64(0.04748150934315741) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.22428512573242) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:6 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002646066015586257 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002646066015586257 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:238.4140625 - response_length/max:542.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:238.4140625 - response_length_non_aborted/max:542.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:363.65625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010357052087783813 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5818912852555513) - timing_s/agent_loop/generate_sequences/max:np.float64(4.9085306860506535) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8859313524881145) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.9085306860506535) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:542 - timing_s/gen:11.682764021679759 - timing_s/reward:0.06351600773632526 - timing_s/old_log_prob:2.910671105608344 - timing_s/adv:0.49789087660610676 - timing_s/update_actor:20.792680598795414 - timing_s/step:36.04587009549141 - timing_s/stop_profile:0.00011053495109081268 - timing_per_token_ms/update_actor:0.13490352688506724 - timing_per_token_ms/gen:0.19141403187862108 - timing_per_token_ms/adv:0.003230330737728585 - perf/total_num_tokens:154130 - perf/time_per_step:36.04587009549141 - perf/throughput:534.4925770680677 (TaskRunner pid=2377588) Training Progress: 92%|█████████▏| 92/100 [1:25:27<05:40, 42.50s/it] (TaskRunner pid=2377588) step:93 - global_seqlen/min:15457 - global_seqlen/max:20834 - global_seqlen/minmax_diff:5377 - global_seqlen/balanced_min:18415 - global_seqlen/balanced_max:18455 - global_seqlen/mean:18423.0 - actor/entropy:0.41477349400520325 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42425447702407837 - training/rollout_probs_diff_mean:0.005878234747797251 - training/rollout_probs_diff_std:0.012953962199389935 - training/rollout_actor_probs_pearson_corr:0.9988547563552856 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0002052783966064 - rollout_corr/rollout_is_ratio_fraction_high:1.703693669696804e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.407387339393608e-05 - rollout_corr/rollout_is_max:4.696316242218018 - rollout_corr/rollout_is_min:0.29898959398269653 - rollout_corr/rollout_is_std:0.040980733931064606 - rollout_corr/rollout_is_eff_sample_size:0.9983239893029506 - rollout_corr/rollout_is_seq_mean:1.0001827478408813 - rollout_corr/rollout_is_seq_std:0.0029300351161509752 - rollout_corr/rollout_is_seq_max:1.0157015323638916 - rollout_corr/rollout_is_seq_min:0.9913196563720703 - rollout_corr/rollout_is_seq_max_deviation:0.0157015323638916 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5297262356616557) - rollout_corr/training_log_ppl:np.float64(0.4191977974260226) - rollout_corr/kl:np.float64(0.0008948687736705097) - rollout_corr/k3_kl:np.float64(0.0011211829413468877) - rollout_corr/rollout_ppl:np.float64(1.5282964161597192) - rollout_corr/rollout_log_ppl:np.float64(0.418302928970661) - rollout_corr/log_ppl_diff:np.float64(0.0008948684553615749) - rollout_corr/log_ppl_abs_diff:np.float64(0.002572127676103264) - rollout_corr/log_ppl_diff_max:np.float64(0.010169625282287598) - rollout_corr/log_ppl_diff_min:np.float64(-0.009163498878479004) - rollout_corr/ppl_ratio:np.float64(1.0009004019666463) - rollout_corr/chi2_token:np.float64(0.002695041010156274) - rollout_corr/chi2_seq:np.float64(0.7000831146724522) - actor/pg_loss:np.float64(0.006859057582914829) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015379139122160268) - actor/ppo_kl:np.float64(0.00018736743199188055) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(133969.8173064671) - self_distillation/token_reweight_w_std:np.float64(1823779.3558803052) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000981303397566) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14890081819612533) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.46300897747278214) - perf/mfu/actor:np.float64(0.04534523826707963) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.97018432617188) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:6 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006359036546200514 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006359036546200514 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:229.28125 - response_length/max:670.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:229.28125 - response_length_non_aborted/max:670.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001506321132183075 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5258204378187656) - timing_s/agent_loop/generate_sequences/max:np.float64(5.300253491848707) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.830272475279344) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.300253491848707) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:670 - timing_s/gen:11.97874958254397 - timing_s/reward:0.057910580188035965 - timing_s/old_log_prob:2.9104053396731615 - timing_s/adv:0.4859201181679964 - timing_s/update_actor:20.671803083270788 - timing_s/step:36.19257167354226 - timing_s/stop_profile:0.00010884925723075867 - timing_per_token_ms/update_actor:0.14025812220641853 - timing_per_token_ms/gen:0.20408119092517324 - timing_per_token_ms/adv:0.003296966551104573 - perf/total_num_tokens:147384 - perf/time_per_step:36.19257167354226 - perf/throughput:509.02710551148004 (TaskRunner pid=2377588) Training Progress: 93%|█████████▎| 93/100 [1:26:03<04:44, 40.62s/it] (TaskRunner pid=2377588) step:94 - global_seqlen/min:14425 - global_seqlen/max:19878 - global_seqlen/minmax_diff:5453 - global_seqlen/balanced_min:16429 - global_seqlen/balanced_max:16434 - global_seqlen/mean:16432.5 - actor/entropy:0.43463438749313354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20012031495571136 - training/rollout_probs_diff_mean:0.006288039963692427 - training/rollout_probs_diff_std:0.013051224872469902 - training/rollout_actor_probs_pearson_corr:0.9988447427749634 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9999005794525146 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.7049716916517355e-05 - rollout_corr/rollout_is_max:1.5375944375991821 - rollout_corr/rollout_is_min:0.4818863570690155 - rollout_corr/rollout_is_std:0.04255897179245949 - rollout_corr/rollout_is_eff_sample_size:0.9981915336931861 - rollout_corr/rollout_is_seq_mean:0.9999184012413025 - rollout_corr/rollout_is_seq_std:0.0029622744768857956 - rollout_corr/rollout_is_seq_max:1.0091873407363892 - rollout_corr/rollout_is_seq_min:0.990716814994812 - rollout_corr/rollout_is_seq_max_deviation:0.009283185005187988 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5455019446089864) - rollout_corr/training_log_ppl:np.float64(0.4302993090241216) - rollout_corr/kl:np.float64(0.0007909124758782582) - rollout_corr/k3_kl:np.float64(0.0011846215156765538) - rollout_corr/rollout_ppl:np.float64(1.544189362321049) - rollout_corr/rollout_log_ppl:np.float64(0.42950839595869184) - rollout_corr/log_ppl_diff:np.float64(0.0007909130654297769) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024577174917794764) - rollout_corr/log_ppl_diff_max:np.float64(0.010443806648254395) - rollout_corr/log_ppl_diff_min:np.float64(-0.007998019456863403) - rollout_corr/ppl_ratio:np.float64(1.000795932020992) - rollout_corr/chi2_token:np.float64(0.0030952903907746077) - rollout_corr/chi2_seq:np.float64(0.6447298661805689) - actor/pg_loss:np.float64(0.004465584876015782) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000802594579909055) - actor/ppo_kl:np.float64(-0.00022167980297638223) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(37231.120757297846) - self_distillation/token_reweight_w_std:np.float64(427712.56028361246) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9983186365570873) - self_distillation/token_reweight_w_clip_frac:np.float64(0.123998536815634) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.40267710387706757) - perf/mfu/actor:np.float64(0.04063135388763145) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.99001693725586) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:6 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00413029408082366 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00413029408082366 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:229.109375 - response_length/max:533.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:229.109375 - response_length_non_aborted/max:533.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:284.40625 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001487191766500473 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6242477353662252) - timing_s/agent_loop/generate_sequences/max:np.float64(4.684996096417308) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.780280881204817) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.684996096417308) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:533 - timing_s/gen:11.302994087338448 - timing_s/reward:0.057898255065083504 - timing_s/old_log_prob:2.8574047423899174 - timing_s/adv:0.4710292723029852 - timing_s/update_actor:20.76949243992567 - timing_s/step:35.549358669668436 - timing_s/stop_profile:0.00011142902076244354 - timing_per_token_ms/update_actor:0.15799096637703994 - timing_per_token_ms/gen:0.19271285015580794 - timing_per_token_ms/adv:0.0035830615571503513 - perf/total_num_tokens:131460 - perf/time_per_step:35.549358669668436 - perf/throughput:462.24462592121534 (TaskRunner pid=2377588) Training Progress: 94%|█████████▍| 94/100 [1:26:39<03:54, 39.12s/it] (TaskRunner pid=2377588) step:95 - global_seqlen/min:14505 - global_seqlen/max:19447 - global_seqlen/minmax_diff:4942 - global_seqlen/balanced_min:16885 - global_seqlen/balanced_max:16943 - global_seqlen/mean:16893.375 - actor/entropy:0.40254759788513184 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1748221218585968 - training/rollout_probs_diff_mean:0.0062261782586574554 - training/rollout_probs_diff_std:0.013322706334292889 - training/rollout_actor_probs_pearson_corr:0.9987474679946899 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.000057578086853 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5758583545684814 - rollout_corr/rollout_is_min:0.5620222687721252 - rollout_corr/rollout_is_std:0.0426834337413311 - rollout_corr/rollout_is_eff_sample_size:0.9981816751744341 - rollout_corr/rollout_is_seq_mean:1.0000698566436768 - rollout_corr/rollout_is_seq_std:0.0029274565167725086 - rollout_corr/rollout_is_seq_max:1.0096288919448853 - rollout_corr/rollout_is_seq_min:0.9894712567329407 - rollout_corr/rollout_is_seq_max_deviation:0.010528743267059326 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4907503658905625) - rollout_corr/training_log_ppl:np.float64(0.39545443921815604) - rollout_corr/kl:np.float64(0.001116184725507452) - rollout_corr/k3_kl:np.float64(0.0011958868253714172) - rollout_corr/rollout_ppl:np.float64(1.4890606999397278) - rollout_corr/rollout_log_ppl:np.float64(0.39433825574815273) - rollout_corr/log_ppl_diff:np.float64(0.0011161834700033069) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026863421080634) - rollout_corr/log_ppl_diff_max:np.float64(0.0126495361328125) - rollout_corr/log_ppl_diff_min:np.float64(-0.008571147918701172) - rollout_corr/ppl_ratio:np.float64(1.0011223210021853) - rollout_corr/chi2_token:np.float64(0.0025384388864040375) - rollout_corr/chi2_seq:np.float64(1.3367056755814701) - actor/pg_loss:np.float64(0.00664030690677464) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013014837059017736) - actor/ppo_kl:np.float64(0.00026066816148428984) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(28485.3158807524) - self_distillation/token_reweight_w_std:np.float64(402547.63218703016) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998683985322714) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12180487395380624) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.39125820249319077) - perf/mfu/actor:np.float64(0.042062913386408995) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.92643356323242) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:6 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0016088924603536725 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0016088924603536725 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:236.41796875 - response_length/max:546.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:236.41796875 - response_length_non_aborted/max:546.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.5 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.819811046123505e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.431061465293169) - timing_s/agent_loop/generate_sequences/max:np.float64(5.089055087417364) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9110498828231357) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.089055087417364) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:546 - timing_s/gen:12.41839449107647 - timing_s/reward:0.05636488273739815 - timing_s/old_log_prob:2.83126556687057 - timing_s/adv:0.49369583651423454 - timing_s/update_actor:20.665844686329365 - timing_s/step:36.55556072294712 - timing_s/stop_profile:0.00011430494487285614 - timing_per_token_ms/update_actor:0.1529138248450159 - timing_per_token_ms/gen:0.20518471475433256 - timing_per_token_ms/adv:0.0036530284543070473 - perf/total_num_tokens:135147 - perf/time_per_step:36.55556072294712 - perf/throughput:462.12873406686595 (TaskRunner pid=2377588) Training Progress: 95%|█████████▌| 95/100 [1:27:15<03:11, 38.36s/it] (TaskRunner pid=2377588) step:96 - global_seqlen/min:13914 - global_seqlen/max:21203 - global_seqlen/minmax_diff:7289 - global_seqlen/balanced_min:17510 - global_seqlen/balanced_max:17516 - global_seqlen/mean:17511.875 - actor/entropy:0.3947932720184326 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3904871344566345 - training/rollout_probs_diff_mean:0.006139610894024372 - training/rollout_probs_diff_std:0.013206885196268559 - training/rollout_actor_probs_pearson_corr:0.9987712502479553 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.000056505203247 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5891429939074442e-05 - rollout_corr/rollout_is_max:1.774428129196167 - rollout_corr/rollout_is_min:0.4781637489795685 - rollout_corr/rollout_is_std:0.04286736622452736 - rollout_corr/rollout_is_eff_sample_size:0.998165996967404 - rollout_corr/rollout_is_seq_mean:1.0000444650650024 - rollout_corr/rollout_is_seq_std:0.0029045171104371548 - rollout_corr/rollout_is_seq_max:1.0101375579833984 - rollout_corr/rollout_is_seq_min:0.9906045198440552 - rollout_corr/rollout_is_seq_max_deviation:0.010137557983398438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4840585528872907) - rollout_corr/training_log_ppl:np.float64(0.39096204267116264) - rollout_corr/kl:np.float64(0.001380846692413229) - rollout_corr/k3_kl:np.float64(0.0011636833953616588) - rollout_corr/rollout_ppl:np.float64(1.482026272919029) - rollout_corr/rollout_log_ppl:np.float64(0.38958119886228815) - rollout_corr/log_ppl_diff:np.float64(0.0013808438088744879) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029026162810623646) - rollout_corr/log_ppl_diff_max:np.float64(0.011550426483154297) - rollout_corr/log_ppl_diff_min:np.float64(-0.009650111198425293) - rollout_corr/ppl_ratio:np.float64(1.0013875453732908) - rollout_corr/chi2_token:np.float64(0.0019992664456367493) - rollout_corr/chi2_seq:np.float64(0.7233279764186591) - actor/pg_loss:np.float64(0.009002521983347833) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009446693484278512) - actor/ppo_kl:np.float64(0.000500841857686396) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(104403.70773658413) - self_distillation/token_reweight_w_std:np.float64(1446357.480080368) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997071377001703) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16106312299962156) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.4429244324564934) - perf/mfu/actor:np.float64(0.041864112416560656) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.2278289794922) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:6 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002600235166028142 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002600235166028142 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:245.80859375 - response_length/max:568.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:245.80859375 - response_length_non_aborted/max:568.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:301.4375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014579668641090393 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3747573643922806) - timing_s/agent_loop/generate_sequences/max:np.float64(5.129914239048958) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0549576484409044) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.129914239048958) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:568 - timing_s/gen:11.912157226353884 - timing_s/reward:0.05577950365841389 - timing_s/old_log_prob:2.8709400221705437 - timing_s/adv:0.5066861715167761 - timing_s/update_actor:21.39893756620586 - timing_s/step:36.83123058080673 - timing_s/stop_profile:3.4576281905174255e-05 - timing_per_token_ms/update_actor:0.15274590503733795 - timing_per_token_ms/gen:0.18930120975660503 - timing_per_token_ms/adv:0.0036167327279116035 - perf/total_num_tokens:140095 - perf/time_per_step:36.83123058080673 - perf/throughput:475.4626637190255 (TaskRunner pid=2377588) Training Progress: 96%|█████████▌| 96/100 [1:27:52<02:31, 37.91s/it] (TaskRunner pid=2377588) step:97 - global_seqlen/min:14202 - global_seqlen/max:20065 - global_seqlen/minmax_diff:5863 - global_seqlen/balanced_min:17452 - global_seqlen/balanced_max:17455 - global_seqlen/mean:17454.0 - actor/entropy:0.4278377294540405 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12245899438858032 - training/rollout_probs_diff_mean:0.006036087870597839 - training/rollout_probs_diff_std:0.012526719830930233 - training/rollout_actor_probs_pearson_corr:0.998920738697052 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.0001471042633057 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5871000869083218e-05 - rollout_corr/rollout_is_max:1.5717827081680298 - rollout_corr/rollout_is_min:0.4538624584674835 - rollout_corr/rollout_is_std:0.04135716333985329 - rollout_corr/rollout_is_eff_sample_size:0.9982930996503491 - rollout_corr/rollout_is_seq_mean:1.0001170635223389 - rollout_corr/rollout_is_seq_std:0.002829844830557704 - rollout_corr/rollout_is_seq_max:1.0071420669555664 - rollout_corr/rollout_is_seq_min:0.9868662357330322 - rollout_corr/rollout_is_seq_max_deviation:0.013133764266967773 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5366868297569454) - rollout_corr/training_log_ppl:np.float64(0.4249948668293655) - rollout_corr/kl:np.float64(0.0011487816395430173) - rollout_corr/k3_kl:np.float64(0.001014784923654588) - rollout_corr/rollout_ppl:np.float64(1.5349369887262583) - rollout_corr/rollout_log_ppl:np.float64(0.42384608497377485) - rollout_corr/log_ppl_diff:np.float64(0.0011487818555906415) - rollout_corr/log_ppl_abs_diff:np.float64(0.002529876190237701) - rollout_corr/log_ppl_diff_max:np.float64(0.01464909315109253) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058668553829193115) - rollout_corr/ppl_ratio:np.float64(1.0011543908622116) - rollout_corr/chi2_token:np.float64(0.0018499793950468302) - rollout_corr/chi2_seq:np.float64(0.6211200577672571) - actor/pg_loss:np.float64(0.0055677155032753944) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005267545884635183) - actor/ppo_kl:np.float64(0.0003954675982464906) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(34281.936448936816) - self_distillation/token_reweight_w_std:np.float64(509362.89015717537) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9971434485632926) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12872212124057114) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.36344193667173386) - perf/mfu/actor:np.float64(0.04338788444793293) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.2425880432129) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:6 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020870366133749485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020870366133749485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:246.125 - response_length/max:471.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:246.125 - response_length_non_aborted/max:471.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.3125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.1275674700737e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8456854578107595) - timing_s/agent_loop/generate_sequences/max:np.float64(4.521015455946326) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1125747761689126) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.521015455946326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:185 - timing_s/agent_loop/slowest/response_length:471 - timing_s/gen:11.450798900797963 - timing_s/reward:0.055612821131944656 - timing_s/old_log_prob:2.8146979566663504 - timing_s/adv:0.4819042608141899 - timing_s/update_actor:20.624557754024863 - timing_s/step:35.50594742037356 - timing_s/stop_profile:0.00011634454131126404 - timing_per_token_ms/update_actor:0.14770652682783936 - timing_per_token_ms/gen:0.1817356351701048 - timing_per_token_ms/adv:0.003451245135887117 - perf/total_num_tokens:139632 - perf/time_per_step:35.50594742037356 - perf/throughput:491.57961603877027 (TaskRunner pid=2377588) Training Progress: 97%|█████████▋| 97/100 [1:28:28<01:51, 37.20s/it] (TaskRunner pid=2377588) step:98 - global_seqlen/min:14094 - global_seqlen/max:21465 - global_seqlen/minmax_diff:7371 - global_seqlen/balanced_min:17663 - global_seqlen/balanced_max:17837 - global_seqlen/mean:17686.0 - actor/entropy:0.4419797658920288 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22960913181304932 - training/rollout_probs_diff_mean:0.005934121552854776 - training/rollout_probs_diff_std:0.012329128570854664 - training/rollout_actor_probs_pearson_corr:0.9989829659461975 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.4609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9999937415122986 - rollout_corr/rollout_is_ratio_fraction_high:1.5307372450479306e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.47039532661438 - rollout_corr/rollout_is_min:0.5519552826881409 - rollout_corr/rollout_is_std:0.04011788219213486 - rollout_corr/rollout_is_eff_sample_size:0.9983931415314893 - rollout_corr/rollout_is_seq_mean:1.000035047531128 - rollout_corr/rollout_is_seq_std:0.002616406185552478 - rollout_corr/rollout_is_seq_max:1.0072439908981323 - rollout_corr/rollout_is_seq_min:0.9915816187858582 - rollout_corr/rollout_is_seq_max_deviation:0.008418381214141846 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5622488385997713) - rollout_corr/training_log_ppl:np.float64(0.4412493983982131) - rollout_corr/kl:np.float64(0.0009715645814267759) - rollout_corr/k3_kl:np.float64(0.0010152437158694738) - rollout_corr/rollout_ppl:np.float64(1.5606801309622824) - rollout_corr/rollout_log_ppl:np.float64(0.44027783087221906) - rollout_corr/log_ppl_diff:np.float64(0.0009715675259940326) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024324438418261707) - rollout_corr/log_ppl_diff_max:np.float64(0.012042462825775146) - rollout_corr/log_ppl_diff_min:np.float64(-0.006255775690078735) - rollout_corr/ppl_ratio:np.float64(1.0009767988231033) - rollout_corr/chi2_token:np.float64(0.0020780335180461407) - rollout_corr/chi2_seq:np.float64(0.6138318656012416) - actor/pg_loss:np.float64(0.008576334337703884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009876204467218486) - actor/ppo_kl:np.float64(0.00018930831549113236) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4609375) - self_distillation/token_reweight_w_mean:np.float64(84236.79843885638) - self_distillation/token_reweight_w_std:np.float64(1252939.0888161017) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002153136301786) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15693358180578798) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.47966376692056656) - perf/mfu/actor:np.float64(0.04382159887159613) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(206.1462287902832) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:6 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0032604702282696962 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0032604702282696962 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:255.1875 - response_length/max:760.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:255.1875 - response_length_non_aborted/max:760.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.492039680480957e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7709837183356285) - timing_s/agent_loop/generate_sequences/max:np.float64(6.34854906052351) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0786310864496045) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.34854906052351) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:760 - timing_s/gen:13.463648416101933 - timing_s/reward:0.057271191850304604 - timing_s/old_log_prob:2.8944747261703014 - timing_s/adv:0.4749757442623377 - timing_s/update_actor:20.708794854581356 - timing_s/step:37.688242591917515 - timing_s/stop_profile:0.00011444464325904846 - timing_per_token_ms/update_actor:0.14636431962132024 - timing_per_token_ms/gen:0.20609307519137174 - timing_per_token_ms/adv:0.0033570037336193717 - perf/total_num_tokens:141488 - perf/time_per_step:37.688242591917515 - perf/throughput:469.27101885596744 (TaskRunner pid=2377588) Training Progress: 98%|█████████▊| 98/100 [1:29:06<01:14, 37.36s/it] (TaskRunner pid=2377588) step:99 - global_seqlen/min:15032 - global_seqlen/max:21175 - global_seqlen/minmax_diff:6143 - global_seqlen/balanced_min:17338 - global_seqlen/balanced_max:17414 - global_seqlen/mean:17348.375 - actor/entropy:0.4338042438030243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.13754761219024658 - training/rollout_probs_diff_mean:0.0061327191069722176 - training/rollout_probs_diff_std:0.01257069781422615 - training/rollout_actor_probs_pearson_corr:0.9989312291145325 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.0002034902572632 - rollout_corr/rollout_is_ratio_fraction_high:1.5054120012791827e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.080000162124634 - rollout_corr/rollout_is_min:0.6076822280883789 - rollout_corr/rollout_is_std:0.04208151623606682 - rollout_corr/rollout_is_eff_sample_size:0.9982329891138849 - rollout_corr/rollout_is_seq_mean:1.000204086303711 - rollout_corr/rollout_is_seq_std:0.002738645998761058 - rollout_corr/rollout_is_seq_max:1.0093317031860352 - rollout_corr/rollout_is_seq_min:0.99202561378479 - rollout_corr/rollout_is_seq_max_deviation:0.009331703186035156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.542434255592525) - rollout_corr/training_log_ppl:np.float64(0.4297622809535824) - rollout_corr/kl:np.float64(0.000954678694265354) - rollout_corr/k3_kl:np.float64(0.0010423342322383178) - rollout_corr/rollout_ppl:np.float64(1.540923796594143) - rollout_corr/rollout_log_ppl:np.float64(0.42880760214757174) - rollout_corr/log_ppl_diff:np.float64(0.0009546788060106337) - rollout_corr/log_ppl_abs_diff:np.float64(0.002415620896499604) - rollout_corr/log_ppl_diff_max:np.float64(0.01578676700592041) - rollout_corr/log_ppl_diff_min:np.float64(-0.007883846759796143) - rollout_corr/ppl_ratio:np.float64(1.0009596885647625) - rollout_corr/chi2_token:np.float64(0.0022040612529963255) - rollout_corr/chi2_seq:np.float64(0.6168480273336172) - actor/pg_loss:np.float64(0.006773949251510203) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010614463863021228) - actor/ppo_kl:np.float64(0.0002702048197278373) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(28364.966470204527) - self_distillation/token_reweight_w_std:np.float64(425038.0857956918) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004266444593668) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13330395321827382) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.3726640082895756) - perf/mfu/actor:np.float64(0.04232074698033128) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.85714960098267) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:7 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018535384442657232 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018535384442657232 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:259.48046875 - response_length/max:672.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:259.48046875 - response_length_non_aborted/max:672.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.65625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00029327720403671265 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7771310210227966) - timing_s/agent_loop/generate_sequences/max:np.float64(5.738869354128838) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.157900535305089) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.738869354128838) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:672 - timing_s/gen:13.445963209494948 - timing_s/reward:0.05128566734492779 - timing_s/old_log_prob:3.132158486172557 - timing_s/adv:0.5237186495214701 - timing_s/update_actor:21.05120411142707 - timing_s/step:38.350514786317945 - timing_s/stop_profile:0.00012153387069702148 - timing_per_token_ms/update_actor:0.15167994200773177 - timing_per_token_ms/gen:0.20241713775264497 - timing_per_token_ms/adv:0.0037735425473673333 - perf/total_num_tokens:138787 - perf/time_per_step:38.350514786317945 - perf/throughput:452.3635496592933 (TaskRunner pid=2377588) Training Progress: 99%|█████████▉| 99/100 [1:29:44<00:37, 37.81s/it] (TaskRunner pid=2377588) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2377588) validation generation end (TaskRunner pid=2377588) [prompt] system (TaskRunner pid=2377588) (TaskRunner pid=2377588) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) ... (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2377588) (TaskRunner pid=2377588) A (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) user (TaskRunner pid=2377588) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2377588) (TaskRunner pid=2377588) A: -0.01 (TaskRunner pid=2377588) B: 1.69 (TaskRunner pid=2377588) C: 2.49 (TaskRunner pid=2377588) D: 0.45 (TaskRunner pid=2377588) Please reason step by step. (TaskRunner pid=2377588) assistant (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) [response] (TaskRunner pid=2377588) To determine the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR', we need to evaluate the balance of hydrophobic and hydrophilic residues, as well as the presence of stabilizing interactions such as hydrogen bonds and secondary structures. (TaskRunner pid=2377588) (TaskRunner pid=2377588) The sequence contains a mix of both hydrophilic and hydrophobic residues. Notably, there are several charged residues (e.g., lysine, arginine, glutamic acid, aspartic acid), which can contribute to stabilizing interactions in the protein's folded state. Additionally, the presence of repeated amino acids like 'RR' and 'AA' suggests potential for forming stable secondary structures like alpha-helices or beta-sheets. (TaskRunner pid=2377588) (TaskRunner pid=2377588) However, the sequence also includes some hydrophilic residues (e.g., serine, threonine, glutamine) which may affect the overall folding stability. A score of 0.45 (Option D) suggests a moderate level of stability, which is reasonable given the balance of hydrophilic and hydrophobic residues and the presence of stabilizing interactions. (TaskRunner pid=2377588) (TaskRunner pid=2377588) Among the options provided, 0.45 is the most plausible score for the folding stability of this protein sequence, as it reflects a moderate but not extreme level of stability. (TaskRunner pid=2377588) (TaskRunner pid=2377588) (TaskRunner pid=2377588) D (TaskRunner pid=2377588) (TaskRunner pid=2377588) [ground_truth] B (TaskRunner pid=2377588) [score] 0.0 (TaskRunner pid=2377588) [acc] 0.0 (TaskRunner pid=2377588) [pred] D (TaskRunner pid=2377588) [incorrect_format] 1 (TaskRunner pid=2377588) [feedback] (TaskRunner pid=2377588) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2377588) step:100 - global_seqlen/min:15201 - global_seqlen/max:22595 - global_seqlen/minmax_diff:7394 - global_seqlen/balanced_min:19240 - global_seqlen/balanced_max:20182 - global_seqlen/mean:19359.5 - actor/entropy:0.3639138340950012 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27729934453964233 - training/rollout_probs_diff_mean:0.005966469645500183 - training/rollout_probs_diff_std:0.01337695773690939 - training/rollout_actor_probs_pearson_corr:0.9986874461174011 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9998152852058411 - rollout_corr/rollout_is_ratio_fraction_high:4.59418079117313e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.53139353642473e-05 - rollout_corr/rollout_is_max:2.4048268795013428 - rollout_corr/rollout_is_min:0.3131374716758728 - rollout_corr/rollout_is_std:0.042716238647699356 - rollout_corr/rollout_is_eff_sample_size:0.9981778743517461 - rollout_corr/rollout_is_seq_mean:0.9998635053634644 - rollout_corr/rollout_is_seq_std:0.0027923036832362413 - rollout_corr/rollout_is_seq_max:1.008629322052002 - rollout_corr/rollout_is_seq_min:0.9914166331291199 - rollout_corr/rollout_is_seq_max_deviation:0.008629322052001953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.456413078121841) - rollout_corr/training_log_ppl:np.float64(0.37094445162802003) - rollout_corr/kl:np.float64(0.0011583137813406807) - rollout_corr/k3_kl:np.float64(0.0012635472568831574) - rollout_corr/rollout_ppl:np.float64(1.4546512076631188) - rollout_corr/rollout_log_ppl:np.float64(0.3697861341643147) - rollout_corr/log_ppl_diff:np.float64(0.0011583174637053162) - rollout_corr/log_ppl_abs_diff:np.float64(0.002755315537797287) - rollout_corr/log_ppl_diff_max:np.float64(0.010634154081344604) - rollout_corr/log_ppl_diff_min:np.float64(-0.007962644100189209) - rollout_corr/ppl_ratio:np.float64(1.0011649164371192) - rollout_corr/chi2_token:np.float64(0.003004896454513073) - rollout_corr/chi2_seq:np.float64(0.8863912406377494) - actor/pg_loss:np.float64(0.007820420083589852) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010761070025182562) - actor/ppo_kl:np.float64(7.754255021552581e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(45580.425170551986) - self_distillation/token_reweight_w_std:np.float64(679798.1475089006) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9963978114537895) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13672411735751666) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.39938923716545105) - perf/mfu/actor:np.float64(0.04707397319538395) - perf/max_memory_allocated_gb:np.float64(135.0777406692505) - perf/max_memory_reserved_gb:np.float64(149.6015625) - perf/cpu_memory_used_gb:np.float64(205.93875122070312) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.55) - val-aux/sciknoweval/reward/std@16:np.float64(0.1705123962141389) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.61904) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1400715293582534) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.48004) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.147132195408043) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.55058) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16941156579650474) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.674) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09758132520114339) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4220399999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11484012157584442) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.55496) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1354735155461244) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.71124) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06916601903650618) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3761) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08117633716216308) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5593400000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09364957204441302) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.73856) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04218677040162369) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.3436) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.048294469414788314) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.55908) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06533731091763299) - val-aux/sciknoweval/score/mean@16:np.float64(0.55) - val-aux/sciknoweval/score/std@16:np.float64(0.1705123962141389) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.61904) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1400715293582534) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.48004) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.147132195408043) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.55058) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16941156579650474) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.674) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09758132520114339) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4220399999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11484012157584442) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.55496) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1354735155461244) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.71124) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06916601903650618) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3761) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08117633716216308) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5593400000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09364957204441302) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.73856) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04218677040162369) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.3436) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.048294469414788314) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.55908) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06533731091763299) - val-core/sciknoweval/acc/mean@16:np.float64(0.55) - val-aux/sciknoweval/acc/std@16:np.float64(0.1705123962141389) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.61904) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1400715293582534) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.48004) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.147132195408043) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.55058) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16941156579650474) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.674) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09758132520114339) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4220399999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11484012157584442) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.55496) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1354735155461244) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.71124) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06916601903650618) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3761) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08117633716216308) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5593400000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09364957204441302) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.73856) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04218677040162369) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.3436) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.048294469414788314) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.55908) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06533731091763299) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.004841229182759271) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0008935323161475471) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9977199999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.00635622529493724) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99894) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.004480669592817572) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.99556) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.008311822904754409) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99972) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0023498085028359223) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.99216) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.009763933633531108) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9875) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.009682458365518542) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:7 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005199081264436245 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005199081264436245 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:255.078125 - response_length/max:1625.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:255.078125 - response_length_non_aborted/max:1625.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.90625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00020011886954307556 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9916328880935907) - timing_s/agent_loop/generate_sequences/max:np.float64(11.257305985316634) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1975462599875755) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.257305985316634) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1625 - timing_s/gen:18.558317134156823 - timing_s/reward:0.05930858105421066 - timing_s/old_log_prob:2.932212023064494 - timing_s/adv:0.669283602386713 - timing_s/update_actor:21.07729810476303 - timing_s/step:43.38669454306364 - timing_s/testing:78.02890569157898 - timing_s/save_checkpoint:12.830345639958978 - timing_s/stop_profile:0.00021155737340450287 - timing_per_token_ms/update_actor:0.13609144157108288 - timing_per_token_ms/gen:0.2842008749488028 - timing_per_token_ms/adv:0.004321415857761778 - perf/total_num_tokens:154876 - perf/time_per_step:43.38669454306364 - perf/throughput:446.20822590632366 (TaskRunner pid=2377588) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2377588) "np.float64(0.55), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2377588) "np.float64(0.1705123962141389), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2377588) "np.float64(0.61904), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2377588) "np.float64(0.1400715293582534), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2377588) "np.float64(0.48004), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2377588) "np.float64(0.147132195408043), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2377588) "np.float64(0.55058), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2377588) "np.float64(0.16941156579650474), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2377588) "np.float64(0.674), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2377588) "np.float64(0.09758132520114339), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2377588) "np.float64(0.4220399999999999), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2377588) "np.float64(0.11484012157584442), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2377588) "np.float64(0.55496), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2377588) "np.float64(0.1354735155461244), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2377588) "np.float64(0.71124), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2377588) "np.float64(0.06916601903650618), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2377588) "np.float64(0.3761), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2377588) "np.float64(0.08117633716216308), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2377588) "np.float64(0.5593400000000001), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2377588) "np.float64(0.09364957204441302), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2377588) "np.float64(0.73856), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2377588) "np.float64(0.04218677040162369), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2377588) "np.float64(0.3436), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2377588) "np.float64(0.048294469414788314), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2377588) "np.float64(0.55908), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2377588) "np.float64(0.06533731091763299), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2377588) "np.float64(0.55), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2377588) "np.float64(0.1705123962141389), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2377588) "np.float64(0.61904), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2377588) "np.float64(0.1400715293582534), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2377588) "np.float64(0.48004), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2377588) "np.float64(0.147132195408043), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2377588) "np.float64(0.55058), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2377588) "np.float64(0.16941156579650474), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2377588) "np.float64(0.674), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2377588) "np.float64(0.09758132520114339), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2377588) "np.float64(0.4220399999999999), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2377588) "np.float64(0.11484012157584442), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2377588) "np.float64(0.55496), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2377588) "np.float64(0.1354735155461244), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2377588) "np.float64(0.71124), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2377588) "np.float64(0.06916601903650618), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2377588) "np.float64(0.3761), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2377588) "np.float64(0.08117633716216308), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2377588) "np.float64(0.5593400000000001), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2377588) "np.float64(0.09364957204441302), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2377588) "np.float64(0.73856), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2377588) "np.float64(0.04218677040162369), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2377588) "np.float64(0.3436), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2377588) "np.float64(0.048294469414788314), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2377588) "np.float64(0.55908), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2377588) "np.float64(0.06533731091763299), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2377588) "np.float64(0.55), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2377588) "np.float64(0.1705123962141389), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2377588) "np.float64(0.61904), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2377588) "np.float64(0.1400715293582534), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2377588) "np.float64(0.48004), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2377588) "np.float64(0.147132195408043), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2377588) "np.float64(0.55058), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2377588) "np.float64(0.16941156579650474), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2377588) "np.float64(0.674), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2377588) "np.float64(0.09758132520114339), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2377588) "np.float64(0.4220399999999999), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2377588) "np.float64(0.11484012157584442), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2377588) "np.float64(0.55496), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2377588) "np.float64(0.1354735155461244), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2377588) "np.float64(0.71124), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2377588) "np.float64(0.06916601903650618), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2377588) "np.float64(0.3761), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2377588) "np.float64(0.08117633716216308), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2377588) "np.float64(0.5593400000000001), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2377588) "np.float64(0.09364957204441302), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2377588) "np.float64(0.73856), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2377588) "np.float64(0.04218677040162369), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2377588) "np.float64(0.3436), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2377588) "np.float64(0.048294469414788314), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2377588) "np.float64(0.55908), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2377588) 'np.float64(0.06533731091763299), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.99875), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2377588) 'np.float64(0.004841229182759271), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2377588) 'np.float64(0.9999600000000001), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2377588) 'np.float64(0.0008935323161475471), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2377588) 'np.float64(0.9977199999999999), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2377588) 'np.float64(0.00635622529493724), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.99894), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2377588) 'np.float64(0.004480669592817572), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(0.99556), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2377588) 'np.float64(0.008311822904754409), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(0.99972), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2377588) 'np.float64(0.0023498085028359223), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(0.99216), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2377588) 'np.float64(0.009763933633531108), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(0.9875), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2377588) 'np.float64(0.009682458365518542), ' (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2377588) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2377588) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2377588) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2377588) Training Progress: 100%|██████████| 100/100 [1:31:59<00:00, 66.76s/it] Training Progress: 100%|██████████| 100/100 [1:31:59<00:00, 55.19s/it] (TaskRunner pid=2377588) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2377588) wandb: uploading output.log; uploading history steps 98-99, summary, console lines 601-719 (TaskRunner pid=2377588) wandb: uploading data (TaskRunner pid=2377588) wandb: (TaskRunner pid=2377588) wandb: Run history: (TaskRunner pid=2377588) wandb: actor/entropy ▂▂▄▁▅▇█▅▃▄▄▅▅▅▅▇▄▇█▆▇▅▄▅▃▂▃▂▃▃▁▂▂▁▂▂▂▄▅▃ (TaskRunner pid=2377588) wandb: actor/grad_norm █▅▃▅▄▄▅▃▃▄▅▆▄▃▁▄█▆▃▄▃▄▃▄▃▂▃▄▄▅▃▂▃▃▃▂▄▄▂▁ (TaskRunner pid=2377588) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2377588) wandb: actor/lr ▁▆██████████████████████████████████████ (TaskRunner pid=2377588) wandb: actor/pg_clipfrac ▁▂▂▂▁▂▃▂▃▃█▃▂▅▄▅▄▃▂▂▂▄▅▅▇▅▄▃▅▅▄▃▅▅▆▅▅▄▃▃ (TaskRunner pid=2377588) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2377588) wandb: actor/pg_loss ▃▃▃▃▅▅▅▅▇▄█▂▆▅▆▄▄▄▄▃▄▃▃▅▃▃▆▁▂▂▁▃▃▂▄▄▂▄▂▃ (TaskRunner pid=2377588) wandb: actor/ppo_kl ▃▂▂▃▂▁▅▄▅▅▅▄▆▄▄▄▅▇█▃▄▁▄█▅▆▅▃▃▆▄▅▇▄▄▇▅▁█▅ (TaskRunner pid=2377588) wandb: critic/advantages/max █████████████████████████████▁█████▁████ (TaskRunner pid=2377588) wandb: critic/advantages/mean ▄▆█▃▆▆▇▇█▁▆▆▆▇▇▁▆▆▆▆▆▇▆▇▅▆▆▆▇▆▇▆▆▆▇▇▆▆▇▆ (TaskRunner pid=2377588) wandb: +220 ... (TaskRunner pid=2377588) wandb: (TaskRunner pid=2377588) wandb: Run summary: (TaskRunner pid=2377588) wandb: actor/entropy 0.36391 (TaskRunner pid=2377588) wandb: actor/grad_norm 0.39939 (TaskRunner pid=2377588) wandb: actor/kl_loss 0 (TaskRunner pid=2377588) wandb: actor/lr 0.0 (TaskRunner pid=2377588) wandb: actor/pg_clipfrac 0.00108 (TaskRunner pid=2377588) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2377588) wandb: actor/pg_loss 0.00782 (TaskRunner pid=2377588) wandb: actor/ppo_kl 8e-05 (TaskRunner pid=2377588) wandb: critic/advantages/max 0.875 (TaskRunner pid=2377588) wandb: critic/advantages/mean -0.0052 (TaskRunner pid=2377588) wandb: +220 ... (TaskRunner pid=2377588) wandb: (TaskRunner pid=2377588) wandb: 🚀 View run qwen3gen-biology-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/gj54iy5b (TaskRunner pid=2377588) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2377588) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2377588) wandb: Find logs at: ./wandb/run-20260703_091731-gj54iy5b/logs (TaskRunner pid=2377588) Exception ignored in atexit callback: .teardown_atexit at 0x77a81f17b920> (TaskRunner pid=2377588) Traceback (most recent call last): (TaskRunner pid=2377588) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2377588) conn.teardown(hooks.exit_code) (TaskRunner pid=2377588) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2377588) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2377588) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2377588) return future.result() (TaskRunner pid=2377588) ^^^^^^^^^^^^^^^ (TaskRunner pid=2377588) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2377588) return self.__get_result() (TaskRunner pid=2377588) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2377588) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2377588) raise self._exception (TaskRunner pid=2377588) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2377588) return await fn() (TaskRunner pid=2377588) ^^^^^^^^^^ (TaskRunner pid=2377588) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2377588) await self._client.close() (TaskRunner pid=2377588) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2377588) await self._writer.wait_closed() (TaskRunner pid=2377588) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2377588) await self._protocol._get_close_waiter(self) (TaskRunner pid=2377588) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-03 10:49:42] Finished Qwen/Qwen3-8B biology rlsd_tr. [2026-07-03 10:49:42] Starting Qwen/Qwen3-8B material grpo. Experiment: qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: material Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet Ray tmp: /tmp/ray_q3g_material_grpo_Qwen3_8B 2026-07-03 10:49:44,074 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/material Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_material_grpo_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/material +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_material_grpo_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/material', 'EXPERIMENT': 'qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 10:49:53,802 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2403642) TaskRunner hostname: mole-workspace-rw, PID: 2403642 (TaskRunner pid=2403642) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2403642) 'calculate_entropy': False, (TaskRunner pid=2403642) 'calculate_sum_pi_squared': False, (TaskRunner pid=2403642) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2403642) 'async_save': False, (TaskRunner pid=2403642) 'load_contents': ['model', (TaskRunner pid=2403642) 'optimizer', (TaskRunner pid=2403642) 'extra'], (TaskRunner pid=2403642) 'save_contents': ['model', (TaskRunner pid=2403642) 'optimizer', (TaskRunner pid=2403642) 'extra']}, (TaskRunner pid=2403642) 'clip_ratio': 0.2, (TaskRunner pid=2403642) 'clip_ratio_c': 3.0, (TaskRunner pid=2403642) 'clip_ratio_high': 0.28, (TaskRunner pid=2403642) 'clip_ratio_low': 0.2, (TaskRunner pid=2403642) 'data_loader_seed': 42, (TaskRunner pid=2403642) 'entropy_checkpointing': False, (TaskRunner pid=2403642) 'entropy_coeff': 0, (TaskRunner pid=2403642) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2403642) 'freeze_vision_tower': False, (TaskRunner pid=2403642) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2403642) 'dtype': 'bfloat16', (TaskRunner pid=2403642) 'entropy_checkpointing': False, (TaskRunner pid=2403642) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2403642) 'forward_only': False, (TaskRunner pid=2403642) 'forward_prefetch': False, (TaskRunner pid=2403642) 'fsdp_size': -1, (TaskRunner pid=2403642) 'full_determinism': False, (TaskRunner pid=2403642) 'model_dtype': 'fp32', (TaskRunner pid=2403642) 'offload_policy': False, (TaskRunner pid=2403642) 'optimizer_offload': False, (TaskRunner pid=2403642) 'param_offload': False, (TaskRunner pid=2403642) 'reshard_after_forward': True, (TaskRunner pid=2403642) 'seed': 42, (TaskRunner pid=2403642) 'strategy': 'fsdp', (TaskRunner pid=2403642) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2403642) 'use_orig_params': False, (TaskRunner pid=2403642) 'use_torch_compile': True, (TaskRunner pid=2403642) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2403642) 'grad_clip': 1.0, (TaskRunner pid=2403642) 'kl_loss_coef': 0.0, (TaskRunner pid=2403642) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2403642) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2403642) 'loss_scale_factor': None, (TaskRunner pid=2403642) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2403642) 'betas': [0.9, 0.999], (TaskRunner pid=2403642) 'clip_grad': 1.0, (TaskRunner pid=2403642) 'lr': 1e-06, (TaskRunner pid=2403642) 'lr_scheduler_type': 'constant', (TaskRunner pid=2403642) 'lr_warmup_steps': 10, (TaskRunner pid=2403642) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2403642) 'min_lr_ratio': 0.0, (TaskRunner pid=2403642) 'num_cycles': 0.5, (TaskRunner pid=2403642) 'optimizer': 'AdamW', (TaskRunner pid=2403642) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2403642) 'override_optimizer_config': None, (TaskRunner pid=2403642) 'total_training_steps': -1, (TaskRunner pid=2403642) 'warmup_style': None, (TaskRunner pid=2403642) 'weight_decay': 0.01}, (TaskRunner pid=2403642) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2403642) 'clip_cov_lb': 1.0, (TaskRunner pid=2403642) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2403642) 'clip_cov_ub': 5.0, (TaskRunner pid=2403642) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2403642) 'loss_mode': 'vanilla', (TaskRunner pid=2403642) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2403642) 'ppo_epochs': 1, (TaskRunner pid=2403642) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2403642) 'ppo_micro_batch_size': None, (TaskRunner pid=2403642) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2403642) 'ppo_mini_batch_size': 8, (TaskRunner pid=2403642) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2403642) 'all_ranks': False, (TaskRunner pid=2403642) 'enable': False, (TaskRunner pid=2403642) 'ranks': [], (TaskRunner pid=2403642) 'save_path': 'outputs/profile', (TaskRunner pid=2403642) 'tool': None, (TaskRunner pid=2403642) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2403642) 'analysis': True, (TaskRunner pid=2403642) 'contents': [], (TaskRunner pid=2403642) 'discrete': False, (TaskRunner pid=2403642) 'level': 'level0'}, (TaskRunner pid=2403642) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2403642) 'discrete': False}, (TaskRunner pid=2403642) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2403642) 'step_end': None, (TaskRunner pid=2403642) 'step_start': 0}, (TaskRunner pid=2403642) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2403642) 'stack_depth': 32, (TaskRunner pid=2403642) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2403642) 'rollout_n': 8, (TaskRunner pid=2403642) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2403642) 'mode': 'disabled', (TaskRunner pid=2403642) 'record_file': None, (TaskRunner pid=2403642) 'replay_file': None}, (TaskRunner pid=2403642) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2403642) 'alpha': 0.5, (TaskRunner pid=2403642) 'distillation_add_tail': True, (TaskRunner pid=2403642) 'distillation_topk': 100, (TaskRunner pid=2403642) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2403642) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2403642) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2403642) 'enable': False, (TaskRunner pid=2403642) 'loss_weight': 0.0, (TaskRunner pid=2403642) 'mask': 'all'}, (TaskRunner pid=2403642) 'feedback_template': '\n' (TaskRunner pid=2403642) 'The ' (TaskRunner pid=2403642) 'following ' (TaskRunner pid=2403642) 'is ' (TaskRunner pid=2403642) 'feedback ' (TaskRunner pid=2403642) 'from ' (TaskRunner pid=2403642) 'your ' (TaskRunner pid=2403642) 'unsuccessful ' (TaskRunner pid=2403642) 'earlier ' (TaskRunner pid=2403642) 'attempt:\n' (TaskRunner pid=2403642) '\n' (TaskRunner pid=2403642) '{feedback_raw}', (TaskRunner pid=2403642) 'full_logit_distillation': True, (TaskRunner pid=2403642) 'include_environment_feedback': True, (TaskRunner pid=2403642) 'is_clip': 2, (TaskRunner pid=2403642) 'max_reprompt_len': 22528, (TaskRunner pid=2403642) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2403642) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2403642) '\n' (TaskRunner pid=2403642) 'Correctly ' (TaskRunner pid=2403642) 'solve ' (TaskRunner pid=2403642) 'the ' (TaskRunner pid=2403642) 'original ' (TaskRunner pid=2403642) 'question.', (TaskRunner pid=2403642) 'reprompt_truncation': 'right', (TaskRunner pid=2403642) 'solution_template': '\n' (TaskRunner pid=2403642) 'Correct ' (TaskRunner pid=2403642) 'solution:\n' (TaskRunner pid=2403642) '\n' (TaskRunner pid=2403642) '{successful_previous_attempt}', (TaskRunner pid=2403642) 'srpo_dynamic_weighting': False, (TaskRunner pid=2403642) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2403642) 'success_reward_threshold': 0.5, (TaskRunner pid=2403642) 'teacher_regularization': 'ema', (TaskRunner pid=2403642) 'teacher_update_rate': 0.0, (TaskRunner pid=2403642) 'token_reweight_decay_steps': None, (TaskRunner pid=2403642) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2403642) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2403642) 'shuffle': False, (TaskRunner pid=2403642) 'strategy': 'fsdp', (TaskRunner pid=2403642) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2403642) 'tau_neg': 1.05, (TaskRunner pid=2403642) 'tau_pos': 1.0, (TaskRunner pid=2403642) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2403642) 'use_dynamic_bsz': False, (TaskRunner pid=2403642) 'use_fused_kernels': False, (TaskRunner pid=2403642) 'use_kl_loss': False, (TaskRunner pid=2403642) 'use_prefix_grouper': False, (TaskRunner pid=2403642) 'use_remove_padding': True, (TaskRunner pid=2403642) 'use_torch_compile': True}, (TaskRunner pid=2403642) 'hybrid_engine': True, (TaskRunner pid=2403642) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2403642) 'custom_chat_template': None, (TaskRunner pid=2403642) 'enable_activation_offload': False, (TaskRunner pid=2403642) 'enable_gradient_checkpointing': True, (TaskRunner pid=2403642) 'exclude_modules': None, (TaskRunner pid=2403642) 'external_lib': None, (TaskRunner pid=2403642) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2403642) 'hf_config_path': None, (TaskRunner pid=2403642) 'lora_adapter_path': None, (TaskRunner pid=2403642) 'lora_alpha': 16, (TaskRunner pid=2403642) 'lora_rank': 0, (TaskRunner pid=2403642) 'override_config': {}, (TaskRunner pid=2403642) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2403642) 'target_modules': 'all-linear', (TaskRunner pid=2403642) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2403642) 'num_shards': 4}, (TaskRunner pid=2403642) 'tokenizer_path': None, (TaskRunner pid=2403642) 'trust_remote_code': True, (TaskRunner pid=2403642) 'use_fused_kernels': False, (TaskRunner pid=2403642) 'use_liger': False, (TaskRunner pid=2403642) 'use_remove_padding': True, (TaskRunner pid=2403642) 'use_shm': False}, (TaskRunner pid=2403642) 'nccl_timeout': 600, (TaskRunner pid=2403642) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2403642) 'entropy_checkpointing': False, (TaskRunner pid=2403642) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2403642) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2403642) 'dtype': 'bfloat16', (TaskRunner pid=2403642) 'entropy_checkpointing': False, (TaskRunner pid=2403642) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2403642) 'forward_only': True, (TaskRunner pid=2403642) 'forward_prefetch': False, (TaskRunner pid=2403642) 'fsdp_size': -1, (TaskRunner pid=2403642) 'full_determinism': False, (TaskRunner pid=2403642) 'model_dtype': 'fp32', (TaskRunner pid=2403642) 'offload_policy': False, (TaskRunner pid=2403642) 'optimizer_offload': False, (TaskRunner pid=2403642) 'param_offload': False, (TaskRunner pid=2403642) 'reshard_after_forward': True, (TaskRunner pid=2403642) 'seed': 42, (TaskRunner pid=2403642) 'strategy': 'fsdp', (TaskRunner pid=2403642) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2403642) 'use_orig_params': False, (TaskRunner pid=2403642) 'use_torch_compile': True, (TaskRunner pid=2403642) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2403642) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2403642) 'log_prob_micro_batch_size': None, (TaskRunner pid=2403642) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2403642) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2403642) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2403642) 'all_ranks': False, (TaskRunner pid=2403642) 'enable': False, (TaskRunner pid=2403642) 'ranks': [], (TaskRunner pid=2403642) 'save_path': 'outputs/profile', (TaskRunner pid=2403642) 'tool': None, (TaskRunner pid=2403642) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2403642) 'analysis': True, (TaskRunner pid=2403642) 'contents': [], (TaskRunner pid=2403642) 'discrete': False, (TaskRunner pid=2403642) 'level': 'level0'}, (TaskRunner pid=2403642) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2403642) 'discrete': False}, (TaskRunner pid=2403642) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2403642) 'step_end': None, (TaskRunner pid=2403642) 'step_start': 0}, (TaskRunner pid=2403642) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2403642) 'stack_depth': 32, (TaskRunner pid=2403642) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2403642) 'rollout_n': 8, (TaskRunner pid=2403642) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2403642) 'mode': 'disabled', (TaskRunner pid=2403642) 'record_file': None, (TaskRunner pid=2403642) 'replay_file': None}, (TaskRunner pid=2403642) 'strategy': 'fsdp', (TaskRunner pid=2403642) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2403642) 'use_torch_compile': True}, (TaskRunner pid=2403642) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2403642) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2403642) 'agent_loop_config_path': None, (TaskRunner pid=2403642) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2403642) 'name': None, (TaskRunner pid=2403642) 'path': None}, (TaskRunner pid=2403642) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2403642) 'num_workers': 8}, (TaskRunner pid=2403642) 'calculate_log_probs': True, (TaskRunner pid=2403642) 'cudagraph_capture_sizes': None, (TaskRunner pid=2403642) 'data_parallel_size': 1, (TaskRunner pid=2403642) 'disable_log_stats': True, (TaskRunner pid=2403642) 'do_sample': True, (TaskRunner pid=2403642) 'dtype': 'bfloat16', (TaskRunner pid=2403642) 'enable_chunked_prefill': True, (TaskRunner pid=2403642) 'enable_prefix_caching': True, (TaskRunner pid=2403642) 'enable_rollout_routing_replay': False, (TaskRunner pid=2403642) 'enforce_eager': False, (TaskRunner pid=2403642) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2403642) 'expert_parallel_size': 1, (TaskRunner pid=2403642) 'free_cache_engine': True, (TaskRunner pid=2403642) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2403642) 'ignore_eos': False, (TaskRunner pid=2403642) 'layered_summon': False, (TaskRunner pid=2403642) 'load_format': 'dummy', (TaskRunner pid=2403642) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2403642) 'log_prob_micro_batch_size': None, (TaskRunner pid=2403642) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2403642) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2403642) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2403642) 'max_model_len': 10240, (TaskRunner pid=2403642) 'max_num_batched_tokens': 10240, (TaskRunner pid=2403642) 'max_num_seqs': 1024, (TaskRunner pid=2403642) 'mode': 'async', (TaskRunner pid=2403642) 'multi_stage_wake_up': False, (TaskRunner pid=2403642) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2403642) 'enable': False, (TaskRunner pid=2403642) 'format': 'hermes', (TaskRunner pid=2403642) 'interaction_config_path': None, (TaskRunner pid=2403642) 'max_assistant_turns': None, (TaskRunner pid=2403642) 'max_parallel_calls': 1, (TaskRunner pid=2403642) 'max_tool_response_length': 256, (TaskRunner pid=2403642) 'max_user_turns': None, (TaskRunner pid=2403642) 'num_repeat_rollouts': None, (TaskRunner pid=2403642) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2403642) 'tool_config_path': None, (TaskRunner pid=2403642) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2403642) 'use_inference_chat_template': False}, (TaskRunner pid=2403642) 'n': 8, (TaskRunner pid=2403642) 'name': 'vllm', (TaskRunner pid=2403642) 'over_sample_rate': 0, (TaskRunner pid=2403642) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2403642) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2403642) 'all_ranks': False, (TaskRunner pid=2403642) 'enable': False, (TaskRunner pid=2403642) 'ranks': [], (TaskRunner pid=2403642) 'save_path': 'outputs/profile', (TaskRunner pid=2403642) 'tool': None, (TaskRunner pid=2403642) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2403642) 'analysis': True, (TaskRunner pid=2403642) 'contents': [], (TaskRunner pid=2403642) 'discrete': False, (TaskRunner pid=2403642) 'level': 'level0'}, (TaskRunner pid=2403642) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2403642) 'discrete': False}, (TaskRunner pid=2403642) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2403642) 'step_end': None, (TaskRunner pid=2403642) 'step_start': 0}, (TaskRunner pid=2403642) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2403642) 'stack_depth': 32, (TaskRunner pid=2403642) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2403642) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2403642) 'enable': False, (TaskRunner pid=2403642) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2403642) 'port': 9090, (TaskRunner pid=2403642) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2403642) 'prompt_length': 2048, (TaskRunner pid=2403642) 'quantization': None, (TaskRunner pid=2403642) 'quantization_config_file': None, (TaskRunner pid=2403642) 'response_length': 8192, (TaskRunner pid=2403642) 'scheduling_policy': 'fcfs', (TaskRunner pid=2403642) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2403642) 'skip_rollout': False, (TaskRunner pid=2403642) 'skip_tokenizer_init': True, (TaskRunner pid=2403642) 'temperature': 1.0, (TaskRunner pid=2403642) 'tensor_model_parallel_size': 2, (TaskRunner pid=2403642) 'top_k': -1, (TaskRunner pid=2403642) 'top_p': 1.0, (TaskRunner pid=2403642) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2403642) 'backend': None, (TaskRunner pid=2403642) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2403642) 'token2text': False}, (TaskRunner pid=2403642) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2403642) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2403642) 'do_sample': True, (TaskRunner pid=2403642) 'n': 16, (TaskRunner pid=2403642) 'temperature': 0.6, (TaskRunner pid=2403642) 'top_k': -1, (TaskRunner pid=2403642) 'top_p': 0.95}}}, (TaskRunner pid=2403642) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2403642) 'adv_estimator': 'grpo', (TaskRunner pid=2403642) 'gamma': 1.0, (TaskRunner pid=2403642) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2403642) 'horizon': 10000, (TaskRunner pid=2403642) 'kl_coef': 0.001, (TaskRunner pid=2403642) 'target_kl': 0.1, (TaskRunner pid=2403642) 'type': 'fixed'}, (TaskRunner pid=2403642) 'kl_penalty': 'kl', (TaskRunner pid=2403642) 'lam': 1.0, (TaskRunner pid=2403642) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2403642) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2403642) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2403642) 'loss_type': 'ppo_clip', (TaskRunner pid=2403642) 'rollout_is': 'token', (TaskRunner pid=2403642) 'rollout_is_batch_normalize': False, (TaskRunner pid=2403642) 'rollout_is_threshold': 2.0, (TaskRunner pid=2403642) 'rollout_rs': None, (TaskRunner pid=2403642) 'rollout_rs_threshold': None}, (TaskRunner pid=2403642) 'use_kl_in_reward': False, (TaskRunner pid=2403642) 'use_pf_ppo': False}, (TaskRunner pid=2403642) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2403642) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2403642) 'async_save': False, (TaskRunner pid=2403642) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2403642) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2403642) 'cliprange_value': 0.5, (TaskRunner pid=2403642) 'data_loader_seed': 42, (TaskRunner pid=2403642) 'enable': None, (TaskRunner pid=2403642) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2403642) 'forward_micro_batch_size': None, (TaskRunner pid=2403642) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2403642) 'grad_clip': 1.0, (TaskRunner pid=2403642) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2403642) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2403642) 'enable_activation_offload': False, (TaskRunner pid=2403642) 'enable_gradient_checkpointing': True, (TaskRunner pid=2403642) 'external_lib': None, (TaskRunner pid=2403642) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2403642) 'dtype': 'bfloat16', (TaskRunner pid=2403642) 'entropy_checkpointing': False, (TaskRunner pid=2403642) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2403642) 'forward_only': False, (TaskRunner pid=2403642) 'forward_prefetch': False, (TaskRunner pid=2403642) 'fsdp_size': -1, (TaskRunner pid=2403642) 'full_determinism': False, (TaskRunner pid=2403642) 'model_dtype': 'fp32', (TaskRunner pid=2403642) 'offload_policy': False, (TaskRunner pid=2403642) 'optimizer_offload': False, (TaskRunner pid=2403642) 'param_offload': False, (TaskRunner pid=2403642) 'reshard_after_forward': True, (TaskRunner pid=2403642) 'seed': 42, (TaskRunner pid=2403642) 'strategy': 'fsdp', (TaskRunner pid=2403642) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2403642) 'use_orig_params': False, (TaskRunner pid=2403642) 'use_torch_compile': True, (TaskRunner pid=2403642) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2403642) 'lora_alpha': 16, (TaskRunner pid=2403642) 'lora_rank': 0, (TaskRunner pid=2403642) 'override_config': {}, (TaskRunner pid=2403642) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2403642) 'target_modules': 'all-linear', (TaskRunner pid=2403642) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2403642) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2403642) 'trust_remote_code': True, (TaskRunner pid=2403642) 'use_remove_padding': False, (TaskRunner pid=2403642) 'use_shm': False}, (TaskRunner pid=2403642) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2403642) 'betas': [0.9, 0.999], (TaskRunner pid=2403642) 'clip_grad': 1.0, (TaskRunner pid=2403642) 'lr': 1e-05, (TaskRunner pid=2403642) 'lr_scheduler_type': 'constant', (TaskRunner pid=2403642) 'lr_warmup_steps': -1, (TaskRunner pid=2403642) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2403642) 'min_lr_ratio': 0.0, (TaskRunner pid=2403642) 'num_cycles': 0.5, (TaskRunner pid=2403642) 'optimizer': 'AdamW', (TaskRunner pid=2403642) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2403642) 'override_optimizer_config': None, (TaskRunner pid=2403642) 'total_training_steps': -1, (TaskRunner pid=2403642) 'warmup_style': None, (TaskRunner pid=2403642) 'weight_decay': 0.01}, (TaskRunner pid=2403642) 'ppo_epochs': 1, (TaskRunner pid=2403642) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2403642) 'ppo_micro_batch_size': None, (TaskRunner pid=2403642) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2403642) 'ppo_mini_batch_size': 8, (TaskRunner pid=2403642) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2403642) 'all_ranks': False, (TaskRunner pid=2403642) 'enable': False, (TaskRunner pid=2403642) 'ranks': [], (TaskRunner pid=2403642) 'save_path': 'outputs/profile', (TaskRunner pid=2403642) 'tool': None, (TaskRunner pid=2403642) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2403642) 'analysis': True, (TaskRunner pid=2403642) 'contents': [], (TaskRunner pid=2403642) 'discrete': False, (TaskRunner pid=2403642) 'level': 'level0'}, (TaskRunner pid=2403642) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2403642) 'discrete': False}, (TaskRunner pid=2403642) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2403642) 'step_end': None, (TaskRunner pid=2403642) 'step_start': 0}, (TaskRunner pid=2403642) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2403642) 'stack_depth': 32, (TaskRunner pid=2403642) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2403642) 'rollout_n': 8, (TaskRunner pid=2403642) 'shuffle': False, (TaskRunner pid=2403642) 'strategy': 'fsdp', (TaskRunner pid=2403642) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2403642) 'use_dynamic_bsz': False}, (TaskRunner pid=2403642) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2403642) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2403642) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2403642) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2403642) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2403642) 'dataloader_num_workers': 8, (TaskRunner pid=2403642) 'filter_overlong_prompts': True, (TaskRunner pid=2403642) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2403642) 'image_key': 'images', (TaskRunner pid=2403642) 'image_patch_size': 14, (TaskRunner pid=2403642) 'max_prompt_length': 2048, (TaskRunner pid=2403642) 'max_response_length': 8192, (TaskRunner pid=2403642) 'prompt_key': 'prompt', (TaskRunner pid=2403642) 'return_full_prompt': False, (TaskRunner pid=2403642) 'return_multi_modal_inputs': True, (TaskRunner pid=2403642) 'return_raw_chat': True, (TaskRunner pid=2403642) 'return_raw_input_ids': False, (TaskRunner pid=2403642) 'reward_fn_key': 'data_source', (TaskRunner pid=2403642) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2403642) 'seed': None, (TaskRunner pid=2403642) 'shuffle': True, (TaskRunner pid=2403642) 'tokenizer': None, (TaskRunner pid=2403642) 'tool_config_path': None, (TaskRunner pid=2403642) 'train_batch_size': 32, (TaskRunner pid=2403642) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet'], (TaskRunner pid=2403642) 'train_max_samples': 3200, (TaskRunner pid=2403642) 'truncation': 'error', (TaskRunner pid=2403642) 'trust_remote_code': True, (TaskRunner pid=2403642) 'use_shm': False, (TaskRunner pid=2403642) 'val_batch_size': None, (TaskRunner pid=2403642) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet'], (TaskRunner pid=2403642) 'val_max_samples': -1, (TaskRunner pid=2403642) 'validation_shuffle': False, (TaskRunner pid=2403642) 'video_key': 'videos'}, (TaskRunner pid=2403642) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2403642) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2403642) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2403642) 'cuda-memory-usage': 'true', (TaskRunner pid=2403642) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2403642) 'discrete': False, (TaskRunner pid=2403642) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2403642) 'capture-range-end': None, (TaskRunner pid=2403642) 'cuda-graph-trace': 'graph', (TaskRunner pid=2403642) 'cuda-memory-usage': 'true', (TaskRunner pid=2403642) 'kill': 'none', (TaskRunner pid=2403642) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2403642) 'torch_memory': {'context': 'all', (TaskRunner pid=2403642) 'kw_args': {}, (TaskRunner pid=2403642) 'stack_depth': 32, (TaskRunner pid=2403642) 'stacks': 'all', (TaskRunner pid=2403642) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2403642) 'profile_continuous_steps': False, (TaskRunner pid=2403642) 'save_path': 'outputs/profile', (TaskRunner pid=2403642) 'steps': None, (TaskRunner pid=2403642) 'tool': None}, (TaskRunner pid=2403642) 'max_model_len': 10240, (TaskRunner pid=2403642) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_material_grpo_Qwen3_8B', (TaskRunner pid=2403642) 'include_dashboard': False, (TaskRunner pid=2403642) 'num_cpus': None, (TaskRunner pid=2403642) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2403642) 'TASK': 'datasets/sciknoweval/material', (TaskRunner pid=2403642) 'USER': 'root'}}}, (TaskRunner pid=2403642) 'timeline_json_file': None}, (TaskRunner pid=2403642) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2403642) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2403642) 'name': 'custom_reward_manager', (TaskRunner pid=2403642) 'path': None}, (TaskRunner pid=2403642) 'name': 'naive', (TaskRunner pid=2403642) 'source': 'register'}, (TaskRunner pid=2403642) 'reward_model': {'enable': False, (TaskRunner pid=2403642) 'enable_resource_pool': False, (TaskRunner pid=2403642) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2403642) 'launch_reward_fn_async': False, (TaskRunner pid=2403642) 'max_length': None, (TaskRunner pid=2403642) 'micro_batch_size': None, (TaskRunner pid=2403642) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2403642) 'model': {'external_lib': None, (TaskRunner pid=2403642) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2403642) 'forward_prefetch': False, (TaskRunner pid=2403642) 'fsdp_size': -1, (TaskRunner pid=2403642) 'param_offload': False, (TaskRunner pid=2403642) 'reshard_after_forward': True, (TaskRunner pid=2403642) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2403642) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2403642) 'override_config': {}, (TaskRunner pid=2403642) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2403642) 'trust_remote_code': False, (TaskRunner pid=2403642) 'use_fused_kernels': False, (TaskRunner pid=2403642) 'use_remove_padding': False, (TaskRunner pid=2403642) 'use_shm': False}, (TaskRunner pid=2403642) 'n_gpus_per_node': 8, (TaskRunner pid=2403642) 'nnodes': 0, (TaskRunner pid=2403642) 'num_workers': 1, (TaskRunner pid=2403642) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2403642) 'all_ranks': False, (TaskRunner pid=2403642) 'enable': False, (TaskRunner pid=2403642) 'ranks': [], (TaskRunner pid=2403642) 'save_path': 'outputs/profile', (TaskRunner pid=2403642) 'tool': None, (TaskRunner pid=2403642) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2403642) 'analysis': True, (TaskRunner pid=2403642) 'contents': [], (TaskRunner pid=2403642) 'discrete': False, (TaskRunner pid=2403642) 'level': 'level0'}, (TaskRunner pid=2403642) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2403642) 'discrete': False}, (TaskRunner pid=2403642) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2403642) 'step_end': None, (TaskRunner pid=2403642) 'step_start': 0}, (TaskRunner pid=2403642) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2403642) 'stack_depth': 32, (TaskRunner pid=2403642) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2403642) 'reward_loop_class_name': None, (TaskRunner pid=2403642) 'reward_loop_module_path': None, (TaskRunner pid=2403642) 'reward_loop_source': 'register', (TaskRunner pid=2403642) 'reward_manager': 'naive', (TaskRunner pid=2403642) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2403642) 'cudagraph_capture_sizes': None, (TaskRunner pid=2403642) 'data_parallel_size': 1, (TaskRunner pid=2403642) 'disable_log_stats': True, (TaskRunner pid=2403642) 'dtype': 'bfloat16', (TaskRunner pid=2403642) 'enable_chunked_prefill': True, (TaskRunner pid=2403642) 'enable_prefix_caching': True, (TaskRunner pid=2403642) 'enforce_eager': True, (TaskRunner pid=2403642) 'engine_kwargs': {}, (TaskRunner pid=2403642) 'expert_parallel_size': 1, (TaskRunner pid=2403642) 'free_cache_engine': True, (TaskRunner pid=2403642) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2403642) 'limit_images': None, (TaskRunner pid=2403642) 'load_format': 'auto', (TaskRunner pid=2403642) 'max_model_len': None, (TaskRunner pid=2403642) 'max_num_batched_tokens': 8192, (TaskRunner pid=2403642) 'max_num_seqs': 1024, (TaskRunner pid=2403642) 'name': '???', (TaskRunner pid=2403642) 'prompt_length': 2048, (TaskRunner pid=2403642) 'response_length': 2048, (TaskRunner pid=2403642) 'skip_tokenizer_init': False, (TaskRunner pid=2403642) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2403642) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2403642) 'memory_limit_mb': 1024, (TaskRunner pid=2403642) 'url': None}, (TaskRunner pid=2403642) 'strategy': 'fsdp', (TaskRunner pid=2403642) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2403642) 'use_dynamic_bsz': False, (TaskRunner pid=2403642) 'use_reward_loop': False}, (TaskRunner pid=2403642) 'trainer': {'balance_batch': True, (TaskRunner pid=2403642) 'critic_warmup': 0, (TaskRunner pid=2403642) 'default_hdfs_dir': None, (TaskRunner pid=2403642) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2403642) 'del_local_ckpt_after_load': False, (TaskRunner pid=2403642) 'device': 'cuda', (TaskRunner pid=2403642) 'esi_redundant_time': 0, (TaskRunner pid=2403642) 'experiment_name': 'qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2403642) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2403642) 'log_val_generations': 0, (TaskRunner pid=2403642) 'logger': ['console', 'wandb'], (TaskRunner pid=2403642) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2403642) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2403642) 'n_gpus_per_node': 8, (TaskRunner pid=2403642) 'nnodes': 1, (TaskRunner pid=2403642) 'project_name': 'SDPO-root', (TaskRunner pid=2403642) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2403642) 'resume_from_path': None, (TaskRunner pid=2403642) 'resume_mode': 'auto', (TaskRunner pid=2403642) 'rollout_data_dir': None, (TaskRunner pid=2403642) 'save_freq': 10, (TaskRunner pid=2403642) 'test_freq': 10, (TaskRunner pid=2403642) 'total_epochs': 30, (TaskRunner pid=2403642) 'total_training_steps': 100, (TaskRunner pid=2403642) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2403642) 'val_before_train': False, (TaskRunner pid=2403642) 'val_only': False, (TaskRunner pid=2403642) 'validation_data_dir': None}, (TaskRunner pid=2403642) 'transfer_queue': {'enable': False}, (TaskRunner pid=2403642) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/material', (TaskRunner pid=2403642) 'dir': '/users/root/SDPO', (TaskRunner pid=2403642) 'log_dir': '/users/root/output', (TaskRunner pid=2403642) 'task': 'datasets/sciknoweval/material'}} (TaskRunner pid=2403642) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2403642) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2403642) use_critic=need_critic(config), (TaskRunner pid=2403642) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2403642) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2403642) Using dataset class: RLHFDataset (TaskRunner pid=2403642) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2403642) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2403642) dataset len: 841 (TaskRunner pid=2403642) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2403642) WARNING:2026-07-03 10:50:02,889:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2403642) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/841 [00:00 (TaskRunner pid=2403642) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2403642) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2403642) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2403642) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2403642) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2403642) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 94/94 [00:00<00:00, 136.10 examples/s] (TaskRunner pid=2403642) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2403642) self.use_critic = need_critic(self.config) (WorkerDict pid=2404022) [W703 10:50:12.420427916 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:43061 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2404021) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2404026) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2404026) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2404027) [W703 10:50:12.893489239 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:43061 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2404021) Model config after override: Qwen3Config { (WorkerDict pid=2404021) "architectures": [ (WorkerDict pid=2404021) "Qwen3ForCausalLM" (WorkerDict pid=2404021) ], (WorkerDict pid=2404021) "attention_bias": false, (WorkerDict pid=2404021) "attention_dropout": 0.0, (WorkerDict pid=2404021) "dtype": "bfloat16", (WorkerDict pid=2404021) "eos_token_id": 151645, (WorkerDict pid=2404021) "head_dim": 128, (WorkerDict pid=2404021) "hidden_act": "silu", (WorkerDict pid=2404021) "hidden_size": 4096, (WorkerDict pid=2404021) "initializer_range": 0.02, (WorkerDict pid=2404021) "intermediate_size": 12288, (WorkerDict pid=2404021) "layer_types": [ (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention", (WorkerDict pid=2404021) "full_attention" (WorkerDict pid=2404021) ], (WorkerDict pid=2404021) "max_position_embeddings": 40960, (WorkerDict pid=2404021) "max_window_layers": 36, (WorkerDict pid=2404021) "model_type": "qwen3", (WorkerDict pid=2404021) "num_attention_heads": 32, (WorkerDict pid=2404021) "num_hidden_layers": 36, (WorkerDict pid=2404021) "num_key_value_heads": 8, (WorkerDict pid=2404021) "pad_token_id": 151643, (WorkerDict pid=2404021) "rms_norm_eps": 1e-06, (WorkerDict pid=2404021) "rope_scaling": null, (WorkerDict pid=2404021) "rope_theta": 1000000, (WorkerDict pid=2404021) "sliding_window": null, (WorkerDict pid=2404021) "tie_word_embeddings": false, (WorkerDict pid=2404021) "transformers_version": "4.57.1", (WorkerDict pid=2404021) "use_cache": true, (WorkerDict pid=2404021) "use_sliding_window": false, (WorkerDict pid=2404021) "vocab_size": 151936 (WorkerDict pid=2404021) } (WorkerDict pid=2404021) (WorkerDict pid=2404026) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2404026) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2404026) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2404021) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2404021) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2404022) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2404022) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2404027) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.90s/it] [repeated 15x across cluster] (WorkerDict pid=2404027) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.20s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.60s/it] [repeated 7x across cluster] (WorkerDict pid=2404021) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2404021) Actor use_remove_padding=True (WorkerDict pid=2404021) Actor use_fused_kernels=False (WorkerDict pid=2404021) Actor use_prefix_grouper=False (WorkerDict pid=2404023) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2404023) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2404027) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2404027) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2404022) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2404021) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2404021) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2404021) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2404021) warnings.warn( (WorkerDict pid=2404021) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2404021) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2403642) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2403642) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2404028) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2404028) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2404910) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2404910) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2404908) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2404908) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2404908) ['serve', (vLLMHttpServer pid=2404908) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2404908) '--dtype', (vLLMHttpServer pid=2404908) 'bfloat16', (vLLMHttpServer pid=2404908) '--load_format', (vLLMHttpServer pid=2404908) 'dummy', (vLLMHttpServer pid=2404908) '--trust_remote_code', (vLLMHttpServer pid=2404908) '--max_model_len', (vLLMHttpServer pid=2404908) '40960', (vLLMHttpServer pid=2404908) '--max_num_seqs', (vLLMHttpServer pid=2404908) '1024', (vLLMHttpServer pid=2404908) '--enable_chunked_prefill', (vLLMHttpServer pid=2404908) '--max_num_batched_tokens', (vLLMHttpServer pid=2404908) '10240', (vLLMHttpServer pid=2404908) '--enable_prefix_caching', (vLLMHttpServer pid=2404908) '--enable_sleep_mode', (vLLMHttpServer pid=2404908) '--logprobs_mode', (vLLMHttpServer pid=2404908) 'processed_logprobs', (vLLMHttpServer pid=2404908) '--disable_custom_all_reduce', (vLLMHttpServer pid=2404908) '--gpu_memory_utilization', (vLLMHttpServer pid=2404908) '0.8', (vLLMHttpServer pid=2404908) '--disable_log_stats', (vLLMHttpServer pid=2404908) '--tensor_parallel_size', (vLLMHttpServer pid=2404908) '2', (vLLMHttpServer pid=2404908) '--seed', (vLLMHttpServer pid=2404908) '0', (vLLMHttpServer pid=2404908) '--override_generation_config', (vLLMHttpServer pid=2404908) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2404908) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2404908) '--hf_overrides', (vLLMHttpServer pid=2404908) '{}', (vLLMHttpServer pid=2404908) '--scheduling_policy', (vLLMHttpServer pid=2404908) 'fcfs'] (WorkerDict pid=2404028) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2404909) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2404910) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2404908) WARNING 07-03 10:51:03 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2404023) WARNING 07-03 10:51:11 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2404911) WARNING 07-03 10:51:03 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2404025) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2404022) [rank1]:W0703 10:51:22.715000 2404022 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2404022) [rank1]:W0703 10:51:22.715000 2404022 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2404022) [rank1]:W0703 10:51:22.715000 2404022 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2404022) [rank1]:W0703 10:51:22.715000 2404022 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2404022) [rank1]:W0703 10:51:22.715000 2404022 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2404022) [rank1]:W0703 10:51:22.715000 2404022 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/aj/faj75csb7ofozv5in77ssqc762na7cdfwj3wkvfkkmz6awioro7y/.2404026.129108552971968.tmp' (vLLMHttpServer pid=2404911) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2404911) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2404911) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2404911) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2404025) 2026-07-03 10:51:25,589 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2404025) 2026-07-03 10:51:25,605 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2404021) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2403642) (TaskRunner pid=2403642) A (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) user (TaskRunner pid=2403642) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2403642) (TaskRunner pid=2403642) Lattice Parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) a = 7.82 Å (TaskRunner pid=2403642) b = 7.82 Å (TaskRunner pid=2403642) c = 17.88 Å (TaskRunner pid=2403642) α = 90.00º (TaskRunner pid=2403642) β = 90.00º (TaskRunner pid=2403642) γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) A: 1025.67 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) B: 1094.31 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) C: 1200.45 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) D: 1150.78 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) Please reason step by step. (TaskRunner pid=2403642) assistant (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) [response] (TaskRunner pid=2403642) The given crystal structure is orthorhombic, as all the angles (α, β, γ) are 90.00º. The volume $ V $ of a unit cell in an orthorhombic system is calculated using the formula: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) V = a \times b \times c (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Where: (TaskRunner pid=2403642) - $ a = 7.82 $ Å (TaskRunner pid=2403642) - $ b = 7.82 $ Å (TaskRunner pid=2403642) - $ c = 17.88 $ Å (TaskRunner pid=2403642) (TaskRunner pid=2403642) Substituting the values: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) V = 7.82 \times 7.82 \times 17.88 (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Now multiply by $ 17.88 $: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) 61.1524 \times 17.88 = 1094.31 (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Thus, the volume of the unit cell is $ 1094.31 $ ų. (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) B (TaskRunner pid=2403642) (TaskRunner pid=2403642) [ground_truth] B (TaskRunner pid=2403642) [score] 1.0 (TaskRunner pid=2403642) [acc] 1.0 (TaskRunner pid=2403642) [pred] B (TaskRunner pid=2403642) [incorrect_format] 1 (TaskRunner pid=2403642) [feedback] (TaskRunner pid=2403642) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2403642) step:10 - global_seqlen/min:21528 - global_seqlen/max:30348 - global_seqlen/minmax_diff:8820 - global_seqlen/balanced_min:25597 - global_seqlen/balanced_max:25600 - global_seqlen/mean:25598.625 - actor/entropy:0.16920506954193115 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6033204197883606 - training/rollout_probs_diff_mean:0.0029068614821881056 - training/rollout_probs_diff_std:0.009937559254467487 - training/rollout_actor_probs_pearson_corr:0.9988886713981628 - rollout_corr/rollout_is_mean:0.999866783618927 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9597660866566002e-05 - rollout_corr/rollout_is_max:1.6955915689468384 - rollout_corr/rollout_is_min:0.11169125884771347 - rollout_corr/rollout_is_std:0.028244443237781525 - rollout_corr/rollout_is_eff_sample_size:0.9992025302719866 - rollout_corr/rollout_is_seq_mean:0.9998682737350464 - rollout_corr/rollout_is_seq_std:0.0015377062372863293 - rollout_corr/rollout_is_seq_max:1.004624605178833 - rollout_corr/rollout_is_seq_min:0.994768500328064 - rollout_corr/rollout_is_seq_max_deviation:0.005231499671936035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1872184653766453) - rollout_corr/training_log_ppl:np.float64(0.16784915810421808) - rollout_corr/kl:np.float64(0.000722991060116307) - rollout_corr/k3_kl:np.float64(0.0005227115022989892) - rollout_corr/rollout_ppl:np.float64(1.1863347007893026) - rollout_corr/rollout_log_ppl:np.float64(0.16712616474978859) - rollout_corr/log_ppl_diff:np.float64(0.0007229933544294909) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014674967678729445) - rollout_corr/log_ppl_diff_max:np.float64(0.006441354751586914) - rollout_corr/log_ppl_diff_min:np.float64(-0.003609776496887207) - rollout_corr/ppl_ratio:np.float64(1.0007247407920659) - rollout_corr/chi2_token:np.float64(0.0006014511454850435) - rollout_corr/chi2_seq:np.float64(0.217362450202927) - actor/pg_loss:np.float64(-0.00012342154514044523) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00042188452925984166) - actor/ppo_kl:np.float64(0.0001569013978262035) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.241903368383646) - perf/mfu/actor:np.float64(0.10273077133253324) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(105.04884338378906) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6795212765957447) - val-aux/sciknoweval/reward/std@16:np.float64(0.12984114398296573) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7279042553191489) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.08743747296198659) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6308191489361703) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.12907308985602772) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6793829787234041) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.13009488275314807) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7579255319148935) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.05743236777991458) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.580531914893617) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12283448124701997) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.697404255319149) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.09072076266889331) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7801914893617022) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.039544738199845254) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.526968085106383) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10635135878078301) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7053510638297872) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.05191007682205249) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7965425531914894) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.024488405943791508) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.481127659574468) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07748643019338206) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7081489361702129) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.026119348743445903) - val-aux/sciknoweval/score/mean@16:np.float64(0.6795212765957447) - val-aux/sciknoweval/score/std@16:np.float64(0.12984114398296573) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7279042553191489) - val-aux/sciknoweval/score/best@2/std:np.float64(0.08743747296198659) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6308191489361703) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.12907308985602772) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6793829787234041) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.13009488275314807) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7579255319148935) - val-aux/sciknoweval/score/best@4/std:np.float64(0.05743236777991458) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.580531914893617) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12283448124701997) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.697404255319149) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.09072076266889331) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7801914893617022) - val-aux/sciknoweval/score/best@8/std:np.float64(0.039544738199845254) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.526968085106383) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10635135878078301) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7053510638297872) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.05191007682205249) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7965425531914894) - val-aux/sciknoweval/score/best@16/std:np.float64(0.024488405943791508) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.481127659574468) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07748643019338206) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7081489361702129) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.026119348743445903) - val-core/sciknoweval/acc/mean@16:np.float64(0.6795212765957447) - val-aux/sciknoweval/acc/std@16:np.float64(0.12984114398296573) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7279042553191489) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.08743747296198659) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6308191489361703) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.12907308985602772) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6793829787234041) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.13009488275314807) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7579255319148935) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.05743236777991458) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.580531914893617) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12283448124701997) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.697404255319149) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.09072076266889331) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7801914893617022) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.039544738199845254) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.526968085106383) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10635135878078301) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7053510638297872) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.05191007682205249) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7965425531914894) - val-core/sciknoweval/acc/best@16/std:np.float64(0.024488405943791508) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.481127659574468) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07748643019338206) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7081489361702129) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.026119348743445903) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9993351063829787) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0025751219057230166) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999787234042554) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.00047528314688699314) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9987872340425531) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0033809709015623617) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.999436170212766) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0023833348897965814) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9976382978723404) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.004421182396145962) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9998723404255319) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0011583538373460597) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9958297872340426) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0051935817199633555) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9933510638297872) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.005150243811446033) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005119888577610254 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005119888577610254 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.64453125 - response_length/max:850.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:398.64453125 - response_length_non_aborted/max:850.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.3125 - prompt_length/max:483.0 - prompt_length/min:267.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013617239892482758 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5323186237365007) - timing_s/agent_loop/generate_sequences/max:np.float64(7.833362892270088) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.64455915162398) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.833362892270088) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:850 - timing_s/gen:14.731748191639781 - timing_s/reward:0.06321951374411583 - timing_s/old_log_prob:3.0473394505679607 - timing_s/adv:0.1209035012871027 - timing_s/update_actor:13.126259731128812 - timing_s/step:31.175520030781627 - timing_s/testing:95.16854247637093 - timing_s/save_checkpoint:12.76433370821178 - timing_s/stop_profile:0.0001818723976612091 - timing_per_token_ms/gen:0.14435389642283697 - timing_per_token_ms/adv:0.0005903808372866838 - timing_per_token_ms/update_actor:0.06409650777692558 - perf/total_num_tokens:204789 - perf/time_per_step:31.175520030781627 - perf/throughput:821.1130070877665 (TaskRunner pid=2403642) Training Progress: 10%|█ | 10/100 [07:14<1:36:26, 64.29s/it] (TaskRunner pid=2403642) step:11 - global_seqlen/min:23579 - global_seqlen/max:28112 - global_seqlen/minmax_diff:4533 - global_seqlen/balanced_min:26452 - global_seqlen/balanced_max:26455 - global_seqlen/mean:26453.625 - actor/entropy:0.1441550999879837 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5545997023582458 - training/rollout_probs_diff_mean:0.0024969361256808043 - training/rollout_probs_diff_std:0.009402581490576267 - training/rollout_actor_probs_pearson_corr:0.9987636208534241 - rollout_corr/rollout_is_mean:1.0001752376556396 - rollout_corr/rollout_is_ratio_fraction_high:8.951189556682948e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.6853567760554142e-05 - rollout_corr/rollout_is_max:3.7230892181396484 - rollout_corr/rollout_is_min:0.25212857127189636 - rollout_corr/rollout_is_std:0.02670188620686531 - rollout_corr/rollout_is_eff_sample_size:0.9992878743513974 - rollout_corr/rollout_is_seq_mean:1.0002554655075073 - rollout_corr/rollout_is_seq_std:0.0014197564451023936 - rollout_corr/rollout_is_seq_max:1.004997730255127 - rollout_corr/rollout_is_seq_min:0.9965838193893433 - rollout_corr/rollout_is_seq_max_deviation:0.004997730255126953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1556855565868318) - rollout_corr/training_log_ppl:np.float64(0.14192374088452198) - rollout_corr/kl:np.float64(0.0001279704757504163) - rollout_corr/k3_kl:np.float64(0.0004536967828556726) - rollout_corr/rollout_ppl:np.float64(1.1555262096226215) - rollout_corr/rollout_log_ppl:np.float64(0.14179576889728196) - rollout_corr/log_ppl_diff:np.float64(0.00012797198724001646) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011966362362727523) - rollout_corr/log_ppl_diff_max:np.float64(0.005566880106925964) - rollout_corr/log_ppl_diff_min:np.float64(-0.005283966660499573) - rollout_corr/ppl_ratio:np.float64(1.0001292265951633) - rollout_corr/chi2_token:np.float64(0.0016099370550364256) - rollout_corr/chi2_seq:np.float64(0.5782006285153329) - actor/pg_loss:np.float64(5.143752787262201e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019089993247689563) - actor/ppo_kl:np.float64(5.072445345533083e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12007458135485649) - perf/mfu/actor:np.float64(0.10334472437206974) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(108.11588668823242) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003309702267870307 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003309702267870307 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:436.39453125 - response_length/max:721.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:436.39453125 - response_length_non_aborted/max:721.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.28125 - prompt_length/max:481.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001160837709903717 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0758140552788973) - timing_s/agent_loop/generate_sequences/max:np.float64(7.61514799669385) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.149380202659813) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.61514799669385) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:721 - timing_s/gen:14.069710811600089 - timing_s/reward:0.06073213554918766 - timing_s/old_log_prob:3.2710697315633297 - timing_s/adv:0.1479395367205143 - timing_s/update_actor:13.371990287676454 - timing_s/step:31.018236611038446 - timing_s/stop_profile:4.745647311210632e-05 - timing_per_token_ms/gen:0.12594064297824045 - timing_per_token_ms/adv:0.0006990513432493387 - timing_per_token_ms/update_actor:0.06318600138769476 - perf/total_num_tokens:211629 - perf/time_per_step:31.018236611038446 - perf/throughput:852.8410345089044 (TaskRunner pid=2403642) Training Progress: 11%|█ | 11/100 [07:45<1:20:16, 54.11s/it] (TaskRunner pid=2403642) step:12 - global_seqlen/min:22402 - global_seqlen/max:29419 - global_seqlen/minmax_diff:7017 - global_seqlen/balanced_min:26481 - global_seqlen/balanced_max:26487 - global_seqlen/mean:26485.25 - actor/entropy:0.1846352517604828 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3155989646911621 - training/rollout_probs_diff_mean:0.0028140994254499674 - training/rollout_probs_diff_std:0.009286953136324883 - training/rollout_actor_probs_pearson_corr:0.9989963173866272 - rollout_corr/rollout_is_mean:1.0000858306884766 - rollout_corr/rollout_is_ratio_fraction_high:8.870752935763448e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.5524312257766724 - rollout_corr/rollout_is_std:0.027619199827313423 - rollout_corr/rollout_is_eff_sample_size:0.9992378802349873 - rollout_corr/rollout_is_seq_mean:1.0001217126846313 - rollout_corr/rollout_is_seq_std:0.0015678335912525654 - rollout_corr/rollout_is_seq_max:1.0049930810928345 - rollout_corr/rollout_is_seq_min:0.9935046434402466 - rollout_corr/rollout_is_seq_max_deviation:0.006495356559753418 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.197012455202639) - rollout_corr/training_log_ppl:np.float64(0.1768503768616938) - rollout_corr/kl:np.float64(0.0005232393070224717) - rollout_corr/k3_kl:np.float64(0.0008367889608820178) - rollout_corr/rollout_ppl:np.float64(1.1963813910260797) - rollout_corr/rollout_log_ppl:np.float64(0.17632713728380622) - rollout_corr/log_ppl_diff:np.float64(0.0005232395778875798) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015441769937751815) - rollout_corr/log_ppl_diff_max:np.float64(0.03289683163166046) - rollout_corr/log_ppl_diff_min:np.float64(-0.020482316613197327) - rollout_corr/ppl_ratio:np.float64(1.000529296696186) - rollout_corr/chi2_token:np.float64(0.0047122761607170105) - rollout_corr/chi2_seq:np.float64(5.705244723241776) - actor/pg_loss:np.float64(2.2202730178833008e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040041032389126485) - actor/ppo_kl:np.float64(0.0002610394977891062) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2557632587850094) - perf/mfu/actor:np.float64(0.10529765568035229) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.64424991607666) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005666193319484591 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005666193319484591 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.3515625 - response_length/max:797.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.3515625 - response_length_non_aborted/max:797.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:387.3125 - prompt_length/max:482.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.065253496170044e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6090983245521784) - timing_s/agent_loop/generate_sequences/max:np.float64(7.733725985512137) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.158535132286488) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.733725985512137) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:786 - timing_s/gen:14.906016860157251 - timing_s/reward:0.06484229490160942 - timing_s/old_log_prob:3.085481446236372 - timing_s/adv:0.11875537969172001 - timing_s/update_actor:12.98678758367896 - timing_s/step:31.24809272401035 - timing_s/stop_profile:3.450550138950348e-05 - timing_per_token_ms/gen:0.13222759567246742 - timing_per_token_ms/adv:0.000560478849981216 - timing_per_token_ms/update_actor:0.061292547661806855 - perf/total_num_tokens:211882 - perf/time_per_step:31.24809272401035 - perf/throughput:847.5797301909987 (TaskRunner pid=2403642) Training Progress: 12%|█▏ | 12/100 [08:17<1:09:10, 47.16s/it] (TaskRunner pid=2403642) step:13 - global_seqlen/min:25377 - global_seqlen/max:30122 - global_seqlen/minmax_diff:4745 - global_seqlen/balanced_min:27079 - global_seqlen/balanced_max:27081 - global_seqlen/mean:27079.625 - actor/entropy:0.1729048788547516 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2444952130317688 - training/rollout_probs_diff_mean:0.0027501848526299 - training/rollout_probs_diff_std:0.009192431345582008 - training/rollout_actor_probs_pearson_corr:0.9989957809448242 - rollout_corr/rollout_is_mean:0.9998618960380554 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6526812314987183 - rollout_corr/rollout_is_min:0.5106894969940186 - rollout_corr/rollout_is_std:0.027012590318918228 - rollout_corr/rollout_is_eff_sample_size:0.9992704948770252 - rollout_corr/rollout_is_seq_mean:0.9998829364776611 - rollout_corr/rollout_is_seq_std:0.0015230097342282534 - rollout_corr/rollout_is_seq_max:1.006705641746521 - rollout_corr/rollout_is_seq_min:0.9953020215034485 - rollout_corr/rollout_is_seq_max_deviation:0.006705641746520996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.188228433020413) - rollout_corr/training_log_ppl:np.float64(0.1695324133397662) - rollout_corr/kl:np.float64(0.0005957026775860186) - rollout_corr/k3_kl:np.float64(0.00043430524907250856) - rollout_corr/rollout_ppl:np.float64(1.1875129132531583) - rollout_corr/rollout_log_ppl:np.float64(0.1689367104190751) - rollout_corr/log_ppl_diff:np.float64(0.0005957029206911102) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012361184635665268) - rollout_corr/log_ppl_diff_max:np.float64(0.005819700658321381) - rollout_corr/log_ppl_diff_min:np.float64(-0.003224298357963562) - rollout_corr/ppl_ratio:np.float64(1.000597006175667) - rollout_corr/chi2_token:np.float64(0.0005501273553818464) - rollout_corr/chi2_seq:np.float64(0.09272657055407763) - actor/pg_loss:np.float64(-7.470371201634407e-07) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014200475698089576) - actor/ppo_kl:np.float64(8.690033108127437e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13939961977303028) - perf/mfu/actor:np.float64(0.10884410844931278) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.16784048080444) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007487679249607027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007487679249607027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:430.39453125 - response_length/max:683.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:430.39453125 - response_length_non_aborted/max:683.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:415.84375 - prompt_length/max:493.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.849474549293518e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5356736686080694) - timing_s/agent_loop/generate_sequences/max:np.float64(7.184362847357988) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.028661246171396) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.184362847357988) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:488 - timing_s/agent_loop/slowest/response_length:665 - timing_s/gen:13.875789875164628 - timing_s/reward:0.08754915185272694 - timing_s/old_log_prob:3.1806381102651358 - timing_s/adv:0.14010637439787388 - timing_s/update_actor:12.896027885377407 - timing_s/step:30.271038146689534 - timing_s/stop_profile:2.8403475880622864e-05 - timing_per_token_ms/gen:0.1259363218264912 - timing_per_token_ms/adv:0.0006467333576345402 - timing_per_token_ms/update_actor:0.05952827949693454 - perf/total_num_tokens:216637 - perf/time_per_step:30.271038146689534 - perf/throughput:894.5720615452843 (TaskRunner pid=2403642) Training Progress: 13%|█▎ | 13/100 [08:47<1:00:58, 42.05s/it] (TaskRunner pid=2403642) step:14 - global_seqlen/min:20977 - global_seqlen/max:29238 - global_seqlen/minmax_diff:8261 - global_seqlen/balanced_min:25164 - global_seqlen/balanced_max:25170 - global_seqlen/mean:25167.5 - actor/entropy:0.16507954895496368 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977966308594 - training/rollout_probs_diff_mean:0.0026576712261885405 - training/rollout_probs_diff_std:0.00907220970839262 - training/rollout_actor_probs_pearson_corr:0.9990447759628296 - rollout_corr/rollout_is_mean:1.000095248222351 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.685792974778451e-06 - rollout_corr/rollout_is_max:1.6395798921585083 - rollout_corr/rollout_is_min:0.4166649878025055 - rollout_corr/rollout_is_std:0.0269794724881649 - rollout_corr/rollout_is_eff_sample_size:0.9992728755912055 - rollout_corr/rollout_is_seq_mean:1.0000946521759033 - rollout_corr/rollout_is_seq_std:0.0014211699599400163 - rollout_corr/rollout_is_seq_max:1.0052399635314941 - rollout_corr/rollout_is_seq_min:0.995132327079773 - rollout_corr/rollout_is_seq_max_deviation:0.005239963531494141 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1816609147936106) - rollout_corr/training_log_ppl:np.float64(0.1643792792747263) - rollout_corr/kl:np.float64(0.0004290493978587051) - rollout_corr/k3_kl:np.float64(0.0004259060699780548) - rollout_corr/rollout_ppl:np.float64(1.1811475306749344) - rollout_corr/rollout_log_ppl:np.float64(0.16395022800861625) - rollout_corr/log_ppl_diff:np.float64(0.0004290512661100365) - rollout_corr/log_ppl_abs_diff:np.float64(0.001253184695087839) - rollout_corr/log_ppl_diff_max:np.float64(0.006546080112457275) - rollout_corr/log_ppl_diff_min:np.float64(-0.004463165998458862) - rollout_corr/ppl_ratio:np.float64(1.0004304558970034) - rollout_corr/chi2_token:np.float64(0.0008710860274732113) - rollout_corr/chi2_seq:np.float64(0.33716148510575294) - actor/pg_loss:np.float64(-3.031059168279171e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016462049688925617) - actor/ppo_kl:np.float64(0.00014458334416866947) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12229474447667599) - perf/mfu/actor:np.float64(0.10208644290646234) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.20569229125977) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0031842044554650784 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0031842044554650784 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:403.296875 - response_length/max:936.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:403.296875 - response_length_non_aborted/max:936.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.1875 - prompt_length/max:519.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5698915123939514e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1408378798514605) - timing_s/agent_loop/generate_sequences/max:np.float64(8.30212122388184) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7140189427445875) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.30212122388184) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:936 - timing_s/gen:15.170544736087322 - timing_s/reward:0.06094013340771198 - timing_s/old_log_prob:3.1517421398311853 - timing_s/adv:0.12233048304915428 - timing_s/update_actor:12.77594661898911 - timing_s/step:31.38437505811453 - timing_s/stop_profile:6.346404552459717e-05 - timing_per_token_ms/gen:0.14693875417542251 - timing_per_token_ms/adv:0.0006075816184024748 - timing_per_token_ms/update_actor:0.063454587359636 - perf/total_num_tokens:201340 - perf/time_per_step:31.38437505811453 - perf/throughput:801.911777863898 (TaskRunner pid=2403642) Training Progress: 14%|█▍ | 14/100 [09:18<55:40, 38.84s/it] (TaskRunner pid=2403642) step:15 - global_seqlen/min:22220 - global_seqlen/max:28895 - global_seqlen/minmax_diff:6675 - global_seqlen/balanced_min:26398 - global_seqlen/balanced_max:26402 - global_seqlen/mean:26400.5 - actor/entropy:0.15334376692771912 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31862038373947144 - training/rollout_probs_diff_mean:0.002467168727889657 - training/rollout_probs_diff_std:0.008850067853927612 - training/rollout_actor_probs_pearson_corr:0.99895840883255 - rollout_corr/rollout_is_mean:0.999837338924408 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8703776597976685 - rollout_corr/rollout_is_min:0.5231994986534119 - rollout_corr/rollout_is_std:0.025291837751865387 - rollout_corr/rollout_is_eff_sample_size:0.9993606128205051 - rollout_corr/rollout_is_seq_mean:0.9998326897621155 - rollout_corr/rollout_is_seq_std:0.0014355891617015004 - rollout_corr/rollout_is_seq_max:1.0044710636138916 - rollout_corr/rollout_is_seq_min:0.9946416020393372 - rollout_corr/rollout_is_seq_max_deviation:0.005358397960662842 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1715559563599527) - rollout_corr/training_log_ppl:np.float64(0.15571669759083306) - rollout_corr/kl:np.float64(0.0006178469196100878) - rollout_corr/k3_kl:np.float64(0.00039293118835814766) - rollout_corr/rollout_ppl:np.float64(1.1708267177455127) - rollout_corr/rollout_log_ppl:np.float64(0.1550988501185202) - rollout_corr/log_ppl_diff:np.float64(0.0006178474723128602) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012598117464222014) - rollout_corr/log_ppl_diff_max:np.float64(0.006419658660888672) - rollout_corr/log_ppl_diff_min:np.float64(-0.004078000783920288) - rollout_corr/ppl_ratio:np.float64(1.0006193004082888) - rollout_corr/chi2_token:np.float64(0.00034064752981066704) - rollout_corr/chi2_seq:np.float64(0.33131241356022656) - actor/pg_loss:np.float64(7.49519094824791e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001743871712278633) - actor/ppo_kl:np.float64(9.903572419744933e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2359774075448513) - perf/mfu/actor:np.float64(0.10741188729449186) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.21630764007568) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0016187583096325397 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0016187583096325397 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:435.265625 - response_length/max:741.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:435.265625 - response_length_non_aborted/max:741.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:389.75 - prompt_length/max:512.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.758872091770172e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.033079355955124) - timing_s/agent_loop/generate_sequences/max:np.float64(7.490394292399287) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.156952525649103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.490394292399287) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:722 - timing_s/gen:14.472805431112647 - timing_s/reward:0.0683264397084713 - timing_s/old_log_prob:3.1418946012854576 - timing_s/adv:0.11782016232609749 - timing_s/update_actor:12.759607169777155 - timing_s/step:30.647193098440766 - timing_s/stop_profile:0.00011414289474487305 - timing_per_token_ms/gen:0.12988481738084365 - timing_per_token_ms/adv:0.0005578500517324364 - timing_per_token_ms/update_actor:0.060413662476928254 - perf/total_num_tokens:211204 - perf/time_per_step:30.647193098440766 - perf/throughput:861.4328860460365 (TaskRunner pid=2403642) Training Progress: 15%|█▌ | 15/100 [09:49<51:32, 36.38s/it] (TaskRunner pid=2403642) step:16 - global_seqlen/min:23242 - global_seqlen/max:29683 - global_seqlen/minmax_diff:6441 - global_seqlen/balanced_min:26899 - global_seqlen/balanced_max:26903 - global_seqlen/mean:26901.125 - actor/entropy:0.17101728916168213 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31862038373947144 - training/rollout_probs_diff_mean:0.0025425369385629892 - training/rollout_probs_diff_std:0.008662575855851173 - training/rollout_actor_probs_pearson_corr:0.999067485332489 - rollout_corr/rollout_is_mean:1.0001250505447388 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.979804078990128e-06 - rollout_corr/rollout_is_max:1.8446787595748901 - rollout_corr/rollout_is_min:0.27226293087005615 - rollout_corr/rollout_is_std:0.02593412436544895 - rollout_corr/rollout_is_eff_sample_size:0.9993281113448161 - rollout_corr/rollout_is_seq_mean:1.0001810789108276 - rollout_corr/rollout_is_seq_std:0.0015763085102662444 - rollout_corr/rollout_is_seq_max:1.0065312385559082 - rollout_corr/rollout_is_seq_min:0.9940209984779358 - rollout_corr/rollout_is_seq_max_deviation:0.006531238555908203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.193248980678618) - rollout_corr/training_log_ppl:np.float64(0.17239582387992414) - rollout_corr/kl:np.float64(0.00033440753456126515) - rollout_corr/k3_kl:np.float64(0.00037891795724220856) - rollout_corr/rollout_ppl:np.float64(1.1928353495895863) - rollout_corr/rollout_log_ppl:np.float64(0.17206141452334123) - rollout_corr/log_ppl_diff:np.float64(0.0003344093565829098) - rollout_corr/log_ppl_abs_diff:np.float64(0.001199286969495006) - rollout_corr/log_ppl_diff_max:np.float64(0.009442910552024841) - rollout_corr/log_ppl_diff_min:np.float64(-0.005542337894439697) - rollout_corr/ppl_ratio:np.float64(1.0003358356188983) - rollout_corr/chi2_token:np.float64(0.0008554125670343637) - rollout_corr/chi2_seq:np.float64(0.4001388291362673) - actor/pg_loss:np.float64(-1.7592916265130043e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.427388002019143e-05) - actor/ppo_kl:np.float64(0.0001517259071359689) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1951090656220913) - perf/mfu/actor:np.float64(0.10736358975617949) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.19661712646484) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002055252669379115 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.002055252669379115 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:435.00390625 - response_length/max:835.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:435.00390625 - response_length_non_aborted/max:835.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:405.65625 - prompt_length/max:495.0 - prompt_length/min:268.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.904615581035614e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1833369322121143) - timing_s/agent_loop/generate_sequences/max:np.float64(7.8781713005155325) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.695604027634545) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.8781713005155325) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:835 - timing_s/gen:15.18984092772007 - timing_s/reward:0.05374291352927685 - timing_s/old_log_prob:3.123084019869566 - timing_s/adv:0.1425618678331375 - timing_s/update_actor:12.955157721415162 - timing_s/step:31.49756983295083 - timing_s/stop_profile:3.855116665363312e-05 - timing_per_token_ms/gen:0.13640180069970698 - timing_per_token_ms/adv:0.0006624345070751572 - timing_per_token_ms/update_actor:0.06019802945701696 - perf/total_num_tokens:215209 - perf/time_per_step:31.49756983295083 - perf/throughput:854.0698581722863 (TaskRunner pid=2403642) Training Progress: 16%|█▌ | 16/100 [10:20<48:53, 34.92s/it] (TaskRunner pid=2403642) step:17 - global_seqlen/min:22126 - global_seqlen/max:30070 - global_seqlen/minmax_diff:7944 - global_seqlen/balanced_min:25601 - global_seqlen/balanced_max:25610 - global_seqlen/mean:25606.375 - actor/entropy:0.16802534461021423 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4241411089897156 - training/rollout_probs_diff_mean:0.0025332875084131956 - training/rollout_probs_diff_std:0.008655770681798458 - training/rollout_actor_probs_pearson_corr:0.9990943670272827 - rollout_corr/rollout_is_mean:0.9999657869338989 - rollout_corr/rollout_is_ratio_fraction_high:9.704686817713082e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.704686817713082e-06 - rollout_corr/rollout_is_max:2.0695576667785645 - rollout_corr/rollout_is_min:0.48783329129219055 - rollout_corr/rollout_is_std:0.025280050933361053 - rollout_corr/rollout_is_eff_sample_size:0.9993612081053422 - rollout_corr/rollout_is_seq_mean:0.9999219179153442 - rollout_corr/rollout_is_seq_std:0.0015641668578609824 - rollout_corr/rollout_is_seq_max:1.0051617622375488 - rollout_corr/rollout_is_seq_min:0.9921011924743652 - rollout_corr/rollout_is_seq_max_deviation:0.007898807525634766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.188482933677733) - rollout_corr/training_log_ppl:np.float64(0.16806937995715998) - rollout_corr/kl:np.float64(0.0004929015229606648) - rollout_corr/k3_kl:np.float64(0.00041387354062649706) - rollout_corr/rollout_ppl:np.float64(1.187876682728529) - rollout_corr/rollout_log_ppl:np.float64(0.16757647686608834) - rollout_corr/log_ppl_diff:np.float64(0.0004929030910716392) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013073813388473354) - rollout_corr/log_ppl_diff_max:np.float64(0.008626759052276611) - rollout_corr/log_ppl_diff_min:np.float64(-0.0042053088545799255) - rollout_corr/ppl_ratio:np.float64(1.0004944938700646) - rollout_corr/chi2_token:np.float64(0.0006821004208177328) - rollout_corr/chi2_seq:np.float64(0.3241713694296777) - actor/pg_loss:np.float64(9.176251478493214e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010633002148097148) - actor/ppo_kl:np.float64(5.967334392664725e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16828186437487602) - perf/mfu/actor:np.float64(0.10254237405965684) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.18576049804688) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036283396184444427 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036283396184444427 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:402.51171875 - response_length/max:755.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:402.51171875 - response_length_non_aborted/max:755.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.6875 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010741129517555237 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4098624978214502) - timing_s/agent_loop/generate_sequences/max:np.float64(7.328284703195095) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.6529731439222815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.328284703195095) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:755 - timing_s/gen:14.528358956798911 - timing_s/reward:0.06316086649894714 - timing_s/old_log_prob:3.1723644994199276 - timing_s/adv:0.12884124740958214 - timing_s/update_actor:13.108599437400699 - timing_s/step:31.074970627203584 - timing_s/stop_profile:0.0001179128885269165 - timing_per_token_ms/gen:0.14099316748152627 - timing_per_token_ms/adv:0.0006289510298196355 - timing_per_token_ms/update_actor:0.06399089795705512 - perf/total_num_tokens:204851 - perf/time_per_step:31.074970627203584 - perf/throughput:824.0192825020315 (TaskRunner pid=2403642) Training Progress: 17%|█▋ | 17/100 [10:52<46:44, 33.78s/it] (TaskRunner pid=2403642) step:18 - global_seqlen/min:22716 - global_seqlen/max:27857 - global_seqlen/minmax_diff:5141 - global_seqlen/balanced_min:25547 - global_seqlen/balanced_max:25558 - global_seqlen/mean:25553.75 - actor/entropy:0.15348316729068756 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41023707389831543 - training/rollout_probs_diff_mean:0.002396379830315709 - training/rollout_probs_diff_std:0.008675065822899342 - training/rollout_actor_probs_pearson_corr:0.9990106225013733 - rollout_corr/rollout_is_mean:1.0001230239868164 - rollout_corr/rollout_is_ratio_fraction_high:9.547268746246118e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.525377035140991 - rollout_corr/rollout_is_min:0.535515308380127 - rollout_corr/rollout_is_std:0.024909522384405136 - rollout_corr/rollout_is_eff_sample_size:0.9993800194714868 - rollout_corr/rollout_is_seq_mean:1.000164270401001 - rollout_corr/rollout_is_seq_std:0.001482548424974084 - rollout_corr/rollout_is_seq_max:1.006403923034668 - rollout_corr/rollout_is_seq_min:0.9943401217460632 - rollout_corr/rollout_is_seq_max_deviation:0.006403923034667969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1755624981597066) - rollout_corr/training_log_ppl:np.float64(0.1577926207028213) - rollout_corr/kl:np.float64(0.0004000797638387399) - rollout_corr/k3_kl:np.float64(0.00042349859928947353) - rollout_corr/rollout_ppl:np.float64(1.175103010609746) - rollout_corr/rollout_log_ppl:np.float64(0.15739254072104814) - rollout_corr/log_ppl_diff:np.float64(0.0004000799817731604) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012859311391366646) - rollout_corr/log_ppl_diff_max:np.float64(0.006476789712905884) - rollout_corr/log_ppl_diff_min:np.float64(-0.005883336067199707) - rollout_corr/ppl_ratio:np.float64(1.0004015415906906) - rollout_corr/chi2_token:np.float64(0.0009252710733562708) - rollout_corr/chi2_seq:np.float64(0.45614964910782874) - actor/pg_loss:np.float64(9.575032163411379e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002660133432073053) - actor/ppo_kl:np.float64(0.00022335322293542959) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18165946006774902) - perf/mfu/actor:np.float64(0.10286695858314113) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.21480178833008) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004518244881182909 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004518244881182909 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:409.1484375 - response_length/max:880.0 - response_length/min:170.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:409.1484375 - response_length_non_aborted/max:880.0 - response_length_non_aborted/min:170.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:389.40625 - prompt_length/max:492.0 - prompt_length/min:206.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012827478349208832 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1651914436370134) - timing_s/agent_loop/generate_sequences/max:np.float64(8.112665966153145) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.75771035838261) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.112665966153145) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:880 - timing_s/gen:15.17405853793025 - timing_s/reward:0.060957103967666626 - timing_s/old_log_prob:3.141259476542473 - timing_s/adv:0.1505921334028244 - timing_s/update_actor:13.009109269827604 - timing_s/step:31.61887457035482 - timing_s/stop_profile:0.0001099761575460434 - timing_per_token_ms/gen:0.1448708114980643 - timing_per_token_ms/adv:0.000736644002361808 - timing_per_token_ms/update_actor:0.06363600875521011 - perf/total_num_tokens:204430 - perf/time_per_step:31.61887457035482 - perf/throughput:808.1802514235801 (TaskRunner pid=2403642) Training Progress: 18%|█▊ | 18/100 [11:23<45:18, 33.15s/it] (TaskRunner pid=2403642) step:19 - global_seqlen/min:24113 - global_seqlen/max:31499 - global_seqlen/minmax_diff:7386 - global_seqlen/balanced_min:27522 - global_seqlen/balanced_max:27526 - global_seqlen/mean:27523.875 - actor/entropy:0.18350636959075928 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4400348365306854 - training/rollout_probs_diff_mean:0.0025148489512503147 - training/rollout_probs_diff_std:0.008488977327942848 - training/rollout_actor_probs_pearson_corr:0.9992618560791016 - rollout_corr/rollout_is_mean:0.9999606609344482 - rollout_corr/rollout_is_ratio_fraction_high:8.207013706851285e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.165530204772949 - rollout_corr/rollout_is_min:0.5154091119766235 - rollout_corr/rollout_is_std:0.02545158751308918 - rollout_corr/rollout_is_eff_sample_size:0.9993523979621741 - rollout_corr/rollout_is_seq_mean:0.9999680519104004 - rollout_corr/rollout_is_seq_std:0.0013802233152091503 - rollout_corr/rollout_is_seq_max:1.0054303407669067 - rollout_corr/rollout_is_seq_min:0.9942456483840942 - rollout_corr/rollout_is_seq_max_deviation:0.005754351615905762 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.210047378204763) - rollout_corr/training_log_ppl:np.float64(0.18686755485396134) - rollout_corr/kl:np.float64(0.0003091387440541382) - rollout_corr/k3_kl:np.float64(0.0003994344855300369) - rollout_corr/rollout_ppl:np.float64(1.2096817316487432) - rollout_corr/rollout_log_ppl:np.float64(0.18655841430881992) - rollout_corr/log_ppl_diff:np.float64(0.00030914054514141753) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011480295725050382) - rollout_corr/log_ppl_diff_max:np.float64(0.0067489296197891235) - rollout_corr/log_ppl_diff_min:np.float64(-0.00647541880607605) - rollout_corr/ppl_ratio:np.float64(1.0003103516064584) - rollout_corr/chi2_token:np.float64(0.0010224655270576477) - rollout_corr/chi2_seq:np.float64(0.5422194402199239) - actor/pg_loss:np.float64(-3.9114849641919136e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001304197849094635) - actor/ppo_kl:np.float64(-7.434647464321653e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17187580466270447) - perf/mfu/actor:np.float64(0.11026519680088508) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.16053438186646) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009684276301413774 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009684276301413774 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:475.96484375 - response_length/max:846.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:475.96484375 - response_length_non_aborted/max:846.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.15625 - prompt_length/max:494.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015503913164138794 - timing_s/agent_loop/generate_sequences/min:np.float64(2.144832994788885) - timing_s/agent_loop/generate_sequences/max:np.float64(8.308098182082176) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.560128854158393) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.308098182082176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:449 - timing_s/agent_loop/slowest/response_length:811 - timing_s/gen:14.88607195392251 - timing_s/reward:0.06521516665816307 - timing_s/old_log_prob:3.224108999595046 - timing_s/adv:0.14479263313114643 - timing_s/update_actor:13.007811995223165 - timing_s/step:31.41670709103346 - timing_s/stop_profile:0.00011463277041912079 - timing_per_token_ms/gen:0.12217019667224067 - timing_per_token_ms/adv:0.0006575774356406321 - timing_per_token_ms/update_actor:0.05907513020615359 - perf/total_num_tokens:220191 - perf/time_per_step:31.41670709103346 - perf/throughput:876.090384655733 (TaskRunner pid=2403642) Training Progress: 19%|█▉ | 19/100 [11:55<44:04, 32.65s/it] (TaskRunner pid=2403642) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2403642) validation generation end (TaskRunner pid=2403642) [prompt] system (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2403642) (TaskRunner pid=2403642) A (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) user (TaskRunner pid=2403642) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2403642) (TaskRunner pid=2403642) Lattice Parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) a = 7.82 Å (TaskRunner pid=2403642) b = 7.82 Å (TaskRunner pid=2403642) c = 17.88 Å (TaskRunner pid=2403642) α = 90.00º (TaskRunner pid=2403642) β = 90.00º (TaskRunner pid=2403642) γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) A: 1025.67 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) B: 1094.31 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) C: 1200.45 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) D: 1150.78 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) Please reason step by step. (TaskRunner pid=2403642) assistant (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) [response] (TaskRunner pid=2403642) To calculate the volume of a unit cell in a crystal structure, we use the formula for the volume of a unit cell based on the lattice parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \times b \times c \times \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) In this case, the crystal structure is given with the following lattice parameters: (TaskRunner pid=2403642) - $ a = 7.82 $ Å (TaskRunner pid=2403642) - $ b = 7.82 $ Å (TaskRunner pid=2403642) - $ c = 17.88 $ Å (TaskRunner pid=2403642) - $ \alpha = 90.00^\circ $ (TaskRunner pid=2403642) - $ \beta = 90.00^\circ $ (TaskRunner pid=2403642) - $ \gamma = 90.00^\circ $ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Since all angles are 90 degrees, the crystal structure is orthorhombic, and the formula simplifies to: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \times b \times c $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Now, substitute the values: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Then multiply by $ 17.88 $: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Therefore, the volume of the unit cell is $ 1094.31 $ ų. (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) B (TaskRunner pid=2403642) (TaskRunner pid=2403642) [ground_truth] B (TaskRunner pid=2403642) [score] 1.0 (TaskRunner pid=2403642) [acc] 1.0 (TaskRunner pid=2403642) [pred] B (TaskRunner pid=2403642) [incorrect_format] 1 (TaskRunner pid=2403642) [feedback] (TaskRunner pid=2403642) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2403642) step:20 - global_seqlen/min:23527 - global_seqlen/max:31800 - global_seqlen/minmax_diff:8273 - global_seqlen/balanced_min:27764 - global_seqlen/balanced_max:27771 - global_seqlen/mean:27768.625 - actor/entropy:0.1863677203655243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25539880990982056 - training/rollout_probs_diff_mean:0.002553188009187579 - training/rollout_probs_diff_std:0.008304450660943985 - training/rollout_actor_probs_pearson_corr:0.9992569088935852 - rollout_corr/rollout_is_mean:0.9998255372047424 - rollout_corr/rollout_is_ratio_fraction_high:8.369811439479236e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.369811439479236e-06 - rollout_corr/rollout_is_max:2.0695643424987793 - rollout_corr/rollout_is_min:0.40586209297180176 - rollout_corr/rollout_is_std:0.025391798466444016 - rollout_corr/rollout_is_eff_sample_size:0.9993553743445184 - rollout_corr/rollout_is_seq_mean:0.9998011589050293 - rollout_corr/rollout_is_seq_std:0.0013334499672055244 - rollout_corr/rollout_is_seq_max:1.0043513774871826 - rollout_corr/rollout_is_seq_min:0.9964183568954468 - rollout_corr/rollout_is_seq_max_deviation:0.004351377487182617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.210513724014163) - rollout_corr/training_log_ppl:np.float64(0.18781593269523) - rollout_corr/kl:np.float64(0.0005289665862315829) - rollout_corr/k3_kl:np.float64(0.0003631379920179256) - rollout_corr/rollout_ppl:np.float64(1.2098644091747701) - rollout_corr/rollout_log_ppl:np.float64(0.18728696474136086) - rollout_corr/log_ppl_diff:np.float64(0.0005289679538691416) - rollout_corr/log_ppl_abs_diff:np.float64(0.001086680407752283) - rollout_corr/log_ppl_diff_max:np.float64(0.005179837346076965) - rollout_corr/log_ppl_diff_min:np.float64(-0.003173530101776123) - rollout_corr/ppl_ratio:np.float64(1.0005300352349877) - rollout_corr/chi2_token:np.float64(0.0004152453038841486) - rollout_corr/chi2_seq:np.float64(0.07480436656624079) - actor/pg_loss:np.float64(2.3769563995301723e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.079792703734711e-05) - actor/ppo_kl:np.float64(-1.0977940139156317e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1225764099508524) - perf/mfu/actor:np.float64(0.11097826177378524) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.25468444824219) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.706781914893617) - val-aux/sciknoweval/reward/std@16:np.float64(0.09874159305871691) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7449893617021276) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.0670867911174988) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6692127659574468) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.09798249294138696) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7065957446808511) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.10005103385584223) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7665851063829787) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.04321020147130527) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6289255319148936) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09109460458607277) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7163404255319149) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.07276144717910131) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7829042553191489) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.03043673029185372) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5888936170212765) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07593238143528759) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7209148936170212) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.04468436283740652) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7954787234042554) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.021678038074192666) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5562765957446808) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05018701047232671) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7211170212765958) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.02631430733093111) - val-aux/sciknoweval/score/mean@16:np.float64(0.706781914893617) - val-aux/sciknoweval/score/std@16:np.float64(0.09874159305871691) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7449893617021276) - val-aux/sciknoweval/score/best@2/std:np.float64(0.0670867911174988) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6692127659574468) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.09798249294138696) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7065957446808511) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.10005103385584223) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7665851063829787) - val-aux/sciknoweval/score/best@4/std:np.float64(0.04321020147130527) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6289255319148936) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09109460458607277) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7163404255319149) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.07276144717910131) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7829042553191489) - val-aux/sciknoweval/score/best@8/std:np.float64(0.03043673029185372) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5888936170212765) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07593238143528759) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7209148936170212) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.04468436283740652) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7954787234042554) - val-aux/sciknoweval/score/best@16/std:np.float64(0.021678038074192666) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5562765957446808) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05018701047232671) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7211170212765958) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.02631430733093111) - val-core/sciknoweval/acc/mean@16:np.float64(0.706781914893617) - val-aux/sciknoweval/acc/std@16:np.float64(0.09874159305871691) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7449893617021276) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.0670867911174988) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6692127659574468) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.09798249294138696) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7065957446808511) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.10005103385584223) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7665851063829787) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.04321020147130527) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6289255319148936) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09109460458607277) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7163404255319149) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.07276144717910131) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7829042553191489) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.03043673029185372) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5888936170212765) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07593238143528759) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7209148936170212) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.04468436283740652) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7954787234042554) - val-core/sciknoweval/acc/best@16/std:np.float64(0.021678038074192666) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5562765957446808) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05018701047232671) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7211170212765958) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.02631430733093111) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0001014839654089883 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0001014839654089883 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:466.70703125 - response_length/max:879.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:466.70703125 - response_length_non_aborted/max:879.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.0625 - prompt_length/max:492.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015135854482650757 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8971739206463099) - timing_s/agent_loop/generate_sequences/max:np.float64(8.296176321804523) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.369741026603151) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.296176321804523) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:482 - timing_s/agent_loop/slowest/response_length:848 - timing_s/gen:14.875661510974169 - timing_s/reward:0.06638186797499657 - timing_s/old_log_prob:3.1252263449132442 - timing_s/adv:0.12130717560648918 - timing_s/update_actor:13.124596381559968 - timing_s/step:31.401353472843766 - timing_s/testing:45.161780742928386 - timing_s/save_checkpoint:13.100456409156322 - timing_s/stop_profile:0.00016569718718528748 - timing_per_token_ms/gen:0.124506486695968 - timing_per_token_ms/adv:0.0005460622177299434 - timing_per_token_ms/update_actor:0.05908015062665134 - perf/total_num_tokens:222149 - perf/time_per_step:31.401353472843766 - perf/throughput:884.3129970182531 (TaskRunner pid=2403642) Training Progress: 20%|██ | 20/100 [13:24<1:06:23, 49.79s/it] (TaskRunner pid=2403642) step:21 - global_seqlen/min:24020 - global_seqlen/max:30816 - global_seqlen/minmax_diff:6796 - global_seqlen/balanced_min:27042 - global_seqlen/balanced_max:27050 - global_seqlen/mean:27046.625 - actor/entropy:0.20988589525222778 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23094487190246582 - training/rollout_probs_diff_mean:0.002792060375213623 - training/rollout_probs_diff_std:0.008543950505554676 - training/rollout_actor_probs_pearson_corr:0.9993190765380859 - rollout_corr/rollout_is_mean:0.9999420642852783 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.816615343093872 - rollout_corr/rollout_is_min:0.6647822260856628 - rollout_corr/rollout_is_std:0.026228930801153183 - rollout_corr/rollout_is_eff_sample_size:0.9993122780583822 - rollout_corr/rollout_is_seq_mean:0.9999385476112366 - rollout_corr/rollout_is_seq_std:0.00129312661010772 - rollout_corr/rollout_is_seq_max:1.0037506818771362 - rollout_corr/rollout_is_seq_min:0.9962911605834961 - rollout_corr/rollout_is_seq_max_deviation:0.0037506818771362305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2395123722963035) - rollout_corr/training_log_ppl:np.float64(0.21087656814052025) - rollout_corr/kl:np.float64(0.00034929181821574673) - rollout_corr/k3_kl:np.float64(0.0003811120066359308) - rollout_corr/rollout_ppl:np.float64(1.239079408813268) - rollout_corr/rollout_log_ppl:np.float64(0.21052727483038325) - rollout_corr/log_ppl_diff:np.float64(0.00034929331013699993) - rollout_corr/log_ppl_abs_diff:np.float64(0.001078697205230128) - rollout_corr/log_ppl_diff_max:np.float64(0.004425495862960815) - rollout_corr/log_ppl_diff_min:np.float64(-0.0028105080127716064) - rollout_corr/ppl_ratio:np.float64(1.0003502799663693) - rollout_corr/chi2_token:np.float64(0.0008452925831079483) - rollout_corr/chi2_seq:np.float64(0.35264981212094426) - actor/pg_loss:np.float64(-6.608013063669205e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001253350560546096) - actor/ppo_kl:np.float64(-7.613605790623978e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15066424570977688) - perf/mfu/actor:np.float64(0.10792687021589811) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.82352495193481) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004071439150720835 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.004071439150720835 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:451.05078125 - response_length/max:825.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:451.05078125 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.15625 - prompt_length/max:524.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.632452666759491e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0553892366588116) - timing_s/agent_loop/generate_sequences/max:np.float64(8.023678202182055) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.208998567650269) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.023678202182055) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:446 - timing_s/agent_loop/slowest/response_length:810 - timing_s/gen:14.172827698290348 - timing_s/reward:0.0629449412226677 - timing_s/old_log_prob:3.3852025605738163 - timing_s/adv:0.11489835940301418 - timing_s/update_actor:13.015619862824678 - timing_s/step:30.850695865228772 - timing_s/stop_profile:0.00012831203639507294 - timing_per_token_ms/gen:0.12274140850176539 - timing_per_token_ms/adv:0.000531019856465521 - timing_per_token_ms/update_actor:0.06015362296970823 - perf/total_num_tokens:216373 - perf/time_per_step:30.850695865228772 - perf/throughput:876.6941633392371 (TaskRunner pid=2403642) Training Progress: 21%|██ | 21/100 [13:55<58:05, 44.12s/it] (TaskRunner pid=2403642) step:22 - global_seqlen/min:20457 - global_seqlen/max:31713 - global_seqlen/minmax_diff:11256 - global_seqlen/balanced_min:26522 - global_seqlen/balanced_max:26527 - global_seqlen/mean:26525.25 - actor/entropy:0.1900346577167511 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413552701473236 - training/rollout_probs_diff_mean:0.00256634084507823 - training/rollout_probs_diff_std:0.00823039747774601 - training/rollout_actor_probs_pearson_corr:0.9992367625236511 - rollout_corr/rollout_is_mean:0.9999353885650635 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.67016933625564e-06 - rollout_corr/rollout_is_max:1.6402086019515991 - rollout_corr/rollout_is_min:0.43521374464035034 - rollout_corr/rollout_is_std:0.024964498355984688 - rollout_corr/rollout_is_eff_sample_size:0.9993770429423398 - rollout_corr/rollout_is_seq_mean:0.9999351501464844 - rollout_corr/rollout_is_seq_std:0.0014069859171286225 - rollout_corr/rollout_is_seq_max:1.005880355834961 - rollout_corr/rollout_is_seq_min:0.995682418346405 - rollout_corr/rollout_is_seq_max_deviation:0.0058803558349609375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.217024851590395) - rollout_corr/training_log_ppl:np.float64(0.19277588855766226) - rollout_corr/kl:np.float64(0.00034876816037332503) - rollout_corr/k3_kl:np.float64(0.00038947746364215163) - rollout_corr/rollout_ppl:np.float64(1.216602017171681) - rollout_corr/rollout_log_ppl:np.float64(0.19242711797414813) - rollout_corr/log_ppl_diff:np.float64(0.00034877058351412416) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010859863250516355) - rollout_corr/log_ppl_diff_max:np.float64(0.0049463361501693726) - rollout_corr/log_ppl_diff_min:np.float64(-0.005746215581893921) - rollout_corr/ppl_ratio:np.float64(1.0003498834557831) - rollout_corr/chi2_token:np.float64(0.0008665753994137049) - rollout_corr/chi2_seq:np.float64(0.2999505987390876) - actor/pg_loss:np.float64(9.697419591248035e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.08482979866676e-05) - actor/ppo_kl:np.float64(-6.385789393270613e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12580287642776966) - perf/mfu/actor:np.float64(0.10640031863300181) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(110.78891372680664) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014565885066986084 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014565885066986084 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:450.5390625 - response_length/max:790.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:450.5390625 - response_length_non_aborted/max:790.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:378.375 - prompt_length/max:509.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.725242853164673e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7559421602636576) - timing_s/agent_loop/generate_sequences/max:np.float64(7.862461443990469) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.1768025901401415) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.862461443990469) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:760 - timing_s/gen:15.114545667544007 - timing_s/reward:0.06437323614954948 - timing_s/old_log_prob:3.082932820543647 - timing_s/adv:0.11840634606778622 - timing_s/update_actor:12.972170302644372 - timing_s/step:31.439034961163998 - timing_s/stop_profile:0.00011073052883148193 - timing_per_token_ms/gen:0.13104567157002903 - timing_per_token_ms/adv:0.0005579888317159415 - timing_per_token_ms/update_actor:0.06113123487358447 - perf/total_num_tokens:212202 - perf/time_per_step:31.439034961163998 - perf/throughput:843.7043322979252 (TaskRunner pid=2403642) Training Progress: 22%|██▏ | 22/100 [14:27<52:25, 40.33s/it] (TaskRunner pid=2403642) step:23 - global_seqlen/min:18473 - global_seqlen/max:30908 - global_seqlen/minmax_diff:12435 - global_seqlen/balanced_min:25645 - global_seqlen/balanced_max:25648 - global_seqlen/mean:25647.125 - actor/entropy:0.20922939479351044 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2637213468551636 - training/rollout_probs_diff_mean:0.0029920015949755907 - training/rollout_probs_diff_std:0.009136729873716831 - training/rollout_actor_probs_pearson_corr:0.9991100430488586 - rollout_corr/rollout_is_mean:1.000084638595581 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9838535785675049 - rollout_corr/rollout_is_min:0.5637778639793396 - rollout_corr/rollout_is_std:0.02762783132493496 - rollout_corr/rollout_is_eff_sample_size:0.9992375231521491 - rollout_corr/rollout_is_seq_mean:1.0001695156097412 - rollout_corr/rollout_is_seq_std:0.0014690716052427888 - rollout_corr/rollout_is_seq_max:1.0058791637420654 - rollout_corr/rollout_is_seq_min:0.9964756965637207 - rollout_corr/rollout_is_seq_max_deviation:0.00587916374206543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2296825596131384) - rollout_corr/training_log_ppl:np.float64(0.20483429163869005) - rollout_corr/kl:np.float64(0.00022616685472609532) - rollout_corr/k3_kl:np.float64(0.0003908713183022883) - rollout_corr/rollout_ppl:np.float64(1.2293974561616778) - rollout_corr/rollout_log_ppl:np.float64(0.20460812396777328) - rollout_corr/log_ppl_diff:np.float64(0.00022616767091676593) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011755102314054966) - rollout_corr/log_ppl_diff_max:np.float64(0.0060980916023254395) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052372366189956665) - rollout_corr/ppl_ratio:np.float64(1.0002273020800203) - rollout_corr/chi2_token:np.float64(0.0011190951336175203) - rollout_corr/chi2_seq:np.float64(0.6697086337953806) - actor/pg_loss:np.float64(-6.491632666438818e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018655569419934181) - actor/ppo_kl:np.float64(3.843692887528505e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1308762775734067) - perf/mfu/actor:np.float64(0.10322630604294634) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(110.69140243530273) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002776920795440674 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002776920795440674 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:417.78515625 - response_length/max:783.0 - response_length/min:179.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.78515625 - response_length_non_aborted/max:783.0 - response_length_non_aborted/min:179.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.6875 - prompt_length/max:493.0 - prompt_length/min:257.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.929202497005463e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.208122979849577) - timing_s/agent_loop/generate_sequences/max:np.float64(7.101036481559277) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.8324967114022) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.101036481559277) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:438 - timing_s/agent_loop/slowest/response_length:706 - timing_s/gen:14.253962934017181 - timing_s/reward:0.06467375718057156 - timing_s/old_log_prob:3.1388892475515604 - timing_s/adv:0.12416808865964413 - timing_s/update_actor:13.07398303039372 - timing_s/step:30.743178697302938 - timing_s/stop_profile:6.946362555027008e-05 - timing_per_token_ms/gen:0.13327314740135557 - timing_per_token_ms/adv:0.000605175476099388 - timing_per_token_ms/update_actor:0.06372050975691096 - perf/total_num_tokens:205177 - perf/time_per_step:30.743178697302938 - perf/throughput:834.2379053422343 (TaskRunner pid=2403642) Training Progress: 23%|██▎ | 23/100 [14:58<48:04, 37.46s/it] (TaskRunner pid=2403642) step:24 - global_seqlen/min:23618 - global_seqlen/max:30976 - global_seqlen/minmax_diff:7358 - global_seqlen/balanced_min:27454 - global_seqlen/balanced_max:27459 - global_seqlen/mean:27456.875 - actor/entropy:0.16937562823295593 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5486339330673218 - training/rollout_probs_diff_mean:0.0023563068825751543 - training/rollout_probs_diff_std:0.008330320939421654 - training/rollout_actor_probs_pearson_corr:0.9992015957832336 - rollout_corr/rollout_is_mean:1.000078558921814 - rollout_corr/rollout_is_ratio_fraction_high:8.243275260610972e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.243275260610972e-06 - rollout_corr/rollout_is_max:3.039975881576538 - rollout_corr/rollout_is_min:0.0006037261919118464 - rollout_corr/rollout_is_std:0.02445065602660179 - rollout_corr/rollout_is_eff_sample_size:0.9994027607393757 - rollout_corr/rollout_is_seq_mean:1.0001696348190308 - rollout_corr/rollout_is_seq_std:0.0013404276687651873 - rollout_corr/rollout_is_seq_max:1.0070313215255737 - rollout_corr/rollout_is_seq_min:0.997212290763855 - rollout_corr/rollout_is_seq_max_deviation:0.0070313215255737305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1992141208611429) - rollout_corr/training_log_ppl:np.float64(0.17708433591906214) - rollout_corr/kl:np.float64(0.00021467350635373528) - rollout_corr/k3_kl:np.float64(0.00040859091361511446) - rollout_corr/rollout_ppl:np.float64(1.198959435801953) - rollout_corr/rollout_log_ppl:np.float64(0.17686966038309038) - rollout_corr/log_ppl_diff:np.float64(0.00021467553597176448) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010494204252609052) - rollout_corr/log_ppl_diff_max:np.float64(0.016319751739501953) - rollout_corr/log_ppl_diff_min:np.float64(-0.003981977701187134) - rollout_corr/ppl_ratio:np.float64(1.0002161071170121) - rollout_corr/chi2_token:np.float64(0.0011550325434654951) - rollout_corr/chi2_seq:np.float64(0.5893677773419768) - actor/pg_loss:np.float64(8.695642463862896e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.039850561341154e-05) - actor/ppo_kl:np.float64(5.909442530871445e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14581617712974548) - perf/mfu/actor:np.float64(0.10952445859984875) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(110.68953323364258) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006563708302564919 - critic/advantages/max:0.5 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0006563708302564919 - critic/returns/max:0.5 - critic/returns/min:-0.75 - response_length/mean:473.87109375 - response_length/max:818.0 - response_length/min:176.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:473.87109375 - response_length_non_aborted/max:818.0 - response_length_non_aborted/min:176.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.15625 - prompt_length/max:477.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.136430263519287e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1059566028416157) - timing_s/agent_loop/generate_sequences/max:np.float64(7.97863014601171) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.456470786826685) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.97863014601171) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:459 - timing_s/agent_loop/slowest/response_length:762 - timing_s/gen:15.442488372325897 - timing_s/reward:0.06564980186522007 - timing_s/old_log_prob:3.1257457230240107 - timing_s/adv:0.1359486449509859 - timing_s/update_actor:13.1055824495852 - timing_s/step:31.963432287797332 - timing_s/stop_profile:0.00011222809553146362 - timing_per_token_ms/gen:0.127296686799432 - timing_per_token_ms/adv:0.0006189189636065007 - timing_per_token_ms/update_actor:0.059664393934056585 - perf/total_num_tokens:219655 - perf/time_per_step:31.963432287797332 - perf/throughput:859.0089685231395 (TaskRunner pid=2403642) Training Progress: 24%|██▍ | 24/100 [15:30<45:22, 35.82s/it] (TaskRunner pid=2403642) step:25 - global_seqlen/min:22266 - global_seqlen/max:31914 - global_seqlen/minmax_diff:9648 - global_seqlen/balanced_min:26848 - global_seqlen/balanced_max:26857 - global_seqlen/mean:26854.5 - actor/entropy:0.2005494087934494 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21512538194656372 - training/rollout_probs_diff_mean:0.002576504833996296 - training/rollout_probs_diff_std:0.008091717958450317 - training/rollout_actor_probs_pearson_corr:0.9993591904640198 - rollout_corr/rollout_is_mean:1.0000380277633667 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6483047008514404 - rollout_corr/rollout_is_min:0.5925564169883728 - rollout_corr/rollout_is_std:0.02495972253382206 - rollout_corr/rollout_is_eff_sample_size:0.9993775191858117 - rollout_corr/rollout_is_seq_mean:1.000016212463379 - rollout_corr/rollout_is_seq_std:0.0014132416108623147 - rollout_corr/rollout_is_seq_max:1.0057713985443115 - rollout_corr/rollout_is_seq_min:0.9941661953926086 - rollout_corr/rollout_is_seq_max_deviation:0.005833804607391357 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.247515902388841) - rollout_corr/training_log_ppl:np.float64(0.21253347316815052) - rollout_corr/kl:np.float64(0.0003586823201153777) - rollout_corr/k3_kl:np.float64(0.0004320936768209549) - rollout_corr/rollout_ppl:np.float64(1.2470650793984532) - rollout_corr/rollout_log_ppl:np.float64(0.2121747907367535) - rollout_corr/log_ppl_diff:np.float64(0.00035868243139702827) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011752601276384667) - rollout_corr/log_ppl_diff_max:np.float64(0.005879908800125122) - rollout_corr/log_ppl_diff_min:np.float64(-0.00401286780834198) - rollout_corr/ppl_ratio:np.float64(1.000359978992492) - rollout_corr/chi2_token:np.float64(0.001059497706592083) - rollout_corr/chi2_seq:np.float64(0.5972242541611195) - actor/pg_loss:np.float64(2.3624743334949017e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012215516676405969) - actor/ppo_kl:np.float64(2.806563131052542e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1335355006158352) - perf/mfu/actor:np.float64(0.10680968031632504) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(110.6960678100586) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005718811880797148 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005718811880797148 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:448.765625 - response_length/max:1180.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:448.765625 - response_length_non_aborted/max:1180.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.4375 - prompt_length/max:481.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014773011207580566 - timing_s/agent_loop/generate_sequences/min:np.float64(2.004378417506814) - timing_s/agent_loop/generate_sequences/max:np.float64(9.795474849641323) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.088747393274389) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.795474849641323) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1180 - timing_s/gen:17.24072809331119 - timing_s/reward:0.06484648585319519 - timing_s/old_log_prob:3.1079817228019238 - timing_s/adv:0.11739494651556015 - timing_s/update_actor:13.089253498241305 - timing_s/step:33.707123439759016 - timing_s/stop_profile:0.00011344440281391144 - timing_per_token_ms/gen:0.15007075043793036 - timing_per_token_ms/adv:0.0005464398262654311 - timing_per_token_ms/update_actor:0.06092672316670067 - perf/total_num_tokens:214836 - perf/time_per_step:33.707123439759016 - perf/throughput:796.7010311038275 (TaskRunner pid=2403642) Training Progress: 25%|██▌ | 25/100 [16:03<44:00, 35.21s/it] (TaskRunner pid=2403642) step:26 - global_seqlen/min:25781 - global_seqlen/max:31695 - global_seqlen/minmax_diff:5914 - global_seqlen/balanced_min:28514 - global_seqlen/balanced_max:28518 - global_seqlen/mean:28515.875 - actor/entropy:0.20827430486679077 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2769121527671814 - training/rollout_probs_diff_mean:0.0026053066831082106 - training/rollout_probs_diff_std:0.00794726237654686 - training/rollout_actor_probs_pearson_corr:0.9993575811386108 - rollout_corr/rollout_is_mean:1.0002012252807617 - rollout_corr/rollout_is_ratio_fraction_high:7.904950507509056e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.244551658630371 - rollout_corr/rollout_is_min:0.5874804258346558 - rollout_corr/rollout_is_std:0.024796804413199425 - rollout_corr/rollout_is_eff_sample_size:0.9993857344571517 - rollout_corr/rollout_is_seq_mean:1.0002319812774658 - rollout_corr/rollout_is_seq_std:0.0011953577632084489 - rollout_corr/rollout_is_seq_max:1.0041698217391968 - rollout_corr/rollout_is_seq_min:0.9968327283859253 - rollout_corr/rollout_is_seq_max_deviation:0.004169821739196777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2440492357127368) - rollout_corr/training_log_ppl:np.float64(0.2152066015114542) - rollout_corr/kl:np.float64(0.00024418294762629245) - rollout_corr/k3_kl:np.float64(0.0003427424277333557) - rollout_corr/rollout_ppl:np.float64(1.2437414275482297) - rollout_corr/rollout_log_ppl:np.float64(0.2149624170269817) - rollout_corr/log_ppl_diff:np.float64(0.0002441844844724983) - rollout_corr/log_ppl_abs_diff:np.float64(0.000932207825826481) - rollout_corr/log_ppl_diff_max:np.float64(0.004497110843658447) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034370869398117065) - rollout_corr/ppl_ratio:np.float64(1.0002449133899063) - rollout_corr/chi2_token:np.float64(0.0009027821943163872) - rollout_corr/chi2_seq:np.float64(0.36676088720560074) - actor/pg_loss:np.float64(3.524916246533394e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.288250374884228e-05) - actor/ppo_kl:np.float64(0.00015204964313486613) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09116398356854916) - perf/mfu/actor:np.float64(0.113877495350513) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(110.65935134887695) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000627455476205796 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000627455476205796 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:494.15234375 - response_length/max:880.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:494.15234375 - response_length_non_aborted/max:880.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.96875 - prompt_length/max:491.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016456283628940582 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8963664006441832) - timing_s/agent_loop/generate_sequences/max:np.float64(8.646993298083544) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.699355000702781) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.646993298083544) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:880 - timing_s/gen:15.889902139082551 - timing_s/reward:0.06746228411793709 - timing_s/old_log_prob:3.106580648571253 - timing_s/adv:0.13499674573540688 - timing_s/update_actor:12.984761372208595 - timing_s/step:32.270915891975164 - timing_s/stop_profile:0.00011105276644229889 - timing_per_token_ms/gen:0.1256088957501605 - timing_per_token_ms/adv:0.0005917613686034835 - timing_per_token_ms/update_actor:0.05691900288965618 - perf/total_num_tokens:228127 - perf/time_per_step:32.270915891975164 - perf/throughput:883.6400892820977 (TaskRunner pid=2403642) Training Progress: 26%|██▌ | 26/100 [16:36<42:21, 34.34s/it] (TaskRunner pid=2403642) step:27 - global_seqlen/min:24354 - global_seqlen/max:30450 - global_seqlen/minmax_diff:6096 - global_seqlen/balanced_min:27918 - global_seqlen/balanced_max:27922 - global_seqlen/mean:27920.625 - actor/entropy:0.2055797427892685 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7309482097625732 - training/rollout_probs_diff_mean:0.0024637985043227673 - training/rollout_probs_diff_std:0.00815705768764019 - training/rollout_actor_probs_pearson_corr:0.9992928504943848 - rollout_corr/rollout_is_mean:1.0000125169754028 - rollout_corr/rollout_is_ratio_fraction_high:8.22930087451823e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.645860174903646e-05 - rollout_corr/rollout_is_max:2.0586676597595215 - rollout_corr/rollout_is_min:0.0001314591645495966 - rollout_corr/rollout_is_std:0.02405497618019581 - rollout_corr/rollout_is_eff_sample_size:0.9994216927413265 - rollout_corr/rollout_is_seq_mean:1.0000035762786865 - rollout_corr/rollout_is_seq_std:0.0012129470705986023 - rollout_corr/rollout_is_seq_max:1.003065824508667 - rollout_corr/rollout_is_seq_min:0.9962401390075684 - rollout_corr/rollout_is_seq_max_deviation:0.0037598609924316406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2440797071903944) - rollout_corr/training_log_ppl:np.float64(0.2131779525516322) - rollout_corr/kl:np.float64(0.0003574901133275299) - rollout_corr/k3_kl:np.float64(0.00035757935391700357) - rollout_corr/rollout_ppl:np.float64(1.2436211220920086) - rollout_corr/rollout_log_ppl:np.float64(0.21282046237320174) - rollout_corr/log_ppl_diff:np.float64(0.00035749017843045294) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010061207285616547) - rollout_corr/log_ppl_diff_max:np.float64(0.00536857545375824) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038084089756011963) - rollout_corr/ppl_ratio:np.float64(1.0003584027290344) - rollout_corr/chi2_token:np.float64(0.0006836575921624899) - rollout_corr/chi2_seq:np.float64(0.28105390653945506) - actor/pg_loss:np.float64(-0.00014648318756371737) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.341250309560564e-05) - actor/ppo_kl:np.float64(-8.465370333166078e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15089168027043343) - perf/mfu/actor:np.float64(0.11177476135023583) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.46063613891602) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.80078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004742134769912809 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004742134769912809 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:474.67578125 - response_length/max:811.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:474.67578125 - response_length_non_aborted/max:811.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.84375 - prompt_length/max:494.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00047997571527957916 - timing_s/agent_loop/generate_sequences/min:np.float64(2.266698367893696) - timing_s/agent_loop/generate_sequences/max:np.float64(7.956064598634839) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.491192605360993) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.956064598634839) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:494 - timing_s/agent_loop/slowest/response_length:758 - timing_s/gen:15.56756940856576 - timing_s/reward:0.06229664012789726 - timing_s/old_log_prob:3.189983759075403 - timing_s/adv:0.13729296810925007 - timing_s/update_actor:12.930999640375376 - timing_s/step:31.998288065195084 - timing_s/stop_profile:0.00011795200407505035 - timing_per_token_ms/gen:0.1281102183938524 - timing_per_token_ms/adv:0.0006146574803986752 - timing_per_token_ms/update_actor:0.05789178985237336 - perf/total_num_tokens:223365 - perf/time_per_step:31.998288065195084 - perf/throughput:872.5662117646098 (TaskRunner pid=2403642) Training Progress: 27%|██▋ | 27/100 [17:08<41:04, 33.76s/it] (TaskRunner pid=2403642) step:28 - global_seqlen/min:20775 - global_seqlen/max:31390 - global_seqlen/minmax_diff:10615 - global_seqlen/balanced_min:26861 - global_seqlen/balanced_max:26867 - global_seqlen/mean:26864.875 - actor/entropy:0.24384665489196777 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5723066329956055 - training/rollout_probs_diff_mean:0.0027470295317471027 - training/rollout_probs_diff_std:0.008356913924217224 - training/rollout_actor_probs_pearson_corr:0.9994053840637207 - rollout_corr/rollout_is_mean:0.9999920129776001 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.5305564122390933e-05 - rollout_corr/rollout_is_max:1.5868679285049438 - rollout_corr/rollout_is_min:0.24166789650917053 - rollout_corr/rollout_is_std:0.025605684146285057 - rollout_corr/rollout_is_eff_sample_size:0.9993447785041603 - rollout_corr/rollout_is_seq_mean:0.9999786615371704 - rollout_corr/rollout_is_seq_std:0.0014453264884650707 - rollout_corr/rollout_is_seq_max:1.005670189857483 - rollout_corr/rollout_is_seq_min:0.9954844117164612 - rollout_corr/rollout_is_seq_max_deviation:0.00567018985748291 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2841918719932437) - rollout_corr/training_log_ppl:np.float64(0.24157115184061695) - rollout_corr/kl:np.float64(0.0003922118894976734) - rollout_corr/k3_kl:np.float64(0.0004578474525800402) - rollout_corr/rollout_ppl:np.float64(1.2836904847063124) - rollout_corr/rollout_log_ppl:np.float64(0.2411789383040741) - rollout_corr/log_ppl_diff:np.float64(0.0003922135365428403) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012401825661072508) - rollout_corr/log_ppl_diff_max:np.float64(0.00700107216835022) - rollout_corr/log_ppl_diff_min:np.float64(-0.005011111497879028) - rollout_corr/ppl_ratio:np.float64(1.000393595546484) - rollout_corr/chi2_token:np.float64(0.0010894446168094873) - rollout_corr/chi2_seq:np.float64(0.5639708044473082) - actor/pg_loss:np.float64(5.167769268155098e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.550479020006605e-05) - actor/ppo_kl:np.float64(1.7489767522249622e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.116465725004673) - perf/mfu/actor:np.float64(0.10695600784376426) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.37525177001953) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00028047000523656607 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00028047000523656607 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:463.08984375 - response_length/max:837.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.08984375 - response_length_non_aborted/max:837.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:376.4375 - prompt_length/max:478.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001988764852285385 - timing_s/agent_loop/generate_sequences/min:np.float64(2.185948533937335) - timing_s/agent_loop/generate_sequences/max:np.float64(8.100449938327074) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.3857879123679595) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.100449938327074) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:450 - timing_s/agent_loop/slowest/response_length:837 - timing_s/gen:14.817726872861385 - timing_s/reward:0.06450781039893627 - timing_s/old_log_prob:3.1151426900178194 - timing_s/adv:0.1301942765712738 - timing_s/update_actor:13.202640883624554 - timing_s/step:31.42223551683128 - timing_s/stop_profile:0.000135716050863266 - timing_per_token_ms/gen:0.12499031533147241 - timing_per_token_ms/adv:0.0006057829999733565 - timing_per_token_ms/update_actor:0.06143077570444937 - perf/total_num_tokens:214919 - perf/time_per_step:31.42223551683128 - perf/throughput:854.9638355810129 (TaskRunner pid=2403642) Training Progress: 28%|██▊ | 28/100 [17:40<39:41, 33.08s/it] (TaskRunner pid=2403642) step:29 - global_seqlen/min:22297 - global_seqlen/max:31545 - global_seqlen/minmax_diff:9248 - global_seqlen/balanced_min:28290 - global_seqlen/balanced_max:28300 - global_seqlen/mean:28296.125 - actor/entropy:0.1927935630083084 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28604796528816223 - training/rollout_probs_diff_mean:0.0021387599408626556 - training/rollout_probs_diff_std:0.007081093732267618 - training/rollout_actor_probs_pearson_corr:0.9994828701019287 - rollout_corr/rollout_is_mean:0.999884843826294 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.611747145652771 - rollout_corr/rollout_is_min:0.5320075750350952 - rollout_corr/rollout_is_std:0.02203766442835331 - rollout_corr/rollout_is_eff_sample_size:0.9995145771041962 - rollout_corr/rollout_is_seq_mean:0.9998981952667236 - rollout_corr/rollout_is_seq_std:0.00132310192566365 - rollout_corr/rollout_is_seq_max:1.0052039623260498 - rollout_corr/rollout_is_seq_min:0.9921262264251709 - rollout_corr/rollout_is_seq_max_deviation:0.007873773574829102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2352579105645418) - rollout_corr/training_log_ppl:np.float64(0.20490435884858016) - rollout_corr/kl:np.float64(0.0003952607846713363) - rollout_corr/k3_kl:np.float64(0.0003191322348712333) - rollout_corr/rollout_ppl:np.float64(1.2347394162788987) - rollout_corr/rollout_log_ppl:np.float64(0.20450909563805908) - rollout_corr/log_ppl_diff:np.float64(0.0003952632105210796) - rollout_corr/log_ppl_abs_diff:np.float64(0.001101043846574612) - rollout_corr/log_ppl_diff_max:np.float64(0.008444517850875854) - rollout_corr/log_ppl_diff_min:np.float64(-0.004731416702270508) - rollout_corr/ppl_ratio:np.float64(1.0003964479546994) - rollout_corr/chi2_token:np.float64(0.0005052480846643448) - rollout_corr/chi2_seq:np.float64(0.20438435021787882) - actor/pg_loss:np.float64(5.3944881074130535e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011199153595953248) - actor/ppo_kl:np.float64(2.4016177389896143e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11256356723606586) - perf/mfu/actor:np.float64(0.11208081941034986) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.4299087524414) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002028820803388953 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002028820803388953 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:479.66015625 - response_length/max:881.0 - response_length/min:177.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:479.66015625 - response_length_non_aborted/max:881.0 - response_length_non_aborted/min:177.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:404.59375 - prompt_length/max:512.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001045428216457367 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2093354631215334) - timing_s/agent_loop/generate_sequences/max:np.float64(8.23103902488947) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.504767282072862) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.23103902488947) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:512 - timing_s/agent_loop/slowest/response_length:828 - timing_s/gen:15.497990868985653 - timing_s/reward:0.06586435809731483 - timing_s/old_log_prob:3.1171885281801224 - timing_s/adv:0.12244941666722298 - timing_s/update_actor:13.134156158193946 - timing_s/step:32.024198323488235 - timing_s/stop_profile:0.00018646381795406342 - timing_per_token_ms/gen:0.12621233188362246 - timing_per_token_ms/adv:0.0005409283809497898 - timing_per_token_ms/update_actor:0.05802100180764127 - perf/total_num_tokens:226369 - perf/time_per_step:32.024198323488235 - perf/throughput:883.5857408254348 (TaskRunner pid=2403642) Training Progress: 29%|██▉ | 29/100 [18:12<38:47, 32.78s/it] (TaskRunner pid=2403642) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2403642) validation generation end (TaskRunner pid=2403642) [prompt] system (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2403642) (TaskRunner pid=2403642) A (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) user (TaskRunner pid=2403642) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2403642) (TaskRunner pid=2403642) Lattice Parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) a = 7.82 Å (TaskRunner pid=2403642) b = 7.82 Å (TaskRunner pid=2403642) c = 17.88 Å (TaskRunner pid=2403642) α = 90.00º (TaskRunner pid=2403642) β = 90.00º (TaskRunner pid=2403642) γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) A: 1025.67 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) B: 1094.31 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) C: 1200.45 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) D: 1150.78 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) Please reason step by step. (TaskRunner pid=2403642) assistant (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) [response] (TaskRunner pid=2403642) To calculate the volume of the unit cell for a crystal structure, we use the formula for the volume of a unit cell, which depends on the lattice parameters and the angles between them: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \cdot b \cdot c \cdot \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given: (TaskRunner pid=2403642) - $ a = 7.82 $ Å (TaskRunner pid=2403642) - $ b = 7.82 $ Å (TaskRunner pid=2403642) - $ c = 17.88 $ Å (TaskRunner pid=2403642) - $ \alpha = 90^\circ $ (TaskRunner pid=2403642) - $ \beta = 90^\circ $ (TaskRunner pid=2403642) - $ \gamma = 90^\circ $ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Since all angles are 90°, the crystal structure is orthorhombic. In this case, the formula simplifies to: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \cdot b \cdot c $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Now, substitute the given values: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = 7.82 \cdot 7.82 \cdot 17.88 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) First, calculate $ 7.82 \cdot 7.82 $: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 7.82 \cdot 7.82 = 61.1524 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Now, multiply this by 17.88: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 61.1524 \cdot 17.88 = 1094.31 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) So, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) B (TaskRunner pid=2403642) (TaskRunner pid=2403642) [ground_truth] B (TaskRunner pid=2403642) [score] 1.0 (TaskRunner pid=2403642) [acc] 1.0 (TaskRunner pid=2403642) [pred] B (TaskRunner pid=2403642) [incorrect_format] 1 (TaskRunner pid=2403642) [feedback] (TaskRunner pid=2403642) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2403642) step:30 - global_seqlen/min:23520 - global_seqlen/max:31889 - global_seqlen/minmax_diff:8369 - global_seqlen/balanced_min:29105 - global_seqlen/balanced_max:29106 - global_seqlen/mean:29105.25 - actor/entropy:0.22476468980312347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25539630651474 - training/rollout_probs_diff_mean:0.0022680300753563643 - training/rollout_probs_diff_std:0.0070181735791265965 - training/rollout_actor_probs_pearson_corr:0.9995337128639221 - rollout_corr/rollout_is_mean:1.000045895576477 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7930431365966797 - rollout_corr/rollout_is_min:0.5083495378494263 - rollout_corr/rollout_is_std:0.02269851416349411 - rollout_corr/rollout_is_eff_sample_size:0.9994850427681393 - rollout_corr/rollout_is_seq_mean:1.0000736713409424 - rollout_corr/rollout_is_seq_std:0.001143696834333241 - rollout_corr/rollout_is_seq_max:1.0036154985427856 - rollout_corr/rollout_is_seq_min:0.9971887469291687 - rollout_corr/rollout_is_seq_max_deviation:0.0036154985427856445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2817749911919236) - rollout_corr/training_log_ppl:np.float64(0.23585528284456814) - rollout_corr/kl:np.float64(0.00021687363476452504) - rollout_corr/k3_kl:np.float64(0.00031874458314717913) - rollout_corr/rollout_ppl:np.float64(1.2814569370821118) - rollout_corr/rollout_log_ppl:np.float64(0.2356384065351449) - rollout_corr/log_ppl_diff:np.float64(0.00021687630942324176) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009794712605071254) - rollout_corr/log_ppl_diff_max:np.float64(0.005163669586181641) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038784146308898926) - rollout_corr/ppl_ratio:np.float64(1.000217784428969) - rollout_corr/chi2_token:np.float64(0.0008400704246014357) - rollout_corr/chi2_seq:np.float64(0.4963743803091347) - actor/pg_loss:np.float64(-7.453944999724627e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.395594548142981e-05) - actor/ppo_kl:np.float64(2.2799680227336694e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15676945261657238) - perf/mfu/actor:np.float64(0.11588638795918937) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.46013641357422) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7027925531914894) - val-aux/sciknoweval/reward/std@16:np.float64(0.1152265165634801) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7485106382978723) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.0846900615940614) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6569148936170213) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.10865230852578993) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7026063829787232) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.11444102586889436) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7797127659574468) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.056704653085655574) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.614436170212766) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.0941326641213177) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7107234042553192) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.08665034162618929) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8018191489361703) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.035254136893894374) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5756382978723404) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07668049600750805) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7147765957446808) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.0579906950963037) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8148297872340425) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.017294505266903102) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5442021276595744) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05309335928225127) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7180106382978723) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.03597123942734059) - val-aux/sciknoweval/score/mean@16:np.float64(0.7027925531914894) - val-aux/sciknoweval/score/std@16:np.float64(0.1152265165634801) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7485106382978723) - val-aux/sciknoweval/score/best@2/std:np.float64(0.0846900615940614) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6569148936170213) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.10865230852578993) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7026063829787232) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.11444102586889436) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7797127659574468) - val-aux/sciknoweval/score/best@4/std:np.float64(0.056704653085655574) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.614436170212766) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.0941326641213177) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7107234042553192) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.08665034162618929) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8018191489361703) - val-aux/sciknoweval/score/best@8/std:np.float64(0.035254136893894374) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5756382978723404) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07668049600750805) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7147765957446808) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.0579906950963037) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8148297872340425) - val-aux/sciknoweval/score/best@16/std:np.float64(0.017294505266903102) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5442021276595744) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05309335928225127) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7180106382978723) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.03597123942734059) - val-core/sciknoweval/acc/mean@16:np.float64(0.7027925531914894) - val-aux/sciknoweval/acc/std@16:np.float64(0.1152265165634801) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7485106382978723) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.0846900615940614) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6569148936170213) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.10865230852578993) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7026063829787232) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.11444102586889436) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7797127659574468) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.056704653085655574) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.614436170212766) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.0941326641213177) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7107234042553192) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.08665034162618929) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8018191489361703) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.035254136893894374) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5756382978723404) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07668049600750805) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7147765957446808) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.0579906950963037) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8148297872340425) - val-core/sciknoweval/acc/best@16/std:np.float64(0.017294505266903102) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5442021276595744) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05309335928225127) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7180106382978723) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.03597123942734059) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002027016453212127 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0002027016453212127 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:496.2265625 - response_length/max:938.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:496.2265625 - response_length_non_aborted/max:938.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:413.3125 - prompt_length/max:482.0 - prompt_length/min:206.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011067651212215424 - timing_s/agent_loop/generate_sequences/min:np.float64(1.926969500258565) - timing_s/agent_loop/generate_sequences/max:np.float64(9.089917557314038) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.944559972602292) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.089917557314038) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:938 - timing_s/gen:16.493819031864405 - timing_s/reward:0.06631623953580856 - timing_s/old_log_prob:3.098200213164091 - timing_s/adv:0.11566653102636337 - timing_s/update_actor:12.980147063732147 - timing_s/step:32.83957484923303 - timing_s/testing:47.53535209223628 - timing_s/save_checkpoint:13.027126125991344 - timing_s/stop_profile:5.0555914640426636e-05 - timing_per_token_ms/gen:0.12983783106778032 - timing_per_token_ms/adv:0.000496759738476578 - timing_per_token_ms/update_actor:0.05574658808862726 - perf/total_num_tokens:232842 - perf/time_per_step:32.83957484923303 - perf/throughput:886.2858345037239 (TaskRunner pid=2403642) Training Progress: 30%|███ | 30/100 [19:45<59:28, 50.98s/it] (TaskRunner pid=2403642) step:31 - global_seqlen/min:25888 - global_seqlen/max:30450 - global_seqlen/minmax_diff:4562 - global_seqlen/balanced_min:27424 - global_seqlen/balanced_max:27426 - global_seqlen/mean:27425.375 - actor/entropy:0.2341211438179016 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105886578559875 - training/rollout_probs_diff_mean:0.0023879760410636663 - training/rollout_probs_diff_std:0.007143564056605101 - training/rollout_actor_probs_pearson_corr:0.9994761347770691 - rollout_corr/rollout_is_mean:0.9998959898948669 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.458877346129157e-06 - rollout_corr/rollout_is_max:1.7586430311203003 - rollout_corr/rollout_is_min:0.44323089718818665 - rollout_corr/rollout_is_std:0.023076131939888 - rollout_corr/rollout_is_eff_sample_size:0.9994676564348994 - rollout_corr/rollout_is_seq_mean:0.999855101108551 - rollout_corr/rollout_is_seq_std:0.0011561827268451452 - rollout_corr/rollout_is_seq_max:1.0038914680480957 - rollout_corr/rollout_is_seq_min:0.9953445792198181 - rollout_corr/rollout_is_seq_max_deviation:0.004655420780181885 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2752317558042705) - rollout_corr/training_log_ppl:np.float64(0.23740968311176402) - rollout_corr/kl:np.float64(0.00037503032777941314) - rollout_corr/k3_kl:np.float64(0.00029784045699443595) - rollout_corr/rollout_ppl:np.float64(1.2747451555915177) - rollout_corr/rollout_log_ppl:np.float64(0.23703464957361575) - rollout_corr/log_ppl_diff:np.float64(0.0003750335381482728) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009644899037084542) - rollout_corr/log_ppl_diff_max:np.float64(0.0038155168294906616) - rollout_corr/log_ppl_diff_min:np.float64(-0.004125386476516724) - rollout_corr/ppl_ratio:np.float64(1.0003758335951716) - rollout_corr/chi2_token:np.float64(0.00044544809497892857) - rollout_corr/chi2_seq:np.float64(0.17121168388985097) - actor/pg_loss:np.float64(0.0001104114344343543) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012691650317719905) - actor/ppo_kl:np.float64(-5.702109045202519e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14219314232468605) - perf/mfu/actor:np.float64(0.10953375845039864) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(112.64505767822266) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008942090906202793 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008942090906202793 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:461.79296875 - response_length/max:839.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:461.79296875 - response_length_non_aborted/max:839.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:395.25 - prompt_length/max:492.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.103678166866302e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0917833521962166) - timing_s/agent_loop/generate_sequences/max:np.float64(8.422423332929611) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.35807181410928) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.422423332929611) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:447 - timing_s/agent_loop/slowest/response_length:808 - timing_s/gen:15.17648260295391 - timing_s/reward:0.06344617158174515 - timing_s/old_log_prob:3.1480666026473045 - timing_s/adv:0.111147940158844 - timing_s/update_actor:12.983128607273102 - timing_s/step:31.574141327291727 - timing_s/stop_profile:0.00012382306158542633 - timing_per_token_ms/gen:0.12837600219045933 - timing_per_token_ms/adv:0.0005065926179625802 - timing_per_token_ms/update_actor:0.05917479983078218 - perf/total_num_tokens:219403 - perf/time_per_step:31.574141327291727 - perf/throughput:868.6024020641961 (TaskRunner pid=2403642) Training Progress: 31%|███ | 31/100 [20:17<51:56, 45.17s/it] (TaskRunner pid=2403642) step:32 - global_seqlen/min:21612 - global_seqlen/max:32901 - global_seqlen/minmax_diff:11289 - global_seqlen/balanced_min:27666 - global_seqlen/balanced_max:27672 - global_seqlen/mean:27669.5 - actor/entropy:0.2351425141096115 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.893662691116333 - training/rollout_probs_diff_mean:0.002326859161257744 - training/rollout_probs_diff_std:0.007395750377327204 - training/rollout_actor_probs_pearson_corr:0.9995218515396118 - rollout_corr/rollout_is_mean:0.9999954104423523 - rollout_corr/rollout_is_ratio_fraction_high:8.340283784491476e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.340283784491476e-06 - rollout_corr/rollout_is_max:2.5332539081573486 - rollout_corr/rollout_is_min:0.012144971638917923 - rollout_corr/rollout_is_std:0.022933630272746086 - rollout_corr/rollout_is_eff_sample_size:0.9994742060099695 - rollout_corr/rollout_is_seq_mean:1.0000008344650269 - rollout_corr/rollout_is_seq_std:0.0012937926221638918 - rollout_corr/rollout_is_seq_max:1.0046977996826172 - rollout_corr/rollout_is_seq_min:0.9950659275054932 - rollout_corr/rollout_is_seq_max_deviation:0.004934072494506836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2944361343979836) - rollout_corr/training_log_ppl:np.float64(0.24626350629114313) - rollout_corr/kl:np.float64(0.0002554487013840534) - rollout_corr/k3_kl:np.float64(0.0003589305630811168) - rollout_corr/rollout_ppl:np.float64(1.2940973713994026) - rollout_corr/rollout_log_ppl:np.float64(0.24600805527734337) - rollout_corr/log_ppl_diff:np.float64(0.00025545101379975677) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010613546473905444) - rollout_corr/log_ppl_diff_max:np.float64(0.01161763072013855) - rollout_corr/log_ppl_diff_min:np.float64(-0.0033280551433563232) - rollout_corr/ppl_ratio:np.float64(1.000256673200056) - rollout_corr/chi2_token:np.float64(0.0008953770156949759) - rollout_corr/chi2_seq:np.float64(0.5134820169769228) - actor/pg_loss:np.float64(6.652786396443844e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010245629937344347) - actor/ppo_kl:np.float64(-5.820105452869484e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16398444771766663) - perf/mfu/actor:np.float64(0.11033717682891869) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.75769805908203) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005869474844075739 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005869474844075739 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:468.359375 - response_length/max:848.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:468.359375 - response_length_non_aborted/max:848.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001023951917886734 - timing_s/agent_loop/generate_sequences/min:np.float64(1.942232746630907) - timing_s/agent_loop/generate_sequences/max:np.float64(8.150801707059145) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.316388803075824) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.150801707059145) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:419 - timing_s/agent_loop/slowest/response_length:835 - timing_s/gen:15.041519667953253 - timing_s/reward:0.06522183865308762 - timing_s/old_log_prob:3.1123256366699934 - timing_s/adv:0.11888792738318443 - timing_s/update_actor:12.938601948320866 - timing_s/step:31.36331848986447 - timing_s/stop_profile:6.30207359790802e-05 - timing_per_token_ms/gen:0.125450539349068 - timing_per_token_ms/adv:0.000537089247109563 - timing_per_token_ms/update_actor:0.0584515529207289 - perf/total_num_tokens:221356 - perf/time_per_step:31.36331848986447 - perf/throughput:882.224883471493 (TaskRunner pid=2403642) Training Progress: 32%|███▏ | 32/100 [20:48<46:30, 41.04s/it] (TaskRunner pid=2403642) step:33 - global_seqlen/min:20947 - global_seqlen/max:30495 - global_seqlen/minmax_diff:9548 - global_seqlen/balanced_min:27719 - global_seqlen/balanced_max:27728 - global_seqlen/mean:27724.125 - actor/entropy:0.25870415568351746 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8123438954353333 - training/rollout_probs_diff_mean:0.0024514386896044016 - training/rollout_probs_diff_std:0.007493364159017801 - training/rollout_actor_probs_pearson_corr:0.9996020793914795 - rollout_corr/rollout_is_mean:0.9999789595603943 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.6236797819146886e-05 - rollout_corr/rollout_is_max:1.5526777505874634 - rollout_corr/rollout_is_min:0.0813368558883667 - rollout_corr/rollout_is_std:0.02380339801311493 - rollout_corr/rollout_is_eff_sample_size:0.9994336000282128 - rollout_corr/rollout_is_seq_mean:1.000024437904358 - rollout_corr/rollout_is_seq_std:0.001137533108703792 - rollout_corr/rollout_is_seq_max:1.0037413835525513 - rollout_corr/rollout_is_seq_min:0.9970559477806091 - rollout_corr/rollout_is_seq_max_deviation:0.0037413835525512695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.318345540203154) - rollout_corr/training_log_ppl:np.float64(0.26373951183632016) - rollout_corr/kl:np.float64(0.0002792183334006104) - rollout_corr/k3_kl:np.float64(0.0003289724186430476) - rollout_corr/rollout_ppl:np.float64(1.3179421676322818) - rollout_corr/rollout_log_ppl:np.float64(0.2634602915350115) - rollout_corr/log_ppl_diff:np.float64(0.00027922030130866915) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009092716936720535) - rollout_corr/log_ppl_diff_max:np.float64(0.009413182735443115) - rollout_corr/log_ppl_diff_min:np.float64(-0.003118470311164856) - rollout_corr/ppl_ratio:np.float64(1.0002800580114126) - rollout_corr/chi2_token:np.float64(0.0007109332364052534) - rollout_corr/chi2_seq:np.float64(0.21768948528915644) - actor/pg_loss:np.float64(-2.1165120415389538e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.417932996148011e-05) - actor/ppo_kl:np.float64(-4.76137685723188e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1778383795171976) - perf/mfu/actor:np.float64(0.11100855943794985) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.48153686523438) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00239492766559124 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00239492766559124 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:481.16015625 - response_length/max:929.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:481.16015625 - response_length_non_aborted/max:929.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.21875 - prompt_length/max:480.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.61252236366272e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1162655036896467) - timing_s/agent_loop/generate_sequences/max:np.float64(8.431221760809422) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.50012135515135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.431221760809422) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:459 - timing_s/agent_loop/slowest/response_length:929 - timing_s/gen:15.760489447042346 - timing_s/reward:0.06304218620061874 - timing_s/old_log_prob:3.091986909508705 - timing_s/adv:0.12686722911894321 - timing_s/update_actor:12.838001016527414 - timing_s/step:31.930561792105436 - timing_s/stop_profile:0.00011173449456691742 - timing_per_token_ms/gen:0.12794993746431838 - timing_per_token_ms/adv:0.0005720073632573761 - timing_per_token_ms/update_actor:0.05788280521264158 - perf/total_num_tokens:221793 - perf/time_per_step:31.930561792105436 - perf/throughput:868.2629883090425 (TaskRunner pid=2403642) Training Progress: 33%|███▎ | 33/100 [21:20<42:47, 38.32s/it] (TaskRunner pid=2403642) step:34 - global_seqlen/min:25779 - global_seqlen/max:32632 - global_seqlen/minmax_diff:6853 - global_seqlen/balanced_min:28623 - global_seqlen/balanced_max:28629 - global_seqlen/mean:28625.75 - actor/entropy:0.24488314986228943 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7310589551925659 - training/rollout_probs_diff_mean:0.00249440036714077 - training/rollout_probs_diff_std:0.0074654994532465935 - training/rollout_actor_probs_pearson_corr:0.9995710253715515 - rollout_corr/rollout_is_mean:1.000001072883606 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.6263600628008135e-05 - rollout_corr/rollout_is_max:1.951600193977356 - rollout_corr/rollout_is_min:0.26894107460975647 - rollout_corr/rollout_is_std:0.023488307371735573 - rollout_corr/rollout_is_eff_sample_size:0.9994484845358234 - rollout_corr/rollout_is_seq_mean:1.0000451803207397 - rollout_corr/rollout_is_seq_std:0.0012103441404178739 - rollout_corr/rollout_is_seq_max:1.0049504041671753 - rollout_corr/rollout_is_seq_min:0.9962252974510193 - rollout_corr/rollout_is_seq_max_deviation:0.004950404167175293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3030735217034817) - rollout_corr/training_log_ppl:np.float64(0.2590878018454532) - rollout_corr/kl:np.float64(0.00033822323003818155) - rollout_corr/k3_kl:np.float64(0.00033817789738321835) - rollout_corr/rollout_ppl:np.float64(1.3026120658032596) - rollout_corr/rollout_log_ppl:np.float64(0.2587495753541589) - rollout_corr/log_ppl_diff:np.float64(0.0003382264912943356) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009682540330686606) - rollout_corr/log_ppl_diff_max:np.float64(0.011813685297966003) - rollout_corr/log_ppl_diff_min:np.float64(-0.004161953926086426) - rollout_corr/ppl_ratio:np.float64(1.000339277787134) - rollout_corr/chi2_token:np.float64(0.0006227635312825441) - rollout_corr/chi2_seq:np.float64(0.22475781477987766) - actor/pg_loss:np.float64(0.00017607829067856073) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.003569008797058e-05) - actor/ppo_kl:np.float64(8.233531357149104e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11507174372673035) - perf/mfu/actor:np.float64(0.1140994907811674) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.46148681640625) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008680696482770145 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008680696482770145 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:480.3671875 - response_length/max:934.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.3671875 - response_length_non_aborted/max:934.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:414.1875 - prompt_length/max:482.0 - prompt_length/min:271.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.508803486824036e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.029529247432947) - timing_s/agent_loop/generate_sequences/max:np.float64(8.856205441057682) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.579186692339135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.856205441057682) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:877 - timing_s/gen:15.42210958339274 - timing_s/reward:0.06580507196485996 - timing_s/old_log_prob:3.1205662954598665 - timing_s/adv:0.11770584806799889 - timing_s/update_actor:13.002541430294514 - timing_s/step:31.81529883481562 - timing_s/stop_profile:0.00011952407658100128 - timing_per_token_ms/gen:0.12540951407120807 - timing_per_token_ms/adv:0.0005139858696627987 - timing_per_token_ms/update_actor:0.05677816926322679 - perf/total_num_tokens:229006 - perf/time_per_step:31.81529883481562 - perf/throughput:899.7479529777264 (TaskRunner pid=2403642) Training Progress: 34%|███▍ | 34/100 [21:52<40:01, 36.38s/it] (TaskRunner pid=2403642) step:35 - global_seqlen/min:24227 - global_seqlen/max:29909 - global_seqlen/minmax_diff:5682 - global_seqlen/balanced_min:25979 - global_seqlen/balanced_max:25983 - global_seqlen/mean:25981.75 - actor/entropy:0.2596481442451477 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2622659206390381 - training/rollout_probs_diff_mean:0.0026135449297726154 - training/rollout_probs_diff_std:0.0073624346405267715 - training/rollout_actor_probs_pearson_corr:0.999534547328949 - rollout_corr/rollout_is_mean:1.0000660419464111 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5070068836212158 - rollout_corr/rollout_is_min:0.5062825679779053 - rollout_corr/rollout_is_std:0.02401033416390419 - rollout_corr/rollout_is_eff_sample_size:0.9994239551040021 - rollout_corr/rollout_is_seq_mean:1.0000636577606201 - rollout_corr/rollout_is_seq_std:0.0013239914551377296 - rollout_corr/rollout_is_seq_max:1.0058327913284302 - rollout_corr/rollout_is_seq_min:0.9960933327674866 - rollout_corr/rollout_is_seq_max_deviation:0.005832791328430176 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3059996077790856) - rollout_corr/training_log_ppl:np.float64(0.26158572202257346) - rollout_corr/kl:np.float64(0.0004447607697875) - rollout_corr/k3_kl:np.float64(0.00033729273330607157) - rollout_corr/rollout_ppl:np.float64(1.3054057322442532) - rollout_corr/rollout_log_ppl:np.float64(0.261140958769829) - rollout_corr/log_ppl_diff:np.float64(0.00044476325274445117) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011254070268478245) - rollout_corr/log_ppl_diff_max:np.float64(0.007861137390136719) - rollout_corr/log_ppl_diff_min:np.float64(-0.00546593964099884) - rollout_corr/ppl_ratio:np.float64(1.0004459931515157) - rollout_corr/chi2_token:np.float64(0.0004484797827899456) - rollout_corr/chi2_seq:np.float64(0.2356518420856446) - actor/pg_loss:np.float64(2.136023249477148e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.410318408394232e-06) - actor/ppo_kl:np.float64(0.00019875600776231295) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09427732788026333) - perf/mfu/actor:np.float64(0.1048240639539772) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.44162034988403) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006490063853561878 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006490063853561878 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:427.3359375 - response_length/max:854.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:427.3359375 - response_length_non_aborted/max:854.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.59375 - prompt_length/max:459.0 - prompt_length/min:274.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015152059495449066 - timing_s/agent_loop/generate_sequences/min:np.float64(1.993311908096075) - timing_s/agent_loop/generate_sequences/max:np.float64(7.891210563480854) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.914158031700936) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.891210563480854) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:452 - timing_s/agent_loop/slowest/response_length:854 - timing_s/gen:14.651534689590335 - timing_s/reward:0.06202704273164272 - timing_s/old_log_prob:3.084901884198189 - timing_s/adv:0.12269627302885056 - timing_s/update_actor:12.809735296294093 - timing_s/step:30.8191114962101 - timing_s/stop_profile:0.00011374428868293762 - timing_per_token_ms/gen:0.1339287252928786 - timing_per_token_ms/adv:0.0005903002734075388 - timing_per_token_ms/update_actor:0.0616285243309924 - perf/total_num_tokens:207854 - perf/time_per_step:30.8191114962101 - perf/throughput:843.0402026091842 (TaskRunner pid=2403642) Training Progress: 35%|███▌ | 35/100 [22:23<37:37, 34.74s/it] (TaskRunner pid=2403642) step:36 - global_seqlen/min:23449 - global_seqlen/max:31040 - global_seqlen/minmax_diff:7591 - global_seqlen/balanced_min:27587 - global_seqlen/balanced_max:27592 - global_seqlen/mean:27589.5 - actor/entropy:0.2709210216999054 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491968750953674 - training/rollout_probs_diff_mean:0.002717098919674754 - training/rollout_probs_diff_std:0.007521712686866522 - training/rollout_actor_probs_pearson_corr:0.9994779825210571 - rollout_corr/rollout_is_mean:0.9999476075172424 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.64872407913208 - rollout_corr/rollout_is_min:0.6180930137634277 - rollout_corr/rollout_is_std:0.024854427203536034 - rollout_corr/rollout_is_eff_sample_size:0.9993825197696726 - rollout_corr/rollout_is_seq_mean:0.9999822974205017 - rollout_corr/rollout_is_seq_std:0.0013898462057113647 - rollout_corr/rollout_is_seq_max:1.00667142868042 - rollout_corr/rollout_is_seq_min:0.9957444667816162 - rollout_corr/rollout_is_seq_max_deviation:0.006671428680419922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3220618744380772) - rollout_corr/training_log_ppl:np.float64(0.2742194219317753) - rollout_corr/kl:np.float64(0.00034166475395225326) - rollout_corr/k3_kl:np.float64(0.0003441765950782383) - rollout_corr/rollout_ppl:np.float64(1.321584774646908) - rollout_corr/rollout_log_ppl:np.float64(0.27387775406532455) - rollout_corr/log_ppl_diff:np.float64(0.00034166786645073444) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011080713447881863) - rollout_corr/log_ppl_diff_max:np.float64(0.006220310926437378) - rollout_corr/log_ppl_diff_min:np.float64(-0.005935758352279663) - rollout_corr/ppl_ratio:np.float64(1.0003428137861192) - rollout_corr/chi2_token:np.float64(0.0006969412788748741) - rollout_corr/chi2_seq:np.float64(0.2664695552084595) - actor/pg_loss:np.float64(-3.446557093411684e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.949851553552435e-05) - actor/ppo_kl:np.float64(4.116838032253156e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13451017811894417) - perf/mfu/actor:np.float64(0.11006513992096655) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.43120193481445) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015561387408524752 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015561387408524752 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:464.390625 - response_length/max:878.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:464.390625 - response_length_non_aborted/max:878.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.78125 - prompt_length/max:509.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013710744678974152 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3757042922079563) - timing_s/agent_loop/generate_sequences/max:np.float64(8.467374807223678) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.015290737050236) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.467374807223678) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:271 - timing_s/agent_loop/slowest/response_length:878 - timing_s/gen:15.240626215934753 - timing_s/reward:0.06576763652265072 - timing_s/old_log_prob:3.0849456433206797 - timing_s/adv:0.12345556542277336 - timing_s/update_actor:13.022881884127855 - timing_s/step:31.625891700387 - timing_s/stop_profile:0.00011456385254859924 - timing_per_token_ms/gen:0.12819745479572317 - timing_per_token_ms/adv:0.0005593412594590938 - timing_per_token_ms/update_actor:0.05900289006745254 - perf/total_num_tokens:220716 - perf/time_per_step:31.625891700387 - perf/throughput:872.3706595018281 (TaskRunner pid=2403642) Training Progress: 36%|███▌ | 36/100 [22:55<36:04, 33.82s/it] (TaskRunner pid=2403642) step:37 - global_seqlen/min:20238 - global_seqlen/max:29602 - global_seqlen/minmax_diff:9364 - global_seqlen/balanced_min:25032 - global_seqlen/balanced_max:25035 - global_seqlen/mean:25033.0 - actor/entropy:0.2622256875038147 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23438845574855804 - training/rollout_probs_diff_mean:0.0027199105825275183 - training/rollout_probs_diff_std:0.0077196042984724045 - training/rollout_actor_probs_pearson_corr:0.9994896054267883 - rollout_corr/rollout_is_mean:0.9999966621398926 - rollout_corr/rollout_is_ratio_fraction_high:9.025270628626458e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0277390480041504 - rollout_corr/rollout_is_min:0.6365309953689575 - rollout_corr/rollout_is_std:0.02535303682088852 - rollout_corr/rollout_is_eff_sample_size:0.9993577554631589 - rollout_corr/rollout_is_seq_mean:0.9999998807907104 - rollout_corr/rollout_is_seq_std:0.0014201079029589891 - rollout_corr/rollout_is_seq_max:1.0045517683029175 - rollout_corr/rollout_is_seq_min:0.9944545030593872 - rollout_corr/rollout_is_seq_max_deviation:0.005545496940612793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3239887948147953) - rollout_corr/training_log_ppl:np.float64(0.26963052261271514) - rollout_corr/kl:np.float64(0.00043052457150238865) - rollout_corr/k3_kl:np.float64(0.0004052748620466673) - rollout_corr/rollout_ppl:np.float64(1.3233865266665816) - rollout_corr/rollout_log_ppl:np.float64(0.26919999830715824) - rollout_corr/log_ppl_diff:np.float64(0.0004305243055569008) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012312442268012092) - rollout_corr/log_ppl_diff_max:np.float64(0.004594564437866211) - rollout_corr/log_ppl_diff_min:np.float64(-0.003984987735748291) - rollout_corr/ppl_ratio:np.float64(1.0004317718558013) - rollout_corr/chi2_token:np.float64(0.0007657709065824747) - rollout_corr/chi2_seq:np.float64(0.3679836820811033) - actor/pg_loss:np.float64(-2.334697637706995e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.749956244471832e-05) - actor/ppo_kl:np.float64(9.06747865041524e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15217697247862816) - perf/mfu/actor:np.float64(0.10164436124329765) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.42990493774414) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0048601082526147366 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0048601082526147366 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:432.8125 - response_length/max:895.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:432.8125 - response_length_non_aborted/max:895.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.46875 - prompt_length/max:491.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001987311989068985 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1476458813995123) - timing_s/agent_loop/generate_sequences/max:np.float64(8.263966159895062) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.842065687218565) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.263966159895062) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:895 - timing_s/gen:15.675849946215749 - timing_s/reward:0.06284371577203274 - timing_s/old_log_prob:3.051144268363714 - timing_s/adv:0.11523681879043579 - timing_s/update_actor:12.753763007000089 - timing_s/step:31.746028892695904 - timing_s/stop_profile:0.00010980665683746338 - timing_per_token_ms/gen:0.14147879012830097 - timing_per_token_ms/adv:0.0005754245335678694 - timing_per_token_ms/update_actor:0.06368475116346467 - perf/total_num_tokens:200264 - perf/time_per_step:31.746028892695904 - perf/throughput:788.539570873999 (TaskRunner pid=2403642) Training Progress: 37%|███▋ | 37/100 [23:26<34:52, 33.21s/it] (TaskRunner pid=2403642) step:38 - global_seqlen/min:22776 - global_seqlen/max:30916 - global_seqlen/minmax_diff:8140 - global_seqlen/balanced_min:27403 - global_seqlen/balanced_max:27410 - global_seqlen/mean:27406.75 - actor/entropy:0.2778184711933136 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5546000003814697 - training/rollout_probs_diff_mean:0.0026332649867981672 - training/rollout_probs_diff_std:0.007570348680019379 - training/rollout_actor_probs_pearson_corr:0.9994733929634094 - rollout_corr/rollout_is_mean:1.0000190734863281 - rollout_corr/rollout_is_ratio_fraction_high:8.395601071242709e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6791202142485417e-05 - rollout_corr/rollout_is_max:2.175250768661499 - rollout_corr/rollout_is_min:0.20461466908454895 - rollout_corr/rollout_is_std:0.024789592251181602 - rollout_corr/rollout_is_eff_sample_size:0.9993858535200482 - rollout_corr/rollout_is_seq_mean:1.0000555515289307 - rollout_corr/rollout_is_seq_std:0.0012281483504921198 - rollout_corr/rollout_is_seq_max:1.0044065713882446 - rollout_corr/rollout_is_seq_min:0.9965072870254517 - rollout_corr/rollout_is_seq_max_deviation:0.004406571388244629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3290739133954048) - rollout_corr/training_log_ppl:np.float64(0.27583638252690434) - rollout_corr/kl:np.float64(0.00030358191989154903) - rollout_corr/k3_kl:np.float64(0.00035663118822526485) - rollout_corr/rollout_ppl:np.float64(1.3286687950603664) - rollout_corr/rollout_log_ppl:np.float64(0.2755327984923497) - rollout_corr/log_ppl_diff:np.float64(0.0003035840345546603) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011323641519993544) - rollout_corr/log_ppl_diff_max:np.float64(0.005536884069442749) - rollout_corr/log_ppl_diff_min:np.float64(-0.0041558146476745605) - rollout_corr/ppl_ratio:np.float64(1.0003046747297049) - rollout_corr/chi2_token:np.float64(0.0008237855508923531) - rollout_corr/chi2_seq:np.float64(0.5890746030490845) - actor/pg_loss:np.float64(-7.543503306806087e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.630327238279278e-05) - actor/ppo_kl:np.float64(3.2657569320804214e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1603358257561922) - perf/mfu/actor:np.float64(0.10901964703649844) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.42809295654297) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003431701799854636 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003431701799854636 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:465.2734375 - response_length/max:1004.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:465.2734375 - response_length_non_aborted/max:1004.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.1875 - prompt_length/max:541.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.173791527748108e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7882296908646822) - timing_s/agent_loop/generate_sequences/max:np.float64(8.705533424392343) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.197099420365703) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.705533424392343) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:1004 - timing_s/gen:15.281680893152952 - timing_s/reward:0.06543147936463356 - timing_s/old_log_prob:3.1095343474298716 - timing_s/adv:0.1125567052513361 - timing_s/update_actor:12.957870135083795 - timing_s/step:31.61500228010118 - timing_s/stop_profile:0.00012493692338466644 - timing_per_token_ms/gen:0.12829889088366175 - timing_per_token_ms/adv:0.0005133621518938587 - timing_per_token_ms/update_actor:0.059099811794009666 - perf/total_num_tokens:219254 - perf/time_per_step:31.61500228010118 - perf/throughput:866.8906539111687 (TaskRunner pid=2403642) Training Progress: 38%|███▊ | 38/100 [23:58<33:50, 32.75s/it] (TaskRunner pid=2403642) step:39 - global_seqlen/min:21926 - global_seqlen/max:31836 - global_seqlen/minmax_diff:9910 - global_seqlen/balanced_min:26048 - global_seqlen/balanced_max:26052 - global_seqlen/mean:26050.5 - actor/entropy:0.23975405097007751 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535163402557373 - training/rollout_probs_diff_mean:0.0025534790474921465 - training/rollout_probs_diff_std:0.0075296261347830296 - training/rollout_actor_probs_pearson_corr:0.9994619488716125 - rollout_corr/rollout_is_mean:1.0000462532043457 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9363714456558228 - rollout_corr/rollout_is_min:0.6228123307228088 - rollout_corr/rollout_is_std:0.024104483425617218 - rollout_corr/rollout_is_eff_sample_size:0.9994194303888935 - rollout_corr/rollout_is_seq_mean:1.0000656843185425 - rollout_corr/rollout_is_seq_std:0.001369856996461749 - rollout_corr/rollout_is_seq_max:1.004970669746399 - rollout_corr/rollout_is_seq_min:0.9958161115646362 - rollout_corr/rollout_is_seq_max_deviation:0.004970669746398926 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2866896665655077) - rollout_corr/training_log_ppl:np.float64(0.2466871515935054) - rollout_corr/kl:np.float64(0.0002536588835226894) - rollout_corr/k3_kl:np.float64(0.0003328794842047955) - rollout_corr/rollout_ppl:np.float64(1.2863487033173442) - rollout_corr/rollout_log_ppl:np.float64(0.24643349040707108) - rollout_corr/log_ppl_diff:np.float64(0.00025366118643432856) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010324943286832422) - rollout_corr/log_ppl_diff_max:np.float64(0.004928857088088989) - rollout_corr/log_ppl_diff_min:np.float64(-0.004723459482192993) - rollout_corr/ppl_ratio:np.float64(1.0002546701580286) - rollout_corr/chi2_token:np.float64(0.0008289911784231663) - rollout_corr/chi2_seq:np.float64(0.5572728242259473) - actor/pg_loss:np.float64(7.081078365445137e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(1.9370347672520438e-05) - actor/ppo_kl:np.float64(1.5183129935536499e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12078280001878738) - perf/mfu/actor:np.float64(0.10567685543937032) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.3458366394043) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008759010815992951 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008759010815992951 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:427.015625 - response_length/max:883.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:427.015625 - response_length_non_aborted/max:883.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:387.0625 - prompt_length/max:505.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001429375261068344 - timing_s/agent_loop/generate_sequences/min:np.float64(2.152031809091568) - timing_s/agent_loop/generate_sequences/max:np.float64(7.988802125677466) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.852606363965606) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.988802125677466) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:883 - timing_s/gen:14.53843860886991 - timing_s/reward:0.06197395920753479 - timing_s/old_log_prob:3.0982844587415457 - timing_s/adv:0.11875713244080544 - timing_s/update_actor:12.699045792222023 - timing_s/step:30.60459577664733 - timing_s/stop_profile:0.0001082438975572586 - timing_per_token_ms/gen:0.1329946083726985 - timing_per_token_ms/adv:0.0005698409456670958 - timing_per_token_ms/update_actor:0.06093475073521633 - perf/total_num_tokens:208404 - perf/time_per_step:30.60459577664733 - perf/throughput:851.1956893702119 (TaskRunner pid=2403642) Training Progress: 39%|███▉ | 39/100 [24:29<32:39, 32.12s/it] (TaskRunner pid=2403642) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2403642) validation generation end (TaskRunner pid=2403642) [prompt] system (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2403642) (TaskRunner pid=2403642) A (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) user (TaskRunner pid=2403642) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2403642) (TaskRunner pid=2403642) Lattice Parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) a = 7.82 Å (TaskRunner pid=2403642) b = 7.82 Å (TaskRunner pid=2403642) c = 17.88 Å (TaskRunner pid=2403642) α = 90.00º (TaskRunner pid=2403642) β = 90.00º (TaskRunner pid=2403642) γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) A: 1025.67 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) B: 1094.31 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) C: 1200.45 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) D: 1150.78 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) Please reason step by step. (TaskRunner pid=2403642) assistant (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) [response] (TaskRunner pid=2403642) The volume of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \times b \times c \times \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) For a cubic crystal structure (which this is, since α = β = γ = 90.00º), the formula simplifies to: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \times b \times c $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given: (TaskRunner pid=2403642) - a = 7.82 Å (TaskRunner pid=2403642) - b = 7.82 Å (TaskRunner pid=2403642) - c = 17.88 Å (TaskRunner pid=2403642) (TaskRunner pid=2403642) Substituting these values into the formula: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Then multiply by 17.88: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Therefore, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) B (TaskRunner pid=2403642) (TaskRunner pid=2403642) [ground_truth] B (TaskRunner pid=2403642) [score] 1.0 (TaskRunner pid=2403642) [acc] 1.0 (TaskRunner pid=2403642) [pred] B (TaskRunner pid=2403642) [incorrect_format] 1 (TaskRunner pid=2403642) [feedback] (TaskRunner pid=2403642) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2403642) step:40 - global_seqlen/min:20189 - global_seqlen/max:28119 - global_seqlen/minmax_diff:7930 - global_seqlen/balanced_min:24920 - global_seqlen/balanced_max:24923 - global_seqlen/mean:24921.875 - actor/entropy:0.2692209780216217 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2310577630996704 - training/rollout_probs_diff_mean:0.002756730420514941 - training/rollout_probs_diff_std:0.007605041842907667 - training/rollout_actor_probs_pearson_corr:0.9995124936103821 - rollout_corr/rollout_is_mean:1.0000061988830566 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.4278150796890259 - rollout_corr/rollout_is_min:0.5504452586174011 - rollout_corr/rollout_is_std:0.024521248415112495 - rollout_corr/rollout_is_eff_sample_size:0.9993990696778947 - rollout_corr/rollout_is_seq_mean:1.000065803527832 - rollout_corr/rollout_is_seq_std:0.001380614354275167 - rollout_corr/rollout_is_seq_max:1.0054199695587158 - rollout_corr/rollout_is_seq_min:0.9957116842269897 - rollout_corr/rollout_is_seq_max_deviation:0.00541996955871582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3192760865204036) - rollout_corr/training_log_ppl:np.float64(0.2728104801790323) - rollout_corr/kl:np.float64(0.0001471567377218097) - rollout_corr/k3_kl:np.float64(0.00035515559358145765) - rollout_corr/rollout_ppl:np.float64(1.319087610580027) - rollout_corr/rollout_log_ppl:np.float64(0.272663321782602) - rollout_corr/log_ppl_diff:np.float64(0.0001471583964303136) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011645762133412063) - rollout_corr/log_ppl_diff_max:np.float64(0.0038172006607055664) - rollout_corr/log_ppl_diff_min:np.float64(-0.004808694124221802) - rollout_corr/ppl_ratio:np.float64(1.00014826701954) - rollout_corr/chi2_token:np.float64(0.0011329331900924444) - rollout_corr/chi2_seq:np.float64(0.39967853645794094) - actor/pg_loss:np.float64(-8.308794349431992e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.9222728395980084e-05) - actor/ppo_kl:np.float64(-0.00010214552137752242) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08158384636044502) - perf/mfu/actor:np.float64(0.1011381841099553) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.36322402954102) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7240691489361702) - val-aux/sciknoweval/reward/std@16:np.float64(0.10122444039363532) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7613085106382979) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.06993133531537328) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6865638297872341) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.09972499691001828) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7238404255319149) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.10065446646187623) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7869787234042553) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.04375591618344764) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6497978723404255) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09380131256289198) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7325425531914893) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.07347374709542713) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8037234042553191) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.02647539161149547) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6118829787234042) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0869859468809813) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7364999999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.047862660217948536) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8132234042553192) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.013916766628741731) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5718723404255319) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07324511545629868) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7397127659574468) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.035732993527985556) - val-aux/sciknoweval/score/mean@16:np.float64(0.7240691489361702) - val-aux/sciknoweval/score/std@16:np.float64(0.10122444039363532) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7613085106382979) - val-aux/sciknoweval/score/best@2/std:np.float64(0.06993133531537328) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6865638297872341) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.09972499691001828) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7238404255319149) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.10065446646187623) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7869787234042553) - val-aux/sciknoweval/score/best@4/std:np.float64(0.04375591618344764) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6497978723404255) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09380131256289198) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7325425531914893) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.07347374709542713) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8037234042553191) - val-aux/sciknoweval/score/best@8/std:np.float64(0.02647539161149547) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6118829787234042) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0869859468809813) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7364999999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.047862660217948536) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8132234042553192) - val-aux/sciknoweval/score/best@16/std:np.float64(0.013916766628741731) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5718723404255319) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07324511545629868) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7397127659574468) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.035732993527985556) - val-core/sciknoweval/acc/mean@16:np.float64(0.7240691489361702) - val-aux/sciknoweval/acc/std@16:np.float64(0.10122444039363532) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7613085106382979) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.06993133531537328) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6865638297872341) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.09972499691001828) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7238404255319149) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.10065446646187623) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7869787234042553) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.04375591618344764) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6497978723404255) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09380131256289198) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7325425531914893) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.07347374709542713) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8037234042553191) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.02647539161149547) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6118829787234042) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0869859468809813) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7364999999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.047862660217948536) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8132234042553192) - val-core/sciknoweval/acc/best@16/std:np.float64(0.013916766628741731) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5718723404255319) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07324511545629868) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7397127659574468) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.035732993527985556) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9993351063829787) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0025751219057230166) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999787234042554) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.00047528314688699314) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9988404255319148) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.003315310792311214) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9993829787234043) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.002486632437298573) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9976063829787235) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.00444236518310906) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9998510638297873) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0012498981398063418) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9958936170212767) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.005179048574556801) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9932446808510639) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.00512159745832533) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002173902466893196 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.002173902466893196 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:398.68359375 - response_length/max:789.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:398.68359375 - response_length_non_aborted/max:789.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:380.125 - prompt_length/max:468.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001199766993522644 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9521888811141253) - timing_s/agent_loop/generate_sequences/max:np.float64(7.229736687615514) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.547362951736432) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.229736687615514) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:774 - timing_s/gen:13.955810092389584 - timing_s/reward:0.059946209192276 - timing_s/old_log_prob:3.035145480185747 - timing_s/adv:0.14190049842000008 - timing_s/update_actor:12.781881777569652 - timing_s/step:30.060451973229647 - timing_s/testing:93.20514548011124 - timing_s/save_checkpoint:13.233764106407762 - timing_s/stop_profile:0.0001539122313261032 - timing_per_token_ms/gen:0.13673721223547794 - timing_per_token_ms/adv:0.0007117266378432606 - timing_per_token_ms/update_actor:0.06410975186241831 - perf/total_num_tokens:199375 - perf/time_per_step:30.060451973229647 - perf/throughput:829.0585591392369 (TaskRunner pid=2403642) Training Progress: 40%|████ | 40/100 [26:45<1:03:27, 63.46s/it] (TaskRunner pid=2403642) step:41 - global_seqlen/min:22101 - global_seqlen/max:30341 - global_seqlen/minmax_diff:8240 - global_seqlen/balanced_min:26800 - global_seqlen/balanced_max:26806 - global_seqlen/mean:26804.875 - actor/entropy:0.24527297914028168 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105892539024353 - training/rollout_probs_diff_mean:0.0024506240151822567 - training/rollout_probs_diff_std:0.007172984071075916 - training/rollout_actor_probs_pearson_corr:0.9995378255844116 - rollout_corr/rollout_is_mean:1.0000022649765015 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5441513061523438 - rollout_corr/rollout_is_min:0.5378821492195129 - rollout_corr/rollout_is_std:0.023597173392772675 - rollout_corr/rollout_is_eff_sample_size:0.99944348329181 - rollout_corr/rollout_is_seq_mean:0.9999989867210388 - rollout_corr/rollout_is_seq_std:0.0013331422815099359 - rollout_corr/rollout_is_seq_max:1.0063869953155518 - rollout_corr/rollout_is_seq_min:0.9958586096763611 - rollout_corr/rollout_is_seq_max_deviation:0.006386995315551758 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2987580224871635) - rollout_corr/training_log_ppl:np.float64(0.25461333782732254) - rollout_corr/kl:np.float64(0.00037003947440439333) - rollout_corr/k3_kl:np.float64(0.000317131163605211) - rollout_corr/rollout_ppl:np.float64(1.298254407942295) - rollout_corr/rollout_log_ppl:np.float64(0.25424329534871504) - rollout_corr/log_ppl_diff:np.float64(0.00037004247860750183) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010555475237197243) - rollout_corr/log_ppl_diff_max:np.float64(0.005651324987411499) - rollout_corr/log_ppl_diff_min:np.float64(-0.005803555250167847) - rollout_corr/ppl_ratio:np.float64(1.0003710864111781) - rollout_corr/chi2_token:np.float64(0.0005223318003118038) - rollout_corr/chi2_seq:np.float64(0.19784280378371477) - actor/pg_loss:np.float64(4.0626153349876404e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.391483702827827e-05) - actor/ppo_kl:np.float64(6.689386933178554e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12102004140615463) - perf/mfu/actor:np.float64(0.10706792224524682) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.53229522705078) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009183372603729367 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009183372603729367 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:437.05859375 - response_length/max:997.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.05859375 - response_length_non_aborted/max:997.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:400.59375 - prompt_length/max:499.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.401911377906799e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5086693558841944) - timing_s/agent_loop/generate_sequences/max:np.float64(8.868929112330079) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.9913268628079095) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.868929112330079) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:997 - timing_s/gen:15.21447511576116 - timing_s/reward:0.058584295213222504 - timing_s/old_log_prob:3.25773966871202 - timing_s/adv:0.1243264339864254 - timing_s/update_actor:13.056273797526956 - timing_s/step:31.825263621285558 - timing_s/stop_profile:2.806819975376129e-05 - timing_per_token_ms/gen:0.13598072265554675 - timing_per_token_ms/adv:0.00057977529267729 - timing_per_token_ms/update_actor:0.060885724133795414 - perf/total_num_tokens:214439 - perf/time_per_step:31.825263621285558 - perf/throughput:842.2514678581392 (TaskRunner pid=2403642) Training Progress: 41%|████ | 41/100 [27:17<53:04, 53.98s/it] (TaskRunner pid=2403642) step:42 - global_seqlen/min:25339 - global_seqlen/max:30354 - global_seqlen/minmax_diff:5015 - global_seqlen/balanced_min:27648 - global_seqlen/balanced_max:27657 - global_seqlen/mean:27654.375 - actor/entropy:0.249659925699234 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163419246673584 - training/rollout_probs_diff_mean:0.002381665166467428 - training/rollout_probs_diff_std:0.007057615090161562 - training/rollout_actor_probs_pearson_corr:0.9996173977851868 - rollout_corr/rollout_is_mean:1.0001263618469238 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.602668458479457e-06 - rollout_corr/rollout_is_max:1.9402073621749878 - rollout_corr/rollout_is_min:0.44588688015937805 - rollout_corr/rollout_is_std:0.023735692724585533 - rollout_corr/rollout_is_eff_sample_size:0.9994370531944818 - rollout_corr/rollout_is_seq_mean:1.0001702308654785 - rollout_corr/rollout_is_seq_std:0.0012351945042610168 - rollout_corr/rollout_is_seq_max:1.0057711601257324 - rollout_corr/rollout_is_seq_min:0.9974271655082703 - rollout_corr/rollout_is_seq_max_deviation:0.005771160125732422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3183727939613163) - rollout_corr/training_log_ppl:np.float64(0.2636420357521274) - rollout_corr/kl:np.float64(0.0003492014980753799) - rollout_corr/k3_kl:np.float64(0.00035256788812176865) - rollout_corr/rollout_ppl:np.float64(1.3178962673991919) - rollout_corr/rollout_log_ppl:np.float64(0.2632928293242003) - rollout_corr/log_ppl_diff:np.float64(0.00034920642792712897) - rollout_corr/log_ppl_abs_diff:np.float64(0.001064916534232907) - rollout_corr/log_ppl_diff_max:np.float64(0.01382581889629364) - rollout_corr/log_ppl_diff_min:np.float64(-0.004806399345397949) - rollout_corr/ppl_ratio:np.float64(1.0003504659980536) - rollout_corr/chi2_token:np.float64(0.0006320546381175518) - rollout_corr/chi2_seq:np.float64(0.2667905113194138) - actor/pg_loss:np.float64(2.428167499601841e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(1.0820636816788465e-05) - actor/ppo_kl:np.float64(0.00022477524237096702) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13300273567438126) - perf/mfu/actor:np.float64(0.11066785146384982) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.22729873657227) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015237476909533143 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015237476909533143 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:454.07421875 - response_length/max:830.0 - response_length/min:170.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:454.07421875 - response_length_non_aborted/max:830.0 - response_length_non_aborted/min:170.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:410.125 - prompt_length/max:510.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5350600481033325e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0540550239384174) - timing_s/agent_loop/generate_sequences/max:np.float64(7.967892354354262) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.309633722696162) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.967892354354262) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:393 - timing_s/agent_loop/slowest/response_length:781 - timing_s/gen:15.329625273123384 - timing_s/reward:0.06597327440977097 - timing_s/old_log_prob:3.1255873944610357 - timing_s/adv:0.11859093979001045 - timing_s/update_actor:12.868022060021758 - timing_s/step:31.593635687604547 - timing_s/stop_profile:0.00011480413377285004 - timing_per_token_ms/gen:0.1318756851864059 - timing_per_token_ms/adv:0.0005360405893733381 - timing_per_token_ms/update_actor:0.058164495039310045 - perf/total_num_tokens:221235 - perf/time_per_step:31.593635687604547 - perf/throughput:875.3147397610185 (TaskRunner pid=2403642) Training Progress: 42%|████▏ | 42/100 [27:49<45:42, 47.28s/it] (TaskRunner pid=2403642) step:43 - global_seqlen/min:22391 - global_seqlen/max:31176 - global_seqlen/minmax_diff:8785 - global_seqlen/balanced_min:27125 - global_seqlen/balanced_max:27133 - global_seqlen/mean:27130.5 - actor/entropy:0.2829850912094116 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23087042570114136 - training/rollout_probs_diff_mean:0.0026621695142239332 - training/rollout_probs_diff_std:0.007386308629065752 - training/rollout_actor_probs_pearson_corr:0.9996067881584167 - rollout_corr/rollout_is_mean:1.0000057220458984 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.773489930201322e-05 - rollout_corr/rollout_is_max:1.7430531978607178 - rollout_corr/rollout_is_min:0.22278240323066711 - rollout_corr/rollout_is_std:0.02498597651720047 - rollout_corr/rollout_is_eff_sample_size:0.9993760904567573 - rollout_corr/rollout_is_seq_mean:1.0000249147415161 - rollout_corr/rollout_is_seq_std:0.0012387627502903342 - rollout_corr/rollout_is_seq_max:1.0044147968292236 - rollout_corr/rollout_is_seq_min:0.9966818690299988 - rollout_corr/rollout_is_seq_max_deviation:0.004414796829223633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.347760524135083) - rollout_corr/training_log_ppl:np.float64(0.28952421667054296) - rollout_corr/kl:np.float64(0.0003012928969590156) - rollout_corr/k3_kl:np.float64(0.0003403808943005515) - rollout_corr/rollout_ppl:np.float64(1.3473055744543672) - rollout_corr/rollout_log_ppl:np.float64(0.28922292120114435) - rollout_corr/log_ppl_diff:np.float64(0.0003012954693986103) - rollout_corr/log_ppl_abs_diff:np.float64(0.0009853317314991727) - rollout_corr/log_ppl_diff_max:np.float64(0.004570573568344116) - rollout_corr/log_ppl_diff_min:np.float64(-0.004001826047897339) - rollout_corr/ppl_ratio:np.float64(1.0003021161537617) - rollout_corr/chi2_token:np.float64(0.0007619182579219341) - rollout_corr/chi2_seq:np.float64(0.20320769771933556) - actor/pg_loss:np.float64(3.108277451246977e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001063900026565534) - actor/ppo_kl:np.float64(1.231407157575859e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1578393466770649) - perf/mfu/actor:np.float64(0.10808209957965548) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.30550813674927) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0023088620509952307 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0023088620509952307 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.515625 - response_length/max:898.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.515625 - response_length_non_aborted/max:898.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.3125 - prompt_length/max:494.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016414374113082886 - timing_s/agent_loop/generate_sequences/min:np.float64(2.200499527156353) - timing_s/agent_loop/generate_sequences/max:np.float64(8.465374808758497) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.1702591333087184) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.465374808758497) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:898 - timing_s/gen:15.178781248629093 - timing_s/reward:0.06619619764387608 - timing_s/old_log_prob:3.0212193094193935 - timing_s/adv:0.11106680892407894 - timing_s/update_actor:13.142551416531205 - timing_s/step:31.606105795130134 - timing_s/stop_profile:2.8137117624282837e-05 - timing_per_token_ms/gen:0.13459707417292496 - timing_per_token_ms/adv:0.0005117248526753974 - timing_per_token_ms/update_actor:0.06055247515034373 - perf/total_num_tokens:217044 - perf/time_per_step:31.606105795130134 - perf/throughput:858.3942664705079 (TaskRunner pid=2403642) Training Progress: 43%|████▎ | 43/100 [28:20<40:27, 42.59s/it] (TaskRunner pid=2403642) step:44 - global_seqlen/min:23687 - global_seqlen/max:28489 - global_seqlen/minmax_diff:4802 - global_seqlen/balanced_min:25919 - global_seqlen/balanced_max:25924 - global_seqlen/mean:25921.0 - actor/entropy:0.23762044310569763 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917698621749878 - training/rollout_probs_diff_mean:0.002502892632037401 - training/rollout_probs_diff_std:0.007343742996454239 - training/rollout_actor_probs_pearson_corr:0.9994988441467285 - rollout_corr/rollout_is_mean:0.9999760985374451 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.069801308214664e-06 - rollout_corr/rollout_is_max:1.5526788234710693 - rollout_corr/rollout_is_min:0.48514747619628906 - rollout_corr/rollout_is_std:0.02382592298090458 - rollout_corr/rollout_is_eff_sample_size:0.9994326474348206 - rollout_corr/rollout_is_seq_mean:0.9999829530715942 - rollout_corr/rollout_is_seq_std:0.0012827671598643064 - rollout_corr/rollout_is_seq_max:1.005139708518982 - rollout_corr/rollout_is_seq_min:0.9958560466766357 - rollout_corr/rollout_is_seq_max_deviation:0.005139708518981934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2832285850308836) - rollout_corr/training_log_ppl:np.float64(0.24504046242509503) - rollout_corr/kl:np.float64(0.0004000083458315373) - rollout_corr/k3_kl:np.float64(0.00034762927705855873) - rollout_corr/rollout_ppl:np.float64(1.282694707158953) - rollout_corr/rollout_log_ppl:np.float64(0.24464045157947112) - rollout_corr/log_ppl_diff:np.float64(0.0004000108456239104) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010354671976529062) - rollout_corr/log_ppl_diff_max:np.float64(0.004307284951210022) - rollout_corr/log_ppl_diff_min:np.float64(-0.004696741700172424) - rollout_corr/ppl_ratio:np.float64(1.0004009895492345) - rollout_corr/chi2_token:np.float64(0.0005764113739132881) - rollout_corr/chi2_seq:np.float64(0.057558765169233084) - actor/pg_loss:np.float64(-1.6392907127738e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010265427090416779) - actor/ppo_kl:np.float64(8.106677549335473e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12191110476851463) - perf/mfu/actor:np.float64(0.10458949772185872) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.37454605102539) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002656318014487624 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.002656318014487624 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:430.6875 - response_length/max:820.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:430.6875 - response_length_non_aborted/max:820.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.34375 - prompt_length/max:524.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.637613892555237e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8315623067319393) - timing_s/agent_loop/generate_sequences/max:np.float64(7.903299419209361) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.8979601562678) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.903299419209361) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:281 - timing_s/agent_loop/slowest/response_length:815 - timing_s/gen:14.815047051757574 - timing_s/reward:0.06429556384682655 - timing_s/old_log_prob:3.12994815595448 - timing_s/adv:0.11841119639575481 - timing_s/update_actor:12.86581609211862 - timing_s/step:31.080680061131716 - timing_s/stop_profile:0.00011561065912246704 - timing_per_token_ms/gen:0.13436953137931337 - timing_per_token_ms/adv:0.0005710196192071815 - timing_per_token_ms/update_actor:0.06204340154758025 - perf/total_num_tokens:207368 - perf/time_per_step:31.080680061131716 - perf/throughput:833.990760466525 (TaskRunner pid=2403642) Training Progress: 44%|████▍ | 44/100 [28:52<36:32, 39.15s/it] (TaskRunner pid=2403642) step:45 - global_seqlen/min:20724 - global_seqlen/max:29711 - global_seqlen/minmax_diff:8987 - global_seqlen/balanced_min:25302 - global_seqlen/balanced_max:25306 - global_seqlen/mean:25304.125 - actor/entropy:0.26102858781814575 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19273829460144043 - training/rollout_probs_diff_mean:0.0026539494283497334 - training/rollout_probs_diff_std:0.007585626561194658 - training/rollout_actor_probs_pearson_corr:0.9995617866516113 - rollout_corr/rollout_is_mean:1.000087022781372 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9513788223266602 - rollout_corr/rollout_is_min:0.568621039390564 - rollout_corr/rollout_is_std:0.02511446550488472 - rollout_corr/rollout_is_eff_sample_size:0.9993698993447263 - rollout_corr/rollout_is_seq_mean:1.0000715255737305 - rollout_corr/rollout_is_seq_std:0.0013407886726781726 - rollout_corr/rollout_is_seq_max:1.0047191381454468 - rollout_corr/rollout_is_seq_min:0.9964459538459778 - rollout_corr/rollout_is_seq_max_deviation:0.004719138145446777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3035321370698512) - rollout_corr/training_log_ppl:np.float64(0.2587986676080618) - rollout_corr/kl:np.float64(0.0003129965853041483) - rollout_corr/k3_kl:np.float64(0.0003379961697334011) - rollout_corr/rollout_ppl:np.float64(1.3031289116479456) - rollout_corr/rollout_log_ppl:np.float64(0.258485670099617) - rollout_corr/log_ppl_diff:np.float64(0.0003129975084448233) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011024432169506326) - rollout_corr/log_ppl_diff_max:np.float64(0.003939390182495117) - rollout_corr/log_ppl_diff_min:np.float64(-0.004243284463882446) - rollout_corr/ppl_ratio:np.float64(1.0003140484914184) - rollout_corr/chi2_token:np.float64(0.000735103152692318) - rollout_corr/chi2_seq:np.float64(0.5922304301057011) - actor/pg_loss:np.float64(2.7036177925765514e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015170505503192544) - actor/ppo_kl:np.float64(5.9133477579464966e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17508666589856148) - perf/mfu/actor:np.float64(0.10034117675794652) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(111.28366470336914) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:1 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004022400826215744 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004022400826215744 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:420.25390625 - response_length/max:906.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.25390625 - response_length_non_aborted/max:906.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:370.5 - prompt_length/max:483.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010674819350242615 - timing_s/agent_loop/generate_sequences/min:np.float64(2.033995635807514) - timing_s/agent_loop/generate_sequences/max:np.float64(8.038469087332487) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.785468361864332) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.038469087332487) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:904 - timing_s/gen:15.00672054849565 - timing_s/reward:0.06179456599056721 - timing_s/old_log_prob:3.060260822996497 - timing_s/adv:0.11811300739645958 - timing_s/update_actor:13.051573330536485 - timing_s/step:31.38392587006092 - timing_s/stop_profile:4.0376558899879456e-05 - timing_per_token_ms/gen:0.13948710831896316 - timing_per_token_ms/adv:0.0005834671589931463 - timing_per_token_ms/update_actor:0.06447354596600596 - perf/total_num_tokens:202433 - perf/time_per_step:31.38392587006092 - perf/throughput:806.2765985608951 (TaskRunner pid=2403642) Training Progress: 45%|████▌ | 45/100 [29:23<33:45, 36.83s/it] (TaskRunner pid=2403642) step:46 - global_seqlen/min:23160 - global_seqlen/max:28431 - global_seqlen/minmax_diff:5271 - global_seqlen/balanced_min:25774 - global_seqlen/balanced_max:25777 - global_seqlen/mean:25775.375 - actor/entropy:0.2477809637784958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413540482521057 - training/rollout_probs_diff_mean:0.0025389681104570627 - training/rollout_probs_diff_std:0.007506586145609617 - training/rollout_actor_probs_pearson_corr:0.9995504021644592 - rollout_corr/rollout_is_mean:1.0000053644180298 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.605976939201355 - rollout_corr/rollout_is_min:0.5050538778305054 - rollout_corr/rollout_is_std:0.02421550080180168 - rollout_corr/rollout_is_eff_sample_size:0.9994139531569982 - rollout_corr/rollout_is_seq_mean:1.0000325441360474 - rollout_corr/rollout_is_seq_std:0.0013734105741605163 - rollout_corr/rollout_is_seq_max:1.0052671432495117 - rollout_corr/rollout_is_seq_min:0.9960712194442749 - rollout_corr/rollout_is_seq_max_deviation:0.005267143249511719 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.301206385716796) - rollout_corr/training_log_ppl:np.float64(0.25428796322376) - rollout_corr/kl:np.float64(0.0002562967966284191) - rollout_corr/k3_kl:np.float64(0.0003318889672527803) - rollout_corr/rollout_ppl:np.float64(1.3008486884646118) - rollout_corr/rollout_log_ppl:np.float64(0.25403166448813863) - rollout_corr/log_ppl_diff:np.float64(0.0002562987356213853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010766688355943188) - rollout_corr/log_ppl_diff_max:np.float64(0.0048729777336120605) - rollout_corr/log_ppl_diff_min:np.float64(-0.004036739468574524) - rollout_corr/ppl_ratio:np.float64(1.000257351435721) - rollout_corr/chi2_token:np.float64(0.0008127458859235048) - rollout_corr/chi2_seq:np.float64(0.213467349531129) - actor/pg_loss:np.float64(-2.8856913559138775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.868811063512112e-05) - actor/ppo_kl:np.float64(-1.7918505959002573e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1389861050993204) - perf/mfu/actor:np.float64(0.10119552924817618) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(149.76423168182373) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006029889453202486 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006029889453202486 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:424.48046875 - response_length/max:872.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:424.48046875 - response_length_non_aborted/max:872.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:381.0 - prompt_length/max:481.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.641230821609497e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0706292800605297) - timing_s/agent_loop/generate_sequences/max:np.float64(8.012809984385967) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.769642198676593) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.012809984385967) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:280 - timing_s/agent_loop/slowest/response_length:872 - timing_s/gen:15.134837241843343 - timing_s/reward:0.06418384425342083 - timing_s/old_log_prob:7.101993776857853 - timing_s/adv:0.15979714132845402 - timing_s/update_actor:13.256720028817654 - timing_s/step:35.80505630001426 - timing_s/stop_profile:6.691180169582367e-05 - timing_per_token_ms/gen:0.13927721609912247 - timing_per_token_ms/adv:0.0007749506133686416 - timing_per_token_ms/update_actor:0.06428965644931284 - perf/total_num_tokens:206203 - perf/time_per_step:35.80505630001426 - perf/throughput:719.8808677753632 (TaskRunner pid=2403642) Training Progress: 46%|████▌ | 46/100 [29:59<32:52, 36.53s/it] (TaskRunner pid=2403642) step:47 - global_seqlen/min:21281 - global_seqlen/max:28817 - global_seqlen/minmax_diff:7536 - global_seqlen/balanced_min:26442 - global_seqlen/balanced_max:26454 - global_seqlen/mean:26450.75 - actor/entropy:0.27556195855140686 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19444799423217773 - training/rollout_probs_diff_mean:0.002736711408942938 - training/rollout_probs_diff_std:0.0076127080246806145 - training/rollout_actor_probs_pearson_corr:0.9995213150978088 - rollout_corr/rollout_is_mean:1.0000603199005127 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.98327289178269e-06 - rollout_corr/rollout_is_max:1.6245206594467163 - rollout_corr/rollout_is_min:0.21295833587646484 - rollout_corr/rollout_is_std:0.024904737249016762 - rollout_corr/rollout_is_eff_sample_size:0.9993802575945846 - rollout_corr/rollout_is_seq_mean:1.000070571899414 - rollout_corr/rollout_is_seq_std:0.0013441572664305568 - rollout_corr/rollout_is_seq_max:1.0038058757781982 - rollout_corr/rollout_is_seq_min:0.9955684542655945 - rollout_corr/rollout_is_seq_max_deviation:0.004431545734405518 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3388532362878323) - rollout_corr/training_log_ppl:np.float64(0.2835671211942099) - rollout_corr/kl:np.float64(0.00031976593474931925) - rollout_corr/k3_kl:np.float64(0.00036261727734654414) - rollout_corr/rollout_ppl:np.float64(1.338419138919562) - rollout_corr/rollout_log_ppl:np.float64(0.2832473532325821) - rollout_corr/log_ppl_diff:np.float64(0.00031976796162780374) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011114145017927513) - rollout_corr/log_ppl_diff_max:np.float64(0.005098104476928711) - rollout_corr/log_ppl_diff_min:np.float64(-0.00347292423248291) - rollout_corr/ppl_ratio:np.float64(1.0003207672853023) - rollout_corr/chi2_token:np.float64(0.0008113444782793522) - rollout_corr/chi2_seq:np.float64(0.28644665400497615) - actor/pg_loss:np.float64(-4.764122422784567e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001026247907702782) - actor/ppo_kl:np.float64(6.258977063389182e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1877906322479248) - perf/mfu/actor:np.float64(0.10244830140367661) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(129.80683517456055) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009874862618744373 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009874862618744373 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.8359375 - response_length/max:1078.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.8359375 - response_length_non_aborted/max:1078.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.75 - prompt_length/max:490.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.805197775363922e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8029491119086742) - timing_s/agent_loop/generate_sequences/max:np.float64(9.529983641579747) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.154630884913786) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.529983641579747) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:1078 - timing_s/gen:20.076213896274567 - timing_s/reward:0.05995168723165989 - timing_s/old_log_prob:3.1332308892160654 - timing_s/adv:0.1481637991964817 - timing_s/update_actor:13.528764003887773 - timing_s/step:37.01840306445956 - timing_s/stop_profile:5.839020013809204e-05 - timing_per_token_ms/gen:0.18035011315577504 - timing_per_token_ms/adv:0.0007001871364539839 - timing_per_token_ms/update_actor:0.06393374480821798 - perf/total_num_tokens:211606 - perf/time_per_step:37.01840306445956 - perf/throughput:714.5297422458156 (TaskRunner pid=2403642) Training Progress: 47%|████▋ | 47/100 [30:36<32:24, 36.68s/it] (TaskRunner pid=2403642) step:48 - global_seqlen/min:22277 - global_seqlen/max:30571 - global_seqlen/minmax_diff:8294 - global_seqlen/balanced_min:26173 - global_seqlen/balanced_max:26178 - global_seqlen/mean:26176.25 - actor/entropy:0.2385566085577011 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24515900015830994 - training/rollout_probs_diff_mean:0.0024435243103653193 - training/rollout_probs_diff_std:0.00727314967662096 - training/rollout_actor_probs_pearson_corr:0.9995198845863342 - rollout_corr/rollout_is_mean:1.0000426769256592 - rollout_corr/rollout_is_ratio_fraction_high:9.144612704403698e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.144612704403698e-06 - rollout_corr/rollout_is_max:2.0436415672302246 - rollout_corr/rollout_is_min:0.34062984585762024 - rollout_corr/rollout_is_std:0.023853426799178123 - rollout_corr/rollout_is_eff_sample_size:0.9994313376218713 - rollout_corr/rollout_is_seq_mean:1.0000296831130981 - rollout_corr/rollout_is_seq_std:0.0014473841292783618 - rollout_corr/rollout_is_seq_max:1.0059645175933838 - rollout_corr/rollout_is_seq_min:0.9947035908699036 - rollout_corr/rollout_is_seq_max_deviation:0.005964517593383789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2863476877100766) - rollout_corr/training_log_ppl:np.float64(0.24573822057573125) - rollout_corr/kl:np.float64(0.0003334146562288254) - rollout_corr/k3_kl:np.float64(0.0003732351611951401) - rollout_corr/rollout_ppl:np.float64(1.2859108792617917) - rollout_corr/rollout_log_ppl:np.float64(0.24540480367431883) - rollout_corr/log_ppl_diff:np.float64(0.00033341690141242) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011924598220502958) - rollout_corr/log_ppl_diff_max:np.float64(0.0059059858322143555) - rollout_corr/log_ppl_diff_min:np.float64(-0.00526835024356842) - rollout_corr/ppl_ratio:np.float64(1.0003347243182361) - rollout_corr/chi2_token:np.float64(0.0008353295270353556) - rollout_corr/chi2_seq:np.float64(0.3313969324808568) - actor/pg_loss:np.float64(-4.039297346025705e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.439815504360013e-05) - actor/ppo_kl:np.float64(6.001183691495271e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09218540880829096) - perf/mfu/actor:np.float64(0.10079389831904431) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(123.64847230911255) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:1 - critic/score/mean:0.85546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.85546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0028942700009793043 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0028942700009793043 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:427.1640625 - response_length/max:825.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:427.1640625 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.84375 - prompt_length/max:488.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.29004430770874e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9806438311934471) - timing_s/agent_loop/generate_sequences/max:np.float64(7.873849323019385) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.021167605707888) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.873849323019385) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:479 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:14.795848157256842 - timing_s/reward:0.06539638713002205 - timing_s/old_log_prob:3.167770102620125 - timing_s/adv:0.1251517105847597 - timing_s/update_actor:13.308212032541633 - timing_s/step:31.549153676256537 - timing_s/stop_profile:0.00012654997408390045 - timing_per_token_ms/gen:0.1353023040515833 - timing_per_token_ms/adv:0.0005976396093059534 - timing_per_token_ms/update_actor:0.06355098625921222 - perf/total_num_tokens:209410 - perf/time_per_step:31.549153676256537 - perf/throughput:829.6973753593868 (TaskRunner pid=2403642) Training Progress: 48%|████▊ | 48/100 [31:07<30:28, 35.16s/it] (TaskRunner pid=2403642) step:49 - global_seqlen/min:21747 - global_seqlen/max:29435 - global_seqlen/minmax_diff:7688 - global_seqlen/balanced_min:25439 - global_seqlen/balanced_max:25505 - global_seqlen/mean:25449.75 - actor/entropy:0.254334956407547 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25539761781692505 - training/rollout_probs_diff_mean:0.002560534281656146 - training/rollout_probs_diff_std:0.007452011574059725 - training/rollout_actor_probs_pearson_corr:0.999518096446991 - rollout_corr/rollout_is_mean:0.9999451041221619 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.916302219091449e-05 - rollout_corr/rollout_is_max:1.7746413946151733 - rollout_corr/rollout_is_min:0.41666513681411743 - rollout_corr/rollout_is_std:0.024758314713835716 - rollout_corr/rollout_is_eff_sample_size:0.9993874013402829 - rollout_corr/rollout_is_seq_mean:0.9999779462814331 - rollout_corr/rollout_is_seq_std:0.0014876748900860548 - rollout_corr/rollout_is_seq_max:1.0042344331741333 - rollout_corr/rollout_is_seq_min:0.994238555431366 - rollout_corr/rollout_is_seq_max_deviation:0.005761444568634033 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3119332161732018) - rollout_corr/training_log_ppl:np.float64(0.264481845399132) - rollout_corr/kl:np.float64(0.0004063297049228254) - rollout_corr/k3_kl:np.float64(0.0003945719078330967) - rollout_corr/rollout_ppl:np.float64(1.311392041388899) - rollout_corr/rollout_log_ppl:np.float64(0.26407551308511756) - rollout_corr/log_ppl_diff:np.float64(0.0004063323140144348) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013702319120056927) - rollout_corr/log_ppl_diff_max:np.float64(0.007741332054138184) - rollout_corr/log_ppl_diff_min:np.float64(-0.004774883389472961) - rollout_corr/ppl_ratio:np.float64(1.0004080620128661) - rollout_corr/chi2_token:np.float64(0.000757753849029541) - rollout_corr/chi2_seq:np.float64(0.3059838619083166) - actor/pg_loss:np.float64(5.814828909933567e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010606717933114851) - actor/ppo_kl:np.float64(5.833629540230589e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16979572549462318) - perf/mfu/actor:np.float64(0.1016400765494086) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(113.97234725952148) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:1 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00041071255691349506 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00041071255691349506 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:401.8359375 - response_length/max:1363.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:401.8359375 - response_length_non_aborted/max:1363.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:393.46875 - prompt_length/max:486.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010422617197036743 - timing_s/agent_loop/generate_sequences/min:np.float64(1.753613343462348) - timing_s/agent_loop/generate_sequences/max:np.float64(10.484201986342669) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.581973708147416) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.484201986342669) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:1363 - timing_s/gen:17.40170544758439 - timing_s/reward:0.06906182132661343 - timing_s/old_log_prob:3.1587615106254816 - timing_s/adv:0.14425324462354183 - timing_s/update_actor:13.074285212904215 - timing_s/step:33.933415714651346 - timing_s/stop_profile:0.00012540258467197418 - timing_per_token_ms/gen:0.16916210214430244 - timing_per_token_ms/adv:0.0007085199492310427 - timing_per_token_ms/update_actor:0.06421617703957905 - perf/total_num_tokens:203598 - perf/time_per_step:33.933415714651346 - perf/throughput:749.9908118301107 (TaskRunner pid=2403642) Training Progress: 49%|████▉ | 49/100 [31:41<29:35, 34.81s/it] (TaskRunner pid=2403642) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2403642) validation generation end (TaskRunner pid=2403642) [prompt] system (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2403642) (TaskRunner pid=2403642) A (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) user (TaskRunner pid=2403642) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2403642) (TaskRunner pid=2403642) Lattice Parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) a = 7.82 Å (TaskRunner pid=2403642) b = 7.82 Å (TaskRunner pid=2403642) c = 17.88 Å (TaskRunner pid=2403642) α = 90.00º (TaskRunner pid=2403642) β = 90.00º (TaskRunner pid=2403642) γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) A: 1025.67 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) B: 1094.31 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) C: 1200.45 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) D: 1150.78 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) Please reason step by step. (TaskRunner pid=2403642) assistant (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) [response] (TaskRunner pid=2403642) To calculate the volume of a unit cell, we use the formula for the volume of a unit cell in a crystal structure: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \times b \times c \times \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) In this case, the crystal structure is described by lattice parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) - a = 7.82 Å (TaskRunner pid=2403642) - b = 7.82 Å (TaskRunner pid=2403642) - c = 17.88 Å (TaskRunner pid=2403642) - α = 90.00º (TaskRunner pid=2403642) - β = 90.00º (TaskRunner pid=2403642) - γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) Since α, β, and γ are all 90 degrees, the crystal is a cubic structure (or at least has orthogonal axes). For a cubic crystal, the volume simplifies to: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \times b \times c $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Substituting the given values: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Now, multiply by 17.88: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Therefore, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) B (TaskRunner pid=2403642) (TaskRunner pid=2403642) [ground_truth] B (TaskRunner pid=2403642) [score] 1.0 (TaskRunner pid=2403642) [acc] 1.0 (TaskRunner pid=2403642) [pred] B (TaskRunner pid=2403642) [incorrect_format] 1 (TaskRunner pid=2403642) [feedback] (TaskRunner pid=2403642) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2403642) step:50 - global_seqlen/min:18531 - global_seqlen/max:28752 - global_seqlen/minmax_diff:10221 - global_seqlen/balanced_min:24590 - global_seqlen/balanced_max:24595 - global_seqlen/mean:24592.875 - actor/entropy:0.22492657601833344 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7781625986099243 - training/rollout_probs_diff_mean:0.002371629234403372 - training/rollout_probs_diff_std:0.00784727931022644 - training/rollout_actor_probs_pearson_corr:0.9994186162948608 - rollout_corr/rollout_is_mean:0.9999779462814331 - rollout_corr/rollout_is_ratio_fraction_high:1.0639316315064207e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0639316315064207e-05 - rollout_corr/rollout_is_max:2.165529489517212 - rollout_corr/rollout_is_min:0.04572895169258118 - rollout_corr/rollout_is_std:0.02394072338938713 - rollout_corr/rollout_is_eff_sample_size:0.999427170058055 - rollout_corr/rollout_is_seq_mean:1.0000346899032593 - rollout_corr/rollout_is_seq_std:0.001483708037994802 - rollout_corr/rollout_is_seq_max:1.006122350692749 - rollout_corr/rollout_is_seq_min:0.9957391619682312 - rollout_corr/rollout_is_seq_max_deviation:0.0061223506927490234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2762280572205782) - rollout_corr/training_log_ppl:np.float64(0.2360334648838034) - rollout_corr/kl:np.float64(0.0004081390215331382) - rollout_corr/k3_kl:np.float64(0.0004092514230435995) - rollout_corr/rollout_ppl:np.float64(1.2756929220631719) - rollout_corr/rollout_log_ppl:np.float64(0.23562532204960007) - rollout_corr/log_ppl_diff:np.float64(0.00040814283420331776) - rollout_corr/log_ppl_abs_diff:np.float64(0.001212061324622482) - rollout_corr/log_ppl_diff_max:np.float64(0.015186533331871033) - rollout_corr/log_ppl_diff_min:np.float64(-0.003930956125259399) - rollout_corr/ppl_ratio:np.float64(1.0004097421187907) - rollout_corr/chi2_token:np.float64(0.0007466780953109264) - rollout_corr/chi2_seq:np.float64(0.2214487304445356) - actor/pg_loss:np.float64(5.0445785745978355e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.287216749318759e-05) - actor/ppo_kl:np.float64(0.00011426370583222933) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1364841517060995) - perf/mfu/actor:np.float64(0.0992731123109947) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(114.06961441040039) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7406914893617021) - val-aux/sciknoweval/reward/std@16:np.float64(0.06782258470826572) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7657872340425533) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.04877188049764316) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7148404255319148) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.06455783589932332) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7409255319148936) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.06713673383437797) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7843297872340426) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.03278506425382436) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6915957446808512) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.05908327474495294) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7453297872340426) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.049206425903886845) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7974042553191488) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.01955746993586378) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6668829787234042) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.051961064875554894) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7445212765957446) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.031088513575260223) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8041808510638296) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.00968861258916768) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6427446808510638) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04082562011514557) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7425106382978722) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.022038691077849427) - val-aux/sciknoweval/score/mean@16:np.float64(0.7406914893617021) - val-aux/sciknoweval/score/std@16:np.float64(0.06782258470826572) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7657872340425533) - val-aux/sciknoweval/score/best@2/std:np.float64(0.04877188049764316) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7148404255319148) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.06455783589932332) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7409255319148936) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.06713673383437797) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7843297872340426) - val-aux/sciknoweval/score/best@4/std:np.float64(0.03278506425382436) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6915957446808512) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.05908327474495294) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7453297872340426) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.049206425903886845) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7974042553191488) - val-aux/sciknoweval/score/best@8/std:np.float64(0.01955746993586378) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6668829787234042) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.051961064875554894) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7445212765957446) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.031088513575260223) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8041808510638296) - val-aux/sciknoweval/score/best@16/std:np.float64(0.00968861258916768) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6427446808510638) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04082562011514557) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7425106382978722) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.022038691077849427) - val-core/sciknoweval/acc/mean@16:np.float64(0.7406914893617021) - val-aux/sciknoweval/acc/std@16:np.float64(0.06782258470826572) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7657872340425533) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.04877188049764316) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7148404255319148) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.06455783589932332) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7409255319148936) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.06713673383437797) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7843297872340426) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.03278506425382436) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6915957446808512) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.05908327474495294) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7453297872340426) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.049206425903886845) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7974042553191488) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.01955746993586378) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6668829787234042) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.051961064875554894) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7445212765957446) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.031088513575260223) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8041808510638296) - val-core/sciknoweval/acc/best@16/std:np.float64(0.00968861258916768) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6427446808510638) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04082562011514557) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7425106382978722) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.022038691077849427) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:1 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005066974554210901 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.005066974554210901 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:367.15234375 - response_length/max:793.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:367.15234375 - response_length_non_aborted/max:793.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.375 - prompt_length/max:492.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.704327046871185e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.972106758505106) - timing_s/agent_loop/generate_sequences/max:np.float64(7.237190572544932) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.16025916540093) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.237190572544932) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:492 - timing_s/agent_loop/slowest/response_length:774 - timing_s/gen:13.96083409152925 - timing_s/reward:0.06105468235909939 - timing_s/old_log_prob:3.0737576838582754 - timing_s/adv:0.11763521283864975 - timing_s/update_actor:12.941408473998308 - timing_s/step:30.238582840189338 - timing_s/testing:45.02190750092268 - timing_s/save_checkpoint:14.179249055683613 - timing_s/stop_profile:0.0001403260976076126 - timing_per_token_ms/gen:0.148533732926868 - timing_per_token_ms/adv:0.0005979130786795451 - timing_per_token_ms/update_actor:0.06577824102508505 - perf/total_num_tokens:196743 - perf/time_per_step:30.238582840189338 - perf/throughput:813.294562446036 (TaskRunner pid=2403642) Training Progress: 50%|█████ | 50/100 [33:11<42:40, 51.22s/it] (TaskRunner pid=2403642) step:51 - global_seqlen/min:21651 - global_seqlen/max:30533 - global_seqlen/minmax_diff:8882 - global_seqlen/balanced_min:24618 - global_seqlen/balanced_max:24626 - global_seqlen/mean:24623.375 - actor/entropy:0.2378806620836258 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31757521629333496 - training/rollout_probs_diff_mean:0.0024352639447897673 - training/rollout_probs_diff_std:0.007294767536222935 - training/rollout_actor_probs_pearson_corr:0.999496579170227 - rollout_corr/rollout_is_mean:1.0000085830688477 - rollout_corr/rollout_is_ratio_fraction_high:1.0320023648091592e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.7408487796783447 - rollout_corr/rollout_is_min:0.5327962040901184 - rollout_corr/rollout_is_std:0.024035144597291946 - rollout_corr/rollout_is_eff_sample_size:0.9994226453138363 - rollout_corr/rollout_is_seq_mean:0.9999585747718811 - rollout_corr/rollout_is_seq_std:0.0015011404175311327 - rollout_corr/rollout_is_seq_max:1.004288673400879 - rollout_corr/rollout_is_seq_min:0.9942871928215027 - rollout_corr/rollout_is_seq_max_deviation:0.0057128071784973145 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3018347015604377) - rollout_corr/training_log_ppl:np.float64(0.2531574094318785) - rollout_corr/kl:np.float64(0.00036208893200040393) - rollout_corr/k3_kl:np.float64(0.00035660695331785064) - rollout_corr/rollout_ppl:np.float64(1.3012964907102287) - rollout_corr/rollout_log_ppl:np.float64(0.25279531987325754) - rollout_corr/log_ppl_diff:np.float64(0.00036208955862093717) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011715776199707761) - rollout_corr/log_ppl_diff_max:np.float64(0.006238698959350586) - rollout_corr/log_ppl_diff_min:np.float64(-0.003474891185760498) - rollout_corr/ppl_ratio:np.float64(1.0003633103333414) - rollout_corr/chi2_token:np.float64(0.0007233740761876106) - rollout_corr/chi2_seq:np.float64(0.3191821831278503) - actor/pg_loss:np.float64(0.00010972644668072462) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.342345659504645e-05) - actor/ppo_kl:np.float64(6.792625837803712e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09654535725712776) - perf/mfu/actor:np.float64(0.10045939727212769) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(107.78156566619873) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:1 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000906872097402811 - critic/advantages/max:0.875 - critic/advantages/min:-0.25 - critic/returns/mean:0.000906872097402811 - critic/returns/max:0.875 - critic/returns/min:-0.25 - response_length/mean:378.51171875 - response_length/max:976.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:378.51171875 - response_length_non_aborted/max:976.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.96875 - prompt_length/max:519.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012427382171154022 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5273418612778187) - timing_s/agent_loop/generate_sequences/max:np.float64(8.357269065454602) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9491922540328233) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.357269065454602) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:976 - timing_s/gen:14.852619636803865 - timing_s/reward:0.05984634719789028 - timing_s/old_log_prob:3.2959805000573397 - timing_s/adv:0.1256413720548153 - timing_s/update_actor:12.754862451925874 - timing_s/step:31.188012816011906 - timing_s/stop_profile:0.00012112408876419067 - timing_per_token_ms/gen:0.15327939026000129 - timing_per_token_ms/adv:0.0006378155515583023 - timing_per_token_ms/update_actor:0.06474976750712419 - perf/total_num_tokens:196987 - perf/time_per_step:31.188012816011906 - perf/throughput:789.5140721296092 (TaskRunner pid=2403642) Training Progress: 51%|█████ | 51/100 [33:42<36:56, 45.23s/it] (TaskRunner pid=2403642) step:52 - global_seqlen/min:23073 - global_seqlen/max:29801 - global_seqlen/minmax_diff:6728 - global_seqlen/balanced_min:26508 - global_seqlen/balanced_max:26513 - global_seqlen/mean:26511.0 - actor/entropy:0.2446550875902176 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2773001194000244 - training/rollout_probs_diff_mean:0.0023589127231389284 - training/rollout_probs_diff_std:0.007042487617582083 - training/rollout_actor_probs_pearson_corr:0.9995072484016418 - rollout_corr/rollout_is_mean:0.9999248385429382 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.183073416352272e-06 - rollout_corr/rollout_is_max:1.4406086206436157 - rollout_corr/rollout_is_min:0.44539976119995117 - rollout_corr/rollout_is_std:0.02308904379606247 - rollout_corr/rollout_is_eff_sample_size:0.9994670610232405 - rollout_corr/rollout_is_seq_mean:0.9999663233757019 - rollout_corr/rollout_is_seq_std:0.001290986081585288 - rollout_corr/rollout_is_seq_max:1.004225730895996 - rollout_corr/rollout_is_seq_min:0.995941162109375 - rollout_corr/rollout_is_seq_max_deviation:0.004225730895996094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2991151995956898) - rollout_corr/training_log_ppl:np.float64(0.2535376882078708) - rollout_corr/kl:np.float64(0.0003585750972416868) - rollout_corr/k3_kl:np.float64(0.00032570855380242847) - rollout_corr/rollout_ppl:np.float64(1.2986278631724417) - rollout_corr/rollout_log_ppl:np.float64(0.2531791118817637) - rollout_corr/log_ppl_diff:np.float64(0.00035857632610714063) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011525552763487212) - rollout_corr/log_ppl_diff_max:np.float64(0.004387974739074707) - rollout_corr/log_ppl_diff_min:np.float64(-0.006045117974281311) - rollout_corr/ppl_ratio:np.float64(1.0003596900496632) - rollout_corr/chi2_token:np.float64(0.0005872410256415606) - rollout_corr/chi2_seq:np.float64(0.2268612238112837) - actor/pg_loss:np.float64(-8.898775558918715e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.063289280675235e-05) - actor/ppo_kl:np.float64(3.672884257088782e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1358618326485157) - perf/mfu/actor:np.float64(0.1070155509069357) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(108.36789321899414) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:1 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0010089902207255363 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0010089902207255363 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:425.375 - response_length/max:767.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:425.375 - response_length_non_aborted/max:767.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:403.09375 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014198385179042816 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3870915044099092) - timing_s/agent_loop/generate_sequences/max:np.float64(7.209513144567609) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.068675200593134) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.209513144567609) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:406 - timing_s/agent_loop/slowest/response_length:759 - timing_s/gen:13.877084165811539 - timing_s/reward:0.0635684635490179 - timing_s/old_log_prob:3.1054749377071857 - timing_s/adv:0.1410121750086546 - timing_s/update_actor:12.906041007488966 - timing_s/step:30.17966585420072 - timing_s/stop_profile:3.808364272117615e-05 - timing_per_token_ms/gen:0.12743428744684412 - timing_per_token_ms/adv:0.0006648757827347827 - timing_per_token_ms/update_actor:0.06085229247995627 - perf/total_num_tokens:212088 - perf/time_per_step:30.17966585420072 - perf/throughput:878.4391493290811 (TaskRunner pid=2403642) Training Progress: 52%|█████▏ | 52/100 [34:12<32:34, 40.72s/it] (TaskRunner pid=2403642) step:53 - global_seqlen/min:21755 - global_seqlen/max:28435 - global_seqlen/minmax_diff:6680 - global_seqlen/balanced_min:24298 - global_seqlen/balanced_max:24302 - global_seqlen/mean:24300.625 - actor/entropy:0.24457094073295593 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5994806289672852 - training/rollout_probs_diff_mean:0.0024542806204408407 - training/rollout_probs_diff_std:0.007331932429224253 - training/rollout_actor_probs_pearson_corr:0.9994651675224304 - rollout_corr/rollout_is_mean:1.000049114227295 - rollout_corr/rollout_is_ratio_fraction_high:2.151717671949882e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.587857246398926 - rollout_corr/rollout_is_min:0.7702428102493286 - rollout_corr/rollout_is_std:0.02378586307168007 - rollout_corr/rollout_is_eff_sample_size:0.9994346716979495 - rollout_corr/rollout_is_seq_mean:1.00004243850708 - rollout_corr/rollout_is_seq_std:0.0014012084575369954 - rollout_corr/rollout_is_seq_max:1.005181908607483 - rollout_corr/rollout_is_seq_min:0.9958944916725159 - rollout_corr/rollout_is_seq_max_deviation:0.00518190860748291 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.289477339014411) - rollout_corr/training_log_ppl:np.float64(0.24894744424091186) - rollout_corr/kl:np.float64(0.0003029550313176088) - rollout_corr/k3_kl:np.float64(0.0003627692941421401) - rollout_corr/rollout_ppl:np.float64(1.289075882639736) - rollout_corr/rollout_log_ppl:np.float64(0.2486444886744721) - rollout_corr/log_ppl_diff:np.float64(0.0003029555664397776) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011978519905824214) - rollout_corr/log_ppl_diff_max:np.float64(0.0045301467180252075) - rollout_corr/log_ppl_diff_min:np.float64(-0.004763618111610413) - rollout_corr/ppl_ratio:np.float64(1.0003041608724743) - rollout_corr/chi2_token:np.float64(0.0008704829961061478) - rollout_corr/chi2_seq:np.float64(0.24797223578207195) - actor/pg_loss:np.float64(-0.00013095582835376263) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.983969549561152e-05) - actor/ppo_kl:np.float64(4.8199360510814415e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1310713840648532) - perf/mfu/actor:np.float64(0.09757008713741944) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.20830535888672) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020118560642004013 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020118560642004013 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:363.08203125 - response_length/max:798.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:363.08203125 - response_length_non_aborted/max:798.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:502.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00022395513951778412 - timing_s/agent_loop/generate_sequences/min:np.float64(1.381840230897069) - timing_s/agent_loop/generate_sequences/max:np.float64(6.928327467292547) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.167328959978477) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.928327467292547) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:502 - timing_s/agent_loop/slowest/response_length:798 - timing_s/gen:13.77582342736423 - timing_s/reward:0.05633910559117794 - timing_s/old_log_prob:3.225114507600665 - timing_s/adv:0.12230853363871574 - timing_s/update_actor:13.057347904890776 - timing_s/step:30.358734093606472 - timing_s/stop_profile:0.00011031143367290497 - timing_per_token_ms/gen:0.1482084092068148 - timing_per_token_ms/adv:0.0006291429419959144 - timing_per_token_ms/update_actor:0.06716569998143451 - perf/total_num_tokens:194405 - perf/time_per_step:30.358734093606472 - perf/throughput:800.4492191628535 (TaskRunner pid=2403642) Training Progress: 53%|█████▎ | 53/100 [34:43<29:34, 37.75s/it] (TaskRunner pid=2403642) step:54 - global_seqlen/min:21771 - global_seqlen/max:29996 - global_seqlen/minmax_diff:8225 - global_seqlen/balanced_min:26633 - global_seqlen/balanced_max:26640 - global_seqlen/mean:26637.625 - actor/entropy:0.22380009293556213 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22949397563934326 - training/rollout_probs_diff_mean:0.0022567082196474075 - training/rollout_probs_diff_std:0.006998219061642885 - training/rollout_actor_probs_pearson_corr:0.9995344877243042 - rollout_corr/rollout_is_mean:1.0000628232955933 - rollout_corr/rollout_is_ratio_fraction_high:9.23284278542269e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.167020082473755 - rollout_corr/rollout_is_min:0.5657718777656555 - rollout_corr/rollout_is_std:0.02301405742764473 - rollout_corr/rollout_is_eff_sample_size:0.9994706335038355 - rollout_corr/rollout_is_seq_mean:1.0000866651535034 - rollout_corr/rollout_is_seq_std:0.0013085813261568546 - rollout_corr/rollout_is_seq_max:1.0036485195159912 - rollout_corr/rollout_is_seq_min:0.9966868162155151 - rollout_corr/rollout_is_seq_max_deviation:0.003648519515991211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2876649326644838) - rollout_corr/training_log_ppl:np.float64(0.24327065303805284) - rollout_corr/kl:np.float64(0.00037481868857369705) - rollout_corr/k3_kl:np.float64(0.0003726331062168242) - rollout_corr/rollout_ppl:np.float64(1.2871439228765666) - rollout_corr/rollout_log_ppl:np.float64(0.24289583274367033) - rollout_corr/log_ppl_diff:np.float64(0.0003748202943825163) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011444683550507762) - rollout_corr/log_ppl_diff_max:np.float64(0.006301254034042358) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038700103759765625) - rollout_corr/ppl_ratio:np.float64(1.0003759877290577) - rollout_corr/chi2_token:np.float64(0.0007948835846036673) - rollout_corr/chi2_seq:np.float64(0.26042517158202827) - actor/pg_loss:np.float64(-0.0001082469243556261) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.538802492490504e-05) - actor/ppo_kl:np.float64(0.0001657101322258825) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11135829612612724) - perf/mfu/actor:np.float64(0.1082633724102717) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.19610595703125) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00029083454865030944 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00029083454865030944 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:423.08203125 - response_length/max:969.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.08203125 - response_length_non_aborted/max:969.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:409.34375 - prompt_length/max:524.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015332549810409546 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0272562000900507) - timing_s/agent_loop/generate_sequences/max:np.float64(8.707748917862773) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7737743248508195) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.707748917862773) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:969 - timing_s/gen:15.431535312905908 - timing_s/reward:0.06327725574374199 - timing_s/old_log_prob:3.0882180761545897 - timing_s/adv:0.1180159579962492 - timing_s/update_actor:12.672546118497849 - timing_s/step:31.4620493222028 - timing_s/stop_profile:0.00011998787522315979 - timing_per_token_ms/gen:0.1424769438634454 - timing_per_token_ms/adv:0.0005538029291099019 - timing_per_token_ms/update_actor:0.05946732356252598 - perf/total_num_tokens:213101 - perf/time_per_step:31.4620493222028 - perf/throughput:846.6589295313894 (TaskRunner pid=2403642) Training Progress: 54%|█████▍ | 54/100 [35:15<27:30, 35.88s/it] (TaskRunner pid=2403642) step:55 - global_seqlen/min:22757 - global_seqlen/max:30719 - global_seqlen/minmax_diff:7962 - global_seqlen/balanced_min:26797 - global_seqlen/balanced_max:26801 - global_seqlen/mean:26799.75 - actor/entropy:0.2322537750005722 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016374111175537 - training/rollout_probs_diff_mean:0.002216401044279337 - training/rollout_probs_diff_std:0.00684860534965992 - training/rollout_actor_probs_pearson_corr:0.9995991587638855 - rollout_corr/rollout_is_mean:0.9999573826789856 - rollout_corr/rollout_is_ratio_fraction_high:9.197432518703863e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.8394865037407726e-05 - rollout_corr/rollout_is_max:2.096909761428833 - rollout_corr/rollout_is_min:0.3293769061565399 - rollout_corr/rollout_is_std:0.02277715504169464 - rollout_corr/rollout_is_eff_sample_size:0.9994813510988554 - rollout_corr/rollout_is_seq_mean:0.9999474883079529 - rollout_corr/rollout_is_seq_std:0.0012583937495946884 - rollout_corr/rollout_is_seq_max:1.006894826889038 - rollout_corr/rollout_is_seq_min:0.9962959885597229 - rollout_corr/rollout_is_seq_max_deviation:0.006894826889038086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2898584022186697) - rollout_corr/training_log_ppl:np.float64(0.24666903459728928) - rollout_corr/kl:np.float64(0.0002695444498144184) - rollout_corr/k3_kl:np.float64(0.00033902103465521805) - rollout_corr/rollout_ppl:np.float64(1.2895028642378747) - rollout_corr/rollout_log_ppl:np.float64(0.24639948902040487) - rollout_corr/log_ppl_diff:np.float64(0.0002695455768844113) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011074139329139143) - rollout_corr/log_ppl_diff_max:np.float64(0.004107072949409485) - rollout_corr/log_ppl_diff_min:np.float64(-0.004751652479171753) - rollout_corr/ppl_ratio:np.float64(1.0002705552615225) - rollout_corr/chi2_token:np.float64(0.0008234675042331219) - rollout_corr/chi2_seq:np.float64(0.48434280999936163) - actor/pg_loss:np.float64(4.3725594878196716e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.63317041976552e-05) - actor/ppo_kl:np.float64(-7.033058565752981e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1018903236836195) - perf/mfu/actor:np.float64(0.10775877878208463) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.22493743896484) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023579916451126337 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023579916451126337 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:424.7109375 - response_length/max:954.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:424.7109375 - response_length_non_aborted/max:954.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.78125 - prompt_length/max:519.0 - prompt_length/min:271.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013876333832740784 - timing_s/agent_loop/generate_sequences/min:np.float64(2.095417447388172) - timing_s/agent_loop/generate_sequences/max:np.float64(8.385497575625777) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.71868045329029) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.385497575625777) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:954 - timing_s/gen:15.350444003939629 - timing_s/reward:0.0659722164273262 - timing_s/old_log_prob:3.0554297920316458 - timing_s/adv:0.11823453195393085 - timing_s/update_actor:12.858605690300465 - timing_s/step:31.537192882969975 - timing_s/stop_profile:0.0001096036285161972 - timing_per_token_ms/gen:0.14118466607747576 - timing_per_token_ms/adv:0.0005514721776972306 - timing_per_token_ms/update_actor:0.0599753994454261 - perf/total_num_tokens:214398 - perf/time_per_step:31.537192882969975 - perf/throughput:849.7823537893829 (TaskRunner pid=2403642) Training Progress: 55%|█████▌ | 55/100 [35:46<25:56, 34.59s/it] (TaskRunner pid=2403642) step:56 - global_seqlen/min:18837 - global_seqlen/max:31049 - global_seqlen/minmax_diff:12212 - global_seqlen/balanced_min:25345 - global_seqlen/balanced_max:25347 - global_seqlen/mean:25346.0 - actor/entropy:0.25467386841773987 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491804838180542 - training/rollout_probs_diff_mean:0.002417678711935878 - training/rollout_probs_diff_std:0.007147489581257105 - training/rollout_actor_probs_pearson_corr:0.9995514154434204 - rollout_corr/rollout_is_mean:0.9999701976776123 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5092898607254028 - rollout_corr/rollout_is_min:0.5352914929389954 - rollout_corr/rollout_is_std:0.023758282884955406 - rollout_corr/rollout_is_eff_sample_size:0.999435862444797 - rollout_corr/rollout_is_seq_mean:0.9999875426292419 - rollout_corr/rollout_is_seq_std:0.0014447674620896578 - rollout_corr/rollout_is_seq_max:1.004019021987915 - rollout_corr/rollout_is_seq_min:0.9958305358886719 - rollout_corr/rollout_is_seq_max_deviation:0.004169464111328125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3189649819396436) - rollout_corr/training_log_ppl:np.float64(0.26553016485559056) - rollout_corr/kl:np.float64(0.00045735904856680776) - rollout_corr/k3_kl:np.float64(0.0003336688204953475) - rollout_corr/rollout_ppl:np.float64(1.3182676555588841) - rollout_corr/rollout_log_ppl:np.float64(0.2650728039225214) - rollout_corr/log_ppl_diff:np.float64(0.0004573609330691397) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012234431924298406) - rollout_corr/log_ppl_diff_max:np.float64(0.005470991134643555) - rollout_corr/log_ppl_diff_min:np.float64(-0.003992438316345215) - rollout_corr/ppl_ratio:np.float64(1.0004586917348206) - rollout_corr/chi2_token:np.float64(0.0004137852229177952) - rollout_corr/chi2_seq:np.float64(0.20288011711090803) - actor/pg_loss:np.float64(-4.3228152208030224e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.432743940516957e-05) - actor/ppo_kl:np.float64(0.00014076509722471542) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06149144750088453) - perf/mfu/actor:np.float64(0.10336162570898155) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.06003189086914) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.78515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024021996650844812 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024021996650844812 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:410.59375 - response_length/max:1051.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:410.59375 - response_length_non_aborted/max:1051.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:381.46875 - prompt_length/max:480.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011040642857551575 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2151698395609856) - timing_s/agent_loop/generate_sequences/max:np.float64(8.617751153185964) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9353341244204785) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.617751153185964) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:1051 - timing_s/gen:15.921105803921819 - timing_s/reward:0.06280032359063625 - timing_s/old_log_prob:3.02948241494596 - timing_s/adv:0.12095407582819462 - timing_s/update_actor:12.801099264994264 - timing_s/step:32.021815275773406 - timing_s/stop_profile:0.0001158788800239563 - timing_per_token_ms/gen:0.15146801320421854 - timing_per_token_ms/adv:0.0005965146168438541 - timing_per_token_ms/update_actor:0.06313175286531536 - perf/total_num_tokens:202768 - perf/time_per_step:32.021815275773406 - perf/throughput:791.5228971786588 (TaskRunner pid=2403642) Training Progress: 56%|█████▌ | 56/100 [36:18<24:48, 33.84s/it] (TaskRunner pid=2403642) step:57 - global_seqlen/min:20714 - global_seqlen/max:27341 - global_seqlen/minmax_diff:6627 - global_seqlen/balanced_min:24062 - global_seqlen/balanced_max:24063 - global_seqlen/mean:24062.75 - actor/entropy:0.247794970870018 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163609981536865 - training/rollout_probs_diff_mean:0.0026478164363652468 - training/rollout_probs_diff_std:0.00758332060649991 - training/rollout_actor_probs_pearson_corr:0.9995037913322449 - rollout_corr/rollout_is_mean:0.9999039769172668 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2746400645701215e-05 - rollout_corr/rollout_is_max:1.474251627922058 - rollout_corr/rollout_is_min:0.4279352128505707 - rollout_corr/rollout_is_std:0.025090720504522324 - rollout_corr/rollout_is_eff_sample_size:0.9993709708778578 - rollout_corr/rollout_is_seq_mean:0.9998193979263306 - rollout_corr/rollout_is_seq_std:0.00137531163636595 - rollout_corr/rollout_is_seq_max:1.0043578147888184 - rollout_corr/rollout_is_seq_min:0.9959853291511536 - rollout_corr/rollout_is_seq_max_deviation:0.004357814788818359 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2967904824763536) - rollout_corr/training_log_ppl:np.float64(0.2543451767269289) - rollout_corr/kl:np.float64(0.000545830731464747) - rollout_corr/k3_kl:np.float64(0.0003583562258455686) - rollout_corr/rollout_ppl:np.float64(1.296079382300377) - rollout_corr/rollout_log_ppl:np.float64(0.25379934372904245) - rollout_corr/log_ppl_diff:np.float64(0.000545832997886464) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012242147058714181) - rollout_corr/log_ppl_diff_max:np.float64(0.0052535682916641235) - rollout_corr/log_ppl_diff_min:np.float64(-0.00345398485660553) - rollout_corr/ppl_ratio:np.float64(1.0005470658652484) - rollout_corr/chi2_token:np.float64(0.0003389487974345684) - rollout_corr/chi2_seq:np.float64(0.048471454065293074) - actor/pg_loss:np.float64(-4.220055416226387e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.6725049792730715e-05) - actor/ppo_kl:np.float64(3.711784023319176e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08370932471007109) - perf/mfu/actor:np.float64(0.09839206597889999) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(105.99935150146484) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006781270494684577 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.0006781270494684577 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:343.4609375 - response_length/max:884.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.4609375 - response_length_non_aborted/max:884.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:408.5 - prompt_length/max:499.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.432699203491211e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1311710253357887) - timing_s/agent_loop/generate_sequences/max:np.float64(7.473232988268137) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.930545251489093) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.473232988268137) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:884 - timing_s/gen:14.094484962522984 - timing_s/reward:0.061067309230566025 - timing_s/old_log_prob:3.0981795713305473 - timing_s/adv:0.12110709398984909 - timing_s/update_actor:12.573839627206326 - timing_s/step:30.03517392836511 - timing_s/stop_profile:0.0001140441745519638 - timing_per_token_ms/gen:0.1602993990687963 - timing_per_token_ms/adv:0.000629121224661817 - timing_per_token_ms/update_actor:0.06531796878581171 - perf/total_num_tokens:192502 - perf/time_per_step:30.03517392836511 - perf/throughput:801.1523441612311 (TaskRunner pid=2403642) Training Progress: 57%|█████▋ | 57/100 [36:48<23:26, 32.71s/it] (TaskRunner pid=2403642) step:58 - global_seqlen/min:23061 - global_seqlen/max:28276 - global_seqlen/minmax_diff:5215 - global_seqlen/balanced_min:25176 - global_seqlen/balanced_max:25182 - global_seqlen/mean:25179.875 - actor/entropy:0.21638013422489166 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083588361740112 - training/rollout_probs_diff_mean:0.002310627605766058 - training/rollout_probs_diff_std:0.007523712236434221 - training/rollout_actor_probs_pearson_corr:0.9994233250617981 - rollout_corr/rollout_is_mean:1.0000600814819336 - rollout_corr/rollout_is_ratio_fraction_high:9.657077498559374e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.931415499711875e-05 - rollout_corr/rollout_is_max:2.903904676437378 - rollout_corr/rollout_is_min:0.3459939956665039 - rollout_corr/rollout_is_std:0.0234451275318861 - rollout_corr/rollout_is_eff_sample_size:0.999450627941437 - rollout_corr/rollout_is_seq_mean:1.0000741481781006 - rollout_corr/rollout_is_seq_std:0.0013978402130305767 - rollout_corr/rollout_is_seq_max:1.0069079399108887 - rollout_corr/rollout_is_seq_min:0.9964627623558044 - rollout_corr/rollout_is_seq_max_deviation:0.006907939910888672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2778812977485359) - rollout_corr/training_log_ppl:np.float64(0.23755318802432157) - rollout_corr/kl:np.float64(0.0002802361841816037) - rollout_corr/k3_kl:np.float64(0.00033016265430774183) - rollout_corr/rollout_ppl:np.float64(1.2775348122231662) - rollout_corr/rollout_log_ppl:np.float64(0.23727294862328563) - rollout_corr/log_ppl_diff:np.float64(0.00028023940103594214) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010713046794990078) - rollout_corr/log_ppl_diff_max:np.float64(0.005381345748901367) - rollout_corr/log_ppl_diff_min:np.float64(-0.007450282573699951) - rollout_corr/ppl_ratio:np.float64(1.0002813362516463) - rollout_corr/chi2_token:np.float64(0.0007960675284266472) - rollout_corr/chi2_seq:np.float64(0.3917784618679434) - actor/pg_loss:np.float64(-5.9889862313866615e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.172775687853573e-05) - actor/ppo_kl:np.float64(4.416867541578995e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12709747441112995) - perf/mfu/actor:np.float64(0.10145917324682335) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.06658554077148) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005021680262871087 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005021680262871087 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:404.49609375 - response_length/max:832.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:404.49609375 - response_length_non_aborted/max:832.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:382.375 - prompt_length/max:482.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010803155601024628 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5115187149494886) - timing_s/agent_loop/generate_sequences/max:np.float64(7.589536972343922) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.612800014750974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.589536972343922) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:482 - timing_s/agent_loop/slowest/response_length:832 - timing_s/gen:14.042075123637915 - timing_s/reward:0.05917769856750965 - timing_s/old_log_prob:3.049703635275364 - timing_s/adv:0.1191564816981554 - timing_s/update_actor:12.995382741093636 - timing_s/step:30.358434250578284 - timing_s/stop_profile:0.0001139957457780838 - timing_per_token_ms/gen:0.13560540336296042 - timing_per_token_ms/adv:0.0005915263762139179 - timing_per_token_ms/update_actor:0.06451274450872788 - perf/total_num_tokens:201439 - perf/time_per_step:30.358434250578284 - perf/throughput:829.4194223643257 (TaskRunner pid=2403642) Training Progress: 58%|█████▊ | 58/100 [37:19<22:25, 32.03s/it] (TaskRunner pid=2403642) step:59 - global_seqlen/min:20172 - global_seqlen/max:28781 - global_seqlen/minmax_diff:8609 - global_seqlen/balanced_min:23349 - global_seqlen/balanced_max:23358 - global_seqlen/mean:23354.375 - actor/entropy:0.24409091472625732 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2924416661262512 - training/rollout_probs_diff_mean:0.002486287383362651 - training/rollout_probs_diff_std:0.007413405925035477 - training/rollout_actor_probs_pearson_corr:0.9995101094245911 - rollout_corr/rollout_is_mean:0.9999902248382568 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.23826318688225e-05 - rollout_corr/rollout_is_max:1.4768394231796265 - rollout_corr/rollout_is_min:0.24649836122989655 - rollout_corr/rollout_is_std:0.024291684851050377 - rollout_corr/rollout_is_eff_sample_size:0.9994105001503535 - rollout_corr/rollout_is_seq_mean:1.0000391006469727 - rollout_corr/rollout_is_seq_std:0.001515738433226943 - rollout_corr/rollout_is_seq_max:1.005221962928772 - rollout_corr/rollout_is_seq_min:0.9950911402702332 - rollout_corr/rollout_is_seq_max_deviation:0.005221962928771973 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.310080023482442) - rollout_corr/training_log_ppl:np.float64(0.26051520198234357) - rollout_corr/kl:np.float64(0.000524200898032956) - rollout_corr/k3_kl:np.float64(0.0004534090181920192) - rollout_corr/rollout_ppl:np.float64(1.309392552357167) - rollout_corr/rollout_log_ppl:np.float64(0.2599910004355479) - rollout_corr/log_ppl_diff:np.float64(0.0005242015467956662) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013801687164232135) - rollout_corr/log_ppl_diff_max:np.float64(0.006640344858169556) - rollout_corr/log_ppl_diff_min:np.float64(-0.005571231245994568) - rollout_corr/ppl_ratio:np.float64(1.0005259348545223) - rollout_corr/chi2_token:np.float64(0.0007969602011144161) - rollout_corr/chi2_seq:np.float64(0.15531086595728993) - actor/pg_loss:np.float64(-2.4826964363455772e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.681123972564819e-05) - actor/ppo_kl:np.float64(0.0002450501577317965) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13004800118505955) - perf/mfu/actor:np.float64(0.0959988976426908) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.06728744506836) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001934698666445911 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001934698666445911 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:349.04296875 - response_length/max:790.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:349.04296875 - response_length_non_aborted/max:790.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:380.78125 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001479554921388626 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7181454841047525) - timing_s/agent_loop/generate_sequences/max:np.float64(7.045558724552393) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.890985884645488) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.045558724552393) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:785 - timing_s/gen:13.8492118883878 - timing_s/reward:0.057231469079852104 - timing_s/old_log_prob:3.05007535032928 - timing_s/adv:0.12209023907780647 - timing_s/update_actor:12.677439650520682 - timing_s/step:29.850827749818563 - timing_s/stop_profile:0.00012044981122016907 - timing_per_token_ms/gen:0.15499090021137932 - timing_per_token_ms/adv:0.0006534655662900767 - timing_per_token_ms/update_actor:0.0678536658041624 - perf/total_num_tokens:186835 - perf/time_per_step:29.850827749818563 - perf/throughput:782.3694269296084 (TaskRunner pid=2403642) Training Progress: 59%|█████▉ | 59/100 [37:49<21:26, 31.39s/it] (TaskRunner pid=2403642) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2403642) validation generation end (TaskRunner pid=2403642) [prompt] system (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2403642) (TaskRunner pid=2403642) A (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) user (TaskRunner pid=2403642) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2403642) (TaskRunner pid=2403642) Lattice Parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) a = 7.82 Å (TaskRunner pid=2403642) b = 7.82 Å (TaskRunner pid=2403642) c = 17.88 Å (TaskRunner pid=2403642) α = 90.00º (TaskRunner pid=2403642) β = 90.00º (TaskRunner pid=2403642) γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) A: 1025.67 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) B: 1094.31 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) C: 1200.45 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) D: 1150.78 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) Please reason step by step. (TaskRunner pid=2403642) assistant (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) [response] (TaskRunner pid=2403642) The crystal structure is described as having lattice parameters a = 7.82 Å, b = 7.82 Å, c = 17.88 Å, with all angles equal to 90 degrees, which indicates a cubic or orthorhombic structure. Since all angles are 90 degrees, this is an orthorhombic unit cell. (TaskRunner pid=2403642) (TaskRunner pid=2403642) The volume of a unit cell in an orthorhombic structure is calculated using the formula: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \times b \times c $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Substituting the given values: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = 7.82 \, \text{Å} \times 7.82 \, \text{Å} \times 17.88 \, \text{Å} $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) First, calculate the product of a and b: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 7.82 \times 7.82 = 61.1524 \, \text{Å}^2 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Then multiply by c: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 61.1524 \times 17.88 = 1094.31 \, \text{Å}^3 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Thus, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) B (TaskRunner pid=2403642) (TaskRunner pid=2403642) [ground_truth] B (TaskRunner pid=2403642) [score] 1.0 (TaskRunner pid=2403642) [acc] 1.0 (TaskRunner pid=2403642) [pred] B (TaskRunner pid=2403642) [incorrect_format] 1 (TaskRunner pid=2403642) [feedback] (TaskRunner pid=2403642) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2403642) step:60 - global_seqlen/min:20680 - global_seqlen/max:28400 - global_seqlen/minmax_diff:7720 - global_seqlen/balanced_min:23980 - global_seqlen/balanced_max:23992 - global_seqlen/mean:23988.5 - actor/entropy:0.2360708862543106 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48631659150123596 - training/rollout_probs_diff_mean:0.002380544785410166 - training/rollout_probs_diff_std:0.007412472274154425 - training/rollout_actor_probs_pearson_corr:0.9995028972625732 - rollout_corr/rollout_is_mean:0.9999828338623047 - rollout_corr/rollout_is_ratio_fraction_high:2.126483195752371e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.63820219039917 - rollout_corr/rollout_is_min:0.5089420080184937 - rollout_corr/rollout_is_std:0.0243162102997303 - rollout_corr/rollout_is_eff_sample_size:0.9994089522585684 - rollout_corr/rollout_is_seq_mean:0.9999392032623291 - rollout_corr/rollout_is_seq_std:0.0015173176070675254 - rollout_corr/rollout_is_seq_max:1.0061588287353516 - rollout_corr/rollout_is_seq_min:0.9955165982246399 - rollout_corr/rollout_is_seq_max_deviation:0.0061588287353515625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2967511527240276) - rollout_corr/training_log_ppl:np.float64(0.2504257837572368) - rollout_corr/kl:np.float64(0.0005130456188339183) - rollout_corr/k3_kl:np.float64(0.00039365157081761026) - rollout_corr/rollout_ppl:np.float64(1.2960526859387755) - rollout_corr/rollout_log_ppl:np.float64(0.24991273554041982) - rollout_corr/log_ppl_diff:np.float64(0.0005130482168169692) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012746355059789494) - rollout_corr/log_ppl_diff_max:np.float64(0.007074087858200073) - rollout_corr/log_ppl_diff_min:np.float64(-0.004114046692848206) - rollout_corr/ppl_ratio:np.float64(1.000514515209943) - rollout_corr/chi2_token:np.float64(0.0005633197724819183) - rollout_corr/chi2_seq:np.float64(0.19360803021118045) - actor/pg_loss:np.float64(2.6261783204972744e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.017660366022028e-05) - actor/ppo_kl:np.float64(0.00012672674524605299) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10184402484446764) - perf/mfu/actor:np.float64(0.09759851413831924) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.0626106262207) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7533244680851063) - val-aux/sciknoweval/reward/std@16:np.float64(0.05264515376564715) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7701702127659574) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.03719160916663892) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.736276595744681) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.048824922865621616) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7530531914893616) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.05238510513576182) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7838297872340426) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.03035824019330265) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7190744680851064) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.047430696791599204) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.755372340425532) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.03210443043930431) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7968510638297872) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.02641430219640358) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6982340425531915) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.04790582542211792) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7554361702127659) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.01487437652497796) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.808627659574468) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.01922845505585631) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6755851063829788) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04116780524473105) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7553404255319148) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.008410939966742359) - val-aux/sciknoweval/score/mean@16:np.float64(0.7533244680851063) - val-aux/sciknoweval/score/std@16:np.float64(0.05264515376564715) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7701702127659574) - val-aux/sciknoweval/score/best@2/std:np.float64(0.03719160916663892) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.736276595744681) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.048824922865621616) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7530531914893616) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.05238510513576182) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7838297872340426) - val-aux/sciknoweval/score/best@4/std:np.float64(0.03035824019330265) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7190744680851064) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.047430696791599204) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.755372340425532) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.03210443043930431) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7968510638297872) - val-aux/sciknoweval/score/best@8/std:np.float64(0.02641430219640358) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6982340425531915) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.04790582542211792) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7554361702127659) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.01487437652497796) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.808627659574468) - val-aux/sciknoweval/score/best@16/std:np.float64(0.01922845505585631) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6755851063829788) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04116780524473105) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7553404255319148) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.008410939966742359) - val-core/sciknoweval/acc/mean@16:np.float64(0.7533244680851063) - val-aux/sciknoweval/acc/std@16:np.float64(0.05264515376564715) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7701702127659574) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.03719160916663892) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.736276595744681) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.048824922865621616) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7530531914893616) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.05238510513576182) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7838297872340426) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.03035824019330265) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7190744680851064) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.047430696791599204) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.755372340425532) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.03210443043930431) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7968510638297872) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.02641430219640358) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6982340425531915) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.04790582542211792) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7554361702127659) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.01487437652497796) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.808627659574468) - val-core/sciknoweval/acc/best@16/std:np.float64(0.01922845505585631) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6755851063829788) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04116780524473105) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7553404255319148) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.008410939966742359) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00800355151295662 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00800355151295662 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:367.390625 - response_length/max:909.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:367.390625 - response_length_non_aborted/max:909.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:382.25 - prompt_length/max:495.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011824630200862885 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3282297514379025) - timing_s/agent_loop/generate_sequences/max:np.float64(7.722264399752021) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.283295893306786) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.722264399752021) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:438 - timing_s/agent_loop/slowest/response_length:909 - timing_s/gen:14.402702420949936 - timing_s/reward:0.06062391586601734 - timing_s/old_log_prob:3.0862164348363876 - timing_s/adv:0.1184315737336874 - timing_s/update_actor:12.883418533951044 - timing_s/step:30.63865765556693 - timing_s/testing:41.323619935661554 - timing_s/save_checkpoint:12.727264175191522 - timing_s/stop_profile:0.00015765242278575897 - timing_per_token_ms/gen:0.15313552525145596 - timing_per_token_ms/adv:0.0006171268197974415 - timing_per_token_ms/update_actor:0.06713330624023513 - perf/total_num_tokens:191908 - perf/time_per_step:30.63865765556693 - perf/throughput:782.9487920023604 (TaskRunner pid=2403642) Training Progress: 60%|██████ | 60/100 [39:14<31:36, 47.41s/it] (TaskRunner pid=2403642) step:61 - global_seqlen/min:20712 - global_seqlen/max:26801 - global_seqlen/minmax_diff:6089 - global_seqlen/balanced_min:23429 - global_seqlen/balanced_max:23433 - global_seqlen/mean:23431.0 - actor/entropy:0.28184887766838074 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18471911549568176 - training/rollout_probs_diff_mean:0.0026123705320060253 - training/rollout_probs_diff_std:0.007209790870547295 - training/rollout_actor_probs_pearson_corr:0.9995990991592407 - rollout_corr/rollout_is_mean:0.9999793767929077 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.4511632919311523 - rollout_corr/rollout_is_min:0.6871916055679321 - rollout_corr/rollout_is_std:0.024666661396622658 - rollout_corr/rollout_is_eff_sample_size:0.9993919257653825 - rollout_corr/rollout_is_seq_mean:0.9999731183052063 - rollout_corr/rollout_is_seq_std:0.0016377248102799058 - rollout_corr/rollout_is_seq_max:1.0047904253005981 - rollout_corr/rollout_is_seq_min:0.9949299097061157 - rollout_corr/rollout_is_seq_max_deviation:0.005070090293884277 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.352380177937448) - rollout_corr/training_log_ppl:np.float64(0.2933591184410034) - rollout_corr/kl:np.float64(0.0003880567859333439) - rollout_corr/k3_kl:np.float64(0.0003783676123703117) - rollout_corr/rollout_ppl:np.float64(1.351829611696303) - rollout_corr/rollout_log_ppl:np.float64(0.2929710615426302) - rollout_corr/log_ppl_diff:np.float64(0.00038805689837317914) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013752124359598383) - rollout_corr/log_ppl_diff_max:np.float64(0.0052146315574646) - rollout_corr/log_ppl_diff_min:np.float64(-0.004297047853469849) - rollout_corr/ppl_ratio:np.float64(1.0003896711859852) - rollout_corr/chi2_token:np.float64(0.0007408889941871166) - rollout_corr/chi2_seq:np.float64(0.2107625617645681) - actor/pg_loss:np.float64(-2.980930730700493e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.4723101887502708e-05) - actor/ppo_kl:np.float64(4.149326706581746e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.05792093137279153) - perf/mfu/actor:np.float64(0.09517745982226297) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(107.07573699951172) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00027224450604990125 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.00027224450604990125 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:319.25 - response_length/max:737.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:319.25 - response_length_non_aborted/max:737.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.96875 - prompt_length/max:509.0 - prompt_length/min:269.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014871545135974884 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4992734659463167) - timing_s/agent_loop/generate_sequences/max:np.float64(6.306750409305096) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.624103704605659) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.306750409305096) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:287 - timing_s/agent_loop/slowest/response_length:737 - timing_s/gen:12.51429340802133 - timing_s/reward:0.05750600807368755 - timing_s/old_log_prob:3.1164417024701834 - timing_s/adv:0.11376292444765568 - timing_s/update_actor:12.820102855563164 - timing_s/step:28.724578784778714 - timing_s/stop_profile:0.00010818615555763245 - timing_per_token_ms/gen:0.15312124862986162 - timing_per_token_ms/adv:0.0006069039117390192 - timing_per_token_ms/update_actor:0.06839284951326856 - perf/total_num_tokens:187448 - perf/time_per_step:28.724578784778714 - perf/throughput:815.7125706022953 (TaskRunner pid=2403642) Training Progress: 61%|██████ | 61/100 [39:42<27:11, 41.82s/it] (TaskRunner pid=2403642) step:62 - global_seqlen/min:18620 - global_seqlen/max:28360 - global_seqlen/minmax_diff:9740 - global_seqlen/balanced_min:23900 - global_seqlen/balanced_max:23903 - global_seqlen/mean:23901.875 - actor/entropy:0.22449631989002228 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34021374583244324 - training/rollout_probs_diff_mean:0.0022299615666270256 - training/rollout_probs_diff_std:0.007056043017655611 - training/rollout_actor_probs_pearson_corr:0.9995299577713013 - rollout_corr/rollout_is_mean:0.999995768070221 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1769656086689793e-05 - rollout_corr/rollout_is_max:1.5585025548934937 - rollout_corr/rollout_is_min:0.09886863827705383 - rollout_corr/rollout_is_std:0.022548340260982513 - rollout_corr/rollout_is_eff_sample_size:0.9994918307473496 - rollout_corr/rollout_is_seq_mean:1.0000267028808594 - rollout_corr/rollout_is_seq_std:0.00136755988933146 - rollout_corr/rollout_is_seq_max:1.003589153289795 - rollout_corr/rollout_is_seq_min:0.9956575632095337 - rollout_corr/rollout_is_seq_max_deviation:0.004342436790466309 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2885832339525223) - rollout_corr/training_log_ppl:np.float64(0.24286671256413683) - rollout_corr/kl:np.float64(0.0003957929533555671) - rollout_corr/k3_kl:np.float64(0.000319797927829768) - rollout_corr/rollout_ppl:np.float64(1.2880218625068665) - rollout_corr/rollout_log_ppl:np.float64(0.24247091660799924) - rollout_corr/log_ppl_diff:np.float64(0.0003957959561375901) - rollout_corr/log_ppl_abs_diff:np.float64(0.001179586790385656) - rollout_corr/log_ppl_diff_max:np.float64(0.005161285400390625) - rollout_corr/log_ppl_diff_min:np.float64(-0.004351675510406494) - rollout_corr/ppl_ratio:np.float64(1.0003969802055508) - rollout_corr/chi2_token:np.float64(0.00047266576439142227) - rollout_corr/chi2_seq:np.float64(0.14369945717044175) - actor/pg_loss:np.float64(3.2330979593098164e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.0179311731189955e-05) - actor/ppo_kl:np.float64(0.00012892496328453795) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08937415108084679) - perf/mfu/actor:np.float64(0.09680483357257347) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.24963760375977) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:1.4966638445912395e-05 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:1.4966638445912395e-05 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:358.87109375 - response_length/max:988.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:358.87109375 - response_length_non_aborted/max:988.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.0625 - prompt_length/max:541.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012834742665290833 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3620898555964231) - timing_s/agent_loop/generate_sequences/max:np.float64(8.082440830767155) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.747333021972736) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.082440830767155) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:988 - timing_s/gen:14.8345803655684 - timing_s/reward:0.0613545086234808 - timing_s/old_log_prob:3.105897920206189 - timing_s/adv:0.11996661499142647 - timing_s/update_actor:12.829736707732081 - timing_s/step:31.0376286637038 - timing_s/stop_profile:0.00011411495506763458 - timing_per_token_ms/gen:0.16147185037246137 - timing_per_token_ms/adv:0.0006273912349524173 - timing_per_token_ms/update_actor:0.06709586961133845 - perf/total_num_tokens:191215 - perf/time_per_step:31.0376286637038 - perf/throughput:770.0934649028605 (TaskRunner pid=2403642) Training Progress: 62%|██████▏ | 62/100 [40:13<24:26, 38.60s/it] (TaskRunner pid=2403642) step:63 - global_seqlen/min:19818 - global_seqlen/max:26499 - global_seqlen/minmax_diff:6681 - global_seqlen/balanced_min:23404 - global_seqlen/balanced_max:23410 - global_seqlen/mean:23408.125 - actor/entropy:0.27887508273124695 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032555460929871 - training/rollout_probs_diff_mean:0.002610958879813552 - training/rollout_probs_diff_std:0.007603233214467764 - training/rollout_actor_probs_pearson_corr:0.9995499849319458 - rollout_corr/rollout_is_mean:0.9998920559883118 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.616157118813135e-05 - rollout_corr/rollout_is_max:1.424423098564148 - rollout_corr/rollout_is_min:0.4286360740661621 - rollout_corr/rollout_is_std:0.02481122314929962 - rollout_corr/rollout_is_eff_sample_size:0.9993846628923608 - rollout_corr/rollout_is_seq_mean:0.9998520016670227 - rollout_corr/rollout_is_seq_std:0.0016024266369640827 - rollout_corr/rollout_is_seq_max:1.0046424865722656 - rollout_corr/rollout_is_seq_min:0.994684636592865 - rollout_corr/rollout_is_seq_max_deviation:0.00531536340713501 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.360725431703031) - rollout_corr/training_log_ppl:np.float64(0.2997116361220833) - rollout_corr/kl:np.float64(0.0005157101149642074) - rollout_corr/k3_kl:np.float64(0.00036520069393475296) - rollout_corr/rollout_ppl:np.float64(1.3599829538725317) - rollout_corr/rollout_log_ppl:np.float64(0.2991959253558889) - rollout_corr/log_ppl_diff:np.float64(0.0005157107661943883) - rollout_corr/log_ppl_abs_diff:np.float64(0.001327492733253166) - rollout_corr/log_ppl_diff_max:np.float64(0.0076059699058532715) - rollout_corr/log_ppl_diff_min:np.float64(-0.0042442381381988525) - rollout_corr/ppl_ratio:np.float64(1.0005172591190785) - rollout_corr/chi2_token:np.float64(0.00040490622632205486) - rollout_corr/chi2_seq:np.float64(0.13411166868172586) - actor/pg_loss:np.float64(2.5239423848688602e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.4135695639415644e-05) - actor/ppo_kl:np.float64(3.2647710623834314e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0763076157309115) - perf/mfu/actor:np.float64(0.09596526124371177) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.12528991699219) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038225792814046144 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038225792814046144 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:324.06640625 - response_length/max:997.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.06640625 - response_length_non_aborted/max:997.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.4375 - prompt_length/max:483.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.467212319374084e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.324038576334715) - timing_s/agent_loop/generate_sequences/max:np.float64(7.843218516558409) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6979008383350447) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.843218516558409) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:997 - timing_s/gen:14.753351049497724 - timing_s/reward:0.05980104021728039 - timing_s/old_log_prob:3.047181662172079 - timing_s/adv:0.12265457585453987 - timing_s/update_actor:12.744738161563873 - timing_s/step:30.815232565626502 - timing_s/stop_profile:0.00011097453534603119 - timing_per_token_ms/gen:0.17783477838379147 - timing_per_token_ms/adv:0.0006549786444586008 - timing_per_token_ms/update_actor:0.06805723526320387 - perf/total_num_tokens:187265 - perf/time_per_step:30.815232565626502 - perf/throughput:759.6283737319927 (TaskRunner pid=2403642) Training Progress: 63%|██████▎ | 63/100 [40:44<22:22, 36.28s/it] (TaskRunner pid=2403642) step:64 - global_seqlen/min:20169 - global_seqlen/max:25066 - global_seqlen/minmax_diff:4897 - global_seqlen/balanced_min:23043 - global_seqlen/balanced_max:23048 - global_seqlen/mean:23046.25 - actor/entropy:0.2526596784591675 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.611835241317749 - training/rollout_probs_diff_mean:0.00250222091563046 - training/rollout_probs_diff_std:0.007975784130394459 - training/rollout_actor_probs_pearson_corr:0.9994510412216187 - rollout_corr/rollout_is_mean:0.9999549388885498 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.7495785111095756e-05 - rollout_corr/rollout_is_max:1.5585436820983887 - rollout_corr/rollout_is_min:0.24953652918338776 - rollout_corr/rollout_is_std:0.024059930816292763 - rollout_corr/rollout_is_eff_sample_size:0.9994213355271035 - rollout_corr/rollout_is_seq_mean:0.999975323677063 - rollout_corr/rollout_is_seq_std:0.001513405004516244 - rollout_corr/rollout_is_seq_max:1.004136085510254 - rollout_corr/rollout_is_seq_min:0.9950628876686096 - rollout_corr/rollout_is_seq_max_deviation:0.004937112331390381 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3078968808986247) - rollout_corr/training_log_ppl:np.float64(0.2614987124252366) - rollout_corr/kl:np.float64(0.0003369308145106942) - rollout_corr/k3_kl:np.float64(0.00036924221276990465) - rollout_corr/rollout_ppl:np.float64(1.3074609604664147) - rollout_corr/rollout_log_ppl:np.float64(0.2611617771326564) - rollout_corr/log_ppl_diff:np.float64(0.00033693529258016497) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012274637265363708) - rollout_corr/log_ppl_diff_max:np.float64(0.009813323616981506) - rollout_corr/log_ppl_diff_min:np.float64(-0.003359168767929077) - rollout_corr/ppl_ratio:np.float64(1.0003384032752365) - rollout_corr/chi2_token:np.float64(0.000732076819986105) - rollout_corr/chi2_seq:np.float64(0.19428053102456033) - actor/pg_loss:np.float64(-5.030236206948757e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.072281131404452e-05) - actor/ppo_kl:np.float64(-8.341730296290706e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0369609696790576) - perf/mfu/actor:np.float64(0.09428030451113695) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.12153482437134) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006679094512946904 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006679094512946904 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:328.9765625 - response_length/max:976.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:328.9765625 - response_length_non_aborted/max:976.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.21875 - prompt_length/max:473.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012168474495410919 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3025468364357948) - timing_s/agent_loop/generate_sequences/max:np.float64(7.783195244148374) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.66632833393669) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.783195244148374) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:976 - timing_s/gen:14.508876765146852 - timing_s/reward:0.05970977991819382 - timing_s/old_log_prob:3.0077879298478365 - timing_s/adv:0.12053753063082695 - timing_s/update_actor:12.580914795398712 - timing_s/step:30.365346198901534 - timing_s/stop_profile:0.00011391565203666687 - timing_per_token_ms/gen:0.17227762194717106 - timing_per_token_ms/adv:0.0006537806076413026 - timing_per_token_ms/update_actor:0.06823732058034773 - perf/total_num_tokens:184370 - perf/time_per_step:30.365346198901534 - perf/throughput:758.9654947136318 (TaskRunner pid=2403642) Training Progress: 64%|██████▍ | 64/100 [41:15<20:42, 34.52s/it] (TaskRunner pid=2403642) step:65 - global_seqlen/min:20002 - global_seqlen/max:28030 - global_seqlen/minmax_diff:8028 - global_seqlen/balanced_min:23359 - global_seqlen/balanced_max:23360 - global_seqlen/mean:23359.25 - actor/entropy:0.26026061177253723 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15484067797660828 - training/rollout_probs_diff_mean:0.00245410343632102 - training/rollout_probs_diff_std:0.007108656223863363 - training/rollout_actor_probs_pearson_corr:0.9995998740196228 - rollout_corr/rollout_is_mean:0.9998462200164795 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1785504284489434e-05 - rollout_corr/rollout_is_max:1.645729422569275 - rollout_corr/rollout_is_min:0.2046152651309967 - rollout_corr/rollout_is_std:0.024017779156565666 - rollout_corr/rollout_is_eff_sample_size:0.9994233597444102 - rollout_corr/rollout_is_seq_mean:0.9998414516448975 - rollout_corr/rollout_is_seq_std:0.0014760512858629227 - rollout_corr/rollout_is_seq_max:1.0046924352645874 - rollout_corr/rollout_is_seq_min:0.9952191114425659 - rollout_corr/rollout_is_seq_max_deviation:0.004780888557434082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3243399313651025) - rollout_corr/training_log_ppl:np.float64(0.2717995819548378) - rollout_corr/kl:np.float64(0.0004931192334041157) - rollout_corr/k3_kl:np.float64(0.00032269406943896684) - rollout_corr/rollout_ppl:np.float64(1.3236418650485575) - rollout_corr/rollout_log_ppl:np.float64(0.2713064616546035) - rollout_corr/log_ppl_diff:np.float64(0.0004931203002342954) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012263796961633489) - rollout_corr/log_ppl_diff_max:np.float64(0.005062103271484375) - rollout_corr/log_ppl_diff_min:np.float64(-0.004357486963272095) - rollout_corr/ppl_ratio:np.float64(1.00049435836263) - rollout_corr/chi2_token:np.float64(0.0002919696271419525) - rollout_corr/chi2_seq:np.float64(0.05615922715514898) - actor/pg_loss:np.float64(6.77348580211401e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.6549815831676824e-05) - actor/ppo_kl:np.float64(2.9900374386215844e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07062362832948565) - perf/mfu/actor:np.float64(0.09549129968903941) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.11310195922852) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005215085693635046 - critic/advantages/max:0.75 - critic/advantages/min:-0.625 - critic/returns/mean:0.0005215085693635046 - critic/returns/max:0.75 - critic/returns/min:-0.625 - response_length/mean:331.4453125 - response_length/max:856.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.4453125 - response_length_non_aborted/max:856.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:398.53125 - prompt_length/max:490.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010526925325393677 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3068315219134092) - timing_s/agent_loop/generate_sequences/max:np.float64(7.289796233177185) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.759431266633328) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.289796233177185) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:856 - timing_s/gen:13.990133194252849 - timing_s/reward:0.059712598100304604 - timing_s/old_log_prob:2.9942075554281473 - timing_s/adv:0.12356245517730713 - timing_s/update_actor:12.67472019791603 - timing_s/step:29.930364733561873 - timing_s/stop_profile:0.00011428631842136383 - timing_per_token_ms/gen:0.16488076834711665 - timing_per_token_ms/adv:0.0006612073117571579 - timing_per_token_ms/update_actor:0.06782495263073532 - perf/total_num_tokens:186874 - perf/time_per_step:29.930364733561873 - perf/throughput:780.4532356335279 (TaskRunner pid=2403642) Training Progress: 65%|██████▌ | 65/100 [41:45<19:20, 33.16s/it] (TaskRunner pid=2403642) step:66 - global_seqlen/min:16888 - global_seqlen/max:25871 - global_seqlen/minmax_diff:8983 - global_seqlen/balanced_min:22834 - global_seqlen/balanced_max:22836 - global_seqlen/mean:22834.875 - actor/entropy:0.21960677206516266 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18740245699882507 - training/rollout_probs_diff_mean:0.002241658978164196 - training/rollout_probs_diff_std:0.006920246407389641 - training/rollout_actor_probs_pearson_corr:0.9995320439338684 - rollout_corr/rollout_is_mean:0.999957263469696 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.4334044456481934 - rollout_corr/rollout_is_min:0.6489461064338684 - rollout_corr/rollout_is_std:0.02259587123990059 - rollout_corr/rollout_is_eff_sample_size:0.9994895680773683 - rollout_corr/rollout_is_seq_mean:0.9999371767044067 - rollout_corr/rollout_is_seq_std:0.0015744587872177362 - rollout_corr/rollout_is_seq_max:1.0062541961669922 - rollout_corr/rollout_is_seq_min:0.9941355586051941 - rollout_corr/rollout_is_seq_max_deviation:0.0062541961669921875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2800852260552347) - rollout_corr/training_log_ppl:np.float64(0.2388803932844894) - rollout_corr/kl:np.float64(0.0003673631687366452) - rollout_corr/k3_kl:np.float64(0.00030391120974115893) - rollout_corr/rollout_ppl:np.float64(1.2795846569351852) - rollout_corr/rollout_log_ppl:np.float64(0.23851302760886028) - rollout_corr/log_ppl_diff:np.float64(0.0003673656756291166) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012513460969785228) - rollout_corr/log_ppl_diff_max:np.float64(0.0068174004554748535) - rollout_corr/log_ppl_diff_min:np.float64(-0.004504576325416565) - rollout_corr/ppl_ratio:np.float64(1.0003687869757414) - rollout_corr/chi2_token:np.float64(0.0004690245259553194) - rollout_corr/chi2_seq:np.float64(0.12948566675186157) - actor/pg_loss:np.float64(-8.635222911834717e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(1.3391224911174504e-05) - actor/ppo_kl:np.float64(1.975016734201418e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06018312368541956) - perf/mfu/actor:np.float64(0.09330624794717815) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.09537124633789) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0019717176910489798 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0019717176910489798 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:325.40234375 - response_length/max:822.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:325.40234375 - response_length_non_aborted/max:822.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.1875 - prompt_length/max:475.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.301941514015198e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2752054762095213) - timing_s/agent_loop/generate_sequences/max:np.float64(6.971778251230717) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6238729834512924) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.971778251230717) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:822 - timing_s/gen:13.410331742838025 - timing_s/reward:0.05954797938466072 - timing_s/old_log_prob:2.9815644156187773 - timing_s/adv:0.11963827721774578 - timing_s/update_actor:12.643205320462584 - timing_s/step:29.304401816800237 - timing_s/stop_profile:0.00015919841825962067 - timing_per_token_ms/gen:0.16098257857265674 - timing_per_token_ms/adv:0.0006549098539938679 - timing_per_token_ms/update_actor:0.0692099547318662 - perf/total_num_tokens:182679 - perf/time_per_step:29.304401816800237 - perf/throughput:779.2302038019677 (TaskRunner pid=2403642) Training Progress: 66%|██████▌ | 66/100 [42:14<18:08, 32.02s/it] (TaskRunner pid=2403642) step:67 - global_seqlen/min:21129 - global_seqlen/max:23608 - global_seqlen/minmax_diff:2479 - global_seqlen/balanced_min:22432 - global_seqlen/balanced_max:22436 - global_seqlen/mean:22434.375 - actor/entropy:0.2391292154788971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9956695437431335 - training/rollout_probs_diff_mean:0.002481213305145502 - training/rollout_probs_diff_std:0.008158995769917965 - training/rollout_actor_probs_pearson_corr:0.9994234442710876 - rollout_corr/rollout_is_mean:1.0000455379486084 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.474420762155205e-05 - rollout_corr/rollout_is_max:1.775370717048645 - rollout_corr/rollout_is_min:0.00026000928482972085 - rollout_corr/rollout_is_std:0.024126727133989334 - rollout_corr/rollout_is_eff_sample_size:0.9994183587518422 - rollout_corr/rollout_is_seq_mean:1.0000216960906982 - rollout_corr/rollout_is_seq_std:0.0014827295672148466 - rollout_corr/rollout_is_seq_max:1.0066418647766113 - rollout_corr/rollout_is_seq_min:0.9953941106796265 - rollout_corr/rollout_is_seq_max_deviation:0.006641864776611328 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2991883950307965) - rollout_corr/training_log_ppl:np.float64(0.25347993172181305) - rollout_corr/kl:np.float64(0.0003144908936771884) - rollout_corr/k3_kl:np.float64(0.0003958175733913549) - rollout_corr/rollout_ppl:np.float64(1.2987574469298124) - rollout_corr/rollout_log_ppl:np.float64(0.25316543960070703) - rollout_corr/log_ppl_diff:np.float64(0.00031449212110601366) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012145455984864384) - rollout_corr/log_ppl_diff_max:np.float64(0.007181018590927124) - rollout_corr/log_ppl_diff_min:np.float64(-0.005029022693634033) - rollout_corr/ppl_ratio:np.float64(1.0003158359322697) - rollout_corr/chi2_token:np.float64(0.0009363754652440548) - rollout_corr/chi2_seq:np.float64(0.2373002462554723) - actor/pg_loss:np.float64(1.0611605830490589e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.2922226384689566e-05) - actor/ppo_kl:np.float64(-7.573282338313447e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.060480975080281496) - perf/mfu/actor:np.float64(0.09189412437163944) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.12161254882812) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:2 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017661177553236485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017661177553236485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:315.73046875 - response_length/max:867.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.73046875 - response_length_non_aborted/max:867.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.34375 - prompt_length/max:472.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014377757906913757 - timing_s/agent_loop/generate_sequences/min:np.float64(1.256573947146535) - timing_s/agent_loop/generate_sequences/max:np.float64(7.064186401665211) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4151445413008332) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.064186401665211) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:867 - timing_s/gen:14.483078742399812 - timing_s/reward:0.058952899649739265 - timing_s/old_log_prob:2.9972836077213287 - timing_s/adv:0.12383461557328701 - timing_s/update_actor:12.739469554275274 - timing_s/step:30.491643711924553 - timing_s/stop_profile:0.00011830590665340424 - timing_per_token_ms/gen:0.1791861474804188 - timing_per_token_ms/adv:0.0006899825355803705 - timing_per_token_ms/update_actor:0.07098186128583521 - perf/total_num_tokens:179475 - perf/time_per_step:30.491643711924553 - perf/throughput:735.754858345877 (TaskRunner pid=2403642) Training Progress: 67%|██████▋ | 67/100 [42:45<17:21, 31.57s/it] (TaskRunner pid=2403642) step:68 - global_seqlen/min:17478 - global_seqlen/max:24987 - global_seqlen/minmax_diff:7509 - global_seqlen/balanced_min:21376 - global_seqlen/balanced_max:21379 - global_seqlen/mean:21377.75 - actor/entropy:0.2649036645889282 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9373443126678467 - training/rollout_probs_diff_mean:0.002694986527785659 - training/rollout_probs_diff_std:0.008782610297203064 - training/rollout_actor_probs_pearson_corr:0.999350905418396 - rollout_corr/rollout_is_mean:1.0000685453414917 - rollout_corr/rollout_is_ratio_fraction_high:2.6007126507465728e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.901069067069329e-05 - rollout_corr/rollout_is_max:6.416873931884766 - rollout_corr/rollout_is_min:0.0026309515815228224 - rollout_corr/rollout_is_std:0.025890419259667397 - rollout_corr/rollout_is_eff_sample_size:0.9993300161348888 - rollout_corr/rollout_is_seq_mean:0.9999976754188538 - rollout_corr/rollout_is_seq_std:0.0016944073140621185 - rollout_corr/rollout_is_seq_max:1.0089281797409058 - rollout_corr/rollout_is_seq_min:0.9944455027580261 - rollout_corr/rollout_is_seq_max_deviation:0.008928179740905762 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3318745321594179) - rollout_corr/training_log_ppl:np.float64(0.2706391986721428) - rollout_corr/kl:np.float64(0.0005419229773551137) - rollout_corr/k3_kl:np.float64(0.0004925900274201922) - rollout_corr/rollout_ppl:np.float64(1.3310793451964855) - rollout_corr/rollout_log_ppl:np.float64(0.2700972726306645) - rollout_corr/log_ppl_diff:np.float64(0.0005419260414782912) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014799507334828377) - rollout_corr/log_ppl_diff_max:np.float64(0.026928484439849854) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034056007862091064) - rollout_corr/ppl_ratio:np.float64(1.0005449776072055) - rollout_corr/chi2_token:np.float64(0.000718752620741725) - rollout_corr/chi2_seq:np.float64(0.20082624815404415) - actor/pg_loss:np.float64(-4.2798928916454315e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.8068650357890874e-05) - actor/ppo_kl:np.float64(8.310840217617965e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1402107272297144) - perf/mfu/actor:np.float64(0.08789061359051692) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.2520523071289) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:2 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004600010288413614 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0004600010288413614 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:300.3984375 - response_length/max:758.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:300.3984375 - response_length_non_aborted/max:758.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:367.65625 - prompt_length/max:491.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00020078755915164948 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3047517966479063) - timing_s/agent_loop/generate_sequences/max:np.float64(6.5038600247353315) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4670009663968813) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.5038600247353315) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:758 - timing_s/gen:13.26128751784563 - timing_s/reward:0.05853509530425072 - timing_s/old_log_prob:3.011771934106946 - timing_s/adv:0.13655235059559345 - timing_s/update_actor:12.3400493953377 - timing_s/step:28.899806262925267 - timing_s/stop_profile:0.00012821145355701447 - timing_per_token_ms/gen:0.17244398738453656 - timing_per_token_ms/adv:0.0007984490334319178 - timing_per_token_ms/update_actor:0.07215474848462596 - perf/total_num_tokens:171022 - perf/time_per_step:28.899806262925267 - perf/throughput:739.7194917332336 (TaskRunner pid=2403642) Training Progress: 68%|██████▊ | 68/100 [43:14<16:25, 30.78s/it] (TaskRunner pid=2403642) step:69 - global_seqlen/min:18174 - global_seqlen/max:25281 - global_seqlen/minmax_diff:7107 - global_seqlen/balanced_min:22244 - global_seqlen/balanced_max:22250 - global_seqlen/mean:22247.875 - actor/entropy:0.2577400505542755 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846359670162201 - training/rollout_probs_diff_mean:0.0026801007334142923 - training/rollout_probs_diff_std:0.007650184910744429 - training/rollout_actor_probs_pearson_corr:0.9994962215423584 - rollout_corr/rollout_is_mean:1.000064492225647 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5282295942306519 - rollout_corr/rollout_is_min:0.6526354551315308 - rollout_corr/rollout_is_std:0.025199759751558304 - rollout_corr/rollout_is_eff_sample_size:0.9993654941771045 - rollout_corr/rollout_is_seq_mean:1.0000888109207153 - rollout_corr/rollout_is_seq_std:0.0015610881382599473 - rollout_corr/rollout_is_seq_max:1.0052450895309448 - rollout_corr/rollout_is_seq_min:0.9963499903678894 - rollout_corr/rollout_is_seq_max_deviation:0.005245089530944824 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3210419910028577) - rollout_corr/training_log_ppl:np.float64(0.26073044213990215) - rollout_corr/kl:np.float64(0.0002998370159144059) - rollout_corr/k3_kl:np.float64(0.0003779013446205681) - rollout_corr/rollout_ppl:np.float64(1.3205989100970328) - rollout_corr/rollout_log_ppl:np.float64(0.2604306025314145) - rollout_corr/log_ppl_diff:np.float64(0.0002998396084876731) - rollout_corr/log_ppl_abs_diff:np.float64(0.001366822500131093) - rollout_corr/log_ppl_diff_max:np.float64(0.0053320229053497314) - rollout_corr/log_ppl_diff_min:np.float64(-0.005263388156890869) - rollout_corr/ppl_ratio:np.float64(1.0003015059046447) - rollout_corr/chi2_token:np.float64(0.0009072565007954836) - rollout_corr/chi2_seq:np.float64(0.4102727370336652) - actor/pg_loss:np.float64(-5.396897904574871e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(1.595677440491272e-05) - actor/ppo_kl:np.float64(5.623625808581778e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.050544146448373795) - perf/mfu/actor:np.float64(0.0909921613438466) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.04339218139648) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:2 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013345195911824703 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013345195911824703 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:332.58984375 - response_length/max:779.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:332.58984375 - response_length_non_aborted/max:779.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.65625 - prompt_length/max:461.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001451447606086731 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2613888587802649) - timing_s/agent_loop/generate_sequences/max:np.float64(6.885465707629919) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.741194956128311) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.885465707629919) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:779 - timing_s/gen:13.309920981526375 - timing_s/reward:0.04865463450551033 - timing_s/old_log_prob:2.97403771802783 - timing_s/adv:0.10951554961502552 - timing_s/update_actor:12.642627412453294 - timing_s/step:29.118866013363004 - timing_s/stop_profile:6.626173853874207e-05 - timing_per_token_ms/gen:0.15632431299726782 - timing_per_token_ms/adv:0.0006153146627207403 - timing_per_token_ms/update_actor:0.07103278072879597 - perf/total_num_tokens:177983 - perf/time_per_step:29.118866013363004 - perf/throughput:764.0364494204608 (TaskRunner pid=2403642) Training Progress: 69%|██████▉ | 69/100 [43:43<15:39, 30.30s/it] (TaskRunner pid=2403642) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2403642) validation generation end (TaskRunner pid=2403642) [prompt] system (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2403642) (TaskRunner pid=2403642) A (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) user (TaskRunner pid=2403642) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2403642) (TaskRunner pid=2403642) Lattice Parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) a = 7.82 Å (TaskRunner pid=2403642) b = 7.82 Å (TaskRunner pid=2403642) c = 17.88 Å (TaskRunner pid=2403642) α = 90.00º (TaskRunner pid=2403642) β = 90.00º (TaskRunner pid=2403642) γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) A: 1025.67 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) B: 1094.31 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) C: 1200.45 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) D: 1150.78 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) Please reason step by step. (TaskRunner pid=2403642) assistant (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) [response] (TaskRunner pid=2403642) The volume $ V $ of a unit cell in a crystal structure can be calculated using the formula for a unit cell with given lattice parameters. For a unit cell with orthorhombic symmetry (where all angles are 90 degrees), the formula is: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) V = a \times b \times c (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given: (TaskRunner pid=2403642) - $ a = 7.82 $ Å (TaskRunner pid=2403642) - $ b = 7.82 $ Å (TaskRunner pid=2403642) - $ c = 17.88 $ Å (TaskRunner pid=2403642) (TaskRunner pid=2403642) Substituting these values into the formula: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) V = 7.82 \times 7.82 \times 17.88 (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Now, multiply by $ 17.88 $: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) 61.1524 \times 17.88 = 1094.31 (TaskRunner pid=2403642) $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Thus, the volume of the unit cell is approximately $ 1094.31 $ ų. (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) B (TaskRunner pid=2403642) (TaskRunner pid=2403642) [ground_truth] B (TaskRunner pid=2403642) [score] 1.0 (TaskRunner pid=2403642) [acc] 1.0 (TaskRunner pid=2403642) [pred] B (TaskRunner pid=2403642) [incorrect_format] 1 (TaskRunner pid=2403642) [feedback] (TaskRunner pid=2403642) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2403642) step:70 - global_seqlen/min:20129 - global_seqlen/max:27403 - global_seqlen/minmax_diff:7274 - global_seqlen/balanced_min:22945 - global_seqlen/balanced_max:22953 - global_seqlen/mean:22948.75 - actor/entropy:0.2474832534790039 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.697953999042511 - training/rollout_probs_diff_mean:0.0025176035705953836 - training/rollout_probs_diff_std:0.007814755663275719 - training/rollout_actor_probs_pearson_corr:0.9994720220565796 - rollout_corr/rollout_is_mean:0.9999285340309143 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.189371778309578e-05 - rollout_corr/rollout_is_max:1.6485027074813843 - rollout_corr/rollout_is_min:0.024857966229319572 - rollout_corr/rollout_is_std:0.024460405111312866 - rollout_corr/rollout_is_eff_sample_size:0.9994020463382516 - rollout_corr/rollout_is_seq_mean:0.9999470710754395 - rollout_corr/rollout_is_seq_std:0.0016255476512014866 - rollout_corr/rollout_is_seq_max:1.0055850744247437 - rollout_corr/rollout_is_seq_min:0.994884729385376 - rollout_corr/rollout_is_seq_max_deviation:0.005585074424743652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.314159161876887) - rollout_corr/training_log_ppl:np.float64(0.2612044901034096) - rollout_corr/kl:np.float64(0.0005048151205322426) - rollout_corr/k3_kl:np.float64(0.0004144159162251526) - rollout_corr/rollout_ppl:np.float64(1.3134744726121426) - rollout_corr/rollout_log_ppl:np.float64(0.260699674137868) - rollout_corr/log_ppl_diff:np.float64(0.0005048159655416384) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012870829232269898) - rollout_corr/log_ppl_diff_max:np.float64(0.009302586317062378) - rollout_corr/log_ppl_diff_min:np.float64(-0.0068359375) - rollout_corr/ppl_ratio:np.float64(1.0005064497236162) - rollout_corr/chi2_token:np.float64(0.0006057738792151213) - rollout_corr/chi2_seq:np.float64(0.13440353889018297) - actor/pg_loss:np.float64(-6.187940016388893e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.0769394874805585e-05) - actor/ppo_kl:np.float64(0.00011248510861205929) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11688512889668345) - perf/mfu/actor:np.float64(0.09366797054721682) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.0029411315918) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7593085106382979) - val-aux/sciknoweval/reward/std@16:np.float64(0.03784596039508453) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7730106382978723) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.03245061258021307) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.746063829787234) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.02930060432284286) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7596914893617022) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.037449532559174645) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7857340425531916) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.029635888676231206) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7364255319148935) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.023699060838306855) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7574255319148936) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.02639621551830921) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7981914893617023) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.02430626418681097) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7282872340425532) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.022082307000321155) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7558297872340426) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.015649633880830637) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8088191489361701) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.017956496405970527) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.7177127659574468) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.020493030326682645) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7542765957446809) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.01130426401943146) - val-aux/sciknoweval/score/mean@16:np.float64(0.7593085106382979) - val-aux/sciknoweval/score/std@16:np.float64(0.03784596039508453) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7730106382978723) - val-aux/sciknoweval/score/best@2/std:np.float64(0.03245061258021307) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.746063829787234) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.02930060432284286) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7596914893617022) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.037449532559174645) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7857340425531916) - val-aux/sciknoweval/score/best@4/std:np.float64(0.029635888676231206) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7364255319148935) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.023699060838306855) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7574255319148936) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.02639621551830921) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7981914893617023) - val-aux/sciknoweval/score/best@8/std:np.float64(0.02430626418681097) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7282872340425532) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.022082307000321155) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7558297872340426) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.015649633880830637) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8088191489361701) - val-aux/sciknoweval/score/best@16/std:np.float64(0.017956496405970527) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.7177127659574468) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.020493030326682645) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7542765957446809) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.01130426401943146) - val-core/sciknoweval/acc/mean@16:np.float64(0.7593085106382979) - val-aux/sciknoweval/acc/std@16:np.float64(0.03784596039508453) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7730106382978723) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.03245061258021307) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.746063829787234) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.02930060432284286) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7596914893617022) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.037449532559174645) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7857340425531916) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.029635888676231206) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7364255319148935) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.023699060838306855) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7574255319148936) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.02639621551830921) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7981914893617023) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.02430626418681097) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7282872340425532) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.022082307000321155) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7558297872340426) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.015649633880830637) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8088191489361701) - val-core/sciknoweval/acc/best@16/std:np.float64(0.017956496405970527) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.7177127659574468) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.020493030326682645) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7542765957446809) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.01130426401943146) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:2 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003361461916938424 - critic/advantages/max:0.625 - critic/advantages/min:-0.375 - critic/returns/mean:0.003361461916938424 - critic/returns/max:0.625 - critic/returns/min:-0.375 - response_length/mean:328.4296875 - response_length/max:803.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:328.4296875 - response_length_non_aborted/max:803.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.71875 - prompt_length/max:512.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2191317081451416e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.687570558860898) - timing_s/agent_loop/generate_sequences/max:np.float64(6.912776779383421) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6014377262836206) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.912776779383421) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:803 - timing_s/gen:13.990839572623372 - timing_s/reward:0.060982152819633484 - timing_s/old_log_prob:2.999350441619754 - timing_s/adv:0.12235800549387932 - timing_s/update_actor:12.692470544949174 - timing_s/step:29.955675810575485 - timing_s/testing:41.514320719987154 - timing_s/save_checkpoint:12.88465422205627 - timing_s/stop_profile:3.7822872400283813e-05 - timing_per_token_ms/gen:0.16640309679848916 - timing_per_token_ms/adv:0.0006664742387596237 - timing_per_token_ms/update_actor:0.06913486870172217 - perf/total_num_tokens:183590 - perf/time_per_step:29.955675810575485 - perf/throughput:766.0902109208374 (TaskRunner pid=2403642) Training Progress: 70%|███████ | 70/100 [45:07<23:16, 46.53s/it] (TaskRunner pid=2403642) step:71 - global_seqlen/min:20204 - global_seqlen/max:27625 - global_seqlen/minmax_diff:7421 - global_seqlen/balanced_min:23791 - global_seqlen/balanced_max:23796 - global_seqlen/mean:23793.75 - actor/entropy:0.23197931051254272 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9734354019165039 - training/rollout_probs_diff_mean:0.0025075948797166348 - training/rollout_probs_diff_std:0.008153514936566353 - training/rollout_actor_probs_pearson_corr:0.9994100332260132 - rollout_corr/rollout_is_mean:0.9999135136604309 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2147904019220732e-05 - rollout_corr/rollout_is_max:1.6415942907333374 - rollout_corr/rollout_is_min:0.018141763284802437 - rollout_corr/rollout_is_std:0.02477516233921051 - rollout_corr/rollout_is_eff_sample_size:0.9993864488349556 - rollout_corr/rollout_is_seq_mean:0.9999602437019348 - rollout_corr/rollout_is_seq_std:0.0016766986809670925 - rollout_corr/rollout_is_seq_max:1.0064085721969604 - rollout_corr/rollout_is_seq_min:0.9950252175331116 - rollout_corr/rollout_is_seq_max_deviation:0.006408572196960449 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.294114553835243) - rollout_corr/training_log_ppl:np.float64(0.25148121974780224) - rollout_corr/kl:np.float64(0.0005309915984341718) - rollout_corr/k3_kl:np.float64(0.00038931291395272183) - rollout_corr/rollout_ppl:np.float64(1.2933996450155973) - rollout_corr/rollout_log_ppl:np.float64(0.2509502254106337) - rollout_corr/log_ppl_diff:np.float64(0.0005309943371685222) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013700544222956523) - rollout_corr/log_ppl_diff_max:np.float64(0.01174214482307434) - rollout_corr/log_ppl_diff_min:np.float64(-0.004778727889060974) - rollout_corr/ppl_ratio:np.float64(1.0005327749531716) - rollout_corr/chi2_token:np.float64(0.0004457335453480482) - rollout_corr/chi2_seq:np.float64(0.12818227359093726) - actor/pg_loss:np.float64(-2.503779251128435e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.104266059177462e-05) - actor/ppo_kl:np.float64(0.00013277838810599185) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0483645498752594) - perf/mfu/actor:np.float64(0.09704824978962062) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.64053344726562) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:2 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008568470366299152 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008568470366299152 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:352.7421875 - response_length/max:988.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.7421875 - response_length_non_aborted/max:988.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.8125 - prompt_length/max:482.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.001761019229889e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8390790149569511) - timing_s/agent_loop/generate_sequences/max:np.float64(7.671897960826755) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.904139525635401) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.671897960826755) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:988 - timing_s/gen:14.035376226529479 - timing_s/reward:0.0553808081895113 - timing_s/old_log_prob:3.246291721239686 - timing_s/adv:0.15561982430517673 - timing_s/update_actor:12.717371648177505 - timing_s/step:30.32148109935224 - timing_s/stop_profile:0.00012207403779029846 - timing_per_token_ms/gen:0.15542708053564128 - timing_per_token_ms/adv:0.0008175457016295075 - timing_per_token_ms/update_actor:0.06681046308472553 - perf/total_num_tokens:190350 - perf/time_per_step:30.32148109935224 - perf/throughput:784.7159550695004 (TaskRunner pid=2403642) Training Progress: 71%|███████ | 71/100 [45:37<20:08, 41.69s/it] (TaskRunner pid=2403642) step:72 - global_seqlen/min:21608 - global_seqlen/max:27136 - global_seqlen/minmax_diff:5528 - global_seqlen/balanced_min:24254 - global_seqlen/balanced_max:24258 - global_seqlen/mean:24256.25 - actor/entropy:0.19864249229431152 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1831313669681549 - training/rollout_probs_diff_mean:0.0021919270511716604 - training/rollout_probs_diff_std:0.006964217871427536 - training/rollout_actor_probs_pearson_corr:0.9995092749595642 - rollout_corr/rollout_is_mean:0.999876856803894 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5391958951950073 - rollout_corr/rollout_is_min:0.6549360752105713 - rollout_corr/rollout_is_std:0.022048480808734894 - rollout_corr/rollout_is_eff_sample_size:0.9995141007300881 - rollout_corr/rollout_is_seq_mean:0.9998931884765625 - rollout_corr/rollout_is_seq_std:0.0013704376760870218 - rollout_corr/rollout_is_seq_max:1.003908634185791 - rollout_corr/rollout_is_seq_min:0.9955668449401855 - rollout_corr/rollout_is_seq_max_deviation:0.004433155059814453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2588270916603506) - rollout_corr/training_log_ppl:np.float64(0.2237605569389416) - rollout_corr/kl:np.float64(0.000427495997047056) - rollout_corr/k3_kl:np.float64(0.0003172674925409069) - rollout_corr/rollout_ppl:np.float64(1.2582736886106431) - rollout_corr/rollout_log_ppl:np.float64(0.22333305978099816) - rollout_corr/log_ppl_diff:np.float64(0.000427497157943435) - rollout_corr/log_ppl_abs_diff:np.float64(0.001277009301702492) - rollout_corr/log_ppl_diff_max:np.float64(0.0061948299407958984) - rollout_corr/log_ppl_diff_min:np.float64(-0.004580110311508179) - rollout_corr/ppl_ratio:np.float64(1.0004288996569812) - rollout_corr/chi2_token:np.float64(0.0004238623660057783) - rollout_corr/chi2_seq:np.float64(0.11839799722656608) - actor/pg_loss:np.float64(-8.923967834562063e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.7999737540085334e-05) - actor/ppo_kl:np.float64(4.3629116550647495e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06856918195262551) - perf/mfu/actor:np.float64(0.09946419479837837) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.11327743530273) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:2 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:1.2195121598779224e-05 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:1.2195121598779224e-05 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:360.3515625 - response_length/max:797.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.3515625 - response_length_non_aborted/max:797.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.65625 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.410083293914795e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1268845815211535) - timing_s/agent_loop/generate_sequences/max:np.float64(7.048265373334289) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9732546996310703) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.048265373334289) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:770 - timing_s/gen:13.639525232836604 - timing_s/reward:0.060637516900897026 - timing_s/old_log_prob:3.149775017052889 - timing_s/adv:0.12171594798564911 - timing_s/update_actor:12.700207613408566 - timing_s/step:29.76059423200786 - timing_s/stop_profile:0.00012917257845401764 - timing_per_token_ms/gen:0.1478539320632694 - timing_per_token_ms/adv:0.0006272401339121314 - timing_per_token_ms/update_actor:0.06544811962591376 - perf/total_num_tokens:194050 - perf/time_per_step:29.76059423200786 - perf/throughput:815.0458895713891 (TaskRunner pid=2403642) Training Progress: 72%|███████▏ | 72/100 [46:07<17:47, 38.12s/it] (TaskRunner pid=2403642) step:73 - global_seqlen/min:19053 - global_seqlen/max:28796 - global_seqlen/minmax_diff:9743 - global_seqlen/balanced_min:23456 - global_seqlen/balanced_max:23464 - global_seqlen/mean:23461.25 - actor/entropy:0.22118522226810455 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2151082158088684 - training/rollout_probs_diff_mean:0.0023402885999530554 - training/rollout_probs_diff_std:0.007146602030843496 - training/rollout_actor_probs_pearson_corr:0.9995121359825134 - rollout_corr/rollout_is_mean:1.00008225440979 - rollout_corr/rollout_is_ratio_fraction_high:1.1420479495427571e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1420479495427571e-05 - rollout_corr/rollout_is_max:2.6541292667388916 - rollout_corr/rollout_is_min:0.45500603318214417 - rollout_corr/rollout_is_std:0.024027705192565918 - rollout_corr/rollout_is_eff_sample_size:0.9994232406725769 - rollout_corr/rollout_is_seq_mean:1.000099778175354 - rollout_corr/rollout_is_seq_std:0.0016823429614305496 - rollout_corr/rollout_is_seq_max:1.0069019794464111 - rollout_corr/rollout_is_seq_min:0.995582640171051 - rollout_corr/rollout_is_seq_max_deviation:0.006901979446411133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2888292078860104) - rollout_corr/training_log_ppl:np.float64(0.23963962492416613) - rollout_corr/kl:np.float64(0.00026983858866547195) - rollout_corr/k3_kl:np.float64(0.0003792109853435477) - rollout_corr/rollout_ppl:np.float64(1.2885228227823973) - rollout_corr/rollout_log_ppl:np.float64(0.23936978423444089) - rollout_corr/log_ppl_diff:np.float64(0.00026984068972524256) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012609190453076735) - rollout_corr/log_ppl_diff_max:np.float64(0.007149204611778259) - rollout_corr/log_ppl_diff_min:np.float64(-0.006377771496772766) - rollout_corr/ppl_ratio:np.float64(1.0002713806461543) - rollout_corr/chi2_token:np.float64(0.0011249079834669828) - rollout_corr/chi2_seq:np.float64(0.2845069239847362) - actor/pg_loss:np.float64(4.49218787252903e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.43538634601282e-06) - actor/ppo_kl:np.float64(4.757973179181363e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1339050605893135) - perf/mfu/actor:np.float64(0.09533829770092217) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.18564987182617) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012262739473953843 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:0.0012262739473953843 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:342.0390625 - response_length/max:847.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:342.0390625 - response_length_non_aborted/max:847.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.125 - prompt_length/max:494.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.323974907398224e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.573115924373269) - timing_s/agent_loop/generate_sequences/max:np.float64(7.276758732274175) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.74214253291575) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.276758732274175) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:446 - timing_s/agent_loop/slowest/response_length:838 - timing_s/gen:13.863008869811893 - timing_s/reward:0.05912385508418083 - timing_s/old_log_prob:3.2045726757496595 - timing_s/adv:0.11945493891835213 - timing_s/update_actor:12.806744307279587 - timing_s/step:30.131799463182688 - timing_s/stop_profile:0.00011930055916309357 - timing_per_token_ms/gen:0.15832220449295234 - timing_per_token_ms/adv:0.0006364480735167144 - timing_per_token_ms/update_actor:0.06823349303255148 - perf/total_num_tokens:187690 - perf/time_per_step:30.131799463182688 - perf/throughput:778.6209392727019 (TaskRunner pid=2403642) Training Progress: 73%|███████▎ | 73/100 [46:37<16:04, 35.74s/it] (TaskRunner pid=2403642) step:74 - global_seqlen/min:18126 - global_seqlen/max:26046 - global_seqlen/minmax_diff:7920 - global_seqlen/balanced_min:22541 - global_seqlen/balanced_max:22548 - global_seqlen/mean:22545.0 - actor/entropy:0.260564386844635 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16881024837493896 - training/rollout_probs_diff_mean:0.0027874174993485212 - training/rollout_probs_diff_std:0.0077921925112605095 - training/rollout_actor_probs_pearson_corr:0.9994821548461914 - rollout_corr/rollout_is_mean:1.000143051147461 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.586216926574707 - rollout_corr/rollout_is_min:0.6137421727180481 - rollout_corr/rollout_is_std:0.025568412616848946 - rollout_corr/rollout_is_eff_sample_size:0.9993470405186317 - rollout_corr/rollout_is_seq_mean:1.0001143217086792 - rollout_corr/rollout_is_seq_std:0.0015101114986464381 - rollout_corr/rollout_is_seq_max:1.0060126781463623 - rollout_corr/rollout_is_seq_min:0.9956173896789551 - rollout_corr/rollout_is_seq_max_deviation:0.006012678146362305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3129614721983671) - rollout_corr/training_log_ppl:np.float64(0.26557272001809906) - rollout_corr/kl:np.float64(0.0003072068811249551) - rollout_corr/k3_kl:np.float64(0.0003651777730624417) - rollout_corr/rollout_ppl:np.float64(1.312554101459682) - rollout_corr/rollout_log_ppl:np.float64(0.26526551083952654) - rollout_corr/log_ppl_diff:np.float64(0.0003072091785725206) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012191955756861717) - rollout_corr/log_ppl_diff_max:np.float64(0.004254922270774841) - rollout_corr/log_ppl_diff_min:np.float64(-0.005225449800491333) - rollout_corr/ppl_ratio:np.float64(1.0003084915224463) - rollout_corr/chi2_token:np.float64(0.0008592354133725166) - rollout_corr/chi2_seq:np.float64(0.40940430806949735) - actor/pg_loss:np.float64(6.841961294412613e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.0106103572034044e-05) - actor/ppo_kl:np.float64(8.371459145450899e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.05365279596298933) - perf/mfu/actor:np.float64(0.09285714441128551) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.1694221496582) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:2 - critic/score/mean:0.8359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009363640565425158 - critic/advantages/max:0.25 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0009363640565425158 - critic/returns/max:0.25 - critic/returns/min:-0.75 - response_length/mean:318.09375 - response_length/max:961.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:318.09375 - response_length_non_aborted/max:961.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:386.4375 - prompt_length/max:481.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012472458183765411 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4892469886690378) - timing_s/agent_loop/generate_sequences/max:np.float64(7.678874081000686) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.469133668273571) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.678874081000686) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:961 - timing_s/gen:14.281971836462617 - timing_s/reward:0.06201468966901302 - timing_s/old_log_prob:2.9973433893173933 - timing_s/adv:0.12412707321345806 - timing_s/update_actor:12.666662221774459 - timing_s/step:30.221048586070538 - timing_s/stop_profile:5.814991891384125e-05 - timing_per_token_ms/gen:0.1753852519459502 - timing_per_token_ms/adv:0.0006882184143571638 - timing_per_token_ms/update_actor:0.07022988590471534 - perf/total_num_tokens:180360 - perf/time_per_step:30.221048586070538 - perf/throughput:746.0032346591515 (TaskRunner pid=2403642) Training Progress: 74%|███████▍ | 74/100 [47:08<14:46, 34.09s/it] (TaskRunner pid=2403642) step:75 - global_seqlen/min:19411 - global_seqlen/max:26389 - global_seqlen/minmax_diff:6978 - global_seqlen/balanced_min:22835 - global_seqlen/balanced_max:22839 - global_seqlen/mean:22837.125 - actor/entropy:0.26598554849624634 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12735021114349365 - training/rollout_probs_diff_mean:0.0026302863843739033 - training/rollout_probs_diff_std:0.007375993765890598 - training/rollout_actor_probs_pearson_corr:0.9995732307434082 - rollout_corr/rollout_is_mean:0.9998831152915955 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.4389069080352783 - rollout_corr/rollout_is_min:0.6956850290298462 - rollout_corr/rollout_is_std:0.024746274575591087 - rollout_corr/rollout_is_eff_sample_size:0.9993877585302487 - rollout_corr/rollout_is_seq_mean:0.999903678894043 - rollout_corr/rollout_is_seq_std:0.001642070128582418 - rollout_corr/rollout_is_seq_max:1.006669282913208 - rollout_corr/rollout_is_seq_min:0.9947586059570312 - rollout_corr/rollout_is_seq_max_deviation:0.006669282913208008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3507126765325665) - rollout_corr/training_log_ppl:np.float64(0.28722253389423713) - rollout_corr/kl:np.float64(0.0003447701374001966) - rollout_corr/k3_kl:np.float64(0.0003724003217939753) - rollout_corr/rollout_ppl:np.float64(1.350269827991724) - rollout_corr/rollout_log_ppl:np.float64(0.286877762133372) - rollout_corr/log_ppl_diff:np.float64(0.00034477176086511463) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012942667381139472) - rollout_corr/log_ppl_diff_max:np.float64(0.005844295024871826) - rollout_corr/log_ppl_diff_min:np.float64(-0.006119906902313232) - rollout_corr/ppl_ratio:np.float64(1.0003462715540081) - rollout_corr/chi2_token:np.float64(0.0008182954043149948) - rollout_corr/chi2_seq:np.float64(0.32993280817754567) - actor/pg_loss:np.float64(2.505222801119089e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.734913768013939e-06) - actor/ppo_kl:np.float64(-8.327829606002979e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06800705939531326) - perf/mfu/actor:np.float64(0.09338293257404955) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.09074020385742) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:2 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0049235341139137745 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0049235341139137745 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:319.53515625 - response_length/max:740.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:319.53515625 - response_length_non_aborted/max:740.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.125 - prompt_length/max:493.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.5044347643852234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9954011496156454) - timing_s/agent_loop/generate_sequences/max:np.float64(6.424747133627534) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6563248890815885) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.424747133627534) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:740 - timing_s/gen:13.63974541053176 - timing_s/reward:0.06020428612828255 - timing_s/old_log_prob:2.9700843077152967 - timing_s/adv:0.11695144511759281 - timing_s/update_actor:12.65475906431675 - timing_s/step:29.526658721268177 - timing_s/stop_profile:2.7820467948913574e-05 - timing_per_token_ms/gen:0.16674301549530884 - timing_per_token_ms/adv:0.0006401388370777452 - timing_per_token_ms/update_actor:0.06926637582618625 - perf/total_num_tokens:182697 - perf/time_per_step:29.526658721268177 - perf/throughput:773.4408832229406 (TaskRunner pid=2403642) Training Progress: 75%|███████▌ | 75/100 [47:37<13:38, 32.73s/it] (TaskRunner pid=2403642) step:76 - global_seqlen/min:18401 - global_seqlen/max:23716 - global_seqlen/minmax_diff:5315 - global_seqlen/balanced_min:21350 - global_seqlen/balanced_max:21353 - global_seqlen/mean:21352.0 - actor/entropy:0.29671216011047363 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932341992855072 - training/rollout_probs_diff_mean:0.002921811304986477 - training/rollout_probs_diff_std:0.007850022055208683 - training/rollout_actor_probs_pearson_corr:0.9994902014732361 - rollout_corr/rollout_is_mean:0.9999136924743652 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6809651899384335e-05 - rollout_corr/rollout_is_max:1.6220922470092773 - rollout_corr/rollout_is_min:0.3131377696990967 - rollout_corr/rollout_is_std:0.026378488168120384 - rollout_corr/rollout_is_eff_sample_size:0.9993046592121337 - rollout_corr/rollout_is_seq_mean:0.9999048113822937 - rollout_corr/rollout_is_seq_std:0.001711147604510188 - rollout_corr/rollout_is_seq_max:1.0068550109863281 - rollout_corr/rollout_is_seq_min:0.9951537847518921 - rollout_corr/rollout_is_seq_max_deviation:0.006855010986328125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3681150269694626) - rollout_corr/training_log_ppl:np.float64(0.3019279351574369) - rollout_corr/kl:np.float64(0.00037963077198455153) - rollout_corr/k3_kl:np.float64(0.00038022535707682437) - rollout_corr/rollout_ppl:np.float64(1.3675709716044366) - rollout_corr/rollout_log_ppl:np.float64(0.30154830423998646) - rollout_corr/log_ppl_diff:np.float64(0.0003796309174504131) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012849238410126418) - rollout_corr/log_ppl_diff_max:np.float64(0.005592137575149536) - rollout_corr/log_ppl_diff_min:np.float64(-0.0062049925327301025) - rollout_corr/ppl_ratio:np.float64(1.0003810485359281) - rollout_corr/chi2_token:np.float64(0.0007722426671534777) - rollout_corr/chi2_seq:np.float64(0.23392329807393253) - actor/pg_loss:np.float64(-9.966082870960236e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.643598342430778e-05) - actor/ppo_kl:np.float64(-7.630967955130075e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1309410072863102) - perf/mfu/actor:np.float64(0.08796069759198259) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.3106803894043) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:2 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013857238227501512 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013857238227501512 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:291.40625 - response_length/max:715.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.40625 - response_length_non_aborted/max:715.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:375.84375 - prompt_length/max:471.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.397596836090088e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8652194179594517) - timing_s/agent_loop/generate_sequences/max:np.float64(6.0747572015970945) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.322189557933598) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.0747572015970945) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:699 - timing_s/gen:13.099566580727696 - timing_s/reward:0.05862412601709366 - timing_s/old_log_prob:3.0302375964820385 - timing_s/adv:0.11894695088267326 - timing_s/update_actor:12.693279696628451 - timing_s/step:29.086983252316713 - timing_s/stop_profile:0.0001168716698884964 - timing_per_token_ms/gen:0.17559740724836054 - timing_per_token_ms/adv:0.0006963454880261408 - timing_per_token_ms/update_actor:0.07430966476576228 - perf/total_num_tokens:170816 - perf/time_per_step:29.086983252316713 - perf/throughput:734.074063809947 (TaskRunner pid=2403642) Training Progress: 76%|███████▌ | 76/100 [48:06<12:39, 31.65s/it] (TaskRunner pid=2403642) step:77 - global_seqlen/min:20873 - global_seqlen/max:27379 - global_seqlen/minmax_diff:6506 - global_seqlen/balanced_min:24520 - global_seqlen/balanced_max:24522 - global_seqlen/mean:24521.25 - actor/entropy:0.21010300517082214 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9525610208511353 - training/rollout_probs_diff_mean:0.002213509287685156 - training/rollout_probs_diff_std:0.007591753266751766 - training/rollout_actor_probs_pearson_corr:0.9994261264801025 - rollout_corr/rollout_is_mean:0.9999380707740784 - rollout_corr/rollout_is_ratio_fraction_high:1.0898707841988653e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.269612352596596e-05 - rollout_corr/rollout_is_max:2.0322699546813965 - rollout_corr/rollout_is_min:1.2050280020048376e-05 - rollout_corr/rollout_is_std:0.02297777123749256 - rollout_corr/rollout_is_eff_sample_size:0.9994720625032244 - rollout_corr/rollout_is_seq_mean:0.9999042749404907 - rollout_corr/rollout_is_seq_std:0.0015725604025647044 - rollout_corr/rollout_is_seq_max:1.0046145915985107 - rollout_corr/rollout_is_seq_min:0.9941759705543518 - rollout_corr/rollout_is_seq_max_deviation:0.005824029445648193 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2686810540035367) - rollout_corr/training_log_ppl:np.float64(0.22998299021855928) - rollout_corr/kl:np.float64(0.0006073243849300525) - rollout_corr/k3_kl:np.float64(0.0005009522885188744) - rollout_corr/rollout_ppl:np.float64(1.2678813147358596) - rollout_corr/rollout_log_ppl:np.float64(0.22937566417385824) - rollout_corr/log_ppl_diff:np.float64(0.0006073260447010398) - rollout_corr/log_ppl_abs_diff:np.float64(0.001405421266099438) - rollout_corr/log_ppl_diff_max:np.float64(0.05526180565357208) - rollout_corr/log_ppl_diff_min:np.float64(-0.004332512617111206) - rollout_corr/ppl_ratio:np.float64(1.0006146889645606) - rollout_corr/chi2_token:np.float64(0.00042232475243508816) - rollout_corr/chi2_seq:np.float64(0.13180363178253174) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(2.2369566373470207e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.09757031732597408) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.31251525878906) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:2 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:358.4140625 - response_length/max:697.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:358.4140625 - response_length_non_aborted/max:697.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.875 - prompt_length/max:510.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000134320929646492 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9717217683792114) - timing_s/agent_loop/generate_sequences/max:np.float64(6.515862211585045) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.02899411416729) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.515862211585045) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:420 - timing_s/agent_loop/slowest/response_length:697 - timing_s/gen:13.739952562376857 - timing_s/reward:0.061353178694844246 - timing_s/old_log_prob:3.051538873463869 - timing_s/adv:0.11956898868083954 - timing_s/update_actor:12.933352803811431 - timing_s/step:29.99167255870998 - timing_s/stop_profile:0.00017154216766357422 - timing_per_token_ms/gen:0.14974772284997773 - timing_per_token_ms/adv:0.0006095171977409366 - timing_per_token_ms/update_actor:0.06592931031152281 - perf/total_num_tokens:196170 - perf/time_per_step:29.99167255870998 - perf/throughput:817.6019510748728 (TaskRunner pid=2403642) Training Progress: 77%|███████▋ | 77/100 [48:36<11:56, 31.17s/it] (TaskRunner pid=2403642) step:78 - global_seqlen/min:21151 - global_seqlen/max:25607 - global_seqlen/minmax_diff:4456 - global_seqlen/balanced_min:23008 - global_seqlen/balanced_max:23012 - global_seqlen/mean:23011.25 - actor/entropy:0.2615104913711548 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4400331377983093 - training/rollout_probs_diff_mean:0.0026174753438681364 - training/rollout_probs_diff_std:0.007456088438630104 - training/rollout_actor_probs_pearson_corr:0.9995262026786804 - rollout_corr/rollout_is_mean:0.9998167753219604 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2719733604171779e-05 - rollout_corr/rollout_is_max:1.454869270324707 - rollout_corr/rollout_is_min:0.46178218722343445 - rollout_corr/rollout_is_std:0.024490846320986748 - rollout_corr/rollout_is_eff_sample_size:0.9994004984726531 - rollout_corr/rollout_is_seq_mean:0.9998263716697693 - rollout_corr/rollout_is_seq_std:0.0016342015005648136 - rollout_corr/rollout_is_seq_max:1.005255103111267 - rollout_corr/rollout_is_seq_min:0.9942693710327148 - rollout_corr/rollout_is_seq_max_deviation:0.005730628967285156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.330007707234472) - rollout_corr/training_log_ppl:np.float64(0.27612566971220076) - rollout_corr/kl:np.float64(0.0005081333625520656) - rollout_corr/k3_kl:np.float64(0.00034416606136744576) - rollout_corr/rollout_ppl:np.float64(1.3292845152318478) - rollout_corr/rollout_log_ppl:np.float64(0.2756175332906423) - rollout_corr/log_ppl_diff:np.float64(0.0005081364215584472) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013684313307749107) - rollout_corr/log_ppl_diff_max:np.float64(0.006606578826904297) - rollout_corr/log_ppl_diff_min:np.float64(-0.004864111542701721) - rollout_corr/ppl_ratio:np.float64(1.00050969584845) - rollout_corr/chi2_token:np.float64(0.00036964891478419304) - rollout_corr/chi2_seq:np.float64(0.0903538609854877) - actor/pg_loss:np.float64(2.2566295228898525e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(1.0493426182200949e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.020936077926307917) - perf/mfu/actor:np.float64(0.09409445481639155) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.23200035095215) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004102113889530301 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004102113889530301 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:307.1015625 - response_length/max:779.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:307.1015625 - response_length_non_aborted/max:779.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.0 - prompt_length/max:493.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012887082993984222 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6045669373124838) - timing_s/agent_loop/generate_sequences/max:np.float64(6.628957685083151) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5532616262498777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.628957685083151) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:490 - timing_s/agent_loop/slowest/response_length:779 - timing_s/gen:13.354373624548316 - timing_s/reward:0.06010809354484081 - timing_s/old_log_prob:3.0047842897474766 - timing_s/adv:0.11696131527423859 - timing_s/update_actor:12.723615363240242 - timing_s/step:29.346568485721946 - timing_s/stop_profile:0.00011821836233139038 - timing_per_token_ms/gen:0.16986407215330224 - timing_per_token_ms/adv:0.0006353485538282285 - timing_per_token_ms/update_actor:0.06911627662143648 - perf/total_num_tokens:184090 - perf/time_per_step:29.346568485721946 - perf/throughput:784.1206378590981 (TaskRunner pid=2403642) Training Progress: 78%|███████▊ | 78/100 [49:06<11:13, 30.63s/it] (TaskRunner pid=2403642) step:79 - global_seqlen/min:18926 - global_seqlen/max:25995 - global_seqlen/minmax_diff:7069 - global_seqlen/balanced_min:23504 - global_seqlen/balanced_max:23505 - global_seqlen/mean:23504.5 - actor/entropy:0.24806106090545654 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102373421192169 - training/rollout_probs_diff_mean:0.0023650529328733683 - training/rollout_probs_diff_std:0.0071376836858689785 - training/rollout_actor_probs_pearson_corr:0.9995306730270386 - rollout_corr/rollout_is_mean:1.0001239776611328 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1717834240698721e-05 - rollout_corr/rollout_is_max:1.7178680896759033 - rollout_corr/rollout_is_min:0.4388447403907776 - rollout_corr/rollout_is_std:0.023599699139595032 - rollout_corr/rollout_is_eff_sample_size:0.99944348329181 - rollout_corr/rollout_is_seq_mean:1.0001518726348877 - rollout_corr/rollout_is_seq_std:0.0014191392110660672 - rollout_corr/rollout_is_seq_max:1.004914402961731 - rollout_corr/rollout_is_seq_min:0.9959833025932312 - rollout_corr/rollout_is_seq_max_deviation:0.004914402961730957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3128615743480623) - rollout_corr/training_log_ppl:np.float64(0.26136835565557703) - rollout_corr/kl:np.float64(0.00019384605705319302) - rollout_corr/k3_kl:np.float64(0.0003144730202855328) - rollout_corr/rollout_ppl:np.float64(1.3125933022238314) - rollout_corr/rollout_log_ppl:np.float64(0.26117450813762844) - rollout_corr/log_ppl_diff:np.float64(0.00019384751794859767) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011084461002610624) - rollout_corr/log_ppl_diff_max:np.float64(0.0037963688373565674) - rollout_corr/log_ppl_diff_min:np.float64(-0.004113912582397461) - rollout_corr/ppl_ratio:np.float64(1.0001948736608028) - rollout_corr/chi2_token:np.float64(0.0008782581426203251) - rollout_corr/chi2_seq:np.float64(0.2646336832549423) - actor/pg_loss:np.float64(1.6814330592751503e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.797860573395155e-06) - actor/ppo_kl:np.float64(5.1398302620242475e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.03347182273864746) - perf/mfu/actor:np.float64(0.09539341168894122) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.25070190429688) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003666217438876629 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003666217438876629 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:333.359375 - response_length/max:757.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:333.359375 - response_length_non_aborted/max:757.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.15625 - prompt_length/max:494.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00027751922607421875 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0520722176879644) - timing_s/agent_loop/generate_sequences/max:np.float64(6.681050945073366) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8504629519884475) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.681050945073366) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:422 - timing_s/agent_loop/slowest/response_length:757 - timing_s/gen:13.767290970310569 - timing_s/reward:0.0555742122232914 - timing_s/old_log_prob:3.168600793927908 - timing_s/adv:0.11057946272194386 - timing_s/update_actor:12.864728827029467 - timing_s/step:30.068919649347663 - timing_s/stop_profile:0.00011127069592475891 - timing_per_token_ms/gen:0.16132283771163075 - timing_per_token_ms/adv:0.0005880760211977699 - timing_per_token_ms/update_actor:0.06841630765932835 - perf/total_num_tokens:188036 - perf/time_per_step:30.068919649347663 - perf/throughput:781.6875456152254 (TaskRunner pid=2403642) Training Progress: 79%|███████▉ | 79/100 [49:36<10:43, 30.63s/it] (TaskRunner pid=2403642) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2403642) validation generation end (TaskRunner pid=2403642) [prompt] system (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2403642) (TaskRunner pid=2403642) A (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) user (TaskRunner pid=2403642) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2403642) (TaskRunner pid=2403642) Lattice Parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) a = 7.82 Å (TaskRunner pid=2403642) b = 7.82 Å (TaskRunner pid=2403642) c = 17.88 Å (TaskRunner pid=2403642) α = 90.00º (TaskRunner pid=2403642) β = 90.00º (TaskRunner pid=2403642) γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) A: 1025.67 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) B: 1094.31 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) C: 1200.45 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) D: 1150.78 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) Please reason step by step. (TaskRunner pid=2403642) assistant (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) [response] (TaskRunner pid=2403642) The volume of a unit cell in a crystal structure can be calculated using the formula for the volume of a parallelepiped defined by the lattice vectors: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \cdot b \cdot c \cdot \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) In this case, the crystal structure is a unit cell with all angles equal to 90°, which means it is a **primitive orthorhombic** unit cell. For such a structure, the formula simplifies to: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \cdot b \cdot c $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given: (TaskRunner pid=2403642) - a = 7.82 Å (TaskRunner pid=2403642) - b = 7.82 Å (TaskRunner pid=2403642) - c = 17.88 Å (TaskRunner pid=2403642) (TaskRunner pid=2403642) Now, calculate the volume: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = 7.82 \cdot 7.82 \cdot 17.88 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) First, calculate $ 7.82 \cdot 7.82 $: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 7.82 \cdot 7.82 = 61.1524 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Then multiply by 17.88: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ 61.1524 \cdot 17.88 = 1094.31 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Therefore, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) B (TaskRunner pid=2403642) (TaskRunner pid=2403642) [ground_truth] B (TaskRunner pid=2403642) [score] 1.0 (TaskRunner pid=2403642) [acc] 1.0 (TaskRunner pid=2403642) [pred] B (TaskRunner pid=2403642) [incorrect_format] 1 (TaskRunner pid=2403642) [feedback] (TaskRunner pid=2403642) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2403642) step:80 - global_seqlen/min:20780 - global_seqlen/max:26076 - global_seqlen/minmax_diff:5296 - global_seqlen/balanced_min:23379 - global_seqlen/balanced_max:23381 - global_seqlen/mean:23380.0 - actor/entropy:0.2778765857219696 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23994797468185425 - training/rollout_probs_diff_mean:0.0025989830028265715 - training/rollout_probs_diff_std:0.007374095730483532 - training/rollout_actor_probs_pearson_corr:0.9996104836463928 - rollout_corr/rollout_is_mean:0.9999150037765503 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1238986189709976e-05 - rollout_corr/rollout_is_max:1.4688119888305664 - rollout_corr/rollout_is_min:0.30741897225379944 - rollout_corr/rollout_is_std:0.02457466721534729 - rollout_corr/rollout_is_eff_sample_size:0.9993962121006339 - rollout_corr/rollout_is_seq_mean:0.999893307685852 - rollout_corr/rollout_is_seq_std:0.0016186292050406337 - rollout_corr/rollout_is_seq_max:1.0051453113555908 - rollout_corr/rollout_is_seq_min:0.9950850605964661 - rollout_corr/rollout_is_seq_max_deviation:0.00514531135559082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3502345071174204) - rollout_corr/training_log_ppl:np.float64(0.289737257262459) - rollout_corr/kl:np.float64(0.0005005977039438392) - rollout_corr/k3_kl:np.float64(0.000352018927735287) - rollout_corr/rollout_ppl:np.float64(1.3495450923219323) - rollout_corr/rollout_log_ppl:np.float64(0.2892366573069012) - rollout_corr/log_ppl_diff:np.float64(0.0005005999555578455) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013773568934993818) - rollout_corr/log_ppl_diff_max:np.float64(0.006760120391845703) - rollout_corr/log_ppl_diff_min:np.float64(-0.004993259906768799) - rollout_corr/ppl_ratio:np.float64(1.0005022925324738) - rollout_corr/chi2_token:np.float64(0.00039854785427451134) - rollout_corr/chi2_seq:np.float64(0.2169972979463637) - actor/pg_loss:np.float64(-0.00011445709969848394) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.960122886492172e-05) - actor/ppo_kl:np.float64(6.888215898825933e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12062663584947586) - perf/mfu/actor:np.float64(0.09535256392760892) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.24382019042969) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7506648936170213) - val-aux/sciknoweval/reward/std@16:np.float64(0.03880700651700434) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7653510638297872) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.029800870503189608) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7373936170212767) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.0355050684444622) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7515851063829787) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.03928199705006994) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7755851063829787) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.02222459000259867) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7243510638297872) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.03196328070235044) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7520319148936169) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.027285394753994465) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7842234042553193) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.016187705596573548) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7106489361702127) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.029416911005616678) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7507765957446809) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.015715671528587116) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7905000000000002) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.010857614698018669) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6966276595744681) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.024822143344578985) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.748595744680851) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.01193164443697504) - val-aux/sciknoweval/score/mean@16:np.float64(0.7506648936170213) - val-aux/sciknoweval/score/std@16:np.float64(0.03880700651700434) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7653510638297872) - val-aux/sciknoweval/score/best@2/std:np.float64(0.029800870503189608) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7373936170212767) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.0355050684444622) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7515851063829787) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.03928199705006994) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7755851063829787) - val-aux/sciknoweval/score/best@4/std:np.float64(0.02222459000259867) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7243510638297872) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.03196328070235044) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7520319148936169) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.027285394753994465) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7842234042553193) - val-aux/sciknoweval/score/best@8/std:np.float64(0.016187705596573548) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7106489361702127) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.029416911005616678) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7507765957446809) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.015715671528587116) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7905000000000002) - val-aux/sciknoweval/score/best@16/std:np.float64(0.010857614698018669) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6966276595744681) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.024822143344578985) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.748595744680851) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.01193164443697504) - val-core/sciknoweval/acc/mean@16:np.float64(0.7506648936170213) - val-aux/sciknoweval/acc/std@16:np.float64(0.03880700651700434) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7653510638297872) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.029800870503189608) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7373936170212767) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.0355050684444622) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7515851063829787) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.03928199705006994) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7755851063829787) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.02222459000259867) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7243510638297872) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.03196328070235044) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7520319148936169) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.027285394753994465) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7842234042553193) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.016187705596573548) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7106489361702127) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.029416911005616678) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7507765957446809) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.015715671528587116) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7905000000000002) - val-core/sciknoweval/acc/best@16/std:np.float64(0.010857614698018669) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6966276595744681) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.024822143344578985) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.748595744680851) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.01193164443697504) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001062084105797112 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001062084105797112 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:347.5625 - response_length/max:801.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:347.5625 - response_length_non_aborted/max:801.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.0625 - prompt_length/max:488.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.711459279060364e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8344800118356943) - timing_s/agent_loop/generate_sequences/max:np.float64(6.994876081123948) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8829950507715694) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.994876081123948) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:801 - timing_s/gen:13.721989672631025 - timing_s/reward:0.06023436784744263 - timing_s/old_log_prob:3.066313661634922 - timing_s/adv:0.11803494021296501 - timing_s/update_actor:12.887175291776657 - timing_s/step:29.93937893025577 - timing_s/testing:42.340679252520204 - timing_s/save_checkpoint:12.914493877440691 - timing_s/stop_profile:3.894791007041931e-05 - timing_per_token_ms/gen:0.1542212469950439 - timing_per_token_ms/adv:0.0006310679010530636 - timing_per_token_ms/update_actor:0.06890063778751421 - perf/total_num_tokens:187040 - perf/time_per_step:29.93937893025577 - perf/throughput:780.911322658498 (TaskRunner pid=2403642) Training Progress: 80%|████████ | 80/100 [51:02<15:40, 47.01s/it] (TaskRunner pid=2403642) step:81 - global_seqlen/min:18017 - global_seqlen/max:28164 - global_seqlen/minmax_diff:10147 - global_seqlen/balanced_min:21751 - global_seqlen/balanced_max:21756 - global_seqlen/mean:21754.5 - actor/entropy:0.2879306674003601 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4863179922103882 - training/rollout_probs_diff_mean:0.0025793167296797037 - training/rollout_probs_diff_std:0.007397323846817017 - training/rollout_actor_probs_pearson_corr:0.9995822906494141 - rollout_corr/rollout_is_mean:1.0000742673873901 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.5967281544581056e-05 - rollout_corr/rollout_is_max:1.549249529838562 - rollout_corr/rollout_is_min:0.29609444737434387 - rollout_corr/rollout_is_std:0.024538258090615273 - rollout_corr/rollout_is_eff_sample_size:0.999398593413883 - rollout_corr/rollout_is_seq_mean:1.0001031160354614 - rollout_corr/rollout_is_seq_std:0.0016027017263695598 - rollout_corr/rollout_is_seq_max:1.0047954320907593 - rollout_corr/rollout_is_seq_min:0.9945899248123169 - rollout_corr/rollout_is_seq_max_deviation:0.0054100751876831055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3673228165134788) - rollout_corr/training_log_ppl:np.float64(0.2993711391609395) - rollout_corr/kl:np.float64(0.00029862392293544815) - rollout_corr/k3_kl:np.float64(0.00041313828008071596) - rollout_corr/rollout_ppl:np.float64(1.366905136499554) - rollout_corr/rollout_log_ppl:np.float64(0.2990725145646138) - rollout_corr/log_ppl_diff:np.float64(0.0002986245963256806) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013556781050283462) - rollout_corr/log_ppl_diff_max:np.float64(0.006214797496795654) - rollout_corr/log_ppl_diff_min:np.float64(-0.004116475582122803) - rollout_corr/ppl_ratio:np.float64(1.0003002183511853) - rollout_corr/chi2_token:np.float64(0.001083672046661377) - rollout_corr/chi2_seq:np.float64(0.25717995106242597) - actor/pg_loss:np.float64(4.586763679981232e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(7.988616901499768e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1057282155379653) - perf/mfu/actor:np.float64(0.08956160130432747) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(107.28913879394531) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0001314593682764098 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0001314593682764098 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:300.859375 - response_length/max:690.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:300.859375 - response_length_non_aborted/max:690.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:378.96875 - prompt_length/max:524.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6717782020568848e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.362369280308485) - timing_s/agent_loop/generate_sequences/max:np.float64(6.274985628202558) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3638904910403653) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.274985628202558) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:688 - timing_s/gen:12.462707381695509 - timing_s/reward:0.05432135425508022 - timing_s/old_log_prob:3.1264919862151146 - timing_s/adv:0.11341601237654686 - timing_s/update_actor:12.697534121572971 - timing_s/step:28.537394113838673 - timing_s/stop_profile:0.00011392123997211456 - timing_per_token_ms/gen:0.1618113137067711 - timing_per_token_ms/adv:0.0006516813324630931 - timing_per_token_ms/update_actor:0.07295923901705953 - perf/total_num_tokens:174036 - perf/time_per_step:28.537394113838673 - perf/throughput:762.3155748986402 (TaskRunner pid=2403642) Training Progress: 81%|████████ | 81/100 [51:30<13:08, 41.48s/it] (TaskRunner pid=2403642) step:82 - global_seqlen/min:20421 - global_seqlen/max:24727 - global_seqlen/minmax_diff:4306 - global_seqlen/balanced_min:22230 - global_seqlen/balanced_max:22232 - global_seqlen/mean:22231.125 - actor/entropy:0.32268545031547546 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15518474578857422 - training/rollout_probs_diff_mean:0.0027826493605971336 - training/rollout_probs_diff_std:0.0072498926892876625 - training/rollout_actor_probs_pearson_corr:0.9995722770690918 - rollout_corr/rollout_is_mean:1.000037431716919 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.266447998204967e-05 - rollout_corr/rollout_is_max:1.586849331855774 - rollout_corr/rollout_is_min:0.3824073374271393 - rollout_corr/rollout_is_std:0.025383582338690758 - rollout_corr/rollout_is_eff_sample_size:0.9993560886789191 - rollout_corr/rollout_is_seq_mean:1.0000476837158203 - rollout_corr/rollout_is_seq_std:0.001457554753869772 - rollout_corr/rollout_is_seq_max:1.0035213232040405 - rollout_corr/rollout_is_seq_min:0.9956403374671936 - rollout_corr/rollout_is_seq_max_deviation:0.0043596625328063965 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4052514918148518) - rollout_corr/training_log_ppl:np.float64(0.32896969285502564) - rollout_corr/kl:np.float64(0.00020728115616858167) - rollout_corr/k3_kl:np.float64(0.00039224870215548435) - rollout_corr/rollout_ppl:np.float64(1.4049864276312292) - rollout_corr/rollout_log_ppl:np.float64(0.32876241035410203) - rollout_corr/log_ppl_diff:np.float64(0.00020728250092361122) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013390206877375022) - rollout_corr/log_ppl_diff_max:np.float64(0.005917102098464966) - rollout_corr/log_ppl_diff_min:np.float64(-0.004551202058792114) - rollout_corr/ppl_ratio:np.float64(1.0002087897155434) - rollout_corr/chi2_token:np.float64(0.0011829480063170195) - rollout_corr/chi2_seq:np.float64(0.323864143807441) - actor/pg_loss:np.float64(-2.7368776500225067e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.27955094689969e-05) - actor/ppo_kl:np.float64(-7.605274061872613e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06758737610653043) - perf/mfu/actor:np.float64(0.09014706940472565) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.4574966430664) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002290687756612897 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.002290687756612897 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:308.44140625 - response_length/max:903.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.44140625 - response_length_non_aborted/max:903.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:386.28125 - prompt_length/max:510.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011214055120944977 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8820667397230864) - timing_s/agent_loop/generate_sequences/max:np.float64(7.370335368439555) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4890292429554393) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.370335368439555) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:903 - timing_s/gen:14.1277469471097 - timing_s/reward:0.0628491472452879 - timing_s/old_log_prob:2.9511485043913126 - timing_s/adv:0.12012522853910923 - timing_s/update_actor:12.779416888952255 - timing_s/step:30.128240322694182 - timing_s/stop_profile:0.00010496936738491058 - timing_per_token_ms/gen:0.17892056771203124 - timing_per_token_ms/adv:0.0006754338148604109 - timing_per_token_ms/update_actor:0.07185543291754384 - perf/total_num_tokens:177849 - perf/time_per_step:30.128240322694182 - perf/throughput:737.8832869722678 (TaskRunner pid=2403642) Training Progress: 82%|████████▏ | 82/100 [52:00<11:25, 38.09s/it] (TaskRunner pid=2403642) step:83 - global_seqlen/min:20784 - global_seqlen/max:24821 - global_seqlen/minmax_diff:4037 - global_seqlen/balanced_min:22452 - global_seqlen/balanced_max:22456 - global_seqlen/mean:22454.375 - actor/entropy:0.28845345973968506 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24496477842330933 - training/rollout_probs_diff_mean:0.0025984023232012987 - training/rollout_probs_diff_std:0.007140541914850473 - training/rollout_actor_probs_pearson_corr:0.9996364712715149 - rollout_corr/rollout_is_mean:0.9999492764472961 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6114643812179565 - rollout_corr/rollout_is_min:0.5105829834938049 - rollout_corr/rollout_is_std:0.02469322830438614 - rollout_corr/rollout_is_eff_sample_size:0.9993904969951359 - rollout_corr/rollout_is_seq_mean:1.0000005960464478 - rollout_corr/rollout_is_seq_std:0.001637228881008923 - rollout_corr/rollout_is_seq_max:1.0061671733856201 - rollout_corr/rollout_is_seq_min:0.9956061244010925 - rollout_corr/rollout_is_seq_max_deviation:0.006167173385620117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3608696786686778) - rollout_corr/training_log_ppl:np.float64(0.298661266482668) - rollout_corr/kl:np.float64(0.0002497441816160517) - rollout_corr/k3_kl:np.float64(0.0003628389737002635) - rollout_corr/rollout_ppl:np.float64(1.360557473730296) - rollout_corr/rollout_log_ppl:np.float64(0.2984115183062386) - rollout_corr/log_ppl_diff:np.float64(0.0002497481764294207) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013755984837189317) - rollout_corr/log_ppl_diff_max:np.float64(0.004447370767593384) - rollout_corr/log_ppl_diff_min:np.float64(-0.006148874759674072) - rollout_corr/ppl_ratio:np.float64(1.000251316698268) - rollout_corr/chi2_token:np.float64(0.0009627032559365034) - rollout_corr/chi2_seq:np.float64(0.50378093495965) - actor/pg_loss:np.float64(-1.185922883450985e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.797973315362469e-05) - actor/ppo_kl:np.float64(-6.918077795292632e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.048877181485295296) - perf/mfu/actor:np.float64(0.09183496883024235) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.3936767578125) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006229627761058509 - critic/advantages/max:0.75 - critic/advantages/min:-0.375 - critic/returns/mean:-0.0006229627761058509 - critic/returns/max:0.75 - critic/returns/min:-0.375 - response_length/mean:310.38671875 - response_length/max:659.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:310.38671875 - response_length_non_aborted/max:659.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.3125 - prompt_length/max:492.0 - prompt_length/min:267.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000145619735121727 - timing_s/agent_loop/generate_sequences/min:np.float64(2.076954545453191) - timing_s/agent_loop/generate_sequences/max:np.float64(6.009804593399167) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.534746044242638) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.009804593399167) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:439 - timing_s/agent_loop/slowest/response_length:645 - timing_s/gen:12.931843714788556 - timing_s/reward:0.05970763601362705 - timing_s/old_log_prob:2.95658066496253 - timing_s/adv:0.12099779397249222 - timing_s/update_actor:12.547399116680026 - timing_s/step:28.704043405130506 - timing_s/stop_profile:0.00011953897774219513 - timing_per_token_ms/gen:0.1627486340727741 - timing_per_token_ms/adv:0.0006735758286107508 - timing_per_token_ms/update_actor:0.0698494119557994 - perf/total_num_tokens:179635 - perf/time_per_step:28.704043405130506 - perf/throughput:782.272193609718 (TaskRunner pid=2403642) Training Progress: 83%|████████▎ | 83/100 [52:29<09:59, 35.29s/it] (TaskRunner pid=2403642) step:84 - global_seqlen/min:18888 - global_seqlen/max:27836 - global_seqlen/minmax_diff:8948 - global_seqlen/balanced_min:23252 - global_seqlen/balanced_max:23255 - global_seqlen/mean:23253.375 - actor/entropy:0.26086071133613586 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22949278354644775 - training/rollout_probs_diff_mean:0.0024000119883567095 - training/rollout_probs_diff_std:0.006870268378406763 - training/rollout_actor_probs_pearson_corr:0.9995623230934143 - rollout_corr/rollout_is_mean:1.0000234842300415 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.447716474533081 - rollout_corr/rollout_is_min:0.6345890760421753 - rollout_corr/rollout_is_std:0.023518739268183708 - rollout_corr/rollout_is_eff_sample_size:0.9994470556038549 - rollout_corr/rollout_is_seq_mean:1.0000110864639282 - rollout_corr/rollout_is_seq_std:0.0014413440367206931 - rollout_corr/rollout_is_seq_max:1.0039033889770508 - rollout_corr/rollout_is_seq_min:0.9952189922332764 - rollout_corr/rollout_is_seq_max_deviation:0.004781007766723633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3491347096860409) - rollout_corr/training_log_ppl:np.float64(0.28971572226146236) - rollout_corr/kl:np.float64(0.00028144666232599036) - rollout_corr/k3_kl:np.float64(0.0003370657866810234) - rollout_corr/rollout_ppl:np.float64(1.348737888969481) - rollout_corr/rollout_log_ppl:np.float64(0.2894342727086041) - rollout_corr/log_ppl_diff:np.float64(0.00028144955285824835) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012165008520241827) - rollout_corr/log_ppl_diff_max:np.float64(0.00624963641166687) - rollout_corr/log_ppl_diff_min:np.float64(-0.004302710294723511) - rollout_corr/ppl_ratio:np.float64(1.000282758846879) - rollout_corr/chi2_token:np.float64(0.0007954011671245098) - rollout_corr/chi2_seq:np.float64(0.2558585754595697) - actor/pg_loss:np.float64(2.7131754904985428e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.780518636631314e-05) - actor/ppo_kl:np.float64(-1.1325415965046659e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.03505889978259802) - perf/mfu/actor:np.float64(0.09341460669569832) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.45610427856445) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.83203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018751678289845586 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.0018751678289845586 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:334.60546875 - response_length/max:836.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.60546875 - response_length_non_aborted/max:836.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:392.0625 - prompt_length/max:519.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011912919580936432 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8652551472187042) - timing_s/agent_loop/generate_sequences/max:np.float64(7.026093482971191) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.726336318446556) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.026093482971191) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:519 - timing_s/agent_loop/slowest/response_length:836 - timing_s/gen:14.720471816137433 - timing_s/reward:0.06028051674365997 - timing_s/old_log_prob:2.986829526722431 - timing_s/adv:0.11731350980699062 - timing_s/update_actor:12.968602947890759 - timing_s/step:30.940340334549546 - timing_s/stop_profile:0.00016866251826286316 - timing_per_token_ms/gen:0.17184968089911665 - timing_per_token_ms/adv:0.0006306262521407679 - timing_per_token_ms/update_actor:0.06971355205368446 - perf/total_num_tokens:186027 - perf/time_per_step:30.940340334549546 - perf/throughput:751.5552430441144 (TaskRunner pid=2403642) Training Progress: 84%|████████▍ | 84/100 [53:00<09:03, 34.00s/it] (TaskRunner pid=2403642) step:85 - global_seqlen/min:19239 - global_seqlen/max:26701 - global_seqlen/minmax_diff:7462 - global_seqlen/balanced_min:22624 - global_seqlen/balanced_max:22640 - global_seqlen/mean:22635.75 - actor/entropy:0.3317025303840637 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6983705163002014 - training/rollout_probs_diff_mean:0.00267635565251112 - training/rollout_probs_diff_std:0.007437863387167454 - training/rollout_actor_probs_pearson_corr:0.999566912651062 - rollout_corr/rollout_is_mean:0.9997298717498779 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4775165002210997e-05 - rollout_corr/rollout_is_max:1.4265085458755493 - rollout_corr/rollout_is_min:0.20711393654346466 - rollout_corr/rollout_is_std:0.024886781349778175 - rollout_corr/rollout_is_eff_sample_size:0.9993807338411205 - rollout_corr/rollout_is_seq_mean:0.9997522830963135 - rollout_corr/rollout_is_seq_std:0.001566800638101995 - rollout_corr/rollout_is_seq_max:1.006334662437439 - rollout_corr/rollout_is_seq_min:0.995550274848938 - rollout_corr/rollout_is_seq_max_deviation:0.006334662437438965 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4340935898944736) - rollout_corr/training_log_ppl:np.float64(0.34704944033001084) - rollout_corr/kl:np.float64(0.0006850926394577073) - rollout_corr/k3_kl:np.float64(0.00040617002476039943) - rollout_corr/rollout_ppl:np.float64(1.433068967424333) - rollout_corr/rollout_log_ppl:np.float64(0.346364343771711) - rollout_corr/log_ppl_diff:np.float64(0.000685096558299847) - rollout_corr/log_ppl_abs_diff:np.float64(0.001369192890706472) - rollout_corr/log_ppl_diff_max:np.float64(0.005961418151855469) - rollout_corr/log_ppl_diff_min:np.float64(-0.005674391984939575) - rollout_corr/ppl_ratio:np.float64(1.000686737941578) - rollout_corr/chi2_token:np.float64(0.00025137816555798054) - rollout_corr/chi2_seq:np.float64(0.04498665127903223) - actor/pg_loss:np.float64(8.018629159778357e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.963335999898845e-05) - actor/ppo_kl:np.float64(0.00010742824364484704) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09896238800138235) - perf/mfu/actor:np.float64(0.09221646713692358) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.29238510131836) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0025595843326300383 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0025595843326300383 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:315.3359375 - response_length/max:920.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.3359375 - response_length_non_aborted/max:920.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:392.03125 - prompt_length/max:541.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010436028242111206 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8224622216075659) - timing_s/agent_loop/generate_sequences/max:np.float64(7.511046390980482) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6148865458744694) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.511046390980482) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:541 - timing_s/agent_loop/slowest/response_length:920 - timing_s/gen:14.36024017445743 - timing_s/reward:0.06420428119599819 - timing_s/old_log_prob:2.9934521317481995 - timing_s/adv:0.1144099049270153 - timing_s/update_actor:12.755359781906009 - timing_s/step:30.37469002045691 - timing_s/stop_profile:0.0001283697783946991 - timing_per_token_ms/gen:0.17788866256791405 - timing_per_token_ms/adv:0.0006317987305866566 - timing_per_token_ms/update_actor:0.07043813316272936 - perf/total_num_tokens:181086 - perf/time_per_step:30.37469002045691 - perf/throughput:745.2174815530678 (TaskRunner pid=2403642) Training Progress: 85%|████████▌ | 85/100 [53:31<08:13, 32.92s/it] (TaskRunner pid=2403642) step:86 - global_seqlen/min:19108 - global_seqlen/max:24507 - global_seqlen/minmax_diff:5399 - global_seqlen/balanced_min:21176 - global_seqlen/balanced_max:21184 - global_seqlen/mean:21181.75 - actor/entropy:0.3438378870487213 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.822213351726532 - training/rollout_probs_diff_mean:0.0030232411809265614 - training/rollout_probs_diff_std:0.008595702238380909 - training/rollout_actor_probs_pearson_corr:0.9994766116142273 - rollout_corr/rollout_is_mean:1.0000633001327515 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.151559551246464e-05 - rollout_corr/rollout_is_max:1.739801049232483 - rollout_corr/rollout_is_min:2.8528873372124508e-05 - rollout_corr/rollout_is_std:0.026786576956510544 - rollout_corr/rollout_is_eff_sample_size:0.999282993753028 - rollout_corr/rollout_is_seq_mean:1.000069499015808 - rollout_corr/rollout_is_seq_std:0.0017593996599316597 - rollout_corr/rollout_is_seq_max:1.005941390991211 - rollout_corr/rollout_is_seq_min:0.9954469799995422 - rollout_corr/rollout_is_seq_max_deviation:0.0059413909912109375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.419941368047148) - rollout_corr/training_log_ppl:np.float64(0.34356227729585953) - rollout_corr/kl:np.float64(0.0003319210262313632) - rollout_corr/k3_kl:np.float64(0.0004817469163640453) - rollout_corr/rollout_ppl:np.float64(1.4194591976702213) - rollout_corr/rollout_log_ppl:np.float64(0.3432303518638946) - rollout_corr/log_ppl_diff:np.float64(0.00033192543196491897) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014975782542023808) - rollout_corr/log_ppl_diff_max:np.float64(0.02202536165714264) - rollout_corr/log_ppl_diff_min:np.float64(-0.005789533257484436) - rollout_corr/ppl_ratio:np.float64(1.0003345324657857) - rollout_corr/chi2_token:np.float64(0.0011146750766783953) - rollout_corr/chi2_seq:np.float64(0.34562928532250226) - actor/pg_loss:np.float64(2.0878855139017105e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.801963642530609e-06) - actor/ppo_kl:np.float64(-2.804078054907677e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.03001499082893133) - perf/mfu/actor:np.float64(0.08669465010272584) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.32245254516602) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.83203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0003598018374759704 - critic/advantages/max:0.375 - critic/advantages/min:-0.625 - critic/returns/mean:-0.0003598018374759704 - critic/returns/max:0.375 - critic/returns/min:-0.625 - response_length/mean:282.2734375 - response_length/max:971.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:282.2734375 - response_length_non_aborted/max:971.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.65625 - prompt_length/max:483.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011705607175827026 - timing_s/agent_loop/generate_sequences/min:np.float64(1.650878168642521) - timing_s/agent_loop/generate_sequences/max:np.float64(7.470661237835884) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.279068757787172) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.470661237835884) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:971 - timing_s/gen:14.963778465986252 - timing_s/reward:0.05691719613969326 - timing_s/old_log_prob:2.998625362291932 - timing_s/adv:0.13489815592765808 - timing_s/update_actor:12.720323342829943 - timing_s/step:30.960941610857844 - timing_s/stop_profile:0.00012388452887535095 - timing_per_token_ms/gen:0.2070767272700209 - timing_per_token_ms/adv:0.0007960753710603355 - timing_per_token_ms/update_actor:0.0750665274518745 - perf/total_num_tokens:169454 - perf/time_per_step:30.960941610857844 - perf/throughput:684.144244261992 (TaskRunner pid=2403642) Training Progress: 86%|████████▌ | 86/100 [54:02<07:32, 32.35s/it] (TaskRunner pid=2403642) step:87 - global_seqlen/min:19166 - global_seqlen/max:25802 - global_seqlen/minmax_diff:6636 - global_seqlen/balanced_min:22712 - global_seqlen/balanced_max:22718 - global_seqlen/mean:22715.375 - actor/entropy:0.2796725332736969 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17467910051345825 - training/rollout_probs_diff_mean:0.002430505584925413 - training/rollout_probs_diff_std:0.006751000415533781 - training/rollout_actor_probs_pearson_corr:0.9996549487113953 - rollout_corr/rollout_is_mean:0.9999507665634155 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.643362045288086 - rollout_corr/rollout_is_min:0.6558393836021423 - rollout_corr/rollout_is_std:0.023985495790839195 - rollout_corr/rollout_is_eff_sample_size:0.9994250267530548 - rollout_corr/rollout_is_seq_mean:0.9999579787254333 - rollout_corr/rollout_is_seq_std:0.0015496320556849241 - rollout_corr/rollout_is_seq_max:1.0060547590255737 - rollout_corr/rollout_is_seq_min:0.9959386587142944 - rollout_corr/rollout_is_seq_max_deviation:0.0060547590255737305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3713900046423078) - rollout_corr/training_log_ppl:np.float64(0.307053470693063) - rollout_corr/kl:np.float64(0.0003921358480982462) - rollout_corr/k3_kl:np.float64(0.00034856318789877605) - rollout_corr/rollout_ppl:np.float64(1.3708278317935765) - rollout_corr/rollout_log_ppl:np.float64(0.30666133074555546) - rollout_corr/log_ppl_diff:np.float64(0.00039213994750753045) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012159571633674204) - rollout_corr/log_ppl_diff_max:np.float64(0.0056915283203125) - rollout_corr/log_ppl_diff_min:np.float64(-0.003270745277404785) - rollout_corr/ppl_ratio:np.float64(1.0003934551496059) - rollout_corr/chi2_token:np.float64(0.0006380670238286257) - rollout_corr/chi2_seq:np.float64(0.17924676439724863) - actor/pg_loss:np.float64(4.446483217179775e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(3.656892177161808e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.04396366095170379) - perf/mfu/actor:np.float64(0.09381662705297268) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.2486572265625) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003371447790414095 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003371447790414095 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:297.04296875 - response_length/max:791.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:297.04296875 - response_length_non_aborted/max:791.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.8125 - prompt_length/max:518.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015601515769958496 - timing_s/agent_loop/generate_sequences/min:np.float64(1.120266143232584) - timing_s/agent_loop/generate_sequences/max:np.float64(6.718985510990024) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.512847683778091) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.718985510990024) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:791 - timing_s/gen:13.762186324223876 - timing_s/reward:0.05853886716067791 - timing_s/old_log_prob:2.9617889877408743 - timing_s/adv:0.12188956327736378 - timing_s/update_actor:12.450166650116444 - timing_s/step:29.440568823367357 - timing_s/stop_profile:0.00014316104352474213 - timing_per_token_ms/gen:0.18097900298809722 - timing_per_token_ms/adv:0.0006707437323693962 - timing_per_token_ms/update_actor:0.06851178249377593 - perf/total_num_tokens:181723 - perf/time_per_step:29.440568823367357 - perf/throughput:771.5671234575643 (TaskRunner pid=2403642) Training Progress: 87%|████████▋ | 87/100 [54:31<06:49, 31.50s/it] (TaskRunner pid=2403642) step:88 - global_seqlen/min:19663 - global_seqlen/max:25654 - global_seqlen/minmax_diff:5991 - global_seqlen/balanced_min:22652 - global_seqlen/balanced_max:22666 - global_seqlen/mean:22661.25 - actor/entropy:0.2953377664089203 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797217309474945 - training/rollout_probs_diff_mean:0.0025792568922042847 - training/rollout_probs_diff_std:0.00730118015781045 - training/rollout_actor_probs_pearson_corr:0.999626874923706 - rollout_corr/rollout_is_mean:0.9999725818634033 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8165900707244873 - rollout_corr/rollout_is_min:0.5354034304618835 - rollout_corr/rollout_is_std:0.025083592161536217 - rollout_corr/rollout_is_eff_sample_size:0.9993710899372364 - rollout_corr/rollout_is_seq_mean:1.0000321865081787 - rollout_corr/rollout_is_seq_std:0.0015428932383656502 - rollout_corr/rollout_is_seq_max:1.00552499294281 - rollout_corr/rollout_is_seq_min:0.9950153231620789 - rollout_corr/rollout_is_seq_max_deviation:0.005524992942810059 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3939012219198048) - rollout_corr/training_log_ppl:np.float64(0.31474355689715594) - rollout_corr/kl:np.float64(0.0004259274350104647) - rollout_corr/k3_kl:np.float64(0.00037728191566088753) - rollout_corr/rollout_ppl:np.float64(1.3932252889499068) - rollout_corr/rollout_log_ppl:np.float64(0.314317626907723) - rollout_corr/log_ppl_diff:np.float64(0.00042592998943291605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014040337118785828) - rollout_corr/log_ppl_diff_max:np.float64(0.005751430988311768) - rollout_corr/log_ppl_diff_min:np.float64(-0.005627959966659546) - rollout_corr/ppl_ratio:np.float64(1.000427548540756) - rollout_corr/chi2_token:np.float64(0.0006589947734028101) - rollout_corr/chi2_seq:np.float64(0.39225143822841346) - actor/pg_loss:np.float64(-3.2695126719772816e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.326313172801747e-05) - actor/ppo_kl:np.float64(0.00012912661738972986) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07530395220965147) - perf/mfu/actor:np.float64(0.09325487071830077) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.34903717041016) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029977471567690372 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0029977471567690372 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:322.5078125 - response_length/max:1015.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.5078125 - response_length_non_aborted/max:1015.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.65625 - prompt_length/max:490.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010713748633861542 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9955077525228262) - timing_s/agent_loop/generate_sequences/max:np.float64(7.942003663629293) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6675338528002612) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.942003663629293) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:1015 - timing_s/gen:15.68840309791267 - timing_s/reward:0.06001050025224686 - timing_s/old_log_prob:2.9801001977175474 - timing_s/adv:0.12975222058594227 - timing_s/update_actor:12.51058503985405 - timing_s/step:31.4559672344476 - timing_s/stop_profile:0.00011013075709342957 - timing_per_token_ms/gen:0.19001965913995142 - timing_per_token_ms/adv:0.0007157163692754275 - timing_per_token_ms/update_actor:0.06900868795771443 - perf/total_num_tokens:181290 - perf/time_per_step:31.4559672344476 - perf/throughput:720.4118007594928 (TaskRunner pid=2403642) Training Progress: 88%|████████▊ | 88/100 [55:03<06:17, 31.50s/it] (TaskRunner pid=2403642) step:89 - global_seqlen/min:21430 - global_seqlen/max:28221 - global_seqlen/minmax_diff:6791 - global_seqlen/balanced_min:24733 - global_seqlen/balanced_max:24735 - global_seqlen/mean:24734.125 - actor/entropy:0.3050276041030884 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7560598850250244 - training/rollout_probs_diff_mean:0.002423952566459775 - training/rollout_probs_diff_std:0.007534638978540897 - training/rollout_actor_probs_pearson_corr:0.9996098875999451 - rollout_corr/rollout_is_mean:0.9999791383743286 - rollout_corr/rollout_is_ratio_fraction_high:1.1024750165233854e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.3074251405196264e-05 - rollout_corr/rollout_is_max:2.385235548019409 - rollout_corr/rollout_is_min:0.14922362565994263 - rollout_corr/rollout_is_std:0.024784782901406288 - rollout_corr/rollout_is_eff_sample_size:0.9993859725829729 - rollout_corr/rollout_is_seq_mean:0.9999644160270691 - rollout_corr/rollout_is_seq_std:0.0017730066319927573 - rollout_corr/rollout_is_seq_max:1.0053385496139526 - rollout_corr/rollout_is_seq_min:0.9937400221824646 - rollout_corr/rollout_is_seq_max_deviation:0.0062599778175354 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4235105477273464) - rollout_corr/training_log_ppl:np.float64(0.3350572675553849) - rollout_corr/kl:np.float64(0.0004977377876294042) - rollout_corr/k3_kl:np.float64(0.00043506097114232034) - rollout_corr/rollout_ppl:np.float64(1.4227662342600524) - rollout_corr/rollout_log_ppl:np.float64(0.33455952878284734) - rollout_corr/log_ppl_diff:np.float64(0.0004977387725375593) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014161956496536732) - rollout_corr/log_ppl_diff_max:np.float64(0.006734251976013184) - rollout_corr/log_ppl_diff_min:np.float64(-0.004191935062408447) - rollout_corr/ppl_ratio:np.float64(1.000499605666846) - rollout_corr/chi2_token:np.float64(0.0007642176933586597) - rollout_corr/chi2_seq:np.float64(0.3206390021368861) - actor/pg_loss:np.float64(5.431764293462038e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.499900716036791e-05) - actor/ppo_kl:np.float64(0.00011755074425323642) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21098365262150764) - perf/mfu/actor:np.float64(0.09995132241417987) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.35758590698242) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:3 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0021939254365861416 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:0.0021939254365861416 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:354.31640625 - response_length/max:789.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:354.31640625 - response_length_non_aborted/max:789.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:418.625 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.185262024402618e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6536885611712933) - timing_s/agent_loop/generate_sequences/max:np.float64(6.9988903403282166) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.955004553521576) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.9988903403282166) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:789 - timing_s/gen:13.682155484333634 - timing_s/reward:0.06117759272456169 - timing_s/old_log_prob:3.0277791507542133 - timing_s/adv:0.13126206398010254 - timing_s/update_actor:12.846789751201868 - timing_s/step:29.835316451266408 - timing_s/stop_profile:2.7783215045928955e-05 - timing_per_token_ms/gen:0.15084235140657776 - timing_per_token_ms/adv:0.0006633652088971337 - timing_per_token_ms/update_actor:0.0649244199623085 - perf/total_num_tokens:197873 - perf/time_per_step:29.835316451266408 - perf/throughput:829.0217078944413 (TaskRunner pid=2403642) Training Progress: 89%|████████▉ | 89/100 [55:32<05:41, 31.00s/it] (TaskRunner pid=2403642) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2403642) validation generation end (TaskRunner pid=2403642) [prompt] system (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2403642) (TaskRunner pid=2403642) A (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) user (TaskRunner pid=2403642) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2403642) (TaskRunner pid=2403642) Lattice Parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) a = 7.82 Å (TaskRunner pid=2403642) b = 7.82 Å (TaskRunner pid=2403642) c = 17.88 Å (TaskRunner pid=2403642) α = 90.00º (TaskRunner pid=2403642) β = 90.00º (TaskRunner pid=2403642) γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) A: 1025.67 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) B: 1094.31 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) C: 1200.45 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) D: 1150.78 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) Please reason step by step. (TaskRunner pid=2403642) assistant (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) [response] (TaskRunner pid=2403642) The given crystal structure is a unit cell with lattice parameters a = 7.82 Å, b = 7.82 Å, c = 17.88 Å, and all angles α, β, γ are 90°, which indicates that the unit cell is orthorhombic. (TaskRunner pid=2403642) (TaskRunner pid=2403642) For an orthorhombic unit cell, the volume (V) of the unit cell is calculated using the formula: (TaskRunner pid=2403642) (TaskRunner pid=2403642) V = a × b × c (TaskRunner pid=2403642) (TaskRunner pid=2403642) Substituting the given values: (TaskRunner pid=2403642) (TaskRunner pid=2403642) V = 7.82 Å × 7.82 Å × 17.88 Å (TaskRunner pid=2403642) (TaskRunner pid=2403642) First, calculate 7.82 × 7.82: (TaskRunner pid=2403642) (TaskRunner pid=2403642) 7.82 × 7.82 = 61.1524 (TaskRunner pid=2403642) (TaskRunner pid=2403642) Then multiply that by 17.88: (TaskRunner pid=2403642) (TaskRunner pid=2403642) 61.1524 × 17.88 ≈ 1094.31 (TaskRunner pid=2403642) (TaskRunner pid=2403642) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) B (TaskRunner pid=2403642) (TaskRunner pid=2403642) [ground_truth] B (TaskRunner pid=2403642) [score] 1.0 (TaskRunner pid=2403642) [acc] 1.0 (TaskRunner pid=2403642) [pred] B (TaskRunner pid=2403642) [incorrect_format] 1 (TaskRunner pid=2403642) [feedback] (TaskRunner pid=2403642) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2403642) step:90 - global_seqlen/min:20047 - global_seqlen/max:28634 - global_seqlen/minmax_diff:8587 - global_seqlen/balanced_min:23235 - global_seqlen/balanced_max:23239 - global_seqlen/mean:23238.25 - actor/entropy:0.34761369228363037 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2881855070590973 - training/rollout_probs_diff_mean:0.002914301585406065 - training/rollout_probs_diff_std:0.00754848076030612 - training/rollout_actor_probs_pearson_corr:0.9996027946472168 - rollout_corr/rollout_is_mean:0.9999606609344482 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.3682652681600302e-05 - rollout_corr/rollout_is_max:1.6418794393539429 - rollout_corr/rollout_is_min:0.3688293397426605 - rollout_corr/rollout_is_std:0.026443934068083763 - rollout_corr/rollout_is_eff_sample_size:0.9993012069606744 - rollout_corr/rollout_is_seq_mean:0.9999961256980896 - rollout_corr/rollout_is_seq_std:0.0017832440789788961 - rollout_corr/rollout_is_seq_max:1.0080827474594116 - rollout_corr/rollout_is_seq_min:0.9929141402244568 - rollout_corr/rollout_is_seq_max_deviation:0.008082747459411621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4519235398620367) - rollout_corr/training_log_ppl:np.float64(0.36163410761218984) - rollout_corr/kl:np.float64(0.0004622778555010143) - rollout_corr/k3_kl:np.float64(0.0004612364294018789) - rollout_corr/rollout_ppl:np.float64(1.451245247386396) - rollout_corr/rollout_log_ppl:np.float64(0.36117182987800334) - rollout_corr/log_ppl_diff:np.float64(0.0004622777341865003) - rollout_corr/log_ppl_abs_diff:np.float64(0.001413370599038899) - rollout_corr/log_ppl_diff_max:np.float64(0.008033037185668945) - rollout_corr/log_ppl_diff_min:np.float64(-0.007330834865570068) - rollout_corr/ppl_ratio:np.float64(1.0004640952683985) - rollout_corr/chi2_token:np.float64(0.0009107908699661493) - rollout_corr/chi2_seq:np.float64(0.3217647692654282) - actor/pg_loss:np.float64(1.382431946694851e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(1.2271325431356672e-05) - actor/ppo_kl:np.float64(8.642425431104606e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.01626203558407724) - perf/mfu/actor:np.float64(0.09550263987107718) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.47405242919922) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7593085106382979) - val-aux/sciknoweval/reward/std@16:np.float64(0.05324482498876583) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7813297872340427) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.033221983145840114) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7378085106382979) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.056841760464986006) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7595957446808509) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.0533915093622308) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7919361702127661) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.015034088527283876) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7136276595744682) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.053166060035626864) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.766095744680851) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.0424948683594337) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7966170212765958) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0047140238339681995) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6905531914893618) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.04196233640540949) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7699468085106383) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.028759654583830934) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7978191489361702) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0007503582967907387) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6730106382978723) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.025702318653018304) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7716063829787233) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.019305424740207522) - val-aux/sciknoweval/score/mean@16:np.float64(0.7593085106382979) - val-aux/sciknoweval/score/std@16:np.float64(0.05324482498876583) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7813297872340427) - val-aux/sciknoweval/score/best@2/std:np.float64(0.033221983145840114) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7378085106382979) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.056841760464986006) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7595957446808509) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.0533915093622308) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7919361702127661) - val-aux/sciknoweval/score/best@4/std:np.float64(0.015034088527283876) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7136276595744682) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.053166060035626864) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.766095744680851) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.0424948683594337) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7966170212765958) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0047140238339681995) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6905531914893618) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.04196233640540949) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7699468085106383) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.028759654583830934) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7978191489361702) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0007503582967907387) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6730106382978723) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.025702318653018304) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7716063829787233) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.019305424740207522) - val-core/sciknoweval/acc/mean@16:np.float64(0.7593085106382979) - val-aux/sciknoweval/acc/std@16:np.float64(0.05324482498876583) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7813297872340427) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.033221983145840114) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7378085106382979) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.056841760464986006) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7595957446808509) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.0533915093622308) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7919361702127661) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.015034088527283876) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7136276595744682) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.053166060035626864) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.766095744680851) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.0424948683594337) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7966170212765958) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0047140238339681995) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6905531914893618) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.04196233640540949) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7699468085106383) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.028759654583830934) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7978191489361702) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0007503582967907387) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6730106382978723) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.025702318653018304) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7716063829787233) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.019305424740207522) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:3 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003238602774217725 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.003238602774217725 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:329.8828125 - response_length/max:791.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:329.8828125 - response_length_non_aborted/max:791.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:509.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6615336537361145e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3552416246384382) - timing_s/agent_loop/generate_sequences/max:np.float64(6.563702668994665) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7758583198083215) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.563702668994665) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:450 - timing_s/agent_loop/slowest/response_length:791 - timing_s/gen:13.95789504237473 - timing_s/reward:0.057368138805031776 - timing_s/old_log_prob:3.16178429313004 - timing_s/adv:0.15473546087741852 - timing_s/update_actor:12.370095839723945 - timing_s/step:29.78493888862431 - timing_s/testing:42.92627669684589 - timing_s/save_checkpoint:13.298548012971878 - timing_s/stop_profile:0.00014620088040828705 - timing_per_token_ms/gen:0.16527998866044677 - timing_per_token_ms/adv:0.0008323317207482196 - timing_per_token_ms/update_actor:0.06653951911032427 - perf/total_num_tokens:185906 - perf/time_per_step:29.78493888862431 - perf/throughput:780.2013657605767 (TaskRunner pid=2403642) Training Progress: 90%|█████████ | 90/100 [56:59<07:55, 47.52s/it] (TaskRunner pid=2403642) step:91 - global_seqlen/min:21635 - global_seqlen/max:25466 - global_seqlen/minmax_diff:3831 - global_seqlen/balanced_min:23610 - global_seqlen/balanced_max:23624 - global_seqlen/mean:23619.75 - actor/entropy:0.35342922806739807 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21307647228240967 - training/rollout_probs_diff_mean:0.0028058807365596294 - training/rollout_probs_diff_std:0.007240186911076307 - training/rollout_actor_probs_pearson_corr:0.9996463656425476 - rollout_corr/rollout_is_mean:0.999942421913147 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6107667684555054 - rollout_corr/rollout_is_min:0.6440470814704895 - rollout_corr/rollout_is_std:0.025918031111359596 - rollout_corr/rollout_is_eff_sample_size:0.9993284684924016 - rollout_corr/rollout_is_seq_mean:0.999967098236084 - rollout_corr/rollout_is_seq_std:0.0016982742818072438 - rollout_corr/rollout_is_seq_max:1.0109471082687378 - rollout_corr/rollout_is_seq_min:0.9943731427192688 - rollout_corr/rollout_is_seq_max_deviation:0.010947108268737793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5175115521997213) - rollout_corr/training_log_ppl:np.float64(0.39346077786467504) - rollout_corr/kl:np.float64(0.0004233603360290772) - rollout_corr/k3_kl:np.float64(0.00041688081708457503) - rollout_corr/rollout_ppl:np.float64(1.5167877241037786) - rollout_corr/rollout_log_ppl:np.float64(0.3930374163610395) - rollout_corr/log_ppl_diff:np.float64(0.00042336150363553315) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012970607058377936) - rollout_corr/log_ppl_diff_max:np.float64(0.007829904556274414) - rollout_corr/log_ppl_diff_min:np.float64(-0.010272711515426636) - rollout_corr/ppl_ratio:np.float64(1.0004250255879015) - rollout_corr/chi2_token:np.float64(0.0008352859877049923) - rollout_corr/chi2_seq:np.float64(0.39926669048145413) - actor/pg_loss:np.float64(-0.00021755159832537174) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.048991831950843e-05) - actor/ppo_kl:np.float64(3.1574491124786164e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13898340100422502) - perf/mfu/actor:np.float64(0.09595114108973381) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.60107803344727) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:3 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018889433704316616 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018889433704316616 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:382.0546875 - response_length/max:873.0 - response_length/min:67.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.0546875 - response_length_non_aborted/max:873.0 - response_length_non_aborted/min:67.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:356.0625 - prompt_length/max:477.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.923265755176544e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8506027832627296) - timing_s/agent_loop/generate_sequences/max:np.float64(7.396001921966672) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8466681276404415) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.396001921966672) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:821 - timing_s/gen:13.723110236227512 - timing_s/reward:0.0578249916434288 - timing_s/old_log_prob:3.260933877900243 - timing_s/adv:0.11665872484445572 - timing_s/update_actor:12.915504146367311 - timing_s/step:30.13380137272179 - timing_s/stop_profile:0.00018453411757946014 - timing_per_token_ms/gen:0.14030949263059028 - timing_per_token_ms/adv:0.0006173791257552245 - timing_per_token_ms/update_actor:0.0683511899277475 - perf/total_num_tokens:188958 - perf/time_per_step:30.13380137272179 - perf/throughput:783.8290864086418 (TaskRunner pid=2403642) Training Progress: 91%|█████████ | 91/100 [57:29<06:20, 42.32s/it] (TaskRunner pid=2403642) step:92 - global_seqlen/min:22074 - global_seqlen/max:29852 - global_seqlen/minmax_diff:7778 - global_seqlen/balanced_min:24847 - global_seqlen/balanced_max:24849 - global_seqlen/mean:24848.0 - actor/entropy:0.3595642149448395 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22151517868041992 - training/rollout_probs_diff_mean:0.0029321240726858377 - training/rollout_probs_diff_std:0.00750844506546855 - training/rollout_actor_probs_pearson_corr:0.9995761513710022 - rollout_corr/rollout_is_mean:1.0001015663146973 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5213674306869507 - rollout_corr/rollout_is_min:0.6559000015258789 - rollout_corr/rollout_is_std:0.026828821748495102 - rollout_corr/rollout_is_eff_sample_size:0.9992808510663643 - rollout_corr/rollout_is_seq_mean:1.0001308917999268 - rollout_corr/rollout_is_seq_std:0.0014905876014381647 - rollout_corr/rollout_is_seq_max:1.0047581195831299 - rollout_corr/rollout_is_seq_min:0.9957191944122314 - rollout_corr/rollout_is_seq_max_deviation:0.004758119583129883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4703190927393734) - rollout_corr/training_log_ppl:np.float64(0.3702059087372618) - rollout_corr/kl:np.float64(0.00043767166106079003) - rollout_corr/k3_kl:np.float64(0.0004534227082331199) - rollout_corr/rollout_ppl:np.float64(1.4696018886752427) - rollout_corr/rollout_log_ppl:np.float64(0.369768237680546) - rollout_corr/log_ppl_diff:np.float64(0.00043767105671577156) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013431093539111316) - rollout_corr/log_ppl_diff_max:np.float64(0.00604403018951416) - rollout_corr/log_ppl_diff_min:np.float64(-0.004392862319946289) - rollout_corr/ppl_ratio:np.float64(1.0004392443224788) - rollout_corr/chi2_token:np.float64(0.0009216954931616783) - rollout_corr/chi2_seq:np.float64(0.4805670231580734) - actor/pg_loss:np.float64(4.529592115432024e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.733584970992524e-05) - actor/ppo_kl:np.float64(0.000197046403414447) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11538068484514952) - perf/mfu/actor:np.float64(0.100455938692909) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.47601699829102) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:3 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006862935610115528 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006862935610115528 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.0625 - response_length/max:908.0 - response_length/min:177.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.0625 - response_length_non_aborted/max:908.0 - response_length_non_aborted/min:177.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.4375 - prompt_length/max:476.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011311471462249756 - timing_s/agent_loop/generate_sequences/min:np.float64(2.233831040561199) - timing_s/agent_loop/generate_sequences/max:np.float64(7.89228375069797) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.414751171891112) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.89228375069797) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:908 - timing_s/gen:15.144404793158174 - timing_s/reward:0.059919705614447594 - timing_s/old_log_prob:3.056266037747264 - timing_s/adv:0.11873866431415081 - timing_s/update_actor:12.893685279414058 - timing_s/step:31.356594663113356 - timing_s/stop_profile:0.00011335499584674835 - timing_per_token_ms/gen:0.15483809906304366 - timing_per_token_ms/adv:0.0005973250579229255 - timing_per_token_ms/update_actor:0.06486279217348508 - perf/total_num_tokens:198784 - perf/time_per_step:31.356594663113356 - perf/throughput:792.432987923596 (TaskRunner pid=2403642) Training Progress: 92%|█████████▏| 92/100 [58:00<05:12, 39.05s/it] (TaskRunner pid=2403642) step:93 - global_seqlen/min:20407 - global_seqlen/max:28006 - global_seqlen/minmax_diff:7599 - global_seqlen/balanced_min:24042 - global_seqlen/balanced_max:24044 - global_seqlen/mean:24043.25 - actor/entropy:0.3005387485027313 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4241411089897156 - training/rollout_probs_diff_mean:0.002642803592607379 - training/rollout_probs_diff_std:0.007475018501281738 - training/rollout_actor_probs_pearson_corr:0.9995802044868469 - rollout_corr/rollout_is_mean:1.000084638595581 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0518122508074157e-05 - rollout_corr/rollout_is_max:1.5421065092086792 - rollout_corr/rollout_is_min:0.4320620894432068 - rollout_corr/rollout_is_std:0.025235213339328766 - rollout_corr/rollout_is_eff_sample_size:0.9993635892517825 - rollout_corr/rollout_is_seq_mean:1.0000985860824585 - rollout_corr/rollout_is_seq_std:0.0016015670262277126 - rollout_corr/rollout_is_seq_max:1.0049409866333008 - rollout_corr/rollout_is_seq_min:0.9953435659408569 - rollout_corr/rollout_is_seq_max_deviation:0.004940986633300781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4219031422398984) - rollout_corr/training_log_ppl:np.float64(0.3349203703983221) - rollout_corr/kl:np.float64(0.00030562687818047607) - rollout_corr/k3_kl:np.float64(0.00046132832230227905) - rollout_corr/rollout_ppl:np.float64(1.421450737863779) - rollout_corr/rollout_log_ppl:np.float64(0.334614738705568) - rollout_corr/log_ppl_diff:np.float64(0.00030563169275410473) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013734388921875507) - rollout_corr/log_ppl_diff_max:np.float64(0.007729679346084595) - rollout_corr/log_ppl_diff_min:np.float64(-0.004610300064086914) - rollout_corr/ppl_ratio:np.float64(1.0003073420375586) - rollout_corr/chi2_token:np.float64(0.001192569499835372) - rollout_corr/chi2_seq:np.float64(0.4643619789276272) - actor/pg_loss:np.float64(7.481081411242485e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.265500319888815e-05) - actor/ppo_kl:np.float64(4.5113914802996646e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12982227839529514) - perf/mfu/actor:np.float64(0.0976179469248955) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.53211975097656) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003967961762100458 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:0.003967961762100458 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:371.3828125 - response_length/max:898.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:371.3828125 - response_length_non_aborted/max:898.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.96875 - prompt_length/max:492.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.372776210308075e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.136307967826724) - timing_s/agent_loop/generate_sequences/max:np.float64(7.731545675545931) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.174539480387466) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.731545675545931) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:892 - timing_s/gen:14.656834904104471 - timing_s/reward:0.06079719215631485 - timing_s/old_log_prob:3.0328547917306423 - timing_s/adv:0.13069446943700314 - timing_s/update_actor:12.899016531184316 - timing_s/step:30.867851842194796 - timing_s/stop_profile:0.00012446008622646332 - timing_per_token_ms/gen:0.15416238828811737 - timing_per_token_ms/adv:0.0006794758894752329 - timing_per_token_ms/update_actor:0.0670615273059191 - perf/total_num_tokens:192346 - perf/time_per_step:30.867851842194796 - perf/throughput:778.9090774089466 (TaskRunner pid=2403642) Training Progress: 93%|█████████▎| 93/100 [58:31<04:16, 36.61s/it] (TaskRunner pid=2403642) step:94 - global_seqlen/min:21534 - global_seqlen/max:29660 - global_seqlen/minmax_diff:8126 - global_seqlen/balanced_min:25407 - global_seqlen/balanced_max:25422 - global_seqlen/mean:25417.125 - actor/entropy:0.2767510414123535 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135532975196838 - training/rollout_probs_diff_mean:0.0025500222109258175 - training/rollout_probs_diff_std:0.0073505425825715065 - training/rollout_actor_probs_pearson_corr:0.9995822310447693 - rollout_corr/rollout_is_mean:0.9999530911445618 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.667346603237092e-06 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:0.2820626199245453 - rollout_corr/rollout_is_std:0.024969272315502167 - rollout_corr/rollout_is_eff_sample_size:0.999376804820774 - rollout_corr/rollout_is_seq_mean:0.9999873042106628 - rollout_corr/rollout_is_seq_std:0.0015164752257987857 - rollout_corr/rollout_is_seq_max:1.0061312913894653 - rollout_corr/rollout_is_seq_min:0.9966298937797546 - rollout_corr/rollout_is_seq_max_deviation:0.006131291389465332 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3853449211455882) - rollout_corr/training_log_ppl:np.float64(0.3129881990898866) - rollout_corr/kl:np.float64(0.0004005604543886321) - rollout_corr/k3_kl:np.float64(0.0004372991470091847) - rollout_corr/rollout_ppl:np.float64(1.3847369984723628) - rollout_corr/rollout_log_ppl:np.float64(0.3125876357080415) - rollout_corr/log_ppl_diff:np.float64(0.0004005633818451315) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013818044390063733) - rollout_corr/log_ppl_diff_max:np.float64(0.005264222621917725) - rollout_corr/log_ppl_diff_min:np.float64(-0.005869626998901367) - rollout_corr/ppl_ratio:np.float64(1.000402232632041) - rollout_corr/chi2_token:np.float64(0.0009447787888348103) - rollout_corr/chi2_seq:np.float64(0.45626901066862047) - actor/pg_loss:np.float64(2.013996709138155e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001884714770312712) - actor/ppo_kl:np.float64(3.779682675641993e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13156491424888372) - perf/mfu/actor:np.float64(0.10275564150104508) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.4635009765625) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:3 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000761303526815027 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:0.000761303526815027 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:404.06640625 - response_length/max:928.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:404.06640625 - response_length_non_aborted/max:928.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.21875 - prompt_length/max:483.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010003708302974701 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8698799815028906) - timing_s/agent_loop/generate_sequences/max:np.float64(8.220686331391335) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.46872838906711) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.220686331391335) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:928 - timing_s/gen:15.075250079855323 - timing_s/reward:0.062137238681316376 - timing_s/old_log_prob:3.039712043479085 - timing_s/adv:0.13286671228706837 - timing_s/update_actor:12.929252305999398 - timing_s/step:31.326194984838367 - timing_s/stop_profile:2.9284507036209106e-05 - timing_per_token_ms/gen:0.14573766765455984 - timing_per_token_ms/adv:0.0006534310641303273 - timing_per_token_ms/update_actor:0.06358534012992913 - perf/total_num_tokens:203337 - perf/time_per_step:31.326194984838367 - perf/throughput:811.3696863695603 (TaskRunner pid=2403642) Training Progress: 94%|█████████▍| 94/100 [59:02<03:30, 35.03s/it] (TaskRunner pid=2403642) step:95 - global_seqlen/min:22838 - global_seqlen/max:27064 - global_seqlen/minmax_diff:4226 - global_seqlen/balanced_min:25386 - global_seqlen/balanced_max:25389 - global_seqlen/mean:25387.875 - actor/entropy:0.2577309012413025 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.893662691116333 - training/rollout_probs_diff_mean:0.0024719759821891785 - training/rollout_probs_diff_std:0.007829725742340088 - training/rollout_actor_probs_pearson_corr:0.9994516968727112 - rollout_corr/rollout_is_mean:1.0001028776168823 - rollout_corr/rollout_is_ratio_fraction_high:9.833324838837143e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.933329935534857e-05 - rollout_corr/rollout_is_max:2.013819932937622 - rollout_corr/rollout_is_min:0.012144971638917923 - rollout_corr/rollout_is_std:0.024124257266521454 - rollout_corr/rollout_is_eff_sample_size:0.9994184778225121 - rollout_corr/rollout_is_seq_mean:1.0001139640808105 - rollout_corr/rollout_is_seq_std:0.0013081853976473212 - rollout_corr/rollout_is_seq_max:1.0043644905090332 - rollout_corr/rollout_is_seq_min:0.9966682195663452 - rollout_corr/rollout_is_seq_max_deviation:0.004364490509033203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3321691052988172) - rollout_corr/training_log_ppl:np.float64(0.2772668176476145) - rollout_corr/kl:np.float64(0.0003558635719524972) - rollout_corr/k3_kl:np.float64(0.00039730372719759544) - rollout_corr/rollout_ppl:np.float64(1.3316714563407004) - rollout_corr/rollout_log_ppl:np.float64(0.27691095299087465) - rollout_corr/log_ppl_diff:np.float64(0.0003558646567398682) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012297478533582762) - rollout_corr/log_ppl_diff_max:np.float64(0.007804989814758301) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037243664264678955) - rollout_corr/ppl_ratio:np.float64(1.000357209937647) - rollout_corr/chi2_token:np.float64(0.0008305651135742664) - rollout_corr/chi2_seq:np.float64(0.3522763052023947) - actor/pg_loss:np.float64(-4.14624810218811e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.430264829759835e-05) - actor/ppo_kl:np.float64(0.0001356113971553441) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09277421608567238) - perf/mfu/actor:np.float64(0.10337933781473602) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.35813522338867) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:3 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005334579036571085 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005334579036571085 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:397.24609375 - response_length/max:889.0 - response_length/min:170.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:397.24609375 - response_length_non_aborted/max:889.0 - response_length_non_aborted/min:170.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.125 - prompt_length/max:493.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.716302156448364e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2121583987027407) - timing_s/agent_loop/generate_sequences/max:np.float64(7.992377787828445) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.505745570415456) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.992377787828445) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:889 - timing_s/gen:14.996155738830566 - timing_s/reward:0.051952021196484566 - timing_s/old_log_prob:3.1451192516833544 - timing_s/adv:0.11712789162993431 - timing_s/update_actor:12.810398852452636 - timing_s/step:31.155440155416727 - timing_s/stop_profile:0.00011650100350379944 - timing_per_token_ms/gen:0.1474620752134379 - timing_per_token_ms/adv:0.0005766920805203976 - timing_per_token_ms/update_actor:0.0630734103014364 - perf/total_num_tokens:203103 - perf/time_per_step:31.155440155416727 - perf/throughput:814.8777508311347 (TaskRunner pid=2403642) Training Progress: 95%|█████████▌| 95/100 [59:34<02:49, 33.89s/it] (TaskRunner pid=2403642) step:96 - global_seqlen/min:21482 - global_seqlen/max:29626 - global_seqlen/minmax_diff:8144 - global_seqlen/balanced_min:26147 - global_seqlen/balanced_max:26151 - global_seqlen/mean:26149.375 - actor/entropy:0.27476727962493896 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2821924090385437 - training/rollout_probs_diff_mean:0.002515679458156228 - training/rollout_probs_diff_std:0.007363538723438978 - training/rollout_actor_probs_pearson_corr:0.9994983077049255 - rollout_corr/rollout_is_mean:1.0000098943710327 - rollout_corr/rollout_is_ratio_fraction_high:1.907468686113134e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.167215347290039 - rollout_corr/rollout_is_min:0.569692075252533 - rollout_corr/rollout_is_std:0.024926267564296722 - rollout_corr/rollout_is_eff_sample_size:0.9993789479189513 - rollout_corr/rollout_is_seq_mean:1.0000368356704712 - rollout_corr/rollout_is_seq_std:0.0015217246254906058 - rollout_corr/rollout_is_seq_max:1.0109775066375732 - rollout_corr/rollout_is_seq_min:0.9955570101737976 - rollout_corr/rollout_is_seq_max_deviation:0.010977506637573242 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3714127410203218) - rollout_corr/training_log_ppl:np.float64(0.30417355547251645) - rollout_corr/kl:np.float64(0.0004562080985355266) - rollout_corr/k3_kl:np.float64(0.0004123068538604002) - rollout_corr/rollout_ppl:np.float64(1.370782433077693) - rollout_corr/rollout_log_ppl:np.float64(0.30371734405343886) - rollout_corr/log_ppl_diff:np.float64(0.0004562114190775901) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011979831324424595) - rollout_corr/log_ppl_diff_max:np.float64(0.0051811933517456055) - rollout_corr/log_ppl_diff_min:np.float64(-0.006937205791473389) - rollout_corr/ppl_ratio:np.float64(1.000457545509562) - rollout_corr/chi2_token:np.float64(0.0007856255397200584) - rollout_corr/chi2_seq:np.float64(0.5816815719008446) - actor/pg_loss:np.float64(-5.977042019367218e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.639629201439675e-05) - actor/ppo_kl:np.float64(0.0001432839844417444) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.04847895633429289) - perf/mfu/actor:np.float64(0.10562419744407212) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.48655700683594) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:3 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0001442523207515478 - critic/advantages/max:0.375 - critic/advantages/min:-0.625 - critic/returns/mean:0.0001442523207515478 - critic/returns/max:0.375 - critic/returns/min:-0.625 - response_length/mean:409.57421875 - response_length/max:843.0 - response_length/min:170.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:409.57421875 - response_length_non_aborted/max:843.0 - response_length_non_aborted/min:170.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.59375 - prompt_length/max:505.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.192417979240417e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.067355601117015) - timing_s/agent_loop/generate_sequences/max:np.float64(7.962549405172467) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.60729234876635) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.962549405172467) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:479 - timing_s/agent_loop/slowest/response_length:843 - timing_s/gen:14.895837729796767 - timing_s/reward:0.06217483803629875 - timing_s/old_log_prob:3.1998401042073965 - timing_s/adv:0.1344296932220459 - timing_s/update_actor:12.865752601996064 - timing_s/step:31.24854612722993 - timing_s/stop_profile:2.9239803552627563e-05 - timing_per_token_ms/gen:0.14206672067788353 - timing_per_token_ms/adv:0.0006426047143671976 - timing_per_token_ms/update_actor:0.061501243347097515 - perf/total_num_tokens:209195 - perf/time_per_step:31.24854612722993 - perf/throughput:836.8189321042837 (TaskRunner pid=2403642) Training Progress: 96%|█████████▌| 96/100 [1:00:05<02:12, 33.10s/it] (TaskRunner pid=2403642) step:97 - global_seqlen/min:20859 - global_seqlen/max:29785 - global_seqlen/minmax_diff:8926 - global_seqlen/balanced_min:24566 - global_seqlen/balanced_max:24575 - global_seqlen/mean:24572.5 - actor/entropy:0.28956183791160583 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3534809648990631 - training/rollout_probs_diff_mean:0.002682067221030593 - training/rollout_probs_diff_std:0.007446713745594025 - training/rollout_actor_probs_pearson_corr:0.9995591044425964 - rollout_corr/rollout_is_mean:0.9999961256980896 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.936397671699524 - rollout_corr/rollout_is_min:0.5016483664512634 - rollout_corr/rollout_is_std:0.025280050933361053 - rollout_corr/rollout_is_eff_sample_size:0.9993613271623948 - rollout_corr/rollout_is_seq_mean:1.0000274181365967 - rollout_corr/rollout_is_seq_std:0.0013985984260216355 - rollout_corr/rollout_is_seq_max:1.0056803226470947 - rollout_corr/rollout_is_seq_min:0.9960852861404419 - rollout_corr/rollout_is_seq_max_deviation:0.0056803226470947266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3826120416633785) - rollout_corr/training_log_ppl:np.float64(0.3127178924914915) - rollout_corr/kl:np.float64(0.0003792919155714003) - rollout_corr/k3_kl:np.float64(0.00042534956372719535) - rollout_corr/rollout_ppl:np.float64(1.382072074804455) - rollout_corr/rollout_log_ppl:np.float64(0.3123385964281624) - rollout_corr/log_ppl_diff:np.float64(0.00037929606332909316) - rollout_corr/log_ppl_abs_diff:np.float64(0.00126680287939962) - rollout_corr/log_ppl_diff_max:np.float64(0.006162881851196289) - rollout_corr/log_ppl_diff_min:np.float64(-0.004757493734359741) - rollout_corr/ppl_ratio:np.float64(1.0003806476015598) - rollout_corr/chi2_token:np.float64(0.0009697633795440197) - rollout_corr/chi2_seq:np.float64(0.3747521855402738) - actor/pg_loss:np.float64(-1.547869760543108e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.352384894446004e-05) - actor/ppo_kl:np.float64(6.558593235084231e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08563856221735477) - perf/mfu/actor:np.float64(0.09803966351839774) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.51029205322266) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:3 - critic/score/mean:0.83984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003381997812539339 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003381997812539339 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:401.078125 - response_length/max:956.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:401.078125 - response_length_non_aborted/max:956.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:366.8125 - prompt_length/max:490.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.434663474559784e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1238491851836443) - timing_s/agent_loop/generate_sequences/max:np.float64(8.404852511361241) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.601780556979065) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.404852511361241) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:956 - timing_s/gen:15.47561683692038 - timing_s/reward:0.06152399443089962 - timing_s/old_log_prob:3.0726423896849155 - timing_s/adv:0.1360377948731184 - timing_s/update_actor:13.11331277154386 - timing_s/step:31.946508400142193 - timing_s/stop_profile:0.0001276116818189621 - timing_per_token_ms/gen:0.15072282555729072 - timing_per_token_ms/adv:0.0006920225601440553 - timing_per_token_ms/update_actor:0.06670725796898902 - perf/total_num_tokens:196580 - perf/time_per_step:31.946508400142193 - perf/throughput:769.1763898645846 (TaskRunner pid=2403642) Training Progress: 97%|█████████▋| 97/100 [1:00:37<01:38, 32.77s/it] (TaskRunner pid=2403642) step:98 - global_seqlen/min:21789 - global_seqlen/max:28252 - global_seqlen/minmax_diff:6463 - global_seqlen/balanced_min:23858 - global_seqlen/balanced_max:23860 - global_seqlen/mean:23859.5 - actor/entropy:0.36053964495658875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463629484176636 - training/rollout_probs_diff_mean:0.003115718485787511 - training/rollout_probs_diff_std:0.007648924365639687 - training/rollout_actor_probs_pearson_corr:0.9995946288108826 - rollout_corr/rollout_is_mean:0.9998584389686584 - rollout_corr/rollout_is_ratio_fraction_high:1.1389002793293912e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1389002793293912e-05 - rollout_corr/rollout_is_max:2.498016595840454 - rollout_corr/rollout_is_min:0.2521287500858307 - rollout_corr/rollout_is_std:0.027102908119559288 - rollout_corr/rollout_is_eff_sample_size:0.9992657334826961 - rollout_corr/rollout_is_seq_mean:0.9998365044593811 - rollout_corr/rollout_is_seq_std:0.0016116970218718052 - rollout_corr/rollout_is_seq_max:1.0038925409317017 - rollout_corr/rollout_is_seq_min:0.9945869445800781 - rollout_corr/rollout_is_seq_max_deviation:0.005413055419921875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.469639522023499) - rollout_corr/training_log_ppl:np.float64(0.3758351509895874) - rollout_corr/kl:np.float64(0.0006500774032822676) - rollout_corr/k3_kl:np.float64(0.0005059725597362785) - rollout_corr/rollout_ppl:np.float64(1.4686928312294185) - rollout_corr/rollout_log_ppl:np.float64(0.3751850715925684) - rollout_corr/log_ppl_diff:np.float64(0.0006500793970189989) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014755238080397248) - rollout_corr/log_ppl_diff_max:np.float64(0.006156846880912781) - rollout_corr/log_ppl_diff_min:np.float64(-0.004006743431091309) - rollout_corr/ppl_ratio:np.float64(1.0006518627051264) - rollout_corr/chi2_token:np.float64(0.002016712911427021) - rollout_corr/chi2_seq:np.float64(0.6680954364128411) - actor/pg_loss:np.float64(-2.2069783881306648e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(9.759571342371487e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.060099998489022255) - perf/mfu/actor:np.float64(0.09587603943024309) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.60213470458984) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:3 - critic/score/mean:0.9296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.9296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001405118266120553 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.001405118266120553 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:342.984375 - response_length/max:814.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:342.984375 - response_length_non_aborted/max:814.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:402.625 - prompt_length/max:486.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.733065962791443e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1825728360563517) - timing_s/agent_loop/generate_sequences/max:np.float64(7.279220985248685) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.037709981312219) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.279220985248685) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:486 - timing_s/agent_loop/slowest/response_length:814 - timing_s/gen:14.624581962823868 - timing_s/reward:0.06081410311162472 - timing_s/old_log_prob:3.118553228676319 - timing_s/adv:0.13524136133491993 - timing_s/update_actor:13.040348125621676 - timing_s/step:31.065068090334535 - timing_s/stop_profile:3.069266676902771e-05 - timing_per_token_ms/gen:0.16655940461509575 - timing_per_token_ms/adv:0.000708529942658689 - timing_per_token_ms/update_actor:0.06831842728065171 - perf/total_num_tokens:190876 - perf/time_per_step:31.065068090334535 - perf/throughput:768.0491776363932 (TaskRunner pid=2403642) Training Progress: 98%|█████████▊| 98/100 [1:01:08<01:04, 32.26s/it] (TaskRunner pid=2403642) step:99 - global_seqlen/min:20296 - global_seqlen/max:26640 - global_seqlen/minmax_diff:6344 - global_seqlen/balanced_min:24543 - global_seqlen/balanced_max:24549 - global_seqlen/mean:24547.25 - actor/entropy:0.37918776273727417 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24402102828025818 - training/rollout_probs_diff_mean:0.002885072724893689 - training/rollout_probs_diff_std:0.007192612625658512 - training/rollout_actor_probs_pearson_corr:0.9996479749679565 - rollout_corr/rollout_is_mean:0.9999138116836548 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.599069595336914 - rollout_corr/rollout_is_min:0.7051597237586975 - rollout_corr/rollout_is_std:0.025952504947781563 - rollout_corr/rollout_is_eff_sample_size:0.9993268018058533 - rollout_corr/rollout_is_seq_mean:0.9999309182167053 - rollout_corr/rollout_is_seq_std:0.0014725772198289633 - rollout_corr/rollout_is_seq_max:1.0040785074234009 - rollout_corr/rollout_is_seq_min:0.9944436550140381 - rollout_corr/rollout_is_seq_max_deviation:0.005556344985961914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.520531804766506) - rollout_corr/training_log_ppl:np.float64(0.40119856897217687) - rollout_corr/kl:np.float64(0.00043854576770385734) - rollout_corr/k3_kl:np.float64(0.00043854525202391414) - rollout_corr/rollout_ppl:np.float64(1.5198842682875693) - rollout_corr/rollout_log_ppl:np.float64(0.40076001826673746) - rollout_corr/log_ppl_diff:np.float64(0.0004385507054394111) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013136765192029998) - rollout_corr/log_ppl_diff_max:np.float64(0.005993157625198364) - rollout_corr/log_ppl_diff_min:np.float64(-0.004958093166351318) - rollout_corr/ppl_ratio:np.float64(1.0004400596953928) - rollout_corr/chi2_token:np.float64(0.0009022736921906471) - rollout_corr/chi2_seq:np.float64(0.3876523340586573) - actor/pg_loss:np.float64(2.1395273506641388e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(1.1412551813805294e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07385709136724472) - perf/mfu/actor:np.float64(0.09943231597237594) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.54281997680664) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:3 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004624001740012318 - critic/advantages/max:0.625 - critic/advantages/min:-0.375 - critic/returns/mean:0.0004624001740012318 - critic/returns/max:0.625 - critic/returns/min:-0.375 - response_length/mean:372.7578125 - response_length/max:945.0 - response_length/min:182.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:372.7578125 - response_length_non_aborted/max:945.0 - response_length_non_aborted/min:182.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.34375 - prompt_length/max:480.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.4596229195594788e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3601644691079855) - timing_s/agent_loop/generate_sequences/max:np.float64(8.149387737736106) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.401201009575743) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.149387737736106) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:945 - timing_s/gen:15.560864474624395 - timing_s/reward:0.06195022352039814 - timing_s/old_log_prob:2.985416041687131 - timing_s/adv:0.1166917011141777 - timing_s/update_actor:12.82098894380033 - timing_s/step:31.63107174821198 - timing_s/stop_profile:0.0001179184764623642 - timing_per_token_ms/gen:0.16306734511165086 - timing_per_token_ms/adv:0.0005942198266311792 - timing_per_token_ms/update_actor:0.0652872976799862 - perf/total_num_tokens:196378 - perf/time_per_step:31.63107174821198 - perf/throughput:776.0486332995526 (TaskRunner pid=2403642) Training Progress: 99%|█████████▉| 99/100 [1:01:40<00:32, 32.09s/it] (TaskRunner pid=2403642) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2403642) validation generation end (TaskRunner pid=2403642) [prompt] system (TaskRunner pid=2403642) (TaskRunner pid=2403642) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) ... (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2403642) (TaskRunner pid=2403642) A (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) user (TaskRunner pid=2403642) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2403642) (TaskRunner pid=2403642) Lattice Parameters: (TaskRunner pid=2403642) (TaskRunner pid=2403642) a = 7.82 Å (TaskRunner pid=2403642) b = 7.82 Å (TaskRunner pid=2403642) c = 17.88 Å (TaskRunner pid=2403642) α = 90.00º (TaskRunner pid=2403642) β = 90.00º (TaskRunner pid=2403642) γ = 90.00º (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) A: 1025.67 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) B: 1094.31 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) C: 1200.45 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) D: 1150.78 ų (TaskRunner pid=2403642) (TaskRunner pid=2403642) Please reason step by step. (TaskRunner pid=2403642) assistant (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) [response] (TaskRunner pid=2403642) The given crystal structure has lattice parameters a = 7.82 Å, b = 7.82 Å, c = 17.88 Å, and all angles α, β, γ are 90.00º. This indicates that the crystal structure is orthorhombic, as all angles are 90 degrees and the lattice parameters are not equal. (TaskRunner pid=2403642) (TaskRunner pid=2403642) The volume of the unit cell in a crystal structure is calculated using the formula: (TaskRunner pid=2403642) (TaskRunner pid=2403642) $$ V = a \times b \times c $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Plugging in the values: (TaskRunner pid=2403642) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2403642) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Now multiply that result by 17.88: (TaskRunner pid=2403642) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2403642) (TaskRunner pid=2403642) Therefore, the volume of the unit cell is approximately 1094.31 ų, which corresponds to option B. (TaskRunner pid=2403642) (TaskRunner pid=2403642) (TaskRunner pid=2403642) B (TaskRunner pid=2403642) (TaskRunner pid=2403642) [ground_truth] B (TaskRunner pid=2403642) [score] 1.0 (TaskRunner pid=2403642) [acc] 1.0 (TaskRunner pid=2403642) [pred] B (TaskRunner pid=2403642) [incorrect_format] 1 (TaskRunner pid=2403642) [feedback] (TaskRunner pid=2403642) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2403642) step:100 - global_seqlen/min:20619 - global_seqlen/max:28642 - global_seqlen/minmax_diff:8023 - global_seqlen/balanced_min:24226 - global_seqlen/balanced_max:24234 - global_seqlen/mean:24231.0 - actor/entropy:0.41687414050102234 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2016180157661438 - training/rollout_probs_diff_mean:0.0030894633382558823 - training/rollout_probs_diff_std:0.007387364748865366 - training/rollout_actor_probs_pearson_corr:0.9996216893196106 - rollout_corr/rollout_is_mean:0.9999920725822449 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6438628435134888 - rollout_corr/rollout_is_min:0.5598989725112915 - rollout_corr/rollout_is_std:0.02741776406764984 - rollout_corr/rollout_is_eff_sample_size:0.9992487118690302 - rollout_corr/rollout_is_seq_mean:0.9999960064888 - rollout_corr/rollout_is_seq_std:0.0014865802368149161 - rollout_corr/rollout_is_seq_max:1.0055478811264038 - rollout_corr/rollout_is_seq_min:0.9939270615577698 - rollout_corr/rollout_is_seq_max_deviation:0.006072938442230225 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.5640644407831132) - rollout_corr/training_log_ppl:np.float64(0.4244483573420439) - rollout_corr/kl:np.float64(0.0003323329324800284) - rollout_corr/k3_kl:np.float64(0.0004623864396933186) - rollout_corr/rollout_ppl:np.float64(1.5636177295818925) - rollout_corr/rollout_log_ppl:np.float64(0.4241160247474909) - rollout_corr/log_ppl_diff:np.float64(0.0003323325945530087) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012461927544791251) - rollout_corr/log_ppl_diff_max:np.float64(0.006919294595718384) - rollout_corr/log_ppl_diff_min:np.float64(-0.004684805870056152) - rollout_corr/ppl_ratio:np.float64(1.00033376715146) - rollout_corr/chi2_token:np.float64(0.0012154490686953068) - rollout_corr/chi2_seq:np.float64(0.4026260869577527) - actor/pg_loss:np.float64(-1.95368193089962e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.2877348328620428e-05) - actor/ppo_kl:np.float64(-5.772552391647423e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1109579335898161) - perf/mfu/actor:np.float64(0.09790439064664765) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.49691772460938) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7799202127659575) - val-aux/sciknoweval/reward/std@16:np.float64(0.052101428059437495) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.799936170212766) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.03891400102821863) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7602021276595745) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.0484960635134995) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7798829787234042) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.052006938967530975) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8139574468085107) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.027455777877525876) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7414893617021275) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.04241546758668117) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7823297872340426) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.038877017813791076) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8243297872340425) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.019003297583575607) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7248191489361702) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.035434813866661946) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7831382978723406) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.02545128949215922) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8322765957446808) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.013309078916541482) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.7087659574468085) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.027047818566610645) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7843510638297873) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.01931287084263904) - val-aux/sciknoweval/score/mean@16:np.float64(0.7799202127659575) - val-aux/sciknoweval/score/std@16:np.float64(0.052101428059437495) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.799936170212766) - val-aux/sciknoweval/score/best@2/std:np.float64(0.03891400102821863) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7602021276595745) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.0484960635134995) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7798829787234042) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.052006938967530975) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8139574468085107) - val-aux/sciknoweval/score/best@4/std:np.float64(0.027455777877525876) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7414893617021275) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.04241546758668117) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7823297872340426) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.038877017813791076) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8243297872340425) - val-aux/sciknoweval/score/best@8/std:np.float64(0.019003297583575607) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7248191489361702) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.035434813866661946) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7831382978723406) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.02545128949215922) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8322765957446808) - val-aux/sciknoweval/score/best@16/std:np.float64(0.013309078916541482) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.7087659574468085) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.027047818566610645) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7843510638297873) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.01931287084263904) - val-core/sciknoweval/acc/mean@16:np.float64(0.7799202127659575) - val-aux/sciknoweval/acc/std@16:np.float64(0.052101428059437495) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.799936170212766) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.03891400102821863) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7602021276595745) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.0484960635134995) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7798829787234042) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.052006938967530975) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8139574468085107) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.027455777877525876) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7414893617021275) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.04241546758668117) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7823297872340426) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.038877017813791076) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8243297872340425) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.019003297583575607) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7248191489361702) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.035434813866661946) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7831382978723406) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.02545128949215922) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8322765957446808) - val-core/sciknoweval/acc/best@16/std:np.float64(0.013309078916541482) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.7087659574468085) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.027047818566610645) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7843510638297873) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.01931287084263904) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:3 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009803137509152293 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009803137509152293 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:366.09375 - response_length/max:851.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:366.09375 - response_length_non_aborted/max:851.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.125 - prompt_length/max:479.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011714361608028412 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6114151515066624) - timing_s/agent_loop/generate_sequences/max:np.float64(7.582470629364252) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.872075782623142) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.582470629364252) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:439 - timing_s/agent_loop/slowest/response_length:812 - timing_s/gen:15.210048558190465 - timing_s/reward:0.0619368739426136 - timing_s/old_log_prob:3.036892930045724 - timing_s/adv:0.1176000852137804 - timing_s/update_actor:12.983655489981174 - timing_s/step:31.49581053480506 - timing_s/testing:44.8417251277715 - timing_s/save_checkpoint:12.71921988017857 - timing_s/stop_profile:0.00015413016080856323 - timing_per_token_ms/gen:0.16229245153852395 - timing_per_token_ms/adv:0.0006066613285346271 - timing_per_token_ms/update_actor:0.06697853725589727 - perf/total_num_tokens:193848 - perf/time_per_step:31.49581053480506 - perf/throughput:769.3404166634499 (TaskRunner pid=2403642) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2403642) "np.float64(0.7799202127659575), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2403642) "np.float64(0.052101428059437495), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2403642) "np.float64(0.799936170212766), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2403642) "np.float64(0.03891400102821863), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2403642) "np.float64(0.7602021276595745), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2403642) "np.float64(0.0484960635134995), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2403642) "np.float64(0.7798829787234042), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2403642) "np.float64(0.052006938967530975), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2403642) "np.float64(0.8139574468085107), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2403642) "np.float64(0.027455777877525876), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2403642) "np.float64(0.7414893617021275), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2403642) "np.float64(0.04241546758668117), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2403642) "np.float64(0.7823297872340426), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2403642) "np.float64(0.038877017813791076), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2403642) "np.float64(0.8243297872340425), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2403642) "np.float64(0.019003297583575607), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2403642) "np.float64(0.7248191489361702), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2403642) "np.float64(0.035434813866661946), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2403642) "np.float64(0.7831382978723406), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2403642) "np.float64(0.02545128949215922), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2403642) "np.float64(0.8322765957446808), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2403642) 'np.float64(0.013309078916541482), ' (TaskRunner pid=2403642) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.7087659574468085), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.027047818566610645), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7843510638297873), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.01931287084263904), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/mean@16': np.float64(0.7799202127659575), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/std@16': np.float64(0.052101428059437495), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.799936170212766), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.03891400102821863), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.7602021276595745), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.0484960635134995), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.7798829787234042), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.052006938967530975), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.8139574468085107), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.027455777877525876), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.7414893617021275), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.04241546758668117), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.7823297872340426), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.038877017813791076), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8243297872340425), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.019003297583575607), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.7248191489361702), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.035434813866661946), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.7831382978723406), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.02545128949215922), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8322765957446808), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.013309078916541482), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.7087659574468085), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.027047818566610645), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7843510638297873), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.01931287084263904), " (TaskRunner pid=2403642) "'val-core/sciknoweval/acc/mean@16': np.float64(0.7799202127659575), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/std@16': np.float64(0.052101428059437495), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.799936170212766), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.03891400102821863), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.7602021276595745), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.0484960635134995), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.7798829787234042), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.052006938967530975), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.8139574468085107), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.027455777877525876), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.7414893617021275), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.04241546758668117), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.7823297872340426), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.038877017813791076), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8243297872340425), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.019003297583575607), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.7248191489361702), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.035434813866661946), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.7831382978723406), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.02545128949215922), " (TaskRunner pid=2403642) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8322765957446808), " (TaskRunner pid=2403642) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.013309078916541482), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.7087659574468085), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.027047818566610645), " (TaskRunner pid=2403642) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7843510638297873), " (TaskRunner pid=2403642) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.01931287084263904), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/std@16': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/best@2/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/worst@2/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/maj@2/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/worst@4/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/maj@4/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/worst@8/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/worst@16/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2403642) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2403642) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2403642) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2403642) Training Progress: 100%|██████████| 100/100 [1:03:09<00:00, 49.20s/it] Training Progress: 100%|██████████| 100/100 [1:03:09<00:00, 37.89s/it] (TaskRunner pid=2403642) wandb: updating run metadata (TaskRunner pid=2403642) wandb: updating run metadata; uploading output.log; uploading wandb-summary.json (TaskRunner pid=2403642) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2403642) wandb: uploading history steps 98-99, summary, console lines 939-1046 (TaskRunner pid=2403642) wandb: (TaskRunner pid=2403642) wandb: Run history: (TaskRunner pid=2403642) wandb: actor/entropy ▁▁▂▁▂▂▂▂▂▂▃▃▄▄▄▄▄▅▄▄▄▄▄▄▃▃▄▃▄▃▅▅▆▅▆▅▅▄▇█ (TaskRunner pid=2403642) wandb: actor/grad_norm ▅▅█▆▅▃▄▄▄▃▃▃▄▃▂▃▃▄▃▃▄▃▃▃▃▂▃▂▂▂▃▂▃▃▂▂▂▅▁▂ (TaskRunner pid=2403642) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2403642) wandb: actor/lr ▁▂▃▄▇███████████████████████████████████ (TaskRunner pid=2403642) wandb: actor/pg_clipfrac ▅▅▄█▃▃▄▂▂▂▁▂▂▂▁▂▂▂▂▃▂▂▂▂▂▁▁▁▂▁▂▁▁▁▁▂▂▃▁▁ (TaskRunner pid=2403642) wandb: actor/pg_clipfrac_lower ▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2403642) wandb: actor/pg_loss ▆▂▄▇▄█▆▁▆▃▆▃▅▆▆▄█▃▆▅▄▆▅▆▄▅▅▃▃▆▅▆▂▄▅▄▅▆▅▅ (TaskRunner pid=2403642) wandb: actor/ppo_kl ▆▆▁▄▄▅▅▂▄▂▄▄▄▃▄▅▅▃▅▅▂█▆▆▄▄▅▆▄▄▅▅▆▃▆▇▄▆▆▃ (TaskRunner pid=2403642) wandb: critic/advantages/max ▇█████▇█████▇█▅▆▇████▆▅███▁▁█▇▁█▅▁██▃█▅▆ (TaskRunner pid=2403642) wandb: critic/advantages/mean █▆█▆▆▅▆▆▄▄▄▄▃▁▂▅▂▆▄▄▄▃▄▄▃▄▄▄▅▄▄▄▅▄▄▅▅▃▆▂ (TaskRunner pid=2403642) wandb: +204 ... (TaskRunner pid=2403642) wandb: (TaskRunner pid=2403642) wandb: Run summary: (TaskRunner pid=2403642) wandb: actor/entropy 0.41687 (TaskRunner pid=2403642) wandb: actor/grad_norm 0.11096 (TaskRunner pid=2403642) wandb: actor/kl_loss 0 (TaskRunner pid=2403642) wandb: actor/lr 0.0 (TaskRunner pid=2403642) wandb: actor/pg_clipfrac 2e-05 (TaskRunner pid=2403642) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2403642) wandb: actor/pg_loss -0.0 (TaskRunner pid=2403642) wandb: actor/ppo_kl -6e-05 (TaskRunner pid=2403642) wandb: critic/advantages/max 0.375 (TaskRunner pid=2403642) wandb: critic/advantages/mean -0.00098 (TaskRunner pid=2403642) wandb: +204 ... (TaskRunner pid=2403642) wandb: (TaskRunner pid=2403642) wandb: 🚀 View run qwen3gen-material-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/1ud3umaz (TaskRunner pid=2403642) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2403642) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2403642) wandb: Find logs at: ./wandb/run-20260703_105138-1ud3umaz/logs (TaskRunner pid=2403642) Exception ignored in atexit callback: .teardown_atexit at 0x70b6218f3880> (TaskRunner pid=2403642) Traceback (most recent call last): (TaskRunner pid=2403642) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2403642) conn.teardown(hooks.exit_code) (TaskRunner pid=2403642) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2403642) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2403642) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2403642) return future.result() (TaskRunner pid=2403642) ^^^^^^^^^^^^^^^ (TaskRunner pid=2403642) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2403642) return self.__get_result() (TaskRunner pid=2403642) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2403642) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2403642) raise self._exception (TaskRunner pid=2403642) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2403642) return await fn() (TaskRunner pid=2403642) ^^^^^^^^^^ (TaskRunner pid=2403642) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2403642) await self._client.close() (TaskRunner pid=2403642) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2403642) await self._writer.wait_closed() (TaskRunner pid=2403642) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2403642) await self._protocol._get_close_waiter(self) (TaskRunner pid=2403642) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-03 11:54:55] Finished Qwen/Qwen3-8B material grpo. [2026-07-03 11:54:55] Starting Qwen/Qwen3-8B material rlsd_tr. Experiment: qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: material Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet Ray tmp: /tmp/ray_q3g_material_rlsd_tr_Qwen3_8B 2026-07-03 11:54:56,944 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/material Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_material_rlsd_tr_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/material +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_material_rlsd_tr_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/material', 'EXPERIMENT': 'qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 11:55:06,864 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2425326) TaskRunner hostname: mole-workspace-rw, PID: 2425326 (TaskRunner pid=2425326) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2425326) 'calculate_entropy': False, (TaskRunner pid=2425326) 'calculate_sum_pi_squared': False, (TaskRunner pid=2425326) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2425326) 'async_save': False, (TaskRunner pid=2425326) 'load_contents': ['model', (TaskRunner pid=2425326) 'optimizer', (TaskRunner pid=2425326) 'extra'], (TaskRunner pid=2425326) 'save_contents': ['model', (TaskRunner pid=2425326) 'optimizer', (TaskRunner pid=2425326) 'extra']}, (TaskRunner pid=2425326) 'clip_ratio': 0.2, (TaskRunner pid=2425326) 'clip_ratio_c': 3.0, (TaskRunner pid=2425326) 'clip_ratio_high': 0.28, (TaskRunner pid=2425326) 'clip_ratio_low': 0.2, (TaskRunner pid=2425326) 'data_loader_seed': 42, (TaskRunner pid=2425326) 'entropy_checkpointing': False, (TaskRunner pid=2425326) 'entropy_coeff': 0, (TaskRunner pid=2425326) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2425326) 'freeze_vision_tower': False, (TaskRunner pid=2425326) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2425326) 'dtype': 'bfloat16', (TaskRunner pid=2425326) 'entropy_checkpointing': False, (TaskRunner pid=2425326) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2425326) 'forward_only': False, (TaskRunner pid=2425326) 'forward_prefetch': False, (TaskRunner pid=2425326) 'fsdp_size': -1, (TaskRunner pid=2425326) 'full_determinism': False, (TaskRunner pid=2425326) 'model_dtype': 'fp32', (TaskRunner pid=2425326) 'offload_policy': False, (TaskRunner pid=2425326) 'optimizer_offload': False, (TaskRunner pid=2425326) 'param_offload': False, (TaskRunner pid=2425326) 'reshard_after_forward': True, (TaskRunner pid=2425326) 'seed': 42, (TaskRunner pid=2425326) 'strategy': 'fsdp', (TaskRunner pid=2425326) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2425326) 'use_orig_params': False, (TaskRunner pid=2425326) 'use_torch_compile': True, (TaskRunner pid=2425326) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2425326) 'grad_clip': 1.0, (TaskRunner pid=2425326) 'kl_loss_coef': 0.001, (TaskRunner pid=2425326) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2425326) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2425326) 'loss_scale_factor': None, (TaskRunner pid=2425326) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2425326) 'betas': [0.9, 0.999], (TaskRunner pid=2425326) 'clip_grad': 1.0, (TaskRunner pid=2425326) 'lr': 1e-06, (TaskRunner pid=2425326) 'lr_scheduler_type': 'constant', (TaskRunner pid=2425326) 'lr_warmup_steps': 10, (TaskRunner pid=2425326) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2425326) 'min_lr_ratio': 0.0, (TaskRunner pid=2425326) 'num_cycles': 0.5, (TaskRunner pid=2425326) 'optimizer': 'AdamW', (TaskRunner pid=2425326) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2425326) 'override_optimizer_config': None, (TaskRunner pid=2425326) 'total_training_steps': -1, (TaskRunner pid=2425326) 'warmup_style': None, (TaskRunner pid=2425326) 'weight_decay': 0.01}, (TaskRunner pid=2425326) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2425326) 'clip_cov_lb': 1.0, (TaskRunner pid=2425326) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2425326) 'clip_cov_ub': 5.0, (TaskRunner pid=2425326) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2425326) 'loss_mode': 'rlsd', (TaskRunner pid=2425326) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2425326) 'ppo_epochs': 1, (TaskRunner pid=2425326) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2425326) 'ppo_micro_batch_size': None, (TaskRunner pid=2425326) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2425326) 'ppo_mini_batch_size': 8, (TaskRunner pid=2425326) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2425326) 'all_ranks': False, (TaskRunner pid=2425326) 'enable': False, (TaskRunner pid=2425326) 'ranks': [], (TaskRunner pid=2425326) 'save_path': 'outputs/profile', (TaskRunner pid=2425326) 'tool': None, (TaskRunner pid=2425326) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2425326) 'analysis': True, (TaskRunner pid=2425326) 'contents': [], (TaskRunner pid=2425326) 'discrete': False, (TaskRunner pid=2425326) 'level': 'level0'}, (TaskRunner pid=2425326) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2425326) 'discrete': False}, (TaskRunner pid=2425326) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2425326) 'step_end': None, (TaskRunner pid=2425326) 'step_start': 0}, (TaskRunner pid=2425326) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2425326) 'stack_depth': 32, (TaskRunner pid=2425326) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2425326) 'rollout_n': 8, (TaskRunner pid=2425326) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2425326) 'mode': 'disabled', (TaskRunner pid=2425326) 'record_file': None, (TaskRunner pid=2425326) 'replay_file': None}, (TaskRunner pid=2425326) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2425326) 'alpha': 0.5, (TaskRunner pid=2425326) 'distillation_add_tail': True, (TaskRunner pid=2425326) 'distillation_topk': 100, (TaskRunner pid=2425326) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2425326) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2425326) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2425326) 'enable': False, (TaskRunner pid=2425326) 'loss_weight': 0.0, (TaskRunner pid=2425326) 'mask': 'all'}, (TaskRunner pid=2425326) 'feedback_template': '\n' (TaskRunner pid=2425326) 'The ' (TaskRunner pid=2425326) 'following ' (TaskRunner pid=2425326) 'is ' (TaskRunner pid=2425326) 'feedback ' (TaskRunner pid=2425326) 'from ' (TaskRunner pid=2425326) 'your ' (TaskRunner pid=2425326) 'unsuccessful ' (TaskRunner pid=2425326) 'earlier ' (TaskRunner pid=2425326) 'attempt:\n' (TaskRunner pid=2425326) '\n' (TaskRunner pid=2425326) '{feedback_raw}', (TaskRunner pid=2425326) 'full_logit_distillation': True, (TaskRunner pid=2425326) 'include_environment_feedback': True, (TaskRunner pid=2425326) 'is_clip': 2, (TaskRunner pid=2425326) 'max_reprompt_len': 10240, (TaskRunner pid=2425326) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2425326) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2425326) '\n' (TaskRunner pid=2425326) 'Correctly ' (TaskRunner pid=2425326) 'solve ' (TaskRunner pid=2425326) 'the ' (TaskRunner pid=2425326) 'original ' (TaskRunner pid=2425326) 'question.', (TaskRunner pid=2425326) 'reprompt_truncation': 'right', (TaskRunner pid=2425326) 'solution_template': '\n' (TaskRunner pid=2425326) 'Correct ' (TaskRunner pid=2425326) 'solution:\n' (TaskRunner pid=2425326) '\n' (TaskRunner pid=2425326) '{successful_previous_attempt}', (TaskRunner pid=2425326) 'srpo_dynamic_weighting': False, (TaskRunner pid=2425326) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2425326) 'success_reward_threshold': 0.5, (TaskRunner pid=2425326) 'teacher_regularization': 'trust-region', (TaskRunner pid=2425326) 'teacher_update_rate': 0.1, (TaskRunner pid=2425326) 'token_reweight_decay_steps': 0, (TaskRunner pid=2425326) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2425326) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2425326) 'shuffle': False, (TaskRunner pid=2425326) 'strategy': 'fsdp', (TaskRunner pid=2425326) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2425326) 'tau_neg': 1.05, (TaskRunner pid=2425326) 'tau_pos': 1.0, (TaskRunner pid=2425326) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2425326) 'use_dynamic_bsz': False, (TaskRunner pid=2425326) 'use_fused_kernels': False, (TaskRunner pid=2425326) 'use_kl_loss': False, (TaskRunner pid=2425326) 'use_prefix_grouper': False, (TaskRunner pid=2425326) 'use_remove_padding': True, (TaskRunner pid=2425326) 'use_torch_compile': True}, (TaskRunner pid=2425326) 'hybrid_engine': True, (TaskRunner pid=2425326) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2425326) 'custom_chat_template': None, (TaskRunner pid=2425326) 'enable_activation_offload': False, (TaskRunner pid=2425326) 'enable_gradient_checkpointing': True, (TaskRunner pid=2425326) 'exclude_modules': None, (TaskRunner pid=2425326) 'external_lib': None, (TaskRunner pid=2425326) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2425326) 'hf_config_path': None, (TaskRunner pid=2425326) 'lora_adapter_path': None, (TaskRunner pid=2425326) 'lora_alpha': 16, (TaskRunner pid=2425326) 'lora_rank': 0, (TaskRunner pid=2425326) 'override_config': {}, (TaskRunner pid=2425326) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2425326) 'target_modules': 'all-linear', (TaskRunner pid=2425326) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2425326) 'num_shards': 4}, (TaskRunner pid=2425326) 'tokenizer_path': None, (TaskRunner pid=2425326) 'trust_remote_code': True, (TaskRunner pid=2425326) 'use_fused_kernels': False, (TaskRunner pid=2425326) 'use_liger': False, (TaskRunner pid=2425326) 'use_remove_padding': True, (TaskRunner pid=2425326) 'use_shm': False}, (TaskRunner pid=2425326) 'nccl_timeout': 600, (TaskRunner pid=2425326) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2425326) 'entropy_checkpointing': False, (TaskRunner pid=2425326) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2425326) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2425326) 'dtype': 'bfloat16', (TaskRunner pid=2425326) 'entropy_checkpointing': False, (TaskRunner pid=2425326) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2425326) 'forward_only': True, (TaskRunner pid=2425326) 'forward_prefetch': False, (TaskRunner pid=2425326) 'fsdp_size': -1, (TaskRunner pid=2425326) 'full_determinism': False, (TaskRunner pid=2425326) 'model_dtype': 'fp32', (TaskRunner pid=2425326) 'offload_policy': False, (TaskRunner pid=2425326) 'optimizer_offload': False, (TaskRunner pid=2425326) 'param_offload': False, (TaskRunner pid=2425326) 'reshard_after_forward': True, (TaskRunner pid=2425326) 'seed': 42, (TaskRunner pid=2425326) 'strategy': 'fsdp', (TaskRunner pid=2425326) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2425326) 'use_orig_params': False, (TaskRunner pid=2425326) 'use_torch_compile': True, (TaskRunner pid=2425326) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2425326) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2425326) 'log_prob_micro_batch_size': None, (TaskRunner pid=2425326) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2425326) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2425326) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2425326) 'all_ranks': False, (TaskRunner pid=2425326) 'enable': False, (TaskRunner pid=2425326) 'ranks': [], (TaskRunner pid=2425326) 'save_path': 'outputs/profile', (TaskRunner pid=2425326) 'tool': None, (TaskRunner pid=2425326) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2425326) 'analysis': True, (TaskRunner pid=2425326) 'contents': [], (TaskRunner pid=2425326) 'discrete': False, (TaskRunner pid=2425326) 'level': 'level0'}, (TaskRunner pid=2425326) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2425326) 'discrete': False}, (TaskRunner pid=2425326) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2425326) 'step_end': None, (TaskRunner pid=2425326) 'step_start': 0}, (TaskRunner pid=2425326) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2425326) 'stack_depth': 32, (TaskRunner pid=2425326) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2425326) 'rollout_n': 8, (TaskRunner pid=2425326) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2425326) 'mode': 'disabled', (TaskRunner pid=2425326) 'record_file': None, (TaskRunner pid=2425326) 'replay_file': None}, (TaskRunner pid=2425326) 'strategy': 'fsdp', (TaskRunner pid=2425326) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2425326) 'use_torch_compile': True}, (TaskRunner pid=2425326) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2425326) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2425326) 'agent_loop_config_path': None, (TaskRunner pid=2425326) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2425326) 'name': None, (TaskRunner pid=2425326) 'path': None}, (TaskRunner pid=2425326) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2425326) 'num_workers': 8}, (TaskRunner pid=2425326) 'calculate_log_probs': True, (TaskRunner pid=2425326) 'cudagraph_capture_sizes': None, (TaskRunner pid=2425326) 'data_parallel_size': 1, (TaskRunner pid=2425326) 'disable_log_stats': True, (TaskRunner pid=2425326) 'do_sample': True, (TaskRunner pid=2425326) 'dtype': 'bfloat16', (TaskRunner pid=2425326) 'enable_chunked_prefill': True, (TaskRunner pid=2425326) 'enable_prefix_caching': True, (TaskRunner pid=2425326) 'enable_rollout_routing_replay': False, (TaskRunner pid=2425326) 'enforce_eager': False, (TaskRunner pid=2425326) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2425326) 'expert_parallel_size': 1, (TaskRunner pid=2425326) 'free_cache_engine': True, (TaskRunner pid=2425326) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2425326) 'ignore_eos': False, (TaskRunner pid=2425326) 'layered_summon': False, (TaskRunner pid=2425326) 'load_format': 'dummy', (TaskRunner pid=2425326) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2425326) 'log_prob_micro_batch_size': None, (TaskRunner pid=2425326) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2425326) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2425326) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2425326) 'max_model_len': 10240, (TaskRunner pid=2425326) 'max_num_batched_tokens': 10240, (TaskRunner pid=2425326) 'max_num_seqs': 1024, (TaskRunner pid=2425326) 'mode': 'async', (TaskRunner pid=2425326) 'multi_stage_wake_up': False, (TaskRunner pid=2425326) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2425326) 'enable': False, (TaskRunner pid=2425326) 'format': 'hermes', (TaskRunner pid=2425326) 'interaction_config_path': None, (TaskRunner pid=2425326) 'max_assistant_turns': None, (TaskRunner pid=2425326) 'max_parallel_calls': 1, (TaskRunner pid=2425326) 'max_tool_response_length': 256, (TaskRunner pid=2425326) 'max_user_turns': None, (TaskRunner pid=2425326) 'num_repeat_rollouts': None, (TaskRunner pid=2425326) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2425326) 'tool_config_path': None, (TaskRunner pid=2425326) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2425326) 'use_inference_chat_template': False}, (TaskRunner pid=2425326) 'n': 8, (TaskRunner pid=2425326) 'name': 'vllm', (TaskRunner pid=2425326) 'over_sample_rate': 0, (TaskRunner pid=2425326) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2425326) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2425326) 'all_ranks': False, (TaskRunner pid=2425326) 'enable': False, (TaskRunner pid=2425326) 'ranks': [], (TaskRunner pid=2425326) 'save_path': 'outputs/profile', (TaskRunner pid=2425326) 'tool': None, (TaskRunner pid=2425326) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2425326) 'analysis': True, (TaskRunner pid=2425326) 'contents': [], (TaskRunner pid=2425326) 'discrete': False, (TaskRunner pid=2425326) 'level': 'level0'}, (TaskRunner pid=2425326) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2425326) 'discrete': False}, (TaskRunner pid=2425326) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2425326) 'step_end': None, (TaskRunner pid=2425326) 'step_start': 0}, (TaskRunner pid=2425326) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2425326) 'stack_depth': 32, (TaskRunner pid=2425326) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2425326) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2425326) 'enable': False, (TaskRunner pid=2425326) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2425326) 'port': 9090, (TaskRunner pid=2425326) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2425326) 'prompt_length': 2048, (TaskRunner pid=2425326) 'quantization': None, (TaskRunner pid=2425326) 'quantization_config_file': None, (TaskRunner pid=2425326) 'response_length': 8192, (TaskRunner pid=2425326) 'scheduling_policy': 'fcfs', (TaskRunner pid=2425326) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2425326) 'skip_rollout': False, (TaskRunner pid=2425326) 'skip_tokenizer_init': True, (TaskRunner pid=2425326) 'temperature': 1.0, (TaskRunner pid=2425326) 'tensor_model_parallel_size': 2, (TaskRunner pid=2425326) 'top_k': -1, (TaskRunner pid=2425326) 'top_p': 1.0, (TaskRunner pid=2425326) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2425326) 'backend': None, (TaskRunner pid=2425326) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2425326) 'token2text': False}, (TaskRunner pid=2425326) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2425326) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2425326) 'do_sample': True, (TaskRunner pid=2425326) 'n': 16, (TaskRunner pid=2425326) 'temperature': 0.6, (TaskRunner pid=2425326) 'top_k': -1, (TaskRunner pid=2425326) 'top_p': 0.95}}}, (TaskRunner pid=2425326) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2425326) 'adv_estimator': 'grpo', (TaskRunner pid=2425326) 'gamma': 1.0, (TaskRunner pid=2425326) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2425326) 'horizon': 10000, (TaskRunner pid=2425326) 'kl_coef': 0.001, (TaskRunner pid=2425326) 'target_kl': 0.1, (TaskRunner pid=2425326) 'type': 'fixed'}, (TaskRunner pid=2425326) 'kl_penalty': 'kl', (TaskRunner pid=2425326) 'lam': 1.0, (TaskRunner pid=2425326) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2425326) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2425326) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2425326) 'loss_type': 'ppo_clip', (TaskRunner pid=2425326) 'rollout_is': 'token', (TaskRunner pid=2425326) 'rollout_is_batch_normalize': False, (TaskRunner pid=2425326) 'rollout_is_threshold': 2.0, (TaskRunner pid=2425326) 'rollout_rs': None, (TaskRunner pid=2425326) 'rollout_rs_threshold': None}, (TaskRunner pid=2425326) 'use_kl_in_reward': False, (TaskRunner pid=2425326) 'use_pf_ppo': False}, (TaskRunner pid=2425326) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2425326) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2425326) 'async_save': False, (TaskRunner pid=2425326) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2425326) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2425326) 'cliprange_value': 0.5, (TaskRunner pid=2425326) 'data_loader_seed': 42, (TaskRunner pid=2425326) 'enable': None, (TaskRunner pid=2425326) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2425326) 'forward_micro_batch_size': None, (TaskRunner pid=2425326) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2425326) 'grad_clip': 1.0, (TaskRunner pid=2425326) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2425326) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2425326) 'enable_activation_offload': False, (TaskRunner pid=2425326) 'enable_gradient_checkpointing': True, (TaskRunner pid=2425326) 'external_lib': None, (TaskRunner pid=2425326) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2425326) 'dtype': 'bfloat16', (TaskRunner pid=2425326) 'entropy_checkpointing': False, (TaskRunner pid=2425326) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2425326) 'forward_only': False, (TaskRunner pid=2425326) 'forward_prefetch': False, (TaskRunner pid=2425326) 'fsdp_size': -1, (TaskRunner pid=2425326) 'full_determinism': False, (TaskRunner pid=2425326) 'model_dtype': 'fp32', (TaskRunner pid=2425326) 'offload_policy': False, (TaskRunner pid=2425326) 'optimizer_offload': False, (TaskRunner pid=2425326) 'param_offload': False, (TaskRunner pid=2425326) 'reshard_after_forward': True, (TaskRunner pid=2425326) 'seed': 42, (TaskRunner pid=2425326) 'strategy': 'fsdp', (TaskRunner pid=2425326) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2425326) 'use_orig_params': False, (TaskRunner pid=2425326) 'use_torch_compile': True, (TaskRunner pid=2425326) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2425326) 'lora_alpha': 16, (TaskRunner pid=2425326) 'lora_rank': 0, (TaskRunner pid=2425326) 'override_config': {}, (TaskRunner pid=2425326) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2425326) 'target_modules': 'all-linear', (TaskRunner pid=2425326) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2425326) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2425326) 'trust_remote_code': True, (TaskRunner pid=2425326) 'use_remove_padding': False, (TaskRunner pid=2425326) 'use_shm': False}, (TaskRunner pid=2425326) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2425326) 'betas': [0.9, 0.999], (TaskRunner pid=2425326) 'clip_grad': 1.0, (TaskRunner pid=2425326) 'lr': 1e-05, (TaskRunner pid=2425326) 'lr_scheduler_type': 'constant', (TaskRunner pid=2425326) 'lr_warmup_steps': -1, (TaskRunner pid=2425326) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2425326) 'min_lr_ratio': 0.0, (TaskRunner pid=2425326) 'num_cycles': 0.5, (TaskRunner pid=2425326) 'optimizer': 'AdamW', (TaskRunner pid=2425326) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2425326) 'override_optimizer_config': None, (TaskRunner pid=2425326) 'total_training_steps': -1, (TaskRunner pid=2425326) 'warmup_style': None, (TaskRunner pid=2425326) 'weight_decay': 0.01}, (TaskRunner pid=2425326) 'ppo_epochs': 1, (TaskRunner pid=2425326) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2425326) 'ppo_micro_batch_size': None, (TaskRunner pid=2425326) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2425326) 'ppo_mini_batch_size': 8, (TaskRunner pid=2425326) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2425326) 'all_ranks': False, (TaskRunner pid=2425326) 'enable': False, (TaskRunner pid=2425326) 'ranks': [], (TaskRunner pid=2425326) 'save_path': 'outputs/profile', (TaskRunner pid=2425326) 'tool': None, (TaskRunner pid=2425326) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2425326) 'analysis': True, (TaskRunner pid=2425326) 'contents': [], (TaskRunner pid=2425326) 'discrete': False, (TaskRunner pid=2425326) 'level': 'level0'}, (TaskRunner pid=2425326) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2425326) 'discrete': False}, (TaskRunner pid=2425326) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2425326) 'step_end': None, (TaskRunner pid=2425326) 'step_start': 0}, (TaskRunner pid=2425326) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2425326) 'stack_depth': 32, (TaskRunner pid=2425326) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2425326) 'rollout_n': 8, (TaskRunner pid=2425326) 'shuffle': False, (TaskRunner pid=2425326) 'strategy': 'fsdp', (TaskRunner pid=2425326) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2425326) 'use_dynamic_bsz': False}, (TaskRunner pid=2425326) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2425326) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2425326) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2425326) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2425326) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2425326) 'dataloader_num_workers': 8, (TaskRunner pid=2425326) 'filter_overlong_prompts': True, (TaskRunner pid=2425326) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2425326) 'image_key': 'images', (TaskRunner pid=2425326) 'image_patch_size': 14, (TaskRunner pid=2425326) 'max_prompt_length': 2048, (TaskRunner pid=2425326) 'max_response_length': 8192, (TaskRunner pid=2425326) 'prompt_key': 'prompt', (TaskRunner pid=2425326) 'return_full_prompt': False, (TaskRunner pid=2425326) 'return_multi_modal_inputs': True, (TaskRunner pid=2425326) 'return_raw_chat': True, (TaskRunner pid=2425326) 'return_raw_input_ids': False, (TaskRunner pid=2425326) 'reward_fn_key': 'data_source', (TaskRunner pid=2425326) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2425326) 'seed': None, (TaskRunner pid=2425326) 'shuffle': True, (TaskRunner pid=2425326) 'tokenizer': None, (TaskRunner pid=2425326) 'tool_config_path': None, (TaskRunner pid=2425326) 'train_batch_size': 32, (TaskRunner pid=2425326) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet'], (TaskRunner pid=2425326) 'train_max_samples': 3200, (TaskRunner pid=2425326) 'truncation': 'error', (TaskRunner pid=2425326) 'trust_remote_code': True, (TaskRunner pid=2425326) 'use_shm': False, (TaskRunner pid=2425326) 'val_batch_size': None, (TaskRunner pid=2425326) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet'], (TaskRunner pid=2425326) 'val_max_samples': -1, (TaskRunner pid=2425326) 'validation_shuffle': False, (TaskRunner pid=2425326) 'video_key': 'videos'}, (TaskRunner pid=2425326) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2425326) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2425326) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2425326) 'cuda-memory-usage': 'true', (TaskRunner pid=2425326) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2425326) 'discrete': False, (TaskRunner pid=2425326) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2425326) 'capture-range-end': None, (TaskRunner pid=2425326) 'cuda-graph-trace': 'graph', (TaskRunner pid=2425326) 'cuda-memory-usage': 'true', (TaskRunner pid=2425326) 'kill': 'none', (TaskRunner pid=2425326) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2425326) 'torch_memory': {'context': 'all', (TaskRunner pid=2425326) 'kw_args': {}, (TaskRunner pid=2425326) 'stack_depth': 32, (TaskRunner pid=2425326) 'stacks': 'all', (TaskRunner pid=2425326) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2425326) 'profile_continuous_steps': False, (TaskRunner pid=2425326) 'save_path': 'outputs/profile', (TaskRunner pid=2425326) 'steps': None, (TaskRunner pid=2425326) 'tool': None}, (TaskRunner pid=2425326) 'max_model_len': 10240, (TaskRunner pid=2425326) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_material_rlsd_tr_Qwen3_8B', (TaskRunner pid=2425326) 'include_dashboard': False, (TaskRunner pid=2425326) 'num_cpus': None, (TaskRunner pid=2425326) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2425326) 'TASK': 'datasets/sciknoweval/material', (TaskRunner pid=2425326) 'USER': 'root'}}}, (TaskRunner pid=2425326) 'timeline_json_file': None}, (TaskRunner pid=2425326) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2425326) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2425326) 'name': 'custom_reward_manager', (TaskRunner pid=2425326) 'path': None}, (TaskRunner pid=2425326) 'name': 'naive', (TaskRunner pid=2425326) 'source': 'register'}, (TaskRunner pid=2425326) 'reward_model': {'enable': False, (TaskRunner pid=2425326) 'enable_resource_pool': False, (TaskRunner pid=2425326) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2425326) 'launch_reward_fn_async': False, (TaskRunner pid=2425326) 'max_length': None, (TaskRunner pid=2425326) 'micro_batch_size': None, (TaskRunner pid=2425326) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2425326) 'model': {'external_lib': None, (TaskRunner pid=2425326) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2425326) 'forward_prefetch': False, (TaskRunner pid=2425326) 'fsdp_size': -1, (TaskRunner pid=2425326) 'param_offload': False, (TaskRunner pid=2425326) 'reshard_after_forward': True, (TaskRunner pid=2425326) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2425326) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2425326) 'override_config': {}, (TaskRunner pid=2425326) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2425326) 'trust_remote_code': False, (TaskRunner pid=2425326) 'use_fused_kernels': False, (TaskRunner pid=2425326) 'use_remove_padding': False, (TaskRunner pid=2425326) 'use_shm': False}, (TaskRunner pid=2425326) 'n_gpus_per_node': 8, (TaskRunner pid=2425326) 'nnodes': 0, (TaskRunner pid=2425326) 'num_workers': 1, (TaskRunner pid=2425326) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2425326) 'all_ranks': False, (TaskRunner pid=2425326) 'enable': False, (TaskRunner pid=2425326) 'ranks': [], (TaskRunner pid=2425326) 'save_path': 'outputs/profile', (TaskRunner pid=2425326) 'tool': None, (TaskRunner pid=2425326) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2425326) 'analysis': True, (TaskRunner pid=2425326) 'contents': [], (TaskRunner pid=2425326) 'discrete': False, (TaskRunner pid=2425326) 'level': 'level0'}, (TaskRunner pid=2425326) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2425326) 'discrete': False}, (TaskRunner pid=2425326) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2425326) 'step_end': None, (TaskRunner pid=2425326) 'step_start': 0}, (TaskRunner pid=2425326) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2425326) 'stack_depth': 32, (TaskRunner pid=2425326) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2425326) 'reward_loop_class_name': None, (TaskRunner pid=2425326) 'reward_loop_module_path': None, (TaskRunner pid=2425326) 'reward_loop_source': 'register', (TaskRunner pid=2425326) 'reward_manager': 'naive', (TaskRunner pid=2425326) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2425326) 'cudagraph_capture_sizes': None, (TaskRunner pid=2425326) 'data_parallel_size': 1, (TaskRunner pid=2425326) 'disable_log_stats': True, (TaskRunner pid=2425326) 'dtype': 'bfloat16', (TaskRunner pid=2425326) 'enable_chunked_prefill': True, (TaskRunner pid=2425326) 'enable_prefix_caching': True, (TaskRunner pid=2425326) 'enforce_eager': True, (TaskRunner pid=2425326) 'engine_kwargs': {}, (TaskRunner pid=2425326) 'expert_parallel_size': 1, (TaskRunner pid=2425326) 'free_cache_engine': True, (TaskRunner pid=2425326) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2425326) 'limit_images': None, (TaskRunner pid=2425326) 'load_format': 'auto', (TaskRunner pid=2425326) 'max_model_len': None, (TaskRunner pid=2425326) 'max_num_batched_tokens': 8192, (TaskRunner pid=2425326) 'max_num_seqs': 1024, (TaskRunner pid=2425326) 'name': '???', (TaskRunner pid=2425326) 'prompt_length': 2048, (TaskRunner pid=2425326) 'response_length': 2048, (TaskRunner pid=2425326) 'skip_tokenizer_init': False, (TaskRunner pid=2425326) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2425326) 'sandbox_fusion': { (TaskRunner pid=2425326) 'max_concurrent': 64, (TaskRunner pid=2425326) 'memory_limit_mb': 1024, (TaskRunner pid=2425326) 'url': None}, (TaskRunner pid=2425326) 'strategy': 'fsdp', (TaskRunner pid=2425326) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2425326) 'use_dynamic_bsz': False, (TaskRunner pid=2425326) 'use_reward_loop': False}, (TaskRunner pid=2425326) 'trainer': (TaskRunner pid=2425326) {'balance_batch': True, (TaskRunner pid=2425326) 'critic_warmup': 0, (TaskRunner pid=2425326) 'default_hdfs_dir': None, (TaskRunner pid=2425326) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2425326) 'del_local_ckpt_after_load': False, (TaskRunner pid=2425326) 'device': 'cuda', (TaskRunner pid=2425326) 'esi_redundant_time': 0, (TaskRunner pid=2425326) 'experiment_name': 'qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2425326) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2425326) 'log_val_generations': 0, (TaskRunner pid=2425326) 'logger': ['console', 'wandb'], (TaskRunner pid=2425326) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2425326) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2425326) 'n_gpus_per_node': 8, (TaskRunner pid=2425326) 'nnodes': 1, (TaskRunner pid=2425326) 'project_name': 'SDPO-root', (TaskRunner pid=2425326) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2425326) 'resume_from_path': None, (TaskRunner pid=2425326) 'resume_mode': 'auto', (TaskRunner pid=2425326) 'rollout_data_dir': None, (TaskRunner pid=2425326) 'save_freq': 10, (TaskRunner pid=2425326) 'test_freq': 10, (TaskRunner pid=2425326) 'total_epochs': 30, (TaskRunner pid=2425326) 'total_training_steps': 100, (TaskRunner pid=2425326) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2425326) 'val_before_train': False, (TaskRunner pid=2425326) 'val_only': False, (TaskRunner pid=2425326) 'validation_data_dir': None}, (TaskRunner pid=2425326) 'transfer_queue': {'enable': False}, (TaskRunner pid=2425326) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/material', (TaskRunner pid=2425326) 'dir': '/users/root/SDPO', (TaskRunner pid=2425326) 'log_dir': '/users/root/output', (TaskRunner pid=2425326) 'task': 'datasets/sciknoweval/material'}} (TaskRunner pid=2425326) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2425326) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2425326) use_critic=need_critic(config), (TaskRunner pid=2425326) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2425326) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2425326) Using dataset class: RLHFDataset (TaskRunner pid=2425326) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2425326) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2425326) dataset len: 841 (TaskRunner pid=2425326) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2425326) WARNING:2026-07-03 11:55:15,772:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2425326) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/841 [00:00 (TaskRunner pid=2425326) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2425326) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2425326) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2425326) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2425326) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2425326) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 94/94 [00:00<00:00, 149.00 examples/s] (TaskRunner pid=2425326) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2425326) self.use_critic = need_critic(self.config) (WorkerDict pid=2425705) [W703 11:55:24.030244946 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56351 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2425703) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2425703) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2425703) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2425707) [W703 11:55:25.743412095 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56351 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2425703) Model config after override: Qwen3Config { (WorkerDict pid=2425703) "architectures": [ (WorkerDict pid=2425703) "Qwen3ForCausalLM" (WorkerDict pid=2425703) ], (WorkerDict pid=2425703) "attention_bias": false, (WorkerDict pid=2425703) "attention_dropout": 0.0, (WorkerDict pid=2425703) "dtype": "bfloat16", (WorkerDict pid=2425703) "eos_token_id": 151645, (WorkerDict pid=2425703) "head_dim": 128, (WorkerDict pid=2425703) "hidden_act": "silu", (WorkerDict pid=2425703) "hidden_size": 4096, (WorkerDict pid=2425703) "initializer_range": 0.02, (WorkerDict pid=2425703) "intermediate_size": 12288, (WorkerDict pid=2425703) "layer_types": [ (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention" (WorkerDict pid=2425703) ], (WorkerDict pid=2425703) "max_position_embeddings": 40960, (WorkerDict pid=2425703) "max_window_layers": 36, (WorkerDict pid=2425703) "model_type": "qwen3", (WorkerDict pid=2425703) "num_attention_heads": 32, (WorkerDict pid=2425703) "num_hidden_layers": 36, (WorkerDict pid=2425703) "num_key_value_heads": 8, (WorkerDict pid=2425703) "pad_token_id": 151643, (WorkerDict pid=2425703) "rms_norm_eps": 1e-06, (WorkerDict pid=2425703) "rope_scaling": null, (WorkerDict pid=2425703) "rope_theta": 1000000, (WorkerDict pid=2425703) "sliding_window": null, (WorkerDict pid=2425703) "tie_word_embeddings": false, (WorkerDict pid=2425703) "transformers_version": "4.57.1", (WorkerDict pid=2425703) "use_cache": true, (WorkerDict pid=2425703) "use_sliding_window": false, (WorkerDict pid=2425703) "vocab_size": 151936 (WorkerDict pid=2425703) } (WorkerDict pid=2425703) (WorkerDict pid=2425703) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2425703) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2425704) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2425703) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2425704) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2425704) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2425709) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.96s/it] [repeated 15x across cluster] (WorkerDict pid=2425709) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.25s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.64s/it] [repeated 7x across cluster] (WorkerDict pid=2425703) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2425703) Actor use_remove_padding=True (WorkerDict pid=2425703) Actor use_fused_kernels=False (WorkerDict pid=2425703) Actor use_prefix_grouper=False (WorkerDict pid=2425709) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2425709) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2425704) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2425704) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2425704) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2425703) reference model: Qwen/Qwen3-8B (WorkerDict pid=2425703) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2425703) warnings.warn( (WorkerDict pid=2425703) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2425703) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2425703) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2425703) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2425703) Model config after override: Qwen3Config { (WorkerDict pid=2425703) "architectures": [ (WorkerDict pid=2425703) "Qwen3ForCausalLM" (WorkerDict pid=2425703) ], (WorkerDict pid=2425703) "attention_bias": false, (WorkerDict pid=2425703) "attention_dropout": 0.0, (WorkerDict pid=2425703) "dtype": "bfloat16", (WorkerDict pid=2425703) "eos_token_id": 151645, (WorkerDict pid=2425703) "head_dim": 128, (WorkerDict pid=2425703) "hidden_act": "silu", (WorkerDict pid=2425703) "hidden_size": 4096, (WorkerDict pid=2425703) "initializer_range": 0.02, (WorkerDict pid=2425703) "intermediate_size": 12288, (WorkerDict pid=2425703) "layer_types": [ (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention", (WorkerDict pid=2425703) "full_attention" (WorkerDict pid=2425703) ], (WorkerDict pid=2425703) "max_position_embeddings": 40960, (WorkerDict pid=2425703) "max_window_layers": 36, (WorkerDict pid=2425703) "model_type": "qwen3", (WorkerDict pid=2425703) "num_attention_heads": 32, (WorkerDict pid=2425703) "num_hidden_layers": 36, (WorkerDict pid=2425703) "num_key_value_heads": 8, (WorkerDict pid=2425703) "pad_token_id": 151643, (WorkerDict pid=2425703) "rms_norm_eps": 1e-06, (WorkerDict pid=2425703) "rope_scaling": null, (WorkerDict pid=2425703) "rope_theta": 1000000, (WorkerDict pid=2425703) "sliding_window": null, (WorkerDict pid=2425703) "tie_word_embeddings": false, (WorkerDict pid=2425703) "transformers_version": "4.57.1", (WorkerDict pid=2425703) "use_cache": true, (WorkerDict pid=2425703) "use_sliding_window": false, (WorkerDict pid=2425703) "vocab_size": 151936 (WorkerDict pid=2425703) } (WorkerDict pid=2425703) (WorkerDict pid=2425707) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (WorkerDict pid=2425707) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2425703) Ref use_remove_padding=True (WorkerDict pid=2425703) Ref use_fused_kernels=False (WorkerDict pid=2425703) Ref use_prefix_grouper=False (WorkerDict pid=2425708) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2425708) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2425326) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2425326) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2425708) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.91s/it] [repeated 15x across cluster] (WorkerDict pid=2425708) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.22s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.61s/it] [repeated 7x across cluster] (vLLMHttpServer pid=2426631) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2426631) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2426631) ['serve', (vLLMHttpServer pid=2426631) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2426631) '--dtype', (vLLMHttpServer pid=2426631) 'bfloat16', (vLLMHttpServer pid=2426631) '--load_format', (vLLMHttpServer pid=2426631) 'dummy', (vLLMHttpServer pid=2426631) '--trust_remote_code', (vLLMHttpServer pid=2426631) '--max_model_len', (vLLMHttpServer pid=2426631) '40960', (vLLMHttpServer pid=2426631) '--max_num_seqs', (vLLMHttpServer pid=2426631) '1024', (vLLMHttpServer pid=2426631) '--enable_chunked_prefill', (vLLMHttpServer pid=2426631) '--max_num_batched_tokens', (vLLMHttpServer pid=2426631) '10240', (vLLMHttpServer pid=2426631) '--enable_prefix_caching', (vLLMHttpServer pid=2426631) '--enable_sleep_mode', (vLLMHttpServer pid=2426631) '--logprobs_mode', (vLLMHttpServer pid=2426631) 'processed_logprobs', (vLLMHttpServer pid=2426631) '--disable_custom_all_reduce', (vLLMHttpServer pid=2426631) '--gpu_memory_utilization', (vLLMHttpServer pid=2426631) '0.8', (vLLMHttpServer pid=2426631) '--disable_log_stats', (vLLMHttpServer pid=2426631) '--tensor_parallel_size', (vLLMHttpServer pid=2426631) '2', (vLLMHttpServer pid=2426631) '--seed', (vLLMHttpServer pid=2426631) '0', (vLLMHttpServer pid=2426631) '--override_generation_config', (vLLMHttpServer pid=2426631) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2426631) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2426631) '--hf_overrides', (vLLMHttpServer pid=2426631) '{}', (vLLMHttpServer pid=2426631) '--scheduling_policy', (vLLMHttpServer pid=2426631) 'fcfs'] (vLLMHttpServer pid=2426631) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2426631) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2426631) WARNING 07-03 11:56:40 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2425709) WARNING 07-03 11:56:49 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2426634) WARNING 07-03 11:56:41 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2425705) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2425708) [rank5]:W0703 11:56:59.632000 2425708 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2425708) [rank5]:W0703 11:56:59.632000 2425708 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2425708) [rank5]:W0703 11:56:59.632000 2425708 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2425708) [rank5]:W0703 11:56:59.632000 2425708 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2425708) [rank5]:W0703 11:56:59.632000 2425708 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2425708) [rank5]:W0703 11:56:59.632000 2425708 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/aj/faj75csb7ofozv5in77ssqc762na7cdfwj3wkvfkkmz6awioro7y/.2425707.138841746503360.tmp' (vLLMHttpServer pid=2426634) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2426634) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2426634) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2426634) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2425705) 2026-07-03 11:57:02,581 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2425705) 2026-07-03 11:57:02,597 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2425703) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2425326) (TaskRunner pid=2425326) A (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) user (TaskRunner pid=2425326) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2425326) (TaskRunner pid=2425326) Lattice Parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) a = 7.82 Å (TaskRunner pid=2425326) b = 7.82 Å (TaskRunner pid=2425326) c = 17.88 Å (TaskRunner pid=2425326) α = 90.00º (TaskRunner pid=2425326) β = 90.00º (TaskRunner pid=2425326) γ = 90.00º (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) A: 1025.67 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) B: 1094.31 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) C: 1200.45 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) D: 1150.78 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) Please reason step by step. (TaskRunner pid=2425326) assistant (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) [response] (TaskRunner pid=2425326) The given crystal structure has lattice parameters a = 7.82 Å, b = 7.82 Å, c = 17.88 Å, and all angles (α, β, γ) are 90.00º, indicating a cubic crystal system. The volume $ V $ of a unit cell in a cubic system is calculated using the formula: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) V = a \times b \times c (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Substituting the given values: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) V = 7.82 \times 7.82 \times 17.88 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Then multiply by 17.88: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) 61.1524 \times 17.88 = 1094.31 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Thus, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) B (TaskRunner pid=2425326) (TaskRunner pid=2425326) [ground_truth] B (TaskRunner pid=2425326) [score] 1.0 (TaskRunner pid=2425326) [acc] 1.0 (TaskRunner pid=2425326) [pred] B (TaskRunner pid=2425326) [incorrect_format] 1 (TaskRunner pid=2425326) [feedback] (TaskRunner pid=2425326) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2425326) step:10 - global_seqlen/min:21584 - global_seqlen/max:29632 - global_seqlen/minmax_diff:8048 - global_seqlen/balanced_min:25339 - global_seqlen/balanced_max:25344 - global_seqlen/mean:25342.25 - actor/entropy:0.17639316618442535 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5623742341995239 - training/rollout_probs_diff_mean:0.00297564291395247 - training/rollout_probs_diff_std:0.010022003203630447 - training/rollout_actor_probs_pearson_corr:0.9988273978233337 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0002245903015137 - rollout_corr/rollout_is_ratio_fraction_high:3.9999198634177446e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9999599317088723e-05 - rollout_corr/rollout_is_max:3.417344331741333 - rollout_corr/rollout_is_min:0.29307252168655396 - rollout_corr/rollout_is_std:0.0296165831387043 - rollout_corr/rollout_is_eff_sample_size:0.9991238647521595 - rollout_corr/rollout_is_seq_mean:1.0002385377883911 - rollout_corr/rollout_is_seq_std:0.0017602697480469942 - rollout_corr/rollout_is_seq_max:1.0098330974578857 - rollout_corr/rollout_is_seq_min:0.99287348985672 - rollout_corr/rollout_is_seq_max_deviation:0.009833097457885742 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1970365992747247) - rollout_corr/training_log_ppl:np.float64(0.17606113059446216) - rollout_corr/kl:np.float64(0.00041335755928439966) - rollout_corr/k3_kl:np.float64(0.0006021688509321166) - rollout_corr/rollout_ppl:np.float64(1.1965299416333437) - rollout_corr/rollout_log_ppl:np.float64(0.17564777103689266) - rollout_corr/log_ppl_diff:np.float64(0.0004133595575694926) - rollout_corr/log_ppl_abs_diff:np.float64(0.001399658045556862) - rollout_corr/log_ppl_diff_max:np.float64(0.007768571376800537) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074193477630615234) - rollout_corr/ppl_ratio:np.float64(1.000415229005739) - rollout_corr/chi2_token:np.float64(0.0016666329465806484) - rollout_corr/chi2_seq:np.float64(0.5307988335844129) - actor/pg_loss:np.float64(0.0035515406634658575) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041831631688182824) - actor/ppo_kl:np.float64(0.00018670088022787468) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(93607.4896354326) - self_distillation/token_reweight_w_std:np.float64(1211600.9292068654) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003265906125307) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0656057171872817) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.3258971497416496) - perf/mfu/actor:np.float64(0.057390654797647184) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(194.34308242797852) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6555851063829787) - val-aux/sciknoweval/reward/std@16:np.float64(0.1378521080587476) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7133085106382979) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.09345261532381192) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5977872340425531) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14093939606080857) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6554574468085107) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1372207073818318) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.746095744680851) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0453184714249063) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5367872340425532) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12224983940234527) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6742553191489362) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11224171051652175) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.760627659574468) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.017753508240277174) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4841702127659574) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08932642643393333) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6877234042553192) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08319623031067264) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7653617021276596) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0048366150101044865) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4493510638297872) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.051336435350715415) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6949574468085107) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05558888914624856) - val-aux/sciknoweval/score/mean@16:np.float64(0.6555851063829787) - val-aux/sciknoweval/score/std@16:np.float64(0.1378521080587476) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7133085106382979) - val-aux/sciknoweval/score/best@2/std:np.float64(0.09345261532381192) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5977872340425531) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14093939606080857) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6554574468085107) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1372207073818318) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.746095744680851) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0453184714249063) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5367872340425532) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12224983940234527) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6742553191489362) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.11224171051652175) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.760627659574468) - val-aux/sciknoweval/score/best@8/std:np.float64(0.017753508240277174) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4841702127659574) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08932642643393333) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6877234042553192) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08319623031067264) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7653617021276596) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0048366150101044865) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4493510638297872) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.051336435350715415) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6949574468085107) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05558888914624856) - val-core/sciknoweval/acc/mean@16:np.float64(0.6555851063829787) - val-aux/sciknoweval/acc/std@16:np.float64(0.1378521080587476) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7133085106382979) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.09345261532381192) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5977872340425531) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14093939606080857) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6554574468085107) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1372207073818318) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.746095744680851) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0453184714249063) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5367872340425532) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12224983940234527) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6742553191489362) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.11224171051652175) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.760627659574468) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.017753508240277174) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4841702127659574) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08932642643393333) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6877234042553192) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08319623031067264) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7653617021276596) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0048366150101044865) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4493510638297872) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.051336435350715415) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6949574468085107) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05558888914624856) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006057378835976124 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006057378835976124 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:390.6328125 - response_length/max:880.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:390.6328125 - response_length_non_aborted/max:880.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.3125 - prompt_length/max:483.0 - prompt_length/min:267.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010654516518115997 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4434602744877338) - timing_s/agent_loop/generate_sequences/max:np.float64(7.469076165929437) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.506298832835455) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.469076165929437) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:880 - timing_s/gen:14.317543858662248 - timing_s/reward:0.0659487321972847 - timing_s/old_log_prob:3.2587016820907593 - timing_s/adv:0.5505860913544893 - timing_s/update_actor:22.788952961564064 - timing_s/step:41.06847350113094 - timing_s/testing:41.18692493438721 - timing_s/save_checkpoint:13.125189991667867 - timing_s/stop_profile:0.0001685861498117447 - timing_per_token_ms/adv:0.0027157518144328608 - timing_per_token_ms/update_actor:0.11240592765817983 - timing_per_token_ms/gen:0.14317257513511977 - perf/total_num_tokens:202738 - perf/time_per_step:41.06847350113094 - perf/throughput:617.0730937759868 (TaskRunner pid=2425326) Training Progress: 10%|█ | 10/100 [08:11<1:27:13, 58.15s/it] (TaskRunner pid=2425326) step:11 - global_seqlen/min:23133 - global_seqlen/max:28243 - global_seqlen/minmax_diff:5110 - global_seqlen/balanced_min:25525 - global_seqlen/balanced_max:25529 - global_seqlen/mean:25526.75 - actor/entropy:0.13923069834709167 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3595540523529053 - training/rollout_probs_diff_mean:0.002506059827283025 - training/rollout_probs_diff_std:0.009211650118231773 - training/rollout_actor_probs_pearson_corr:0.998887836933136 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0000337362289429 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6297050714492798 - rollout_corr/rollout_is_min:0.5049138069152832 - rollout_corr/rollout_is_std:0.026764316484332085 - rollout_corr/rollout_is_eff_sample_size:0.9992841841384783 - rollout_corr/rollout_is_seq_mean:1.0000133514404297 - rollout_corr/rollout_is_seq_std:0.0015445039607584476 - rollout_corr/rollout_is_seq_max:1.0065466165542603 - rollout_corr/rollout_is_seq_min:0.9930241703987122 - rollout_corr/rollout_is_seq_max_deviation:0.006975829601287842 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1522930269129574) - rollout_corr/training_log_ppl:np.float64(0.13917134414077736) - rollout_corr/kl:np.float64(0.0005147107860317135) - rollout_corr/k3_kl:np.float64(0.0004735012372378833) - rollout_corr/rollout_ppl:np.float64(1.1516877599060535) - rollout_corr/rollout_log_ppl:np.float64(0.1386566325527383) - rollout_corr/log_ppl_diff:np.float64(0.0005147115880390629) - rollout_corr/log_ppl_abs_diff:np.float64(0.001257772310054861) - rollout_corr/log_ppl_diff_max:np.float64(0.007037058472633362) - rollout_corr/log_ppl_diff_min:np.float64(-0.005003094673156738) - rollout_corr/ppl_ratio:np.float64(1.0005161934532225) - rollout_corr/chi2_token:np.float64(0.0008689719252288342) - rollout_corr/chi2_seq:np.float64(0.1595524800941348) - actor/pg_loss:np.float64(0.0011308909161016345) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022216506886252319) - actor/ppo_kl:np.float64(0.00015927736775145362) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(16985.92218018137) - self_distillation/token_reweight_w_std:np.float64(351130.6301018601) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002653282135725) - self_distillation/token_reweight_w_clip_frac:np.float64(0.021051821066066623) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.25365722738206387) - perf/mfu/actor:np.float64(0.05824596645962646) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.24867153167725) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000572855758946389 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000572855758946389 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:407.4296875 - response_length/max:690.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:407.4296875 - response_length_non_aborted/max:690.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.28125 - prompt_length/max:481.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001808851957321167 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7019501961767673) - timing_s/agent_loop/generate_sequences/max:np.float64(7.185342380777001) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.885585967407678) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.185342380777001) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:690 - timing_s/gen:13.46023834683001 - timing_s/reward:0.05591420643031597 - timing_s/old_log_prob:3.209773236885667 - timing_s/adv:0.5637547224760056 - timing_s/update_actor:22.877223005518317 - timing_s/step:40.27534000016749 - timing_s/stop_profile:0.00011360272765159607 - timing_per_token_ms/adv:0.0027606076100365577 - timing_per_token_ms/update_actor:0.11202573283672186 - timing_per_token_ms/gen:0.12905062555684466 - perf/total_num_tokens:204214 - perf/time_per_step:40.27534000016749 - perf/throughput:633.8059467628043 (TaskRunner pid=2425326) Training Progress: 11%|█ | 11/100 [08:51<1:18:10, 52.70s/it] (TaskRunner pid=2425326) step:12 - global_seqlen/min:20942 - global_seqlen/max:26854 - global_seqlen/minmax_diff:5912 - global_seqlen/balanced_min:24478 - global_seqlen/balanced_max:24482 - global_seqlen/mean:24480.5 - actor/entropy:0.16560876369476318 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35140830278396606 - training/rollout_probs_diff_mean:0.002876439830288291 - training/rollout_probs_diff_std:0.009866539388895035 - training/rollout_actor_probs_pearson_corr:0.9988297820091248 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.000069260597229 - rollout_corr/rollout_is_ratio_fraction_high:1.034211709338706e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.102635309915058e-05 - rollout_corr/rollout_is_max:4.401683330535889 - rollout_corr/rollout_is_min:0.27226293087005615 - rollout_corr/rollout_is_std:0.028915980830788612 - rollout_corr/rollout_is_eff_sample_size:0.9991645645790151 - rollout_corr/rollout_is_seq_mean:1.0001089572906494 - rollout_corr/rollout_is_seq_std:0.0018035214161500335 - rollout_corr/rollout_is_seq_max:1.0091018676757812 - rollout_corr/rollout_is_seq_min:0.9952834248542786 - rollout_corr/rollout_is_seq_max_deviation:0.00910186767578125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.176113523542881) - rollout_corr/training_log_ppl:np.float64(0.15972186841099756) - rollout_corr/kl:np.float64(0.0005557158914193394) - rollout_corr/k3_kl:np.float64(0.0005383371243823376) - rollout_corr/rollout_ppl:np.float64(1.1754575315862894) - rollout_corr/rollout_log_ppl:np.float64(0.159166151235695) - rollout_corr/log_ppl_diff:np.float64(0.0005557171753025614) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015308187503251247) - rollout_corr/log_ppl_diff_max:np.float64(0.006714761257171631) - rollout_corr/log_ppl_diff_min:np.float64(-0.005140185356140137) - rollout_corr/ppl_ratio:np.float64(1.000557643827051) - rollout_corr/chi2_token:np.float64(0.001142165856435895) - rollout_corr/chi2_seq:np.float64(0.8536760269198567) - actor/pg_loss:np.float64(0.00240803649649024) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033394563706679037) - actor/ppo_kl:np.float64(0.0002117141521171817) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(61640.22243928304) - self_distillation/token_reweight_w_std:np.float64(870763.4397258498) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999482547864318) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04829584649996832) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.23177143931388855) - perf/mfu/actor:np.float64(0.05500504927244522) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.91437911987305) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031685661524534225 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031685661524534225 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:377.703125 - response_length/max:856.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:377.703125 - response_length_non_aborted/max:856.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:387.3125 - prompt_length/max:482.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.083561599254608e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.091922763735056) - timing_s/agent_loop/generate_sequences/max:np.float64(7.5729265213012695) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.351463138547842) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.5729265213012695) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:838 - timing_s/gen:15.129444852471352 - timing_s/reward:0.06270519271492958 - timing_s/old_log_prob:3.0209708996117115 - timing_s/adv:0.5570559594780207 - timing_s/update_actor:23.19953622855246 - timing_s/step:42.055938908830285 - timing_s/stop_profile:0.00011912547051906586 - timing_per_token_ms/adv:0.0028443861414085736 - timing_per_token_ms/update_actor:0.11845926466244797 - timing_per_token_ms/gen:0.1564704924137607 - perf/total_num_tokens:195844 - perf/time_per_step:42.055938908830285 - perf/throughput:582.0937692788009 (TaskRunner pid=2425326) Training Progress: 12%|█▏ | 12/100 [09:33<1:12:33, 49.47s/it] (TaskRunner pid=2425326) step:13 - global_seqlen/min:22843 - global_seqlen/max:27080 - global_seqlen/minmax_diff:4237 - global_seqlen/balanced_min:24516 - global_seqlen/balanced_max:24518 - global_seqlen/mean:24517.25 - actor/entropy:0.15799753367900848 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.999995231628418 - training/rollout_probs_diff_mean:0.0029094803612679243 - training/rollout_probs_diff_std:0.010655753314495087 - training/rollout_actor_probs_pearson_corr:0.9985682368278503 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.625 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.625 - rollout_corr/rollout_is_mean:0.9997681975364685 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2301019271253608e-05 - rollout_corr/rollout_is_max:1.7753705978393555 - rollout_corr/rollout_is_min:5.715029374186997e-07 - rollout_corr/rollout_is_std:0.028576970100402832 - rollout_corr/rollout_is_eff_sample_size:0.9991836065722173 - rollout_corr/rollout_is_seq_mean:0.9997411370277405 - rollout_corr/rollout_is_seq_std:0.0016131839947775006 - rollout_corr/rollout_is_seq_max:1.0061041116714478 - rollout_corr/rollout_is_seq_min:0.9933863282203674 - rollout_corr/rollout_is_seq_max_deviation:0.006613671779632568 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1687511443160474) - rollout_corr/training_log_ppl:np.float64(0.15300021482107695) - rollout_corr/kl:np.float64(0.0008388460865909053) - rollout_corr/k3_kl:np.float64(0.0006182161795891261) - rollout_corr/rollout_ppl:np.float64(1.1677398411557078) - rollout_corr/rollout_log_ppl:np.float64(0.15216136802337132) - rollout_corr/log_ppl_diff:np.float64(0.000838846797705628) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016679515683790669) - rollout_corr/log_ppl_diff_max:np.float64(0.02418047934770584) - rollout_corr/log_ppl_diff_min:np.float64(-0.004306040704250336) - rollout_corr/ppl_ratio:np.float64(1.0008421933744103) - rollout_corr/chi2_token:np.float64(0.000714912312105298) - rollout_corr/chi2_seq:np.float64(0.22370657115243375) - actor/pg_loss:np.float64(0.0012121170293539762) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016096123908937443) - actor/ppo_kl:np.float64(-4.754266964823728e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(26895.081849052105) - self_distillation/token_reweight_w_std:np.float64(552510.5327741585) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.99996522301808) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02378630399471149) - self_distillation/empty_target_batch:np.float64(0.375) - actor/grad_norm:np.float64(0.14329994656145573) - perf/mfu/actor:np.float64(0.055844523004072354) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.88736724853516) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004566133487969637 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.004566133487969637 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:350.3203125 - response_length/max:628.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:350.3203125 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:415.84375 - prompt_length/max:493.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.695758879184723e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7831363286823034) - timing_s/agent_loop/generate_sequences/max:np.float64(6.392013613134623) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.23555718951684) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.392013613134623) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:438 - timing_s/agent_loop/slowest/response_length:628 - timing_s/gen:13.628588801249862 - timing_s/reward:0.05980396829545498 - timing_s/old_log_prob:3.022281412035227 - timing_s/adv:0.5457603670656681 - timing_s/update_actor:22.67483279109001 - timing_s/step:40.02329970896244 - timing_s/stop_profile:0.00012593157589435577 - timing_per_token_ms/adv:0.0027825325386496656 - timing_per_token_ms/update_actor:0.11560652597196877 - timing_per_token_ms/gen:0.15196570996688144 - perf/total_num_tokens:196138 - perf/time_per_step:40.02329970896244 - perf/throughput:612.5744298516656 (TaskRunner pid=2425326) Training Progress: 13%|█▎ | 13/100 [10:13<1:07:36, 46.63s/it] (TaskRunner pid=2425326) step:14 - global_seqlen/min:19037 - global_seqlen/max:25125 - global_seqlen/minmax_diff:6088 - global_seqlen/balanced_min:22643 - global_seqlen/balanced_max:22645 - global_seqlen/mean:22644.25 - actor/entropy:0.14876748621463776 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4311178922653198 - training/rollout_probs_diff_mean:0.0028640730306506157 - training/rollout_probs_diff_std:0.010166647844016552 - training/rollout_actor_probs_pearson_corr:0.9985819458961487 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.71875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999682307243347 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2039778994221706e-05 - rollout_corr/rollout_is_max:1.7173131704330444 - rollout_corr/rollout_is_min:0.015287053771317005 - rollout_corr/rollout_is_std:0.02836867794394493 - rollout_corr/rollout_is_eff_sample_size:0.9991958652393542 - rollout_corr/rollout_is_seq_mean:0.9999401569366455 - rollout_corr/rollout_is_seq_std:0.0017750123515725136 - rollout_corr/rollout_is_seq_max:1.008059024810791 - rollout_corr/rollout_is_seq_min:0.9954343438148499 - rollout_corr/rollout_is_seq_max_deviation:0.008059024810791016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.156547600403428) - rollout_corr/training_log_ppl:np.float64(0.14360159163334174) - rollout_corr/kl:np.float64(0.0006690974439980124) - rollout_corr/k3_kl:np.float64(0.0005123109068421172) - rollout_corr/rollout_ppl:np.float64(1.1557744019664824) - rollout_corr/rollout_log_ppl:np.float64(0.14293249235197436) - rollout_corr/log_ppl_diff:np.float64(0.0006690992813673802) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015188633187790401) - rollout_corr/log_ppl_diff_max:np.float64(0.01147855818271637) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071881115436553955) - rollout_corr/ppl_ratio:np.float64(1.0006714367773384) - rollout_corr/chi2_token:np.float64(0.0006734333001077175) - rollout_corr/chi2_seq:np.float64(0.2402285693679005) - actor/pg_loss:np.float64(0.0010356203420087695) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002713244098231371) - actor/ppo_kl:np.float64(0.00016327230558488282) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_w_mean:np.float64(94791.48625119124) - self_distillation/token_reweight_w_std:np.float64(1204151.7757004234) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003668137360364) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02809872478246689) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.19146711006760597) - perf/mfu/actor:np.float64(0.05182206348917821) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.85970306396484) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017532928613945842 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017532928613945842 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:324.4453125 - response_length/max:755.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.4453125 - response_length_non_aborted/max:755.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.1875 - prompt_length/max:519.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.522400796413422e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8827556539326906) - timing_s/agent_loop/generate_sequences/max:np.float64(6.701478620991111) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6339415991824353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.701478620991111) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:755 - timing_s/gen:13.754035361111164 - timing_s/reward:0.06031708978116512 - timing_s/old_log_prob:3.024625912308693 - timing_s/adv:0.5282524079084396 - timing_s/update_actor:22.739469228312373 - timing_s/step:40.193049723282456 - timing_s/stop_profile:0.00011224672198295593 - timing_per_token_ms/adv:0.0029160405395875313 - timing_per_token_ms/update_actor:0.12552562586701024 - timing_per_token_ms/gen:0.1655955520372651 - perf/total_num_tokens:181154 - perf/time_per_step:40.193049723282456 - perf/throughput:563.3872063926257 (TaskRunner pid=2425326) Training Progress: 14%|█▍ | 14/100 [10:53<1:04:03, 44.70s/it] (TaskRunner pid=2425326) step:15 - global_seqlen/min:20357 - global_seqlen/max:26318 - global_seqlen/minmax_diff:5961 - global_seqlen/balanced_min:23560 - global_seqlen/balanced_max:23564 - global_seqlen/mean:23562.5 - actor/entropy:0.14631496369838715 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016349673271179 - training/rollout_probs_diff_mean:0.002760756528005004 - training/rollout_probs_diff_std:0.009806844405829906 - training/rollout_actor_probs_pearson_corr:0.9987110495567322 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0000892877578735 - rollout_corr/rollout_is_ratio_fraction_high:2.2541815269505605e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:4.229223728179932 - rollout_corr/rollout_is_min:0.5123857259750366 - rollout_corr/rollout_is_std:0.029010646045207977 - rollout_corr/rollout_is_eff_sample_size:0.9991590901402961 - rollout_corr/rollout_is_seq_mean:1.000056266784668 - rollout_corr/rollout_is_seq_std:0.0018069344805553555 - rollout_corr/rollout_is_seq_max:1.0083919763565063 - rollout_corr/rollout_is_seq_min:0.9939211010932922 - rollout_corr/rollout_is_seq_max_deviation:0.008391976356506348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1575869643129408) - rollout_corr/training_log_ppl:np.float64(0.14444846322294325) - rollout_corr/kl:np.float64(0.0006515819939068024) - rollout_corr/k3_kl:np.float64(0.0005389036207645859) - rollout_corr/rollout_ppl:np.float64(1.1568320151418447) - rollout_corr/rollout_log_ppl:np.float64(0.14379688013286795) - rollout_corr/log_ppl_diff:np.float64(0.0006515830900752917) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015404976875288412) - rollout_corr/log_ppl_diff_max:np.float64(0.00785207748413086) - rollout_corr/log_ppl_diff_min:np.float64(-0.008581697940826416) - rollout_corr/ppl_ratio:np.float64(1.0006536967121065) - rollout_corr/chi2_token:np.float64(0.0009729461744427681) - rollout_corr/chi2_seq:np.float64(0.3421388922724873) - actor/pg_loss:np.float64(0.0018546859500929713) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004201540004942217) - actor/ppo_kl:np.float64(0.0002769002403564258) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(111500.54993374902) - self_distillation/token_reweight_w_std:np.float64(1809348.6934464541) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001659584231675) - self_distillation/token_reweight_w_clip_frac:np.float64(0.044835288790636696) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.28271641582250595) - perf/mfu/actor:np.float64(0.05300478413685146) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.90932846069336) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009932487737387419 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009932487737387419 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:346.578125 - response_length/max:695.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:346.578125 - response_length_non_aborted/max:695.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:389.75 - prompt_length/max:512.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001225881278514862 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7689055316150188) - timing_s/agent_loop/generate_sequences/max:np.float64(6.614128939807415) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.024020847733482) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.614128939807415) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:695 - timing_s/gen:13.781829055398703 - timing_s/reward:0.06101464480161667 - timing_s/old_log_prob:2.9474459923803806 - timing_s/adv:0.5514954477548599 - timing_s/update_actor:23.208873661234975 - timing_s/step:40.63824023865163 - timing_s/stop_profile:0.00012914463877677917 - timing_per_token_ms/adv:0.0029257052931292303 - timing_per_token_ms/update_actor:0.12312399820283806 - timing_per_token_ms/gen:0.1553337209255523 - perf/total_num_tokens:188500 - perf/time_per_step:40.63824023865163 - perf/throughput:579.8110317185772 (TaskRunner pid=2425326) Training Progress: 15%|█▌ | 15/100 [11:34<1:01:36, 43.49s/it] (TaskRunner pid=2425326) step:16 - global_seqlen/min:21151 - global_seqlen/max:26600 - global_seqlen/minmax_diff:5449 - global_seqlen/balanced_min:24328 - global_seqlen/balanced_max:24340 - global_seqlen/mean:24336.75 - actor/entropy:0.1754712462425232 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43991324305534363 - training/rollout_probs_diff_mean:0.002930275397375226 - training/rollout_probs_diff_std:0.009878823533654213 - training/rollout_actor_probs_pearson_corr:0.9988336563110352 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.000189185142517 - rollout_corr/rollout_is_ratio_fraction_high:2.2015277863829397e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1007638931914698e-05 - rollout_corr/rollout_is_max:3.340711832046509 - rollout_corr/rollout_is_min:0.12017349898815155 - rollout_corr/rollout_is_std:0.02910090796649456 - rollout_corr/rollout_is_eff_sample_size:0.9991542107998265 - rollout_corr/rollout_is_seq_mean:1.000267505645752 - rollout_corr/rollout_is_seq_std:0.0017066416330635548 - rollout_corr/rollout_is_seq_max:1.0081307888031006 - rollout_corr/rollout_is_seq_min:0.9957355260848999 - rollout_corr/rollout_is_seq_max_deviation:0.008130788803100586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1941090752370656) - rollout_corr/training_log_ppl:np.float64(0.17241633862431627) - rollout_corr/kl:np.float64(0.00014409434729500958) - rollout_corr/k3_kl:np.float64(0.0005055610327104887) - rollout_corr/rollout_ppl:np.float64(1.1939395442605019) - rollout_corr/rollout_log_ppl:np.float64(0.17227224242378725) - rollout_corr/log_ppl_diff:np.float64(0.00014409620052902028) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012714471304207109) - rollout_corr/log_ppl_diff_max:np.float64(0.007993444800376892) - rollout_corr/log_ppl_diff_min:np.float64(-0.004106059670448303) - rollout_corr/ppl_ratio:np.float64(1.0001455154269934) - rollout_corr/chi2_token:np.float64(0.0017365424428135157) - rollout_corr/chi2_seq:np.float64(0.48078759643249214) - actor/pg_loss:np.float64(0.0014060178073123097) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002084290190396132) - actor/ppo_kl:np.float64(-5.135476640383274e-05) - actor/pg_clipfrac_lower:np.float64(1.1388484381313901e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(46786.27347472962) - self_distillation/token_reweight_w_std:np.float64(763521.2723203925) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998286094050854) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03509276293334551) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.1977279968559742) - perf/mfu/actor:np.float64(0.05489509643176197) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.75335693359375) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001325044548138976 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001325044548138976 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:354.8671875 - response_length/max:863.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:354.8671875 - response_length_non_aborted/max:863.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:405.65625 - prompt_length/max:495.0 - prompt_length/min:268.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011777877807617188 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5929318312555552) - timing_s/agent_loop/generate_sequences/max:np.float64(7.403834100812674) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.146243010429316) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.403834100812674) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:452 - timing_s/agent_loop/slowest/response_length:863 - timing_s/gen:14.302462866529822 - timing_s/reward:0.06051816791296005 - timing_s/old_log_prob:3.0093026775866747 - timing_s/adv:0.5386755000799894 - timing_s/update_actor:23.026992704719305 - timing_s/step:41.02336362935603 - timing_s/stop_profile:0.00010964833199977875 - timing_per_token_ms/adv:0.002766780178536521 - timing_per_token_ms/update_actor:0.11827273929714992 - timing_per_token_ms/gen:0.15743635236036613 - perf/total_num_tokens:194694 - perf/time_per_step:41.02336362935603 - perf/throughput:593.2412129800297 (TaskRunner pid=2425326) Training Progress: 16%|█▌ | 16/100 [12:15<59:51, 42.76s/it] (TaskRunner pid=2425326) step:17 - global_seqlen/min:20062 - global_seqlen/max:26482 - global_seqlen/minmax_diff:6420 - global_seqlen/balanced_min:23020 - global_seqlen/balanced_max:23024 - global_seqlen/mean:23022.25 - actor/entropy:0.16389913856983185 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19498690962791443 - training/rollout_probs_diff_mean:0.00290701468475163 - training/rollout_probs_diff_std:0.009640076197683811 - training/rollout_actor_probs_pearson_corr:0.9987958073616028 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.55078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:0.9999469518661499 - rollout_corr/rollout_is_ratio_fraction_high:1.214034273289144e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.214034273289144e-05 - rollout_corr/rollout_is_max:2.1111412048339844 - rollout_corr/rollout_is_min:0.48321589827537537 - rollout_corr/rollout_is_std:0.028326625004410744 - rollout_corr/rollout_is_eff_sample_size:0.9991978885434327 - rollout_corr/rollout_is_seq_mean:0.9999359846115112 - rollout_corr/rollout_is_seq_std:0.0017924453131854534 - rollout_corr/rollout_is_seq_max:1.0070422887802124 - rollout_corr/rollout_is_seq_min:0.9944337010383606 - rollout_corr/rollout_is_seq_max_deviation:0.007042288780212402 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1799702616408467) - rollout_corr/training_log_ppl:np.float64(0.16088108645635657) - rollout_corr/kl:np.float64(0.0005829943140627236) - rollout_corr/k3_kl:np.float64(0.0004866512913253018) - rollout_corr/rollout_ppl:np.float64(1.1792868948541582) - rollout_corr/rollout_log_ppl:np.float64(0.16029809114843374) - rollout_corr/log_ppl_diff:np.float64(0.0005829953079228289) - rollout_corr/log_ppl_abs_diff:np.float64(0.001544620368804317) - rollout_corr/log_ppl_diff_max:np.float64(0.009654134511947632) - rollout_corr/log_ppl_diff_min:np.float64(-0.006283402442932129) - rollout_corr/ppl_ratio:np.float64(1.0005852151662111) - rollout_corr/chi2_token:np.float64(0.0007389525417238474) - rollout_corr/chi2_seq:np.float64(0.31677811266854405) - actor/pg_loss:np.float64(0.0011101557174697518) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.436478249976062e-05) - actor/ppo_kl:np.float64(0.00010106923795805134) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55078125) - self_distillation/token_reweight_w_mean:np.float64(50939.54964332422) - self_distillation/token_reweight_w_std:np.float64(836517.6223717686) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006192203145474) - self_distillation/token_reweight_w_clip_frac:np.float64(0.029739680663624313) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.17823269963264465) - perf/mfu/actor:np.float64(0.05247941166656937) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.76335906982422) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00027619278989732265 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00027619278989732265 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.7578125 - response_length/max:563.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:321.7578125 - response_length_non_aborted/max:563.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.6875 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.299255907535553e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1765111833810806) - timing_s/agent_loop/generate_sequences/max:np.float64(5.669759042561054) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.829668088175822) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.669759042561054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:555 - timing_s/gen:13.049837600439787 - timing_s/reward:0.06018427200615406 - timing_s/old_log_prob:3.0347642116248608 - timing_s/adv:0.5222656074911356 - timing_s/update_actor:22.92831893078983 - timing_s/step:39.68191853351891 - timing_s/stop_profile:0.00011275522410869598 - timing_per_token_ms/adv:0.0028356568509329866 - timing_per_token_ms/update_actor:0.12448999842972466 - timing_per_token_ms/gen:0.1584294961811313 - perf/total_num_tokens:184178 - perf/time_per_step:39.68191853351891 - perf/throughput:580.1697813716678 (TaskRunner pid=2425326) Training Progress: 17%|█▋ | 17/100 [12:55<57:53, 41.85s/it] (TaskRunner pid=2425326) step:18 - global_seqlen/min:20589 - global_seqlen/max:24439 - global_seqlen/minmax_diff:3850 - global_seqlen/balanced_min:22796 - global_seqlen/balanced_max:22808 - global_seqlen/mean:22804.25 - actor/entropy:0.14592979848384857 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43096956610679626 - training/rollout_probs_diff_mean:0.0027572556864470243 - training/rollout_probs_diff_std:0.009901202283799648 - training/rollout_actor_probs_pearson_corr:0.9987658262252808 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:1.000087857246399 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2085176422260702e-05 - rollout_corr/rollout_is_max:1.678128957748413 - rollout_corr/rollout_is_min:0.2909236252307892 - rollout_corr/rollout_is_std:0.028930407017469406 - rollout_corr/rollout_is_eff_sample_size:0.9991639695284223 - rollout_corr/rollout_is_seq_mean:1.0000859498977661 - rollout_corr/rollout_is_seq_std:0.0018274225294589996 - rollout_corr/rollout_is_seq_max:1.006705641746521 - rollout_corr/rollout_is_seq_min:0.9936803579330444 - rollout_corr/rollout_is_seq_max_deviation:0.006705641746520996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1651787175796926) - rollout_corr/training_log_ppl:np.float64(0.14963296141650062) - rollout_corr/kl:np.float64(0.0005520073682241389) - rollout_corr/k3_kl:np.float64(0.000504673919003551) - rollout_corr/rollout_ppl:np.float64(1.1645195069722831) - rollout_corr/rollout_log_ppl:np.float64(0.14908095342980232) - rollout_corr/log_ppl_diff:np.float64(0.0005520079866982996) - rollout_corr/log_ppl_abs_diff:np.float64(0.001438016741303727) - rollout_corr/log_ppl_diff_max:np.float64(0.008772194385528564) - rollout_corr/log_ppl_diff_min:np.float64(-0.004790559411048889) - rollout_corr/ppl_ratio:np.float64(1.0005540039855987) - rollout_corr/chi2_token:np.float64(0.0008889026939868927) - rollout_corr/chi2_seq:np.float64(0.19275876367464662) - actor/pg_loss:np.float64(0.0009842339204624295) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016771803802839713) - actor/ppo_kl:np.float64(0.00019636925287880302) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(41495.434180613374) - self_distillation/token_reweight_w_std:np.float64(678382.9856499105) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997132562566549) - self_distillation/token_reweight_w_clip_frac:np.float64(0.030070643435465172) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.1628562556579709) - perf/mfu/actor:np.float64(0.051830712186790084) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.74004364013672) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018233510199934244 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0018233510199934244 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:323.2265625 - response_length/max:825.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:323.2265625 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:389.40625 - prompt_length/max:492.0 - prompt_length/min:206.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016924738883972168 - timing_s/agent_loop/generate_sequences/min:np.float64(2.006256742402911) - timing_s/agent_loop/generate_sequences/max:np.float64(7.097431987524033) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.763510379641957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.097431987524033) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:14.250134736299515 - timing_s/reward:0.05953882820904255 - timing_s/old_log_prob:2.9710872787982225 - timing_s/adv:0.5348277427256107 - timing_s/update_actor:22.808317894116044 - timing_s/step:40.71119520626962 - timing_s/stop_profile:0.00011086836457252502 - timing_per_token_ms/adv:0.0029316231772893797 - timing_per_token_ms/update_actor:0.125022297894669 - timing_per_token_ms/gen:0.172215390910733 - perf/total_num_tokens:182434 - perf/time_per_step:40.71119520626962 - perf/throughput:560.1469051561545 (TaskRunner pid=2425326) Training Progress: 18%|█▊ | 18/100 [13:36<56:45, 41.53s/it] (TaskRunner pid=2425326) step:19 - global_seqlen/min:20728 - global_seqlen/max:27101 - global_seqlen/minmax_diff:6373 - global_seqlen/balanced_min:23735 - global_seqlen/balanced_max:23739 - global_seqlen/mean:23737.25 - actor/entropy:0.150180384516716 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22949278354644775 - training/rollout_probs_diff_mean:0.002771648345515132 - training/rollout_probs_diff_std:0.00962701253592968 - training/rollout_actor_probs_pearson_corr:0.9988031387329102 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.000072956085205 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6179542541503906 - rollout_corr/rollout_is_min:0.5627838969230652 - rollout_corr/rollout_is_std:0.028339248150587082 - rollout_corr/rollout_is_eff_sample_size:0.9991976505072335 - rollout_corr/rollout_is_seq_mean:1.000105381011963 - rollout_corr/rollout_is_seq_std:0.0016573466127738357 - rollout_corr/rollout_is_seq_max:1.006208896636963 - rollout_corr/rollout_is_seq_min:0.994246244430542 - rollout_corr/rollout_is_seq_max_deviation:0.006208896636962891 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1647716057486832) - rollout_corr/training_log_ppl:np.float64(0.15018300905649085) - rollout_corr/kl:np.float64(0.00027759616153133493) - rollout_corr/k3_kl:np.float64(0.00045629992244755613) - rollout_corr/rollout_ppl:np.float64(1.1644534692168236) - rollout_corr/rollout_log_ppl:np.float64(0.14990541196311824) - rollout_corr/log_ppl_diff:np.float64(0.00027759709337260574) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013563748070737347) - rollout_corr/log_ppl_diff_max:np.float64(0.006419584155082703) - rollout_corr/log_ppl_diff_min:np.float64(-0.0054564327001571655) - rollout_corr/ppl_ratio:np.float64(1.0002791315782815) - rollout_corr/chi2_token:np.float64(0.001294158399105072) - rollout_corr/chi2_seq:np.float64(0.5072512249462306) - actor/pg_loss:np.float64(0.0019059174228459597) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024170369306375505) - actor/ppo_kl:np.float64(-3.959948672971336e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(12786.887558560818) - self_distillation/token_reweight_w_std:np.float64(239622.63726934168) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005574417300522) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03681229500216432) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2085462026298046) - perf/mfu/actor:np.float64(0.05353441975860699) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.7970199584961) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0037669024895876646 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0037669024895876646 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:357.6328125 - response_length/max:672.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:357.6328125 - response_length_non_aborted/max:672.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.15625 - prompt_length/max:494.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015586428344249725 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1946402303874493) - timing_s/agent_loop/generate_sequences/max:np.float64(6.443620601668954) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.013251065494842) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.443620601668954) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:406 - timing_s/agent_loop/slowest/response_length:672 - timing_s/gen:13.341445690020919 - timing_s/reward:0.06141888350248337 - timing_s/old_log_prob:2.991618812084198 - timing_s/adv:0.5749543886631727 - timing_s/update_actor:23.15056392364204 - timing_s/step:40.20719136483967 - timing_s/stop_profile:0.00011139735579490662 - timing_per_token_ms/adv:0.0030277011272534346 - timing_per_token_ms/update_actor:0.12191051998252767 - timing_per_token_ms/gen:0.14572214966053826 - perf/total_num_tokens:189898 - perf/time_per_step:40.20719136483967 - perf/throughput:590.373244045037 (TaskRunner pid=2425326) Training Progress: 19%|█▉ | 19/100 [14:16<55:32, 41.15s/it] (TaskRunner pid=2425326) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2425326) validation generation end (TaskRunner pid=2425326) [prompt] system (TaskRunner pid=2425326) (TaskRunner pid=2425326) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2425326) (TaskRunner pid=2425326) A (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) user (TaskRunner pid=2425326) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2425326) (TaskRunner pid=2425326) Lattice Parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) a = 7.82 Å (TaskRunner pid=2425326) b = 7.82 Å (TaskRunner pid=2425326) c = 17.88 Å (TaskRunner pid=2425326) α = 90.00º (TaskRunner pid=2425326) β = 90.00º (TaskRunner pid=2425326) γ = 90.00º (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) A: 1025.67 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) B: 1094.31 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) C: 1200.45 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) D: 1150.78 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) Please reason step by step. (TaskRunner pid=2425326) assistant (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) [response] (TaskRunner pid=2425326) The given crystal structure is orthorhombic, as all angles (α, β, γ) are 90.00º. The volume $ V $ of a unit cell in an orthorhombic system is calculated using the formula: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) V = a \times b \times c (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Substituting the given lattice parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) V = 7.82 \, \text{Å} \times 7.82 \, \text{Å} \times 17.88 \, \text{Å} (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Then multiply by 17.88: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) 61.1524 \times 17.88 = 1094.31 \, \text{Å}^3 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Thus, the correct volume of the unit cell is 1094.31 ų. (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) B (TaskRunner pid=2425326) (TaskRunner pid=2425326) [ground_truth] B (TaskRunner pid=2425326) [score] 1.0 (TaskRunner pid=2425326) [acc] 1.0 (TaskRunner pid=2425326) [pred] B (TaskRunner pid=2425326) [incorrect_format] 1 (TaskRunner pid=2425326) [feedback] (TaskRunner pid=2425326) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2425326) step:20 - global_seqlen/min:20727 - global_seqlen/max:27737 - global_seqlen/minmax_diff:7010 - global_seqlen/balanced_min:23695 - global_seqlen/balanced_max:23702 - global_seqlen/mean:23699.875 - actor/entropy:0.1525348573923111 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31862038373947144 - training/rollout_probs_diff_mean:0.0029412272851914167 - training/rollout_probs_diff_std:0.010256594978272915 - training/rollout_actor_probs_pearson_corr:0.9986908435821533 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.999910295009613 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.3007811250863597e-05 - rollout_corr/rollout_is_max:1.9659780263900757 - rollout_corr/rollout_is_min:0.27226293087005615 - rollout_corr/rollout_is_std:0.029111146926879883 - rollout_corr/rollout_is_eff_sample_size:0.9991532587389499 - rollout_corr/rollout_is_seq_mean:0.999940037727356 - rollout_corr/rollout_is_seq_std:0.0016126699047163129 - rollout_corr/rollout_is_seq_max:1.0052599906921387 - rollout_corr/rollout_is_seq_min:0.9946086406707764 - rollout_corr/rollout_is_seq_max_deviation:0.005391359329223633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.162003489676863) - rollout_corr/training_log_ppl:np.float64(0.14800938246480655) - rollout_corr/kl:np.float64(0.0004912164907806549) - rollout_corr/k3_kl:np.float64(0.00046629576226564495) - rollout_corr/rollout_ppl:np.float64(1.1614080583676696) - rollout_corr/rollout_log_ppl:np.float64(0.14751816556963604) - rollout_corr/log_ppl_diff:np.float64(0.0004912168951705098) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013770487275905907) - rollout_corr/log_ppl_diff_max:np.float64(0.006052792072296143) - rollout_corr/log_ppl_diff_min:np.float64(-0.003998763859272003) - rollout_corr/ppl_ratio:np.float64(1.0004927921108902) - rollout_corr/chi2_token:np.float64(0.0008720688056200743) - rollout_corr/chi2_seq:np.float64(0.20135757699608803) - actor/pg_loss:np.float64(0.0011970872292295098) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019925688775401795) - actor/ppo_kl:np.float64(-1.4209182749524985e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(7654.9028110676445) - self_distillation/token_reweight_w_std:np.float64(124788.77614233922) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999270187690854) - self_distillation/token_reweight_w_clip_frac:np.float64(0.030343565143994056) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.14196937903761864) - perf/mfu/actor:np.float64(0.05396931844995495) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.8521385192871) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6888297872340425) - val-aux/sciknoweval/reward/std@16:np.float64(0.11079258764978504) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7310212765957446) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.06887196928165733) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6454787234042554) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.11678760945598075) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6891702127659574) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.11065702494895947) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7552659574468084) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.03606974918312295) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5989148936170212) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11207610098957337) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7035212765957447) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.08173885842932514) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7672234042553193) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.01835681121887613) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5511489361702129) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0984233069786272) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7089574468085107) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.050778492735805904) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7725531914893617) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.008178772725978603) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5082446808510638) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07216821985267445) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7104787234042553) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.03203893141992611) - val-aux/sciknoweval/score/mean@16:np.float64(0.6888297872340425) - val-aux/sciknoweval/score/std@16:np.float64(0.11079258764978504) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7310212765957446) - val-aux/sciknoweval/score/best@2/std:np.float64(0.06887196928165733) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6454787234042554) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.11678760945598075) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6891702127659574) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.11065702494895947) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7552659574468084) - val-aux/sciknoweval/score/best@4/std:np.float64(0.03606974918312295) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5989148936170212) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11207610098957337) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7035212765957447) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.08173885842932514) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7672234042553193) - val-aux/sciknoweval/score/best@8/std:np.float64(0.01835681121887613) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5511489361702129) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0984233069786272) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7089574468085107) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.050778492735805904) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7725531914893617) - val-aux/sciknoweval/score/best@16/std:np.float64(0.008178772725978603) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5082446808510638) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07216821985267445) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7104787234042553) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.03203893141992611) - val-core/sciknoweval/acc/mean@16:np.float64(0.6888297872340425) - val-aux/sciknoweval/acc/std@16:np.float64(0.11079258764978504) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7310212765957446) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.06887196928165733) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6454787234042554) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.11678760945598075) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6891702127659574) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.11065702494895947) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7552659574468084) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.03606974918312295) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5989148936170212) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11207610098957337) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7035212765957447) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.08173885842932514) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7672234042553193) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.01835681121887613) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5511489361702129) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0984233069786272) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7089574468085107) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.050778492735805904) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7725531914893617) - val-core/sciknoweval/acc/best@16/std:np.float64(0.008178772725978603) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5082446808510638) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07216821985267445) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7104787234042553) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.03203893141992611) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0047784349881112576 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0047784349881112576 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:339.55859375 - response_length/max:721.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.55859375 - response_length_non_aborted/max:721.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.0625 - prompt_length/max:492.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.912438690662384e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7952663730829954) - timing_s/agent_loop/generate_sequences/max:np.float64(6.682307660579681) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.0160775354816) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.682307660579681) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:479 - timing_s/agent_loop/slowest/response_length:721 - timing_s/gen:13.19336243160069 - timing_s/reward:0.05916779488325119 - timing_s/old_log_prob:3.0719532389193773 - timing_s/adv:0.5339687280356884 - timing_s/update_actor:22.6343802344054 - timing_s/step:39.58918163925409 - timing_s/testing:41.29565432853997 - timing_s/save_checkpoint:13.315809043124318 - timing_s/stop_profile:0.0001597180962562561 - timing_per_token_ms/adv:0.002816305613614462 - timing_per_token_ms/update_actor:0.11938027222931238 - timing_per_token_ms/gen:0.1517751956423285 - perf/total_num_tokens:189599 - perf/time_per_step:39.58918163925409 - perf/throughput:598.6452363668141 (TaskRunner pid=2425326) Training Progress: 20%|██ | 20/100 [15:50<1:16:07, 57.10s/it] (TaskRunner pid=2425326) step:21 - global_seqlen/min:20660 - global_seqlen/max:25901 - global_seqlen/minmax_diff:5241 - global_seqlen/balanced_min:23335 - global_seqlen/balanced_max:23343 - global_seqlen/mean:23339.75 - actor/entropy:0.1605978012084961 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772907614707947 - training/rollout_probs_diff_mean:0.0030818735249340534 - training/rollout_probs_diff_std:0.010238205082714558 - training/rollout_actor_probs_pearson_corr:0.9987055659294128 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0001020431518555 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.33062201004941e-05 - rollout_corr/rollout_is_max:1.8625247478485107 - rollout_corr/rollout_is_min:0.13879206776618958 - rollout_corr/rollout_is_std:0.029451090842485428 - rollout_corr/rollout_is_eff_sample_size:0.9991337418908153 - rollout_corr/rollout_is_seq_mean:1.000085711479187 - rollout_corr/rollout_is_seq_std:0.001832032692618668 - rollout_corr/rollout_is_seq_max:1.0050110816955566 - rollout_corr/rollout_is_seq_min:0.9940919280052185 - rollout_corr/rollout_is_seq_max_deviation:0.005908071994781494 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1757359504699707) - rollout_corr/training_log_ppl:np.float64(0.15899174506193958) - rollout_corr/kl:np.float64(0.0005214925452143859) - rollout_corr/k3_kl:np.float64(0.0005145951237324198) - rollout_corr/rollout_ppl:np.float64(1.175128533039242) - rollout_corr/rollout_log_ppl:np.float64(0.15847025081893662) - rollout_corr/log_ppl_diff:np.float64(0.0005214942430029623) - rollout_corr/log_ppl_abs_diff:np.float64(0.001540094926895108) - rollout_corr/log_ppl_diff_max:np.float64(0.01268022507429123) - rollout_corr/log_ppl_diff_min:np.float64(-0.004476115107536316) - rollout_corr/ppl_ratio:np.float64(1.0005237653385848) - rollout_corr/chi2_token:np.float64(0.000967380590736866) - rollout_corr/chi2_seq:np.float64(0.49231302621774375) - actor/pg_loss:np.float64(0.0008376834448426962) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014917148973836447) - actor/ppo_kl:np.float64(0.00014587882370187266) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(12218.91463384754) - self_distillation/token_reweight_w_std:np.float64(170807.23045986396) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000466012628749) - self_distillation/token_reweight_w_clip_frac:np.float64(0.025998536788392812) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.1443084478378296) - perf/mfu/actor:np.float64(0.05338657488221517) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(197.68404006958008) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002923473948612809 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002923473948612809 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.2109375 - response_length/max:627.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.2109375 - response_length_non_aborted/max:627.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.15625 - prompt_length/max:524.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015726126730442047 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1458057519048452) - timing_s/agent_loop/generate_sequences/max:np.float64(6.271111750975251) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.088328967132838) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.271111750975251) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:321 - timing_s/agent_loop/slowest/response_length:604 - timing_s/gen:12.701564520597458 - timing_s/reward:0.05455047823488712 - timing_s/old_log_prob:3.1934533920139074 - timing_s/adv:0.5404726825654507 - timing_s/update_actor:22.422428233549 - timing_s/step:39.01280113309622 - timing_s/stop_profile:0.00011268444359302521 - timing_per_token_ms/adv:0.0028945933577129716 - timing_per_token_ms/update_actor:0.12008712729115029 - timing_per_token_ms/gen:0.14801273126293446 - perf/total_num_tokens:186718 - perf/time_per_step:39.01280113309622 - perf/throughput:598.2587592306951 (TaskRunner pid=2425326) Training Progress: 21%|██ | 21/100 [16:29<1:08:03, 51.69s/it] (TaskRunner pid=2425326) step:22 - global_seqlen/min:17725 - global_seqlen/max:27066 - global_seqlen/minmax_diff:9341 - global_seqlen/balanced_min:22584 - global_seqlen/balanced_max:22586 - global_seqlen/mean:22584.875 - actor/entropy:0.15070903301239014 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8226406574249268 - training/rollout_probs_diff_mean:0.002898238133639097 - training/rollout_probs_diff_std:0.010579418390989304 - training/rollout_actor_probs_pearson_corr:0.9985734224319458 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.9998201727867126 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.3862077796366066e-05 - rollout_corr/rollout_is_max:1.9132329225540161 - rollout_corr/rollout_is_min:0.034405920654535294 - rollout_corr/rollout_is_std:0.02868003025650978 - rollout_corr/rollout_is_eff_sample_size:0.9991780129114268 - rollout_corr/rollout_is_seq_mean:0.9998583793640137 - rollout_corr/rollout_is_seq_std:0.0018176808953285217 - rollout_corr/rollout_is_seq_max:1.0082428455352783 - rollout_corr/rollout_is_seq_min:0.9950010180473328 - rollout_corr/rollout_is_seq_max_deviation:0.00824284553527832 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1599832442589104) - rollout_corr/training_log_ppl:np.float64(0.14644497753761243) - rollout_corr/kl:np.float64(0.000640428488739353) - rollout_corr/k3_kl:np.float64(0.0004915901672006839) - rollout_corr/rollout_ppl:np.float64(1.1592276142910123) - rollout_corr/rollout_log_ppl:np.float64(0.14580454777023988) - rollout_corr/log_ppl_diff:np.float64(0.0006404297673725523) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014647771386080422) - rollout_corr/log_ppl_diff_max:np.float64(0.006041541695594788) - rollout_corr/log_ppl_diff_min:np.float64(-0.005858853459358215) - rollout_corr/ppl_ratio:np.float64(1.0006423159502447) - rollout_corr/chi2_token:np.float64(0.0006925887428224087) - rollout_corr/chi2_seq:np.float64(0.03699774341657758) - actor/pg_loss:np.float64(0.0015359516255557537) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022245777108764742) - actor/ppo_kl:np.float64(2.9938944005714063e-05) - actor/pg_clipfrac_lower:np.float64(1.2361550943751354e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(12695.067828800296) - self_distillation/token_reweight_w_std:np.float64(233788.51185772906) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000253927661106) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02838251144567039) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.19917604327201843) - perf/mfu/actor:np.float64(0.05172575909415755) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(197.19318389892578) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006770864129066467 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006770864129066467 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:327.40234375 - response_length/max:617.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.40234375 - response_length_non_aborted/max:617.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:378.375 - prompt_length/max:509.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017551518976688385 - timing_s/agent_loop/generate_sequences/min:np.float64(1.250782249495387) - timing_s/agent_loop/generate_sequences/max:np.float64(5.923344105482101) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7955115795339225) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.923344105482101) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:615 - timing_s/gen:13.718998352065682 - timing_s/reward:0.06021725572645664 - timing_s/old_log_prob:3.0550778657197952 - timing_s/adv:0.5346001498401165 - timing_s/update_actor:22.610661946237087 - timing_s/step:40.06782971136272 - timing_s/stop_profile:0.00011607073247432709 - timing_per_token_ms/adv:0.002958839432585505 - timing_per_token_ms/update_actor:0.1251427224316998 - timing_per_token_ms/gen:0.16368189884943843 - perf/total_num_tokens:180679 - perf/time_per_step:40.06782971136272 - perf/throughput:563.6660423760167 (TaskRunner pid=2425326) Training Progress: 22%|██▏ | 22/100 [17:09<1:02:40, 48.22s/it] (TaskRunner pid=2425326) step:23 - global_seqlen/min:16066 - global_seqlen/max:27288 - global_seqlen/minmax_diff:11222 - global_seqlen/balanced_min:22127 - global_seqlen/balanced_max:22131 - global_seqlen/mean:22129.625 - actor/entropy:0.1543504297733307 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4668269157409668 - training/rollout_probs_diff_mean:0.0029893529135733843 - training/rollout_probs_diff_std:0.010312956757843494 - training/rollout_actor_probs_pearson_corr:0.9986623525619507 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.73828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0000005960464478 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.268826235900633e-05 - rollout_corr/rollout_is_max:1.6697542667388916 - rollout_corr/rollout_is_min:0.48396989703178406 - rollout_corr/rollout_is_std:0.029727065935730934 - rollout_corr/rollout_is_eff_sample_size:0.9991170817700432 - rollout_corr/rollout_is_seq_mean:0.9999738335609436 - rollout_corr/rollout_is_seq_std:0.0019343866733834147 - rollout_corr/rollout_is_seq_max:1.0052357912063599 - rollout_corr/rollout_is_seq_min:0.9925729632377625 - rollout_corr/rollout_is_seq_max_deviation:0.007427036762237549 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1613912233151495) - rollout_corr/training_log_ppl:np.float64(0.1482296362228226) - rollout_corr/kl:np.float64(0.0005620803344044134) - rollout_corr/k3_kl:np.float64(0.00048597054205856693) - rollout_corr/rollout_ppl:np.float64(1.1607246096245944) - rollout_corr/rollout_log_ppl:np.float64(0.14766755513846874) - rollout_corr/log_ppl_diff:np.float64(0.0005620810843538493) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015226107789203525) - rollout_corr/log_ppl_diff_max:np.float64(0.009226351976394653) - rollout_corr/log_ppl_diff_min:np.float64(-0.004461884498596191) - rollout_corr/ppl_ratio:np.float64(1.0005641290917993) - rollout_corr/chi2_token:np.float64(0.0008328419644385576) - rollout_corr/chi2_seq:np.float64(0.15795218478888273) - actor/pg_loss:np.float64(0.0010325053008273244) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012042900743836071) - actor/ppo_kl:np.float64(8.446178508592084e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_w_mean:np.float64(40982.084487203276) - self_distillation/token_reweight_w_std:np.float64(662032.4441022928) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005126828327775) - self_distillation/token_reweight_w_clip_frac:np.float64(0.025580635032383725) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.1630999743938446) - perf/mfu/actor:np.float64(0.050710404347214785) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(197.17858123779297) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002686739433556795 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002686739433556795 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:307.86328125 - response_length/max:607.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:307.86328125 - response_length_non_aborted/max:607.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.6875 - prompt_length/max:493.0 - prompt_length/min:257.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.814223110675812e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6565362866967916) - timing_s/agent_loop/generate_sequences/max:np.float64(5.703358609229326) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5760044023700175) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.703358609229326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:607 - timing_s/gen:13.013954853639007 - timing_s/reward:0.05813980847597122 - timing_s/old_log_prob:3.153484107926488 - timing_s/adv:0.5319961532950401 - timing_s/update_actor:22.73805571347475 - timing_s/step:39.58344977721572 - timing_s/stop_profile:2.896040678024292e-05 - timing_per_token_ms/adv:0.003004999820913369 - timing_per_token_ms/update_actor:0.1284367432427953 - timing_per_token_ms/gen:0.16512446999402391 - perf/total_num_tokens:177037 - perf/time_per_step:39.58344977721572 - perf/throughput:559.0625659094989 (TaskRunner pid=2425326) Training Progress: 23%|██▎ | 23/100 [17:49<58:33, 45.63s/it] (TaskRunner pid=2425326) step:24 - global_seqlen/min:19871 - global_seqlen/max:26117 - global_seqlen/minmax_diff:6246 - global_seqlen/balanced_min:22997 - global_seqlen/balanced_max:23000 - global_seqlen/mean:22999.125 - actor/entropy:0.13676773011684418 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22949346899986267 - training/rollout_probs_diff_mean:0.00260890438221395 - training/rollout_probs_diff_std:0.009457413107156754 - training/rollout_actor_probs_pearson_corr:0.9987369179725647 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.8515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.9999614357948303 - rollout_corr/rollout_is_ratio_fraction_high:1.1675559107970912e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.502667823340744e-05 - rollout_corr/rollout_is_max:2.3356175422668457 - rollout_corr/rollout_is_min:0.3921355903148651 - rollout_corr/rollout_is_std:0.027214836329221725 - rollout_corr/rollout_is_eff_sample_size:0.9992600198693956 - rollout_corr/rollout_is_seq_mean:0.999939501285553 - rollout_corr/rollout_is_seq_std:0.0015718950890004635 - rollout_corr/rollout_is_seq_max:1.0046483278274536 - rollout_corr/rollout_is_seq_min:0.9949087500572205 - rollout_corr/rollout_is_seq_max_deviation:0.005091249942779541 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1496808505617082) - rollout_corr/training_log_ppl:np.float64(0.13687511737953173) - rollout_corr/kl:np.float64(0.0004387259168705526) - rollout_corr/k3_kl:np.float64(0.00045451685279829235) - rollout_corr/rollout_ppl:np.float64(1.1491743493825197) - rollout_corr/rollout_log_ppl:np.float64(0.13643639061774593) - rollout_corr/log_ppl_diff:np.float64(0.0004387267617858015) - rollout_corr/log_ppl_abs_diff:np.float64(0.00128690085693961) - rollout_corr/log_ppl_diff_max:np.float64(0.005763456225395203) - rollout_corr/log_ppl_diff_min:np.float64(-0.004293650388717651) - rollout_corr/ppl_ratio:np.float64(1.0004402522463351) - rollout_corr/chi2_token:np.float64(0.0009429841302335262) - rollout_corr/chi2_seq:np.float64(0.39535231585614383) - actor/pg_loss:np.float64(0.0010843543568626046) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.718408753338736e-05) - actor/ppo_kl:np.float64(-7.393197986260702e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_w_mean:np.float64(22220.938996580895) - self_distillation/token_reweight_w_std:np.float64(409145.10946293676) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995599235408008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.022219942387891933) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.15542156994342804) - perf/mfu/actor:np.float64(0.052601399516352225) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(197.20500946044922) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.8515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001507606590166688 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.001507606590166688 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:334.56640625 - response_length/max:629.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.56640625 - response_length_non_aborted/max:629.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.15625 - prompt_length/max:477.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.4721026420593262e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.010031895712018) - timing_s/agent_loop/generate_sequences/max:np.float64(6.218087187036872) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.0288265772978775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.218087187036872) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:459 - timing_s/agent_loop/slowest/response_length:629 - timing_s/gen:13.156386936083436 - timing_s/reward:0.058851540088653564 - timing_s/old_log_prob:3.1598457992076874 - timing_s/adv:0.5560367330908775 - timing_s/update_actor:22.7517912928015 - timing_s/step:39.72860596328974 - timing_s/stop_profile:3.557652235031128e-05 - timing_per_token_ms/adv:0.003022053736233865 - timing_per_token_ms/update_actor:0.12365574392939677 - timing_per_token_ms/gen:0.15360817915075992 - perf/total_num_tokens:183993 - perf/time_per_step:39.72860596328974 - perf/throughput:578.9059153309278 (TaskRunner pid=2425326) Training Progress: 24%|██▍ | 24/100 [18:29<55:33, 43.87s/it] (TaskRunner pid=2425326) step:25 - global_seqlen/min:18321 - global_seqlen/max:25199 - global_seqlen/minmax_diff:6878 - global_seqlen/balanced_min:22271 - global_seqlen/balanced_max:22278 - global_seqlen/mean:22275.5 - actor/entropy:0.14500775933265686 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21093684434890747 - training/rollout_probs_diff_mean:0.0027230808045715094 - training/rollout_probs_diff_std:0.009582200087606907 - training/rollout_actor_probs_pearson_corr:0.9987487196922302 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9998183846473694 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.5558452762197703e-05 - rollout_corr/rollout_is_max:1.5865436792373657 - rollout_corr/rollout_is_min:0.32034292817115784 - rollout_corr/rollout_is_std:0.027499165385961533 - rollout_corr/rollout_is_eff_sample_size:0.9992440697114005 - rollout_corr/rollout_is_seq_mean:0.9998300671577454 - rollout_corr/rollout_is_seq_std:0.0015994401182979345 - rollout_corr/rollout_is_seq_max:1.005348563194275 - rollout_corr/rollout_is_seq_min:0.9952322840690613 - rollout_corr/rollout_is_seq_max_deviation:0.005348563194274902 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1642502681352198) - rollout_corr/training_log_ppl:np.float64(0.14770634190062992) - rollout_corr/kl:np.float64(0.0004993382486020437) - rollout_corr/k3_kl:np.float64(0.0004678560410127375) - rollout_corr/rollout_ppl:np.float64(1.163647570181638) - rollout_corr/rollout_log_ppl:np.float64(0.14720700281031895) - rollout_corr/log_ppl_diff:np.float64(0.0004993390903109685) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014981203275965527) - rollout_corr/log_ppl_diff_max:np.float64(0.007449612021446228) - rollout_corr/log_ppl_diff_min:np.float64(-0.004783540964126587) - rollout_corr/ppl_ratio:np.float64(1.0005014017224312) - rollout_corr/chi2_token:np.float64(0.0008534942753612995) - rollout_corr/chi2_seq:np.float64(0.24121627327986062) - actor/pg_loss:np.float64(0.0011007965076714754) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024173177780539845) - actor/ppo_kl:np.float64(-6.909645383146312e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(120953.98852239526) - self_distillation/token_reweight_w_std:np.float64(1569361.12798946) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008900640532374) - self_distillation/token_reweight_w_clip_frac:np.float64(0.031809512540348805) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.23915600031614304) - perf/mfu/actor:np.float64(0.05094053058519299) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(197.14062881469727) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005090923979878426 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005090923979878426 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:305.671875 - response_length/max:580.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.671875 - response_length_non_aborted/max:580.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.4375 - prompt_length/max:481.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.154575824737549e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1762989610433578) - timing_s/agent_loop/generate_sequences/max:np.float64(5.670870838686824) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.514597332527046) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.670870838686824) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:470 - timing_s/agent_loop/slowest/response_length:575 - timing_s/gen:12.807685051113367 - timing_s/reward:0.05949760600924492 - timing_s/old_log_prob:3.13385272026062 - timing_s/adv:0.550875648856163 - timing_s/update_actor:22.692448277026415 - timing_s/step:39.331494400277734 - timing_s/stop_profile:0.00011282041668891907 - timing_per_token_ms/adv:0.0030912642188512213 - timing_per_token_ms/update_actor:0.1273397245686203 - timing_per_token_ms/gen:0.16367230295856167 - perf/total_num_tokens:178204 - perf/time_per_step:39.331494400277734 - perf/throughput:566.352749613366 (TaskRunner pid=2425326) Training Progress: 25%|██▌ | 25/100 [19:08<53:08, 42.52s/it] (TaskRunner pid=2425326) step:26 - global_seqlen/min:21549 - global_seqlen/max:26767 - global_seqlen/minmax_diff:5218 - global_seqlen/balanced_min:23515 - global_seqlen/balanced_max:23518 - global_seqlen/mean:23516.625 - actor/entropy:0.13138793408870697 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.635709285736084 - training/rollout_probs_diff_mean:0.002796715358272195 - training/rollout_probs_diff_std:0.01032547652721405 - training/rollout_actor_probs_pearson_corr:0.9984015822410583 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9999243021011353 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.467847272986546e-05 - rollout_corr/rollout_is_max:1.7744193077087402 - rollout_corr/rollout_is_min:0.29728731513023376 - rollout_corr/rollout_is_std:0.028742309659719467 - rollout_corr/rollout_is_eff_sample_size:0.999174442522392 - rollout_corr/rollout_is_seq_mean:0.9999179840087891 - rollout_corr/rollout_is_seq_std:0.0018690851284191012 - rollout_corr/rollout_is_seq_max:1.0080863237380981 - rollout_corr/rollout_is_seq_min:0.9940353631973267 - rollout_corr/rollout_is_seq_max_deviation:0.008086323738098145 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1414004624821246) - rollout_corr/training_log_ppl:np.float64(0.13071745182242012) - rollout_corr/kl:np.float64(0.0004450980413324146) - rollout_corr/k3_kl:np.float64(0.0005135989932227858) - rollout_corr/rollout_ppl:np.float64(1.140865498688072) - rollout_corr/rollout_log_ppl:np.float64(0.13027235258050496) - rollout_corr/log_ppl_diff:np.float64(0.00044509924191515893) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015409416664624587) - rollout_corr/log_ppl_diff_max:np.float64(0.008853912353515625) - rollout_corr/log_ppl_diff_min:np.float64(-0.005702786147594452) - rollout_corr/ppl_ratio:np.float64(1.000447248108685) - rollout_corr/chi2_token:np.float64(0.0011436883360147476) - rollout_corr/chi2_seq:np.float64(0.6314138330053538) - actor/pg_loss:np.float64(0.0011609047651290894) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002807997716445243) - actor/ppo_kl:np.float64(-7.160629040470212e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(97124.35309204645) - self_distillation/token_reweight_w_std:np.float64(1541826.973082643) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000492023769766) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02323406282812357) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.14892314560711384) - perf/mfu/actor:np.float64(0.05380290758664994) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(197.13983535766602) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0030618200544267893 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0030618200544267893 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:337.92578125 - response_length/max:632.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:337.92578125 - response_length_non_aborted/max:632.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.96875 - prompt_length/max:491.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000143393874168396 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0222364645451307) - timing_s/agent_loop/generate_sequences/max:np.float64(6.113833889365196) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9542879644650384) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.113833889365196) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:632 - timing_s/gen:13.15871767140925 - timing_s/reward:0.059898363426327705 - timing_s/old_log_prob:3.165998887270689 - timing_s/adv:0.5314182974398136 - timing_s/update_actor:22.61581559292972 - timing_s/step:39.61940640769899 - timing_s/stop_profile:0.00011337921023368835 - timing_per_token_ms/adv:0.002824694750202323 - timing_per_token_ms/update_actor:0.12021184796356685 - timing_per_token_ms/gen:0.1521080774417604 - perf/total_num_tokens:188133 - perf/time_per_step:39.61940640769899 - perf/throughput:593.5632845682959 (TaskRunner pid=2425326) Training Progress: 26%|██▌ | 26/100 [19:48<51:23, 41.66s/it] (TaskRunner pid=2425326) step:27 - global_seqlen/min:19791 - global_seqlen/max:24654 - global_seqlen/minmax_diff:4863 - global_seqlen/balanced_min:22366 - global_seqlen/balanced_max:22371 - global_seqlen/mean:22369.25 - actor/entropy:0.10306167602539062 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31862038373947144 - training/rollout_probs_diff_mean:0.002335259458050132 - training/rollout_probs_diff_std:0.009463846683502197 - training/rollout_actor_probs_pearson_corr:0.9983808398246765 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.81640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.999907910823822 - rollout_corr/rollout_is_ratio_fraction_high:2.5938317776308395e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.11165452003479 - rollout_corr/rollout_is_min:0.589073896408081 - rollout_corr/rollout_is_std:0.026092225685715675 - rollout_corr/rollout_is_eff_sample_size:0.999319539879347 - rollout_corr/rollout_is_seq_mean:0.999942421913147 - rollout_corr/rollout_is_seq_std:0.001715592690743506 - rollout_corr/rollout_is_seq_max:1.0085691213607788 - rollout_corr/rollout_is_seq_min:0.9957650899887085 - rollout_corr/rollout_is_seq_max_deviation:0.008569121360778809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1074565374292433) - rollout_corr/training_log_ppl:np.float64(0.10104348854656564) - rollout_corr/kl:np.float64(0.0006335088800497601) - rollout_corr/k3_kl:np.float64(0.0005041445960429769) - rollout_corr/rollout_ppl:np.float64(1.1067317719571292) - rollout_corr/rollout_log_ppl:np.float64(0.10040997985197464) - rollout_corr/log_ppl_diff:np.float64(0.0006335086945910007) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015751363680465147) - rollout_corr/log_ppl_diff_max:np.float64(0.00826522707939148) - rollout_corr/log_ppl_diff_min:np.float64(-0.005233548581600189) - rollout_corr/ppl_ratio:np.float64(1.0006358111277223) - rollout_corr/chi2_token:np.float64(0.0006916692946106195) - rollout_corr/chi2_seq:np.float64(0.19716444029472768) - actor/pg_loss:np.float64(0.0012738716322928667) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003194190621798043) - actor/ppo_kl:np.float64(0.00022773302247180993) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_w_mean:np.float64(77265.22965213796) - self_distillation/token_reweight_w_std:np.float64(1212805.6968070571) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997106643859297) - self_distillation/token_reweight_w_clip_frac:np.float64(0.031144319364102557) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.21108173206448555) - perf/mfu/actor:np.float64(0.05175513238257462) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(198.09855270385742) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017638056306168437 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017638056306168437 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.1953125 - response_length/max:632.0 - response_length/min:132.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.1953125 - response_length_non_aborted/max:632.0 - response_length_non_aborted/min:132.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.84375 - prompt_length/max:494.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003122035413980484 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5815728176385164) - timing_s/agent_loop/generate_sequences/max:np.float64(5.887590665370226) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5390880910490523) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.887590665370226) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:632 - timing_s/gen:13.200261889025569 - timing_s/reward:0.05567370913922787 - timing_s/old_log_prob:3.3497544210404158 - timing_s/adv:0.5831103380769491 - timing_s/update_actor:22.255416925996542 - timing_s/step:39.55169596709311 - timing_s/stop_profile:0.00015717372298240662 - timing_per_token_ms/adv:0.0032584370177640575 - timing_per_token_ms/update_actor:0.12436389757142362 - timing_per_token_ms/gen:0.1711962997565114 - perf/total_num_tokens:178954 - perf/time_per_step:39.55169596709311 - perf/throughput:565.5699320355604 (TaskRunner pid=2425326) Training Progress: 27%|██▋ | 27/100 [20:28<50:03, 41.15s/it] (TaskRunner pid=2425326) step:28 - global_seqlen/min:18667 - global_seqlen/max:26411 - global_seqlen/minmax_diff:7744 - global_seqlen/balanced_min:22005 - global_seqlen/balanced_max:22010 - global_seqlen/mean:22008.25 - actor/entropy:0.1240900456905365 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311316251754761 - training/rollout_probs_diff_mean:0.0027813054621219635 - training/rollout_probs_diff_std:0.01046876423060894 - training/rollout_actor_probs_pearson_corr:0.9983556866645813 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.625 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.625 - rollout_corr/rollout_is_mean:0.9999943971633911 - rollout_corr/rollout_is_ratio_fraction_high:1.2547366168291774e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.0189464673167095e-05 - rollout_corr/rollout_is_max:3.121652126312256 - rollout_corr/rollout_is_min:0.003787645371630788 - rollout_corr/rollout_is_std:0.028779612854123116 - rollout_corr/rollout_is_eff_sample_size:0.9991721812892008 - rollout_corr/rollout_is_seq_mean:0.9999889135360718 - rollout_corr/rollout_is_seq_std:0.0019287803443148732 - rollout_corr/rollout_is_seq_max:1.0120155811309814 - rollout_corr/rollout_is_seq_min:0.9934726357460022 - rollout_corr/rollout_is_seq_max_deviation:0.012015581130981445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1382743194699287) - rollout_corr/training_log_ppl:np.float64(0.12592740579566453) - rollout_corr/kl:np.float64(0.0009248649318678304) - rollout_corr/k3_kl:np.float64(0.0007315356932053874) - rollout_corr/rollout_ppl:np.float64(1.1372004742734134) - rollout_corr/rollout_log_ppl:np.float64(0.12500254029873759) - rollout_corr/log_ppl_diff:np.float64(0.000924865496926941) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019478512549540028) - rollout_corr/log_ppl_diff_max:np.float64(0.01921156421303749) - rollout_corr/log_ppl_diff_min:np.float64(-0.006299503147602081) - rollout_corr/ppl_ratio:np.float64(1.0009291872847825) - rollout_corr/chi2_token:np.float64(0.0008862421382218599) - rollout_corr/chi2_seq:np.float64(0.23664455325342715) - actor/pg_loss:np.float64(0.0004893041914328933) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.747176980250515e-05) - actor/ppo_kl:np.float64(0.0004110109568475906) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(3621.874775248114) - self_distillation/token_reweight_w_std:np.float64(62977.78215696255) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000168199185282) - self_distillation/token_reweight_w_clip_frac:np.float64(0.007946641184389591) - self_distillation/empty_target_batch:np.float64(0.375) - actor/grad_norm:np.float64(0.12617958337068558) - perf/mfu/actor:np.float64(0.05071472735477538) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(198.16430282592773) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002507904777303338 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.002507904777303338 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:311.3203125 - response_length/max:647.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:311.3203125 - response_length_non_aborted/max:647.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:376.4375 - prompt_length/max:478.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013364292681217194 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8450600504875183) - timing_s/agent_loop/generate_sequences/max:np.float64(5.963680539280176) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6438437174947467) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.963680539280176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:287 - timing_s/agent_loop/slowest/response_length:619 - timing_s/gen:12.844708371907473 - timing_s/reward:0.06229527108371258 - timing_s/old_log_prob:3.1527174096554518 - timing_s/adv:0.5103049799799919 - timing_s/update_actor:22.569185046479106 - timing_s/step:39.2304096929729 - timing_s/stop_profile:0.00012934766709804535 - timing_per_token_ms/adv:0.002898373223563845 - timing_per_token_ms/update_actor:0.12818593621982158 - timing_per_token_ms/gen:0.16116726105934243 - perf/total_num_tokens:176066 - perf/time_per_step:39.2304096929729 - perf/throughput:560.9997492313266 (TaskRunner pid=2425326) Training Progress: 28%|██▊ | 28/100 [21:07<48:42, 40.59s/it] (TaskRunner pid=2425326) (TaskRunner pid=2425326) step:29 - global_seqlen/min:18052 - global_seqlen/max:25734 - global_seqlen/minmax_diff:7682 - global_seqlen/balanced_min:22625 - global_seqlen/balanced_max:22629 - global_seqlen/mean:22627.125 - actor/entropy:0.0921759158372879 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2549144923686981 - training/rollout_probs_diff_mean:0.0022082990035414696 - training/rollout_probs_diff_std:0.009376613423228264 - training/rollout_actor_probs_pearson_corr:0.9981832504272461 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:1.00003182888031 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.582611341495067e-05 - rollout_corr/rollout_is_max:1.6483628749847412 - rollout_corr/rollout_is_min:0.4751218259334564 - rollout_corr/rollout_is_std:0.02595939300954342 - rollout_corr/rollout_is_eff_sample_size:0.9993266827570269 - rollout_corr/rollout_is_seq_mean:1.0000574588775635 - rollout_corr/rollout_is_seq_std:0.0017196550033986568 - rollout_corr/rollout_is_seq_max:1.006497859954834 - rollout_corr/rollout_is_seq_min:0.9940721988677979 - rollout_corr/rollout_is_seq_max_deviation:0.006497859954833984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0933433049358428) - rollout_corr/training_log_ppl:np.float64(0.08807740310294321) - rollout_corr/kl:np.float64(0.00027317932717929594) - rollout_corr/k3_kl:np.float64(0.0003994494008381366) - rollout_corr/rollout_ppl:np.float64(1.0930359880439937) - rollout_corr/rollout_log_ppl:np.float64(0.08780422273412114) - rollout_corr/log_ppl_diff:np.float64(0.000273180368822068) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014128925540717319) - rollout_corr/log_ppl_diff_max:np.float64(0.00730479508638382) - rollout_corr/log_ppl_diff_min:np.float64(-0.0051046013832092285) - rollout_corr/ppl_ratio:np.float64(1.0002750388812274) - rollout_corr/chi2_token:np.float64(0.0010566439013928175) - rollout_corr/chi2_seq:np.float64(0.3449188000522554) - actor/pg_loss:np.float64(0.0009986156364902854) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014245773490983993) - actor/ppo_kl:np.float64(-1.469018872057859e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(23437.328467473853) - self_distillation/token_reweight_w_std:np.float64(427153.70655143255) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993212011177093) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02301981179334689) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.1457035057246685) - perf/mfu/actor:np.float64(0.05253759434802585) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(198.28145599365234) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004229026089888066 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004229026089888066 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:302.50390625 - response_length/max:568.0 - response_length/min:137.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:302.50390625 - response_length_non_aborted/max:568.0 - response_length_non_aborted/min:137.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:404.59375 - prompt_length/max:512.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014289654791355133 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7697436530143023) - timing_s/agent_loop/generate_sequences/max:np.float64(5.547379143536091) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5935217021615244) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.547379143536091) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:456 - timing_s/agent_loop/slowest/response_length:552 - timing_s/gen:12.421394154429436 - timing_s/reward:0.05764414742588997 - timing_s/old_log_prob:3.142216619104147 - timing_s/adv:0.5289132613688707 - timing_s/update_actor:22.47075405344367 - timing_s/step:38.70782240666449 - timing_s/stop_profile:0.00010413303971290588 - timing_per_token_ms/adv:0.002921898282309787 - timing_per_token_ms/update_actor:0.12413615325325064 - timing_per_token_ms/gen:0.1603981631749259 - perf/total_num_tokens:181017 - perf/time_per_step:38.70782240666449 - perf/throughput:584.5620753934272 (TaskRunner pid=2425326) Training Progress: 29%|██▉ | 29/100 [21:46<47:22, 40.04s/it] (TaskRunner pid=2425326) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2425326) validation generation end (TaskRunner pid=2425326) [prompt] system (TaskRunner pid=2425326) (TaskRunner pid=2425326) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2425326) (TaskRunner pid=2425326) A (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) user (TaskRunner pid=2425326) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2425326) (TaskRunner pid=2425326) Lattice Parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) a = 7.82 Å (TaskRunner pid=2425326) b = 7.82 Å (TaskRunner pid=2425326) c = 17.88 Å (TaskRunner pid=2425326) α = 90.00º (TaskRunner pid=2425326) β = 90.00º (TaskRunner pid=2425326) γ = 90.00º (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) A: 1025.67 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) B: 1094.31 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) C: 1200.45 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) D: 1150.78 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) Please reason step by step. (TaskRunner pid=2425326) assistant (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) [response] (TaskRunner pid=2425326) The given crystal structure has lattice parameters a = 7.82 Å, b = 7.82 Å, c = 17.88 Å, and all angles (α, β, γ) are 90.00º, indicating a orthorhombic unit cell. (TaskRunner pid=2425326) (TaskRunner pid=2425326) The volume $ V $ of a unit cell in an orthorhombic system is calculated using the formula: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) V = a \times b \times c (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Substituting the given values: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) V = 7.82 \times 7.82 \times 17.88 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Then multiply by 17.88: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) 61.1524 \times 17.88 = 1094.31 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Thus, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) B (TaskRunner pid=2425326) (TaskRunner pid=2425326) [ground_truth] B (TaskRunner pid=2425326) [score] 1.0 (TaskRunner pid=2425326) [acc] 1.0 (TaskRunner pid=2425326) [pred] B (TaskRunner pid=2425326) [incorrect_format] 1 (TaskRunner pid=2425326) [feedback] (TaskRunner pid=2425326) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2425326) step:30 - global_seqlen/min:20198 - global_seqlen/max:26628 - global_seqlen/minmax_diff:6430 - global_seqlen/balanced_min:23735 - global_seqlen/balanced_max:23742 - global_seqlen/mean:23739.625 - actor/entropy:0.10219845175743103 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6695281267166138 - training/rollout_probs_diff_mean:0.0023985919542610645 - training/rollout_probs_diff_std:0.010449547320604324 - training/rollout_actor_probs_pearson_corr:0.9980133175849915 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.53125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0001095533370972 - rollout_corr/rollout_is_ratio_fraction_high:1.1889334018633235e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.755733607453294e-05 - rollout_corr/rollout_is_max:5.522396087646484 - rollout_corr/rollout_is_min:0.21412654221057892 - rollout_corr/rollout_is_std:0.027441667392849922 - rollout_corr/rollout_is_eff_sample_size:0.9992476405980566 - rollout_corr/rollout_is_seq_mean:1.0000834465026855 - rollout_corr/rollout_is_seq_std:0.0018353002378717065 - rollout_corr/rollout_is_seq_max:1.0131018161773682 - rollout_corr/rollout_is_seq_min:0.994994580745697 - rollout_corr/rollout_is_seq_max_deviation:0.013101816177368164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.108945808839053) - rollout_corr/training_log_ppl:np.float64(0.10102521417138632) - rollout_corr/kl:np.float64(0.0006417104669356188) - rollout_corr/k3_kl:np.float64(0.0007572973181737552) - rollout_corr/rollout_ppl:np.float64(1.1082097035832703) - rollout_corr/rollout_log_ppl:np.float64(0.10038350410468411) - rollout_corr/log_ppl_diff:np.float64(0.0006417100667022169) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018683964008232579) - rollout_corr/log_ppl_diff_max:np.float64(0.014286383986473083) - rollout_corr/log_ppl_diff_min:np.float64(-0.00767090916633606) - rollout_corr/ppl_ratio:np.float64(1.0006450458895415) - rollout_corr/chi2_token:np.float64(0.0018558201845735312) - rollout_corr/chi2_seq:np.float64(0.8411497273482382) - actor/pg_loss:np.float64(0.0017660357989370823) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006848026164334442) - actor/ppo_kl:np.float64(0.00030027530577569905) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53125) - self_distillation/token_reweight_w_mean:np.float64(22148.840008513536) - self_distillation/token_reweight_w_std:np.float64(361685.4959547251) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007154732011259) - self_distillation/token_reweight_w_clip_frac:np.float64(0.029575713473604992) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.22181391529738903) - perf/mfu/actor:np.float64(0.05468114347361665) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(198.37141799926758) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6974734042553191) - val-aux/sciknoweval/reward/std@16:np.float64(0.10297707501391735) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7381914893617021) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.07451746654780546) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6570531914893617) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.0985696922009786) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6974893617021276) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.10250368415272754) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7661170212765956) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.04667697761275083) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6182446808510638) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.0872429003551105) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7044893617021277) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.07824496017358458) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7834042553191488) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.02663968507058267) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5810851063829787) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07146643446726411) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7063085106382979) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.05110215661515914) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7924893617021277) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.012076536974041402) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5502021276595744) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05096568380422959) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.704063829787234) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.03393239797882943) - val-aux/sciknoweval/score/mean@16:np.float64(0.6974734042553191) - val-aux/sciknoweval/score/std@16:np.float64(0.10297707501391735) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7381914893617021) - val-aux/sciknoweval/score/best@2/std:np.float64(0.07451746654780546) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6570531914893617) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.0985696922009786) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6974893617021276) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.10250368415272754) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7661170212765956) - val-aux/sciknoweval/score/best@4/std:np.float64(0.04667697761275083) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6182446808510638) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.0872429003551105) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7044893617021277) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.07824496017358458) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7834042553191488) - val-aux/sciknoweval/score/best@8/std:np.float64(0.02663968507058267) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5810851063829787) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07146643446726411) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7063085106382979) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.05110215661515914) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7924893617021277) - val-aux/sciknoweval/score/best@16/std:np.float64(0.012076536974041402) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5502021276595744) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05096568380422959) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.704063829787234) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.03393239797882943) - val-core/sciknoweval/acc/mean@16:np.float64(0.6974734042553191) - val-aux/sciknoweval/acc/std@16:np.float64(0.10297707501391735) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7381914893617021) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.07451746654780546) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6570531914893617) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.0985696922009786) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6974893617021276) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.10250368415272754) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7661170212765956) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.04667697761275083) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6182446808510638) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.0872429003551105) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7044893617021277) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.07824496017358458) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7834042553191488) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.02663968507058267) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5810851063829787) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07146643446726411) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7063085106382979) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.05110215661515914) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7924893617021277) - val-core/sciknoweval/acc/best@16/std:np.float64(0.012076536974041402) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5502021276595744) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05096568380422959) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.704063829787234) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.03393239797882943) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011651547392830253 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011651547392830253 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:328.55078125 - response_length/max:845.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:328.55078125 - response_length_non_aborted/max:845.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:413.3125 - prompt_length/max:482.0 - prompt_length/min:206.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011792033910751343 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0758803375065327) - timing_s/agent_loop/generate_sequences/max:np.float64(7.179892351850867) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.644049288399401) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.179892351850867) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:845 - timing_s/gen:13.753256499767303 - timing_s/reward:0.06654161959886551 - timing_s/old_log_prob:3.298671366646886 - timing_s/adv:0.5284189581871033 - timing_s/update_actor:22.570153038948774 - timing_s/step:40.306780176237226 - timing_s/testing:40.638839192688465 - timing_s/save_checkpoint:13.60086433403194 - timing_s/stop_profile:0.0001432802528142929 - timing_per_token_ms/adv:0.0027823678669476838 - timing_per_token_ms/update_actor:0.11884219442676945 - timing_per_token_ms/gen:0.1635170611916359 - perf/total_num_tokens:189917 - perf/time_per_step:40.306780176237226 - perf/throughput:588.9734902217679 (TaskRunner pid=2425326) Training Progress: 30%|███ | 30/100 [23:20<1:05:49, 56.41s/it] (TaskRunner pid=2425326) step:31 - global_seqlen/min:20844 - global_seqlen/max:25023 - global_seqlen/minmax_diff:4179 - global_seqlen/balanced_min:22623 - global_seqlen/balanced_max:22625 - global_seqlen/mean:22624.125 - actor/entropy:0.10231570899486542 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33947548270225525 - training/rollout_probs_diff_mean:0.002441213233396411 - training/rollout_probs_diff_std:0.00997840240597725 - training/rollout_actor_probs_pearson_corr:0.9981839060783386 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:0.9999410510063171 - rollout_corr/rollout_is_ratio_fraction_high:3.758974708034657e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2529914783954155e-05 - rollout_corr/rollout_is_max:3.3646881580352783 - rollout_corr/rollout_is_min:0.4018668532371521 - rollout_corr/rollout_is_std:0.028051747009158134 - rollout_corr/rollout_is_eff_sample_size:0.9992135991833438 - rollout_corr/rollout_is_seq_mean:0.9999500513076782 - rollout_corr/rollout_is_seq_std:0.00178877345751971 - rollout_corr/rollout_is_seq_max:1.0059913396835327 - rollout_corr/rollout_is_seq_min:0.9913996458053589 - rollout_corr/rollout_is_seq_max_deviation:0.008600354194641113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1037001409567893) - rollout_corr/training_log_ppl:np.float64(0.09721801686100662) - rollout_corr/kl:np.float64(0.0006632166562940967) - rollout_corr/k3_kl:np.float64(0.0005725183017233348) - rollout_corr/rollout_ppl:np.float64(1.102965306956321) - rollout_corr/rollout_log_ppl:np.float64(0.09655479896900943) - rollout_corr/log_ppl_diff:np.float64(0.0006632178919971921) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015754591222503223) - rollout_corr/log_ppl_diff_max:np.float64(0.010033667087554932) - rollout_corr/log_ppl_diff_min:np.float64(-0.007137298583984375) - rollout_corr/ppl_ratio:np.float64(1.0006655387114733) - rollout_corr/chi2_token:np.float64(0.0010873002465814352) - rollout_corr/chi2_seq:np.float64(0.2302088625729084) - actor/pg_loss:np.float64(0.001406671479344368) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002917049291681906) - actor/ppo_kl:np.float64(0.00019202986122568433) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(105553.12214704277) - self_distillation/token_reweight_w_std:np.float64(1536630.4769199342) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998775997664779) - self_distillation/token_reweight_w_clip_frac:np.float64(0.023525838390924037) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.20417143404483795) - perf/mfu/actor:np.float64(0.05114181998467157) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.02905082702637) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034034382551908493 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034034382551908493 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:311.75390625 - response_length/max:716.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:311.75390625 - response_length_non_aborted/max:716.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:395.25 - prompt_length/max:492.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.511170744895935e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7145044170320034) - timing_s/agent_loop/generate_sequences/max:np.float64(6.486898208037019) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6305026560658007) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.486898208037019) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:716 - timing_s/gen:13.34064906090498 - timing_s/reward:0.058349814265966415 - timing_s/old_log_prob:3.119881872087717 - timing_s/adv:0.585785735398531 - timing_s/update_actor:23.007402751594782 - timing_s/step:40.21905040740967 - timing_s/stop_profile:0.00012809596955776215 - timing_per_token_ms/adv:0.0032365104473572512 - timing_per_token_ms/update_actor:0.12711763853626815 - timing_per_token_ms/gen:0.1671572010788881 - perf/total_num_tokens:180993 - perf/time_per_step:40.21905040740967 - perf/throughput:562.5226048557302 (TaskRunner pid=2425326) Training Progress: 31%|███ | 31/100 [24:01<59:18, 51.57s/it] (TaskRunner pid=2425326) step:32 - global_seqlen/min:17465 - global_seqlen/max:26557 - global_seqlen/minmax_diff:9092 - global_seqlen/balanced_min:22365 - global_seqlen/balanced_max:22370 - global_seqlen/mean:22367.75 - actor/entropy:0.10624657571315765 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491804838180542 - training/rollout_probs_diff_mean:0.0025010062381625175 - training/rollout_probs_diff_std:0.010108744725584984 - training/rollout_actor_probs_pearson_corr:0.9982028007507324 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000066757202148 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8165538311004639 - rollout_corr/rollout_is_min:0.5453736186027527 - rollout_corr/rollout_is_std:0.02843792922794819 - rollout_corr/rollout_is_eff_sample_size:0.9991918186557794 - rollout_corr/rollout_is_seq_mean:0.9999681711196899 - rollout_corr/rollout_is_seq_std:0.0018278236966580153 - rollout_corr/rollout_is_seq_max:1.0054386854171753 - rollout_corr/rollout_is_seq_min:0.9935697317123413 - rollout_corr/rollout_is_seq_max_deviation:0.006430268287658691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1141844862140715) - rollout_corr/training_log_ppl:np.float64(0.10589008548413403) - rollout_corr/kl:np.float64(0.0006955666955104078) - rollout_corr/k3_kl:np.float64(0.0005526502317962922) - rollout_corr/rollout_ppl:np.float64(1.1134190135635436) - rollout_corr/rollout_log_ppl:np.float64(0.1051945186118246) - rollout_corr/log_ppl_diff:np.float64(0.0006955668723094277) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016634032290312462) - rollout_corr/log_ppl_diff_max:np.float64(0.006527021527290344) - rollout_corr/log_ppl_diff_min:np.float64(-0.0044723451137542725) - rollout_corr/ppl_ratio:np.float64(1.0006979582831264) - rollout_corr/chi2_token:np.float64(0.0009256161283701658) - rollout_corr/chi2_seq:np.float64(0.372445534914732) - actor/pg_loss:np.float64(0.0014663469046354294) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011217115661565913) - actor/ppo_kl:np.float64(0.00025597080171557707) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(12649.956588850822) - self_distillation/token_reweight_w_std:np.float64(114389.63001844386) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000564639922231) - self_distillation/token_reweight_w_clip_frac:np.float64(0.019761866453336552) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.17176248785108328) - perf/mfu/actor:np.float64(0.050811027766343705) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.6191520690918) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010969723807647824 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010969723807647824 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:302.6796875 - response_length/max:620.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:302.6796875 - response_length_non_aborted/max:620.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.301118552684784e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9586091451346874) - timing_s/agent_loop/generate_sequences/max:np.float64(5.785053916275501) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.603523639976629) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.785053916275501) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:620 - timing_s/gen:12.92170823737979 - timing_s/reward:0.057648180052638054 - timing_s/old_log_prob:2.969446098431945 - timing_s/adv:0.5218771118670702 - timing_s/update_actor:22.88319905474782 - timing_s/step:39.443169720470905 - timing_s/stop_profile:0.00013935379683971405 - timing_per_token_ms/adv:0.0029164595895154305 - timing_per_token_ms/update_actor:0.12788053701617183 - timing_per_token_ms/gen:0.16676184391218787 - perf/total_num_tokens:178942 - perf/time_per_step:39.443169720470905 - perf/throughput:567.0880448634734 (TaskRunner pid=2425326) Training Progress: 32%|███▏ | 32/100 [24:40<54:20, 47.95s/it] (TaskRunner pid=2425326) step:33 - global_seqlen/min:17970 - global_seqlen/max:23726 - global_seqlen/minmax_diff:5756 - global_seqlen/balanced_min:21881 - global_seqlen/balanced_max:21884 - global_seqlen/mean:21882.25 - actor/entropy:0.11248673498630524 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37974897027015686 - training/rollout_probs_diff_mean:0.002393264090642333 - training/rollout_probs_diff_std:0.00961360428482294 - training/rollout_actor_probs_pearson_corr:0.998481810092926 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9996835589408875 - rollout_corr/rollout_is_ratio_fraction_high:1.3081813449389301e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.924544216715731e-05 - rollout_corr/rollout_is_max:2.4000132083892822 - rollout_corr/rollout_is_min:0.3203427791595459 - rollout_corr/rollout_is_std:0.026572102680802345 - rollout_corr/rollout_is_eff_sample_size:0.9992938263651499 - rollout_corr/rollout_is_seq_mean:0.9996833801269531 - rollout_corr/rollout_is_seq_std:0.0017737209564074874 - rollout_corr/rollout_is_seq_max:1.0062614679336548 - rollout_corr/rollout_is_seq_min:0.9934943318367004 - rollout_corr/rollout_is_seq_max_deviation:0.0065056681632995605 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1257116086781025) - rollout_corr/training_log_ppl:np.float64(0.1153708925266983) - rollout_corr/kl:np.float64(0.0009000642832432959) - rollout_corr/k3_kl:np.float64(0.0006722159668157701) - rollout_corr/rollout_ppl:np.float64(1.1247029760852456) - rollout_corr/rollout_log_ppl:np.float64(0.1144708267529495) - rollout_corr/log_ppl_diff:np.float64(0.0009000657737487927) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018840501870727167) - rollout_corr/log_ppl_diff_max:np.float64(0.01474740356206894) - rollout_corr/log_ppl_diff_min:np.float64(-0.008892104029655457) - rollout_corr/ppl_ratio:np.float64(1.000903464620933) - rollout_corr/chi2_token:np.float64(0.0012851853389292955) - rollout_corr/chi2_seq:np.float64(0.42543286248110235) - actor/pg_loss:np.float64(0.0017157087568193674) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005343864218048111) - actor/ppo_kl:np.float64(0.00020211363660394) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(61808.9106729636) - self_distillation/token_reweight_w_std:np.float64(852764.9511296176) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005815126933157) - self_distillation/token_reweight_w_clip_frac:np.float64(0.039750390249537304) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.23709694296121597) - perf/mfu/actor:np.float64(0.049707760742796335) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.51437759399414) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004366055363789201 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004366055363789201 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:298.6015625 - response_length/max:644.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:298.6015625 - response_length_non_aborted/max:644.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.21875 - prompt_length/max:480.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.951554238796234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.24237797036767) - timing_s/agent_loop/generate_sequences/max:np.float64(5.898001089692116) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.218034182275005) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.898001089692116) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:644 - timing_s/gen:13.501362228766084 - timing_s/reward:0.056727657094597816 - timing_s/old_log_prob:2.932834295555949 - timing_s/adv:0.5199083350598812 - timing_s/update_actor:22.946831580251455 - timing_s/step:40.04101702012122 - timing_s/stop_profile:0.00011570751667022705 - timing_per_token_ms/adv:0.002969920455277001 - timing_per_token_ms/update_actor:0.13108130779656718 - timing_per_token_ms/gen:0.17662230486860736 - perf/total_num_tokens:175058 - perf/time_per_step:40.04101702012122 - perf/throughput:546.495859208667 (TaskRunner pid=2425326) Training Progress: 33%|███▎ | 33/100 [25:20<50:54, 45.59s/it] (TaskRunner pid=2425326) step:34 - global_seqlen/min:21088 - global_seqlen/max:25454 - global_seqlen/minmax_diff:4366 - global_seqlen/balanced_min:23192 - global_seqlen/balanced_max:23193 - global_seqlen/mean:23192.5 - actor/entropy:0.12696923315525055 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.317455530166626 - training/rollout_probs_diff_mean:0.0027052101213485003 - training/rollout_probs_diff_std:0.009988086298108101 - training/rollout_actor_probs_pearson_corr:0.9985281229019165 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.80859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:1.0001001358032227 - rollout_corr/rollout_is_ratio_fraction_high:2.5154700779239647e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.463893175125122 - rollout_corr/rollout_is_min:0.5469041466712952 - rollout_corr/rollout_is_std:0.02853209152817726 - rollout_corr/rollout_is_eff_sample_size:0.9991869389956493 - rollout_corr/rollout_is_seq_mean:1.0001479387283325 - rollout_corr/rollout_is_seq_std:0.0017741305055096745 - rollout_corr/rollout_is_seq_max:1.0061523914337158 - rollout_corr/rollout_is_seq_min:0.993314802646637 - rollout_corr/rollout_is_seq_max_deviation:0.006685197353363037 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1375689092092216) - rollout_corr/training_log_ppl:np.float64(0.1276737543521449) - rollout_corr/kl:np.float64(0.0003955075519228579) - rollout_corr/k3_kl:np.float64(0.0004891105344881908) - rollout_corr/rollout_ppl:np.float64(1.137121386360377) - rollout_corr/rollout_log_ppl:np.float64(0.1272782460291637) - rollout_corr/log_ppl_diff:np.float64(0.0003955083229811862) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014175720716593787) - rollout_corr/log_ppl_diff_max:np.float64(0.007905177772045135) - rollout_corr/log_ppl_diff_min:np.float64(-0.006664559245109558) - rollout_corr/ppl_ratio:np.float64(1.000397405354306) - rollout_corr/chi2_token:np.float64(0.0012063093017786741) - rollout_corr/chi2_seq:np.float64(0.39643845544196665) - actor/pg_loss:np.float64(0.0009817357640713453) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016826897945065866) - actor/ppo_kl:np.float64(0.0001194505678521196) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_w_mean:np.float64(43711.79978899378) - self_distillation/token_reweight_w_std:np.float64(769875.5125299292) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000270399497822) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02182861461187713) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.18393283523619175) - perf/mfu/actor:np.float64(0.05226619083988275) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.52132034301758) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007043316145427525 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007043316145427525 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:310.578125 - response_length/max:597.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:310.578125 - response_length_non_aborted/max:597.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:414.1875 - prompt_length/max:482.0 - prompt_length/min:271.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010688602924346924 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7658899314701557) - timing_s/agent_loop/generate_sequences/max:np.float64(5.676453439518809) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7275537907262333) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.676453439518809) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:482 - timing_s/agent_loop/slowest/response_length:549 - timing_s/gen:13.229636035859585 - timing_s/reward:0.06047343648970127 - timing_s/old_log_prob:2.993378309533 - timing_s/adv:0.5474918968975544 - timing_s/update_actor:22.999919889494777 - timing_s/step:39.9173941295594 - timing_s/stop_profile:0.00011380389332771301 - timing_per_token_ms/adv:0.0029508025056459762 - timing_per_token_ms/update_actor:0.12396205610377696 - timing_per_token_ms/gen:0.16639377214694853 - perf/total_num_tokens:185540 - perf/time_per_step:39.9173941295594 - perf/throughput:581.0123758260469 (TaskRunner pid=2425326) Training Progress: 34%|███▍ | 34/100 [26:00<48:17, 43.90s/it] (TaskRunner pid=2425326) step:35 - global_seqlen/min:19976 - global_seqlen/max:24551 - global_seqlen/minmax_diff:4575 - global_seqlen/balanced_min:21273 - global_seqlen/balanced_max:21278 - global_seqlen/mean:21276.125 - actor/entropy:0.1320156753063202 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36655372381210327 - training/rollout_probs_diff_mean:0.002745934296399355 - training/rollout_probs_diff_std:0.010113532654941082 - training/rollout_actor_probs_pearson_corr:0.9984952807426453 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.81640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9999025464057922 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.574679744313471e-05 - rollout_corr/rollout_is_max:1.6714837551116943 - rollout_corr/rollout_is_min:0.029101360589265823 - rollout_corr/rollout_is_std:0.02873193845152855 - rollout_corr/rollout_is_eff_sample_size:0.999174918572789 - rollout_corr/rollout_is_seq_mean:0.999925971031189 - rollout_corr/rollout_is_seq_std:0.0018350593745708466 - rollout_corr/rollout_is_seq_max:1.0064617395401 - rollout_corr/rollout_is_seq_min:0.9949533343315125 - rollout_corr/rollout_is_seq_max_deviation:0.006461739540100098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.140158954076469) - rollout_corr/training_log_ppl:np.float64(0.12918748372612754) - rollout_corr/kl:np.float64(0.0005167052121564097) - rollout_corr/k3_kl:np.float64(0.0005370462090237282) - rollout_corr/rollout_ppl:np.float64(1.1395544721744955) - rollout_corr/rollout_log_ppl:np.float64(0.1286707783001475) - rollout_corr/log_ppl_diff:np.float64(0.0005167054259800352) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016419052044511773) - rollout_corr/log_ppl_diff_max:np.float64(0.011951439082622528) - rollout_corr/log_ppl_diff_min:np.float64(-0.00703805685043335) - rollout_corr/ppl_ratio:np.float64(1.0005191075615585) - rollout_corr/chi2_token:np.float64(0.001026106532663107) - rollout_corr/chi2_seq:np.float64(0.30341047793626785) - actor/pg_loss:np.float64(0.0011919063981622458) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002517128054932982) - actor/ppo_kl:np.float64(-4.123877571160506e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_w_mean:np.float64(22245.829055046663) - self_distillation/token_reweight_w_std:np.float64(408636.0869032652) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995780531316996) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02098444675357314) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.1606460027396679) - perf/mfu/actor:np.float64(0.04926837037338072) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.49019145965576) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000839686137624085 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000839686137624085 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:280.28515625 - response_length/max:596.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:280.28515625 - response_length_non_aborted/max:596.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.59375 - prompt_length/max:459.0 - prompt_length/min:274.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015633739531040192 - timing_s/agent_loop/generate_sequences/min:np.float64(1.261065213009715) - timing_s/agent_loop/generate_sequences/max:np.float64(5.487262776121497) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3739940239684074) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.487262776121497) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:457 - timing_s/agent_loop/slowest/response_length:596 - timing_s/gen:12.814421102404594 - timing_s/reward:0.05708061158657074 - timing_s/old_log_prob:2.890442356467247 - timing_s/adv:0.5211395006626844 - timing_s/update_actor:22.403862675651908 - timing_s/step:38.77386414818466 - timing_s/stop_profile:2.8176233172416687e-05 - timing_per_token_ms/adv:0.003061762307884333 - timing_per_token_ms/update_actor:0.13162560543597523 - timing_per_token_ms/gen:0.17859073630934727 - perf/total_num_tokens:170209 - perf/time_per_step:38.77386414818466 - perf/throughput:548.7233595983009 (TaskRunner pid=2425326) Training Progress: 35%|███▌ | 35/100 [26:39<45:54, 42.38s/it] (TaskRunner pid=2425326) step:36 - global_seqlen/min:20282 - global_seqlen/max:25914 - global_seqlen/minmax_diff:5632 - global_seqlen/balanced_min:22874 - global_seqlen/balanced_max:22876 - global_seqlen/mean:22874.875 - actor/entropy:0.1515883058309555 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.317455530166626 - training/rollout_probs_diff_mean:0.002942807972431183 - training/rollout_probs_diff_std:0.010348225012421608 - training/rollout_actor_probs_pearson_corr:0.9985588192939758 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:1.0001130104064941 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.585033655166626 - rollout_corr/rollout_is_min:0.6092519164085388 - rollout_corr/rollout_is_std:0.029064016416668892 - rollout_corr/rollout_is_eff_sample_size:0.9991562339352135 - rollout_corr/rollout_is_seq_mean:1.0001581907272339 - rollout_corr/rollout_is_seq_std:0.0017724381759762764 - rollout_corr/rollout_is_seq_max:1.0063711404800415 - rollout_corr/rollout_is_seq_min:0.9951498508453369 - rollout_corr/rollout_is_seq_max_deviation:0.006371140480041504 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1587157743051648) - rollout_corr/training_log_ppl:np.float64(0.14470403948507737) - rollout_corr/kl:np.float64(0.0005400172712077822) - rollout_corr/k3_kl:np.float64(0.0005347190636655341) - rollout_corr/rollout_ppl:np.float64(1.1580887311138213) - rollout_corr/rollout_log_ppl:np.float64(0.14416402138158446) - rollout_corr/log_ppl_diff:np.float64(0.0005400181034929119) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015646610627300106) - rollout_corr/log_ppl_diff_max:np.float64(0.00749245285987854) - rollout_corr/log_ppl_diff_min:np.float64(-0.005699381232261658) - rollout_corr/ppl_ratio:np.float64(1.0005421619862318) - rollout_corr/chi2_token:np.float64(0.0010581978131085634) - rollout_corr/chi2_seq:np.float64(0.3867321067955345) - actor/pg_loss:np.float64(0.0007704137824475765) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002766783450169896) - actor/ppo_kl:np.float64(0.00026795737259011077) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(13437.756535453489) - self_distillation/token_reweight_w_std:np.float64(253038.20973212097) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000933702103794) - self_distillation/token_reweight_w_clip_frac:np.float64(0.022569283057237044) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.13522806763648987) - perf/mfu/actor:np.float64(0.05244350556818289) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.4292984008789) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003078529378399253 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003078529378399253 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:317.05859375 - response_length/max:613.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:317.05859375 - response_length_non_aborted/max:613.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.78125 - prompt_length/max:509.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.4436583518981934e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8329475354403257) - timing_s/agent_loop/generate_sequences/max:np.float64(5.8951219990849495) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6756774246459827) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.8951219990849495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:613 - timing_s/gen:13.717934239655733 - timing_s/reward:0.057973263785243034 - timing_s/old_log_prob:2.8934874050319195 - timing_s/adv:0.5061405170708895 - timing_s/update_actor:22.64327273145318 - timing_s/step:39.904685789719224 - timing_s/stop_profile:0.00012103840708732605 - timing_per_token_ms/adv:0.0027658102889681884 - timing_per_token_ms/update_actor:0.1237344069172683 - timing_per_token_ms/gen:0.16900876267024448 - perf/total_num_tokens:182999 - perf/time_per_step:39.904685789719224 - perf/throughput:573.2378177475421 (TaskRunner pid=2425326) Training Progress: 36%|███▌ | 36/100 [27:19<44:25, 41.65s/it] (TaskRunner pid=2425326) step:37 - global_seqlen/min:18214 - global_seqlen/max:23268 - global_seqlen/minmax_diff:5054 - global_seqlen/balanced_min:21453 - global_seqlen/balanced_max:21458 - global_seqlen/mean:21456.0 - actor/entropy:0.15314580500125885 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41912177205085754 - training/rollout_probs_diff_mean:0.002847726223990321 - training/rollout_probs_diff_std:0.009974293410778046 - training/rollout_actor_probs_pearson_corr:0.9986763000488281 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.8515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:1.0001238584518433 - rollout_corr/rollout_is_ratio_fraction_high:1.2167818567832e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2167818567832e-05 - rollout_corr/rollout_is_max:2.73596453666687 - rollout_corr/rollout_is_min:0.2720000147819519 - rollout_corr/rollout_is_std:0.029622619971632957 - rollout_corr/rollout_is_eff_sample_size:0.9991236267512279 - rollout_corr/rollout_is_seq_mean:1.0001177787780762 - rollout_corr/rollout_is_seq_std:0.001862207194790244 - rollout_corr/rollout_is_seq_max:1.0055677890777588 - rollout_corr/rollout_is_seq_min:0.9919551014900208 - rollout_corr/rollout_is_seq_max_deviation:0.008044898509979248 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.164930485188961) - rollout_corr/training_log_ppl:np.float64(0.14994544762885198) - rollout_corr/kl:np.float64(0.00047978433622719585) - rollout_corr/k3_kl:np.float64(0.0005115730669871255) - rollout_corr/rollout_ppl:np.float64(1.1643610489554703) - rollout_corr/rollout_log_ppl:np.float64(0.1494656624199706) - rollout_corr/log_ppl_diff:np.float64(0.000479785208881367) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015769913807162084) - rollout_corr/log_ppl_diff_max:np.float64(0.008887842297554016) - rollout_corr/log_ppl_diff_min:np.float64(-0.005079410970211029) - rollout_corr/ppl_ratio:np.float64(1.0004820607136935) - rollout_corr/chi2_token:np.float64(0.0011041394900530577) - rollout_corr/chi2_seq:np.float64(0.8046807374339551) - actor/pg_loss:np.float64(0.0009413559455424547) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015754969035697286) - actor/ppo_kl:np.float64(0.0001336421836066748) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_w_mean:np.float64(30945.948427035706) - self_distillation/token_reweight_w_std:np.float64(398418.81097031885) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998940988443792) - self_distillation/token_reweight_w_clip_frac:np.float64(0.031333529215771705) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.17728198319673538) - perf/mfu/actor:np.float64(0.048920584162657293) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.46635818481445) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.8515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021643508225679398 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021643508225679398 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:321.03125 - response_length/max:783.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:321.03125 - response_length_non_aborted/max:783.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.46875 - prompt_length/max:491.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000176331028342247 - timing_s/agent_loop/generate_sequences/min:np.float64(1.089025966823101) - timing_s/agent_loop/generate_sequences/max:np.float64(7.002712983638048) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7239339201696566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.002712983638048) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:782 - timing_s/gen:14.263660093769431 - timing_s/reward:0.05694244056940079 - timing_s/old_log_prob:2.8456411864608526 - timing_s/adv:0.5473702624440193 - timing_s/update_actor:22.9073896817863 - timing_s/step:40.70366654358804 - timing_s/stop_profile:0.00013758428394794464 - timing_per_token_ms/adv:0.0031889113910096205 - timing_per_token_ms/update_actor:0.13345561662114502 - timing_per_token_ms/gen:0.17355762792963875 - perf/total_num_tokens:171648 - perf/time_per_step:40.70366654358804 - perf/throughput:527.1269598532007 (TaskRunner pid=2425326) Training Progress: 37%|███▋ | 37/100 [28:00<43:26, 41.38s/it] (TaskRunner pid=2425326) step:38 - global_seqlen/min:19052 - global_seqlen/max:25764 - global_seqlen/minmax_diff:6712 - global_seqlen/balanced_min:23011 - global_seqlen/balanced_max:23014 - global_seqlen/mean:23012.25 - actor/entropy:0.1633475422859192 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.508357584476471 - training/rollout_probs_diff_mean:0.0028399399016052485 - training/rollout_probs_diff_std:0.009923688136041164 - training/rollout_actor_probs_pearson_corr:0.9988518953323364 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.000065565109253 - rollout_corr/rollout_is_ratio_fraction_high:1.1911284673260525e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1911284673260525e-05 - rollout_corr/rollout_is_max:2.890216827392578 - rollout_corr/rollout_is_min:0.3545355796813965 - rollout_corr/rollout_is_std:0.02890154905617237 - rollout_corr/rollout_is_eff_sample_size:0.9991653976510357 - rollout_corr/rollout_is_seq_mean:0.9999657273292542 - rollout_corr/rollout_is_seq_std:0.0017425058176741004 - rollout_corr/rollout_is_seq_max:1.009063482284546 - rollout_corr/rollout_is_seq_min:0.9950194358825684 - rollout_corr/rollout_is_seq_max_deviation:0.009063482284545898 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1723122037947178) - rollout_corr/training_log_ppl:np.float64(0.15486336171306903) - rollout_corr/kl:np.float64(0.000746124695474748) - rollout_corr/k3_kl:np.float64(0.0006364840925243698) - rollout_corr/rollout_ppl:np.float64(1.1714562741108239) - rollout_corr/rollout_log_ppl:np.float64(0.15411723549914313) - rollout_corr/log_ppl_diff:np.float64(0.0007461262139258906) - rollout_corr/log_ppl_abs_diff:np.float64(0.001779028432792984) - rollout_corr/log_ppl_diff_max:np.float64(0.01745888590812683) - rollout_corr/log_ppl_diff_min:np.float64(-0.005290612578392029) - rollout_corr/ppl_ratio:np.float64(1.000749502563849) - rollout_corr/chi2_token:np.float64(0.0008848432917147875) - rollout_corr/chi2_seq:np.float64(0.41483693197369576) - actor/pg_loss:np.float64(0.0016468875110149384) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00032333156786990003) - actor/ppo_kl:np.float64(0.0002796881955546837) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(109265.09118858888) - self_distillation/token_reweight_w_std:np.float64(1775391.109999408) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001167719019577) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0509182026507915) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.25122565031051636) - perf/mfu/actor:np.float64(0.05191056955392058) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.49505996704102) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012462182203307748 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012462182203307748 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:327.9453125 - response_length/max:741.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.9453125 - response_length_non_aborted/max:741.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.1875 - prompt_length/max:541.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010701827704906464 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3369148392230272) - timing_s/agent_loop/generate_sequences/max:np.float64(6.262549916282296) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7103576930894633) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.262549916282296) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:672 - timing_s/gen:13.752059187740088 - timing_s/reward:0.061503008008003235 - timing_s/old_log_prob:2.929329911246896 - timing_s/adv:0.550515603274107 - timing_s/update_actor:23.220404556021094 - timing_s/step:40.601566975936294 - timing_s/stop_profile:0.00011712312698364258 - timing_per_token_ms/adv:0.0029903399454318186 - timing_per_token_ms/update_actor:0.1261306725549495 - timing_per_token_ms/gen:0.16380469290016064 - perf/total_num_tokens:184098 - perf/time_per_step:40.601566975936294 - perf/throughput:566.7823119644344 (TaskRunner pid=2425326) Training Progress: 38%|███▊ | 38/100 [28:40<42:31, 41.16s/it] (TaskRunner pid=2425326) step:39 - global_seqlen/min:19236 - global_seqlen/max:27409 - global_seqlen/minmax_diff:8173 - global_seqlen/balanced_min:22689 - global_seqlen/balanced_max:22691 - global_seqlen/mean:22690.5 - actor/entropy:0.1420234888792038 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6971966028213501 - training/rollout_probs_diff_mean:0.0028620732482522726 - training/rollout_probs_diff_std:0.01034072507172823 - training/rollout_actor_probs_pearson_corr:0.9985620975494385 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000659227371216 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.4261245926027186e-05 - rollout_corr/rollout_is_max:1.6453677415847778 - rollout_corr/rollout_is_min:0.20416899025440216 - rollout_corr/rollout_is_std:0.02925819344818592 - rollout_corr/rollout_is_eff_sample_size:0.9991448092781762 - rollout_corr/rollout_is_seq_mean:1.0000933408737183 - rollout_corr/rollout_is_seq_std:0.0020522912964224815 - rollout_corr/rollout_is_seq_max:1.0066521167755127 - rollout_corr/rollout_is_seq_min:0.99286949634552 - rollout_corr/rollout_is_seq_max_deviation:0.0071305036544799805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1582270697690547) - rollout_corr/training_log_ppl:np.float64(0.1451856552012032) - rollout_corr/kl:np.float64(0.0004661589703989222) - rollout_corr/k3_kl:np.float64(0.0005055180065909326) - rollout_corr/rollout_ppl:np.float64(1.1576528307050467) - rollout_corr/rollout_log_ppl:np.float64(0.1447194957145257) - rollout_corr/log_ppl_diff:np.float64(0.0004661594866774976) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015421572024933994) - rollout_corr/log_ppl_diff_max:np.float64(0.010017722845077515) - rollout_corr/log_ppl_diff_min:np.float64(-0.006251499056816101) - rollout_corr/ppl_ratio:np.float64(1.0004685285966843) - rollout_corr/chi2_token:np.float64(0.0010773444082587957) - rollout_corr/chi2_seq:np.float64(0.30354314390569925) - actor/pg_loss:np.float64(0.00034410133957862854) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.910650533129228e-05) - actor/ppo_kl:np.float64(7.239252952295772e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7578125) - self_distillation/token_reweight_w_mean:np.float64(16097.881471956614) - self_distillation/token_reweight_w_std:np.float64(291332.0561478417) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000020751496777) - self_distillation/token_reweight_w_clip_frac:np.float64(0.00949771508021513) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.07992621650919318) - perf/mfu/actor:np.float64(0.05144791179433724) - perf/max_memory_allocated_gb:np.float64(130.98325157165527) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.4413948059082) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002805206459015608 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002805206459015608 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:322.015625 - response_length/max:743.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.015625 - response_length_non_aborted/max:743.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:387.0625 - prompt_length/max:505.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.531101047992706e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0251261461526155) - timing_s/agent_loop/generate_sequences/max:np.float64(7.094875009730458) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.93991277301393) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.094875009730458) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:743 - timing_s/gen:14.463495245203376 - timing_s/reward:0.05811183154582977 - timing_s/old_log_prob:2.9740923829376698 - timing_s/adv:0.5358271840959787 - timing_s/update_actor:22.99500824138522 - timing_s/step:41.10932963527739 - timing_s/stop_profile:0.0001144707202911377 - timing_per_token_ms/adv:0.002951825566294147 - timing_per_token_ms/update_actor:0.1266775095380513 - timing_per_token_ms/gen:0.1754512014799769 - perf/total_num_tokens:181524 - perf/time_per_step:41.10932963527739 - perf/throughput:551.9549990552136 (TaskRunner pid=2425326) Training Progress: 39%|███▉ | 39/100 [29:21<41:50, 41.16s/it] (TaskRunner pid=2425326) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2425326) validation generation end (TaskRunner pid=2425326) [prompt] system (TaskRunner pid=2425326) (TaskRunner pid=2425326) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2425326) (TaskRunner pid=2425326) A (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) user (TaskRunner pid=2425326) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2425326) (TaskRunner pid=2425326) Lattice Parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) a = 7.82 Å (TaskRunner pid=2425326) b = 7.82 Å (TaskRunner pid=2425326) c = 17.88 Å (TaskRunner pid=2425326) α = 90.00º (TaskRunner pid=2425326) β = 90.00º (TaskRunner pid=2425326) γ = 90.00º (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) A: 1025.67 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) B: 1094.31 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) C: 1200.45 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) D: 1150.78 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) Please reason step by step. (TaskRunner pid=2425326) assistant (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) [response] (TaskRunner pid=2425326) The given crystal structure has lattice parameters a = 7.82 Å, b = 7.82 Å, c = 17.88 Å, and all angles are 90.00º, which indicates a orthorhombic crystal system. (TaskRunner pid=2425326) (TaskRunner pid=2425326) For an orthorhombic system, the volume $ V $ of the unit cell is calculated using the formula: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) V = a \times b \times c (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Substituting the given values: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) V = 7.82 \times 7.82 \times 17.88 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Then multiply by 17.88: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) 61.1524 \times 17.88 = 1094.31 (TaskRunner pid=2425326) $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) B (TaskRunner pid=2425326) (TaskRunner pid=2425326) [ground_truth] B (TaskRunner pid=2425326) [score] 1.0 (TaskRunner pid=2425326) [acc] 1.0 (TaskRunner pid=2425326) [pred] B (TaskRunner pid=2425326) [incorrect_format] 1 (TaskRunner pid=2425326) [feedback] (TaskRunner pid=2425326) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2425326) step:40 - global_seqlen/min:17835 - global_seqlen/max:24761 - global_seqlen/minmax_diff:6926 - global_seqlen/balanced_min:21932 - global_seqlen/balanced_max:22013 - global_seqlen/mean:21943.625 - actor/entropy:0.14098937809467316 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553865313529968 - training/rollout_probs_diff_mean:0.0029350039549171925 - training/rollout_probs_diff_std:0.01030927523970604 - training/rollout_actor_probs_pearson_corr:0.9984802007675171 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.8515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000852346420288 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.2781676559825428e-05 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:0.41667383909225464 - rollout_corr/rollout_is_std:0.02979115955531597 - rollout_corr/rollout_is_eff_sample_size:0.9991133928184341 - rollout_corr/rollout_is_seq_mean:1.0000969171524048 - rollout_corr/rollout_is_seq_std:0.0018581942422315478 - rollout_corr/rollout_is_seq_max:1.0077598094940186 - rollout_corr/rollout_is_seq_min:0.9940147995948792 - rollout_corr/rollout_is_seq_max_deviation:0.007759809494018555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1458545695059001) - rollout_corr/training_log_ppl:np.float64(0.1349656182937906) - rollout_corr/kl:np.float64(0.0004070129167867975) - rollout_corr/k3_kl:np.float64(0.0004980892093442435) - rollout_corr/rollout_ppl:np.float64(1.1453717714175582) - rollout_corr/rollout_log_ppl:np.float64(0.13455860382237006) - rollout_corr/log_ppl_diff:np.float64(0.0004070144714205526) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015183983850874938) - rollout_corr/log_ppl_diff_max:np.float64(0.006893813610076904) - rollout_corr/log_ppl_diff_min:np.float64(-0.00674157589673996) - rollout_corr/ppl_ratio:np.float64(1.0004091300070286) - rollout_corr/chi2_token:np.float64(0.0011687756050378084) - rollout_corr/chi2_seq:np.float64(0.38695761072449386) - actor/pg_loss:np.float64(8.210085798054934e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.997138350721798e-05) - actor/ppo_kl:np.float64(4.544163184760919e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_w_mean:np.float64(13679.636977609713) - self_distillation/token_reweight_w_std:np.float64(279038.5410264209) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003306025173515) - self_distillation/token_reweight_w_clip_frac:np.float64(0.005064020268036984) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.03898233640938997) - perf/mfu/actor:np.float64(0.04943869679337855) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.3865146636963) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7340425531914894) - val-aux/sciknoweval/reward/std@16:np.float64(0.07158614242680836) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7607872340425531) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.06279756648335182) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7073085106382978) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.05685132383706091) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7343085106382979) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.07144667168417636) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7856808510638299) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.05281382505037524) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6870638297872341) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.044398161678960504) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7343723404255318) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.0524517593839488) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8084680851063829) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.042473494022919174) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6688936170212767) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.03476839920524086) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7338829787234042) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.0334935496036813) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8264574468085106) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.027109821964728293) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6521702127659574) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.02519807991996571) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7329999999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.024979263545988832) - val-aux/sciknoweval/score/mean@16:np.float64(0.7340425531914894) - val-aux/sciknoweval/score/std@16:np.float64(0.07158614242680836) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7607872340425531) - val-aux/sciknoweval/score/best@2/std:np.float64(0.06279756648335182) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7073085106382978) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.05685132383706091) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7343085106382979) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.07144667168417636) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7856808510638299) - val-aux/sciknoweval/score/best@4/std:np.float64(0.05281382505037524) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6870638297872341) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.044398161678960504) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7343723404255318) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.0524517593839488) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8084680851063829) - val-aux/sciknoweval/score/best@8/std:np.float64(0.042473494022919174) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6688936170212767) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.03476839920524086) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7338829787234042) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.0334935496036813) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8264574468085106) - val-aux/sciknoweval/score/best@16/std:np.float64(0.027109821964728293) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6521702127659574) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.02519807991996571) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7329999999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.024979263545988832) - val-core/sciknoweval/acc/mean@16:np.float64(0.7340425531914894) - val-aux/sciknoweval/acc/std@16:np.float64(0.07158614242680836) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7607872340425531) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.06279756648335182) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7073085106382978) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.05685132383706091) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7343085106382979) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.07144667168417636) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7856808510638299) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.05281382505037524) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6870638297872341) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.044398161678960504) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7343723404255318) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.0524517593839488) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8084680851063829) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.042473494022919174) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6688936170212767) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.03476839920524086) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7338829787234042) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.0334935496036813) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8264574468085106) - val-core/sciknoweval/acc/best@16/std:np.float64(0.027109821964728293) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6521702127659574) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.02519807991996571) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7329999999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.024979263545988832) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.8515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00013260988635011017 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:-0.00013260988635011017 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:305.61328125 - response_length/max:958.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.61328125 - response_length_non_aborted/max:958.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:380.125 - prompt_length/max:468.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010437145829200745 - timing_s/agent_loop/generate_sequences/min:np.float64(1.845179246738553) - timing_s/agent_loop/generate_sequences/max:np.float64(7.601139415055513) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5523179156225524) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.601139415055513) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:958 - timing_s/gen:14.69277754984796 - timing_s/reward:0.05978761799633503 - timing_s/old_log_prob:2.9359517358243465 - timing_s/adv:0.5586925502866507 - timing_s/update_actor:23.085252614691854 - timing_s/step:41.420557575300336 - timing_s/testing:39.801779298111796 - timing_s/save_checkpoint:13.031020415946841 - timing_s/stop_profile:0.00019116513431072235 - timing_per_token_ms/adv:0.003182544761215676 - timing_per_token_ms/update_actor:0.13150318494945487 - timing_per_token_ms/gen:0.1877983249593921 - perf/total_num_tokens:175549 - perf/time_per_step:41.420557575300336 - perf/throughput:529.7761856563054 (TaskRunner pid=2425326) Training Progress: 40%|████ | 40/100 [30:56<57:06, 57.11s/it] (TaskRunner pid=2425326) step:41 - global_seqlen/min:20580 - global_seqlen/max:26975 - global_seqlen/minmax_diff:6395 - global_seqlen/balanced_min:24255 - global_seqlen/balanced_max:24260 - global_seqlen/mean:24257.625 - actor/entropy:0.13618877530097961 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967532455921173 - training/rollout_probs_diff_mean:0.0028353757224977016 - training/rollout_probs_diff_std:0.010483196936547756 - training/rollout_actor_probs_pearson_corr:0.9983766078948975 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.0000534057617188 - rollout_corr/rollout_is_ratio_fraction_high:1.0927887160505634e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3711548642022535e-05 - rollout_corr/rollout_is_max:2.463893175125122 - rollout_corr/rollout_is_min:0.3244990110397339 - rollout_corr/rollout_is_std:0.029735084623098373 - rollout_corr/rollout_is_eff_sample_size:0.9991167247735356 - rollout_corr/rollout_is_seq_mean:1.000001072883606 - rollout_corr/rollout_is_seq_std:0.0018027505138888955 - rollout_corr/rollout_is_seq_max:1.005057454109192 - rollout_corr/rollout_is_seq_min:0.9950371384620667 - rollout_corr/rollout_is_seq_max_deviation:0.0050574541091918945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1445708661340177) - rollout_corr/training_log_ppl:np.float64(0.1329260604143201) - rollout_corr/kl:np.float64(0.0004669340913103781) - rollout_corr/k3_kl:np.float64(0.0004507001112799003) - rollout_corr/rollout_ppl:np.float64(1.1440314431674778) - rollout_corr/rollout_log_ppl:np.float64(0.1324591252778191) - rollout_corr/log_ppl_diff:np.float64(0.00046693513650097884) - rollout_corr/log_ppl_abs_diff:np.float64(0.001347396111668786) - rollout_corr/log_ppl_diff_max:np.float64(0.005796477198600769) - rollout_corr/log_ppl_diff_min:np.float64(-0.0040665604174137115) - rollout_corr/ppl_ratio:np.float64(1.0004686149768531) - rollout_corr/chi2_token:np.float64(0.0008493810892105103) - rollout_corr/chi2_seq:np.float64(0.331406096694991) - actor/pg_loss:np.float64(0.0016671433113515377) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000255978321092698) - actor/ppo_kl:np.float64(1.2966889821441896e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(22113.47238512803) - self_distillation/token_reweight_w_std:np.float64(394551.27365861356) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004445768427104) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02935507533402415) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.1600772701203823) - perf/mfu/actor:np.float64(0.05537688033283876) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.28454208374023) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002195139182731509 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002195139182731509 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:357.45703125 - response_length/max:769.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:357.45703125 - response_length_non_aborted/max:769.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:400.59375 - prompt_length/max:499.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.463673293590546e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8169757835566998) - timing_s/agent_loop/generate_sequences/max:np.float64(6.993100944906473) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.198472137322824) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.993100944906473) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:715 - timing_s/gen:13.706706086173654 - timing_s/reward:0.06317076832056046 - timing_s/old_log_prob:3.1366200875490904 - timing_s/adv:0.5526393372565508 - timing_s/update_actor:22.887388257309794 - timing_s/step:40.46955984644592 - timing_s/stop_profile:0.0001219920814037323 - timing_per_token_ms/adv:0.002847760947622401 - timing_per_token_ms/update_actor:0.11793914417275905 - timing_per_token_ms/gen:0.14978533353193296 - perf/total_num_tokens:194061 - perf/time_per_step:40.46955984644592 - perf/throughput:599.4042211489565 (TaskRunner pid=2425326) Training Progress: 41%|████ | 41/100 [31:36<51:15, 52.13s/it] (TaskRunner pid=2425326) step:42 - global_seqlen/min:22411 - global_seqlen/max:27248 - global_seqlen/minmax_diff:4837 - global_seqlen/balanced_min:24246 - global_seqlen/balanced_max:24253 - global_seqlen/mean:24250.5 - actor/entropy:0.14001302421092987 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5599758625030518 - training/rollout_probs_diff_mean:0.002688286593183875 - training/rollout_probs_diff_std:0.009930191561579704 - training/rollout_actor_probs_pearson_corr:0.9986210465431213 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.9999199509620667 - rollout_corr/rollout_is_ratio_fraction_high:1.1234440535190515e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.370331978658214e-05 - rollout_corr/rollout_is_max:2.4000132083892822 - rollout_corr/rollout_is_min:0.3642396628856659 - rollout_corr/rollout_is_std:0.028286617249250412 - rollout_corr/rollout_is_eff_sample_size:0.9992003879303721 - rollout_corr/rollout_is_seq_mean:0.9999122023582458 - rollout_corr/rollout_is_seq_std:0.0016149813309311867 - rollout_corr/rollout_is_seq_max:1.0048774480819702 - rollout_corr/rollout_is_seq_min:0.995093584060669 - rollout_corr/rollout_is_seq_max_deviation:0.004906415939331055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1585845770314336) - rollout_corr/training_log_ppl:np.float64(0.14303961391851772) - rollout_corr/kl:np.float64(0.0006641505782014612) - rollout_corr/k3_kl:np.float64(0.0005312824764303059) - rollout_corr/rollout_ppl:np.float64(1.1577940359711647) - rollout_corr/rollout_log_ppl:np.float64(0.14237546218646457) - rollout_corr/log_ppl_diff:np.float64(0.000664151732053142) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014161761864670552) - rollout_corr/log_ppl_diff_max:np.float64(0.009874150156974792) - rollout_corr/log_ppl_diff_min:np.float64(-0.004382699728012085) - rollout_corr/ppl_ratio:np.float64(1.000665974104777) - rollout_corr/chi2_token:np.float64(0.0007279803976416588) - rollout_corr/chi2_seq:np.float64(0.08038255502469838) - actor/pg_loss:np.float64(0.0011525413719937205) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020648631925723748) - actor/ppo_kl:np.float64(0.00015610256760312247) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(26345.013721656986) - self_distillation/token_reweight_w_std:np.float64(467469.8246827084) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015952868852764) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03433779346232768) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.193668182939291) - perf/mfu/actor:np.float64(0.054983356597584125) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.84840393066406) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0037874106783419847 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0037874106783419847 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:347.703125 - response_length/max:646.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:347.703125 - response_length_non_aborted/max:646.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:410.125 - prompt_length/max:510.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010742619633674622 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0047237034887075) - timing_s/agent_loop/generate_sequences/max:np.float64(6.425680071115494) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.337631682101346) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.425680071115494) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:646 - timing_s/gen:13.901440799236298 - timing_s/reward:0.06124081462621689 - timing_s/old_log_prob:2.9742622170597315 - timing_s/adv:0.5317160319536924 - timing_s/update_actor:22.95695862174034 - timing_s/step:40.51259587518871 - timing_s/stop_profile:0.00011090748012065887 - timing_per_token_ms/adv:0.0027407477781576277 - timing_per_token_ms/update_actor:0.11833239841312726 - timing_per_token_ms/gen:0.15617490674556575 - perf/total_num_tokens:194004 - perf/time_per_step:40.51259587518871 - perf/throughput:598.5916102416886 (TaskRunner pid=2425326) Training Progress: 42%|████▏ | 42/100 [32:17<47:02, 48.66s/it] (TaskRunner pid=2425326) step:43 - global_seqlen/min:19395 - global_seqlen/max:28000 - global_seqlen/minmax_diff:8605 - global_seqlen/balanced_min:24044 - global_seqlen/balanced_max:24049 - global_seqlen/mean:24046.875 - actor/entropy:0.1420712172985077 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292536854743958 - training/rollout_probs_diff_mean:0.002893645316362381 - training/rollout_probs_diff_std:0.01048985868692398 - training/rollout_actor_probs_pearson_corr:0.9984310269355774 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58984375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.58984375 - rollout_corr/rollout_is_mean:0.9999460577964783 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.5401404349831864e-05 - rollout_corr/rollout_is_max:1.6487189531326294 - rollout_corr/rollout_is_min:0.2613983452320099 - rollout_corr/rollout_is_std:0.028771327808499336 - rollout_corr/rollout_is_eff_sample_size:0.9991727763495746 - rollout_corr/rollout_is_seq_mean:1.0000054836273193 - rollout_corr/rollout_is_seq_std:0.0017316376324743032 - rollout_corr/rollout_is_seq_max:1.0067939758300781 - rollout_corr/rollout_is_seq_min:0.9957586526870728 - rollout_corr/rollout_is_seq_max_deviation:0.006793975830078125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.157421734649688) - rollout_corr/training_log_ppl:np.float64(0.14304501365404576) - rollout_corr/kl:np.float64(0.0005448067362863185) - rollout_corr/k3_kl:np.float64(0.0005209827327945504) - rollout_corr/rollout_ppl:np.float64(1.1567890807054937) - rollout_corr/rollout_log_ppl:np.float64(0.14250020634790417) - rollout_corr/log_ppl_diff:np.float64(0.0005448073061415926) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015114768029889092) - rollout_corr/log_ppl_diff_max:np.float64(0.006321355700492859) - rollout_corr/log_ppl_diff_min:np.float64(-0.00696495920419693) - rollout_corr/ppl_ratio:np.float64(1.0005467752926052) - rollout_corr/chi2_token:np.float64(0.0010000739712268114) - rollout_corr/chi2_seq:np.float64(0.3491819892078638) - actor/pg_loss:np.float64(0.0009307680884376168) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.09595422752318e-05) - actor/ppo_kl:np.float64(0.00011152088650501213) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.58984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(14319.957484231563) - self_distillation/token_reweight_w_std:np.float64(238024.43798052194) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005522321444005) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0240835617587436) - self_distillation/empty_target_batch:np.float64(0.41015625) - actor/grad_norm:np.float64(0.12713662534952164) - perf/mfu/actor:np.float64(0.0551132476644326) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.6787872314453) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012457011034712195 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012457011034712195 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.15234375 - response_length/max:764.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.15234375 - response_length_non_aborted/max:764.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.3125 - prompt_length/max:494.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001889839768409729 - timing_s/agent_loop/generate_sequences/min:np.float64(1.859173921868205) - timing_s/agent_loop/generate_sequences/max:np.float64(7.096025053411722) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.087591924362641) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.096025053411722) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:764 - timing_s/gen:14.183020344004035 - timing_s/reward:0.06159451976418495 - timing_s/old_log_prob:2.980512648820877 - timing_s/adv:0.5222495682537556 - timing_s/update_actor:22.39236894249916 - timing_s/step:40.22726967558265 - timing_s/stop_profile:0.00012237019836902618 - timing_per_token_ms/adv:0.002714747593261887 - timing_per_token_ms/update_actor:0.11639957864846867 - timing_per_token_ms/gen:0.16098226330549512 - perf/total_num_tokens:192375 - perf/time_per_step:40.22726967558265 - perf/throughput:597.7754690767913 (TaskRunner pid=2425326) Training Progress: 43%|████▎ | 43/100 [32:57<43:50, 46.15s/it] (TaskRunner pid=2425326) step:44 - global_seqlen/min:21992 - global_seqlen/max:25456 - global_seqlen/minmax_diff:3464 - global_seqlen/balanced_min:23423 - global_seqlen/balanced_max:23432 - global_seqlen/mean:23429.125 - actor/entropy:0.13345500826835632 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3015652000904083 - training/rollout_probs_diff_mean:0.0027469515334814787 - training/rollout_probs_diff_std:0.009971688501536846 - training/rollout_actor_probs_pearson_corr:0.9986183643341064 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.72265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0000137090682983 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1071621884184424e-05 - rollout_corr/rollout_is_max:1.8683162927627563 - rollout_corr/rollout_is_min:0.49738770723342896 - rollout_corr/rollout_is_std:0.02936190739274025 - rollout_corr/rollout_is_eff_sample_size:0.9991387400353914 - rollout_corr/rollout_is_seq_mean:0.9999842047691345 - rollout_corr/rollout_is_seq_std:0.0018778472440317273 - rollout_corr/rollout_is_seq_max:1.0074628591537476 - rollout_corr/rollout_is_seq_min:0.992678165435791 - rollout_corr/rollout_is_seq_max_deviation:0.007462859153747559 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1445557121187449) - rollout_corr/training_log_ppl:np.float64(0.13367409240163397) - rollout_corr/kl:np.float64(0.0005900809133603246) - rollout_corr/k3_kl:np.float64(0.00046168545707558906) - rollout_corr/rollout_ppl:np.float64(1.1438749125227332) - rollout_corr/rollout_log_ppl:np.float64(0.13308401135873282) - rollout_corr/log_ppl_diff:np.float64(0.0005900810429011472) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015492462043766864) - rollout_corr/log_ppl_diff_max:np.float64(0.0074571967124938965) - rollout_corr/log_ppl_diff_min:np.float64(-0.008081652224063873) - rollout_corr/ppl_ratio:np.float64(1.0005922082345933) - rollout_corr/chi2_token:np.float64(0.0006860834546387196) - rollout_corr/chi2_seq:np.float64(0.6962482798844576) - actor/pg_loss:np.float64(0.0007348335348069668) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.418954439548543e-05) - actor/ppo_kl:np.float64(0.00012582078771217198) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.72265625) - self_distillation/token_reweight_w_mean:np.float64(20517.69756563753) - self_distillation/token_reweight_w_std:np.float64(358047.4882045661) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000059798359871) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0146756786343758) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.18048310466110706) - perf/mfu/actor:np.float64(0.05380951610340004) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.57110977172852) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006885165348649025 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.006885165348649025 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:352.81640625 - response_length/max:799.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.81640625 - response_length_non_aborted/max:799.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.34375 - prompt_length/max:524.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.593553841114044e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6965281628072262) - timing_s/agent_loop/generate_sequences/max:np.float64(6.971797473728657) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.097140631893126) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.971797473728657) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:799 - timing_s/gen:14.65759794227779 - timing_s/reward:0.061198318377137184 - timing_s/old_log_prob:2.977988762781024 - timing_s/adv:0.5558650456368923 - timing_s/update_actor:22.66115582920611 - timing_s/step:41.000115586444736 - timing_s/stop_profile:0.00011423230171203613 - timing_per_token_ms/adv:0.002965673310659768 - timing_per_token_ms/update_actor:0.1209027003206805 - timing_per_token_ms/gen:0.16228338860594754 - perf/total_num_tokens:187433 - perf/time_per_step:41.000115586444736 - perf/throughput:571.4404621763073 (TaskRunner pid=2425326) Training Progress: 44%|████▍ | 44/100 [33:38<41:38, 44.62s/it] (TaskRunner pid=2425326) step:45 - global_seqlen/min:18176 - global_seqlen/max:26087 - global_seqlen/minmax_diff:7911 - global_seqlen/balanced_min:22796 - global_seqlen/balanced_max:22799 - global_seqlen/mean:22797.625 - actor/entropy:0.1399414837360382 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102373421192169 - training/rollout_probs_diff_mean:0.002902822569012642 - training/rollout_probs_diff_std:0.010340701788663864 - training/rollout_actor_probs_pearson_corr:0.9984586834907532 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.7734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0001752376556396 - rollout_corr/rollout_is_ratio_fraction_high:1.1424262993386947e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2848525986773893e-05 - rollout_corr/rollout_is_max:2.3329057693481445 - rollout_corr/rollout_is_min:0.3676825761795044 - rollout_corr/rollout_is_std:0.030085813254117966 - rollout_corr/rollout_is_eff_sample_size:0.9990960194125961 - rollout_corr/rollout_is_seq_mean:1.0002131462097168 - rollout_corr/rollout_is_seq_std:0.0018944222247228026 - rollout_corr/rollout_is_seq_max:1.0114330053329468 - rollout_corr/rollout_is_seq_min:0.9952909350395203 - rollout_corr/rollout_is_seq_max_deviation:0.011433005332946777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1445479299873114) - rollout_corr/training_log_ppl:np.float64(0.13379971245012712) - rollout_corr/kl:np.float64(0.0005008000899939802) - rollout_corr/k3_kl:np.float64(0.0004867743777481337) - rollout_corr/rollout_ppl:np.float64(1.1439769458957016) - rollout_corr/rollout_log_ppl:np.float64(0.13329891020839568) - rollout_corr/log_ppl_diff:np.float64(0.0005008022417314351) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014860984665574506) - rollout_corr/log_ppl_diff_max:np.float64(0.00550118088722229) - rollout_corr/log_ppl_diff_min:np.float64(-0.009168602526187897) - rollout_corr/ppl_ratio:np.float64(1.00050284457393) - rollout_corr/chi2_token:np.float64(0.0009574964642524719) - rollout_corr/chi2_seq:np.float64(0.4922240450978279) - actor/pg_loss:np.float64(0.0010258276015520096) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016894487498575472) - actor/ppo_kl:np.float64(0.0002360405962065215) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_w_mean:np.float64(9821.606562155532) - self_distillation/token_reweight_w_std:np.float64(178767.35334247636) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001066522672772) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01871949274209328) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.1153863884974271) - perf/mfu/actor:np.float64(0.05208565184680937) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.7615509033203) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:1 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0014894382329657674 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0014894382329657674 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:341.92578125 - response_length/max:868.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:341.92578125 - response_length_non_aborted/max:868.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:370.5 - prompt_length/max:483.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015263818204402924 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9034442100673914) - timing_s/agent_loop/generate_sequences/max:np.float64(7.533223940059543) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.981655356939882) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.533223940059543) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:868 - timing_s/gen:14.541515922173858 - timing_s/reward:0.04938389174640179 - timing_s/old_log_prob:3.0376770105212927 - timing_s/adv:0.5767613369971514 - timing_s/update_actor:22.603338291868567 - timing_s/step:40.843872372061014 - timing_s/stop_profile:0.00011718086898326874 - timing_per_token_ms/adv:0.0031623981500109736 - timing_per_token_ms/update_actor:0.12393472067742017 - timing_per_token_ms/gen:0.1661261001242258 - perf/total_num_tokens:182381 - perf/time_per_step:40.843872372061014 - perf/throughput:558.1651218652462 (TaskRunner pid=2425326) Training Progress: 45%|████▌ | 45/100 [34:19<39:52, 43.50s/it] (TaskRunner pid=2425326) step:46 - global_seqlen/min:21540 - global_seqlen/max:26143 - global_seqlen/minmax_diff:4603 - global_seqlen/balanced_min:23587 - global_seqlen/balanced_max:23591 - global_seqlen/mean:23589.25 - actor/entropy:0.1451496034860611 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21502655744552612 - training/rollout_probs_diff_mean:0.002851153025403619 - training/rollout_probs_diff_std:0.009968206286430359 - training/rollout_actor_probs_pearson_corr:0.9986371994018555 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0002301931381226 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.655013918876648 - rollout_corr/rollout_is_min:0.5358285903930664 - rollout_corr/rollout_is_std:0.028942765668034554 - rollout_corr/rollout_is_eff_sample_size:0.9991633744785383 - rollout_corr/rollout_is_seq_mean:1.0002855062484741 - rollout_corr/rollout_is_seq_std:0.0016697313403710723 - rollout_corr/rollout_is_seq_max:1.005419373512268 - rollout_corr/rollout_is_seq_min:0.9953736066818237 - rollout_corr/rollout_is_seq_max_deviation:0.005419373512268066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1572659551165998) - rollout_corr/training_log_ppl:np.float64(0.14247176656499505) - rollout_corr/kl:np.float64(0.0002932232152055292) - rollout_corr/k3_kl:np.float64(0.00048011249347723606) - rollout_corr/rollout_ppl:np.float64(1.156941320747137) - rollout_corr/rollout_log_ppl:np.float64(0.14217854263552) - rollout_corr/log_ppl_diff:np.float64(0.0002932239294750616) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013674008805537596) - rollout_corr/log_ppl_diff_max:np.float64(0.006510406732559204) - rollout_corr/log_ppl_diff_min:np.float64(-0.004583634436130524) - rollout_corr/ppl_ratio:np.float64(1.0002947926986963) - rollout_corr/chi2_token:np.float64(0.0013452090788632631) - rollout_corr/chi2_seq:np.float64(0.4254507820587605) - actor/pg_loss:np.float64(0.0008674579439684749) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011161280372107285) - actor/ppo_kl:np.float64(0.000148489029569987) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(14773.669593083207) - self_distillation/token_reweight_w_std:np.float64(243772.67391494405) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000106336781755) - self_distillation/token_reweight_w_clip_frac:np.float64(0.021612054741126485) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.1143045425415039) - perf/mfu/actor:np.float64(0.05422613845648462) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.55764389038086) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:1 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023552831262350082 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023552831262350082 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:356.1640625 - response_length/max:654.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:356.1640625 - response_length_non_aborted/max:654.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:381.0 - prompt_length/max:481.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010038353502750397 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9539247006177902) - timing_s/agent_loop/generate_sequences/max:np.float64(6.435800917446613) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.197516475353041) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.435800917446613) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:271 - timing_s/agent_loop/slowest/response_length:654 - timing_s/gen:13.739061588421464 - timing_s/reward:0.061339808627963066 - timing_s/old_log_prob:3.0308387633413076 - timing_s/adv:0.5223424322903156 - timing_s/update_actor:22.62789387628436 - timing_s/step:40.067844934761524 - timing_s/stop_profile:0.00011489540338516235 - timing_per_token_ms/adv:0.0027679050430297466 - timing_per_token_ms/update_actor:0.11990575090499041 - timing_per_token_ms/gen:0.15068395433571108 - perf/total_num_tokens:188714 - perf/time_per_step:40.067844934761524 - perf/throughput:588.7326867319174 (TaskRunner pid=2425326) Training Progress: 46%|████▌ | 46/100 [34:59<38:14, 42.49s/it] (TaskRunner pid=2425326) step:47 - global_seqlen/min:20476 - global_seqlen/max:26782 - global_seqlen/minmax_diff:6306 - global_seqlen/balanced_min:24446 - global_seqlen/balanced_max:24449 - global_seqlen/mean:24447.5 - actor/entropy:0.1583218276500702 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163803696632385 - training/rollout_probs_diff_mean:0.003097226144745946 - training/rollout_probs_diff_std:0.010430659167468548 - training/rollout_actor_probs_pearson_corr:0.998622477054596 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0001773834228516 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.939037799835205 - rollout_corr/rollout_is_min:0.5226768255233765 - rollout_corr/rollout_is_std:0.03018668293952942 - rollout_corr/rollout_is_eff_sample_size:0.9990899507625369 - rollout_corr/rollout_is_seq_mean:1.0002132654190063 - rollout_corr/rollout_is_seq_std:0.0017920753452926874 - rollout_corr/rollout_is_seq_max:1.00586998462677 - rollout_corr/rollout_is_seq_min:0.9940651059150696 - rollout_corr/rollout_is_seq_max_deviation:0.00593489408493042 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.177405840717256) - rollout_corr/training_log_ppl:np.float64(0.1596459786160267) - rollout_corr/kl:np.float64(0.000444874939256934) - rollout_corr/k3_kl:np.float64(0.0005352445203357092) - rollout_corr/rollout_ppl:np.float64(1.1769076697528362) - rollout_corr/rollout_log_ppl:np.float64(0.15920110348815797) - rollout_corr/log_ppl_diff:np.float64(0.00044487512786872685) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015562142943963408) - rollout_corr/log_ppl_diff_max:np.float64(0.008173204958438873) - rollout_corr/log_ppl_diff_min:np.float64(-0.00523337721824646) - rollout_corr/ppl_ratio:np.float64(1.0004469151608646) - rollout_corr/chi2_token:np.float64(0.0012778392992913723) - rollout_corr/chi2_seq:np.float64(0.631908955052495) - actor/pg_loss:np.float64(0.001201812643557787) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00034777516520989593) - actor/ppo_kl:np.float64(0.0001929089082366442) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(33618.56932144542) - self_distillation/token_reweight_w_std:np.float64(630104.3924066984) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997440660372376) - self_distillation/token_reweight_w_clip_frac:np.float64(0.027509495565027464) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.1583699844777584) - perf/mfu/actor:np.float64(0.05528347187737279) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.58264541625977) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002365098800510168 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002365098800510168 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:372.234375 - response_length/max:590.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:372.234375 - response_length_non_aborted/max:590.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.75 - prompt_length/max:490.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.800439536571503e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0056671164929867) - timing_s/agent_loop/generate_sequences/max:np.float64(6.387341894209385) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.528739148467139) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.387341894209385) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:414 - timing_s/agent_loop/slowest/response_length:582 - timing_s/gen:13.414426237344742 - timing_s/reward:0.06257399171590805 - timing_s/old_log_prob:2.9914919547736645 - timing_s/adv:0.5313322227448225 - timing_s/update_actor:23.144446570426226 - timing_s/step:40.23323190957308 - timing_s/stop_profile:0.00011191517114639282 - timing_per_token_ms/adv:0.002716700187876176 - timing_per_token_ms/update_actor:0.11833749141234393 - timing_per_token_ms/gen:0.14077179865408157 - perf/total_num_tokens:195580 - perf/time_per_step:40.23323190957308 - perf/throughput:607.644448125555 (TaskRunner pid=2425326) Training Progress: 47%|████▋ | 47/100 [35:39<36:56, 41.82s/it] (TaskRunner pid=2425326) step:48 - global_seqlen/min:21150 - global_seqlen/max:26798 - global_seqlen/minmax_diff:5648 - global_seqlen/balanced_min:24129 - global_seqlen/balanced_max:24132 - global_seqlen/mean:24130.75 - actor/entropy:0.14493174850940704 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5314710736274719 - training/rollout_probs_diff_mean:0.002818688051775098 - training/rollout_probs_diff_std:0.010344676673412323 - training/rollout_actor_probs_pearson_corr:0.9985598921775818 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.84375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0001366138458252 - rollout_corr/rollout_is_ratio_fraction_high:2.1507688870769925e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.1507688870769925e-05 - rollout_corr/rollout_is_max:3.622957468032837 - rollout_corr/rollout_is_min:0.05226994305849075 - rollout_corr/rollout_is_std:0.02917659282684326 - rollout_corr/rollout_is_eff_sample_size:0.999149688526822 - rollout_corr/rollout_is_seq_mean:1.000110387802124 - rollout_corr/rollout_is_seq_std:0.0019526410615071654 - rollout_corr/rollout_is_seq_max:1.0087071657180786 - rollout_corr/rollout_is_seq_min:0.9937211275100708 - rollout_corr/rollout_is_seq_max_deviation:0.008707165718078613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1530222110450268) - rollout_corr/training_log_ppl:np.float64(0.14057794290420134) - rollout_corr/kl:np.float64(0.00048637970533782493) - rollout_corr/k3_kl:np.float64(0.0006143509694993554) - rollout_corr/rollout_ppl:np.float64(1.1524676536209881) - rollout_corr/rollout_log_ppl:np.float64(0.14009156363317743) - rollout_corr/log_ppl_diff:np.float64(0.00048637927102390677) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016521671932423487) - rollout_corr/log_ppl_diff_max:np.float64(0.007564276456832886) - rollout_corr/log_ppl_diff_min:np.float64(-0.007610246539115906) - rollout_corr/ppl_ratio:np.float64(1.000488800695166) - rollout_corr/chi2_token:np.float64(0.001616457011550665) - rollout_corr/chi2_seq:np.float64(0.7807452781125903) - actor/pg_loss:np.float64(0.000730266678147018) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002534931982154376) - actor/ppo_kl:np.float64(0.00012199131381329664) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_w_mean:np.float64(31238.350169518264) - self_distillation/token_reweight_w_std:np.float64(474846.4062488966) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999360444489866) - self_distillation/token_reweight_w_clip_frac:np.float64(0.017788645636755973) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.18741019908338785) - perf/mfu/actor:np.float64(0.054881577129153286) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.60995864868164) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:1 - critic/score/mean:0.84375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010646305745467544 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010646305745467544 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:363.2421875 - response_length/max:779.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:363.2421875 - response_length_non_aborted/max:779.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.84375 - prompt_length/max:488.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.346380829811096e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5789380948990583) - timing_s/agent_loop/generate_sequences/max:np.float64(7.043883478268981) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.176404346886557) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.043883478268981) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:779 - timing_s/gen:14.109347710385919 - timing_s/reward:0.06204022653400898 - timing_s/old_log_prob:3.0001636762171984 - timing_s/adv:0.5644608493894339 - timing_s/update_actor:22.82312749326229 - timing_s/step:40.64677978679538 - timing_s/stop_profile:0.00011142902076244354 - timing_per_token_ms/adv:0.0029239707084810556 - timing_per_token_ms/update_actor:0.11822636829181797 - timing_per_token_ms/gen:0.15172973126557607 - perf/total_num_tokens:193046 - perf/time_per_step:40.64677978679538 - perf/throughput:593.6694155495974 (TaskRunner pid=2425326) Training Progress: 48%|████▊ | 48/100 [36:20<35:57, 41.49s/it] (TaskRunner pid=2425326) step:49 - global_seqlen/min:21430 - global_seqlen/max:27560 - global_seqlen/minmax_diff:6130 - global_seqlen/balanced_min:24100 - global_seqlen/balanced_max:24104 - global_seqlen/mean:24102.75 - actor/entropy:0.15728282928466797 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.277299165725708 - training/rollout_probs_diff_mean:0.002870401367545128 - training/rollout_probs_diff_std:0.009825697168707848 - training/rollout_actor_probs_pearson_corr:0.9987659454345703 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000709295272827 - rollout_corr/rollout_is_ratio_fraction_high:2.1716941773775034e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1670196056365967 - rollout_corr/rollout_is_min:0.593317985534668 - rollout_corr/rollout_is_std:0.028590524569153786 - rollout_corr/rollout_is_eff_sample_size:0.9991836065722173 - rollout_corr/rollout_is_seq_mean:1.0001214742660522 - rollout_corr/rollout_is_seq_std:0.0017916224896907806 - rollout_corr/rollout_is_seq_max:1.0051995515823364 - rollout_corr/rollout_is_seq_min:0.9944879412651062 - rollout_corr/rollout_is_seq_max_deviation:0.005512058734893799 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1734219631180167) - rollout_corr/training_log_ppl:np.float64(0.15717786941240774) - rollout_corr/kl:np.float64(0.0005783919656501268) - rollout_corr/k3_kl:np.float64(0.0004959159838193727) - rollout_corr/rollout_ppl:np.float64(1.1727482806891203) - rollout_corr/rollout_log_ppl:np.float64(0.15659947633685078) - rollout_corr/log_ppl_diff:np.float64(0.00057839307555696) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014631202866439708) - rollout_corr/log_ppl_diff_max:np.float64(0.008148357272148132) - rollout_corr/log_ppl_diff_min:np.float64(-0.004555851221084595) - rollout_corr/ppl_ratio:np.float64(1.0005802374798805) - rollout_corr/chi2_token:np.float64(0.0008415062911808491) - rollout_corr/chi2_seq:np.float64(0.23110721190460026) - actor/pg_loss:np.float64(0.00141685560811311) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017184585794893792) - actor/ppo_kl:np.float64(0.00026630591867160547) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(18197.53233134863) - self_distillation/token_reweight_w_std:np.float64(351916.0242279185) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004864516668022) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03428729544975795) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.20876117795705795) - perf/mfu/actor:np.float64(0.05458018329697781) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.66826248168945) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00484559265896678 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00484559265896678 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:359.7421875 - response_length/max:828.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:359.7421875 - response_length_non_aborted/max:828.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:393.46875 - prompt_length/max:486.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015664659440517426 - timing_s/agent_loop/generate_sequences/min:np.float64(1.306804582476616) - timing_s/agent_loop/generate_sequences/max:np.float64(7.429824976250529) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.005236965065706) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.429824976250529) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:828 - timing_s/gen:14.722663125023246 - timing_s/reward:0.06185739301145077 - timing_s/old_log_prob:2.9708834551274776 - timing_s/adv:0.533998092636466 - timing_s/update_actor:22.89355112798512 - timing_s/step:41.26890727877617 - timing_s/stop_profile:0.0001263655722141266 - timing_per_token_ms/adv:0.0027693836420971986 - timing_per_token_ms/update_actor:0.11872893719588594 - timing_per_token_ms/gen:0.15986560606579414 - perf/total_num_tokens:192822 - perf/time_per_step:41.26890727877617 - perf/throughput:584.0413907056753 (TaskRunner pid=2425326) Training Progress: 49%|████▉ | 49/100 [37:01<35:13, 41.43s/it] (TaskRunner pid=2425326) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2425326) validation generation end (TaskRunner pid=2425326) [prompt] system (TaskRunner pid=2425326) (TaskRunner pid=2425326) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2425326) (TaskRunner pid=2425326) A (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) user (TaskRunner pid=2425326) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2425326) (TaskRunner pid=2425326) Lattice Parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) a = 7.82 Å (TaskRunner pid=2425326) b = 7.82 Å (TaskRunner pid=2425326) c = 17.88 Å (TaskRunner pid=2425326) α = 90.00º (TaskRunner pid=2425326) β = 90.00º (TaskRunner pid=2425326) γ = 90.00º (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) A: 1025.67 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) B: 1094.31 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) C: 1200.45 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) D: 1150.78 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) Please reason step by step. (TaskRunner pid=2425326) assistant (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) [response] (TaskRunner pid=2425326) The given crystal structure has lattice parameters a = 7.82 Å, b = 7.82 Å, c = 17.88 Å, and all angles α, β, γ are 90.00°, which indicates that the unit cell is orthorhombic. (TaskRunner pid=2425326) (TaskRunner pid=2425326) For an orthorhombic unit cell, the volume $ V $ of the unit cell is calculated using the formula: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ V = a \times b \times c $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Substituting the given values: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Then multiply by 17.88: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) B (TaskRunner pid=2425326) (TaskRunner pid=2425326) [ground_truth] B (TaskRunner pid=2425326) [score] 1.0 (TaskRunner pid=2425326) [acc] 1.0 (TaskRunner pid=2425326) [pred] B (TaskRunner pid=2425326) [incorrect_format] 1 (TaskRunner pid=2425326) [feedback] (TaskRunner pid=2425326) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2425326) step:50 - global_seqlen/min:18129 - global_seqlen/max:28037 - global_seqlen/minmax_diff:9908 - global_seqlen/balanced_min:24010 - global_seqlen/balanced_max:24016 - global_seqlen/mean:24014.5 - actor/entropy:0.16035732626914978 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744114577770233 - training/rollout_probs_diff_mean:0.0029135870281606913 - training/rollout_probs_diff_std:0.009974236600100994 - training/rollout_actor_probs_pearson_corr:0.998715877532959 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9999343156814575 - rollout_corr/rollout_is_ratio_fraction_high:1.1190188160981052e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1190188160981052e-05 - rollout_corr/rollout_is_max:2.2565836906433105 - rollout_corr/rollout_is_min:0.2993377447128296 - rollout_corr/rollout_is_std:0.02937408536672592 - rollout_corr/rollout_is_eff_sample_size:0.9991379070078227 - rollout_corr/rollout_is_seq_mean:0.9999151229858398 - rollout_corr/rollout_is_seq_std:0.0018135372083634138 - rollout_corr/rollout_is_seq_max:1.0075640678405762 - rollout_corr/rollout_is_seq_min:0.9934759140014648 - rollout_corr/rollout_is_seq_max_deviation:0.007564067840576172 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.175829870160669) - rollout_corr/training_log_ppl:np.float64(0.1581116948655108) - rollout_corr/kl:np.float64(0.00059627767032433) - rollout_corr/k3_kl:np.float64(0.0005526660350341217) - rollout_corr/rollout_ppl:np.float64(1.1751363426446915) - rollout_corr/rollout_log_ppl:np.float64(0.15751541723147966) - rollout_corr/log_ppl_diff:np.float64(0.0005962776340311393) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015397017996292561) - rollout_corr/log_ppl_diff_max:np.float64(0.007978282868862152) - rollout_corr/log_ppl_diff_min:np.float64(-0.0065612345933914185) - rollout_corr/ppl_ratio:np.float64(1.0005985118914396) - rollout_corr/chi2_token:np.float64(0.001080595189705491) - rollout_corr/chi2_seq:np.float64(0.42997400439344347) - actor/pg_loss:np.float64(0.0008469953900203109) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002414540476820548) - actor/ppo_kl:np.float64(6.015790615965244e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(43890.94956536731) - self_distillation/token_reweight_w_std:np.float64(817201.1323379974) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998418700415641) - self_distillation/token_reweight_w_clip_frac:np.float64(0.027138499077409506) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.1494966372847557) - perf/mfu/actor:np.float64(0.05476425496633601) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.49509620666504) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7466755319148937) - val-aux/sciknoweval/reward/std@16:np.float64(0.053407092795511396) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7685744680851063) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.04014169769484946) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7242765957446807) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.05065782360309899) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7465744680851063) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.05329993997278359) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7842234042553193) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.024681103201271454) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7045744680851064) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.04192652565659548) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7502127659574468) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.04277851357615754) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7933936170212765) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.012692031734567996) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6888936170212766) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.032613163884582116) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7509893617021277) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.031065193081468847) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7972127659574468) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.004452606778547598) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6755425531914894) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.024261259848393415) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7515957446808511) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.02253054389908937) - val-aux/sciknoweval/score/mean@16:np.float64(0.7466755319148937) - val-aux/sciknoweval/score/std@16:np.float64(0.053407092795511396) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7685744680851063) - val-aux/sciknoweval/score/best@2/std:np.float64(0.04014169769484946) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7242765957446807) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.05065782360309899) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7465744680851063) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.05329993997278359) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7842234042553193) - val-aux/sciknoweval/score/best@4/std:np.float64(0.024681103201271454) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7045744680851064) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.04192652565659548) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7502127659574468) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.04277851357615754) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7933936170212765) - val-aux/sciknoweval/score/best@8/std:np.float64(0.012692031734567996) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6888936170212766) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.032613163884582116) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7509893617021277) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.031065193081468847) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7972127659574468) - val-aux/sciknoweval/score/best@16/std:np.float64(0.004452606778547598) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6755425531914894) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.024261259848393415) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7515957446808511) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.02253054389908937) - val-core/sciknoweval/acc/mean@16:np.float64(0.7466755319148937) - val-aux/sciknoweval/acc/std@16:np.float64(0.053407092795511396) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7685744680851063) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.04014169769484946) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7242765957446807) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.05065782360309899) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7465744680851063) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.05329993997278359) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7842234042553193) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.024681103201271454) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7045744680851064) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.04192652565659548) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7502127659574468) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.04277851357615754) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7933936170212765) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.012692031734567996) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6888936170212766) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.032613163884582116) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7509893617021277) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.031065193081468847) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7972127659574468) - val-core/sciknoweval/acc/best@16/std:np.float64(0.004452606778547598) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6755425531914894) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.024261259848393415) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7515957446808511) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.02253054389908937) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004277449566870928 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004277449566870928 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:349.078125 - response_length/max:739.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:349.078125 - response_length_non_aborted/max:739.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.375 - prompt_length/max:492.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014016032218933105 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7744934242218733) - timing_s/agent_loop/generate_sequences/max:np.float64(6.806796507909894) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.00763895396085) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.806796507909894) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:492 - timing_s/agent_loop/slowest/response_length:739 - timing_s/gen:13.687782673165202 - timing_s/reward:0.06061543710529804 - timing_s/old_log_prob:2.968567503616214 - timing_s/adv:0.5489756148308516 - timing_s/update_actor:22.73579845018685 - timing_s/step:40.089707519859076 - timing_s/testing:41.6073386631906 - timing_s/save_checkpoint:12.920882619917393 - timing_s/stop_profile:4.44464385509491e-05 - timing_per_token_ms/adv:0.002857521574626015 - timing_per_token_ms/update_actor:0.11834411735715322 - timing_per_token_ms/gen:0.15316886747644692 - perf/total_num_tokens:192116 - perf/time_per_step:40.089707519859076 - perf/throughput:599.0190870837367 (TaskRunner pid=2425326) Training Progress: 50%|█████ | 50/100 [38:36<47:50, 57.40s/it] (TaskRunner pid=2425326) step:51 - global_seqlen/min:20681 - global_seqlen/max:27967 - global_seqlen/minmax_diff:7286 - global_seqlen/balanced_min:23835 - global_seqlen/balanced_max:23840 - global_seqlen/mean:23838.25 - actor/entropy:0.1650850474834442 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2733956575393677 - training/rollout_probs_diff_mean:0.0027729461435228586 - training/rollout_probs_diff_std:0.009361513890326023 - training/rollout_actor_probs_pearson_corr:0.9989805221557617 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9999463558197021 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1035334864573088e-05 - rollout_corr/rollout_is_max:1.6391828060150146 - rollout_corr/rollout_is_min:0.492061048746109 - rollout_corr/rollout_is_std:0.02771829441189766 - rollout_corr/rollout_is_eff_sample_size:0.999232285966509 - rollout_corr/rollout_is_seq_mean:0.9999741315841675 - rollout_corr/rollout_is_seq_std:0.0017780638299882412 - rollout_corr/rollout_is_seq_max:1.0069881677627563 - rollout_corr/rollout_is_seq_min:0.992406964302063 - rollout_corr/rollout_is_seq_max_deviation:0.007593035697937012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1927359052933753) - rollout_corr/training_log_ppl:np.float64(0.17125490526814247) - rollout_corr/kl:np.float64(0.00048572005391089856) - rollout_corr/k3_kl:np.float64(0.0004628497603818005) - rollout_corr/rollout_ppl:np.float64(1.1921314564533532) - rollout_corr/rollout_log_ppl:np.float64(0.17076918519160245) - rollout_corr/log_ppl_diff:np.float64(0.00048572007654001936) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014220301018212922) - rollout_corr/log_ppl_diff_max:np.float64(0.009596168994903564) - rollout_corr/log_ppl_diff_min:np.float64(-0.006043359637260437) - rollout_corr/ppl_ratio:np.float64(1.0004876188468188) - rollout_corr/chi2_token:np.float64(0.0008928193710744381) - rollout_corr/chi2_seq:np.float64(0.36601235694251955) - actor/pg_loss:np.float64(0.0007545447442680597) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012426727698766626) - actor/ppo_kl:np.float64(4.306889947258696e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(8465.622773182346) - self_distillation/token_reweight_w_std:np.float64(146480.68177438865) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010926395189017) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02168405632255599) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.14392557181417942) - perf/mfu/actor:np.float64(0.053210625937775015) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.1166114807129) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029919552616775036 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0029919552616775036 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:353.9765625 - response_length/max:694.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:353.9765625 - response_length_non_aborted/max:694.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.96875 - prompt_length/max:519.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.313009023666382e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.405456904321909) - timing_s/agent_loop/generate_sequences/max:np.float64(6.5661417208611965) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.876830842025811) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.5661417208611965) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:684 - timing_s/gen:13.132729981094599 - timing_s/reward:0.05493359826505184 - timing_s/old_log_prob:3.1243450678884983 - timing_s/adv:0.5484253838658333 - timing_s/update_actor:23.13295418024063 - timing_s/step:40.11374968290329 - timing_s/stop_profile:0.00011517293751239777 - timing_per_token_ms/adv:0.0028757636564441247 - timing_per_token_ms/update_actor:0.12130165899468623 - timing_per_token_ms/gen:0.14492407668558785 - perf/total_num_tokens:190706 - perf/time_per_step:40.11374968290329 - perf/throughput:594.2663098922412 (TaskRunner pid=2425326) Training Progress: 51%|█████ | 51/100 [39:16<42:39, 52.23s/it] (TaskRunner pid=2425326) step:52 - global_seqlen/min:20929 - global_seqlen/max:28953 - global_seqlen/minmax_diff:8024 - global_seqlen/balanced_min:25185 - global_seqlen/balanced_max:25193 - global_seqlen/mean:25191.625 - actor/entropy:0.16280372440814972 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2151232361793518 - training/rollout_probs_diff_mean:0.0027323930989950895 - training/rollout_probs_diff_std:0.00925284344702959 - training/rollout_actor_probs_pearson_corr:0.9989708662033081 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.78515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999663829803467 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0168698281631805e-05 - rollout_corr/rollout_is_max:1.868242859840393 - rollout_corr/rollout_is_min:0.3074229955673218 - rollout_corr/rollout_is_std:0.027021415531635284 - rollout_corr/rollout_is_eff_sample_size:0.9992702568062312 - rollout_corr/rollout_is_seq_mean:0.9999210238456726 - rollout_corr/rollout_is_seq_std:0.0015943794278427958 - rollout_corr/rollout_is_seq_max:1.005804181098938 - rollout_corr/rollout_is_seq_min:0.9952024221420288 - rollout_corr/rollout_is_seq_max_deviation:0.005804181098937988 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1793708237819374) - rollout_corr/training_log_ppl:np.float64(0.16213442391745048) - rollout_corr/kl:np.float64(0.0005378835334433063) - rollout_corr/k3_kl:np.float64(0.0005385009871474722) - rollout_corr/rollout_ppl:np.float64(1.1787225706502795) - rollout_corr/rollout_log_ppl:np.float64(0.1615965386881726) - rollout_corr/log_ppl_diff:np.float64(0.0005378852292778902) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014847702332190238) - rollout_corr/log_ppl_diff_max:np.float64(0.00623573362827301) - rollout_corr/log_ppl_diff_min:np.float64(-0.004963576793670654) - rollout_corr/ppl_ratio:np.float64(1.000539786182344) - rollout_corr/chi2_token:np.float64(0.0010742407757788897) - rollout_corr/chi2_seq:np.float64(0.3871990798506886) - actor/pg_loss:np.float64(0.0007529193535447121) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003004613563462044) - actor/ppo_kl:np.float64(6.476392650078822e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78515625) - self_distillation/token_reweight_w_mean:np.float64(14713.069492256269) - self_distillation/token_reweight_w_std:np.float64(269632.1662454984) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998877602629364) - self_distillation/token_reweight_w_clip_frac:np.float64(0.030055424678721465) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.14222942106425762) - perf/mfu/actor:np.float64(0.056224778008784046) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.7989044189453) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:1 - critic/score/mean:0.78515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003757334081456065 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003757334081456065 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:384.14453125 - response_length/max:1083.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:384.14453125 - response_length_non_aborted/max:1083.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:403.09375 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010464712977409363 - timing_s/agent_loop/generate_sequences/min:np.float64(2.042516840621829) - timing_s/agent_loop/generate_sequences/max:np.float64(8.922996774315834) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.519758339592954) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.922996774315834) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:1083 - timing_s/gen:15.847122693434358 - timing_s/reward:0.06258169002830982 - timing_s/old_log_prob:3.0204358845949173 - timing_s/adv:0.5925211496651173 - timing_s/update_actor:23.307167975232005 - timing_s/step:42.91890803910792 - timing_s/stop_profile:0.00012935325503349304 - timing_per_token_ms/adv:0.0029400701109253435 - timing_per_token_ms/update_actor:0.11564938732233433 - timing_per_token_ms/gen:0.1611446161157031 - perf/total_num_tokens:201533 - perf/time_per_step:42.91890803910792 - perf/throughput:586.9586657946952 (TaskRunner pid=2425326) Training Progress: 52%|█████▏ | 52/100 [39:59<39:33, 49.46s/it] (TaskRunner pid=2425326) step:53 - global_seqlen/min:20480 - global_seqlen/max:24979 - global_seqlen/minmax_diff:4499 - global_seqlen/balanced_min:23009 - global_seqlen/balanced_max:23014 - global_seqlen/mean:23012.625 - actor/entropy:0.15226857364177704 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4149150848388672 - training/rollout_probs_diff_mean:0.0029795358423143625 - training/rollout_probs_diff_std:0.010255304165184498 - training/rollout_actor_probs_pearson_corr:0.9986225366592407 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.0000274181365967 - rollout_corr/rollout_is_ratio_fraction_high:2.4199891413445584e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:5.8867363929748535 - rollout_corr/rollout_is_min:0.612064003944397 - rollout_corr/rollout_is_std:0.028760967776179314 - rollout_corr/rollout_is_eff_sample_size:0.9991734904229588 - rollout_corr/rollout_is_seq_mean:0.9999820590019226 - rollout_corr/rollout_is_seq_std:0.0018618700560182333 - rollout_corr/rollout_is_seq_max:1.0082342624664307 - rollout_corr/rollout_is_seq_min:0.9929176568984985 - rollout_corr/rollout_is_seq_max_deviation:0.008234262466430664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1658900491893291) - rollout_corr/training_log_ppl:np.float64(0.1514768806300708) - rollout_corr/kl:np.float64(0.0005496431590059636) - rollout_corr/k3_kl:np.float64(0.0004694314374376063) - rollout_corr/rollout_ppl:np.float64(1.1652461695484817) - rollout_corr/rollout_log_ppl:np.float64(0.15092723770067096) - rollout_corr/log_ppl_diff:np.float64(0.0005496429293998517) - rollout_corr/log_ppl_abs_diff:np.float64(0.001543470520118717) - rollout_corr/log_ppl_diff_max:np.float64(0.0066513121128082275) - rollout_corr/log_ppl_diff_min:np.float64(-0.004514351487159729) - rollout_corr/ppl_ratio:np.float64(1.0005516416858882) - rollout_corr/chi2_token:np.float64(0.0007865177467465401) - rollout_corr/chi2_seq:np.float64(0.43467385205440223) - actor/pg_loss:np.float64(0.0009989301906898618) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015393912690342404) - actor/ppo_kl:np.float64(8.647114800908184e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(24308.17669938528) - self_distillation/token_reweight_w_std:np.float64(413428.8588547886) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998141932301223) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02757751938770525) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.17244473472237587) - perf/mfu/actor:np.float64(0.05212055127204503) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.67004013061523) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003796357777900994 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0003796357777900994 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:322.83203125 - response_length/max:668.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.83203125 - response_length_non_aborted/max:668.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:502.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00025482289493083954 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9840517472475767) - timing_s/agent_loop/generate_sequences/max:np.float64(6.118285588920116) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7894860992382746) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.118285588920116) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:492 - timing_s/agent_loop/slowest/response_length:668 - timing_s/gen:14.044371008872986 - timing_s/reward:0.06149355508387089 - timing_s/old_log_prob:3.120630817487836 - timing_s/adv:0.5551640838384628 - timing_s/update_actor:22.852618919685483 - timing_s/step:40.728004993870854 - timing_s/stop_profile:2.8777867555618286e-05 - timing_per_token_ms/adv:0.0030155408381185482 - timing_per_token_ms/update_actor:0.124130878809379 - timing_per_token_ms/gen:0.16993612449480291 - perf/total_num_tokens:184101 - perf/time_per_step:40.728004993870854 - perf/throughput:565.0319725570444 (TaskRunner pid=2425326) Training Progress: 53%|█████▎ | 53/100 [40:40<36:47, 46.97s/it] (TaskRunner pid=2425326) step:54 - global_seqlen/min:20279 - global_seqlen/max:27363 - global_seqlen/minmax_diff:7084 - global_seqlen/balanced_min:25193 - global_seqlen/balanced_max:25196 - global_seqlen/mean:25194.875 - actor/entropy:0.15839405357837677 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30164676904678345 - training/rollout_probs_diff_mean:0.0026747898664325476 - training/rollout_probs_diff_std:0.009149343706667423 - training/rollout_actor_probs_pearson_corr:0.9989473223686218 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:0.9999547600746155 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6425002813339233 - rollout_corr/rollout_is_min:0.5558498501777649 - rollout_corr/rollout_is_std:0.026692956686019897 - rollout_corr/rollout_is_eff_sample_size:0.9992879933909776 - rollout_corr/rollout_is_seq_mean:0.9999342560768127 - rollout_corr/rollout_is_seq_std:0.0015845621237531304 - rollout_corr/rollout_is_seq_max:1.0048959255218506 - rollout_corr/rollout_is_seq_min:0.9955595135688782 - rollout_corr/rollout_is_seq_max_deviation:0.004895925521850586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1783004244789481) - rollout_corr/training_log_ppl:np.float64(0.1593520959140733) - rollout_corr/kl:np.float64(0.0006535952649509014) - rollout_corr/k3_kl:np.float64(0.0004820261234499412) - rollout_corr/rollout_ppl:np.float64(1.177526906132698) - rollout_corr/rollout_log_ppl:np.float64(0.15869850096351001) - rollout_corr/log_ppl_diff:np.float64(0.0006535949505632743) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013726948818657547) - rollout_corr/log_ppl_diff_max:np.float64(0.0071489810943603516) - rollout_corr/log_ppl_diff_min:np.float64(-0.005629494786262512) - rollout_corr/ppl_ratio:np.float64(1.0006552611012012) - rollout_corr/chi2_token:np.float64(0.0006367254536598921) - rollout_corr/chi2_seq:np.float64(0.6323353212792426) - actor/pg_loss:np.float64(0.0005831223679706454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011379809757272596) - actor/ppo_kl:np.float64(0.00020858292008796298) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(9313.207618546905) - self_distillation/token_reweight_w_std:np.float64(180056.0983797087) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999447213485837) - self_distillation/token_reweight_w_clip_frac:np.float64(0.019204000534955412) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.10884904395788908) - perf/mfu/actor:np.float64(0.05665816464996518) - perf/max_memory_allocated_gb:np.float64(130.98326301574707) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.66030502319336) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007078859489411116 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007078859489411116 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:377.99609375 - response_length/max:910.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:377.99609375 - response_length_non_aborted/max:910.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:409.34375 - prompt_length/max:524.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.901178181171417e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2368022967129946) - timing_s/agent_loop/generate_sequences/max:np.float64(7.938743500038981) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.04109959355992) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.938743500038981) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:910 - timing_s/gen:14.82884312979877 - timing_s/reward:0.061949849128723145 - timing_s/old_log_prob:3.0097642112523317 - timing_s/adv:0.5438911225646734 - timing_s/update_actor:23.000701379030943 - timing_s/step:41.539462050423026 - timing_s/stop_profile:0.00016438402235507965 - timing_per_token_ms/adv:0.0026984214178710623 - timing_per_token_ms/update_actor:0.11411398835592032 - timing_per_token_ms/gen:0.15324277005382797 - perf/total_num_tokens:201559 - perf/time_per_step:41.539462050423026 - perf/throughput:606.5286779452509 (TaskRunner pid=2425326) Training Progress: 54%|█████▍ | 54/100 [41:22<34:46, 45.35s/it] (TaskRunner pid=2425326) step:55 - global_seqlen/min:22028 - global_seqlen/max:26786 - global_seqlen/minmax_diff:4758 - global_seqlen/balanced_min:24965 - global_seqlen/balanced_max:25170 - global_seqlen/mean:24992.625 - actor/entropy:0.15181651711463928 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17278850078582764 - training/rollout_probs_diff_mean:0.0025789199862629175 - training/rollout_probs_diff_std:0.008787320926785469 - training/rollout_actor_probs_pearson_corr:0.9990404844284058 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.9999983310699463 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0607941476337146e-05 - rollout_corr/rollout_is_max:1.6370457410812378 - rollout_corr/rollout_is_min:0.49754756689071655 - rollout_corr/rollout_is_std:0.02620846964418888 - rollout_corr/rollout_is_eff_sample_size:0.9993134685136027 - rollout_corr/rollout_is_seq_mean:0.9999988079071045 - rollout_corr/rollout_is_seq_std:0.0015600925544276834 - rollout_corr/rollout_is_seq_max:1.0062010288238525 - rollout_corr/rollout_is_seq_min:0.9956908226013184 - rollout_corr/rollout_is_seq_max_deviation:0.006201028823852539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1677626050077379) - rollout_corr/training_log_ppl:np.float64(0.15257516079873312) - rollout_corr/kl:np.float64(0.00042375086632562287) - rollout_corr/k3_kl:np.float64(0.00040278795792403344) - rollout_corr/rollout_ppl:np.float64(1.1672604512423277) - rollout_corr/rollout_log_ppl:np.float64(0.15215140884538414) - rollout_corr/log_ppl_diff:np.float64(0.0004237519533489831) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013195802093832754) - rollout_corr/log_ppl_diff_max:np.float64(0.005208849906921387) - rollout_corr/log_ppl_diff_min:np.float64(-0.00468212366104126) - rollout_corr/ppl_ratio:np.float64(1.000425181351602) - rollout_corr/chi2_token:np.float64(0.0007754352409392595) - rollout_corr/chi2_seq:np.float64(0.32551098824478686) - actor/pg_loss:np.float64(0.0008454262278974056) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.39435245122877e-05) - actor/ppo_kl:np.float64(6.83178090241654e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(6225.795452147257) - self_distillation/token_reweight_w_std:np.float64(108468.37706527536) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998256473336369) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01854172901221318) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.12622096296399832) - perf/mfu/actor:np.float64(0.0559628797720936) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.84178924560547) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006364764412865043 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006364764412865043 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:368.23828125 - response_length/max:1313.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:368.23828125 - response_length_non_aborted/max:1313.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.78125 - prompt_length/max:519.0 - prompt_length/min:271.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014849193394184113 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2017166949808598) - timing_s/agent_loop/generate_sequences/max:np.float64(9.992865791544318) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.0017511987898615) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.992865791544318) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:1313 - timing_s/gen:17.006494095548987 - timing_s/reward:0.060051338747143745 - timing_s/old_log_prob:3.0255694836378098 - timing_s/adv:0.5934303794056177 - timing_s/update_actor:22.865236539393663 - timing_s/step:43.64245236851275 - timing_s/stop_profile:0.00012048892676830292 - timing_per_token_ms/adv:0.002968027465130302 - timing_per_token_ms/update_actor:0.11435991887303586 - timing_per_token_ms/gen:0.1804038877631988 - perf/total_num_tokens:199941 - perf/time_per_step:43.64245236851275 - perf/throughput:572.6677499459616 (TaskRunner pid=2425326) Training Progress: 55%|█████▌ | 55/100 [42:06<33:38, 44.86s/it] (TaskRunner pid=2425326) step:56 - global_seqlen/min:19002 - global_seqlen/max:28125 - global_seqlen/minmax_diff:9123 - global_seqlen/balanced_min:24289 - global_seqlen/balanced_max:24294 - global_seqlen/mean:24291.875 - actor/entropy:0.17489738762378693 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28977641463279724 - training/rollout_probs_diff_mean:0.0027698748745024204 - training/rollout_probs_diff_std:0.009292028844356537 - training/rollout_actor_probs_pearson_corr:0.9989330768585205 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:0.9998672604560852 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.068701542157214e-05 - rollout_corr/rollout_is_max:1.6370550394058228 - rollout_corr/rollout_is_min:0.3381432890892029 - rollout_corr/rollout_is_std:0.02769678272306919 - rollout_corr/rollout_is_eff_sample_size:0.9992333572045602 - rollout_corr/rollout_is_seq_mean:0.9998617768287659 - rollout_corr/rollout_is_seq_std:0.0015884299064055085 - rollout_corr/rollout_is_seq_max:1.0066787004470825 - rollout_corr/rollout_is_seq_min:0.994590699672699 - rollout_corr/rollout_is_seq_max_deviation:0.0066787004470825195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1882359734736383) - rollout_corr/training_log_ppl:np.float64(0.16863422583264764) - rollout_corr/kl:np.float64(0.0004804436645375887) - rollout_corr/k3_kl:np.float64(0.00045078001338083595) - rollout_corr/rollout_ppl:np.float64(1.1876508300192654) - rollout_corr/rollout_log_ppl:np.float64(0.16815378133469494) - rollout_corr/log_ppl_diff:np.float64(0.0004804444979527034) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013180674068280496) - rollout_corr/log_ppl_diff_max:np.float64(0.005176126956939697) - rollout_corr/log_ppl_diff_min:np.float64(-0.006251998245716095) - rollout_corr/ppl_ratio:np.float64(1.0004819862078875) - rollout_corr/chi2_token:np.float64(0.0008565529715269804) - rollout_corr/chi2_seq:np.float64(0.26315169618465006) - actor/pg_loss:np.float64(0.0010086237452924252) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010706515604397282) - actor/ppo_kl:np.float64(-6.85965127367183e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(18737.445998198586) - self_distillation/token_reweight_w_std:np.float64(351841.7787493873) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000774189829826) - self_distillation/token_reweight_w_clip_frac:np.float64(0.019534573119017296) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.17343188263475895) - perf/mfu/actor:np.float64(0.05484556116620652) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.69879531860352) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003864593105390668 - critic/advantages/max:0.75 - critic/advantages/min:-0.625 - critic/returns/mean:0.003864593105390668 - critic/returns/max:0.75 - critic/returns/min:-0.625 - response_length/mean:377.65234375 - response_length/max:863.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:377.65234375 - response_length_non_aborted/max:863.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:381.46875 - prompt_length/max:480.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010058470070362091 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7773322444409132) - timing_s/agent_loop/generate_sequences/max:np.float64(7.712428255006671) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.276553831899946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.712428255006671) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:863 - timing_s/gen:14.491283178329468 - timing_s/reward:0.06307733990252018 - timing_s/old_log_prob:3.145606856793165 - timing_s/adv:0.5667275041341782 - timing_s/update_actor:22.992940368130803 - timing_s/step:41.34694854542613 - timing_s/stop_profile:0.00013054534792900085 - timing_per_token_ms/adv:0.0029162400192151603 - timing_per_token_ms/update_actor:0.11831600261471584 - timing_per_token_ms/gen:0.14989070199660182 - perf/total_num_tokens:194335 - perf/time_per_step:41.34694854542613 - perf/throughput:587.5131262301389 (TaskRunner pid=2425326) Training Progress: 56%|█████▌ | 56/100 [42:47<32:08, 43.82s/it] (TaskRunner pid=2425326) step:57 - global_seqlen/min:22180 - global_seqlen/max:26730 - global_seqlen/minmax_diff:4550 - global_seqlen/balanced_min:24761 - global_seqlen/balanced_max:24766 - global_seqlen/mean:24764.375 - actor/entropy:0.18208277225494385 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.380797803401947 - training/rollout_probs_diff_mean:0.0030766604468226433 - training/rollout_probs_diff_std:0.009836580604314804 - training/rollout_actor_probs_pearson_corr:0.9989367127418518 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.000044822692871 - rollout_corr/rollout_is_ratio_fraction_high:1.0690728231566027e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.1381456463132054e-05 - rollout_corr/rollout_is_max:4.1945343017578125 - rollout_corr/rollout_is_min:0.13594895601272583 - rollout_corr/rollout_is_std:0.028684185817837715 - rollout_corr/rollout_is_eff_sample_size:0.9991777748846973 - rollout_corr/rollout_is_seq_mean:1.000114917755127 - rollout_corr/rollout_is_seq_std:0.0016589241567999125 - rollout_corr/rollout_is_seq_max:1.0065020322799683 - rollout_corr/rollout_is_seq_min:0.9957323670387268 - rollout_corr/rollout_is_seq_max_deviation:0.006502032279968262 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1985374288633466) - rollout_corr/training_log_ppl:np.float64(0.17924344843777362) - rollout_corr/kl:np.float64(0.0005058041924144163) - rollout_corr/k3_kl:np.float64(0.000529215224418067) - rollout_corr/rollout_ppl:np.float64(1.197921092621982) - rollout_corr/rollout_log_ppl:np.float64(0.17873764344403753) - rollout_corr/log_ppl_diff:np.float64(0.000505804993736092) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013525268659577705) - rollout_corr/log_ppl_diff_max:np.float64(0.005043789744377136) - rollout_corr/log_ppl_diff_min:np.float64(-0.0049214959144592285) - rollout_corr/ppl_ratio:np.float64(1.0005072951316833) - rollout_corr/chi2_token:np.float64(0.0011630894150584936) - rollout_corr/chi2_seq:np.float64(0.3350441094953567) - actor/pg_loss:np.float64(0.0012816599337384105) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018717671855483786) - actor/ppo_kl:np.float64(0.00018078757759099062) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(18.977482604794204) - self_distillation/token_reweight_w_std:np.float64(325.0625329564209) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999024558346719) - self_distillation/token_reweight_w_clip_frac:np.float64(0.027874489809619263) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.15066221542656422) - perf/mfu/actor:np.float64(0.054658498083677315) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.6921157836914) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004766728263348341 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004766728263348341 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:365.38671875 - response_length/max:790.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:365.38671875 - response_length_non_aborted/max:790.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:408.5 - prompt_length/max:499.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.738095104694366e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0642375592142344) - timing_s/agent_loop/generate_sequences/max:np.float64(7.1702330857515335) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.312129656034813) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.1702330857515335) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:784 - timing_s/gen:14.236141776666045 - timing_s/reward:0.06245327927172184 - timing_s/old_log_prob:3.2373924870043993 - timing_s/adv:0.5536701437085867 - timing_s/update_actor:23.505518032237887 - timing_s/step:41.68215238675475 - timing_s/stop_profile:0.00012402236461639404 - timing_per_token_ms/adv:0.0027946906781848254 - timing_per_token_ms/update_actor:0.11864582708143193 - timing_per_token_ms/gen:0.1521947185309448 - perf/total_num_tokens:198115 - perf/time_per_step:41.68215238675475 - perf/throughput:594.1241894185225 (TaskRunner pid=2425326) Training Progress: 57%|█████▋ | 57/100 [43:29<30:57, 43.20s/it] (TaskRunner pid=2425326) step:58 - global_seqlen/min:22489 - global_seqlen/max:27290 - global_seqlen/minmax_diff:4801 - global_seqlen/balanced_min:24464 - global_seqlen/balanced_max:24473 - global_seqlen/mean:24470.125 - actor/entropy:0.16469363868236542 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.244918555021286 - training/rollout_probs_diff_mean:0.0026008158456534147 - training/rollout_probs_diff_std:0.008907139301300049 - training/rollout_actor_probs_pearson_corr:0.9990557432174683 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.8046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9998775720596313 - rollout_corr/rollout_is_ratio_fraction_high:2.0434645193745382e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.0434645193745382e-05 - rollout_corr/rollout_is_max:3.076564073562622 - rollout_corr/rollout_is_min:5.316741589922458e-05 - rollout_corr/rollout_is_std:0.026616927236318588 - rollout_corr/rollout_is_eff_sample_size:0.9992918026725187 - rollout_corr/rollout_is_seq_mean:0.9999071955680847 - rollout_corr/rollout_is_seq_std:0.001745687099173665 - rollout_corr/rollout_is_seq_max:1.0125819444656372 - rollout_corr/rollout_is_seq_min:0.9947338700294495 - rollout_corr/rollout_is_seq_max_deviation:0.012581944465637207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1783719179220498) - rollout_corr/training_log_ppl:np.float64(0.16035878341062926) - rollout_corr/kl:np.float64(0.0006153967960749185) - rollout_corr/k3_kl:np.float64(0.00047639863944937133) - rollout_corr/rollout_ppl:np.float64(1.177631719969213) - rollout_corr/rollout_log_ppl:np.float64(0.1597433851493406) - rollout_corr/log_ppl_diff:np.float64(0.0006153982612886466) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013544214962166734) - rollout_corr/log_ppl_diff_max:np.float64(0.017868779599666595) - rollout_corr/log_ppl_diff_min:np.float64(-0.0059568434953689575) - rollout_corr/ppl_ratio:np.float64(1.0006175043527037) - rollout_corr/chi2_token:np.float64(0.0005664050113409758) - rollout_corr/chi2_seq:np.float64(0.14223428606055677) - actor/pg_loss:np.float64(0.00035006296820938587) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.173735679723904e-05) - actor/ppo_kl:np.float64(6.755156556303987e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_w_mean:np.float64(2222.0714181936346) - self_distillation/token_reweight_w_std:np.float64(39239.9520744278) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001498765777797) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01538468204671517) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.09138654079288244) - perf/mfu/actor:np.float64(0.055187680810946424) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.6977996826172) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0035351936239749193 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0035351936239749193 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.31640625 - response_length/max:942.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.31640625 - response_length_non_aborted/max:942.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:382.375 - prompt_length/max:482.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.19140875339508e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8807989470660686) - timing_s/agent_loop/generate_sequences/max:np.float64(7.688587138429284) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.266820297380036) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.688587138429284) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:942 - timing_s/gen:14.974682755768299 - timing_s/reward:0.06177697144448757 - timing_s/old_log_prob:3.3405462689697742 - timing_s/adv:0.5724097155034542 - timing_s/update_actor:23.087911941111088 - timing_s/step:42.124475156888366 - timing_s/stop_profile:0.00010747089982032776 - timing_per_token_ms/adv:0.0029240232503075394 - timing_per_token_ms/update_actor:0.11793928280459891 - timing_per_token_ms/gen:0.15300116227936508 - perf/total_num_tokens:195761 - perf/time_per_step:42.124475156888366 - perf/throughput:580.9004126191123 (TaskRunner pid=2425326) Training Progress: 58%|█████▊ | 58/100 [44:11<30:01, 42.88s/it] (TaskRunner pid=2425326) step:59 - global_seqlen/min:20196 - global_seqlen/max:26982 - global_seqlen/minmax_diff:6786 - global_seqlen/balanced_min:23296 - global_seqlen/balanced_max:23301 - global_seqlen/mean:23299.125 - actor/entropy:0.16596408188343048 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163905024528503 - training/rollout_probs_diff_mean:0.0027721028309315443 - training/rollout_probs_diff_std:0.009304988197982311 - training/rollout_actor_probs_pearson_corr:0.9989809989929199 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0000334978103638 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1246948815823998e-05 - rollout_corr/rollout_is_max:1.9688125848770142 - rollout_corr/rollout_is_min:0.4614606499671936 - rollout_corr/rollout_is_std:0.027634304016828537 - rollout_corr/rollout_is_eff_sample_size:0.9992369280146525 - rollout_corr/rollout_is_seq_mean:1.0000215768814087 - rollout_corr/rollout_is_seq_std:0.0015595081495121121 - rollout_corr/rollout_is_seq_max:1.004768967628479 - rollout_corr/rollout_is_seq_min:0.9953278303146362 - rollout_corr/rollout_is_seq_max_deviation:0.004768967628479004 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1875210818834603) - rollout_corr/training_log_ppl:np.float64(0.1683449064294109) - rollout_corr/kl:np.float64(0.0006163650541246568) - rollout_corr/k3_kl:np.float64(0.0004912084286274876) - rollout_corr/rollout_ppl:np.float64(1.1867791321128607) - rollout_corr/rollout_log_ppl:np.float64(0.16772853956354083) - rollout_corr/log_ppl_diff:np.float64(0.0006163668658700772) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014313266074168496) - rollout_corr/log_ppl_diff_max:np.float64(0.0071097612380981445) - rollout_corr/log_ppl_diff_min:np.float64(-0.004202127456665039) - rollout_corr/ppl_ratio:np.float64(1.0006180678028613) - rollout_corr/chi2_token:np.float64(0.000745030352845788) - rollout_corr/chi2_seq:np.float64(0.12269318685866892) - actor/pg_loss:np.float64(0.0005168186035007238) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015859473069212981) - actor/ppo_kl:np.float64(0.00022523770326543513) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(8113.411336590536) - self_distillation/token_reweight_w_std:np.float64(123141.12176522095) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997582165524364) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015467898192582652) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.1160267498344183) - perf/mfu/actor:np.float64(0.05294808405288777) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.59362411499023) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011978000402450562 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011978000402450562 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:347.31640625 - response_length/max:824.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:347.31640625 - response_length_non_aborted/max:824.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:380.78125 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010782480239868164 - timing_s/agent_loop/generate_sequences/min:np.float64(1.891933923587203) - timing_s/agent_loop/generate_sequences/max:np.float64(7.28188518807292) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9935646541052847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.28188518807292) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:824 - timing_s/gen:14.648240432143211 - timing_s/reward:0.0598688330501318 - timing_s/old_log_prob:3.206283261999488 - timing_s/adv:0.5356909520924091 - timing_s/update_actor:22.838558562099934 - timing_s/step:41.37533868290484 - timing_s/stop_profile:0.00011219829320907593 - timing_per_token_ms/adv:0.002873986427024669 - timing_per_token_ms/update_actor:0.12252905721835011 - timing_per_token_ms/gen:0.1647480169620102 - perf/total_num_tokens:186393 - perf/time_per_step:41.37533868290484 - perf/throughput:563.1162364267622 (TaskRunner pid=2425326) Training Progress: 59%|█████▉ | 59/100 [44:52<29:00, 42.45s/it] (TaskRunner pid=2425326) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2425326) validation generation end (TaskRunner pid=2425326) [prompt] system (TaskRunner pid=2425326) (TaskRunner pid=2425326) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2425326) (TaskRunner pid=2425326) A (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) user (TaskRunner pid=2425326) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2425326) (TaskRunner pid=2425326) Lattice Parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) a = 7.82 Å (TaskRunner pid=2425326) b = 7.82 Å (TaskRunner pid=2425326) c = 17.88 Å (TaskRunner pid=2425326) α = 90.00º (TaskRunner pid=2425326) β = 90.00º (TaskRunner pid=2425326) γ = 90.00º (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) A: 1025.67 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) B: 1094.31 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) C: 1200.45 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) D: 1150.78 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) Please reason step by step. (TaskRunner pid=2425326) assistant (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) [response] (TaskRunner pid=2425326) The given crystal structure has orthorhombic symmetry, as all angles are 90 degrees. The volume of a unit cell for an orthorhombic system is calculated using the formula: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ V = a \times b \times c $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Substituting the given lattice parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Now multiply by 17.88: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Thus, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) B (TaskRunner pid=2425326) (TaskRunner pid=2425326) [ground_truth] B (TaskRunner pid=2425326) [score] 1.0 (TaskRunner pid=2425326) [acc] 1.0 (TaskRunner pid=2425326) [pred] B (TaskRunner pid=2425326) [incorrect_format] 1 (TaskRunner pid=2425326) [feedback] (TaskRunner pid=2425326) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (TaskRunner pid=2425326) step:60 - global_seqlen/min:20976 - global_seqlen/max:28367 - global_seqlen/minmax_diff:7391 - global_seqlen/balanced_min:24354 - global_seqlen/balanced_max:24362 - global_seqlen/mean:24358.5 - actor/entropy:0.1625373214483261 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22152966260910034 - training/rollout_probs_diff_mean:0.002746312413364649 - training/rollout_probs_diff_std:0.009204926900565624 - training/rollout_actor_probs_pearson_corr:0.9989629983901978 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:0.999926745891571 - rollout_corr/rollout_is_ratio_fraction_high:1.0308002856618259e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.767468214035034 - rollout_corr/rollout_is_min:0.6319364309310913 - rollout_corr/rollout_is_std:0.02688652276992798 - rollout_corr/rollout_is_eff_sample_size:0.9992776370535978 - rollout_corr/rollout_is_seq_mean:0.9998831152915955 - rollout_corr/rollout_is_seq_std:0.0015677715418860316 - rollout_corr/rollout_is_seq_max:1.0052909851074219 - rollout_corr/rollout_is_seq_min:0.9949941039085388 - rollout_corr/rollout_is_seq_max_deviation:0.005290985107421875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1889118873514235) - rollout_corr/training_log_ppl:np.float64(0.16804096282430692) - rollout_corr/kl:np.float64(0.0003125149191056664) - rollout_corr/k3_kl:np.float64(0.0004305376780280312) - rollout_corr/rollout_ppl:np.float64(1.1885348898358643) - rollout_corr/rollout_log_ppl:np.float64(0.16772844693332445) - rollout_corr/log_ppl_diff:np.float64(0.0003125158909824677) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013071187931927852) - rollout_corr/log_ppl_diff_max:np.float64(0.008523553609848022) - rollout_corr/log_ppl_diff_min:np.float64(-0.006315112113952637) - rollout_corr/ppl_ratio:np.float64(1.0003141060005873) - rollout_corr/chi2_token:np.float64(0.0011115602683275938) - rollout_corr/chi2_seq:np.float64(0.3905038656666875) - actor/pg_loss:np.float64(0.0006367913447320461) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.850907811895013e-05) - actor/ppo_kl:np.float64(-0.00019378854725826677) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(10045.848395209527) - self_distillation/token_reweight_w_std:np.float64(136172.85056805442) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999325245153159) - self_distillation/token_reweight_w_clip_frac:np.float64(0.016230103617999703) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.11269044876098633) - perf/mfu/actor:np.float64(0.055351362219646164) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.59489059448242) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7513297872340425) - val-aux/sciknoweval/reward/std@16:np.float64(0.058702087346603746) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7717765957446808) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.03301362979339628) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7308617021276597) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.0631627294474146) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7509361702127659) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.05809445095212909) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7822978723404256) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.018520896859933612) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7068191489361703) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.06571796147227824) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7589787234042554) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.03858306666407169) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7896489361702128) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.011801345891776433) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6781489361702129) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06317213833689335) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7614574468085106) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.020390976262651492) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7941382978723405) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.00627330968059728) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6489468085106382) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.050747410997637604) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7610957446808512) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.01389893233797774) - val-aux/sciknoweval/score/mean@16:np.float64(0.7513297872340425) - val-aux/sciknoweval/score/std@16:np.float64(0.058702087346603746) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7717765957446808) - val-aux/sciknoweval/score/best@2/std:np.float64(0.03301362979339628) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7308617021276597) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.0631627294474146) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7509361702127659) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.05809445095212909) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7822978723404256) - val-aux/sciknoweval/score/best@4/std:np.float64(0.018520896859933612) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7068191489361703) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.06571796147227824) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7589787234042554) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.03858306666407169) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7896489361702128) - val-aux/sciknoweval/score/best@8/std:np.float64(0.011801345891776433) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6781489361702129) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06317213833689335) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7614574468085106) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.020390976262651492) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7941382978723405) - val-aux/sciknoweval/score/best@16/std:np.float64(0.00627330968059728) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6489468085106382) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.050747410997637604) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7610957446808512) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.01389893233797774) - val-core/sciknoweval/acc/mean@16:np.float64(0.7513297872340425) - val-aux/sciknoweval/acc/std@16:np.float64(0.058702087346603746) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7717765957446808) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.03301362979339628) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7308617021276597) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.0631627294474146) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7509361702127659) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.05809445095212909) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7822978723404256) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.018520896859933612) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7068191489361703) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.06571796147227824) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7589787234042554) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.03858306666407169) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7896489361702128) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.011801345891776433) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6781489361702129) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06317213833689335) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7614574468085106) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.020390976262651492) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7941382978723405) - val-core/sciknoweval/acc/best@16/std:np.float64(0.00627330968059728) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6489468085106382) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.050747410997637604) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7610957446808512) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.01389893233797774) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017484950367361307 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017484950367361307 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:378.953125 - response_length/max:720.0 - response_length/min:158.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:378.953125 - response_length_non_aborted/max:720.0 - response_length_non_aborted/min:158.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:382.25 - prompt_length/max:495.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013697519898414612 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0377895571291447) - timing_s/agent_loop/generate_sequences/max:np.float64(6.957599924877286) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.499870926862059) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.957599924877286) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:438 - timing_s/agent_loop/slowest/response_length:720 - timing_s/gen:14.019913736730814 - timing_s/reward:0.058817384764552116 - timing_s/old_log_prob:3.295192502439022 - timing_s/adv:0.5394449178129435 - timing_s/update_actor:22.97038963623345 - timing_s/step:40.96161608584225 - timing_s/testing:42.28644065745175 - timing_s/save_checkpoint:13.279354803264141 - timing_s/stop_profile:0.00018284469842910767 - timing_per_token_ms/adv:0.00276825809169768 - timing_per_token_ms/update_actor:0.11787666336306345 - timing_per_token_ms/gen:0.1445173147314849 - perf/total_num_tokens:194868 - perf/time_per_step:40.96161608584225 - perf/throughput:594.6664787090551 (TaskRunner pid=2425326) Training Progress: 60%|██████ | 60/100 [46:29<39:07, 58.69s/it] (TaskRunner pid=2425326) step:61 - global_seqlen/min:20942 - global_seqlen/max:26823 - global_seqlen/minmax_diff:5881 - global_seqlen/balanced_min:25153 - global_seqlen/balanced_max:25161 - global_seqlen/mean:25157.875 - actor/entropy:0.16813786327838898 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24496299028396606 - training/rollout_probs_diff_mean:0.002785650547593832 - training/rollout_probs_diff_std:0.009039455093443394 - training/rollout_actor_probs_pearson_corr:0.9989648461341858 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.80859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000181198120117 - rollout_corr/rollout_is_ratio_fraction_high:1.046649140334921e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.093298280669842e-05 - rollout_corr/rollout_is_max:2.5879404544830322 - rollout_corr/rollout_is_min:0.23645037412643433 - rollout_corr/rollout_is_std:0.026822155341506004 - rollout_corr/rollout_is_eff_sample_size:0.9992810891422065 - rollout_corr/rollout_is_seq_mean:0.9999784231185913 - rollout_corr/rollout_is_seq_std:0.0016213229391723871 - rollout_corr/rollout_is_seq_max:1.007311224937439 - rollout_corr/rollout_is_seq_min:0.9955859780311584 - rollout_corr/rollout_is_seq_max_deviation:0.007311224937438965 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1883001108653843) - rollout_corr/training_log_ppl:np.float64(0.17076641073072096) - rollout_corr/kl:np.float64(0.00046844173982929505) - rollout_corr/k3_kl:np.float64(0.00041541116644339127) - rollout_corr/rollout_ppl:np.float64(1.1877418737858534) - rollout_corr/rollout_log_ppl:np.float64(0.17029796764836647) - rollout_corr/log_ppl_diff:np.float64(0.00046844308235449716) - rollout_corr/log_ppl_abs_diff:np.float64(0.001244591076101642) - rollout_corr/log_ppl_diff_max:np.float64(0.005107656121253967) - rollout_corr/log_ppl_diff_min:np.float64(-0.006342723965644836) - rollout_corr/ppl_ratio:np.float64(1.000469823135063) - rollout_corr/chi2_token:np.float64(0.0007278069388121367) - rollout_corr/chi2_seq:np.float64(0.3579556457698345) - actor/pg_loss:np.float64(0.0007300071883946657) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.12041867498192e-05) - actor/ppo_kl:np.float64(5.782134989851251e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_w_mean:np.float64(3637.0788466231897) - self_distillation/token_reweight_w_std:np.float64(82982.7309890319) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997178621124476) - self_distillation/token_reweight_w_clip_frac:np.float64(0.020233669521985576) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.11315641738474369) - perf/mfu/actor:np.float64(0.056785998846264056) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.96517181396484) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031621886882930994 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031621886882930994 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:373.21484375 - response_length/max:745.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:373.21484375 - response_length_non_aborted/max:745.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.96875 - prompt_length/max:509.0 - prompt_length/min:269.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013932399451732635 - timing_s/agent_loop/generate_sequences/min:np.float64(1.379575191065669) - timing_s/agent_loop/generate_sequences/max:np.float64(7.066107423976064) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.400248988589738) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.066107423976064) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:470 - timing_s/agent_loop/slowest/response_length:745 - timing_s/gen:13.82433844730258 - timing_s/reward:0.06097260117530823 - timing_s/old_log_prob:3.280380440875888 - timing_s/adv:0.6567479409277439 - timing_s/update_actor:22.96429688669741 - timing_s/step:40.8726696446538 - timing_s/stop_profile:0.0001276899129152298 - timing_per_token_ms/adv:0.00326313301961982 - timing_per_token_ms/update_actor:0.11410093701622957 - timing_per_token_ms/gen:0.1446923212302584 - perf/total_num_tokens:201263 - perf/time_per_step:40.8726696446538 - perf/throughput:615.5182722029679 (TaskRunner pid=2425326) Training Progress: 61%|██████ | 61/100 [47:10<34:41, 53.37s/it] (TaskRunner pid=2425326) step:62 - global_seqlen/min:19030 - global_seqlen/max:27927 - global_seqlen/minmax_diff:8897 - global_seqlen/balanced_min:24956 - global_seqlen/balanced_max:24968 - global_seqlen/mean:24964.875 - actor/entropy:0.15622001886367798 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4004977345466614 - training/rollout_probs_diff_mean:0.0024245709646493196 - training/rollout_probs_diff_std:0.008639764040708542 - training/rollout_actor_probs_pearson_corr:0.9990466833114624 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.8515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999725222587585 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.962640433514025e-06 - rollout_corr/rollout_is_max:1.712405800819397 - rollout_corr/rollout_is_min:0.44539976119995117 - rollout_corr/rollout_is_std:0.025484351441264153 - rollout_corr/rollout_is_eff_sample_size:0.9993509693049463 - rollout_corr/rollout_is_seq_mean:0.9999911785125732 - rollout_corr/rollout_is_seq_std:0.0014212263049557805 - rollout_corr/rollout_is_seq_max:1.004982829093933 - rollout_corr/rollout_is_seq_min:0.9956592321395874 - rollout_corr/rollout_is_seq_max_deviation:0.0049828290939331055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1682638903148472) - rollout_corr/training_log_ppl:np.float64(0.15048128882335732) - rollout_corr/kl:np.float64(0.000456686800131223) - rollout_corr/k3_kl:np.float64(0.0003724734151830944) - rollout_corr/rollout_ppl:np.float64(1.1677364106290042) - rollout_corr/rollout_log_ppl:np.float64(0.15002460227697156) - rollout_corr/log_ppl_diff:np.float64(0.0004566865463857539) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011863850013469346) - rollout_corr/log_ppl_diff_max:np.float64(0.007962048053741455) - rollout_corr/log_ppl_diff_min:np.float64(-0.004445493221282959) - rollout_corr/ppl_ratio:np.float64(1.0004580072127283) - rollout_corr/chi2_token:np.float64(0.0005830880254507065) - rollout_corr/chi2_seq:np.float64(0.4403882585465908) - actor/pg_loss:np.float64(0.0009687773417681456) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.318608908870374e-05) - actor/ppo_kl:np.float64(0.00010262198350829976) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8515625) - self_distillation/token_reweight_w_mean:np.float64(44.64393255230971) - self_distillation/token_reweight_w_std:np.float64(657.1197780608782) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001448588445783) - self_distillation/token_reweight_w_clip_frac:np.float64(0.021430096472613513) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.09477861924096942) - perf/mfu/actor:np.float64(0.05646728138609545) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.61653518676758) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.8515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0025554171297699213 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0025554171297699213 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:392.08984375 - response_length/max:912.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:392.08984375 - response_length_non_aborted/max:912.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.0625 - prompt_length/max:541.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015456043183803558 - timing_s/agent_loop/generate_sequences/min:np.float64(1.488322202116251) - timing_s/agent_loop/generate_sequences/max:np.float64(8.23784239590168) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.584870089318429) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.23784239590168) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:912 - timing_s/gen:15.623099455609918 - timing_s/reward:0.062247466295957565 - timing_s/old_log_prob:3.0489571429789066 - timing_s/adv:0.6091952454298735 - timing_s/update_actor:23.01017976924777 - timing_s/step:42.44651843607426 - timing_s/stop_profile:0.00012479349970817566 - timing_per_token_ms/adv:0.0030502618450416507 - timing_per_token_ms/update_actor:0.11521277279201163 - timing_per_token_ms/gen:0.15564731711691077 - perf/total_num_tokens:199719 - perf/time_per_step:42.44651843607426 - perf/throughput:588.1489441259561 (TaskRunner pid=2425326) Training Progress: 62%|██████▏ | 62/100 [47:52<31:44, 50.11s/it] (TaskRunner pid=2425326) step:63 - global_seqlen/min:21676 - global_seqlen/max:29352 - global_seqlen/minmax_diff:7676 - global_seqlen/balanced_min:26177 - global_seqlen/balanced_max:26180 - global_seqlen/mean:26178.375 - actor/entropy:0.18619155883789062 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797507882118225 - training/rollout_probs_diff_mean:0.002799584763124585 - training/rollout_probs_diff_std:0.008926978334784508 - training/rollout_actor_probs_pearson_corr:0.9991098642349243 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9999904632568359 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.512666110822465e-06 - rollout_corr/rollout_is_max:1.6014662981033325 - rollout_corr/rollout_is_min:0.3244990110397339 - rollout_corr/rollout_is_std:0.026437172666192055 - rollout_corr/rollout_is_eff_sample_size:0.9993015640890296 - rollout_corr/rollout_is_seq_mean:0.9999660849571228 - rollout_corr/rollout_is_seq_std:0.0015565672656521201 - rollout_corr/rollout_is_seq_max:1.0054783821105957 - rollout_corr/rollout_is_seq_min:0.9923509359359741 - rollout_corr/rollout_is_seq_max_deviation:0.007649064064025879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2034375509247184) - rollout_corr/training_log_ppl:np.float64(0.1827049713829183) - rollout_corr/kl:np.float64(0.0005114174473841437) - rollout_corr/k3_kl:np.float64(0.00042598485559608434) - rollout_corr/rollout_ppl:np.float64(1.2028177841566503) - rollout_corr/rollout_log_ppl:np.float64(0.18219355211476795) - rollout_corr/log_ppl_diff:np.float64(0.0005114192681503482) - rollout_corr/log_ppl_abs_diff:np.float64(0.001406845221936237) - rollout_corr/log_ppl_diff_max:np.float64(0.008712932467460632) - rollout_corr/log_ppl_diff_min:np.float64(-0.005025111138820648) - rollout_corr/ppl_ratio:np.float64(1.0005130991339684) - rollout_corr/chi2_token:np.float64(0.0006853425875306129) - rollout_corr/chi2_seq:np.float64(0.5162155868019909) - actor/pg_loss:np.float64(0.0013690560590475798) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002309702149432269) - actor/ppo_kl:np.float64(0.00011990283803520185) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(3051.147950245766) - self_distillation/token_reweight_w_std:np.float64(45859.22248870792) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996626949869096) - self_distillation/token_reweight_w_clip_frac:np.float64(0.040024787274887785) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.14891982078552246) - perf/mfu/actor:np.float64(0.059230344477378155) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.58012104034424) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001120116445235908 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001120116445235908 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:410.63671875 - response_length/max:742.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:410.63671875 - response_length_non_aborted/max:742.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.4375 - prompt_length/max:483.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015700235962867737 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4416334554553032) - timing_s/agent_loop/generate_sequences/max:np.float64(6.933473754674196) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.507300539124117) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.933473754674196) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:459 - timing_s/agent_loop/slowest/response_length:647 - timing_s/gen:14.023673828691244 - timing_s/reward:0.062430648133158684 - timing_s/old_log_prob:2.9977916087955236 - timing_s/adv:0.6528902146965265 - timing_s/update_actor:22.98061357997358 - timing_s/step:40.80569167621434 - timing_s/stop_profile:0.00015737488865852356 - timing_per_token_ms/adv:0.0031175073638858722 - timing_per_token_ms/update_actor:0.1097309018415657 - timing_per_token_ms/gen:0.13340252683705034 - perf/total_num_tokens:209427 - perf/time_per_step:40.80569167621434 - perf/throughput:641.5373425776138 (TaskRunner pid=2425326) Training Progress: 63%|██████▎ | 63/100 [48:33<29:11, 47.34s/it] (TaskRunner pid=2425326) step:64 - global_seqlen/min:21649 - global_seqlen/max:29898 - global_seqlen/minmax_diff:8249 - global_seqlen/balanced_min:25438 - global_seqlen/balanced_max:25443 - global_seqlen/mean:25441.875 - actor/entropy:0.17299145460128784 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19256961345672607 - training/rollout_probs_diff_mean:0.0027170064859092236 - training/rollout_probs_diff_std:0.008848386816680431 - training/rollout_actor_probs_pearson_corr:0.9989994168281555 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.80078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999280571937561 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6264592409133911 - rollout_corr/rollout_is_min:0.6363797187805176 - rollout_corr/rollout_is_std:0.02642815187573433 - rollout_corr/rollout_is_eff_sample_size:0.9993019212176399 - rollout_corr/rollout_is_seq_mean:0.999889612197876 - rollout_corr/rollout_is_seq_std:0.001410823781043291 - rollout_corr/rollout_is_seq_max:1.0057334899902344 - rollout_corr/rollout_is_seq_min:0.9941492676734924 - rollout_corr/rollout_is_seq_max_deviation:0.005850732326507568 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1918034963309765) - rollout_corr/training_log_ppl:np.float64(0.1726833581342362) - rollout_corr/kl:np.float64(0.000396452359726851) - rollout_corr/k3_kl:np.float64(0.0003939119860660867) - rollout_corr/rollout_ppl:np.float64(1.1913265604525805) - rollout_corr/rollout_log_ppl:np.float64(0.17228690521005774) - rollout_corr/log_ppl_diff:np.float64(0.0003964529241784476) - rollout_corr/log_ppl_abs_diff:np.float64(0.001206114662636537) - rollout_corr/log_ppl_diff_max:np.float64(0.006472155451774597) - rollout_corr/log_ppl_diff_min:np.float64(-0.003871813416481018) - rollout_corr/ppl_ratio:np.float64(1.0003976812586188) - rollout_corr/chi2_token:np.float64(0.0007929231505841017) - rollout_corr/chi2_seq:np.float64(0.31985570839606225) - actor/pg_loss:np.float64(0.0005421672249212861) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.674097307448392e-05) - actor/ppo_kl:np.float64(-8.614134704387766e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_w_mean:np.float64(17.55605116346851) - self_distillation/token_reweight_w_std:np.float64(375.6963393909973) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002221886534244) - self_distillation/token_reweight_w_clip_frac:np.float64(0.016844158904859796) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.07351323496550322) - perf/mfu/actor:np.float64(0.05771520629925124) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.62054443359375) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.80078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016226072330027819 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016226072330027819 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:403.83984375 - response_length/max:802.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:403.83984375 - response_length_non_aborted/max:802.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.21875 - prompt_length/max:473.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.014745354652405e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.129437470808625) - timing_s/agent_loop/generate_sequences/max:np.float64(8.003116760402918) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.911897071731801) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.003116760402918) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:802 - timing_s/gen:14.972860099747777 - timing_s/reward:0.0634386632591486 - timing_s/old_log_prob:3.0743416529148817 - timing_s/adv:0.5757270734757185 - timing_s/update_actor:22.89247501641512 - timing_s/step:41.66854612715542 - timing_s/stop_profile:0.00013096071779727936 - timing_per_token_ms/adv:0.0028286391700479943 - timing_per_token_ms/update_actor:0.11247439023467767 - timing_per_token_ms/gen:0.14482903475182357 - perf/total_num_tokens:203535 - perf/time_per_step:41.66854612715542 - perf/throughput:610.5774586509874 (TaskRunner pid=2425326) Training Progress: 64%|██████▍ | 64/100 [49:15<27:23, 45.65s/it] (TaskRunner pid=2425326) step:65 - global_seqlen/min:22005 - global_seqlen/max:30883 - global_seqlen/minmax_diff:8878 - global_seqlen/balanced_min:26219 - global_seqlen/balanced_max:26223 - global_seqlen/mean:26221.625 - actor/entropy:0.1663258969783783 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3175744414329529 - training/rollout_probs_diff_mean:0.0026298724114894867 - training/rollout_probs_diff_std:0.008909679017961025 - training/rollout_actor_probs_pearson_corr:0.9990459680557251 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:0.9999517202377319 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9659714698791504 - rollout_corr/rollout_is_min:0.5378821492195129 - rollout_corr/rollout_is_std:0.0260442104190588 - rollout_corr/rollout_is_eff_sample_size:0.9993220398749199 - rollout_corr/rollout_is_seq_mean:0.9998959302902222 - rollout_corr/rollout_is_seq_std:0.0014111259952187538 - rollout_corr/rollout_is_seq_max:1.0054465532302856 - rollout_corr/rollout_is_seq_min:0.9965800642967224 - rollout_corr/rollout_is_seq_max_deviation:0.0054465532302856445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1869128532707691) - rollout_corr/training_log_ppl:np.float64(0.16875576815073146) - rollout_corr/kl:np.float64(0.0004878026736196972) - rollout_corr/k3_kl:np.float64(0.00036951068329926784) - rollout_corr/rollout_ppl:np.float64(1.1863231775350869) - rollout_corr/rollout_log_ppl:np.float64(0.16826796504756203) - rollout_corr/log_ppl_diff:np.float64(0.0004878031031694263) - rollout_corr/log_ppl_abs_diff:np.float64(0.001221742873894982) - rollout_corr/log_ppl_diff_max:np.float64(0.008714795112609863) - rollout_corr/log_ppl_diff_min:np.float64(-0.003134891390800476) - rollout_corr/ppl_ratio:np.float64(1.000489059370011) - rollout_corr/chi2_token:np.float64(0.0004872032441198826) - rollout_corr/chi2_seq:np.float64(0.24569943128153682) - actor/pg_loss:np.float64(0.0006585207302123308) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00013484467808666523) - actor/ppo_kl:np.float64(2.8466167123042396e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(62.919017291860655) - self_distillation/token_reweight_w_std:np.float64(1208.9836768980604) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000118033727631) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02347485371865332) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.20872666779905558) - perf/mfu/actor:np.float64(0.058806865577430006) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.7265510559082) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001335279201157391 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001335279201157391 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:420.89453125 - response_length/max:836.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.89453125 - response_length_non_aborted/max:836.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:398.53125 - prompt_length/max:490.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.106231689453125e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.083560824394226) - timing_s/agent_loop/generate_sequences/max:np.float64(8.249770537018776) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.025782150827581) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.249770537018776) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:835 - timing_s/gen:15.664854919537902 - timing_s/reward:0.06951326876878738 - timing_s/old_log_prob:3.027111055329442 - timing_s/adv:0.5791941955685616 - timing_s/update_actor:23.1634508240968 - timing_s/step:42.59725390560925 - timing_s/stop_profile:0.0001240260899066925 - timing_per_token_ms/adv:0.002761052163855985 - timing_per_token_ms/update_actor:0.1104215071724998 - timing_per_token_ms/gen:0.1453828334326806 - perf/total_num_tokens:209773 - perf/time_per_step:42.59725390560925 - perf/throughput:615.5707844008956 (TaskRunner pid=2425326) Training Progress: 65%|██████▌ | 65/100 [49:58<26:06, 44.76s/it] (TaskRunner pid=2425326) step:66 - global_seqlen/min:16342 - global_seqlen/max:28272 - global_seqlen/minmax_diff:11930 - global_seqlen/balanced_min:24902 - global_seqlen/balanced_max:24906 - global_seqlen/mean:24905.0 - actor/entropy:0.1453680694103241 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4400331377983093 - training/rollout_probs_diff_mean:0.002347266301512718 - training/rollout_probs_diff_std:0.008429394103586674 - training/rollout_actor_probs_pearson_corr:0.9990342259407043 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998712539672852 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.00136185210431e-05 - rollout_corr/rollout_is_max:1.6471433639526367 - rollout_corr/rollout_is_min:0.46178218722343445 - rollout_corr/rollout_is_std:0.024240102618932724 - rollout_corr/rollout_is_eff_sample_size:0.9994124052545171 - rollout_corr/rollout_is_seq_mean:0.9998443722724915 - rollout_corr/rollout_is_seq_std:0.001488116686232388 - rollout_corr/rollout_is_seq_max:1.0049697160720825 - rollout_corr/rollout_is_seq_min:0.9944682717323303 - rollout_corr/rollout_is_seq_max_deviation:0.005531728267669678 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1638081297278404) - rollout_corr/training_log_ppl:np.float64(0.14921010341640795) - rollout_corr/kl:np.float64(0.0003408923481164061) - rollout_corr/k3_kl:np.float64(0.00038058763544057683) - rollout_corr/rollout_ppl:np.float64(1.1633959328755736) - rollout_corr/rollout_log_ppl:np.float64(0.1488692093444115) - rollout_corr/log_ppl_diff:np.float64(0.00034089407199644484) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012122128500777762) - rollout_corr/log_ppl_diff_max:np.float64(0.006232038140296936) - rollout_corr/log_ppl_diff_min:np.float64(-0.009414836764335632) - rollout_corr/ppl_ratio:np.float64(1.0003424615133554) - rollout_corr/chi2_token:np.float64(0.0008226935751736164) - rollout_corr/chi2_seq:np.float64(0.423892357153818) - actor/pg_loss:np.float64(0.0008258491288870573) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014899678853907972) - actor/ppo_kl:np.float64(-0.00015697772187905912) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(5992.965957323788) - self_distillation/token_reweight_w_std:np.float64(127280.61096522707) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000441890442744) - self_distillation/token_reweight_w_clip_frac:np.float64(0.019730598985916004) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.12438435479998589) - perf/mfu/actor:np.float64(0.05638331722933918) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.728759765625) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004558699671179056 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.004558699671179056 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:390.09375 - response_length/max:933.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:390.09375 - response_length_non_aborted/max:933.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.1875 - prompt_length/max:475.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010139122605323792 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9046760816127062) - timing_s/agent_loop/generate_sequences/max:np.float64(8.32050434872508) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.469699847191805) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.32050434872508) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:933 - timing_s/gen:15.560051325708628 - timing_s/reward:0.059697385877370834 - timing_s/old_log_prob:3.0313059631735086 - timing_s/adv:0.5737353917211294 - timing_s/update_actor:23.060265477746725 - timing_s/step:42.379563361406326 - timing_s/stop_profile:0.0001245439052581787 - timing_per_token_ms/adv:0.0028796195127541125 - timing_per_token_ms/update_actor:0.11574114373492635 - timing_per_token_ms/gen:0.15581241814576452 - perf/total_num_tokens:199240 - perf/time_per_step:42.379563361406326 - perf/throughput:587.6653279226601 (TaskRunner pid=2425326) Training Progress: 66%|██████▌ | 66/100 [50:40<24:57, 44.06s/it] (TaskRunner pid=2425326) step:67 - global_seqlen/min:22521 - global_seqlen/max:26374 - global_seqlen/minmax_diff:3853 - global_seqlen/balanced_min:24313 - global_seqlen/balanced_max:24317 - global_seqlen/mean:24315.25 - actor/entropy:0.1448637843132019 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42377305030822754 - training/rollout_probs_diff_mean:0.0024852403439581394 - training/rollout_probs_diff_std:0.008906054310500622 - training/rollout_actor_probs_pearson_corr:0.99898362159729 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.80078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0000438690185547 - rollout_corr/rollout_is_ratio_fraction_high:1.043035626935307e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.086071253870614e-05 - rollout_corr/rollout_is_max:2.217512369155884 - rollout_corr/rollout_is_min:0.15181678533554077 - rollout_corr/rollout_is_std:0.025744972750544548 - rollout_corr/rollout_is_eff_sample_size:0.9993376353677934 - rollout_corr/rollout_is_seq_mean:1.0000419616699219 - rollout_corr/rollout_is_seq_std:0.001741582527756691 - rollout_corr/rollout_is_seq_max:1.0054556131362915 - rollout_corr/rollout_is_seq_min:0.9952270984649658 - rollout_corr/rollout_is_seq_max_deviation:0.005455613136291504 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1658252156339586) - rollout_corr/training_log_ppl:np.float64(0.15123848517396254) - rollout_corr/kl:np.float64(0.0003005595714675735) - rollout_corr/k3_kl:np.float64(0.00047102890214034687) - rollout_corr/rollout_ppl:np.float64(1.1654559667222202) - rollout_corr/rollout_log_ppl:np.float64(0.15093792550760554) - rollout_corr/log_ppl_diff:np.float64(0.00030055966635700315) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014758130128029734) - rollout_corr/log_ppl_diff_max:np.float64(0.006961077451705933) - rollout_corr/log_ppl_diff_min:np.float64(-0.006364375352859497) - rollout_corr/ppl_ratio:np.float64(1.0003026039339602) - rollout_corr/chi2_token:np.float64(0.001243996899574995) - rollout_corr/chi2_seq:np.float64(0.4573242797050625) - actor/pg_loss:np.float64(0.00016782758757472038) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(1.8803054445015732e-05) - actor/ppo_kl:np.float64(-3.549655894374837e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_w_mean:np.float64(162.33343863044865) - self_distillation/token_reweight_w_std:np.float64(3453.4582545155426) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000421507284045) - self_distillation/token_reweight_w_clip_frac:np.float64(0.004522646631812677) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.06182404700666666) - perf/mfu/actor:np.float64(0.05466950097703479) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.71934127807617) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:2 - critic/score/mean:0.80078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018396541709080338 - critic/advantages/max:0.375 - critic/advantages/min:-0.625 - critic/returns/mean:-0.0018396541709080338 - critic/returns/max:0.375 - critic/returns/min:-0.625 - response_length/mean:374.5078125 - response_length/max:809.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.5078125 - response_length_non_aborted/max:809.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.34375 - prompt_length/max:472.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014631636440753937 - timing_s/agent_loop/generate_sequences/min:np.float64(2.024880787357688) - timing_s/agent_loop/generate_sequences/max:np.float64(7.44964237511158) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.393396373881842) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.44964237511158) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:809 - timing_s/gen:14.722403204068542 - timing_s/reward:0.05925688520073891 - timing_s/old_log_prob:3.0161336902529 - timing_s/adv:0.5671027302742004 - timing_s/update_actor:22.998604083433747 - timing_s/step:41.44544195756316 - timing_s/stop_profile:0.00011837854981422424 - timing_per_token_ms/adv:0.0029153655127656533 - timing_per_token_ms/update_actor:0.11823137785666273 - timing_per_token_ms/gen:0.1535599140962987 - perf/total_num_tokens:194522 - perf/time_per_step:41.44544195756316 - perf/throughput:586.6809195784879 (TaskRunner pid=2425326) Training Progress: 67%|██████▋ | 67/100 [51:22<23:48, 43.29s/it] (TaskRunner pid=2425326) step:68 - global_seqlen/min:16271 - global_seqlen/max:28326 - global_seqlen/minmax_diff:12055 - global_seqlen/balanced_min:23376 - global_seqlen/balanced_max:23384 - global_seqlen/mean:23380.5 - actor/entropy:0.18183209002017975 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2554054856300354 - training/rollout_probs_diff_mean:0.0028197066858410835 - training/rollout_probs_diff_std:0.009083811193704605 - training/rollout_actor_probs_pearson_corr:0.9990743398666382 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.78125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999996423721313 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1522964743780904e-05 - rollout_corr/rollout_is_max:1.5750139951705933 - rollout_corr/rollout_is_min:0.4281967878341675 - rollout_corr/rollout_is_std:0.026538433507084846 - rollout_corr/rollout_is_eff_sample_size:0.9992963262320694 - rollout_corr/rollout_is_seq_mean:1.0001122951507568 - rollout_corr/rollout_is_seq_std:0.0016389826778322458 - rollout_corr/rollout_is_seq_max:1.0058777332305908 - rollout_corr/rollout_is_seq_min:0.9958250522613525 - rollout_corr/rollout_is_seq_max_deviation:0.00587773323059082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2043992574326694) - rollout_corr/training_log_ppl:np.float64(0.17893013266075286) - rollout_corr/kl:np.float64(0.0003652134504061877) - rollout_corr/k3_kl:np.float64(0.00045952229044843307) - rollout_corr/rollout_ppl:np.float64(1.2039373209699988) - rollout_corr/rollout_log_ppl:np.float64(0.17856491890051984) - rollout_corr/log_ppl_diff:np.float64(0.00036521376023301855) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012924449620186351) - rollout_corr/log_ppl_diff_max:np.float64(0.009212031960487366) - rollout_corr/log_ppl_diff_min:np.float64(-0.0048696547746658325) - rollout_corr/ppl_ratio:np.float64(1.0003667706623673) - rollout_corr/chi2_token:np.float64(0.0010798515286296606) - rollout_corr/chi2_seq:np.float64(0.2757666315883398) - actor/pg_loss:np.float64(0.0013036205200478435) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010660139287210768) - actor/ppo_kl:np.float64(9.799990320047414e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_w_mean:np.float64(6891.412626077421) - self_distillation/token_reweight_w_std:np.float64(127938.69730146386) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00010019633919) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02797325953724794) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.14852935820817947) - perf/mfu/actor:np.float64(0.053255143371093036) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.65814971923828) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:2 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00013586372369900346 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00013586372369900346 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:362.984375 - response_length/max:842.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:362.984375 - response_length_non_aborted/max:842.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:367.65625 - prompt_length/max:491.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.628571569919586e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1437002159655094) - timing_s/agent_loop/generate_sequences/max:np.float64(7.7901336420327425) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.292600261935149) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.7901336420327425) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:842 - timing_s/gen:14.902525736019015 - timing_s/reward:0.060505084693431854 - timing_s/old_log_prob:3.040873734280467 - timing_s/adv:0.5362718719989061 - timing_s/update_actor:22.70746410638094 - timing_s/step:41.33494069240987 - timing_s/stop_profile:3.4963712096214294e-05 - timing_per_token_ms/adv:0.0028670894121110868 - timing_per_token_ms/update_actor:0.1214017242273526 - timing_per_token_ms/gen:0.16037326994123172 - perf/total_num_tokens:187044 - perf/time_per_step:41.33494069240987 - perf/throughput:565.635261799063 (TaskRunner pid=2425326) Training Progress: 68%|██████▊ | 68/100 [52:03<22:46, 42.71s/it] (TaskRunner pid=2425326) step:69 - global_seqlen/min:18407 - global_seqlen/max:25750 - global_seqlen/minmax_diff:7343 - global_seqlen/balanced_min:23594 - global_seqlen/balanced_max:23599 - global_seqlen/mean:23596.875 - actor/entropy:0.16562606394290924 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491426348686218 - training/rollout_probs_diff_mean:0.0027516221161931753 - training/rollout_probs_diff_std:0.009353713132441044 - training/rollout_actor_probs_pearson_corr:0.9988866448402405 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999135136604309 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6370512247085571 - rollout_corr/rollout_is_min:0.5175263285636902 - rollout_corr/rollout_is_std:0.027063293382525444 - rollout_corr/rollout_is_eff_sample_size:0.9992679951393448 - rollout_corr/rollout_is_seq_mean:0.999840259552002 - rollout_corr/rollout_is_seq_std:0.0015955602284520864 - rollout_corr/rollout_is_seq_max:1.0040080547332764 - rollout_corr/rollout_is_seq_min:0.9931057691574097 - rollout_corr/rollout_is_seq_max_deviation:0.006894230842590332 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1826541190966964) - rollout_corr/training_log_ppl:np.float64(0.16283819011005107) - rollout_corr/kl:np.float64(0.0006027205091285737) - rollout_corr/k3_kl:np.float64(0.00046253268788376545) - rollout_corr/rollout_ppl:np.float64(1.1819306765682995) - rollout_corr/rollout_log_ppl:np.float64(0.16223546834953595) - rollout_corr/log_ppl_diff:np.float64(0.0006027217605151236) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013810389646096155) - rollout_corr/log_ppl_diff_max:np.float64(0.006462059915065765) - rollout_corr/log_ppl_diff_min:np.float64(-0.0063353851437568665) - rollout_corr/ppl_ratio:np.float64(1.0006045163609087) - rollout_corr/chi2_token:np.float64(0.00062129320576787) - rollout_corr/chi2_seq:np.float64(0.24325380730442703) - actor/pg_loss:np.float64(0.0005239989841356874) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.407132443404407e-05) - actor/ppo_kl:np.float64(4.226569186727147e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_w_mean:np.float64(40721.495245202444) - self_distillation/token_reweight_w_std:np.float64(783821.5592307698) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000015069032088) - self_distillation/token_reweight_w_clip_frac:np.float64(0.019517847162205726) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.1003046054393053) - perf/mfu/actor:np.float64(0.05316328022214093) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.77106618881226) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:2 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013472663704305887 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013472663704305887 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.74609375 - response_length/max:865.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.74609375 - response_length_non_aborted/max:865.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.65625 - prompt_length/max:461.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.696744382381439e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9459689650684595) - timing_s/agent_loop/generate_sequences/max:np.float64(7.560749892145395) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.470217737412895) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.560749892145395) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:280 - timing_s/agent_loop/slowest/response_length:865 - timing_s/gen:14.918655971065164 - timing_s/reward:0.05108494125306606 - timing_s/old_log_prob:3.0590711664408445 - timing_s/adv:0.5692106038331985 - timing_s/update_actor:22.958233008161187 - timing_s/step:41.59128964133561 - timing_s/stop_profile:2.9379501938819885e-05 - timing_per_token_ms/adv:0.003015285942700032 - timing_per_token_ms/update_actor:0.121616914359217 - timing_per_token_ms/gen:0.15550795821196814 - perf/total_num_tokens:188775 - perf/time_per_step:41.59128964133561 - perf/throughput:567.3513661992387 (TaskRunner pid=2425326) Training Progress: 69%|██████▉ | 69/100 [52:45<21:53, 42.38s/it] (TaskRunner pid=2425326) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2425326) validation generation end (TaskRunner pid=2425326) [prompt] system (TaskRunner pid=2425326) (TaskRunner pid=2425326) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2425326) (TaskRunner pid=2425326) A (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) user (TaskRunner pid=2425326) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2425326) (TaskRunner pid=2425326) Lattice Parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) a = 7.82 Å (TaskRunner pid=2425326) b = 7.82 Å (TaskRunner pid=2425326) c = 17.88 Å (TaskRunner pid=2425326) α = 90.00º (TaskRunner pid=2425326) β = 90.00º (TaskRunner pid=2425326) γ = 90.00º (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) A: 1025.67 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) B: 1094.31 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) C: 1200.45 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) D: 1150.78 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) Please reason step by step. (TaskRunner pid=2425326) assistant (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) [response] (TaskRunner pid=2425326) The given crystal structure has orthorhombic symmetry, as all angles (α, β, γ) are 90.00º. For an orthorhombic system, the volume of the unit cell is calculated using the formula: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ V = a \times b \times c $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Substituting the given lattice parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ V = 7.82 \, \text{Å} \times 7.82 \, \text{Å} \times 17.88 \, \text{Å} $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Then multiply by 17.88: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Therefore, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) B (TaskRunner pid=2425326) (TaskRunner pid=2425326) [ground_truth] B (TaskRunner pid=2425326) [score] 1.0 (TaskRunner pid=2425326) [acc] 1.0 (TaskRunner pid=2425326) [pred] B (TaskRunner pid=2425326) [incorrect_format] 1 (TaskRunner pid=2425326) [feedback] (TaskRunner pid=2425326) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (TaskRunner pid=2425326) step:70 - global_seqlen/min:21386 - global_seqlen/max:29266 - global_seqlen/minmax_diff:7880 - global_seqlen/balanced_min:24961 - global_seqlen/balanced_max:24973 - global_seqlen/mean:24969.375 - actor/entropy:0.15795306861400604 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917734384536743 - training/rollout_probs_diff_mean:0.0028157206252217293 - training/rollout_probs_diff_std:0.009608049876987934 - training/rollout_actor_probs_pearson_corr:0.9988582730293274 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001001358032227 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9951517970184796e-05 - rollout_corr/rollout_is_max:1.5591782331466675 - rollout_corr/rollout_is_min:0.2789737582206726 - rollout_corr/rollout_is_std:0.027834173291921616 - rollout_corr/rollout_is_eff_sample_size:0.9992260966360754 - rollout_corr/rollout_is_seq_mean:1.0000709295272827 - rollout_corr/rollout_is_seq_std:0.0016937156906351447 - rollout_corr/rollout_is_seq_max:1.0067968368530273 - rollout_corr/rollout_is_seq_min:0.9945765137672424 - rollout_corr/rollout_is_seq_max_deviation:0.006796836853027344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1822556359693408) - rollout_corr/training_log_ppl:np.float64(0.1616127286106348) - rollout_corr/kl:np.float64(0.0004549667964592885) - rollout_corr/k3_kl:np.float64(0.0005006182778970469) - rollout_corr/rollout_ppl:np.float64(1.1816930123604834) - rollout_corr/rollout_log_ppl:np.float64(0.16115776021615602) - rollout_corr/log_ppl_diff:np.float64(0.000454968394478783) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013991295418236405) - rollout_corr/log_ppl_diff_max:np.float64(0.007021665573120117) - rollout_corr/log_ppl_diff_min:np.float64(-0.005086250603199005) - rollout_corr/ppl_ratio:np.float64(1.0004567354917526) - rollout_corr/chi2_token:np.float64(0.0010688593611121178) - rollout_corr/chi2_seq:np.float64(0.44107732083648443) - actor/pg_loss:np.float64(0.0014038580702617764) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010900944607783458) - actor/ppo_kl:np.float64(0.00010926450992521097) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7578125) - self_distillation/token_reweight_w_mean:np.float64(1203.3038881937973) - self_distillation/token_reweight_w_std:np.float64(20396.826950604853) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995136295910925) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03277971228817478) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.17008450627326965) - perf/mfu/actor:np.float64(0.05596456541974616) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.7408905029297) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7519946808510638) - val-aux/sciknoweval/reward/std@16:np.float64(0.05755674390197442) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7740212765957446) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.03644780252328991) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7302765957446808) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.060077687206505734) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7525851063829787) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.05737175354614723) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7860851063829787) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.02068874435499379) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7057872340425532) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.058235388972907545) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.757595744680851) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.04341019114870843) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7938191489361702) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.010582465936555888) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6809787234042554) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.05025895080940347) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7615212765957446) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.02625018766837341) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.796936170212766) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.00436341720182558) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6590212765957446) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.036272385319647577) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7620531914893617) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.01645538172128112) - val-aux/sciknoweval/score/mean@16:np.float64(0.7519946808510638) - val-aux/sciknoweval/score/std@16:np.float64(0.05755674390197442) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7740212765957446) - val-aux/sciknoweval/score/best@2/std:np.float64(0.03644780252328991) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7302765957446808) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.060077687206505734) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7525851063829787) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.05737175354614723) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7860851063829787) - val-aux/sciknoweval/score/best@4/std:np.float64(0.02068874435499379) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7057872340425532) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.058235388972907545) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.757595744680851) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.04341019114870843) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7938191489361702) - val-aux/sciknoweval/score/best@8/std:np.float64(0.010582465936555888) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6809787234042554) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.05025895080940347) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7615212765957446) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.02625018766837341) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.796936170212766) - val-aux/sciknoweval/score/best@16/std:np.float64(0.00436341720182558) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6590212765957446) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.036272385319647577) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7620531914893617) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.01645538172128112) - val-core/sciknoweval/acc/mean@16:np.float64(0.7519946808510638) - val-aux/sciknoweval/acc/std@16:np.float64(0.05755674390197442) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7740212765957446) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.03644780252328991) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7302765957446808) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.060077687206505734) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7525851063829787) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.05737175354614723) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7860851063829787) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.02068874435499379) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7057872340425532) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.058235388972907545) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.757595744680851) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.04341019114870843) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7938191489361702) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.010582465936555888) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6809787234042554) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.05025895080940347) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7615212765957446) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.02625018766837341) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.796936170212766) - val-core/sciknoweval/acc/best@16/std:np.float64(0.00436341720182558) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6590212765957446) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.036272385319647577) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7620531914893617) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.01645538172128112) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:2 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:6.983031198615208e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:6.983031198615208e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:391.57421875 - response_length/max:986.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:391.57421875 - response_length_non_aborted/max:986.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.71875 - prompt_length/max:512.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.871375858783722e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.051547607406974) - timing_s/agent_loop/generate_sequences/max:np.float64(8.433207880705595) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.5971244552056305) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.433207880705595) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:986 - timing_s/gen:15.583152512088418 - timing_s/reward:0.06144307926297188 - timing_s/old_log_prob:3.03653734177351 - timing_s/adv:0.5722084566950798 - timing_s/update_actor:23.25078850798309 - timing_s/step:42.592571161687374 - timing_s/testing:42.528791872784495 - timing_s/save_checkpoint:12.928781876340508 - timing_s/stop_profile:0.00016595236957073212 - timing_per_token_ms/adv:0.002864551358890039 - timing_per_token_ms/update_actor:0.1163965282870671 - timing_per_token_ms/gen:0.1554537724538214 - perf/total_num_tokens:199755 - perf/time_per_step:42.592571161687374 - perf/throughput:586.2377949716337 (TaskRunner pid=2425326) Training Progress: 70%|███████ | 70/100 [54:23<29:33, 59.12s/it] (TaskRunner pid=2425326) step:71 - global_seqlen/min:21129 - global_seqlen/max:29382 - global_seqlen/minmax_diff:8253 - global_seqlen/balanced_min:25727 - global_seqlen/balanced_max:25731 - global_seqlen/mean:25729.5 - actor/entropy:0.14619044959545135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4265592098236084 - training/rollout_probs_diff_mean:0.002610948635265231 - training/rollout_probs_diff_std:0.009499951265752316 - training/rollout_actor_probs_pearson_corr:0.9987483620643616 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9998709559440613 - rollout_corr/rollout_is_ratio_fraction_high:9.452867743675597e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.835860323102679e-05 - rollout_corr/rollout_is_max:2.090681791305542 - rollout_corr/rollout_is_min:0.2555420696735382 - rollout_corr/rollout_is_std:0.02667061798274517 - rollout_corr/rollout_is_eff_sample_size:0.9992889457086402 - rollout_corr/rollout_is_seq_mean:0.9999041557312012 - rollout_corr/rollout_is_seq_std:0.0017140944255515933 - rollout_corr/rollout_is_seq_max:1.0067784786224365 - rollout_corr/rollout_is_seq_min:0.9939207434654236 - rollout_corr/rollout_is_seq_max_deviation:0.0067784786224365234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.157512275967747) - rollout_corr/training_log_ppl:np.float64(0.14429704349458916) - rollout_corr/kl:np.float64(0.0005933385952370251) - rollout_corr/k3_kl:np.float64(0.0004896153571110062) - rollout_corr/rollout_ppl:np.float64(1.156822511460632) - rollout_corr/rollout_log_ppl:np.float64(0.14370370344840921) - rollout_corr/log_ppl_diff:np.float64(0.0005933400461799465) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014040681489859708) - rollout_corr/log_ppl_diff_max:np.float64(0.022363394498825073) - rollout_corr/log_ppl_diff_min:np.float64(-0.007897064089775085) - rollout_corr/ppl_ratio:np.float64(1.0005960373673588) - rollout_corr/chi2_token:np.float64(0.0006810310296714306) - rollout_corr/chi2_seq:np.float64(0.23998616635799408) - actor/pg_loss:np.float64(0.0012072301469743252) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.832385194386006e-05) - actor/ppo_kl:np.float64(0.00010519745228787514) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(22375.126596617047) - self_distillation/token_reweight_w_std:np.float64(455294.88350632007) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999144962057471) - self_distillation/token_reweight_w_clip_frac:np.float64(0.023882172885350883) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.12107829935848713) - perf/mfu/actor:np.float64(0.057396841037551355) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.90234565734863) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:2 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004301054868847132 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004301054868847132 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:413.234375 - response_length/max:709.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:413.234375 - response_length_non_aborted/max:709.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.8125 - prompt_length/max:482.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010207295417785645 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9039257634431124) - timing_s/agent_loop/generate_sequences/max:np.float64(7.2027587201446295) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.839908803347498) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.2027587201446295) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:709 - timing_s/gen:14.13414779305458 - timing_s/reward:0.05805518664419651 - timing_s/old_log_prob:3.176641810685396 - timing_s/adv:0.5792755596339703 - timing_s/update_actor:23.260292198508978 - timing_s/step:41.33510657213628 - timing_s/stop_profile:0.00012504681944847107 - timing_per_token_ms/adv:0.002814257756825678 - timing_per_token_ms/update_actor:0.11300400415140684 - timing_per_token_ms/gen:0.133608233382374 - perf/total_num_tokens:205836 - perf/time_per_step:41.33510657213628 - perf/throughput:622.4611990560122 (TaskRunner pid=2425326) Training Progress: 71%|███████ | 71/100 [55:04<26:00, 53.81s/it] (TaskRunner pid=2425326) step:72 - global_seqlen/min:23755 - global_seqlen/max:27582 - global_seqlen/minmax_diff:3827 - global_seqlen/balanced_min:26033 - global_seqlen/balanced_max:26038 - global_seqlen/mean:26036.125 - actor/entropy:0.12893779575824738 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5466005206108093 - training/rollout_probs_diff_mean:0.002355077536776662 - training/rollout_probs_diff_std:0.008884367533028126 - training/rollout_actor_probs_pearson_corr:0.9988211989402771 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:0.9999760985374451 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.390641025674995e-06 - rollout_corr/rollout_is_max:1.6415457725524902 - rollout_corr/rollout_is_min:0.4085317850112915 - rollout_corr/rollout_is_std:0.025675423443317413 - rollout_corr/rollout_is_eff_sample_size:0.9993413259754993 - rollout_corr/rollout_is_seq_mean:0.9999563694000244 - rollout_corr/rollout_is_seq_std:0.001469815499149263 - rollout_corr/rollout_is_seq_max:1.0054359436035156 - rollout_corr/rollout_is_seq_min:0.9952075481414795 - rollout_corr/rollout_is_seq_max_deviation:0.005435943603515625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.142138805706054) - rollout_corr/training_log_ppl:np.float64(0.13091732577595394) - rollout_corr/kl:np.float64(0.0006154191354865723) - rollout_corr/k3_kl:np.float64(0.0004093494105674722) - rollout_corr/rollout_ppl:np.float64(1.1414216659031808) - rollout_corr/rollout_log_ppl:np.float64(0.13030190639256034) - rollout_corr/log_ppl_diff:np.float64(0.0006154193833936006) - rollout_corr/log_ppl_abs_diff:np.float64(0.001276002629310824) - rollout_corr/log_ppl_diff_max:np.float64(0.0072397589683532715) - rollout_corr/log_ppl_diff_min:np.float64(-0.0041563138365745544) - rollout_corr/ppl_ratio:np.float64(1.0006169544067234) - rollout_corr/chi2_token:np.float64(0.00042544701136648655) - rollout_corr/chi2_seq:np.float64(0.3312300390098244) - actor/pg_loss:np.float64(0.000465520890429616) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012200302489873138) - actor/ppo_kl:np.float64(0.00021202303769474184) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(15657.842956456356) - self_distillation/token_reweight_w_std:np.float64(324596.33819807204) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006162282079458) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014747079403605312) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.0753592997789383) - perf/mfu/actor:np.float64(0.05824163963292747) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.82327270507812) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:2 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002990919165313244 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002990919165313244 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:415.97265625 - response_length/max:774.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:415.97265625 - response_length_non_aborted/max:774.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.65625 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.398504436016083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.954980343580246) - timing_s/agent_loop/generate_sequences/max:np.float64(7.557073438540101) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.849845921824453) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.557073438540101) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:774 - timing_s/gen:14.369386786594987 - timing_s/reward:0.06402775645256042 - timing_s/old_log_prob:3.2480822391808033 - timing_s/adv:0.5643929969519377 - timing_s/update_actor:23.314557442441583 - timing_s/step:41.64751455374062 - timing_s/stop_profile:0.00012939423322677612 - timing_per_token_ms/adv:0.002709663001656053 - timing_per_token_ms/update_actor:0.11193369521406114 - timing_per_token_ms/gen:0.13493775682554054 - perf/total_num_tokens:208289 - perf/time_per_step:41.64751455374062 - perf/throughput:625.1543526421923 (TaskRunner pid=2425326) Training Progress: 72%|███████▏ | 72/100 [55:46<23:24, 50.17s/it] (TaskRunner pid=2425326) step:73 - global_seqlen/min:18636 - global_seqlen/max:29660 - global_seqlen/minmax_diff:11024 - global_seqlen/balanced_min:24241 - global_seqlen/balanced_max:24250 - global_seqlen/mean:24247.5 - actor/entropy:0.13270290195941925 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105892539024353 - training/rollout_probs_diff_mean:0.0024152176920324564 - training/rollout_probs_diff_std:0.008945304900407791 - training/rollout_actor_probs_pearson_corr:0.9987912178039551 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9999596476554871 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7426339387893677 - rollout_corr/rollout_is_min:0.5378821492195129 - rollout_corr/rollout_is_std:0.025952504947781563 - rollout_corr/rollout_is_eff_sample_size:0.9993265637082287 - rollout_corr/rollout_is_seq_mean:0.9999666213989258 - rollout_corr/rollout_is_seq_std:0.001694069360382855 - rollout_corr/rollout_is_seq_max:1.009128451347351 - rollout_corr/rollout_is_seq_min:0.9934071898460388 - rollout_corr/rollout_is_seq_max_deviation:0.009128451347351074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1533304639160633) - rollout_corr/training_log_ppl:np.float64(0.13871499759989092) - rollout_corr/kl:np.float64(0.0004179086189708414) - rollout_corr/k3_kl:np.float64(0.0004781059732508197) - rollout_corr/rollout_ppl:np.float64(1.1528317001648247) - rollout_corr/rollout_log_ppl:np.float64(0.13829708714911249) - rollout_corr/log_ppl_diff:np.float64(0.0004179104507784359) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015569298338959925) - rollout_corr/log_ppl_diff_max:np.float64(0.007369577884674072) - rollout_corr/log_ppl_diff_min:np.float64(-0.008280247449874878) - rollout_corr/ppl_ratio:np.float64(1.0004202341660857) - rollout_corr/chi2_token:np.float64(0.0010818280279636383) - rollout_corr/chi2_seq:np.float64(0.5929345737677068) - actor/pg_loss:np.float64(0.0011989542981609702) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.68577708893281e-05) - actor/ppo_kl:np.float64(-1.3919712605892087e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(12149.982365099946) - self_distillation/token_reweight_w_std:np.float64(255379.99605708098) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005448078736663) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02245678317558486) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.12669192161411047) - perf/mfu/actor:np.float64(0.054887508358529194) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.6793441772461) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:2 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00045284064253792167 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00045284064253792167 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:366.609375 - response_length/max:825.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:366.609375 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.125 - prompt_length/max:494.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.810816168785095e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8117183838039637) - timing_s/agent_loop/generate_sequences/max:np.float64(7.202767511829734) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.063807285878283) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.202767511829734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:459 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:14.214554635807872 - timing_s/reward:0.06004488281905651 - timing_s/old_log_prob:3.2171297930181026 - timing_s/adv:0.5325079876929522 - timing_s/update_actor:22.979619529098272 - timing_s/step:41.08560956455767 - timing_s/stop_profile:0.00011685304343700409 - timing_per_token_ms/adv:0.0027451695416689974 - timing_per_token_ms/update_actor:0.11846385982626184 - timing_per_token_ms/gen:0.15145713075702033 - perf/total_num_tokens:193980 - perf/time_per_step:41.08560956455767 - perf/throughput:590.1701412486041 (TaskRunner pid=2425326) Training Progress: 73%|███████▎ | 73/100 [56:27<21:21, 47.46s/it] (TaskRunner pid=2425326) step:74 - global_seqlen/min:17738 - global_seqlen/max:28519 - global_seqlen/minmax_diff:10781 - global_seqlen/balanced_min:24352 - global_seqlen/balanced_max:24355 - global_seqlen/mean:24354.0 - actor/entropy:0.1518096625804901 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24477601051330566 - training/rollout_probs_diff_mean:0.0026547734159976244 - training/rollout_probs_diff_std:0.009086649864912033 - training/rollout_actor_probs_pearson_corr:0.9989684224128723 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.8359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999744296073914 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:0.6066008806228638 - rollout_corr/rollout_is_std:0.026919756084680557 - rollout_corr/rollout_is_eff_sample_size:0.9992757324631958 - rollout_corr/rollout_is_seq_mean:1.0000312328338623 - rollout_corr/rollout_is_seq_std:0.0015839202096685767 - rollout_corr/rollout_is_seq_max:1.0063767433166504 - rollout_corr/rollout_is_seq_min:0.9951472282409668 - rollout_corr/rollout_is_seq_max_deviation:0.006376743316650391 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1675362568348646) - rollout_corr/training_log_ppl:np.float64(0.15320578197133727) - rollout_corr/kl:np.float64(0.0005907460380143448) - rollout_corr/k3_kl:np.float64(0.00047802696350629503) - rollout_corr/rollout_ppl:np.float64(1.1668550157919526) - rollout_corr/rollout_log_ppl:np.float64(0.15261503440706292) - rollout_corr/log_ppl_diff:np.float64(0.000590747564274352) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013946343315183185) - rollout_corr/log_ppl_diff_max:np.float64(0.005996823310852051) - rollout_corr/log_ppl_diff_min:np.float64(-0.0064388662576675415) - rollout_corr/ppl_ratio:np.float64(1.0005924843717366) - rollout_corr/chi2_token:np.float64(0.000995079753920436) - rollout_corr/chi2_seq:np.float64(0.9846179345622659) - actor/pg_loss:np.float64(0.0004339113365858793) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.072660491758143e-05) - actor/ppo_kl:np.float64(0.0002481599828243475) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_w_mean:np.float64(25010.128267090302) - self_distillation/token_reweight_w_std:np.float64(403261.9257906423) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998603481799364) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015681787044741213) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.07713901996612549) - perf/mfu/actor:np.float64(0.05541995222259979) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.67742919921875) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:2 - critic/score/mean:0.8359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-9.123707059188746e-06 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-9.123707059188746e-06 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.625 - response_length/max:708.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.625 - response_length_non_aborted/max:708.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:386.4375 - prompt_length/max:481.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001463126391172409 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4284407533705235) - timing_s/agent_loop/generate_sequences/max:np.float64(6.862213185057044) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.804670477307809) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.862213185057044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:708 - timing_s/gen:14.012208767235279 - timing_s/reward:0.06173849105834961 - timing_s/old_log_prob:3.2487590964883566 - timing_s/adv:0.5537375174462795 - timing_s/update_actor:22.853031177073717 - timing_s/step:40.81501886062324 - timing_s/stop_profile:2.8137117624282837e-05 - timing_per_token_ms/adv:0.0028421281793867513 - timing_per_token_ms/update_actor:0.11729608676743922 - timing_per_token_ms/gen:0.1461066146066408 - perf/total_num_tokens:194832 - perf/time_per_step:40.81501886062324 - perf/throughput:596.6921167711575 (TaskRunner pid=2425326) Training Progress: 74%|███████▍ | 74/100 [57:08<19:42, 45.47s/it] (TaskRunner pid=2425326) step:75 - global_seqlen/min:19394 - global_seqlen/max:27755 - global_seqlen/minmax_diff:8361 - global_seqlen/balanced_min:24882 - global_seqlen/balanced_max:24895 - global_seqlen/mean:24891.625 - actor/entropy:0.14456571638584137 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41412925720214844 - training/rollout_probs_diff_mean:0.0024925703182816505 - training/rollout_probs_diff_std:0.008859826251864433 - training/rollout_actor_probs_pearson_corr:0.9989786744117737 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0000722408294678 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6829358339309692 - rollout_corr/rollout_is_min:0.5139058828353882 - rollout_corr/rollout_is_std:0.02535303682088852 - rollout_corr/rollout_is_eff_sample_size:0.9993578745193887 - rollout_corr/rollout_is_seq_mean:1.0000704526901245 - rollout_corr/rollout_is_seq_std:0.0013349363580346107 - rollout_corr/rollout_is_seq_max:1.0043613910675049 - rollout_corr/rollout_is_seq_min:0.9966480731964111 - rollout_corr/rollout_is_seq_max_deviation:0.004361391067504883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1619159164838493) - rollout_corr/training_log_ppl:np.float64(0.14637414650496794) - rollout_corr/kl:np.float64(0.0002864610944490753) - rollout_corr/k3_kl:np.float64(0.0003769323916529288) - rollout_corr/rollout_ppl:np.float64(1.1615847297944129) - rollout_corr/rollout_log_ppl:np.float64(0.14608768460311694) - rollout_corr/log_ppl_diff:np.float64(0.0002864619018509984) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011751155252568424) - rollout_corr/log_ppl_diff_max:np.float64(0.004867613315582275) - rollout_corr/log_ppl_diff_min:np.float64(-0.0032870694994926453) - rollout_corr/ppl_ratio:np.float64(1.0002875870559365) - rollout_corr/chi2_token:np.float64(0.0009425606112927198) - rollout_corr/chi2_seq:np.float64(0.3932009725831449) - actor/pg_loss:np.float64(0.0005540411220863461) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.731543756657629e-05) - actor/ppo_kl:np.float64(2.2481496540827095e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(14579.109780266648) - self_distillation/token_reweight_w_std:np.float64(280755.2456506435) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002813185565174) - self_distillation/token_reweight_w_clip_frac:np.float64(0.011456080566858873) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.07376006245613098) - perf/mfu/actor:np.float64(0.05670890516263739) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.6297607421875) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:2 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-9.670490544522181e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-9.670490544522181e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:383.73828125 - response_length/max:775.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:383.73828125 - response_length_non_aborted/max:775.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.125 - prompt_length/max:493.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.557225525379181e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9221164751797915) - timing_s/agent_loop/generate_sequences/max:np.float64(7.160478323698044) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.489932823416893) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.160478323698044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:473 - timing_s/agent_loop/slowest/response_length:775 - timing_s/gen:15.294624611735344 - timing_s/reward:0.06329484842717648 - timing_s/old_log_prob:3.371696576476097 - timing_s/adv:0.5590562466531992 - timing_s/update_actor:22.78064789250493 - timing_s/step:42.15948460996151 - timing_s/stop_profile:0.00011246465146541595 - timing_per_token_ms/adv:0.002807451535673139 - timing_per_token_ms/update_actor:0.11439915982034585 - timing_per_token_ms/gen:0.15569107985520061 - perf/total_num_tokens:199133 - perf/time_per_step:42.15948460996151 - perf/throughput:590.4157802279814 (TaskRunner pid=2425326) Training Progress: 75%|███████▌ | 75/100 [57:50<18:32, 44.49s/it] (TaskRunner pid=2425326) step:76 - global_seqlen/min:19023 - global_seqlen/max:25742 - global_seqlen/minmax_diff:6719 - global_seqlen/balanced_min:22863 - global_seqlen/balanced_max:22870 - global_seqlen/mean:22867.625 - actor/entropy:0.16983351111412048 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4560590386390686 - training/rollout_probs_diff_mean:0.0028075729496777058 - training/rollout_probs_diff_std:0.009388630278408527 - training/rollout_actor_probs_pearson_corr:0.9990025758743286 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9999856948852539 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.306140049768146e-05 - rollout_corr/rollout_is_max:1.4818986654281616 - rollout_corr/rollout_is_min:0.3279767334461212 - rollout_corr/rollout_is_std:0.027030237019062042 - rollout_corr/rollout_is_eff_sample_size:0.9992695425945294 - rollout_corr/rollout_is_seq_mean:1.0000501871109009 - rollout_corr/rollout_is_seq_std:0.0017394665628671646 - rollout_corr/rollout_is_seq_max:1.0056047439575195 - rollout_corr/rollout_is_seq_min:0.9949859380722046 - rollout_corr/rollout_is_seq_max_deviation:0.005604743957519531 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1861248877830803) - rollout_corr/training_log_ppl:np.float64(0.16663943161984207) - rollout_corr/kl:np.float64(0.0005234517319987475) - rollout_corr/k3_kl:np.float64(0.00048701030445386095) - rollout_corr/rollout_ppl:np.float64(1.185497789643705) - rollout_corr/rollout_log_ppl:np.float64(0.1661159795585263) - rollout_corr/log_ppl_diff:np.float64(0.0005234520613157656) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015148041384236421) - rollout_corr/log_ppl_diff_max:np.float64(0.009437471628189087) - rollout_corr/log_ppl_diff_min:np.float64(-0.004993893206119537) - rollout_corr/ppl_ratio:np.float64(1.0005255860742182) - rollout_corr/chi2_token:np.float64(0.0008696855511516333) - rollout_corr/chi2_seq:np.float64(0.4529962248634547) - actor/pg_loss:np.float64(0.0009462278103455901) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001746799912325514) - actor/ppo_kl:np.float64(0.00016931543271425653) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(52772.14622266544) - self_distillation/token_reweight_w_std:np.float64(991781.8422375617) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002462787088007) - self_distillation/token_reweight_w_clip_frac:np.float64(0.023418084252625704) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.1341552957892418) - perf/mfu/actor:np.float64(0.052929615809226865) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.67698287963867) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:2 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002022196538746357 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002022196538746357 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:338.76953125 - response_length/max:716.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:338.76953125 - response_length_non_aborted/max:716.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:375.84375 - prompt_length/max:471.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001282915472984314 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9809613283723593) - timing_s/agent_loop/generate_sequences/max:np.float64(6.393184717744589) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8920597993856063) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.393184717744589) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:716 - timing_s/gen:13.496035115793347 - timing_s/reward:0.06097748689353466 - timing_s/old_log_prob:3.1969344280660152 - timing_s/adv:0.53871732018888 - timing_s/update_actor:22.520035944879055 - timing_s/step:39.90068666823208 - timing_s/stop_profile:0.00013415329158306122 - timing_per_token_ms/adv:0.002944759896299244 - timing_per_token_ms/update_actor:0.12309999368582797 - timing_per_token_ms/gen:0.15561873872347476 - perf/total_num_tokens:182941 - perf/time_per_step:39.90068666823208 - perf/throughput:573.11357045408 (TaskRunner pid=2425326) Training Progress: 76%|███████▌ | 76/100 [58:30<17:15, 43.13s/it] (TaskRunner pid=2425326) step:77 - global_seqlen/min:21140 - global_seqlen/max:29886 - global_seqlen/minmax_diff:8746 - global_seqlen/balanced_min:26058 - global_seqlen/balanced_max:26063 - global_seqlen/mean:26061.375 - actor/entropy:0.1314040571451187 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6118554472923279 - training/rollout_probs_diff_mean:0.0022847233340144157 - training/rollout_probs_diff_std:0.008869555778801441 - training/rollout_actor_probs_pearson_corr:0.9988621473312378 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.55078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:0.9998553991317749 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.608455002307892e-06 - rollout_corr/rollout_is_max:1.9780268669128418 - rollout_corr/rollout_is_min:0.16305549442768097 - rollout_corr/rollout_is_std:0.02495972253382206 - rollout_corr/rollout_is_eff_sample_size:0.999377281064019 - rollout_corr/rollout_is_seq_mean:0.9998329877853394 - rollout_corr/rollout_is_seq_std:0.0015425964957103133 - rollout_corr/rollout_is_seq_max:1.008661150932312 - rollout_corr/rollout_is_seq_min:0.9920440316200256 - rollout_corr/rollout_is_seq_max_deviation:0.008661150932312012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1371952700428665) - rollout_corr/training_log_ppl:np.float64(0.12557044533605222) - rollout_corr/kl:np.float64(0.00041996561053991854) - rollout_corr/k3_kl:np.float64(0.0003773384485654674) - rollout_corr/rollout_ppl:np.float64(1.136701645795256) - rollout_corr/rollout_log_ppl:np.float64(0.1251504793690401) - rollout_corr/log_ppl_diff:np.float64(0.00041996596701210365) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013194825223763473) - rollout_corr/log_ppl_diff_max:np.float64(0.008445695042610168) - rollout_corr/log_ppl_diff_min:np.float64(-0.007665909826755524) - rollout_corr/ppl_ratio:np.float64(1.000421627657488) - rollout_corr/chi2_token:np.float64(0.0006627803668379784) - rollout_corr/chi2_seq:np.float64(0.3250666738022119) - actor/pg_loss:np.float64(0.00032114051282405853) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010906540319410851) - actor/ppo_kl:np.float64(-8.278402166705945e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55078125) - self_distillation/token_reweight_w_mean:np.float64(482.4049843470566) - self_distillation/token_reweight_w_std:np.float64(9469.058202488523) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003158545587212) - self_distillation/token_reweight_w_clip_frac:np.float64(0.00614475453767227) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.06303319334983826) - perf/mfu/actor:np.float64(0.05866944429011463) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.73622131347656) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:2 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003772519761696458 - critic/advantages/max:0.875 - critic/advantages/min:-0.25 - critic/returns/mean:-0.003772519761696458 - critic/returns/max:0.875 - critic/returns/min:-0.25 - response_length/mean:406.54296875 - response_length/max:794.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.54296875 - response_length_non_aborted/max:794.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.875 - prompt_length/max:510.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013346970081329346 - timing_s/agent_loop/generate_sequences/min:np.float64(1.98610470443964) - timing_s/agent_loop/generate_sequences/max:np.float64(7.597161207348108) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.669510889070807) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.597161207348108) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:462 - timing_s/agent_loop/slowest/response_length:794 - timing_s/gen:14.85983676277101 - timing_s/reward:0.06423389725387096 - timing_s/old_log_prob:3.186824817210436 - timing_s/adv:0.5260447077453136 - timing_s/update_actor:23.128049712628126 - timing_s/step:41.85332676395774 - timing_s/stop_profile:0.00011503323912620544 - timing_per_token_ms/adv:0.0025231051112293273 - timing_per_token_ms/update_actor:0.11093068627724038 - timing_per_token_ms/gen:0.14278007939246706 - perf/total_num_tokens:208491 - perf/time_per_step:41.85332676395774 - perf/throughput:622.6834762020142 (TaskRunner pid=2425326) Training Progress: 77%|███████▋ | 77/100 [59:12<16:23, 42.76s/it] (TaskRunner pid=2425326) step:78 - global_seqlen/min:22408 - global_seqlen/max:26542 - global_seqlen/minmax_diff:4134 - global_seqlen/balanced_min:25162 - global_seqlen/balanced_max:25167 - global_seqlen/mean:25165.625 - actor/entropy:0.1404900997877121 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975857734680176 - training/rollout_probs_diff_mean:0.0025989068672060966 - training/rollout_probs_diff_std:0.00935492292046547 - training/rollout_actor_probs_pearson_corr:0.9987855553627014 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:1.0001541376113892 - rollout_corr/rollout_is_ratio_fraction_high:3.129792457912117e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0432641829538625e-05 - rollout_corr/rollout_is_max:2.244246006011963 - rollout_corr/rollout_is_min:0.48570868372917175 - rollout_corr/rollout_is_std:0.027534905821084976 - rollout_corr/rollout_is_eff_sample_size:0.9992425223351087 - rollout_corr/rollout_is_seq_mean:1.0001494884490967 - rollout_corr/rollout_is_seq_std:0.0017627255292609334 - rollout_corr/rollout_is_seq_max:1.009850263595581 - rollout_corr/rollout_is_seq_min:0.9946343302726746 - rollout_corr/rollout_is_seq_max_deviation:0.009850263595581055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1456196890212595) - rollout_corr/training_log_ppl:np.float64(0.13408512135356432) - rollout_corr/kl:np.float64(0.0003313364118397999) - rollout_corr/k3_kl:np.float64(0.0004774892564824995) - rollout_corr/rollout_ppl:np.float64(1.1452268385328352) - rollout_corr/rollout_log_ppl:np.float64(0.13375378415366868) - rollout_corr/log_ppl_diff:np.float64(0.0003313371998956427) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014909690798958763) - rollout_corr/log_ppl_diff_max:np.float64(0.009037353098392487) - rollout_corr/log_ppl_diff_min:np.float64(-0.009428516030311584) - rollout_corr/ppl_ratio:np.float64(1.000333433272317) - rollout_corr/chi2_token:np.float64(0.0012364955618977547) - rollout_corr/chi2_seq:np.float64(0.7343874129001051) - actor/pg_loss:np.float64(0.0005279415054246783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.377939436992165e-05) - actor/ppo_kl:np.float64(8.827176546333249e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(2963.168327776948) - self_distillation/token_reweight_w_std:np.float64(69523.39454561111) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997723049018532) - self_distillation/token_reweight_w_clip_frac:np.float64(0.008337422099430114) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.05845398269593716) - perf/mfu/actor:np.float64(0.057231997315823215) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.64541482925415) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:2 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0019535121973603964 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0019535121973603964 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:374.42578125 - response_length/max:766.0 - response_length/min:144.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.42578125 - response_length_non_aborted/max:766.0 - response_length_non_aborted/min:144.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.0 - prompt_length/max:493.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014593824744224548 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7676217835396528) - timing_s/agent_loop/generate_sequences/max:np.float64(7.0610082577914) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.349809826962883) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.0610082577914) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:271 - timing_s/agent_loop/slowest/response_length:713 - timing_s/gen:14.002786139026284 - timing_s/reward:0.0625135526061058 - timing_s/old_log_prob:3.252134330570698 - timing_s/adv:0.5382752940058708 - timing_s/update_actor:22.856297442689538 - timing_s/step:40.80179328843951 - timing_s/stop_profile:0.00010807625949382782 - timing_per_token_ms/adv:0.0026736634496752555 - timing_per_token_ms/update_actor:0.11352935523501571 - timing_per_token_ms/gen:0.14608604987873394 - perf/total_num_tokens:201325 - perf/time_per_step:40.80179328843951 - perf/throughput:616.7774250042644 (TaskRunner pid=2425326) Training Progress: 78%|███████▊ | 78/100 [59:53<15:28, 42.19s/it] (TaskRunner pid=2425326) step:79 - global_seqlen/min:17609 - global_seqlen/max:27080 - global_seqlen/minmax_diff:9471 - global_seqlen/balanced_min:24292 - global_seqlen/balanced_max:24296 - global_seqlen/mean:24294.125 - actor/entropy:0.11888319253921509 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6836330890655518 - training/rollout_probs_diff_mean:0.0023963868152350187 - training/rollout_probs_diff_std:0.009350299835205078 - training/rollout_actor_probs_pearson_corr:0.9986465573310852 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.8046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999861717224121 - rollout_corr/rollout_is_ratio_fraction_high:1.0910241144301835e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0910241144301835e-05 - rollout_corr/rollout_is_max:2.0104925632476807 - rollout_corr/rollout_is_min:0.28233087062835693 - rollout_corr/rollout_is_std:0.02596168965101242 - rollout_corr/rollout_is_eff_sample_size:0.9993264446594591 - rollout_corr/rollout_is_seq_mean:0.9999976754188538 - rollout_corr/rollout_is_seq_std:0.0015455903485417366 - rollout_corr/rollout_is_seq_max:1.0057108402252197 - rollout_corr/rollout_is_seq_min:0.9947482943534851 - rollout_corr/rollout_is_seq_max_deviation:0.0057108402252197266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1248688157647848) - rollout_corr/training_log_ppl:np.float64(0.11555000303633278) - rollout_corr/kl:np.float64(0.00035947103978273276) - rollout_corr/k3_kl:np.float64(0.00038126155526896355) - rollout_corr/rollout_ppl:np.float64(1.124436704441905) - rollout_corr/rollout_log_ppl:np.float64(0.11519053130905377) - rollout_corr/log_ppl_diff:np.float64(0.0003594717272790149) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012188185646664351) - rollout_corr/log_ppl_diff_max:np.float64(0.004965603351593018) - rollout_corr/log_ppl_diff_min:np.float64(-0.003954470157623291) - rollout_corr/ppl_ratio:np.float64(1.000360760372132) - rollout_corr/chi2_token:np.float64(0.0008095051161944866) - rollout_corr/chi2_seq:np.float64(0.17021930450573564) - actor/pg_loss:np.float64(0.000181176932528615) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(1.7104167454817798e-05) - actor/ppo_kl:np.float64(8.604286239943804e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_w_mean:np.float64(3.382794478908181) - self_distillation/token_reweight_w_std:np.float64(40.678365995234344) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996934912633151) - self_distillation/token_reweight_w_clip_frac:np.float64(0.006391824339516461) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.04638487892225385) - perf/mfu/actor:np.float64(0.054837898195592656) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.43430709838867) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002863938279915601 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002863938279915601 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:358.03515625 - response_length/max:685.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:358.03515625 - response_length_non_aborted/max:685.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.15625 - prompt_length/max:494.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00022757239639759064 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9782236218452454) - timing_s/agent_loop/generate_sequences/max:np.float64(6.614131908863783) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.256541164133523) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.614131908863783) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:452 - timing_s/agent_loop/slowest/response_length:685 - timing_s/gen:14.227141605690122 - timing_s/reward:0.060579895973205566 - timing_s/old_log_prob:3.238638214766979 - timing_s/adv:0.5527574010193348 - timing_s/update_actor:22.91782428510487 - timing_s/step:41.10509469918907 - timing_s/stop_profile:2.960488200187683e-05 - timing_per_token_ms/adv:0.002844089882941528 - timing_per_token_ms/update_actor:0.1179185517337261 - timing_per_token_ms/gen:0.1552215499709801 - perf/total_num_tokens:194353 - perf/time_per_step:41.10509469918907 - perf/throughput:591.0246692724268 (TaskRunner pid=2425326) Training Progress: 79%|███████▉ | 79/100 [1:00:34<14:42, 42.00s/it] (TaskRunner pid=2425326) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2425326) validation generation end (TaskRunner pid=2425326) [prompt] system (TaskRunner pid=2425326) (TaskRunner pid=2425326) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2425326) (TaskRunner pid=2425326) A (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) user (TaskRunner pid=2425326) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2425326) (TaskRunner pid=2425326) Lattice Parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) a = 7.82 Å (TaskRunner pid=2425326) b = 7.82 Å (TaskRunner pid=2425326) c = 17.88 Å (TaskRunner pid=2425326) α = 90.00º (TaskRunner pid=2425326) β = 90.00º (TaskRunner pid=2425326) γ = 90.00º (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) A: 1025.67 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) B: 1094.31 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) C: 1200.45 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) D: 1150.78 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) Please reason step by step. (TaskRunner pid=2425326) assistant (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) [response] (TaskRunner pid=2425326) The given crystal structure has orthorhombic symmetry, as indicated by the right angles (α = β = γ = 90.00º). For an orthorhombic system, the volume of the unit cell is calculated using the formula: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ V = a \times b \times c $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Substituting the given lattice parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Next, multiply by 17.88: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Thus, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) B (TaskRunner pid=2425326) (TaskRunner pid=2425326) [ground_truth] B (TaskRunner pid=2425326) [score] 1.0 (TaskRunner pid=2425326) [acc] 1.0 (TaskRunner pid=2425326) [pred] B (TaskRunner pid=2425326) [incorrect_format] 1 (TaskRunner pid=2425326) [feedback] (TaskRunner pid=2425326) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2425326) step:80 - global_seqlen/min:21209 - global_seqlen/max:28791 - global_seqlen/minmax_diff:7582 - global_seqlen/balanced_min:24840 - global_seqlen/balanced_max:24842 - global_seqlen/mean:24841.125 - actor/entropy:0.1547521948814392 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5170373916625977 - training/rollout_probs_diff_mean:0.002725405152887106 - training/rollout_probs_diff_std:0.009661387652158737 - training/rollout_actor_probs_pearson_corr:0.998755156993866 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0000289678573608 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.980181852763053e-05 - rollout_corr/rollout_is_max:1.6338125467300415 - rollout_corr/rollout_is_min:0.2606707215309143 - rollout_corr/rollout_is_std:0.027992188930511475 - rollout_corr/rollout_is_eff_sample_size:0.9992170508295175 - rollout_corr/rollout_is_seq_mean:0.9999718070030212 - rollout_corr/rollout_is_seq_std:0.0015326400753110647 - rollout_corr/rollout_is_seq_max:1.0061142444610596 - rollout_corr/rollout_is_seq_min:0.9952588081359863 - rollout_corr/rollout_is_seq_max_deviation:0.00611424446105957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1599856242537498) - rollout_corr/training_log_ppl:np.float64(0.14582765987870516) - rollout_corr/kl:np.float64(0.0004167129488124033) - rollout_corr/k3_kl:np.float64(0.00045200509000409284) - rollout_corr/rollout_ppl:np.float64(1.159479976631701) - rollout_corr/rollout_log_ppl:np.float64(0.1454109457845334) - rollout_corr/log_ppl_diff:np.float64(0.0004167140941717662) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013146949859219603) - rollout_corr/log_ppl_diff_max:np.float64(0.006044149398803711) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035454928874969482) - rollout_corr/ppl_ratio:np.float64(1.0004181617405266) - rollout_corr/chi2_token:np.float64(0.0009744579438120127) - rollout_corr/chi2_seq:np.float64(0.3122515210416168) - actor/pg_loss:np.float64(0.0007777587743476033) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001616802319404087) - actor/ppo_kl:np.float64(-1.446276943450897e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(1290.9808493084274) - self_distillation/token_reweight_w_std:np.float64(30826.316274836252) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009112362749875) - self_distillation/token_reweight_w_clip_frac:np.float64(0.020249034860171378) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.08906365092843771) - perf/mfu/actor:np.float64(0.056279375927647174) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.7968292236328) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7446808510638298) - val-aux/sciknoweval/reward/std@16:np.float64(0.05538506140454391) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7648191489361702) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.041079114908916525) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7239255319148935) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.052388701912266876) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7438191489361702) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.056122987382457405) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7792659574468086) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.030281883921560036) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7043829787234042) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.04789275998848127) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7473297872340425) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.03985107353500265) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7920531914893617) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.02353342629540201) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.683627659574468) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.041933104272897176) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7491914893617021) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.021075178461829686) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8022021276595744) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.014557086997678886) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6648297872340425) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.029559143408423787) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7491808510638298) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.01130424834722519) - val-aux/sciknoweval/score/mean@16:np.float64(0.7446808510638298) - val-aux/sciknoweval/score/std@16:np.float64(0.05538506140454391) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7648191489361702) - val-aux/sciknoweval/score/best@2/std:np.float64(0.041079114908916525) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7239255319148935) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.052388701912266876) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7438191489361702) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.056122987382457405) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7792659574468086) - val-aux/sciknoweval/score/best@4/std:np.float64(0.030281883921560036) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7043829787234042) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.04789275998848127) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7473297872340425) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.03985107353500265) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7920531914893617) - val-aux/sciknoweval/score/best@8/std:np.float64(0.02353342629540201) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.683627659574468) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.041933104272897176) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7491914893617021) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.021075178461829686) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8022021276595744) - val-aux/sciknoweval/score/best@16/std:np.float64(0.014557086997678886) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6648297872340425) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.029559143408423787) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7491808510638298) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.01130424834722519) - val-core/sciknoweval/acc/mean@16:np.float64(0.7446808510638298) - val-aux/sciknoweval/acc/std@16:np.float64(0.05538506140454391) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7648191489361702) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.041079114908916525) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7239255319148935) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.052388701912266876) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7438191489361702) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.056122987382457405) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7792659574468086) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.030281883921560036) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7043829787234042) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.04789275998848127) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7473297872340425) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.03985107353500265) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7920531914893617) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.02353342629540201) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.683627659574468) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.041933104272897176) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7491914893617021) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.021075178461829686) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8022021276595744) - val-core/sciknoweval/acc/best@16/std:np.float64(0.014557086997678886) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6648297872340425) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.029559143408423787) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7491808510638298) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.01130424834722519) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001391993253491819 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001391993253491819 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:393.22265625 - response_length/max:879.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:393.22265625 - response_length_non_aborted/max:879.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.0625 - prompt_length/max:488.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.312155604362488e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8971087038516998) - timing_s/agent_loop/generate_sequences/max:np.float64(7.854356210678816) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.425177589560917) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.854356210678816) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:452 - timing_s/agent_loop/slowest/response_length:879 - timing_s/gen:14.75103897601366 - timing_s/reward:0.06159013323485851 - timing_s/old_log_prob:3.0885722003877163 - timing_s/adv:0.560110392048955 - timing_s/update_actor:23.414669763296843 - timing_s/step:41.96594016812742 - timing_s/testing:40.07030278630555 - timing_s/save_checkpoint:13.438419211655855 - timing_s/stop_profile:0.00016048364341259003 - timing_per_token_ms/adv:0.0028184632944812027 - timing_per_token_ms/update_actor:0.11782210831482492 - timing_per_token_ms/gen:0.14653592585321273 - perf/total_num_tokens:198729 - perf/time_per_step:41.96594016812742 - perf/throughput:591.9353861841158 (TaskRunner pid=2425326) Training Progress: 80%|████████ | 80/100 [1:02:10<19:21, 58.07s/it] (TaskRunner pid=2425326) step:81 - global_seqlen/min:17496 - global_seqlen/max:28019 - global_seqlen/minmax_diff:10523 - global_seqlen/balanced_min:22611 - global_seqlen/balanced_max:22618 - global_seqlen/mean:22615.375 - actor/entropy:0.12404749542474747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9216691255569458 - training/rollout_probs_diff_mean:0.002631731564179063 - training/rollout_probs_diff_std:0.010278254747390747 - training/rollout_actor_probs_pearson_corr:0.9983770847320557 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.71875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.9999887347221375 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1917956726392731e-05 - rollout_corr/rollout_is_max:1.6952898502349854 - rollout_corr/rollout_is_min:0.07604635506868362 - rollout_corr/rollout_is_std:0.027463378384709358 - rollout_corr/rollout_is_eff_sample_size:0.999246331269986 - rollout_corr/rollout_is_seq_mean:1.0000253915786743 - rollout_corr/rollout_is_seq_std:0.0017810006393119693 - rollout_corr/rollout_is_seq_max:1.007685899734497 - rollout_corr/rollout_is_seq_min:0.9950356483459473 - rollout_corr/rollout_is_seq_max_deviation:0.00768589973449707 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1329951421357691) - rollout_corr/training_log_ppl:np.float64(0.12217923940625042) - rollout_corr/kl:np.float64(0.00022328235970903165) - rollout_corr/k3_kl:np.float64(0.0005613560395403283) - rollout_corr/rollout_ppl:np.float64(1.13274285197258) - rollout_corr/rollout_log_ppl:np.float64(0.12195595684170257) - rollout_corr/log_ppl_diff:np.float64(0.00022328256454784423) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015974190609995276) - rollout_corr/log_ppl_diff_max:np.float64(0.006190508604049683) - rollout_corr/log_ppl_diff_min:np.float64(-0.007130444049835205) - rollout_corr/ppl_ratio:np.float64(1.0002254962455481) - rollout_corr/chi2_token:np.float64(0.0018780133686959743) - rollout_corr/chi2_seq:np.float64(0.5290436635259539) - actor/pg_loss:np.float64(0.0008224116172641516) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012829220668209018) - actor/ppo_kl:np.float64(-0.00017505270996531763) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_w_mean:np.float64(24058.840261494508) - self_distillation/token_reweight_w_std:np.float64(351943.8947785102) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996452361810952) - self_distillation/token_reweight_w_clip_frac:np.float64(0.024172404606360942) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.14087015390396118) - perf/mfu/actor:np.float64(0.051926130920910266) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.75435638427734) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002596624894067645 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002596624894067645 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:327.76171875 - response_length/max:711.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.76171875 - response_length_non_aborted/max:711.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:378.96875 - prompt_length/max:524.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.86522912979126e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2808020189404488) - timing_s/agent_loop/generate_sequences/max:np.float64(6.690686171874404) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.627482264499122) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.690686171874404) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:711 - timing_s/gen:13.24904546327889 - timing_s/reward:0.05488709174096584 - timing_s/old_log_prob:3.159280374646187 - timing_s/adv:0.5504780802875757 - timing_s/update_actor:22.320611890405416 - timing_s/step:39.43916137330234 - timing_s/stop_profile:0.00010585226118564606 - timing_per_token_ms/adv:0.0030426097305902276 - timing_per_token_ms/update_actor:0.1233707814396479 - timing_per_token_ms/gen:0.15790155128033287 - perf/total_num_tokens:180923 - perf/time_per_step:39.43916137330234 - perf/throughput:573.4243379553473 (TaskRunner pid=2425326) Training Progress: 81%|████████ | 81/100 [1:02:49<16:37, 52.49s/it] (TaskRunner pid=2425326) step:82 - global_seqlen/min:21426 - global_seqlen/max:26718 - global_seqlen/minmax_diff:5292 - global_seqlen/balanced_min:23464 - global_seqlen/balanced_max:23474 - global_seqlen/mean:23472.0 - actor/entropy:0.14597472548484802 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3772715628147125 - training/rollout_probs_diff_mean:0.0027662990614771843 - training/rollout_probs_diff_std:0.009847620502114296 - training/rollout_actor_probs_pearson_corr:0.9987053275108337 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9999840259552002 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2500225895782933e-05 - rollout_corr/rollout_is_max:1.758644700050354 - rollout_corr/rollout_is_min:0.39373767375946045 - rollout_corr/rollout_is_std:0.028364475816488266 - rollout_corr/rollout_is_eff_sample_size:0.9991961032747029 - rollout_corr/rollout_is_seq_mean:0.999990701675415 - rollout_corr/rollout_is_seq_std:0.0016806689091026783 - rollout_corr/rollout_is_seq_max:1.0044842958450317 - rollout_corr/rollout_is_seq_min:0.9935813546180725 - rollout_corr/rollout_is_seq_max_deviation:0.00641864538192749 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1481548310257494) - rollout_corr/training_log_ppl:np.float64(0.13636921173747396) - rollout_corr/kl:np.float64(0.0006211503117583561) - rollout_corr/k3_kl:np.float64(0.0004820507922715933) - rollout_corr/rollout_ppl:np.float64(1.1474357061088085) - rollout_corr/rollout_log_ppl:np.float64(0.1357480596198002) - rollout_corr/log_ppl_diff:np.float64(0.0006211521176737733) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015310409144149162) - rollout_corr/log_ppl_diff_max:np.float64(0.00695037841796875) - rollout_corr/log_ppl_diff_min:np.float64(-0.008269622921943665) - rollout_corr/ppl_ratio:np.float64(1.0006232312880456) - rollout_corr/chi2_token:np.float64(0.0006954430136829615) - rollout_corr/chi2_seq:np.float64(0.27697336371056736) - actor/pg_loss:np.float64(0.0010938638588413596) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014887547422404168) - actor/ppo_kl:np.float64(0.00020041486293642663) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_w_mean:np.float64(725.874184731394) - self_distillation/token_reweight_w_std:np.float64(16585.61214940023) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002405804116279) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02034169973921962) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.10360464476980269) - perf/mfu/actor:np.float64(0.05363463253175198) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.11970901489258) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004921924322843552 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004921924322843552 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:347.21875 - response_length/max:830.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:347.21875 - response_length_non_aborted/max:830.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:386.28125 - prompt_length/max:510.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014531612396240234 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4289500638842583) - timing_s/agent_loop/generate_sequences/max:np.float64(7.363815115764737) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.1253841179132) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.363815115764737) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:811 - timing_s/gen:14.541756819933653 - timing_s/reward:0.05919262766838074 - timing_s/old_log_prob:2.918740339577198 - timing_s/adv:0.5743510741740465 - timing_s/update_actor:22.555013416334987 - timing_s/step:40.737937085330486 - timing_s/stop_profile:3.623589873313904e-05 - timing_per_token_ms/adv:0.003058703317644675 - timing_per_token_ms/update_actor:0.1201165932618385 - timing_per_token_ms/gen:0.16359640018825547 - perf/total_num_tokens:187776 - perf/time_per_step:40.737937085330486 - perf/throughput:576.1705594182416 (TaskRunner pid=2425326) Training Progress: 82%|████████▏ | 82/100 [1:03:30<14:41, 48.98s/it] (TaskRunner pid=2425326) step:83 - global_seqlen/min:21063 - global_seqlen/max:24614 - global_seqlen/minmax_diff:3551 - global_seqlen/balanced_min:23076 - global_seqlen/balanced_max:23082 - global_seqlen/mean:23080.75 - actor/entropy:0.12527549266815186 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20161855220794678 - training/rollout_probs_diff_mean:0.0025815563276410103 - training/rollout_probs_diff_std:0.009686398319900036 - training/rollout_actor_probs_pearson_corr:0.9985157251358032 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.999979555606842 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1838522368634585e-05 - rollout_corr/rollout_is_max:1.6963958740234375 - rollout_corr/rollout_is_min:0.26981014013290405 - rollout_corr/rollout_is_std:0.027797745540738106 - rollout_corr/rollout_is_eff_sample_size:0.9992278820119855 - rollout_corr/rollout_is_seq_mean:0.9999013543128967 - rollout_corr/rollout_is_seq_std:0.001572471926920116 - rollout_corr/rollout_is_seq_max:1.0038666725158691 - rollout_corr/rollout_is_seq_min:0.9941545724868774 - rollout_corr/rollout_is_seq_max_deviation:0.005845427513122559 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.129922913853079) - rollout_corr/training_log_ppl:np.float64(0.12065385213645641) - rollout_corr/kl:np.float64(0.0004963102201873681) - rollout_corr/k3_kl:np.float64(0.00042870396250194176) - rollout_corr/rollout_ppl:np.float64(1.1293713334016502) - rollout_corr/rollout_log_ppl:np.float64(0.12015754160529468) - rollout_corr/log_ppl_diff:np.float64(0.0004963105311617255) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013216602674219757) - rollout_corr/log_ppl_diff_max:np.float64(0.0056051164865493774) - rollout_corr/log_ppl_diff_min:np.float64(-0.005131259560585022) - rollout_corr/ppl_ratio:np.float64(1.00049793231301) - rollout_corr/chi2_token:np.float64(0.0007408871315419674) - rollout_corr/chi2_seq:np.float64(0.5475682236719877) - actor/pg_loss:np.float64(0.0003311014734208584) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.298924194605206e-05) - actor/ppo_kl:np.float64(-1.5865895953481868e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(5976.837657049531) - self_distillation/token_reweight_w_std:np.float64(105540.45544876461) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001530186273158) - self_distillation/token_reweight_w_clip_frac:np.float64(0.008285001036711037) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.05502156773582101) - perf/mfu/actor:np.float64(0.05255484396903881) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.24004364013672) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001291878754273057 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.001291878754273057 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:329.9609375 - response_length/max:698.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:329.9609375 - response_length_non_aborted/max:698.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.3125 - prompt_length/max:492.0 - prompt_length/min:267.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.520429790019989e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0772734954953194) - timing_s/agent_loop/generate_sequences/max:np.float64(6.904385980218649) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9355401242282824) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.904385980218649) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:698 - timing_s/gen:14.591104974970222 - timing_s/reward:0.05898437276482582 - timing_s/old_log_prob:3.001984477043152 - timing_s/adv:0.5269759222865105 - timing_s/update_actor:22.867907663807273 - timing_s/step:41.13651973940432 - timing_s/stop_profile:6.701797246932983e-05 - timing_per_token_ms/adv:0.002853979627430383 - timing_per_token_ms/update_actor:0.12384729516917384 - timing_per_token_ms/gen:0.1727371253104087 - perf/total_num_tokens:184646 - perf/time_per_step:41.13651973940432 - perf/throughput:561.0768763671358 (TaskRunner pid=2425326) Training Progress: 83%|████████▎ | 83/100 [1:04:11<13:12, 46.64s/it] (TaskRunner pid=2425326) step:84 - global_seqlen/min:19050 - global_seqlen/max:27314 - global_seqlen/minmax_diff:8264 - global_seqlen/balanced_min:23791 - global_seqlen/balanced_max:23802 - global_seqlen/mean:23799.125 - actor/entropy:0.11515828967094421 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1930849552154541 - training/rollout_probs_diff_mean:0.0024084453471004963 - training/rollout_probs_diff_std:0.009149638004601002 - training/rollout_actor_probs_pearson_corr:0.998576283454895 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.8203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.000038743019104 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6482936143875122 - rollout_corr/rollout_is_min:0.616061270236969 - rollout_corr/rollout_is_std:0.02596168965101242 - rollout_corr/rollout_is_eff_sample_size:0.9993265637082287 - rollout_corr/rollout_is_seq_mean:1.0000572204589844 - rollout_corr/rollout_is_seq_std:0.0016339186113327742 - rollout_corr/rollout_is_seq_max:1.0067466497421265 - rollout_corr/rollout_is_seq_min:0.9952927827835083 - rollout_corr/rollout_is_seq_max_deviation:0.006746649742126465 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1227850238792598) - rollout_corr/training_log_ppl:np.float64(0.11481493657265673) - rollout_corr/kl:np.float64(0.0003574394741372444) - rollout_corr/k3_kl:np.float64(0.0003687933111855557) - rollout_corr/rollout_ppl:np.float64(1.1223909854888916) - rollout_corr/rollout_log_ppl:np.float64(0.11445749627091573) - rollout_corr/log_ppl_diff:np.float64(0.0003574403017410077) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012601374110090546) - rollout_corr/log_ppl_diff_max:np.float64(0.004564203321933746) - rollout_corr/log_ppl_diff_min:np.float64(-0.006322316825389862) - rollout_corr/ppl_ratio:np.float64(1.0003587694372982) - rollout_corr/chi2_token:np.float64(0.0007815023418515921) - rollout_corr/chi2_seq:np.float64(0.2785968375392258) - actor/pg_loss:np.float64(0.0006993752904236317) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.200197028898401e-05) - actor/ppo_kl:np.float64(6.108405310456533e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_w_mean:np.float64(27.973567307693884) - self_distillation/token_reweight_w_std:np.float64(398.3262187499204) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999827864812687) - self_distillation/token_reweight_w_clip_frac:np.float64(0.009426361415535212) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.08874382358044386) - perf/mfu/actor:np.float64(0.05394679103021711) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.16954803466797) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002152180066332221 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:0.002152180066332221 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:351.66015625 - response_length/max:748.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:351.66015625 - response_length_non_aborted/max:748.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:392.0625 - prompt_length/max:519.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.512925028800964e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7282806430011988) - timing_s/agent_loop/generate_sequences/max:np.float64(6.993337128311396) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.107985877271858) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.993337128311396) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:748 - timing_s/gen:13.897366415709257 - timing_s/reward:0.060391005128622055 - timing_s/old_log_prob:2.914033291861415 - timing_s/adv:0.5548912454396486 - timing_s/update_actor:23.01916046254337 - timing_s/step:40.5301332231611 - timing_s/stop_profile:2.902187407016754e-05 - timing_per_token_ms/adv:0.0029144519254365895 - timing_per_token_ms/update_actor:0.12090339698698675 - timing_per_token_ms/gen:0.1543723012019912 - perf/total_num_tokens:190393 - perf/time_per_step:40.5301332231611 - perf/throughput:587.1958246216644 (TaskRunner pid=2425326) Training Progress: 84%|████████▍ | 84/100 [1:04:52<11:56, 44.81s/it] (TaskRunner pid=2425326) step:85 - global_seqlen/min:19911 - global_seqlen/max:26535 - global_seqlen/minmax_diff:6624 - global_seqlen/balanced_min:23181 - global_seqlen/balanced_max:23236 - global_seqlen/mean:23189.5 - actor/entropy:0.1488744616508484 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948741436004639 - training/rollout_probs_diff_mean:0.0026586854364722967 - training/rollout_probs_diff_std:0.009458235464990139 - training/rollout_actor_probs_pearson_corr:0.9988400936126709 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:0.999997615814209 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.1743153663701378e-05 - rollout_corr/rollout_is_max:1.8089348077774048 - rollout_corr/rollout_is_min:0.2346915602684021 - rollout_corr/rollout_is_std:0.02704346552491188 - rollout_corr/rollout_is_eff_sample_size:0.999269304524189 - rollout_corr/rollout_is_seq_mean:0.999980628490448 - rollout_corr/rollout_is_seq_std:0.0015972725814208388 - rollout_corr/rollout_is_seq_max:1.004743218421936 - rollout_corr/rollout_is_seq_min:0.9943519830703735 - rollout_corr/rollout_is_seq_max_deviation:0.005648016929626465 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1522011863999069) - rollout_corr/training_log_ppl:np.float64(0.13857831715722568) - rollout_corr/kl:np.float64(0.0004144432389345809) - rollout_corr/k3_kl:np.float64(0.00039726549428564795) - rollout_corr/rollout_ppl:np.float64(1.1517116916365921) - rollout_corr/rollout_log_ppl:np.float64(0.13816387339466019) - rollout_corr/log_ppl_diff:np.float64(0.0004144437625654973) - rollout_corr/log_ppl_abs_diff:np.float64(0.001279399984923657) - rollout_corr/log_ppl_diff_max:np.float64(0.006227068603038788) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034830868244171143) - rollout_corr/ppl_ratio:np.float64(1.0004158781375736) - rollout_corr/chi2_token:np.float64(0.0007748533971607685) - rollout_corr/chi2_seq:np.float64(0.20484365243464708) - actor/pg_loss:np.float64(0.0009036622941493988) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.688434430339839e-05) - actor/ppo_kl:np.float64(1.6656271835113756e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(15106.58354095975) - self_distillation/token_reweight_w_std:np.float64(301363.08133950847) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999013759661466) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01797304963838542) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.1237591989338398) - perf/mfu/actor:np.float64(0.05327105722159453) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.0658302307129) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004565150942653418 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.004565150942653418 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:332.640625 - response_length/max:959.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:332.640625 - response_length_non_aborted/max:959.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:392.03125 - prompt_length/max:541.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7103349566459656e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.09701538272202) - timing_s/agent_loop/generate_sequences/max:np.float64(7.866923283785582) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.107072352649993) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.866923283785582) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:470 - timing_s/agent_loop/slowest/response_length:959 - timing_s/gen:14.869738360866904 - timing_s/reward:0.060350554063916206 - timing_s/old_log_prob:2.952952789142728 - timing_s/adv:0.6343162413686514 - timing_s/update_actor:22.294486118480563 - timing_s/step:40.89984368532896 - timing_s/stop_profile:5.994364619255066e-05 - timing_per_token_ms/adv:0.0034191996451446312 - timing_per_token_ms/update_actor:0.12017554344897778 - timing_per_token_ms/gen:0.17461762366558908 - perf/total_num_tokens:185516 - perf/time_per_step:40.89984368532896 - perf/throughput:566.9826070342226 (TaskRunner pid=2425326) Training Progress: 85%|████████▌ | 85/100 [1:05:33<10:54, 43.64s/it] (TaskRunner pid=2425326) step:86 - global_seqlen/min:19969 - global_seqlen/max:26446 - global_seqlen/minmax_diff:6477 - global_seqlen/balanced_min:23249 - global_seqlen/balanced_max:23252 - global_seqlen/mean:23250.375 - actor/entropy:0.15242978930473328 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105868697166443 - training/rollout_probs_diff_mean:0.002931097289547324 - training/rollout_probs_diff_std:0.010023131035268307 - training/rollout_actor_probs_pearson_corr:0.998683750629425 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.83984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0000308752059937 - rollout_corr/rollout_is_ratio_fraction_high:1.125986636907328e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.125986636907328e-05 - rollout_corr/rollout_is_max:2.0322604179382324 - rollout_corr/rollout_is_min:0.3792487382888794 - rollout_corr/rollout_is_std:0.029225580394268036 - rollout_corr/rollout_is_eff_sample_size:0.9991467133695389 - rollout_corr/rollout_is_seq_mean:0.9999984502792358 - rollout_corr/rollout_is_seq_std:0.0019490608246996999 - rollout_corr/rollout_is_seq_max:1.010423183441162 - rollout_corr/rollout_is_seq_min:0.9918707013130188 - rollout_corr/rollout_is_seq_max_deviation:0.01042318344116211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1619863198138773) - rollout_corr/training_log_ppl:np.float64(0.14921248353493866) - rollout_corr/kl:np.float64(0.0005324320977670993) - rollout_corr/k3_kl:np.float64(0.0004945815423127442) - rollout_corr/rollout_ppl:np.float64(1.161366907414049) - rollout_corr/rollout_log_ppl:np.float64(0.1486800509155728) - rollout_corr/log_ppl_diff:np.float64(0.0005324326193658635) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015857149701332673) - rollout_corr/log_ppl_diff_max:np.float64(0.008885562419891357) - rollout_corr/log_ppl_diff_min:np.float64(-0.00977480411529541) - rollout_corr/ppl_ratio:np.float64(1.0005347630940378) - rollout_corr/chi2_token:np.float64(0.0009847534820437431) - rollout_corr/chi2_seq:np.float64(0.564132138621062) - actor/pg_loss:np.float64(0.0006826445460319519) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.8786817640357185e-05) - actor/ppo_kl:np.float64(8.414072883056178e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_w_mean:np.float64(4939.58044633409) - self_distillation/token_reweight_w_std:np.float64(109938.30147719354) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997702944092453) - self_distillation/token_reweight_w_clip_frac:np.float64(0.019916853663744405) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.09240363747812808) - perf/mfu/actor:np.float64(0.05295501420496774) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.0387420654297) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.83984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016115683829411864 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016115683829411864 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:346.91796875 - response_length/max:830.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:346.91796875 - response_length_non_aborted/max:830.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.65625 - prompt_length/max:483.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.966797471046448e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8851856123656034) - timing_s/agent_loop/generate_sequences/max:np.float64(7.222102316096425) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.092150884520379) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.222102316096425) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:830 - timing_s/gen:14.30590521544218 - timing_s/reward:0.060830503702163696 - timing_s/old_log_prob:2.9641824401915073 - timing_s/adv:0.5605881288647652 - timing_s/update_actor:22.485920490697026 - timing_s/step:40.46564007177949 - timing_s/stop_profile:0.0001242365688085556 - timing_per_token_ms/adv:0.0030138660605730293 - timing_per_token_ms/update_actor:0.12089009580865377 - timing_per_token_ms/gen:0.16108258228645303 - perf/total_num_tokens:186003 - perf/time_per_step:40.46564007177949 - perf/throughput:574.5707953403827 (TaskRunner pid=2425326) Training Progress: 86%|████████▌ | 86/100 [1:06:13<09:57, 42.70s/it] (TaskRunner pid=2425326) step:87 - global_seqlen/min:21085 - global_seqlen/max:26247 - global_seqlen/minmax_diff:5162 - global_seqlen/balanced_min:24087 - global_seqlen/balanced_max:24090 - global_seqlen/mean:24088.375 - actor/entropy:0.1323355734348297 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917847633361816 - training/rollout_probs_diff_mean:0.0025879396125674248 - training/rollout_probs_diff_std:0.009271741844713688 - training/rollout_actor_probs_pearson_corr:0.9986913204193115 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:0.9999597668647766 - rollout_corr/rollout_is_ratio_fraction_high:1.149068702943623e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.032074213027954 - rollout_corr/rollout_is_min:0.5740495324134827 - rollout_corr/rollout_is_std:0.02641010284423828 - rollout_corr/rollout_is_eff_sample_size:0.9993027545187237 - rollout_corr/rollout_is_seq_mean:0.9999773502349854 - rollout_corr/rollout_is_seq_std:0.0016537748742848635 - rollout_corr/rollout_is_seq_max:1.0089139938354492 - rollout_corr/rollout_is_seq_min:0.9930402040481567 - rollout_corr/rollout_is_seq_max_deviation:0.008913993835449219 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1391075374558568) - rollout_corr/training_log_ppl:np.float64(0.12840614165907027) - rollout_corr/kl:np.float64(0.0004205593369874805) - rollout_corr/k3_kl:np.float64(0.0003839448123414968) - rollout_corr/rollout_ppl:np.float64(1.1386172864586115) - rollout_corr/rollout_log_ppl:np.float64(0.1279855821048841) - rollout_corr/log_ppl_diff:np.float64(0.00042055955418618396) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011943888894165866) - rollout_corr/log_ppl_diff_max:np.float64(0.004606947302818298) - rollout_corr/log_ppl_diff_min:np.float64(-0.007427416741847992) - rollout_corr/ppl_ratio:np.float64(1.0004217766691) - rollout_corr/chi2_token:np.float64(0.0006975787691771984) - rollout_corr/chi2_seq:np.float64(0.2279121340252459) - actor/pg_loss:np.float64(0.0009529207600280643) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015217112922982778) - actor/ppo_kl:np.float64(4.016830045949771e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(204.87072951882146) - self_distillation/token_reweight_w_std:np.float64(4660.289743357804) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996334998868406) - self_distillation/token_reweight_w_clip_frac:np.float64(0.020627167890779674) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.11101096495985985) - perf/mfu/actor:np.float64(0.054963622205453096) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(195.97815322875977) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018227101536467671 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018227101536467671 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:339.94921875 - response_length/max:602.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.94921875 - response_length_non_aborted/max:602.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.8125 - prompt_length/max:518.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012824684381484985 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8554005287587643) - timing_s/agent_loop/generate_sequences/max:np.float64(6.383846677839756) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.200656441644242) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.383846677839756) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:602 - timing_s/gen:13.335279712453485 - timing_s/reward:0.05815145559608936 - timing_s/old_log_prob:2.9563961178064346 - timing_s/adv:0.5521643031388521 - timing_s/update_actor:22.698151176795363 - timing_s/step:39.689814860001206 - timing_s/stop_profile:0.0001172628253698349 - timing_per_token_ms/adv:0.0028653048573162995 - timing_per_token_ms/update_actor:0.11778581565171667 - timing_per_token_ms/gen:0.153231522544193 - perf/total_num_tokens:192707 - perf/time_per_step:39.689814860001206 - perf/throughput:606.9157814156472 (TaskRunner pid=2425326) Training Progress: 87%|████████▋ | 87/100 [1:06:53<09:03, 41.82s/it] (TaskRunner pid=2425326) step:88 - global_seqlen/min:18837 - global_seqlen/max:25671 - global_seqlen/minmax_diff:6834 - global_seqlen/balanced_min:23089 - global_seqlen/balanced_max:23094 - global_seqlen/mean:23092.0 - actor/entropy:0.14068984985351562 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19462502002716064 - training/rollout_probs_diff_mean:0.002681329380720854 - training/rollout_probs_diff_std:0.009461375884711742 - training/rollout_actor_probs_pearson_corr:0.9987494349479675 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9999996423721313 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.3253651306731626e-05 - rollout_corr/rollout_is_max:1.644294023513794 - rollout_corr/rollout_is_min:0.4757203161716461 - rollout_corr/rollout_is_std:0.027249855920672417 - rollout_corr/rollout_is_eff_sample_size:0.9992577582488479 - rollout_corr/rollout_is_seq_mean:0.9998669624328613 - rollout_corr/rollout_is_seq_std:0.001749269780702889 - rollout_corr/rollout_is_seq_max:1.005174160003662 - rollout_corr/rollout_is_seq_min:0.9924665093421936 - rollout_corr/rollout_is_seq_max_deviation:0.0075334906578063965 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1600256217643619) - rollout_corr/training_log_ppl:np.float64(0.14285353356899577) - rollout_corr/kl:np.float64(0.0006477979862360428) - rollout_corr/k3_kl:np.float64(0.00043804328623764377) - rollout_corr/rollout_ppl:np.float64(1.1592506598681211) - rollout_corr/rollout_log_ppl:np.float64(0.1422057347954251) - rollout_corr/log_ppl_diff:np.float64(0.0006477987735706847) - rollout_corr/log_ppl_abs_diff:np.float64(0.001455325818824349) - rollout_corr/log_ppl_diff_max:np.float64(0.008176639676094055) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035071074962615967) - rollout_corr/ppl_ratio:np.float64(1.0006498480215669) - rollout_corr/chi2_token:np.float64(0.0004831866826862097) - rollout_corr/chi2_seq:np.float64(0.22203046013601124) - actor/pg_loss:np.float64(0.0006858368869870901) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016782948659965768) - actor/ppo_kl:np.float64(0.0001333341535927346) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(570.608905545203) - self_distillation/token_reweight_w_std:np.float64(12229.07915653108) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001349451486021) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01699924370041117) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.1187331285327673) - perf/mfu/actor:np.float64(0.05280560311060975) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.03455352783203) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006365687004290521 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006365687004290521 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.96875 - response_length/max:778.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.96875 - response_length_non_aborted/max:778.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.65625 - prompt_length/max:490.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010626576840877533 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6121100429445505) - timing_s/agent_loop/generate_sequences/max:np.float64(7.1616474110633135) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.021635754012095) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.1616474110633135) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:778 - timing_s/gen:14.137096744030714 - timing_s/reward:0.058695437386631966 - timing_s/old_log_prob:2.939195904880762 - timing_s/adv:0.548638079315424 - timing_s/update_actor:22.755281733348966 - timing_s/step:40.52675708755851 - timing_s/stop_profile:6.329268217086792e-05 - timing_per_token_ms/adv:0.0029698492947526412 - timing_per_token_ms/update_actor:0.12317730021949683 - timing_per_token_ms/gen:0.16436955566959718 - perf/total_num_tokens:184736 - perf/time_per_step:40.52675708755851 - perf/throughput:569.7963927908044 (TaskRunner pid=2425326) Training Progress: 88%|████████▊ | 88/100 [1:07:34<08:17, 41.44s/it] (TaskRunner pid=2425326) step:89 - global_seqlen/min:22363 - global_seqlen/max:26624 - global_seqlen/minmax_diff:4261 - global_seqlen/balanced_min:24749 - global_seqlen/balanced_max:24753 - global_seqlen/mean:24751.0 - actor/entropy:0.1324988603591919 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730056643486023 - training/rollout_probs_diff_mean:0.0024554075207561255 - training/rollout_probs_diff_std:0.00897773914039135 - training/rollout_actor_probs_pearson_corr:0.9988168478012085 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.000014305114746 - rollout_corr/rollout_is_ratio_fraction_high:1.1008366527676117e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2016733055352233e-05 - rollout_corr/rollout_is_max:2.2451746463775635 - rollout_corr/rollout_is_min:0.3002515733242035 - rollout_corr/rollout_is_std:0.02629021555185318 - rollout_corr/rollout_is_eff_sample_size:0.9993093019327395 - rollout_corr/rollout_is_seq_mean:1.0000624656677246 - rollout_corr/rollout_is_seq_std:0.0015305746346712112 - rollout_corr/rollout_is_seq_max:1.0051894187927246 - rollout_corr/rollout_is_seq_min:0.9954579472541809 - rollout_corr/rollout_is_seq_max_deviation:0.005189418792724609 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1442256579175591) - rollout_corr/training_log_ppl:np.float64(0.1322182439325843) - rollout_corr/kl:np.float64(0.00022351242467166799) - rollout_corr/k3_kl:np.float64(0.00042760402516250906) - rollout_corr/rollout_ppl:np.float64(1.1439585289917886) - rollout_corr/rollout_log_ppl:np.float64(0.1319947297743056) - rollout_corr/log_ppl_diff:np.float64(0.0002235141582787037) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011873634211951867) - rollout_corr/log_ppl_diff_max:np.float64(0.005702301859855652) - rollout_corr/log_ppl_diff_min:np.float64(-0.004311807453632355) - rollout_corr/ppl_ratio:np.float64(1.0002248019445688) - rollout_corr/chi2_token:np.float64(0.0013164845295250416) - rollout_corr/chi2_seq:np.float64(0.43028471851721406) - actor/pg_loss:np.float64(0.0009632922010496259) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002576569099801418) - actor/ppo_kl:np.float64(-7.108147247625318e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(36720.438478566706) - self_distillation/token_reweight_w_std:np.float64(389112.48932122736) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997167198453099) - self_distillation/token_reweight_w_clip_frac:np.float64(0.026357669281424023) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.17487578466534615) - perf/mfu/actor:np.float64(0.05634677546245617) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.1344871520996) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:3 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002399823861196637 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.002399823861196637 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:354.84375 - response_length/max:644.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:354.84375 - response_length_non_aborted/max:644.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:418.625 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.163997411727905e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9405058342963457) - timing_s/agent_loop/generate_sequences/max:np.float64(6.659634169191122) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.368485727907682) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.659634169191122) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:406 - timing_s/agent_loop/slowest/response_length:644 - timing_s/gen:13.944504266604781 - timing_s/reward:0.06027063727378845 - timing_s/old_log_prob:2.9573883954435587 - timing_s/adv:0.560860263183713 - timing_s/update_actor:22.896426752209663 - timing_s/step:40.506764674559236 - timing_s/stop_profile:0.00011531449854373932 - timing_per_token_ms/adv:0.0028325131468613035 - timing_per_token_ms/update_actor:0.1156338468759326 - timing_per_token_ms/gen:0.15350621165350925 - perf/total_num_tokens:198008 - perf/time_per_step:40.506764674559236 - perf/throughput:611.0337421133306 (TaskRunner pid=2425326) Training Progress: 89%|████████▉ | 89/100 [1:08:14<07:32, 41.17s/it] (TaskRunner pid=2425326) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2425326) validation generation end (TaskRunner pid=2425326) [prompt] system (TaskRunner pid=2425326) (TaskRunner pid=2425326) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2425326) (TaskRunner pid=2425326) A (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) user (TaskRunner pid=2425326) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2425326) (TaskRunner pid=2425326) Lattice Parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) a = 7.82 Å (TaskRunner pid=2425326) b = 7.82 Å (TaskRunner pid=2425326) c = 17.88 Å (TaskRunner pid=2425326) α = 90.00º (TaskRunner pid=2425326) β = 90.00º (TaskRunner pid=2425326) γ = 90.00º (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) A: 1025.67 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) B: 1094.31 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) C: 1200.45 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) D: 1150.78 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) Please reason step by step. (TaskRunner pid=2425326) assistant (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) [response] (TaskRunner pid=2425326) The given crystal structure has orthorhombic symmetry, as all angles (α, β, γ) are 90.00º. For an orthorhombic system, the volume of the unit cell is calculated using the formula: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ V = a \times b \times c $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Substituting the given lattice parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ V = 7.82 \, \text{Å} \times 7.82 \, \text{Å} \times 17.88 \, \text{Å} $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Then multiply by 17.88: (TaskRunner pid=2425326) (TaskRunner pid=2425326) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2425326) (TaskRunner pid=2425326) Thus, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) B (TaskRunner pid=2425326) (TaskRunner pid=2425326) [ground_truth] B (TaskRunner pid=2425326) [score] 1.0 (TaskRunner pid=2425326) [acc] 1.0 (TaskRunner pid=2425326) [pred] B (TaskRunner pid=2425326) [incorrect_format] 1 (TaskRunner pid=2425326) [feedback] (TaskRunner pid=2425326) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (TaskRunner pid=2425326) step:90 - global_seqlen/min:19604 - global_seqlen/max:25499 - global_seqlen/minmax_diff:5895 - global_seqlen/balanced_min:23067 - global_seqlen/balanced_max:23073 - global_seqlen/mean:23071.25 - actor/entropy:0.14256960153579712 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351723432540894 - training/rollout_probs_diff_mean:0.002670942572876811 - training/rollout_probs_diff_std:0.009400460869073868 - training/rollout_actor_probs_pearson_corr:0.9987446665763855 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.80859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999788403511047 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5973143577575684 - rollout_corr/rollout_is_min:0.5164310336112976 - rollout_corr/rollout_is_std:0.026839928701519966 - rollout_corr/rollout_is_eff_sample_size:0.99928001780181 - rollout_corr/rollout_is_seq_mean:1.0000172853469849 - rollout_corr/rollout_is_seq_std:0.001670500379987061 - rollout_corr/rollout_is_seq_max:1.0047962665557861 - rollout_corr/rollout_is_seq_min:0.9956794381141663 - rollout_corr/rollout_is_seq_max_deviation:0.004796266555786133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1533320206217468) - rollout_corr/training_log_ppl:np.float64(0.14133044816117035) - rollout_corr/kl:np.float64(0.0003802113450959155) - rollout_corr/k3_kl:np.float64(0.0005155056666126256) - rollout_corr/rollout_ppl:np.float64(1.152886132709682) - rollout_corr/rollout_log_ppl:np.float64(0.14095023600384593) - rollout_corr/log_ppl_diff:np.float64(0.00038021215732442215) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015572865449939854) - rollout_corr/log_ppl_diff_max:np.float64(0.008414089679718018) - rollout_corr/log_ppl_diff_min:np.float64(-0.0054509639739990234) - rollout_corr/ppl_ratio:np.float64(1.000382244354114) - rollout_corr/chi2_token:np.float64(0.0013116379268467426) - rollout_corr/chi2_seq:np.float64(0.4403390462975949) - actor/pg_loss:np.float64(7.652048952877522e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(1.100352073990507e-05) - actor/ppo_kl:np.float64(2.3073779324533916e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_w_mean:np.float64(1.160266701830551) - self_distillation/token_reweight_w_std:np.float64(3.191360288124997) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997768390458077) - self_distillation/token_reweight_w_clip_frac:np.float64(0.004193740169284865) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.028098714305087924) - perf/mfu/actor:np.float64(0.05342099023043444) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.17328262329102) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7539893617021277) - val-aux/sciknoweval/reward/std@16:np.float64(0.047999869452259585) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7715425531914895) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.031585664717190674) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7356595744680852) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.04820199399455401) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7537765957446809) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.04794069733385525) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7831808510638297) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.01971751283415692) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7178617021276595) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.04727369375358176) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.756872340425532) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.03422105143492403) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7916063829787234) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.012974950223330726) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6977553191489362) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.04258307761762703) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7582446808510639) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.01874842076187824) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7963510638297873) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.006127339565861653) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6789148936170213) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03239854912214365) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7574574468085106) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0094684459327092) - val-aux/sciknoweval/score/mean@16:np.float64(0.7539893617021277) - val-aux/sciknoweval/score/std@16:np.float64(0.047999869452259585) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7715425531914895) - val-aux/sciknoweval/score/best@2/std:np.float64(0.031585664717190674) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7356595744680852) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.04820199399455401) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7537765957446809) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.04794069733385525) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7831808510638297) - val-aux/sciknoweval/score/best@4/std:np.float64(0.01971751283415692) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7178617021276595) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.04727369375358176) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.756872340425532) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.03422105143492403) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7916063829787234) - val-aux/sciknoweval/score/best@8/std:np.float64(0.012974950223330726) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6977553191489362) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.04258307761762703) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7582446808510639) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.01874842076187824) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7963510638297873) - val-aux/sciknoweval/score/best@16/std:np.float64(0.006127339565861653) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6789148936170213) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03239854912214365) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7574574468085106) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0094684459327092) - val-core/sciknoweval/acc/mean@16:np.float64(0.7539893617021277) - val-aux/sciknoweval/acc/std@16:np.float64(0.047999869452259585) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7715425531914895) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.031585664717190674) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7356595744680852) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.04820199399455401) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7537765957446809) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.04794069733385525) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7831808510638297) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.01971751283415692) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7178617021276595) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.04727369375358176) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.756872340425532) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.03422105143492403) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7916063829787234) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.012974950223330726) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6977553191489362) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.04258307761762703) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7582446808510639) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.01874842076187824) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7963510638297873) - val-core/sciknoweval/acc/best@16/std:np.float64(0.006127339565861653) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6789148936170213) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03239854912214365) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7574574468085106) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0094684459327092) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:3 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002036359626799822 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002036359626799822 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:324.6640625 - response_length/max:630.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.6640625 - response_length_non_aborted/max:630.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:509.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001462586224079132 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8894255366176367) - timing_s/agent_loop/generate_sequences/max:np.float64(6.1467319540679455) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.916311514971312) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.1467319540679455) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:606 - timing_s/gen:12.912329528480768 - timing_s/reward:0.06071756221354008 - timing_s/old_log_prob:2.9483610540628433 - timing_s/adv:0.5490490607917309 - timing_s/update_actor:22.28504673205316 - timing_s/step:38.8428458198905 - timing_s/testing:38.358254324644804 - timing_s/save_checkpoint:13.140531793236732 - timing_s/stop_profile:0.00013989396393299103 - timing_per_token_ms/adv:0.002974747037935368 - timing_per_token_ms/update_actor:0.12074035180177256 - timing_per_token_ms/gen:0.1553568535803928 - perf/total_num_tokens:184570 - perf/time_per_step:38.8428458198905 - perf/throughput:593.9639465907969 (TaskRunner pid=2425326) Training Progress: 90%|█████████ | 90/100 [1:09:45<09:19, 55.95s/it] (TaskRunner pid=2425326) step:91 - global_seqlen/min:20497 - global_seqlen/max:25268 - global_seqlen/minmax_diff:4771 - global_seqlen/balanced_min:22088 - global_seqlen/balanced_max:22093 - global_seqlen/mean:22091.25 - actor/entropy:0.14610257744789124 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5466005206108093 - training/rollout_probs_diff_mean:0.002629169262945652 - training/rollout_probs_diff_std:0.009485256858170033 - training/rollout_actor_probs_pearson_corr:0.9987938404083252 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.84765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:1.0000755786895752 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.5055738408118486e-05 - rollout_corr/rollout_is_max:1.9108070135116577 - rollout_corr/rollout_is_min:0.3648514449596405 - rollout_corr/rollout_is_std:0.027369895949959755 - rollout_corr/rollout_is_eff_sample_size:0.9992515686028562 - rollout_corr/rollout_is_seq_mean:1.0000547170639038 - rollout_corr/rollout_is_seq_std:0.001723990892060101 - rollout_corr/rollout_is_seq_max:1.0056302547454834 - rollout_corr/rollout_is_seq_min:0.9936156272888184 - rollout_corr/rollout_is_seq_max_deviation:0.006384372711181641 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1725259199738503) - rollout_corr/training_log_ppl:np.float64(0.15398886370530818) - rollout_corr/kl:np.float64(0.0005335942853097464) - rollout_corr/k3_kl:np.float64(0.00044532014708309475) - rollout_corr/rollout_ppl:np.float64(1.1718511590734124) - rollout_corr/rollout_log_ppl:np.float64(0.15345526859891834) - rollout_corr/log_ppl_diff:np.float64(0.0005335951063898392) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014636352643719874) - rollout_corr/log_ppl_diff_max:np.float64(0.010127514600753784) - rollout_corr/log_ppl_diff_min:np.float64(-0.004548102617263794) - rollout_corr/ppl_ratio:np.float64(1.0005357707850635) - rollout_corr/chi2_token:np.float64(0.0006994069553911686) - rollout_corr/chi2_seq:np.float64(0.23433439433574677) - actor/pg_loss:np.float64(0.0007798228180035949) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021067558282084065) - actor/ppo_kl:np.float64(0.00019650942880389266) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84765625) - self_distillation/token_reweight_w_mean:np.float64(25743.910056033637) - self_distillation/token_reweight_w_std:np.float64(337349.557794077) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999736596364528) - self_distillation/token_reweight_w_clip_frac:np.float64(0.021167089085793123) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.1266098888590932) - perf/mfu/actor:np.float64(0.05037939348765556) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.7395133972168) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:3 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007186426664702594 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007186426664702594 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:334.2890625 - response_length/max:788.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.2890625 - response_length_non_aborted/max:788.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:356.0625 - prompt_length/max:477.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.094388663768768e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1131412126123905) - timing_s/agent_loop/generate_sequences/max:np.float64(6.820985209196806) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8924124979021144) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.820985209196806) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:788 - timing_s/gen:13.513571955263615 - timing_s/reward:0.05442272871732712 - timing_s/old_log_prob:3.0522234085947275 - timing_s/adv:0.5667988136410713 - timing_s/update_actor:22.74277588352561 - timing_s/step:40.0220394693315 - timing_s/stop_profile:0.00011635571718215942 - timing_per_token_ms/adv:0.0032071454401690225 - timing_per_token_ms/update_actor:0.12868656076232451 - timing_per_token_ms/gen:0.15790941544863885 - perf/total_num_tokens:176730 - perf/time_per_step:40.0220394693315 - perf/throughput:551.9771179309417 (TaskRunner pid=2425326) Training Progress: 91%|█████████ | 91/100 [1:10:25<07:40, 51.19s/it] (TaskRunner pid=2425326) step:92 - global_seqlen/min:21156 - global_seqlen/max:25601 - global_seqlen/minmax_diff:4445 - global_seqlen/balanced_min:23914 - global_seqlen/balanced_max:23917 - global_seqlen/mean:23915.875 - actor/entropy:0.15240398049354553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4429726004600525 - training/rollout_probs_diff_mean:0.0027351249009370804 - training/rollout_probs_diff_std:0.00975241232663393 - training/rollout_actor_probs_pearson_corr:0.9988147020339966 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.71875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.0001146793365479 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.320383984828368e-05 - rollout_corr/rollout_is_max:1.9662312269210815 - rollout_corr/rollout_is_min:0.47732794284820557 - rollout_corr/rollout_is_std:0.0285780131816864 - rollout_corr/rollout_is_eff_sample_size:0.9991842016461998 - rollout_corr/rollout_is_seq_mean:1.0001089572906494 - rollout_corr/rollout_is_seq_std:0.001473374548368156 - rollout_corr/rollout_is_seq_max:1.0044150352478027 - rollout_corr/rollout_is_seq_min:0.9953008890151978 - rollout_corr/rollout_is_seq_max_deviation:0.004699110984802246 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.159420844167471) - rollout_corr/training_log_ppl:np.float64(0.14538506651297212) - rollout_corr/kl:np.float64(0.0005853978563141737) - rollout_corr/k3_kl:np.float64(0.0004851882544585351) - rollout_corr/rollout_ppl:np.float64(1.1587399523705244) - rollout_corr/rollout_log_ppl:np.float64(0.144799667839834) - rollout_corr/log_ppl_diff:np.float64(0.0005853986731381156) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014114797013462521) - rollout_corr/log_ppl_diff_max:np.float64(0.006640970706939697) - rollout_corr/log_ppl_diff_min:np.float64(-0.004197612404823303) - rollout_corr/ppl_ratio:np.float64(1.0005870771128684) - rollout_corr/chi2_token:np.float64(0.0007457071915268898) - rollout_corr/chi2_seq:np.float64(0.1930299310479313) - actor/pg_loss:np.float64(0.001052756910212338) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001924268062793999) - actor/ppo_kl:np.float64(0.00029038446403717444) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_w_mean:np.float64(22084.149476626422) - self_distillation/token_reweight_w_std:np.float64(376728.0488451377) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999960406916216) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02320667250023689) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.1322405505925417) - perf/mfu/actor:np.float64(0.05412615204124727) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.32924270629883) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:3 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007000475889071822 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007000475889071822 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.93359375 - response_length/max:609.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.93359375 - response_length_non_aborted/max:609.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.4375 - prompt_length/max:476.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015132687985897064 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9590888172388077) - timing_s/agent_loop/generate_sequences/max:np.float64(6.027879290282726) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.198338339338079) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.027879290282726) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:609 - timing_s/gen:13.35700635239482 - timing_s/reward:0.06141200475394726 - timing_s/old_log_prob:2.957593822851777 - timing_s/adv:0.5372003093361855 - timing_s/update_actor:23.09533855319023 - timing_s/step:40.09786621108651 - timing_s/stop_profile:4.513002932071686e-05 - timing_per_token_ms/adv:0.002807760061759111 - timing_per_token_ms/update_actor:0.12071134002618675 - timing_per_token_ms/gen:0.14783462664934333 - perf/total_num_tokens:191327 - perf/time_per_step:40.09786621108651 - perf/throughput:596.4375978038349 (TaskRunner pid=2425326) Training Progress: 92%|█████████▏| 92/100 [1:11:05<06:22, 47.87s/it] (TaskRunner pid=2425326) step:93 - global_seqlen/min:17767 - global_seqlen/max:25876 - global_seqlen/minmax_diff:8109 - global_seqlen/balanced_min:21939 - global_seqlen/balanced_max:21946 - global_seqlen/mean:21943.25 - actor/entropy:0.12090399861335754 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15149319171905518 - training/rollout_probs_diff_mean:0.002356545068323612 - training/rollout_probs_diff_std:0.009056123904883862 - training/rollout_actor_probs_pearson_corr:0.998685896396637 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.8046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999880194664001 - rollout_corr/rollout_is_ratio_fraction_high:1.2775633877026848e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.128607749938965 - rollout_corr/rollout_is_min:0.5685504674911499 - rollout_corr/rollout_is_std:0.025369489565491676 - rollout_corr/rollout_is_eff_sample_size:0.9993565649024203 - rollout_corr/rollout_is_seq_mean:1.0000287294387817 - rollout_corr/rollout_is_seq_std:0.0017659437144175172 - rollout_corr/rollout_is_seq_max:1.0067225694656372 - rollout_corr/rollout_is_seq_min:0.9932870864868164 - rollout_corr/rollout_is_seq_max_deviation:0.006722569465637207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1301220706664026) - rollout_corr/training_log_ppl:np.float64(0.11933962863258785) - rollout_corr/kl:np.float64(0.0003626701826924261) - rollout_corr/k3_kl:np.float64(0.00046709808577105605) - rollout_corr/rollout_ppl:np.float64(1.129688723012805) - rollout_corr/rollout_log_ppl:np.float64(0.11897695804509567) - rollout_corr/log_ppl_diff:np.float64(0.0003626705874921754) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015627569082425907) - rollout_corr/log_ppl_diff_max:np.float64(0.008959487080574036) - rollout_corr/log_ppl_diff_min:np.float64(-0.007843941450119019) - rollout_corr/ppl_ratio:np.float64(1.0003649715799838) - rollout_corr/chi2_token:np.float64(0.0011762918438762426) - rollout_corr/chi2_seq:np.float64(0.34349790937267244) - actor/pg_loss:np.float64(0.0003992551937699318) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.536668161061243e-05) - actor/ppo_kl:np.float64(3.433879705161402e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_w_mean:np.float64(1485.541782564018) - self_distillation/token_reweight_w_std:np.float64(32076.792206338432) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001281290315092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01311015960527584) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.08833481976762414) - perf/mfu/actor:np.float64(0.049890876456477364) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.14807891845703) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:3 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004495426546782255 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004495426546782255 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:305.7578125 - response_length/max:685.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.7578125 - response_length_non_aborted/max:685.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.96875 - prompt_length/max:492.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.6660581827163696e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0138336513191462) - timing_s/agent_loop/generate_sequences/max:np.float64(6.0606822818517685) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4966725943013444) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.0606822818517685) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:448 - timing_s/agent_loop/slowest/response_length:685 - timing_s/gen:13.186543503776193 - timing_s/reward:0.056796735152602196 - timing_s/old_log_prob:2.910303473472595 - timing_s/adv:0.5177232958376408 - timing_s/update_actor:22.766759756952524 - timing_s/step:39.52156287059188 - timing_s/stop_profile:0.00010934844613075256 - timing_per_token_ms/adv:0.0029492172754585167 - timing_per_token_ms/update_actor:0.12969113370257668 - timing_per_token_ms/gen:0.16846645762036172 - perf/total_num_tokens:175546 - perf/time_per_step:39.52156287059188 - perf/throughput:555.2222231658769 (TaskRunner pid=2425326) Training Progress: 93%|█████████▎| 93/100 [1:11:44<05:17, 45.38s/it] (TaskRunner pid=2425326) step:94 - global_seqlen/min:17725 - global_seqlen/max:26194 - global_seqlen/minmax_diff:8469 - global_seqlen/balanced_min:23294 - global_seqlen/balanced_max:23300 - global_seqlen/mean:23296.375 - actor/entropy:0.12870950996875763 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1791786253452301 - training/rollout_probs_diff_mean:0.0024543788749724627 - training/rollout_probs_diff_std:0.009046412073075771 - training/rollout_actor_probs_pearson_corr:0.9987512230873108 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0000104904174805 - rollout_corr/rollout_is_ratio_fraction_high:1.1564035958144814e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.084258556365967 - rollout_corr/rollout_is_min:0.5468906760215759 - rollout_corr/rollout_is_std:0.026822155341506004 - rollout_corr/rollout_is_eff_sample_size:0.9992809701042712 - rollout_corr/rollout_is_seq_mean:0.9999830722808838 - rollout_corr/rollout_is_seq_std:0.001727128168568015 - rollout_corr/rollout_is_seq_max:1.0052071809768677 - rollout_corr/rollout_is_seq_min:0.9946647882461548 - rollout_corr/rollout_is_seq_max_deviation:0.005335211753845215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1428441368043423) - rollout_corr/training_log_ppl:np.float64(0.13037367451761384) - rollout_corr/kl:np.float64(0.0003746576256418166) - rollout_corr/k3_kl:np.float64(0.00045740963861362616) - rollout_corr/rollout_ppl:np.float64(1.1423959978856146) - rollout_corr/rollout_log_ppl:np.float64(0.12999901587318163) - rollout_corr/log_ppl_diff:np.float64(0.000374658644432202) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015223244263324887) - rollout_corr/log_ppl_diff_max:np.float64(0.0070696622133255005) - rollout_corr/log_ppl_diff_min:np.float64(-0.006661802530288696) - rollout_corr/ppl_ratio:np.float64(1.0003767299931496) - rollout_corr/chi2_token:np.float64(0.0011147765908390284) - rollout_corr/chi2_seq:np.float64(0.7085019601508975) - actor/pg_loss:np.float64(0.000595554243773222) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016299049775625463) - actor/ppo_kl:np.float64(-1.9219931974845395e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(5829.977993725799) - self_distillation/token_reweight_w_std:np.float64(131522.8458643295) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996835256461054) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01764957682462409) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.09286441234871745) - perf/mfu/actor:np.float64(0.05320265824096174) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.21632719039917) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:3 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002400982892140746 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002400982892140746 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:337.79296875 - response_length/max:813.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:337.79296875 - response_length_non_aborted/max:813.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.21875 - prompt_length/max:483.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.01930034160614e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0375642906874418) - timing_s/agent_loop/generate_sequences/max:np.float64(6.939276555553079) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9494444877345813) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.939276555553079) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:813 - timing_s/gen:14.054879320785403 - timing_s/reward:0.06046262010931969 - timing_s/old_log_prob:2.978256171569228 - timing_s/adv:0.5452382154762745 - timing_s/update_actor:22.606472635641694 - timing_s/step:40.333417035639286 - timing_s/stop_profile:0.00010686740279197693 - timing_per_token_ms/adv:0.0029255528782711608 - timing_per_token_ms/update_actor:0.12129823113918847 - timing_per_token_ms/gen:0.16253112831206018 - perf/total_num_tokens:186371 - perf/time_per_step:40.333417035639286 - perf/throughput:577.594875718438 (TaskRunner pid=2425326) Training Progress: 94%|█████████▍| 94/100 [1:12:25<04:23, 43.88s/it] (TaskRunner pid=2425326) step:95 - global_seqlen/min:20448 - global_seqlen/max:27302 - global_seqlen/minmax_diff:6854 - global_seqlen/balanced_min:23397 - global_seqlen/balanced_max:23404 - global_seqlen/mean:23402.125 - actor/entropy:0.11967595666646957 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1548396348953247 - training/rollout_probs_diff_mean:0.00242202403023839 - training/rollout_probs_diff_std:0.00918981060385704 - training/rollout_actor_probs_pearson_corr:0.9986111521720886 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0000094175338745 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.624280571937561 - rollout_corr/rollout_is_min:0.5480358004570007 - rollout_corr/rollout_is_std:0.026753177866339684 - rollout_corr/rollout_is_eff_sample_size:0.9992848983711098 - rollout_corr/rollout_is_seq_mean:0.999933123588562 - rollout_corr/rollout_is_seq_std:0.001586279133334756 - rollout_corr/rollout_is_seq_max:1.005149245262146 - rollout_corr/rollout_is_seq_min:0.9930873513221741 - rollout_corr/rollout_is_seq_max_deviation:0.006912648677825928 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1280051534995437) - rollout_corr/training_log_ppl:np.float64(0.11896156305010663) - rollout_corr/kl:np.float64(0.0003956216399711998) - rollout_corr/k3_kl:np.float64(0.0004248532313830822) - rollout_corr/rollout_ppl:np.float64(1.1275437236763537) - rollout_corr/rollout_log_ppl:np.float64(0.11856594059645431) - rollout_corr/log_ppl_diff:np.float64(0.0003956224536523223) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012849519116571173) - rollout_corr/log_ppl_diff_max:np.float64(0.0077836960554122925) - rollout_corr/log_ppl_diff_min:np.float64(-0.005988866090774536) - rollout_corr/ppl_ratio:np.float64(1.0003972640261054) - rollout_corr/chi2_token:np.float64(0.0009248631540685892) - rollout_corr/chi2_seq:np.float64(0.2403997452929616) - actor/pg_loss:np.float64(0.00034071970731019974) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010108559263244388) - actor/ppo_kl:np.float64(-4.2658330890787965e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(18573.2146856694) - self_distillation/token_reweight_w_std:np.float64(329012.32793184) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002196375280619) - self_distillation/token_reweight_w_clip_frac:np.float64(0.006969294132431969) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.05942569300532341) - perf/mfu/actor:np.float64(0.05214527055013675) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.1522102355957) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003893822431564331 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.003893822431564331 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:335.19140625 - response_length/max:741.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.19140625 - response_length_non_aborted/max:741.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.125 - prompt_length/max:493.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002185329794883728 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9147215336561203) - timing_s/agent_loop/generate_sequences/max:np.float64(6.643760541453958) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.815999772043142) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.643760541453958) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:741 - timing_s/gen:14.171202763915062 - timing_s/reward:0.05920936167240143 - timing_s/old_log_prob:2.974240956827998 - timing_s/adv:0.5470796376466751 - timing_s/update_actor:23.213566860184073 - timing_s/step:41.056342558935285 - timing_s/stop_profile:0.00011067278683185577 - timing_per_token_ms/adv:0.002922168593913347 - timing_per_token_ms/update_actor:0.12399283644211836 - timing_per_token_ms/gen:0.16514821014013753 - perf/total_num_tokens:187217 - perf/time_per_step:41.056342558935285 - perf/throughput:570.0002372692325 (TaskRunner pid=2425326) Training Progress: 95%|█████████▌| 95/100 [1:13:06<03:35, 43.05s/it] (TaskRunner pid=2425326) step:96 - global_seqlen/min:20160 - global_seqlen/max:27007 - global_seqlen/minmax_diff:6847 - global_seqlen/balanced_min:24376 - global_seqlen/balanced_max:24378 - global_seqlen/mean:24377.25 - actor/entropy:0.11729797720909119 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2773001194000244 - training/rollout_probs_diff_mean:0.002411847235634923 - training/rollout_probs_diff_std:0.009429632686078548 - training/rollout_actor_probs_pearson_corr:0.9985706210136414 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.625 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.625 - rollout_corr/rollout_is_mean:1.0000203847885132 - rollout_corr/rollout_is_ratio_fraction_high:1.1028520020772703e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1028520020772703e-05 - rollout_corr/rollout_is_max:2.404789447784424 - rollout_corr/rollout_is_min:0.44539976119995117 - rollout_corr/rollout_is_std:0.02721264585852623 - rollout_corr/rollout_is_eff_sample_size:0.9992601389023397 - rollout_corr/rollout_is_seq_mean:0.999981701374054 - rollout_corr/rollout_is_seq_std:0.001592422486282885 - rollout_corr/rollout_is_seq_max:1.0050090551376343 - rollout_corr/rollout_is_seq_min:0.9955622553825378 - rollout_corr/rollout_is_seq_max_deviation:0.005009055137634277 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1281056944280863) - rollout_corr/training_log_ppl:np.float64(0.11904539880924858) - rollout_corr/kl:np.float64(0.00041123665303999246) - rollout_corr/k3_kl:np.float64(0.0004748523673612226) - rollout_corr/rollout_ppl:np.float64(1.1276400778442621) - rollout_corr/rollout_log_ppl:np.float64(0.11863416097185109) - rollout_corr/log_ppl_diff:np.float64(0.0004112378373974934) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012866570759797469) - rollout_corr/log_ppl_diff_max:np.float64(0.007371693849563599) - rollout_corr/log_ppl_diff_min:np.float64(-0.0064626410603523254) - rollout_corr/ppl_ratio:np.float64(1.0004128504078835) - rollout_corr/chi2_token:np.float64(0.0010835977736860514) - rollout_corr/chi2_seq:np.float64(0.24060776853002608) - actor/pg_loss:np.float64(0.0005744848167523742) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000119695831926947) - actor/ppo_kl:np.float64(1.3767832875544173e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(2840.6781995082274) - self_distillation/token_reweight_w_std:np.float64(53555.554571515124) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997150534763932) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015691433451138437) - self_distillation/empty_target_batch:np.float64(0.375) - actor/grad_norm:np.float64(0.10368238762021065) - perf/mfu/actor:np.float64(0.0549931270347762) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.1422119140625) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:3 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-9.925667836796492e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-9.925667836796492e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:354.1953125 - response_length/max:654.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:354.1953125 - response_length_non_aborted/max:654.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.59375 - prompt_length/max:505.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012639164924621582 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2934284508228302) - timing_s/agent_loop/generate_sequences/max:np.float64(6.372750483453274) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.1832115383585915) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.372750483453274) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:654 - timing_s/gen:13.704486019909382 - timing_s/reward:0.06210590712726116 - timing_s/old_log_prob:2.920646356418729 - timing_s/adv:0.5429920088499784 - timing_s/update_actor:23.081482391804457 - timing_s/step:40.39900127798319 - timing_s/stop_profile:0.00011100992560386658 - timing_per_token_ms/adv:0.0027843173904458997 - timing_per_token_ms/update_actor:0.11835565123119125 - timing_per_token_ms/gen:0.15114019476265944 - perf/total_num_tokens:195018 - perf/time_per_step:40.39900127798319 - perf/throughput:603.4121940852338 (TaskRunner pid=2425326) Training Progress: 96%|█████████▌| 96/100 [1:13:46<02:49, 42.27s/it] (TaskRunner pid=2425326) step:97 - global_seqlen/min:20013 - global_seqlen/max:25357 - global_seqlen/minmax_diff:5344 - global_seqlen/balanced_min:22630 - global_seqlen/balanced_max:22638 - global_seqlen/mean:22635.25 - actor/entropy:0.1280536651611328 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463575839996338 - training/rollout_probs_diff_mean:0.0024065831676125526 - training/rollout_probs_diff_std:0.009107198566198349 - training/rollout_actor_probs_pearson_corr:0.9987413883209229 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.84375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:1.0000966787338257 - rollout_corr/rollout_is_ratio_fraction_high:1.1470783647382632e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1470783647382632e-05 - rollout_corr/rollout_is_max:2.5134615898132324 - rollout_corr/rollout_is_min:0.4878324866294861 - rollout_corr/rollout_is_std:0.025915730744600296 - rollout_corr/rollout_is_eff_sample_size:0.9993290637389448 - rollout_corr/rollout_is_seq_mean:1.000105857849121 - rollout_corr/rollout_is_seq_std:0.00157264550216496 - rollout_corr/rollout_is_seq_max:1.00606107711792 - rollout_corr/rollout_is_seq_min:0.994608998298645 - rollout_corr/rollout_is_seq_max_deviation:0.006061077117919922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1334166121669114) - rollout_corr/training_log_ppl:np.float64(0.12309159211145015) - rollout_corr/kl:np.float64(0.000282021749916872) - rollout_corr/k3_kl:np.float64(0.00044198074789392194) - rollout_corr/rollout_ppl:np.float64(1.1330834380351007) - rollout_corr/rollout_log_ppl:np.float64(0.1228095693004434) - rollout_corr/log_ppl_diff:np.float64(0.00028202281100675464) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012496994895627722) - rollout_corr/log_ppl_diff_max:np.float64(0.007074952125549316) - rollout_corr/log_ppl_diff_min:np.float64(-0.00588814914226532) - rollout_corr/ppl_ratio:np.float64(1.000283483415842) - rollout_corr/chi2_token:np.float64(0.0012291779275983572) - rollout_corr/chi2_seq:np.float64(0.3651922077406198) - actor/pg_loss:np.float64(0.000943023944273591) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001242990306309366) - actor/ppo_kl:np.float64(4.270520436566727e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_w_mean:np.float64(30499.929090753663) - self_distillation/token_reweight_w_std:np.float64(639412.5114061498) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000444219913334) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015111897548194975) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.09439769387245178) - perf/mfu/actor:np.float64(0.051329233916159775) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.2738265991211) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:3 - critic/score/mean:0.84375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00036132970126345754 - critic/advantages/max:0.75 - critic/advantages/min:-0.75 - critic/returns/mean:0.00036132970126345754 - critic/returns/max:0.75 - critic/returns/min:-0.75 - response_length/mean:340.5390625 - response_length/max:751.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:340.5390625 - response_length_non_aborted/max:751.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:366.8125 - prompt_length/max:490.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001009870320558548 - timing_s/agent_loop/generate_sequences/min:np.float64(1.119951417669654) - timing_s/agent_loop/generate_sequences/max:np.float64(7.088973863050342) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.975005234715354) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.088973863050342) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:459 - timing_s/agent_loop/slowest/response_length:751 - timing_s/gen:14.353797556832433 - timing_s/reward:0.059515826404094696 - timing_s/old_log_prob:2.9649640545248985 - timing_s/adv:0.5532467886805534 - timing_s/update_actor:22.970771735534072 - timing_s/step:40.98978105187416 - timing_s/stop_profile:0.00012583844363689423 - timing_per_token_ms/adv:0.0030552279557358184 - timing_per_token_ms/update_actor:0.12685287182345054 - timing_per_token_ms/gen:0.16464931011072098 - perf/total_num_tokens:181082 - perf/time_per_step:40.98978105187416 - perf/throughput:552.2169042902233 (TaskRunner pid=2425326) Training Progress: 97%|█████████▋| 97/100 [1:14:27<02:05, 41.90s/it] (TaskRunner pid=2425326) step:98 - global_seqlen/min:21562 - global_seqlen/max:27701 - global_seqlen/minmax_diff:6139 - global_seqlen/balanced_min:24212 - global_seqlen/balanced_max:24218 - global_seqlen/mean:24216.375 - actor/entropy:0.1430254727602005 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20211440324783325 - training/rollout_probs_diff_mean:0.002688319655135274 - training/rollout_probs_diff_std:0.009453684091567993 - training/rollout_actor_probs_pearson_corr:0.9988110065460205 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.89453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9998697638511658 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6097447872161865 - rollout_corr/rollout_is_min:0.5124151706695557 - rollout_corr/rollout_is_std:0.027273906394839287 - rollout_corr/rollout_is_eff_sample_size:0.9992565679263558 - rollout_corr/rollout_is_seq_mean:0.9998807907104492 - rollout_corr/rollout_is_seq_std:0.0015214976156130433 - rollout_corr/rollout_is_seq_max:1.0076751708984375 - rollout_corr/rollout_is_seq_min:0.9955283999443054 - rollout_corr/rollout_is_seq_max_deviation:0.0076751708984375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.151344778481871) - rollout_corr/training_log_ppl:np.float64(0.13949581736596883) - rollout_corr/kl:np.float64(0.0008577270308975926) - rollout_corr/k3_kl:np.float64(0.0005512973715582348) - rollout_corr/rollout_ppl:np.float64(1.1503589190542698) - rollout_corr/rollout_log_ppl:np.float64(0.13863808915994014) - rollout_corr/log_ppl_diff:np.float64(0.0008577282060286961) - rollout_corr/log_ppl_abs_diff:np.float64(0.001595133486262057) - rollout_corr/log_ppl_diff_max:np.float64(0.008243545889854431) - rollout_corr/log_ppl_diff_min:np.float64(-0.006423532962799072) - rollout_corr/ppl_ratio:np.float64(1.0008599534630775) - rollout_corr/chi2_token:np.float64(0.00048058945685625076) - rollout_corr/chi2_seq:np.float64(0.11350936791859567) - actor/pg_loss:np.float64(0.0004699843702837825) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018328063106309855) - actor/ppo_kl:np.float64(0.0003591571183854114) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_w_mean:np.float64(452.343374283053) - self_distillation/token_reweight_w_std:np.float64(8862.90448090632) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999787393026054) - self_distillation/token_reweight_w_clip_frac:np.float64(0.011719158239429817) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.08010996598750353) - perf/mfu/actor:np.float64(0.05323706476936674) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.16266250610352) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:3 - critic/score/mean:0.89453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.89453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006149417022243142 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006149417022243142 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:354.13671875 - response_length/max:725.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:354.13671875 - response_length_non_aborted/max:725.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:402.625 - prompt_length/max:486.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014889240264892578 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9090332705527544) - timing_s/agent_loop/generate_sequences/max:np.float64(6.56995116174221) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.119346796629543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.56995116174221) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:486 - timing_s/agent_loop/slowest/response_length:725 - timing_s/gen:14.108668487519026 - timing_s/reward:0.06148545444011688 - timing_s/old_log_prob:2.925100963562727 - timing_s/adv:0.5809599440544844 - timing_s/update_actor:23.736185651272535 - timing_s/step:41.49837529100478 - timing_s/stop_profile:0.00011325441300868988 - timing_per_token_ms/adv:0.002998797012633416 - timing_per_token_ms/update_actor:0.12252136029480329 - timing_per_token_ms/gen:0.15562347353841346 - perf/total_num_tokens:193731 - perf/time_per_step:41.49837529100478 - perf/throughput:583.5499541893912 (TaskRunner pid=2425326) Training Progress: 98%|█████████▊| 98/100 [1:15:09<01:23, 41.80s/it] (TaskRunner pid=2425326) step:99 - global_seqlen/min:20094 - global_seqlen/max:26591 - global_seqlen/minmax_diff:6497 - global_seqlen/balanced_min:23799 - global_seqlen/balanced_max:23806 - global_seqlen/mean:23804.125 - actor/entropy:0.15504607558250427 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744115471839905 - training/rollout_probs_diff_mean:0.002842691959813237 - training/rollout_probs_diff_std:0.009904769249260426 - training/rollout_actor_probs_pearson_corr:0.9986906051635742 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.000064492225647 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.117555711971363e-05 - rollout_corr/rollout_is_max:1.6279256343841553 - rollout_corr/rollout_is_min:0.44314810633659363 - rollout_corr/rollout_is_std:0.0281895212829113 - rollout_corr/rollout_is_eff_sample_size:0.9992061008617588 - rollout_corr/rollout_is_seq_mean:1.0000643730163574 - rollout_corr/rollout_is_seq_std:0.0016942165093496442 - rollout_corr/rollout_is_seq_max:1.006766438484192 - rollout_corr/rollout_is_seq_min:0.9935619235038757 - rollout_corr/rollout_is_seq_max_deviation:0.0067664384841918945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1785350814461708) - rollout_corr/training_log_ppl:np.float64(0.15894308649149025) - rollout_corr/kl:np.float64(0.0013515585573031785) - rollout_corr/k3_kl:np.float64(0.0012069832163774663) - rollout_corr/rollout_ppl:np.float64(1.1769968876615167) - rollout_corr/rollout_log_ppl:np.float64(0.1575915266002994) - rollout_corr/log_ppl_diff:np.float64(0.0013515599057427607) - rollout_corr/log_ppl_abs_diff:np.float64(0.002168246872315649) - rollout_corr/log_ppl_diff_max:np.float64(0.06749674677848816) - rollout_corr/log_ppl_diff_min:np.float64(-0.004221692681312561) - rollout_corr/ppl_ratio:np.float64(1.0013778761494905) - rollout_corr/chi2_token:np.float64(0.0007215156219899654) - rollout_corr/chi2_seq:np.float64(0.20517127867788076) - actor/pg_loss:np.float64(0.0009915356058627367) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020690295605163556) - actor/ppo_kl:np.float64(0.000960174699570826) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(7099.986027709674) - self_distillation/token_reweight_w_std:np.float64(116633.08640508627) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997013744432479) - self_distillation/token_reweight_w_clip_frac:np.float64(0.021721423909184523) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.15946919191628695) - perf/mfu/actor:np.float64(0.05383568726534263) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.18372344970703) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:3 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029405683744698763 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0029405683744698763 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:349.53515625 - response_length/max:706.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:349.53515625 - response_length_non_aborted/max:706.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.34375 - prompt_length/max:480.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.759515523910522e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8449258226901293) - timing_s/agent_loop/generate_sequences/max:np.float64(6.61848109215498) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.178213986058836) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.61848109215498) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:706 - timing_s/gen:14.051712546497583 - timing_s/reward:0.0617553386837244 - timing_s/old_log_prob:3.0272172279655933 - timing_s/adv:0.5545117557048798 - timing_s/update_actor:22.762175181880593 - timing_s/step:40.54527600854635 - timing_s/stop_profile:0.00014713220298290253 - timing_per_token_ms/adv:0.002911846978753051 - timing_per_token_ms/update_actor:0.11952852279741744 - timing_per_token_ms/gen:0.15703571201146146 - perf/total_num_tokens:190433 - perf/time_per_step:40.54527600854635 - perf/throughput:587.0998385849547 (TaskRunner pid=2425326) Training Progress: 99%|█████████▉| 99/100 [1:15:50<00:41, 41.44s/it] (TaskRunner pid=2425326) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2425326) validation generation end (TaskRunner pid=2425326) [prompt] system (TaskRunner pid=2425326) (TaskRunner pid=2425326) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) ... (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2425326) (TaskRunner pid=2425326) A (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) user (TaskRunner pid=2425326) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2425326) (TaskRunner pid=2425326) Lattice Parameters: (TaskRunner pid=2425326) (TaskRunner pid=2425326) a = 7.82 Å (TaskRunner pid=2425326) b = 7.82 Å (TaskRunner pid=2425326) c = 17.88 Å (TaskRunner pid=2425326) α = 90.00º (TaskRunner pid=2425326) β = 90.00º (TaskRunner pid=2425326) γ = 90.00º (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) A: 1025.67 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) B: 1094.31 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) C: 1200.45 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) D: 1150.78 ų (TaskRunner pid=2425326) (TaskRunner pid=2425326) Please reason step by step. (TaskRunner pid=2425326) assistant (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) [response] (TaskRunner pid=2425326) The given crystal structure has lattice parameters a = 7.82 Å, b = 7.82 Å, c = 17.88 Å, and all angles are 90 degrees, indicating a orthorhombic unit cell. The volume of a unit cell in an orthorhombic system is calculated using the formula: (TaskRunner pid=2425326) (TaskRunner pid=2425326) V = a × b × c (TaskRunner pid=2425326) (TaskRunner pid=2425326) Substituting the given values: (TaskRunner pid=2425326) (TaskRunner pid=2425326) V = 7.82 × 7.82 × 17.88 (TaskRunner pid=2425326) (TaskRunner pid=2425326) First, calculate 7.82 × 7.82: (TaskRunner pid=2425326) (TaskRunner pid=2425326) 7.82 × 7.82 = 61.1524 (TaskRunner pid=2425326) (TaskRunner pid=2425326) Then multiply by 17.88: (TaskRunner pid=2425326) (TaskRunner pid=2425326) 61.1524 × 17.88 ≈ 1094.31 (TaskRunner pid=2425326) (TaskRunner pid=2425326) Therefore, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2425326) (TaskRunner pid=2425326) (TaskRunner pid=2425326) B (TaskRunner pid=2425326) (TaskRunner pid=2425326) [ground_truth] B (TaskRunner pid=2425326) [score] 1.0 (TaskRunner pid=2425326) [acc] 1.0 (TaskRunner pid=2425326) [pred] B (TaskRunner pid=2425326) [incorrect_format] 1 (TaskRunner pid=2425326) [feedback] (TaskRunner pid=2425326) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2425326) step:100 - global_seqlen/min:20320 - global_seqlen/max:28053 - global_seqlen/minmax_diff:7733 - global_seqlen/balanced_min:23678 - global_seqlen/balanced_max:23681 - global_seqlen/mean:23679.5 - actor/entropy:0.15241530537605286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25462448596954346 - training/rollout_probs_diff_mean:0.002963797887787223 - training/rollout_probs_diff_std:0.010244213975965977 - training/rollout_actor_probs_pearson_corr:0.9986335039138794 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:1.0000261068344116 - rollout_corr/rollout_is_ratio_fraction_high:1.119720491260523e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0033769607543945 - rollout_corr/rollout_is_min:0.567776620388031 - rollout_corr/rollout_is_std:0.029670871794223785 - rollout_corr/rollout_is_eff_sample_size:0.9991205327494357 - rollout_corr/rollout_is_seq_mean:1.0000578165054321 - rollout_corr/rollout_is_seq_std:0.0018322683172300458 - rollout_corr/rollout_is_seq_max:1.0058804750442505 - rollout_corr/rollout_is_seq_min:0.9950834512710571 - rollout_corr/rollout_is_seq_max_deviation:0.005880475044250488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1711667915806174) - rollout_corr/training_log_ppl:np.float64(0.15226043093571207) - rollout_corr/kl:np.float64(0.00048633673415032774) - rollout_corr/k3_kl:np.float64(0.0005004820688583322) - rollout_corr/rollout_ppl:np.float64(1.170548481401056) - rollout_corr/rollout_log_ppl:np.float64(0.15177409329044167) - rollout_corr/log_ppl_diff:np.float64(0.000486337645270396) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015185297597781755) - rollout_corr/log_ppl_diff_max:np.float64(0.008212178945541382) - rollout_corr/log_ppl_diff_min:np.float64(-0.004179961979389191) - rollout_corr/ppl_ratio:np.float64(1.0004883385263383) - rollout_corr/chi2_token:np.float64(0.0010083636734634638) - rollout_corr/chi2_seq:np.float64(0.34577386314049363) - actor/pg_loss:np.float64(0.0005346338730305433) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.5688376050966326e-05) - actor/ppo_kl:np.float64(0.00010043691768402141) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(1.0763714532367885) - self_distillation/token_reweight_w_std:np.float64(0.7491657065111212) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991688379086554) - self_distillation/token_reweight_w_clip_frac:np.float64(0.016922423528740183) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.08581123873591423) - perf/mfu/actor:np.float64(0.05333022759720618) - perf/max_memory_allocated_gb:np.float64(130.98332166671753) - perf/max_memory_reserved_gb:np.float64(140.86328125) - perf/cpu_memory_used_gb:np.float64(196.12500381469727) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7646276595744681) - val-aux/sciknoweval/reward/std@16:np.float64(0.0401737640202366) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7809255319148936) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.03462769102759328) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7494468085106384) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.03367836377537821) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7651063829787235) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.040366939474519616) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7944468085106383) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.027759320424956824) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7366170212765957) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.025172389625263896) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.764436170212766) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.031638291528035434) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8061702127659575) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.019636408581214976) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.726340425531915) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.018341796156463988) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.764223404255319) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.020384144360752216) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.813808510638298) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.011791035148083585) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.7182659574468084) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.01148574534900323) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7633617021276595) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.013016076917955125) - val-aux/sciknoweval/score/mean@16:np.float64(0.7646276595744681) - val-aux/sciknoweval/score/std@16:np.float64(0.0401737640202366) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7809255319148936) - val-aux/sciknoweval/score/best@2/std:np.float64(0.03462769102759328) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7494468085106384) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.03367836377537821) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7651063829787235) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.040366939474519616) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7944468085106383) - val-aux/sciknoweval/score/best@4/std:np.float64(0.027759320424956824) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7366170212765957) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.025172389625263896) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.764436170212766) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.031638291528035434) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8061702127659575) - val-aux/sciknoweval/score/best@8/std:np.float64(0.019636408581214976) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.726340425531915) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.018341796156463988) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.764223404255319) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.020384144360752216) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.813808510638298) - val-aux/sciknoweval/score/best@16/std:np.float64(0.011791035148083585) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.7182659574468084) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.01148574534900323) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7633617021276595) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.013016076917955125) - val-core/sciknoweval/acc/mean@16:np.float64(0.7646276595744681) - val-aux/sciknoweval/acc/std@16:np.float64(0.0401737640202366) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7809255319148936) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.03462769102759328) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7494468085106384) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.03367836377537821) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7651063829787235) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.040366939474519616) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7944468085106383) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.027759320424956824) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7366170212765957) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.025172389625263896) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.764436170212766) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.031638291528035434) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8061702127659575) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.019636408581214976) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.726340425531915) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.018341796156463988) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.764223404255319) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.020384144360752216) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.813808510638298) - val-core/sciknoweval/acc/best@16/std:np.float64(0.011791035148083585) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.7182659574468084) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.01148574534900323) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7633617021276595) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.013016076917955125) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:3 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012260939693078399 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012260939693078399 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:348.859375 - response_length/max:607.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:348.859375 - response_length_non_aborted/max:607.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.125 - prompt_length/max:479.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.56319272518158e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0379578452557325) - timing_s/agent_loop/generate_sequences/max:np.float64(6.074218215420842) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.135219432209851) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.074218215420842) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:414 - timing_s/agent_loop/slowest/response_length:607 - timing_s/gen:13.520746255293489 - timing_s/reward:0.06225111149251461 - timing_s/old_log_prob:2.993232659995556 - timing_s/adv:0.5430567041039467 - timing_s/update_actor:23.16192770190537 - timing_s/step:40.37044422328472 - timing_s/testing:40.37994212470949 - timing_s/save_checkpoint:12.537891793996096 - timing_s/stop_profile:0.00013117678463459015 - timing_per_token_ms/adv:0.0028667027603198267 - timing_per_token_ms/update_actor:0.12226782502747825 - timing_per_token_ms/gen:0.15139456997462142 - perf/total_num_tokens:189436 - perf/time_per_step:40.37044422328472 - perf/throughput:586.5553489833095 (TaskRunner pid=2425326) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2425326) "np.float64(0.7646276595744681), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2425326) "np.float64(0.0401737640202366), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2425326) "np.float64(0.7809255319148936), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2425326) "np.float64(0.03462769102759328), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2425326) "np.float64(0.7494468085106384), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2425326) "np.float64(0.03367836377537821), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2425326) "np.float64(0.7651063829787235), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2425326) "np.float64(0.040366939474519616), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2425326) "np.float64(0.7944468085106383), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2425326) "np.float64(0.027759320424956824), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2425326) "np.float64(0.7366170212765957), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2425326) "np.float64(0.025172389625263896), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2425326) "np.float64(0.764436170212766), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2425326) "np.float64(0.031638291528035434), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2425326) "np.float64(0.8061702127659575), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2425326) "np.float64(0.019636408581214976), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2425326) "np.float64(0.726340425531915), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2425326) "np.float64(0.018341796156463988), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2425326) "np.float64(0.764223404255319), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2425326) "np.float64(0.020384144360752216), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2425326) "np.float64(0.813808510638298), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2425326) 'np.float64(0.011791035148083585), ' (TaskRunner pid=2425326) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.7182659574468084), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.01148574534900323), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7633617021276595), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.013016076917955125), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/mean@16': np.float64(0.7646276595744681), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/std@16': np.float64(0.0401737640202366), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.7809255319148936), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.03462769102759328), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.7494468085106384), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.03367836377537821), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.7651063829787235), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.040366939474519616), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.7944468085106383), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.027759320424956824), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.7366170212765957), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.025172389625263896), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.764436170212766), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.031638291528035434), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8061702127659575), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.019636408581214976), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.726340425531915), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.018341796156463988), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.764223404255319), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.020384144360752216), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.813808510638298), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.011791035148083585), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.7182659574468084), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.01148574534900323), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7633617021276595), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.013016076917955125), " (TaskRunner pid=2425326) "'val-core/sciknoweval/acc/mean@16': np.float64(0.7646276595744681), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/std@16': np.float64(0.0401737640202366), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.7809255319148936), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.03462769102759328), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.7494468085106384), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.03367836377537821), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.7651063829787235), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.040366939474519616), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.7944468085106383), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.027759320424956824), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.7366170212765957), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.025172389625263896), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.764436170212766), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.031638291528035434), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8061702127659575), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.019636408581214976), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.726340425531915), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.018341796156463988), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.764223404255319), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.020384144360752216), " (TaskRunner pid=2425326) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.813808510638298), " (TaskRunner pid=2425326) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.011791035148083585), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.7182659574468084), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.01148574534900323), " (TaskRunner pid=2425326) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7633617021276595), " (TaskRunner pid=2425326) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.013016076917955125), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/std@16': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/best@2/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/worst@2/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/maj@2/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/worst@4/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/maj@4/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/worst@8/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/worst@16/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2425326) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2425326) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2425326) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2425326) Training Progress: 100%|██████████| 100/100 [1:17:23<00:00, 57.02s/it] Training Progress: 100%|██████████| 100/100 [1:17:23<00:00, 46.43s/it] (TaskRunner pid=2425326) wandb: updating run metadata; uploading output.log; uploading wandb-summary.json (TaskRunner pid=2425326) wandb: uploading output.log (TaskRunner pid=2425326) wandb: uploading config.yaml (TaskRunner pid=2425326) wandb: uploading history steps 98-99, summary, console lines 893-1000 (TaskRunner pid=2425326) wandb: (TaskRunner pid=2425326) wandb: Run history: (TaskRunner pid=2425326) wandb: actor/entropy ▄▅▆▆▅▆▆▆▅▂▁▂▂▅▆▅▆▆▆▆▆█▅▆▆▄▅▆▅▃▅▄▅▅▅▃▃▄▅▅ (TaskRunner pid=2425326) wandb: actor/grad_norm █▇▆▇▅▄▄▆▄▄▆▆▆▅▇▄▃▂▃▄▆▅▃▂▃▂▁▁▄▃▂▁▂▄▂▂▃▃▅▂ (TaskRunner pid=2425326) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2425326) wandb: actor/lr ▁▅▆▇████████████████████████████████████ (TaskRunner pid=2425326) wandb: actor/pg_clipfrac ▆▄▅██▃▄▄▅▃▃▄▅▂▄▁▆▄▃▅▂▃▂▄▂▃▁▂▂▂▃▂▂▁▃▂▃▁▃▃ (TaskRunner pid=2425326) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁▁▇▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2425326) wandb: actor/pg_loss ▄▆█▆▆▃▆▃▃▅▅▃▄▃▃▂▃▅▄▄▃▂▃▃▂▄▃▂▁▄▃▂▂▁▁▂▃▂▂▁ (TaskRunner pid=2425326) wandb: actor/ppo_kl ▃▃▄▅▂▆▃▃▄▃▇▆▃▇▄▅▅▄▄▆▄▅▂▁▃▅▆▃▂▃▃▄▃▄▅▃▃▄█▄ (TaskRunner pid=2425326) wandb: critic/advantages/max █████▇▇▇▇▇███▇██▇▇█████▆▂█▇██▇█▆█▆▆▇███▁ (TaskRunner pid=2425326) wandb: critic/advantages/mean ▄▇█▂▆▃▅▆▃▅▅▄▃▃▂▄▂▂▁▃▅▅▄▃▃▂▃▃▃▅▃▁▂▃▂▆▅▂▅▂ (TaskRunner pid=2425326) wandb: +220 ... (TaskRunner pid=2425326) wandb: (TaskRunner pid=2425326) wandb: Run summary: (TaskRunner pid=2425326) wandb: actor/entropy 0.15242 (TaskRunner pid=2425326) wandb: actor/grad_norm 0.08581 (TaskRunner pid=2425326) wandb: actor/kl_loss 0 (TaskRunner pid=2425326) wandb: actor/lr 0.0 (TaskRunner pid=2425326) wandb: actor/pg_clipfrac 6e-05 (TaskRunner pid=2425326) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2425326) wandb: actor/pg_loss 0.00053 (TaskRunner pid=2425326) wandb: actor/ppo_kl 0.0001 (TaskRunner pid=2425326) wandb: critic/advantages/max 0.125 (TaskRunner pid=2425326) wandb: critic/advantages/mean -0.00123 (TaskRunner pid=2425326) wandb: +220 ... (TaskRunner pid=2425326) wandb: (TaskRunner pid=2425326) wandb: 🚀 View run qwen3gen-material-RLSD_TR-Qwen-Qwen3-8B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/skza3cpu (TaskRunner pid=2425326) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2425326) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2425326) wandb: Find logs at: ./wandb/run-20260703_115716-skza3cpu/logs (TaskRunner pid=2425326) Exception ignored in atexit callback: .teardown_atexit at 0x745be0fdf880> (TaskRunner pid=2425326) Traceback (most recent call last): (TaskRunner pid=2425326) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2425326) conn.teardown(hooks.exit_code) (TaskRunner pid=2425326) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2425326) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2425326) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2425326) return future.result() (TaskRunner pid=2425326) ^^^^^^^^^^^^^^^ (TaskRunner pid=2425326) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2425326) return self.__get_result() (TaskRunner pid=2425326) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2425326) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2425326) raise self._exception (TaskRunner pid=2425326) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2425326) return await fn() (TaskRunner pid=2425326) ^^^^^^^^^^ (TaskRunner pid=2425326) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2425326) await self._client.close() (TaskRunner pid=2425326) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2425326) await self._writer.wait_closed() (TaskRunner pid=2425326) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2425326) await self._protocol._get_close_waiter(self) (TaskRunner pid=2425326) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-03 13:14:53] Finished Qwen/Qwen3-8B material rlsd_tr. [2026-07-03 13:14:53] Starting Qwen/Qwen3-8B tooluse grpo. Experiment: qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: tooluse Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-8B Train: /mnt/mole/SDPO/L2T/datasets/tooluse/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/tooluse/test.parquet Ray tmp: /tmp/ray_q3g_tooluse_grpo_Qwen3_8B 2026-07-03 13:14:54,970 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/tooluse Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/tooluse/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/tooluse/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_tooluse_grpo_Qwen3_8B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/tooluse +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-8B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 trainer.max_actor_ckpt_to_keep=100 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_tooluse_grpo_Qwen3_8B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/tooluse', 'EXPERIMENT': 'qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-03 13:15:04,797 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2448026) TaskRunner hostname: mole-workspace-rw, PID: 2448026 (TaskRunner pid=2448026) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2448026) 'calculate_entropy': False, (TaskRunner pid=2448026) 'calculate_sum_pi_squared': False, (TaskRunner pid=2448026) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2448026) 'async_save': False, (TaskRunner pid=2448026) 'load_contents': ['model', (TaskRunner pid=2448026) 'optimizer', (TaskRunner pid=2448026) 'extra'], (TaskRunner pid=2448026) 'save_contents': ['model', (TaskRunner pid=2448026) 'optimizer', (TaskRunner pid=2448026) 'extra']}, (TaskRunner pid=2448026) 'clip_ratio': 0.2, (TaskRunner pid=2448026) 'clip_ratio_c': 3.0, (TaskRunner pid=2448026) 'clip_ratio_high': 0.28, (TaskRunner pid=2448026) 'clip_ratio_low': 0.2, (TaskRunner pid=2448026) 'data_loader_seed': 42, (TaskRunner pid=2448026) 'entropy_checkpointing': False, (TaskRunner pid=2448026) 'entropy_coeff': 0, (TaskRunner pid=2448026) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2448026) 'freeze_vision_tower': False, (TaskRunner pid=2448026) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2448026) 'dtype': 'bfloat16', (TaskRunner pid=2448026) 'entropy_checkpointing': False, (TaskRunner pid=2448026) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2448026) 'forward_only': False, (TaskRunner pid=2448026) 'forward_prefetch': False, (TaskRunner pid=2448026) 'fsdp_size': -1, (TaskRunner pid=2448026) 'full_determinism': False, (TaskRunner pid=2448026) 'model_dtype': 'fp32', (TaskRunner pid=2448026) 'offload_policy': False, (TaskRunner pid=2448026) 'optimizer_offload': False, (TaskRunner pid=2448026) 'param_offload': False, (TaskRunner pid=2448026) 'reshard_after_forward': True, (TaskRunner pid=2448026) 'seed': 42, (TaskRunner pid=2448026) 'strategy': 'fsdp', (TaskRunner pid=2448026) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2448026) 'use_orig_params': False, (TaskRunner pid=2448026) 'use_torch_compile': True, (TaskRunner pid=2448026) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2448026) 'grad_clip': 1.0, (TaskRunner pid=2448026) 'kl_loss_coef': 0.0, (TaskRunner pid=2448026) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2448026) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2448026) 'loss_scale_factor': None, (TaskRunner pid=2448026) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2448026) 'betas': [0.9, 0.999], (TaskRunner pid=2448026) 'clip_grad': 1.0, (TaskRunner pid=2448026) 'lr': 1e-06, (TaskRunner pid=2448026) 'lr_scheduler_type': 'constant', (TaskRunner pid=2448026) 'lr_warmup_steps': 10, (TaskRunner pid=2448026) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2448026) 'min_lr_ratio': 0.0, (TaskRunner pid=2448026) 'num_cycles': 0.5, (TaskRunner pid=2448026) 'optimizer': 'AdamW', (TaskRunner pid=2448026) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2448026) 'override_optimizer_config': None, (TaskRunner pid=2448026) 'total_training_steps': -1, (TaskRunner pid=2448026) 'warmup_style': None, (TaskRunner pid=2448026) 'weight_decay': 0.01}, (TaskRunner pid=2448026) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2448026) 'clip_cov_lb': 1.0, (TaskRunner pid=2448026) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2448026) 'clip_cov_ub': 5.0, (TaskRunner pid=2448026) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2448026) 'loss_mode': 'vanilla', (TaskRunner pid=2448026) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2448026) 'ppo_epochs': 1, (TaskRunner pid=2448026) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2448026) 'ppo_micro_batch_size': None, (TaskRunner pid=2448026) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2448026) 'ppo_mini_batch_size': 8, (TaskRunner pid=2448026) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2448026) 'all_ranks': False, (TaskRunner pid=2448026) 'enable': False, (TaskRunner pid=2448026) 'ranks': [], (TaskRunner pid=2448026) 'save_path': 'outputs/profile', (TaskRunner pid=2448026) 'tool': None, (TaskRunner pid=2448026) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2448026) 'analysis': True, (TaskRunner pid=2448026) 'contents': [], (TaskRunner pid=2448026) 'discrete': False, (TaskRunner pid=2448026) 'level': 'level0'}, (TaskRunner pid=2448026) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2448026) 'discrete': False}, (TaskRunner pid=2448026) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2448026) 'step_end': None, (TaskRunner pid=2448026) 'step_start': 0}, (TaskRunner pid=2448026) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2448026) 'stack_depth': 32, (TaskRunner pid=2448026) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2448026) 'rollout_n': 8, (TaskRunner pid=2448026) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2448026) 'mode': 'disabled', (TaskRunner pid=2448026) 'record_file': None, (TaskRunner pid=2448026) 'replay_file': None}, (TaskRunner pid=2448026) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2448026) 'alpha': 0.5, (TaskRunner pid=2448026) 'distillation_add_tail': True, (TaskRunner pid=2448026) 'distillation_topk': 100, (TaskRunner pid=2448026) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2448026) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2448026) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2448026) 'enable': False, (TaskRunner pid=2448026) 'loss_weight': 0.0, (TaskRunner pid=2448026) 'mask': 'all'}, (TaskRunner pid=2448026) 'feedback_template': '\n' (TaskRunner pid=2448026) 'The ' (TaskRunner pid=2448026) 'following ' (TaskRunner pid=2448026) 'is ' (TaskRunner pid=2448026) 'feedback ' (TaskRunner pid=2448026) 'from ' (TaskRunner pid=2448026) 'your ' (TaskRunner pid=2448026) 'unsuccessful ' (TaskRunner pid=2448026) 'earlier ' (TaskRunner pid=2448026) 'attempt:\n' (TaskRunner pid=2448026) '\n' (TaskRunner pid=2448026) '{feedback_raw}', (TaskRunner pid=2448026) 'full_logit_distillation': True, (TaskRunner pid=2448026) 'include_environment_feedback': True, (TaskRunner pid=2448026) 'is_clip': 2, (TaskRunner pid=2448026) 'max_reprompt_len': 22528, (TaskRunner pid=2448026) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2448026) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2448026) '\n' (TaskRunner pid=2448026) 'Correctly ' (TaskRunner pid=2448026) 'solve ' (TaskRunner pid=2448026) 'the ' (TaskRunner pid=2448026) 'original ' (TaskRunner pid=2448026) 'question.', (TaskRunner pid=2448026) 'reprompt_truncation': 'right', (TaskRunner pid=2448026) 'solution_template': '\n' (TaskRunner pid=2448026) 'Correct ' (TaskRunner pid=2448026) 'solution:\n' (TaskRunner pid=2448026) '\n' (TaskRunner pid=2448026) '{successful_previous_attempt}', (TaskRunner pid=2448026) 'srpo_dynamic_weighting': False, (TaskRunner pid=2448026) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2448026) 'success_reward_threshold': 0.5, (TaskRunner pid=2448026) 'teacher_regularization': 'ema', (TaskRunner pid=2448026) 'teacher_update_rate': 0.0, (TaskRunner pid=2448026) 'token_reweight_decay_steps': None, (TaskRunner pid=2448026) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2448026) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2448026) 'shuffle': False, (TaskRunner pid=2448026) 'strategy': 'fsdp', (TaskRunner pid=2448026) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2448026) 'tau_neg': 1.05, (TaskRunner pid=2448026) 'tau_pos': 1.0, (TaskRunner pid=2448026) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2448026) 'use_dynamic_bsz': False, (TaskRunner pid=2448026) 'use_fused_kernels': False, (TaskRunner pid=2448026) 'use_kl_loss': False, (TaskRunner pid=2448026) 'use_prefix_grouper': False, (TaskRunner pid=2448026) 'use_remove_padding': True, (TaskRunner pid=2448026) 'use_torch_compile': True}, (TaskRunner pid=2448026) 'hybrid_engine': True, (TaskRunner pid=2448026) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2448026) 'custom_chat_template': None, (TaskRunner pid=2448026) 'enable_activation_offload': False, (TaskRunner pid=2448026) 'enable_gradient_checkpointing': True, (TaskRunner pid=2448026) 'exclude_modules': None, (TaskRunner pid=2448026) 'external_lib': None, (TaskRunner pid=2448026) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2448026) 'hf_config_path': None, (TaskRunner pid=2448026) 'lora_adapter_path': None, (TaskRunner pid=2448026) 'lora_alpha': 16, (TaskRunner pid=2448026) 'lora_rank': 0, (TaskRunner pid=2448026) 'override_config': {}, (TaskRunner pid=2448026) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2448026) 'target_modules': 'all-linear', (TaskRunner pid=2448026) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2448026) 'num_shards': 4}, (TaskRunner pid=2448026) 'tokenizer_path': None, (TaskRunner pid=2448026) 'trust_remote_code': True, (TaskRunner pid=2448026) 'use_fused_kernels': False, (TaskRunner pid=2448026) 'use_liger': False, (TaskRunner pid=2448026) 'use_remove_padding': True, (TaskRunner pid=2448026) 'use_shm': False}, (TaskRunner pid=2448026) 'nccl_timeout': 600, (TaskRunner pid=2448026) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2448026) 'entropy_checkpointing': False, (TaskRunner pid=2448026) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2448026) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2448026) 'dtype': 'bfloat16', (TaskRunner pid=2448026) 'entropy_checkpointing': False, (TaskRunner pid=2448026) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2448026) 'forward_only': True, (TaskRunner pid=2448026) 'forward_prefetch': False, (TaskRunner pid=2448026) 'fsdp_size': -1, (TaskRunner pid=2448026) 'full_determinism': False, (TaskRunner pid=2448026) 'model_dtype': 'fp32', (TaskRunner pid=2448026) 'offload_policy': False, (TaskRunner pid=2448026) 'optimizer_offload': False, (TaskRunner pid=2448026) 'param_offload': False, (TaskRunner pid=2448026) 'reshard_after_forward': True, (TaskRunner pid=2448026) 'seed': 42, (TaskRunner pid=2448026) 'strategy': 'fsdp', (TaskRunner pid=2448026) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2448026) 'use_orig_params': False, (TaskRunner pid=2448026) 'use_torch_compile': True, (TaskRunner pid=2448026) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2448026) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2448026) 'log_prob_micro_batch_size': None, (TaskRunner pid=2448026) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2448026) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2448026) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2448026) 'all_ranks': False, (TaskRunner pid=2448026) 'enable': False, (TaskRunner pid=2448026) 'ranks': [], (TaskRunner pid=2448026) 'save_path': 'outputs/profile', (TaskRunner pid=2448026) 'tool': None, (TaskRunner pid=2448026) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2448026) 'analysis': True, (TaskRunner pid=2448026) 'contents': [], (TaskRunner pid=2448026) 'discrete': False, (TaskRunner pid=2448026) 'level': 'level0'}, (TaskRunner pid=2448026) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2448026) 'discrete': False}, (TaskRunner pid=2448026) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2448026) 'step_end': None, (TaskRunner pid=2448026) 'step_start': 0}, (TaskRunner pid=2448026) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2448026) 'stack_depth': 32, (TaskRunner pid=2448026) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2448026) 'rollout_n': 8, (TaskRunner pid=2448026) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2448026) 'mode': 'disabled', (TaskRunner pid=2448026) 'record_file': None, (TaskRunner pid=2448026) 'replay_file': None}, (TaskRunner pid=2448026) 'strategy': 'fsdp', (TaskRunner pid=2448026) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2448026) 'use_torch_compile': True}, (TaskRunner pid=2448026) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2448026) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2448026) 'agent_loop_config_path': None, (TaskRunner pid=2448026) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2448026) 'name': None, (TaskRunner pid=2448026) 'path': None}, (TaskRunner pid=2448026) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2448026) 'num_workers': 8}, (TaskRunner pid=2448026) 'calculate_log_probs': True, (TaskRunner pid=2448026) 'cudagraph_capture_sizes': None, (TaskRunner pid=2448026) 'data_parallel_size': 1, (TaskRunner pid=2448026) 'disable_log_stats': True, (TaskRunner pid=2448026) 'do_sample': True, (TaskRunner pid=2448026) 'dtype': 'bfloat16', (TaskRunner pid=2448026) 'enable_chunked_prefill': True, (TaskRunner pid=2448026) 'enable_prefix_caching': True, (TaskRunner pid=2448026) 'enable_rollout_routing_replay': False, (TaskRunner pid=2448026) 'enforce_eager': False, (TaskRunner pid=2448026) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2448026) 'expert_parallel_size': 1, (TaskRunner pid=2448026) 'free_cache_engine': True, (TaskRunner pid=2448026) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2448026) 'ignore_eos': False, (TaskRunner pid=2448026) 'layered_summon': False, (TaskRunner pid=2448026) 'load_format': 'dummy', (TaskRunner pid=2448026) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2448026) 'log_prob_micro_batch_size': None, (TaskRunner pid=2448026) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2448026) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2448026) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2448026) 'max_model_len': 10240, (TaskRunner pid=2448026) 'max_num_batched_tokens': 10240, (TaskRunner pid=2448026) 'max_num_seqs': 1024, (TaskRunner pid=2448026) 'mode': 'async', (TaskRunner pid=2448026) 'multi_stage_wake_up': False, (TaskRunner pid=2448026) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2448026) 'enable': False, (TaskRunner pid=2448026) 'format': 'hermes', (TaskRunner pid=2448026) 'interaction_config_path': None, (TaskRunner pid=2448026) 'max_assistant_turns': None, (TaskRunner pid=2448026) 'max_parallel_calls': 1, (TaskRunner pid=2448026) 'max_tool_response_length': 256, (TaskRunner pid=2448026) 'max_user_turns': None, (TaskRunner pid=2448026) 'num_repeat_rollouts': None, (TaskRunner pid=2448026) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2448026) 'tool_config_path': None, (TaskRunner pid=2448026) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2448026) 'use_inference_chat_template': False}, (TaskRunner pid=2448026) 'n': 8, (TaskRunner pid=2448026) 'name': 'vllm', (TaskRunner pid=2448026) 'over_sample_rate': 0, (TaskRunner pid=2448026) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2448026) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2448026) 'all_ranks': False, (TaskRunner pid=2448026) 'enable': False, (TaskRunner pid=2448026) 'ranks': [], (TaskRunner pid=2448026) 'save_path': 'outputs/profile', (TaskRunner pid=2448026) 'tool': None, (TaskRunner pid=2448026) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2448026) 'analysis': True, (TaskRunner pid=2448026) 'contents': [], (TaskRunner pid=2448026) 'discrete': False, (TaskRunner pid=2448026) 'level': 'level0'}, (TaskRunner pid=2448026) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2448026) 'discrete': False}, (TaskRunner pid=2448026) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2448026) 'step_end': None, (TaskRunner pid=2448026) 'step_start': 0}, (TaskRunner pid=2448026) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2448026) 'stack_depth': 32, (TaskRunner pid=2448026) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2448026) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2448026) 'enable': False, (TaskRunner pid=2448026) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2448026) 'port': 9090, (TaskRunner pid=2448026) 'served_model_name': 'Qwen/Qwen3-8B'}, (TaskRunner pid=2448026) 'prompt_length': 2048, (TaskRunner pid=2448026) 'quantization': None, (TaskRunner pid=2448026) 'quantization_config_file': None, (TaskRunner pid=2448026) 'response_length': 8192, (TaskRunner pid=2448026) 'scheduling_policy': 'fcfs', (TaskRunner pid=2448026) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2448026) 'skip_rollout': False, (TaskRunner pid=2448026) 'skip_tokenizer_init': True, (TaskRunner pid=2448026) 'temperature': 1.0, (TaskRunner pid=2448026) 'tensor_model_parallel_size': 2, (TaskRunner pid=2448026) 'top_k': -1, (TaskRunner pid=2448026) 'top_p': 1.0, (TaskRunner pid=2448026) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2448026) 'backend': None, (TaskRunner pid=2448026) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2448026) 'token2text': False}, (TaskRunner pid=2448026) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2448026) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2448026) 'do_sample': True, (TaskRunner pid=2448026) 'n': 16, (TaskRunner pid=2448026) 'temperature': 0.6, (TaskRunner pid=2448026) 'top_k': -1, (TaskRunner pid=2448026) 'top_p': 0.95}}}, (TaskRunner pid=2448026) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2448026) 'adv_estimator': 'grpo', (TaskRunner pid=2448026) 'gamma': 1.0, (TaskRunner pid=2448026) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2448026) 'horizon': 10000, (TaskRunner pid=2448026) 'kl_coef': 0.001, (TaskRunner pid=2448026) 'target_kl': 0.1, (TaskRunner pid=2448026) 'type': 'fixed'}, (TaskRunner pid=2448026) 'kl_penalty': 'kl', (TaskRunner pid=2448026) 'lam': 1.0, (TaskRunner pid=2448026) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2448026) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2448026) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2448026) 'loss_type': 'ppo_clip', (TaskRunner pid=2448026) 'rollout_is': 'token', (TaskRunner pid=2448026) 'rollout_is_batch_normalize': False, (TaskRunner pid=2448026) 'rollout_is_threshold': 2.0, (TaskRunner pid=2448026) 'rollout_rs': None, (TaskRunner pid=2448026) 'rollout_rs_threshold': None}, (TaskRunner pid=2448026) 'use_kl_in_reward': False, (TaskRunner pid=2448026) 'use_pf_ppo': False}, (TaskRunner pid=2448026) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2448026) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2448026) 'async_save': False, (TaskRunner pid=2448026) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2448026) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2448026) 'cliprange_value': 0.5, (TaskRunner pid=2448026) 'data_loader_seed': 42, (TaskRunner pid=2448026) 'enable': None, (TaskRunner pid=2448026) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2448026) 'forward_micro_batch_size': None, (TaskRunner pid=2448026) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2448026) 'grad_clip': 1.0, (TaskRunner pid=2448026) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2448026) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2448026) 'enable_activation_offload': False, (TaskRunner pid=2448026) 'enable_gradient_checkpointing': True, (TaskRunner pid=2448026) 'external_lib': None, (TaskRunner pid=2448026) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2448026) 'dtype': 'bfloat16', (TaskRunner pid=2448026) 'entropy_checkpointing': False, (TaskRunner pid=2448026) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2448026) 'forward_only': False, (TaskRunner pid=2448026) 'forward_prefetch': False, (TaskRunner pid=2448026) 'fsdp_size': -1, (TaskRunner pid=2448026) 'full_determinism': False, (TaskRunner pid=2448026) 'model_dtype': 'fp32', (TaskRunner pid=2448026) 'offload_policy': False, (TaskRunner pid=2448026) 'optimizer_offload': False, (TaskRunner pid=2448026) 'param_offload': False, (TaskRunner pid=2448026) 'reshard_after_forward': True, (TaskRunner pid=2448026) 'seed': 42, (TaskRunner pid=2448026) 'strategy': 'fsdp', (TaskRunner pid=2448026) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2448026) 'use_orig_params': False, (TaskRunner pid=2448026) 'use_torch_compile': True, (TaskRunner pid=2448026) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2448026) 'lora_alpha': 16, (TaskRunner pid=2448026) 'lora_rank': 0, (TaskRunner pid=2448026) 'override_config': {}, (TaskRunner pid=2448026) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2448026) 'target_modules': 'all-linear', (TaskRunner pid=2448026) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2448026) 'tokenizer_path': 'Qwen/Qwen3-8B', (TaskRunner pid=2448026) 'trust_remote_code': True, (TaskRunner pid=2448026) 'use_remove_padding': False, (TaskRunner pid=2448026) 'use_shm': False}, (TaskRunner pid=2448026) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2448026) 'betas': [0.9, 0.999], (TaskRunner pid=2448026) 'clip_grad': 1.0, (TaskRunner pid=2448026) 'lr': 1e-05, (TaskRunner pid=2448026) 'lr_scheduler_type': 'constant', (TaskRunner pid=2448026) 'lr_warmup_steps': -1, (TaskRunner pid=2448026) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2448026) 'min_lr_ratio': 0.0, (TaskRunner pid=2448026) 'num_cycles': 0.5, (TaskRunner pid=2448026) 'optimizer': 'AdamW', (TaskRunner pid=2448026) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2448026) 'override_optimizer_config': None, (TaskRunner pid=2448026) 'total_training_steps': -1, (TaskRunner pid=2448026) 'warmup_style': None, (TaskRunner pid=2448026) 'weight_decay': 0.01}, (TaskRunner pid=2448026) 'ppo_epochs': 1, (TaskRunner pid=2448026) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2448026) 'ppo_micro_batch_size': None, (TaskRunner pid=2448026) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2448026) 'ppo_mini_batch_size': 8, (TaskRunner pid=2448026) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2448026) 'all_ranks': False, (TaskRunner pid=2448026) 'enable': False, (TaskRunner pid=2448026) 'ranks': [], (TaskRunner pid=2448026) 'save_path': 'outputs/profile', (TaskRunner pid=2448026) 'tool': None, (TaskRunner pid=2448026) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2448026) 'analysis': True, (TaskRunner pid=2448026) 'contents': [], (TaskRunner pid=2448026) 'discrete': False, (TaskRunner pid=2448026) 'level': 'level0'}, (TaskRunner pid=2448026) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2448026) 'discrete': False}, (TaskRunner pid=2448026) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2448026) 'step_end': None, (TaskRunner pid=2448026) 'step_start': 0}, (TaskRunner pid=2448026) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2448026) 'stack_depth': 32, (TaskRunner pid=2448026) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2448026) 'rollout_n': 8, (TaskRunner pid=2448026) 'shuffle': False, (TaskRunner pid=2448026) 'strategy': 'fsdp', (TaskRunner pid=2448026) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2448026) 'use_dynamic_bsz': False}, (TaskRunner pid=2448026) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2448026) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2448026) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2448026) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2448026) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2448026) 'dataloader_num_workers': 8, (TaskRunner pid=2448026) 'filter_overlong_prompts': True, (TaskRunner pid=2448026) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2448026) 'image_key': 'images', (TaskRunner pid=2448026) 'image_patch_size': 14, (TaskRunner pid=2448026) 'max_prompt_length': 2048, (TaskRunner pid=2448026) 'max_response_length': 8192, (TaskRunner pid=2448026) 'prompt_key': 'prompt', (TaskRunner pid=2448026) 'return_full_prompt': False, (TaskRunner pid=2448026) 'return_multi_modal_inputs': True, (TaskRunner pid=2448026) 'return_raw_chat': True, (TaskRunner pid=2448026) 'return_raw_input_ids': False, (TaskRunner pid=2448026) 'reward_fn_key': 'data_source', (TaskRunner pid=2448026) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2448026) 'seed': None, (TaskRunner pid=2448026) 'shuffle': True, (TaskRunner pid=2448026) 'tokenizer': None, (TaskRunner pid=2448026) 'tool_config_path': None, (TaskRunner pid=2448026) 'train_batch_size': 32, (TaskRunner pid=2448026) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/tooluse/train.parquet'], (TaskRunner pid=2448026) 'train_max_samples': 3200, (TaskRunner pid=2448026) 'truncation': 'error', (TaskRunner pid=2448026) 'trust_remote_code': True, (TaskRunner pid=2448026) 'use_shm': False, (TaskRunner pid=2448026) 'val_batch_size': None, (TaskRunner pid=2448026) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/tooluse/test.parquet'], (TaskRunner pid=2448026) 'val_max_samples': -1, (TaskRunner pid=2448026) 'validation_shuffle': False, (TaskRunner pid=2448026) 'video_key': 'videos'}, (TaskRunner pid=2448026) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2448026) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2448026) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2448026) 'cuda-memory-usage': 'true', (TaskRunner pid=2448026) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2448026) 'discrete': False, (TaskRunner pid=2448026) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2448026) 'capture-range-end': None, (TaskRunner pid=2448026) 'cuda-graph-trace': 'graph', (TaskRunner pid=2448026) 'cuda-memory-usage': 'true', (TaskRunner pid=2448026) 'kill': 'none', (TaskRunner pid=2448026) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2448026) 'torch_memory': {'context': 'all', (TaskRunner pid=2448026) 'kw_args': {}, (TaskRunner pid=2448026) 'stack_depth': 32, (TaskRunner pid=2448026) 'stacks': 'all', (TaskRunner pid=2448026) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2448026) 'profile_continuous_steps': False, (TaskRunner pid=2448026) 'save_path': 'outputs/profile', (TaskRunner pid=2448026) 'steps': None, (TaskRunner pid=2448026) 'tool': None}, (TaskRunner pid=2448026) 'max_model_len': 10240, (TaskRunner pid=2448026) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_tooluse_grpo_Qwen3_8B', (TaskRunner pid=2448026) 'include_dashboard': False, (TaskRunner pid=2448026) 'num_cpus': None, (TaskRunner pid=2448026) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2448026) 'TASK': 'datasets/tooluse', (TaskRunner pid=2448026) 'USER': 'root'}}}, (TaskRunner pid=2448026) 'timeline_json_file': None}, (TaskRunner pid=2448026) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2448026) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2448026) 'name': 'custom_reward_manager', (TaskRunner pid=2448026) 'path': None}, (TaskRunner pid=2448026) 'name': 'naive', (TaskRunner pid=2448026) 'source': 'register'}, (TaskRunner pid=2448026) 'reward_model': {'enable': False, (TaskRunner pid=2448026) 'enable_resource_pool': False, (TaskRunner pid=2448026) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2448026) 'launch_reward_fn_async': False, (TaskRunner pid=2448026) 'max_length': None, (TaskRunner pid=2448026) 'micro_batch_size': None, (TaskRunner pid=2448026) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2448026) 'model': {'external_lib': None, (TaskRunner pid=2448026) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2448026) 'forward_prefetch': False, (TaskRunner pid=2448026) 'fsdp_size': -1, (TaskRunner pid=2448026) 'param_offload': False, (TaskRunner pid=2448026) 'reshard_after_forward': True, (TaskRunner pid=2448026) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2448026) 'input_tokenizer': 'Qwen/Qwen3-8B', (TaskRunner pid=2448026) 'override_config': {}, (TaskRunner pid=2448026) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2448026) 'trust_remote_code': False, (TaskRunner pid=2448026) 'use_fused_kernels': False, (TaskRunner pid=2448026) 'use_remove_padding': False, (TaskRunner pid=2448026) 'use_shm': False}, (TaskRunner pid=2448026) 'n_gpus_per_node': 8, (TaskRunner pid=2448026) 'nnodes': 0, (TaskRunner pid=2448026) 'num_workers': 1, (TaskRunner pid=2448026) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2448026) 'all_ranks': False, (TaskRunner pid=2448026) 'enable': False, (TaskRunner pid=2448026) 'ranks': [], (TaskRunner pid=2448026) 'save_path': 'outputs/profile', (TaskRunner pid=2448026) 'tool': None, (TaskRunner pid=2448026) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2448026) 'analysis': True, (TaskRunner pid=2448026) 'contents': [], (TaskRunner pid=2448026) 'discrete': False, (TaskRunner pid=2448026) 'level': 'level0'}, (TaskRunner pid=2448026) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2448026) 'discrete': False}, (TaskRunner pid=2448026) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2448026) 'step_end': None, (TaskRunner pid=2448026) 'step_start': 0}, (TaskRunner pid=2448026) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2448026) 'stack_depth': 32, (TaskRunner pid=2448026) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2448026) 'reward_loop_class_name': None, (TaskRunner pid=2448026) 'reward_loop_module_path': None, (TaskRunner pid=2448026) 'reward_loop_source': 'register', (TaskRunner pid=2448026) 'reward_manager': 'naive', (TaskRunner pid=2448026) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2448026) 'cudagraph_capture_sizes': None, (TaskRunner pid=2448026) 'data_parallel_size': 1, (TaskRunner pid=2448026) 'disable_log_stats': True, (TaskRunner pid=2448026) 'dtype': 'bfloat16', (TaskRunner pid=2448026) 'enable_chunked_prefill': True, (TaskRunner pid=2448026) 'enable_prefix_caching': True, (TaskRunner pid=2448026) 'enforce_eager': True, (TaskRunner pid=2448026) 'engine_kwargs': {}, (TaskRunner pid=2448026) 'expert_parallel_size': 1, (TaskRunner pid=2448026) 'free_cache_engine': True, (TaskRunner pid=2448026) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2448026) 'limit_images': None, (TaskRunner pid=2448026) 'load_format': 'auto', (TaskRunner pid=2448026) 'max_model_len': None, (TaskRunner pid=2448026) 'max_num_batched_tokens': 8192, (TaskRunner pid=2448026) 'max_num_seqs': 1024, (TaskRunner pid=2448026) 'name': '???', (TaskRunner pid=2448026) 'prompt_length': 2048, (TaskRunner pid=2448026) 'response_length': 2048, (TaskRunner pid=2448026) 'skip_tokenizer_init': False, (TaskRunner pid=2448026) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2448026) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2448026) 'memory_limit_mb': 1024, (TaskRunner pid=2448026) 'url': None}, (TaskRunner pid=2448026) 'strategy': 'fsdp', (TaskRunner pid=2448026) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2448026) 'use_dynamic_bsz': False, (TaskRunner pid=2448026) 'use_reward_loop': False}, (TaskRunner pid=2448026) 'trainer': {'balance_batch': True, (TaskRunner pid=2448026) 'critic_warmup': 0, (TaskRunner pid=2448026) 'default_hdfs_dir': None, (TaskRunner pid=2448026) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2448026) 'del_local_ckpt_after_load': False, (TaskRunner pid=2448026) 'device': 'cuda', (TaskRunner pid=2448026) 'esi_redundant_time': 0, (TaskRunner pid=2448026) 'experiment_name': 'qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2448026) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2448026) 'log_val_generations': 0, (TaskRunner pid=2448026) 'logger': ['console', 'wandb'], (TaskRunner pid=2448026) 'max_actor_ckpt_to_keep': 100, (TaskRunner pid=2448026) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2448026) 'n_gpus_per_node': 8, (TaskRunner pid=2448026) 'nnodes': 1, (TaskRunner pid=2448026) 'project_name': 'SDPO-root', (TaskRunner pid=2448026) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2448026) 'resume_from_path': None, (TaskRunner pid=2448026) 'resume_mode': 'auto', (TaskRunner pid=2448026) 'rollout_data_dir': None, (TaskRunner pid=2448026) 'save_freq': 10, (TaskRunner pid=2448026) 'test_freq': 10, (TaskRunner pid=2448026) 'total_epochs': 30, (TaskRunner pid=2448026) 'total_training_steps': 100, (TaskRunner pid=2448026) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2448026) 'val_before_train': False, (TaskRunner pid=2448026) 'val_only': False, (TaskRunner pid=2448026) 'validation_data_dir': None}, (TaskRunner pid=2448026) 'transfer_queue': {'enable': False}, (TaskRunner pid=2448026) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/tooluse', (TaskRunner pid=2448026) 'dir': '/users/root/SDPO', (TaskRunner pid=2448026) 'log_dir': '/users/root/output', (TaskRunner pid=2448026) 'task': 'datasets/tooluse'}} (TaskRunner pid=2448026) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2448026) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2448026) use_critic=need_critic(config), (TaskRunner pid=2448026) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2448026) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2448026) Using dataset class: RLHFDataset (TaskRunner pid=2448026) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2448026) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2448026) dataset len: 4046 (TaskRunner pid=2448026) selected 3200 random samples out of 4046 (TaskRunner pid=2448026) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2448026) WARNING:2026-07-03 13:15:13,874:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2448026) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/3200 [00:00 (TaskRunner pid=2448026) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2448026) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2448026) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2448026) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2448026) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2448026) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 68/68 [00:00<00:00, 95.98 examples/s] (TaskRunner pid=2448026) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2448026) self.use_critic = need_critic(self.config) (WorkerDict pid=2448439) [W703 13:15:25.171858971 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59271 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2448439) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2448443) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2448443) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2448439) Model config after override: Qwen3Config { (WorkerDict pid=2448439) "architectures": [ (WorkerDict pid=2448439) "Qwen3ForCausalLM" (WorkerDict pid=2448439) ], (WorkerDict pid=2448439) "attention_bias": false, (WorkerDict pid=2448439) "attention_dropout": 0.0, (WorkerDict pid=2448439) "dtype": "bfloat16", (WorkerDict pid=2448439) "eos_token_id": 151645, (WorkerDict pid=2448439) "head_dim": 128, (WorkerDict pid=2448439) "hidden_act": "silu", (WorkerDict pid=2448439) "hidden_size": 4096, (WorkerDict pid=2448439) "initializer_range": 0.02, (WorkerDict pid=2448439) "intermediate_size": 12288, (WorkerDict pid=2448439) "layer_types": [ (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention", (WorkerDict pid=2448439) "full_attention" (WorkerDict pid=2448439) ], (WorkerDict pid=2448439) "max_position_embeddings": 40960, (WorkerDict pid=2448439) "max_window_layers": 36, (WorkerDict pid=2448439) "model_type": "qwen3", (WorkerDict pid=2448439) "num_attention_heads": 32, (WorkerDict pid=2448439) "num_hidden_layers": 36, (WorkerDict pid=2448439) "num_key_value_heads": 8, (WorkerDict pid=2448439) "pad_token_id": 151643, (WorkerDict pid=2448439) "rms_norm_eps": 1e-06, (WorkerDict pid=2448439) "rope_scaling": null, (WorkerDict pid=2448439) "rope_theta": 1000000, (WorkerDict pid=2448439) "sliding_window": null, (WorkerDict pid=2448439) "tie_word_embeddings": false, (WorkerDict pid=2448439) "transformers_version": "4.57.1", (WorkerDict pid=2448439) "use_cache": true, (WorkerDict pid=2448439) "use_sliding_window": false, (WorkerDict pid=2448439) "vocab_size": 151936 (WorkerDict pid=2448439) } (WorkerDict pid=2448439) (WorkerDict pid=2448444) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2448443) [W703 13:15:26.732240078 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59271 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2448443) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2448443) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2448443) Loading checkpoint shards: 0%| | 0/5 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2448439) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2448440) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2448440) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2448442) Loading checkpoint shards: 80%|████████ | 4/5 [00:12<00:02, 2.93s/it] [repeated 15x across cluster] (WorkerDict pid=2448442) Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.22s/it] Loading checkpoint shards: 100%|██████████| 5/5 [00:13<00:00, 2.61s/it] [repeated 7x across cluster] (WorkerDict pid=2448439) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2448439) Actor use_remove_padding=True (WorkerDict pid=2448439) Actor use_fused_kernels=False (WorkerDict pid=2448439) Actor use_prefix_grouper=False (WorkerDict pid=2448442) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2448442) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2448444) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2448444) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2448440) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2448439) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2448439) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2448439) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2448439) warnings.warn( (WorkerDict pid=2448439) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2448439) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2448026) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2448026) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2448443) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2448443) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2449317) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2449317) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2449317) ['serve', (vLLMHttpServer pid=2449317) 'Qwen/Qwen3-8B', (vLLMHttpServer pid=2449317) '--dtype', (vLLMHttpServer pid=2449317) 'bfloat16', (vLLMHttpServer pid=2449317) '--load_format', (vLLMHttpServer pid=2449317) 'dummy', (vLLMHttpServer pid=2449317) '--trust_remote_code', (vLLMHttpServer pid=2449317) '--max_model_len', (vLLMHttpServer pid=2449317) '40960', (vLLMHttpServer pid=2449317) '--max_num_seqs', (vLLMHttpServer pid=2449317) '1024', (vLLMHttpServer pid=2449317) '--enable_chunked_prefill', (vLLMHttpServer pid=2449317) '--max_num_batched_tokens', (vLLMHttpServer pid=2449317) '10240', (vLLMHttpServer pid=2449317) '--enable_prefix_caching', (vLLMHttpServer pid=2449317) '--enable_sleep_mode', (vLLMHttpServer pid=2449317) '--logprobs_mode', (vLLMHttpServer pid=2449317) 'processed_logprobs', (vLLMHttpServer pid=2449317) '--disable_custom_all_reduce', (vLLMHttpServer pid=2449317) '--gpu_memory_utilization', (vLLMHttpServer pid=2449317) '0.8', (vLLMHttpServer pid=2449317) '--disable_log_stats', (vLLMHttpServer pid=2449317) '--tensor_parallel_size', (vLLMHttpServer pid=2449317) '2', (vLLMHttpServer pid=2449317) '--seed', (vLLMHttpServer pid=2449317) '0', (vLLMHttpServer pid=2449317) '--override_generation_config', (vLLMHttpServer pid=2449317) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2449317) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2449317) '--hf_overrides', (vLLMHttpServer pid=2449317) '{}', (vLLMHttpServer pid=2449317) '--scheduling_policy', (vLLMHttpServer pid=2449317) 'fcfs'] (WorkerDict pid=2448446) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2449317) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2449317) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2449318) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2449318) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2449317) WARNING 07-03 13:16:16 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2448439) WARNING 07-03 13:16:24 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2449320) WARNING 07-03 13:16:17 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2448441) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2448441) [rank2]:W0703 13:16:35.880000 2448441 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2448441) [rank2]:W0703 13:16:35.880000 2448441 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2448441) [rank2]:W0703 13:16:35.880000 2448441 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2448441) [rank2]:W0703 13:16:35.880000 2448441 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2448441) [rank2]:W0703 13:16:35.880000 2448441 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2448441) [rank2]:W0703 13:16:35.880000 2448441 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/aj/faj75csb7ofozv5in77ssqc762na7cdfwj3wkvfkkmz6awioro7y/.2448440.123711478494912.tmp' (vLLMHttpServer pid=2449320) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2449320) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (vLLMHttpServer pid=2449320) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2449320) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (WorkerDict pid=2448445) 2026-07-03 13:16:38,703 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2448445) 2026-07-03 13:16:38,718 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2448439) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) [response] Thought: The user is asking for a random axolotl picture. I should use the `getRandomAxolotlImage` tool to retrieve a random image with its source and description. (TaskRunner pid=2448026) (TaskRunner pid=2448026) Action: getRandomAxolotlImage (TaskRunner pid=2448026) Action Input: {} (TaskRunner pid=2448026) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2448026) [score] 1.0 (TaskRunner pid=2448026) [acc] 1.0 (TaskRunner pid=2448026) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2448026) [incorrect_format] 0 (TaskRunner pid=2448026) [feedback] (TaskRunner pid=2448026) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2448026) step:10 - global_seqlen/min:16938 - global_seqlen/max:26271 - global_seqlen/minmax_diff:9333 - global_seqlen/balanced_min:21702 - global_seqlen/balanced_max:21712 - global_seqlen/mean:21707.375 - actor/entropy:0.22459572553634644 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16888222098350525 - training/rollout_probs_diff_mean:0.004130216781049967 - training/rollout_probs_diff_std:0.011616804637014866 - training/rollout_actor_probs_pearson_corr:0.9986513257026672 - rollout_corr/rollout_is_mean:1.0003739595413208 - rollout_corr/rollout_is_ratio_fraction_high:4.5365875848801807e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.075049877166748 - rollout_corr/rollout_is_min:0.5348902344703674 - rollout_corr/rollout_is_std:0.03388721123337746 - rollout_corr/rollout_is_eff_sample_size:0.99885380664554 - rollout_corr/rollout_is_seq_mean:1.0002562999725342 - rollout_corr/rollout_is_seq_std:0.00375300832092762 - rollout_corr/rollout_is_seq_max:1.010035753250122 - rollout_corr/rollout_is_seq_min:0.9895481467247009 - rollout_corr/rollout_is_seq_max_deviation:0.010451853275299072 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2358288876712322) - rollout_corr/training_log_ppl:np.float64(0.20317194943345385) - rollout_corr/kl:np.float64(4.5682639505884026e-05) - rollout_corr/k3_kl:np.float64(0.0005935528575697901) - rollout_corr/rollout_ppl:np.float64(1.2357774288393557) - rollout_corr/rollout_log_ppl:np.float64(0.20312626545637613) - rollout_corr/log_ppl_diff:np.float64(4.56839770777151e-05) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029669152427231893) - rollout_corr/log_ppl_diff_max:np.float64(0.010825961828231812) - rollout_corr/log_ppl_diff_min:np.float64(-0.012095347046852112) - rollout_corr/ppl_ratio:np.float64(1.0000531105324626) - rollout_corr/chi2_token:np.float64(0.002376394812017679) - rollout_corr/chi2_seq:np.float64(0.2696599082555622) - actor/pg_loss:np.float64(-1.2497417628765106e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.650297705666162e-05) - actor/ppo_kl:np.float64(-0.0002640195668668355) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06585756875574589) - perf/mfu/actor:np.float64(0.08890278693472423) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.935546875) - perf/cpu_memory_used_gb:np.float64(99.63834381103516) - actor/lr:np.float64(9e-07) - val-aux/tooluse/reward/mean@16:np.float64(0.5882352941176471) - val-aux/tooluse/reward/std@16:np.float64(0.05549776321396803) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6103382352941177) - val-aux/tooluse/reward/best@2/std:np.float64(0.048134977048089916) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5659411764705883) - val-aux/tooluse/reward/worst@2/std:np.float64(0.04548147090076516) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5884411764705882) - val-aux/tooluse/reward/maj@2/std:np.float64(0.054938289409976554) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6308676470588235) - val-aux/tooluse/reward/best@4/std:np.float64(0.038071203812288) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.549235294117647) - val-aux/tooluse/reward/worst@4/std:np.float64(0.03313091575494612) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5891764705882353) - val-aux/tooluse/reward/maj@4/std:np.float64(0.046293127479058824) - val-aux/tooluse/reward/best@8/mean:np.float64(0.645720588235294) - val-aux/tooluse/reward/best@8/std:np.float64(0.025710807041351088) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5365735294117646) - val-aux/tooluse/reward/worst@8/std:np.float64(0.025367868823877354) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5885588235294118) - val-aux/tooluse/reward/maj@8/std:np.float64(0.035304274011995554) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6553382352941176) - val-aux/tooluse/reward/best@16/std:np.float64(0.01313263738726293) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5263382352941176) - val-aux/tooluse/reward/worst@16/std:np.float64(0.01841625987782561) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5859411764705883) - val-aux/tooluse/reward/maj@16/std:np.float64(0.026153277356207073) - val-aux/tooluse/score/mean@16:np.float64(0.5882352941176471) - val-aux/tooluse/score/std@16:np.float64(0.05549776321396803) - val-aux/tooluse/score/best@2/mean:np.float64(0.6103382352941177) - val-aux/tooluse/score/best@2/std:np.float64(0.048134977048089916) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5659411764705883) - val-aux/tooluse/score/worst@2/std:np.float64(0.04548147090076516) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5884411764705882) - val-aux/tooluse/score/maj@2/std:np.float64(0.054938289409976554) - val-aux/tooluse/score/best@4/mean:np.float64(0.6308676470588235) - val-aux/tooluse/score/best@4/std:np.float64(0.038071203812288) - val-aux/tooluse/score/worst@4/mean:np.float64(0.549235294117647) - val-aux/tooluse/score/worst@4/std:np.float64(0.03313091575494612) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5891764705882353) - val-aux/tooluse/score/maj@4/std:np.float64(0.046293127479058824) - val-aux/tooluse/score/best@8/mean:np.float64(0.645720588235294) - val-aux/tooluse/score/best@8/std:np.float64(0.025710807041351088) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5365735294117646) - val-aux/tooluse/score/worst@8/std:np.float64(0.025367868823877354) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5885588235294118) - val-aux/tooluse/score/maj@8/std:np.float64(0.035304274011995554) - val-aux/tooluse/score/best@16/mean:np.float64(0.6553382352941176) - val-aux/tooluse/score/best@16/std:np.float64(0.01313263738726293) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5263382352941176) - val-aux/tooluse/score/worst@16/std:np.float64(0.01841625987782561) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5859411764705883) - val-aux/tooluse/score/maj@16/std:np.float64(0.026153277356207073) - val-core/tooluse/acc/mean@16:np.float64(0.5882352941176471) - val-aux/tooluse/acc/std@16:np.float64(0.05549776321396803) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6103382352941177) - val-aux/tooluse/acc/best@2/std:np.float64(0.048134977048089916) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5659411764705883) - val-aux/tooluse/acc/worst@2/std:np.float64(0.04548147090076516) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5884411764705882) - val-aux/tooluse/acc/maj@2/std:np.float64(0.054938289409976554) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6308676470588235) - val-aux/tooluse/acc/best@4/std:np.float64(0.038071203812288) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.549235294117647) - val-aux/tooluse/acc/worst@4/std:np.float64(0.03313091575494612) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5891764705882353) - val-aux/tooluse/acc/maj@4/std:np.float64(0.046293127479058824) - val-aux/tooluse/acc/best@8/mean:np.float64(0.645720588235294) - val-aux/tooluse/acc/best@8/std:np.float64(0.025710807041351088) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5365735294117646) - val-aux/tooluse/acc/worst@8/std:np.float64(0.025367868823877354) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5885588235294118) - val-aux/tooluse/acc/maj@8/std:np.float64(0.035304274011995554) - val-core/tooluse/acc/best@16/mean:np.float64(0.6553382352941176) - val-core/tooluse/acc/best@16/std:np.float64(0.01313263738726293) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5263382352941176) - val-aux/tooluse/acc/worst@16/std:np.float64(0.01841625987782561) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5859411764705883) - val-core/tooluse/acc/maj@16/std:np.float64(0.026153277356207073) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.24609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.24609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012759153032675385 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012759153032675385 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:86.10546875 - response_length/max:588.0 - response_length/min:34.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:86.10546875 - response_length_non_aborted/max:588.0 - response_length_non_aborted/min:34.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:592.25 - prompt_length/max:917.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001600012183189392 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6111685000360012) - timing_s/agent_loop/generate_sequences/max:np.float64(3.993231298401952) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0820964776939945) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.993231298401952) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:739 - timing_s/agent_loop/slowest/response_length:588 - timing_s/gen:10.852380968630314 - timing_s/reward:0.08663872629404068 - timing_s/old_log_prob:3.072744118049741 - timing_s/adv:0.14805224910378456 - timing_s/update_actor:12.86603694036603 - timing_s/step:27.073142820969224 - timing_s/testing:25.101161101832986 - timing_s/save_checkpoint:12.745824763551354 - timing_s/stop_profile:0.00014091096818447113 - timing_per_token_ms/update_actor:0.07408793636014276 - timing_per_token_ms/gen:0.4923277670294567 - timing_per_token_ms/adv:0.0008525457886074696 - perf/total_num_tokens:173659 - perf/time_per_step:27.073142820969224 - perf/throughput:801.8047680517822 (TaskRunner pid=2448026) Training Progress: 10%|█ | 10/100 [05:34<1:00:24, 40.28s/it] (TaskRunner pid=2448026) step:11 - global_seqlen/min:18957 - global_seqlen/max:27507 - global_seqlen/minmax_diff:8550 - global_seqlen/balanced_min:22286 - global_seqlen/balanced_max:22292 - global_seqlen/mean:22290.125 - actor/entropy:0.2515323758125305 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2145475596189499 - training/rollout_probs_diff_mean:0.004317344166338444 - training/rollout_probs_diff_std:0.011624038219451904 - training/rollout_actor_probs_pearson_corr:0.9987900257110596 - rollout_corr/rollout_is_mean:0.9998831152915955 - rollout_corr/rollout_is_ratio_fraction_high:4.261666254024021e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0842576026916504 - rollout_corr/rollout_is_min:0.5088409185409546 - rollout_corr/rollout_is_std:0.03583798184990883 - rollout_corr/rollout_is_eff_sample_size:0.9987169298704608 - rollout_corr/rollout_is_seq_mean:1.0000934600830078 - rollout_corr/rollout_is_seq_std:0.004502827767282724 - rollout_corr/rollout_is_seq_max:1.013597846031189 - rollout_corr/rollout_is_seq_min:0.9832828044891357 - rollout_corr/rollout_is_seq_max_deviation:0.016717195510864258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2865458657033741) - rollout_corr/training_log_ppl:np.float64(0.24004999878525268) - rollout_corr/kl:np.float64(0.0004261141003780722) - rollout_corr/k3_kl:np.float64(0.0006533747404109391) - rollout_corr/rollout_ppl:np.float64(1.2859233990311623) - rollout_corr/rollout_log_ppl:np.float64(0.23962388539803214) - rollout_corr/log_ppl_diff:np.float64(0.00042611338722053915) - rollout_corr/log_ppl_abs_diff:np.float64(0.003570256070815958) - rollout_corr/log_ppl_diff_max:np.float64(0.014903545379638672) - rollout_corr/log_ppl_diff_min:np.float64(-0.016898557543754578) - rollout_corr/ppl_ratio:np.float64(1.0004367819055915) - rollout_corr/chi2_token:np.float64(0.001802204642444849) - rollout_corr/chi2_seq:np.float64(0.13842039951123297) - actor/pg_loss:np.float64(-0.0002331784926354885) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016005308498279192) - actor/ppo_kl:np.float64(-0.0001478686267972762) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18062540236860514) - perf/mfu/actor:np.float64(0.09104551706491418) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(104.94073867797852) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.2265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:9.588749526301399e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:9.588749526301399e-05 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:91.66015625 - response_length/max:664.0 - response_length/min:20.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:91.66015625 - response_length_non_aborted/max:664.0 - response_length_non_aborted/min:20.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:604.90625 - prompt_length/max:1164.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015133246779441833 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4236251749098301) - timing_s/agent_loop/generate_sequences/max:np.float64(4.524629222229123) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1302458627178567) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.524629222229123) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:712 - timing_s/agent_loop/slowest/response_length:656 - timing_s/gen:10.923210125416517 - timing_s/reward:0.07565747015178204 - timing_s/old_log_prob:3.318278046324849 - timing_s/adv:0.14237502217292786 - timing_s/update_actor:12.916431237012148 - timing_s/step:27.467644907534122 - timing_s/stop_profile:0.00011134706437587738 - timing_per_token_ms/update_actor:0.07243359580202079 - timing_per_token_ms/gen:0.4655107660522701 - timing_per_token_ms/adv:0.0007984198281353731 - perf/total_num_tokens:178321 - perf/time_per_step:27.467644907534122 - perf/throughput:811.5047749829481 (TaskRunner pid=2448026) Training Progress: 11%|█ | 11/100 [06:02<53:57, 36.37s/it] (TaskRunner pid=2448026) step:12 - global_seqlen/min:18209 - global_seqlen/max:25518 - global_seqlen/minmax_diff:7309 - global_seqlen/balanced_min:21998 - global_seqlen/balanced_max:22005 - global_seqlen/mean:22001.625 - actor/entropy:0.2264634072780609 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.390397310256958 - training/rollout_probs_diff_mean:0.004275402054190636 - training/rollout_probs_diff_std:0.012356446124613285 - training/rollout_actor_probs_pearson_corr:0.9985068440437317 - rollout_corr/rollout_is_mean:0.9999607801437378 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7033050060272217 - rollout_corr/rollout_is_min:0.5040695667266846 - rollout_corr/rollout_is_std:0.0351477712392807 - rollout_corr/rollout_is_eff_sample_size:0.9987661583782108 - rollout_corr/rollout_is_seq_mean:0.9998271465301514 - rollout_corr/rollout_is_seq_std:0.004464537836611271 - rollout_corr/rollout_is_seq_max:1.0135618448257446 - rollout_corr/rollout_is_seq_min:0.9800541400909424 - rollout_corr/rollout_is_seq_max_deviation:0.019945859909057617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.251064259558916) - rollout_corr/training_log_ppl:np.float64(0.21356585128523875) - rollout_corr/kl:np.float64(0.0007212352236649622) - rollout_corr/k3_kl:np.float64(0.0007491549814311327) - rollout_corr/rollout_ppl:np.float64(1.2501619048416615) - rollout_corr/rollout_log_ppl:np.float64(0.2128446147980867) - rollout_corr/log_ppl_diff:np.float64(0.0007212364871520549) - rollout_corr/log_ppl_abs_diff:np.float64(0.00350782394525595) - rollout_corr/log_ppl_diff_max:np.float64(0.025242015719413757) - rollout_corr/log_ppl_diff_min:np.float64(-0.020114261656999588) - rollout_corr/ppl_ratio:np.float64(1.0007328372448683) - rollout_corr/chi2_token:np.float64(0.0016097496263682842) - rollout_corr/chi2_seq:np.float64(0.16390979941934347) - actor/pg_loss:np.float64(0.00012796837836503983) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006051504715287592) - actor/ppo_kl:np.float64(-0.00011961488690914734) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5012923702597618) - perf/mfu/actor:np.float64(0.09116933436869923) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(103.06351470947266) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014852331951260567 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.0014852331951260567 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:91.39453125 - response_length/max:481.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:91.39453125 - response_length_non_aborted/max:481.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:596.15625 - prompt_length/max:937.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015606917440891266 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5142314974218607) - timing_s/agent_loop/generate_sequences/max:np.float64(3.4842497557401657) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1197145631740568) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.4842497557401657) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:752 - timing_s/agent_loop/slowest/response_length:481 - timing_s/gen:10.157807391136885 - timing_s/reward:0.12038173340260983 - timing_s/old_log_prob:3.196213811635971 - timing_s/adv:0.17003314569592476 - timing_s/update_actor:12.417925156652927 - timing_s/step:26.149601677432656 - timing_s/stop_profile:0.00012761540710926056 - timing_per_token_ms/update_actor:0.07055118176869281 - timing_per_token_ms/gen:0.4341499932101075 - timing_per_token_ms/adv:0.000966026064528897 - perf/total_num_tokens:176013 - perf/time_per_step:26.149601677432656 - perf/throughput:841.3751487078139 (TaskRunner pid=2448026) Training Progress: 12%|█▏ | 12/100 [06:28<48:48, 33.28s/it] (TaskRunner pid=2448026) step:13 - global_seqlen/min:19220 - global_seqlen/max:28619 - global_seqlen/minmax_diff:9399 - global_seqlen/balanced_min:23415 - global_seqlen/balanced_max:23886 - global_seqlen/mean:23475.25 - actor/entropy:0.26368364691734314 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24479267001152039 - training/rollout_probs_diff_mean:0.004046679940074682 - training/rollout_probs_diff_std:0.0110176932066679 - training/rollout_actor_probs_pearson_corr:0.9989438056945801 - rollout_corr/rollout_is_mean:1.0000158548355103 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.648826003074646 - rollout_corr/rollout_is_min:0.6109016537666321 - rollout_corr/rollout_is_std:0.033843208104372025 - rollout_corr/rollout_is_eff_sample_size:0.9988559475012312 - rollout_corr/rollout_is_seq_mean:1.0000919103622437 - rollout_corr/rollout_is_seq_std:0.004048100207000971 - rollout_corr/rollout_is_seq_max:1.0141819715499878 - rollout_corr/rollout_is_seq_min:0.987387478351593 - rollout_corr/rollout_is_seq_max_deviation:0.014181971549987793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3086875015869737) - rollout_corr/training_log_ppl:np.float64(0.2554505392035935) - rollout_corr/kl:np.float64(0.0007047487838072186) - rollout_corr/k3_kl:np.float64(0.000720326093542667) - rollout_corr/rollout_ppl:np.float64(1.3077687490731478) - rollout_corr/rollout_log_ppl:np.float64(0.2547457892651437) - rollout_corr/log_ppl_diff:np.float64(0.0007047499384498224) - rollout_corr/log_ppl_abs_diff:np.float64(0.0033863840071717277) - rollout_corr/log_ppl_diff_max:np.float64(0.013867370784282684) - rollout_corr/log_ppl_diff_min:np.float64(-0.020585566759109497) - rollout_corr/ppl_ratio:np.float64(1.0007147532887757) - rollout_corr/chi2_token:np.float64(0.001510791014879942) - rollout_corr/chi2_seq:np.float64(0.1808106794487685) - actor/pg_loss:np.float64(7.940223440527916e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020800951347155205) - actor/ppo_kl:np.float64(0.00016187368791520385) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3534238636493683) - perf/mfu/actor:np.float64(0.0955168230599007) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(102.67752456665039) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.3359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016656514490023255 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016656514490023255 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:102.6015625 - response_length/max:1125.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:102.6015625 - response_length_non_aborted/max:1125.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:631.0 - prompt_length/max:1024.0 - prompt_length/min:308.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011011958122253418 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5238606445491314) - timing_s/agent_loop/generate_sequences/max:np.float64(7.217537606135011) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.287449269919307) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.217537606135011) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:821 - timing_s/agent_loop/slowest/response_length:1125 - timing_s/gen:14.067130556330085 - timing_s/reward:0.06949751637876034 - timing_s/old_log_prob:3.172292560338974 - timing_s/adv:0.14622640796005726 - timing_s/update_actor:12.94165370054543 - timing_s/step:30.493091329932213 - timing_s/stop_profile:0.00012276507914066315 - timing_per_token_ms/update_actor:0.06891116016094306 - timing_per_token_ms/gen:0.5355642486990819 - timing_per_token_ms/adv:0.0007786200783807269 - perf/total_num_tokens:187802 - perf/time_per_step:30.493091329932213 - perf/throughput:769.8547105637841 (TaskRunner pid=2448026) Training Progress: 13%|█▎ | 13/100 [06:59<47:03, 32.46s/it] (TaskRunner pid=2448026) step:14 - global_seqlen/min:14014 - global_seqlen/max:29896 - global_seqlen/minmax_diff:15882 - global_seqlen/balanced_min:20015 - global_seqlen/balanced_max:20153 - global_seqlen/mean:20037.5 - actor/entropy:0.1920662671327591 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3394758701324463 - training/rollout_probs_diff_mean:0.004057586193084717 - training/rollout_probs_diff_std:0.01215781457722187 - training/rollout_actor_probs_pearson_corr:0.9983827471733093 - rollout_corr/rollout_is_mean:1.0005981922149658 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6297097206115723 - rollout_corr/rollout_is_min:0.5632624626159668 - rollout_corr/rollout_is_std:0.03460085019469261 - rollout_corr/rollout_is_eff_sample_size:0.9988057587432629 - rollout_corr/rollout_is_seq_mean:1.000698208808899 - rollout_corr/rollout_is_seq_std:0.004666553810238838 - rollout_corr/rollout_is_seq_max:1.0221474170684814 - rollout_corr/rollout_is_seq_min:0.9891459941864014 - rollout_corr/rollout_is_seq_max_deviation:0.022147417068481445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.215131500735879) - rollout_corr/training_log_ppl:np.float64(0.18961565523932222) - rollout_corr/kl:np.float64(0.00021677033657852007) - rollout_corr/k3_kl:np.float64(0.0006895844757366376) - rollout_corr/rollout_ppl:np.float64(1.2147769443690777) - rollout_corr/rollout_log_ppl:np.float64(0.1893988842057297) - rollout_corr/log_ppl_diff:np.float64(0.00021677103359252214) - rollout_corr/log_ppl_abs_diff:np.float64(0.0039243222563527524) - rollout_corr/log_ppl_diff_max:np.float64(0.015486270189285278) - rollout_corr/log_ppl_diff_min:np.float64(-0.021514996886253357) - rollout_corr/ppl_ratio:np.float64(1.0002301291096956) - rollout_corr/chi2_token:np.float64(0.0023501813411712646) - rollout_corr/chi2_seq:np.float64(0.1891316152177751) - actor/pg_loss:np.float64(-6.94796908646822e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.882812572759576e-05) - actor/ppo_kl:np.float64(0.00029502600899178333) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07354175997897983) - perf/mfu/actor:np.float64(0.08109110087678428) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(102.86065292358398) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.25390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00028953058063052595 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:0.00028953058063052595 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:75.890625 - response_length/max:866.0 - response_length/min:28.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:75.890625 - response_length_non_aborted/max:866.0 - response_length_non_aborted/min:28.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:550.28125 - prompt_length/max:1276.0 - prompt_length/min:222.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011677108705043793 - timing_s/agent_loop/generate_sequences/min:np.float64(0.33113493025302887) - timing_s/agent_loop/generate_sequences/max:np.float64(5.274745363742113) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9125702472083503) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.274745363742113) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:656 - timing_s/agent_loop/slowest/response_length:866 - timing_s/gen:12.378283524885774 - timing_s/reward:0.06466773524880409 - timing_s/old_log_prob:3.0122491847723722 - timing_s/adv:0.15684368647634983 - timing_s/update_actor:12.710077906027436 - timing_s/step:28.418291436508298 - timing_s/stop_profile:0.00012121163308620453 - timing_per_token_ms/update_actor:0.07928931943872387 - timing_per_token_ms/gen:0.6371362736712876 - timing_per_token_ms/adv:0.000978438468349032 - perf/total_num_tokens:160300 - perf/time_per_step:28.418291436508298 - perf/throughput:705.0916500299629 (TaskRunner pid=2448026) Training Progress: 14%|█▍ | 14/100 [07:27<44:47, 31.25s/it] (TaskRunner pid=2448026) step:15 - global_seqlen/min:16386 - global_seqlen/max:23979 - global_seqlen/minmax_diff:7593 - global_seqlen/balanced_min:19846 - global_seqlen/balanced_max:19850 - global_seqlen/mean:19848.0 - actor/entropy:0.277450829744339 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564751386642456 - training/rollout_probs_diff_mean:0.004466681275516748 - training/rollout_probs_diff_std:0.012154882773756981 - training/rollout_actor_probs_pearson_corr:0.9987545013427734 - rollout_corr/rollout_is_mean:1.0000890493392944 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.7384382924064994e-05 - rollout_corr/rollout_is_max:1.6050195693969727 - rollout_corr/rollout_is_min:0.4127389192581177 - rollout_corr/rollout_is_std:0.0365738645195961 - rollout_corr/rollout_is_eff_sample_size:0.9986642583633856 - rollout_corr/rollout_is_seq_mean:0.9997676610946655 - rollout_corr/rollout_is_seq_std:0.004683754872530699 - rollout_corr/rollout_is_seq_max:1.0157780647277832 - rollout_corr/rollout_is_seq_min:0.9855842590332031 - rollout_corr/rollout_is_seq_max_deviation:0.015778064727783203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2563082994893193) - rollout_corr/training_log_ppl:np.float64(0.21411840058863163) - rollout_corr/kl:np.float64(0.0011823924063172342) - rollout_corr/k3_kl:np.float64(0.0007655184164239159) - rollout_corr/rollout_ppl:np.float64(1.2547586322762072) - rollout_corr/rollout_log_ppl:np.float64(0.2129360087128589) - rollout_corr/log_ppl_diff:np.float64(0.001182391875772737) - rollout_corr/log_ppl_abs_diff:np.float64(0.003633197062299587) - rollout_corr/log_ppl_diff_max:np.float64(0.015970945358276367) - rollout_corr/log_ppl_diff_min:np.float64(-0.014236986637115479) - rollout_corr/ppl_ratio:np.float64(1.0011938200332224) - rollout_corr/chi2_token:np.float64(0.0006637941114604473) - rollout_corr/chi2_seq:np.float64(0.09749737288802862) - actor/pg_loss:np.float64(-0.00020381133072078228) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.661231989506632e-05) - actor/ppo_kl:np.float64(0.0003082933639175778) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.30811620503664017) - perf/mfu/actor:np.float64(0.081035896125215) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(102.65209579467773) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018953753169625998 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018953753169625998 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:82.4375 - response_length/max:525.0 - response_length/min:35.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:82.4375 - response_length_non_aborted/max:525.0 - response_length_non_aborted/min:35.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:537.8125 - prompt_length/max:1261.0 - prompt_length/min:222.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001567583531141281 - timing_s/agent_loop/generate_sequences/min:np.float64(0.46213280968368053) - timing_s/agent_loop/generate_sequences/max:np.float64(3.7013799864798784) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0001081719892682) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.7013799864798784) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:552 - timing_s/agent_loop/slowest/response_length:525 - timing_s/gen:10.209611976519227 - timing_s/reward:0.055870141834020615 - timing_s/old_log_prob:3.120828587561846 - timing_s/adv:0.1667728740721941 - timing_s/update_actor:12.859501745551825 - timing_s/step:26.454133488237858 - timing_s/stop_profile:0.00018094293773174286 - timing_per_token_ms/update_actor:0.08098739007426331 - timing_per_token_ms/gen:0.4837761550663015 - timing_per_token_ms/adv:0.0010503128405393118 - perf/total_num_tokens:158784 - perf/time_per_step:26.454133488237858 - perf/throughput:750.2797250503365 (TaskRunner pid=2448026) Training Progress: 15%|█▌ | 15/100 [07:54<42:14, 29.82s/it] (TaskRunner pid=2448026) step:16 - global_seqlen/min:16731 - global_seqlen/max:20889 - global_seqlen/minmax_diff:4158 - global_seqlen/balanced_min:18720 - global_seqlen/balanced_max:18721 - global_seqlen/mean:18720.875 - actor/entropy:0.2085598111152649 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311232805252075 - training/rollout_probs_diff_mean:0.004366713110357523 - training/rollout_probs_diff_std:0.012814168818295002 - training/rollout_actor_probs_pearson_corr:0.9982979893684387 - rollout_corr/rollout_is_mean:0.9998440742492676 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.514199074241333e-05 - rollout_corr/rollout_is_max:1.6273233890533447 - rollout_corr/rollout_is_min:0.37656983733177185 - rollout_corr/rollout_is_std:0.03599397838115692 - rollout_corr/rollout_is_eff_sample_size:0.9987058719601385 - rollout_corr/rollout_is_seq_mean:0.9998511075973511 - rollout_corr/rollout_is_seq_std:0.004597388207912445 - rollout_corr/rollout_is_seq_max:1.0153234004974365 - rollout_corr/rollout_is_seq_min:0.9875076413154602 - rollout_corr/rollout_is_seq_max_deviation:0.015323400497436523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2214972036890686) - rollout_corr/training_log_ppl:np.float64(0.19345460219483357) - rollout_corr/kl:np.float64(0.0011518330316775405) - rollout_corr/k3_kl:np.float64(0.0007168016556846624) - rollout_corr/rollout_ppl:np.float64(1.220014504622668) - rollout_corr/rollout_log_ppl:np.float64(0.1923027670563897) - rollout_corr/log_ppl_diff:np.float64(0.0011518351384438574) - rollout_corr/log_ppl_abs_diff:np.float64(0.0038362245832104236) - rollout_corr/log_ppl_diff_max:np.float64(0.01744517683982849) - rollout_corr/log_ppl_diff_min:np.float64(-0.013636574149131775) - rollout_corr/ppl_ratio:np.float64(1.001163745764643) - rollout_corr/chi2_token:np.float64(0.0005458625964820385) - rollout_corr/chi2_seq:np.float64(0.035545557737350464) - actor/pg_loss:np.float64(-0.00024469068739563227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026454560793354176) - actor/ppo_kl:np.float64(0.0003672881987739629) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.315181627869606) - perf/mfu/actor:np.float64(0.07761951524050657) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(102.68513107299805) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.2734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013937138020992279 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.013937138020992279 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:70.83984375 - response_length/max:326.0 - response_length/min:35.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:70.83984375 - response_length_non_aborted/max:326.0 - response_length_non_aborted/min:35.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:514.1875 - prompt_length/max:963.0 - prompt_length/min:292.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011122971773147583 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5712603759020567) - timing_s/agent_loop/generate_sequences/max:np.float64(2.48131912201643) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9329123529314529) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.48131912201643) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:512 - timing_s/agent_loop/slowest/response_length:326 - timing_s/gen:9.463260661810637 - timing_s/reward:0.06461237743496895 - timing_s/old_log_prob:3.081401403993368 - timing_s/adv:0.1542664822191 - timing_s/update_actor:12.448448494076729 - timing_s/step:25.299719965085387 - timing_s/stop_profile:0.00011725723743438721 - timing_per_token_ms/update_actor:0.08311876777979614 - timing_per_token_ms/gen:0.5218230307036469 - timing_per_token_ms/adv:0.0010300432152550295 - perf/total_num_tokens:149767 - perf/time_per_step:25.299719965085387 - perf/throughput:739.9637239398518 (TaskRunner pid=2448026) Training Progress: 16%|█▌ | 16/100 [08:19<39:51, 28.48s/it] (TaskRunner pid=2448026) step:17 - global_seqlen/min:18153 - global_seqlen/max:24780 - global_seqlen/minmax_diff:6627 - global_seqlen/balanced_min:21745 - global_seqlen/balanced_max:21757 - global_seqlen/mean:21750.375 - actor/entropy:0.2552684247493744 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.462117463350296 - training/rollout_probs_diff_mean:0.00432991236448288 - training/rollout_probs_diff_std:0.012363778427243233 - training/rollout_actor_probs_pearson_corr:0.9986448884010315 - rollout_corr/rollout_is_mean:0.9998199939727783 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.115894888760522e-05 - rollout_corr/rollout_is_max:1.6400694847106934 - rollout_corr/rollout_is_min:0.3678790032863617 - rollout_corr/rollout_is_std:0.03626453876495361 - rollout_corr/rollout_is_eff_sample_size:0.9986862536895913 - rollout_corr/rollout_is_seq_mean:0.9998749494552612 - rollout_corr/rollout_is_seq_std:0.004498427268117666 - rollout_corr/rollout_is_seq_max:1.017617106437683 - rollout_corr/rollout_is_seq_min:0.983132004737854 - rollout_corr/rollout_is_seq_max_deviation:0.017617106437683105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2815878707915545) - rollout_corr/training_log_ppl:np.float64(0.23492227659153286) - rollout_corr/kl:np.float64(0.0007474762335633045) - rollout_corr/k3_kl:np.float64(0.000889538933449785) - rollout_corr/rollout_ppl:np.float64(1.2805242980830371) - rollout_corr/rollout_log_ppl:np.float64(0.2341747986502014) - rollout_corr/log_ppl_diff:np.float64(0.0007474779413314536) - rollout_corr/log_ppl_abs_diff:np.float64(0.004234338601236232) - rollout_corr/log_ppl_diff_max:np.float64(0.019714437425136566) - rollout_corr/log_ppl_diff_min:np.float64(-0.012213677167892456) - rollout_corr/ppl_ratio:np.float64(1.0007626158185303) - rollout_corr/chi2_token:np.float64(0.0020895975176244974) - rollout_corr/chi2_seq:np.float64(0.27599234133958817) - actor/pg_loss:np.float64(-6.311770994216204e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003830975947494153) - actor/ppo_kl:np.float64(-8.472867671116546e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3631122186779976) - perf/mfu/actor:np.float64(0.0898825103887739) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(102.71323442459106) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.23046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.23046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007136915344744921 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007136915344744921 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:96.26171875 - response_length/max:596.0 - response_length/min:34.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:96.26171875 - response_length_non_aborted/max:596.0 - response_length_non_aborted/min:34.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:583.4375 - prompt_length/max:829.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000102238729596138 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5733682308346033) - timing_s/agent_loop/generate_sequences/max:np.float64(3.937309805303812) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1352126555648283) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.937309805303812) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:673 - timing_s/agent_loop/slowest/response_length:596 - timing_s/gen:10.908852050080895 - timing_s/reward:0.06951581127941608 - timing_s/old_log_prob:3.125644288957119 - timing_s/adv:0.1235087439417839 - timing_s/update_actor:12.682970460504293 - timing_s/step:26.9850155711174 - timing_s/stop_profile:0.00012632831931114197 - timing_per_token_ms/update_actor:0.07288937811706864 - timing_per_token_ms/gen:0.4426754879714684 - timing_per_token_ms/adv:0.00070980812941032 - perf/total_num_tokens:174003 - perf/time_per_step:26.9850155711174 - perf/throughput:806.0167667006966 (TaskRunner pid=2448026) Training Progress: 17%|█▋ | 17/100 [08:46<38:47, 28.04s/it] (TaskRunner pid=2448026) step:18 - global_seqlen/min:18598 - global_seqlen/max:28779 - global_seqlen/minmax_diff:10181 - global_seqlen/balanced_min:21951 - global_seqlen/balanced_max:21965 - global_seqlen/mean:21960.25 - actor/entropy:0.20109622180461884 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491804838180542 - training/rollout_probs_diff_mean:0.0040664877742528915 - training/rollout_probs_diff_std:0.01202450506389141 - training/rollout_actor_probs_pearson_corr:0.9984216690063477 - rollout_corr/rollout_is_mean:0.9999580383300781 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.627551555633545 - rollout_corr/rollout_is_min:0.6065316796302795 - rollout_corr/rollout_is_std:0.034376177936792374 - rollout_corr/rollout_is_eff_sample_size:0.9988195542025132 - rollout_corr/rollout_is_seq_mean:0.9999362230300903 - rollout_corr/rollout_is_seq_std:0.003970478195697069 - rollout_corr/rollout_is_seq_max:1.0097718238830566 - rollout_corr/rollout_is_seq_min:0.986251711845398 - rollout_corr/rollout_is_seq_max_deviation:0.01374828815460205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2182102440856397) - rollout_corr/training_log_ppl:np.float64(0.19044010097786668) - rollout_corr/kl:np.float64(0.0008987573962855322) - rollout_corr/k3_kl:np.float64(0.0006865036335392816) - rollout_corr/rollout_ppl:np.float64(1.2170202075503767) - rollout_corr/rollout_log_ppl:np.float64(0.18954134211162454) - rollout_corr/log_ppl_diff:np.float64(0.0008987588662421331) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031810645741643384) - rollout_corr/log_ppl_diff_max:np.float64(0.01513110101222992) - rollout_corr/log_ppl_diff_min:np.float64(-0.012610912322998047) - rollout_corr/ppl_ratio:np.float64(1.0009075498674065) - rollout_corr/chi2_token:np.float64(0.0009968802332878113) - rollout_corr/chi2_seq:np.float64(0.17070100456476212) - actor/pg_loss:np.float64(-0.0002593687968328595) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(1.8780048776534386e-05) - actor/ppo_kl:np.float64(0.00026812962889088965) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13995121233165264) - perf/mfu/actor:np.float64(0.09002687936600015) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(102.59389877319336) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.32421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.32421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009992443025112152 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.009992443025112152 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:93.0390625 - response_length/max:377.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:93.0390625 - response_length_non_aborted/max:377.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:593.21875 - prompt_length/max:1013.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000166209414601326 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5453768484294415) - timing_s/agent_loop/generate_sequences/max:np.float64(2.915865208953619) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.155231946206186) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.915865208953619) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1013 - timing_s/agent_loop/slowest/response_length:377 - timing_s/gen:9.456719694659114 - timing_s/reward:0.06683716364204884 - timing_s/old_log_prob:3.138201732188463 - timing_s/adv:0.14880957081913948 - timing_s/update_actor:12.751586088910699 - timing_s/step:25.65767483226955 - timing_s/stop_profile:0.00011699646711349487 - timing_per_token_ms/update_actor:0.07258333858284115 - timing_per_token_ms/gen:0.3970408806221813 - timing_per_token_ms/adv:0.0008470393712454291 - perf/total_num_tokens:175682 - perf/time_per_step:25.65767483226955 - perf/throughput:855.8940022258247 (TaskRunner pid=2448026) Training Progress: 18%|█▊ | 18/100 [09:12<37:22, 27.35s/it] (TaskRunner pid=2448026) step:19 - global_seqlen/min:22573 - global_seqlen/max:28005 - global_seqlen/minmax_diff:5432 - global_seqlen/balanced_min:24080 - global_seqlen/balanced_max:24090 - global_seqlen/mean:24087.125 - actor/entropy:0.23685698211193085 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2552533745765686 - training/rollout_probs_diff_mean:0.00392612861469388 - training/rollout_probs_diff_std:0.011309838853776455 - training/rollout_actor_probs_pearson_corr:0.9987810254096985 - rollout_corr/rollout_is_mean:1.0003230571746826 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6983729600906372 - rollout_corr/rollout_is_min:0.5636419057846069 - rollout_corr/rollout_is_std:0.033014096319675446 - rollout_corr/rollout_is_eff_sample_size:0.9989119698204697 - rollout_corr/rollout_is_seq_mean:1.0002014636993408 - rollout_corr/rollout_is_seq_std:0.0036058565601706505 - rollout_corr/rollout_is_seq_max:1.0121043920516968 - rollout_corr/rollout_is_seq_min:0.9820637702941895 - rollout_corr/rollout_is_seq_max_deviation:0.017936229705810547 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.234849292319268) - rollout_corr/training_log_ppl:np.float64(0.20395256622578017) - rollout_corr/kl:np.float64(0.0007500166338658687) - rollout_corr/k3_kl:np.float64(0.0007853202840102935) - rollout_corr/rollout_ppl:np.float64(1.233868609648198) - rollout_corr/rollout_log_ppl:np.float64(0.2032025500520831) - rollout_corr/log_ppl_diff:np.float64(0.0007500161736970767) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034327277535339817) - rollout_corr/log_ppl_diff_max:np.float64(0.017930656671524048) - rollout_corr/log_ppl_diff_min:np.float64(-0.01474597305059433) - rollout_corr/ppl_ratio:np.float64(1.0007612633053213) - rollout_corr/chi2_token:np.float64(0.001629961421713233) - rollout_corr/chi2_seq:np.float64(0.15470255119726062) - actor/pg_loss:np.float64(-0.00011331343557685614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031037671215017326) - actor/ppo_kl:np.float64(0.00043131307997157364) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1746865212917328) - perf/mfu/actor:np.float64(0.09740211773252765) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(102.77637481689453) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023119801189750433 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023119801189750433 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:92.50390625 - response_length/max:709.0 - response_length/min:33.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:92.50390625 - response_length_non_aborted/max:709.0 - response_length_non_aborted/min:33.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:660.21875 - prompt_length/max:1132.0 - prompt_length/min:362.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013253279030323029 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5893871765583754) - timing_s/agent_loop/generate_sequences/max:np.float64(4.732097275555134) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.178810004843399) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.732097275555134) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:717 - timing_s/agent_loop/slowest/response_length:709 - timing_s/gen:11.290596036240458 - timing_s/reward:0.06665989942848682 - timing_s/old_log_prob:3.110683126375079 - timing_s/adv:0.189487986266613 - timing_s/update_actor:13.022684141993523 - timing_s/step:27.784102503210306 - timing_s/stop_profile:2.8876587748527527e-05 - timing_per_token_ms/update_actor:0.0675811462658657 - timing_per_token_ms/gen:0.4767786848629897 - timing_per_token_ms/adv:0.0009833468412409794 - perf/total_num_tokens:192697 - perf/time_per_step:27.784102503210306 - perf/throughput:866.9391065346401 (TaskRunner pid=2448026) Training Progress: 19%|█▉ | 19/100 [09:40<37:06, 27.48s/it] (TaskRunner pid=2448026) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2448026) validation generation end (TaskRunner pid=2448026) [prompt] user (TaskRunner pid=2448026) Your task is to answer the user's question using available tools. (TaskRunner pid=2448026) You have access to the following tools: (TaskRunner pid=2448026) Name: Axolotl (TaskRunner pid=2448026) Description: Collection of axolotl pictures and facts (TaskRunner pid=2448026) Documentation: (TaskRunner pid=2448026) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2448026) Parameters: {} (TaskRunner pid=2448026) Output: Successful response. (TaskRunner pid=2448026) - Format: application/json (TaskRunner pid=2448026) - Structure: Object{url, source, description} (TaskRunner pid=2448026) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2448026) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2448026) Output: Successful response. (TaskRunner pid=2448026) - Format: application/json (TaskRunner pid=2448026) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2448026) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2448026) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2448026) Output: Successful response. (TaskRunner pid=2448026) - Format: application/json (TaskRunner pid=2448026) - Structure: Array[Object{fact, source}] (TaskRunner pid=2448026) (TaskRunner pid=2448026) Use the following format: (TaskRunner pid=2448026) Thought: you should always think about what to do (TaskRunner pid=2448026) Action: the action to take, should be one of the tool names. (TaskRunner pid=2448026) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2448026) (TaskRunner pid=2448026) Begin! (TaskRunner pid=2448026) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2448026) assistant (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) [response] Thought: The user is asking for a random axolotl picture. I should use the getRandomAxolotlImage tool to retrieve it. (TaskRunner pid=2448026) Action: getRandomAxolotlImage (TaskRunner pid=2448026) Action Input: {} (TaskRunner pid=2448026) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2448026) [score] 1.0 (TaskRunner pid=2448026) [acc] 1.0 (TaskRunner pid=2448026) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2448026) [incorrect_format] 0 (TaskRunner pid=2448026) [feedback] (TaskRunner pid=2448026) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2448026) step:20 - global_seqlen/min:13213 - global_seqlen/max:25199 - global_seqlen/minmax_diff:11986 - global_seqlen/balanced_min:19757 - global_seqlen/balanced_max:19763 - global_seqlen/mean:19761.25 - actor/entropy:0.23485620319843292 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135369062423706 - training/rollout_probs_diff_mean:0.003971323370933533 - training/rollout_probs_diff_std:0.011302666738629341 - training/rollout_actor_probs_pearson_corr:0.9988259673118591 - rollout_corr/rollout_is_mean:0.9999756813049316 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5716274976730347 - rollout_corr/rollout_is_min:0.6067317128181458 - rollout_corr/rollout_is_std:0.0347040556371212 - rollout_corr/rollout_is_eff_sample_size:0.9987969583907628 - rollout_corr/rollout_is_seq_mean:0.9999486207962036 - rollout_corr/rollout_is_seq_std:0.004191857762634754 - rollout_corr/rollout_is_seq_max:1.0181686878204346 - rollout_corr/rollout_is_seq_min:0.984289288520813 - rollout_corr/rollout_is_seq_max_deviation:0.01816868782043457 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2786077451892197) - rollout_corr/training_log_ppl:np.float64(0.22650701225938974) - rollout_corr/kl:np.float64(0.0007534831750746207) - rollout_corr/k3_kl:np.float64(0.0008513939894877476) - rollout_corr/rollout_ppl:np.float64(1.2776301931589842) - rollout_corr/rollout_log_ppl:np.float64(0.22575352808780735) - rollout_corr/log_ppl_diff:np.float64(0.0007534841715823859) - rollout_corr/log_ppl_abs_diff:np.float64(0.003561749865184538) - rollout_corr/log_ppl_diff_max:np.float64(0.019503533840179443) - rollout_corr/log_ppl_diff_min:np.float64(-0.014394253492355347) - rollout_corr/ppl_ratio:np.float64(1.0007649422623217) - rollout_corr/chi2_token:np.float64(0.001943441340699792) - rollout_corr/chi2_seq:np.float64(0.10392359551042318) - actor/pg_loss:np.float64(-0.00014276907313615084) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006716543293805444) - actor/ppo_kl:np.float64(8.415000964134833e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.43968162685632706) - perf/mfu/actor:np.float64(0.08129970565572053) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(102.76387786865234) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5983455882352942) - val-aux/tooluse/reward/std@16:np.float64(0.061416634311080937) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6236764705882353) - val-aux/tooluse/reward/best@2/std:np.float64(0.05129263703704207) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5728676470588236) - val-aux/tooluse/reward/worst@2/std:np.float64(0.052030501916938386) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5984264705882353) - val-aux/tooluse/reward/maj@2/std:np.float64(0.061980971842375165) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6441323529411765) - val-aux/tooluse/reward/best@4/std:np.float64(0.0392422353310596) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5534117647058823) - val-aux/tooluse/reward/worst@4/std:np.float64(0.04056937057529145) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.6017941176470588) - val-aux/tooluse/reward/maj@4/std:np.float64(0.04919778318465165) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6597205882352941) - val-aux/tooluse/reward/best@8/std:np.float64(0.02621711982315518) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.536720588235294) - val-aux/tooluse/reward/worst@8/std:np.float64(0.030329347577769342) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.6019999999999999) - val-aux/tooluse/reward/maj@8/std:np.float64(0.035266682486970555) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6691029411764705) - val-aux/tooluse/reward/best@16/std:np.float64(0.01510568160913284) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5239558823529411) - val-aux/tooluse/reward/worst@16/std:np.float64(0.01931804814737708) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.6005882352941176) - val-aux/tooluse/reward/maj@16/std:np.float64(0.0226152794781832) - val-aux/tooluse/score/mean@16:np.float64(0.5983455882352942) - val-aux/tooluse/score/std@16:np.float64(0.061416634311080937) - val-aux/tooluse/score/best@2/mean:np.float64(0.6236764705882353) - val-aux/tooluse/score/best@2/std:np.float64(0.05129263703704207) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5728676470588236) - val-aux/tooluse/score/worst@2/std:np.float64(0.052030501916938386) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5984264705882353) - val-aux/tooluse/score/maj@2/std:np.float64(0.061980971842375165) - val-aux/tooluse/score/best@4/mean:np.float64(0.6441323529411765) - val-aux/tooluse/score/best@4/std:np.float64(0.0392422353310596) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5534117647058823) - val-aux/tooluse/score/worst@4/std:np.float64(0.04056937057529145) - val-aux/tooluse/score/maj@4/mean:np.float64(0.6017941176470588) - val-aux/tooluse/score/maj@4/std:np.float64(0.04919778318465165) - val-aux/tooluse/score/best@8/mean:np.float64(0.6597205882352941) - val-aux/tooluse/score/best@8/std:np.float64(0.02621711982315518) - val-aux/tooluse/score/worst@8/mean:np.float64(0.536720588235294) - val-aux/tooluse/score/worst@8/std:np.float64(0.030329347577769342) - val-aux/tooluse/score/maj@8/mean:np.float64(0.6019999999999999) - val-aux/tooluse/score/maj@8/std:np.float64(0.035266682486970555) - val-aux/tooluse/score/best@16/mean:np.float64(0.6691029411764705) - val-aux/tooluse/score/best@16/std:np.float64(0.01510568160913284) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5239558823529411) - val-aux/tooluse/score/worst@16/std:np.float64(0.01931804814737708) - val-aux/tooluse/score/maj@16/mean:np.float64(0.6005882352941176) - val-aux/tooluse/score/maj@16/std:np.float64(0.0226152794781832) - val-core/tooluse/acc/mean@16:np.float64(0.5983455882352942) - val-aux/tooluse/acc/std@16:np.float64(0.061416634311080937) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6236764705882353) - val-aux/tooluse/acc/best@2/std:np.float64(0.05129263703704207) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5728676470588236) - val-aux/tooluse/acc/worst@2/std:np.float64(0.052030501916938386) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5984264705882353) - val-aux/tooluse/acc/maj@2/std:np.float64(0.061980971842375165) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6441323529411765) - val-aux/tooluse/acc/best@4/std:np.float64(0.0392422353310596) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5534117647058823) - val-aux/tooluse/acc/worst@4/std:np.float64(0.04056937057529145) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.6017941176470588) - val-aux/tooluse/acc/maj@4/std:np.float64(0.04919778318465165) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6597205882352941) - val-aux/tooluse/acc/best@8/std:np.float64(0.02621711982315518) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.536720588235294) - val-aux/tooluse/acc/worst@8/std:np.float64(0.030329347577769342) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.6019999999999999) - val-aux/tooluse/acc/maj@8/std:np.float64(0.035266682486970555) - val-core/tooluse/acc/best@16/mean:np.float64(0.6691029411764705) - val-core/tooluse/acc/best@16/std:np.float64(0.01510568160913284) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5239558823529411) - val-aux/tooluse/acc/worst@16/std:np.float64(0.01931804814737708) - val-core/tooluse/acc/maj@16/mean:np.float64(0.6005882352941176) - val-core/tooluse/acc/maj@16/std:np.float64(0.0226152794781832) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0060984110459685326 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0060984110459685326 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:84.0703125 - response_length/max:553.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:84.0703125 - response_length_non_aborted/max:553.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:533.46875 - prompt_length/max:864.0 - prompt_length/min:253.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.92845743894577e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5227947365492582) - timing_s/agent_loop/generate_sequences/max:np.float64(3.849112132564187) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0496431276769727) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.849112132564187) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:536 - timing_s/agent_loop/slowest/response_length:553 - timing_s/gen:10.588456066325307 - timing_s/reward:0.0629133265465498 - timing_s/old_log_prob:2.979782210662961 - timing_s/adv:0.12297306954860687 - timing_s/update_actor:12.712873607873917 - timing_s/step:26.56435263156891 - timing_s/testing:24.196175483986735 - timing_s/save_checkpoint:13.08900959789753 - timing_s/stop_profile:0.00017257221043109894 - timing_per_token_ms/update_actor:0.08041541911489605 - timing_per_token_ms/gen:0.4919829042991036 - timing_per_token_ms/adv:0.0007778674776937623 - perf/total_num_tokens:158090 - perf/time_per_step:26.56435263156891 - perf/throughput:743.9010569569031 (TaskRunner pid=2448026) Training Progress: 20%|██ | 20/100 [10:44<51:13, 38.42s/it] (TaskRunner pid=2448026) step:21 - global_seqlen/min:17017 - global_seqlen/max:24845 - global_seqlen/minmax_diff:7828 - global_seqlen/balanced_min:20628 - global_seqlen/balanced_max:20632 - global_seqlen/mean:20630.25 - actor/entropy:0.1994934380054474 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31754884123802185 - training/rollout_probs_diff_mean:0.0036330241709947586 - training/rollout_probs_diff_std:0.01089955773204565 - training/rollout_actor_probs_pearson_corr:0.998763918876648 - rollout_corr/rollout_is_mean:0.9998865127563477 - rollout_corr/rollout_is_ratio_fraction_high:0.00010691756324376911 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.7407984733581543 - rollout_corr/rollout_is_min:0.6783112287521362 - rollout_corr/rollout_is_std:0.033427655696868896 - rollout_corr/rollout_is_eff_sample_size:0.9988834226305476 - rollout_corr/rollout_is_seq_mean:0.9998796582221985 - rollout_corr/rollout_is_seq_std:0.004669966641813517 - rollout_corr/rollout_is_seq_max:1.0312814712524414 - rollout_corr/rollout_is_seq_min:0.9865773916244507 - rollout_corr/rollout_is_seq_max_deviation:0.031281471252441406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.212485512252897) - rollout_corr/training_log_ppl:np.float64(0.18563543116033543) - rollout_corr/kl:np.float64(0.0009345494108963237) - rollout_corr/k3_kl:np.float64(0.000743327141023542) - rollout_corr/rollout_ppl:np.float64(1.211291047744453) - rollout_corr/rollout_log_ppl:np.float64(0.18470087937748758) - rollout_corr/log_ppl_diff:np.float64(0.0009345517828478478) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036773754181922413) - rollout_corr/log_ppl_diff_max:np.float64(0.016793623566627502) - rollout_corr/log_ppl_diff_min:np.float64(-0.02077123522758484) - rollout_corr/ppl_ratio:np.float64(1.0009464754257351) - rollout_corr/chi2_token:np.float64(0.0012122192420065403) - rollout_corr/chi2_seq:np.float64(0.05832636100240052) - actor/pg_loss:np.float64(-0.0002913718344643712) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022710490702593233) - actor/ppo_kl:np.float64(0.00023803165971991724) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.44960440695285797) - perf/mfu/actor:np.float64(0.08533227939976737) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(112.86812210083008) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012763284146785736 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012763284146785736 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:73.0703125 - response_length/max:258.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:73.0703125 - response_length_non_aborted/max:258.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:571.625 - prompt_length/max:821.0 - prompt_length/min:253.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010188296437263489 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4669682066887617) - timing_s/agent_loop/generate_sequences/max:np.float64(2.1123071759939194) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9833372019638773) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.1123071759939194) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:753 - timing_s/agent_loop/slowest/response_length:258 - timing_s/gen:8.475934758782387 - timing_s/reward:0.0643459353595972 - timing_s/old_log_prob:3.2948878947645426 - timing_s/adv:0.18654972687363625 - timing_s/update_actor:12.694566590711474 - timing_s/step:24.787276620045304 - timing_s/stop_profile:0.00012628361582756042 - timing_per_token_ms/update_actor:0.07691718829577607 - timing_per_token_ms/gen:0.4531131593489996 - timing_per_token_ms/adv:0.0011303166883195564 - perf/total_num_tokens:165042 - perf/time_per_step:24.787276620045304 - perf/throughput:832.2919179962051 (TaskRunner pid=2448026) Training Progress: 21%|██ | 21/100 [11:08<45:13, 34.34s/it] (TaskRunner pid=2448026) step:22 - global_seqlen/min:16623 - global_seqlen/max:30346 - global_seqlen/minmax_diff:13723 - global_seqlen/balanced_min:21220 - global_seqlen/balanced_max:21223 - global_seqlen/mean:21221.25 - actor/entropy:0.20342501997947693 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.11853548884391785 - training/rollout_probs_diff_mean:0.003692347789183259 - training/rollout_probs_diff_std:0.010465496219694614 - training/rollout_actor_probs_pearson_corr:0.9988213181495667 - rollout_corr/rollout_is_mean:0.9999866485595703 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.463401198387146 - rollout_corr/rollout_is_min:0.6363806128501892 - rollout_corr/rollout_is_std:0.03154614195227623 - rollout_corr/rollout_is_eff_sample_size:0.9990059491803874 - rollout_corr/rollout_is_seq_mean:0.9999550580978394 - rollout_corr/rollout_is_seq_std:0.003941687289625406 - rollout_corr/rollout_is_seq_max:1.013562798500061 - rollout_corr/rollout_is_seq_min:0.985810399055481 - rollout_corr/rollout_is_seq_max_deviation:0.014189600944519043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2090849815867841) - rollout_corr/training_log_ppl:np.float64(0.17842540883793845) - rollout_corr/kl:np.float64(0.0005934757994481288) - rollout_corr/k3_kl:np.float64(0.0006870366126681127) - rollout_corr/rollout_ppl:np.float64(1.208236088976264) - rollout_corr/rollout_log_ppl:np.float64(0.1778319299555733) - rollout_corr/log_ppl_diff:np.float64(0.0005934788823651616) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036253461985324975) - rollout_corr/log_ppl_diff_max:np.float64(0.014084994792938232) - rollout_corr/log_ppl_diff_min:np.float64(-0.029756814241409302) - rollout_corr/ppl_ratio:np.float64(1.000605919631198) - rollout_corr/chi2_token:np.float64(0.0016279437113553286) - rollout_corr/chi2_seq:np.float64(0.0695030358619988) - actor/pg_loss:np.float64(-0.00011226569768041372) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018624756921781227) - actor/ppo_kl:np.float64(6.70735505536868e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1671545635908842) - perf/mfu/actor:np.float64(0.08599787874843301) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(109.28922653198242) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.41796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013649425469338894 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013649425469338894 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:78.1640625 - response_length/max:438.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:78.1640625 - response_length_non_aborted/max:438.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:585.0 - prompt_length/max:1521.0 - prompt_length/min:244.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013805367052555084 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5069361124187708) - timing_s/agent_loop/generate_sequences/max:np.float64(3.1828822437673807) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0275113745665294) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.1828822437673807) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:520 - timing_s/agent_loop/slowest/response_length:438 - timing_s/gen:9.953707370907068 - timing_s/reward:0.0674618799239397 - timing_s/old_log_prob:3.1266141925007105 - timing_s/adv:0.18255320377647877 - timing_s/update_actor:12.927832765504718 - timing_s/step:26.353634336963296 - timing_s/stop_profile:0.00011230632662773132 - timing_per_token_ms/update_actor:0.07614910034461164 - timing_per_token_ms/gen:0.4974366502202433 - timing_per_token_ms/adv:0.0010752971889996983 - perf/total_num_tokens:169770 - perf/time_per_step:26.353634336963296 - perf/throughput:805.2494668727844 (TaskRunner pid=2448026) Training Progress: 22%|██▏ | 22/100 [11:35<41:32, 31.96s/it] (TaskRunner pid=2448026) step:23 - global_seqlen/min:16521 - global_seqlen/max:23594 - global_seqlen/minmax_diff:7073 - global_seqlen/balanced_min:20650 - global_seqlen/balanced_max:20654 - global_seqlen/mean:20652.25 - actor/entropy:0.20465673506259918 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27756577730178833 - training/rollout_probs_diff_mean:0.0036886711604893208 - training/rollout_probs_diff_std:0.010615203529596329 - training/rollout_actor_probs_pearson_corr:0.9987795352935791 - rollout_corr/rollout_is_mean:1.0000567436218262 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5713884830474854 - rollout_corr/rollout_is_min:0.6232436299324036 - rollout_corr/rollout_is_std:0.031244277954101562 - rollout_corr/rollout_is_eff_sample_size:0.9990248661513272 - rollout_corr/rollout_is_seq_mean:1.0002140998840332 - rollout_corr/rollout_is_seq_std:0.00401710532605648 - rollout_corr/rollout_is_seq_max:1.0167683362960815 - rollout_corr/rollout_is_seq_min:0.9884085059165955 - rollout_corr/rollout_is_seq_max_deviation:0.016768336296081543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2032523420639336) - rollout_corr/training_log_ppl:np.float64(0.17796000392991118) - rollout_corr/kl:np.float64(5.591605728194615e-05) - rollout_corr/k3_kl:np.float64(0.0007072796297192818) - rollout_corr/rollout_ppl:np.float64(1.2032720684073865) - rollout_corr/rollout_log_ppl:np.float64(0.1779040871260804) - rollout_corr/log_ppl_diff:np.float64(5.591680383076891e-05) - rollout_corr/log_ppl_abs_diff:np.float64(0.003544284300005529) - rollout_corr/log_ppl_diff_max:np.float64(0.014830626547336578) - rollout_corr/log_ppl_diff_min:np.float64(-0.023066699504852295) - rollout_corr/ppl_ratio:np.float64(1.0000684873666614) - rollout_corr/chi2_token:np.float64(0.0028226797003299) - rollout_corr/chi2_seq:np.float64(0.14784812927246094) - actor/pg_loss:np.float64(-0.00014821125660091639) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005488207025337033) - actor/ppo_kl:np.float64(-0.00022811098898145588) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23881025798618793) - perf/mfu/actor:np.float64(0.08538115311950467) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(108.91734313964844) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.23828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.23828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00013568696158472449 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.00013568696158472449 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:75.5703125 - response_length/max:575.0 - response_length/min:30.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:75.5703125 - response_length_non_aborted/max:575.0 - response_length_non_aborted/min:30.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:569.8125 - prompt_length/max:1109.0 - prompt_length/min:320.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001461803913116455 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5939608886837959) - timing_s/agent_loop/generate_sequences/max:np.float64(4.017370196059346) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0364576009087614) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.017370196059346) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:749 - timing_s/agent_loop/slowest/response_length:575 - timing_s/gen:10.673288064077497 - timing_s/reward:0.0640814621001482 - timing_s/old_log_prob:3.241801545023918 - timing_s/adv:0.1636136882007122 - timing_s/update_actor:12.696003831923008 - timing_s/step:26.935870859771967 - timing_s/stop_profile:0.0001154821366071701 - timing_per_token_ms/update_actor:0.07684395061024227 - timing_per_token_ms/gen:0.5517051620013179 - timing_per_token_ms/adv:0.0009902897275158409 - perf/total_num_tokens:165218 - perf/time_per_step:26.935870859771967 - perf/throughput:766.7192238749409 (TaskRunner pid=2448026) Training Progress: 23%|██▎ | 23/100 [12:02<39:06, 30.47s/it] (TaskRunner pid=2448026) step:24 - global_seqlen/min:19600 - global_seqlen/max:24926 - global_seqlen/minmax_diff:5326 - global_seqlen/balanced_min:22148 - global_seqlen/balanced_max:22337 - global_seqlen/mean:22172.625 - actor/entropy:0.15866845846176147 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.318619966506958 - training/rollout_probs_diff_mean:0.0030198050662875175 - training/rollout_probs_diff_std:0.010547653771936893 - training/rollout_actor_probs_pearson_corr:0.998584508895874 - rollout_corr/rollout_is_mean:0.9998258352279663 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6253578662872314 - rollout_corr/rollout_is_min:0.6065307855606079 - rollout_corr/rollout_is_std:0.030177796259522438 - rollout_corr/rollout_is_eff_sample_size:0.9990899507625369 - rollout_corr/rollout_is_seq_mean:0.9997492432594299 - rollout_corr/rollout_is_seq_std:0.0041739013977348804 - rollout_corr/rollout_is_seq_max:1.0161190032958984 - rollout_corr/rollout_is_seq_min:0.9881100058555603 - rollout_corr/rollout_is_seq_max_deviation:0.016119003295898438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1661933697760105) - rollout_corr/training_log_ppl:np.float64(0.14853941459296038) - rollout_corr/kl:np.float64(0.0008280829907789666) - rollout_corr/k3_kl:np.float64(0.0005883863503304276) - rollout_corr/rollout_ppl:np.float64(1.165204216260463) - rollout_corr/rollout_log_ppl:np.float64(0.14771133106114576) - rollout_corr/log_ppl_diff:np.float64(0.0008280835318146273) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032514531776541844) - rollout_corr/log_ppl_diff_max:np.float64(0.019103258848190308) - rollout_corr/log_ppl_diff_min:np.float64(-0.012403517961502075) - rollout_corr/ppl_ratio:np.float64(1.000837619183585) - rollout_corr/chi2_token:np.float64(0.0006681184750050306) - rollout_corr/chi2_seq:np.float64(0.03422388737089932) - actor/pg_loss:np.float64(-0.00031874573323875666) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010295051888533635) - actor/ppo_kl:np.float64(9.302371081787442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.51068065315485) - perf/mfu/actor:np.float64(0.0902185160373687) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(108.87287139892578) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.37890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014314080588519573 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014314080588519573 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:77.70703125 - response_length/max:694.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:77.70703125 - response_length_non_aborted/max:694.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:615.1875 - prompt_length/max:917.0 - prompt_length/min:348.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010121800005435944 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4148831907659769) - timing_s/agent_loop/generate_sequences/max:np.float64(4.839786099269986) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9898142371384893) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.839786099269986) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:824 - timing_s/agent_loop/slowest/response_length:694 - timing_s/gen:11.505728840827942 - timing_s/reward:0.06582257151603699 - timing_s/old_log_prob:3.0712999496608973 - timing_s/adv:0.12183690816164017 - timing_s/update_actor:12.823335360735655 - timing_s/step:27.683567823842168 - timing_s/stop_profile:0.00011288747191429138 - timing_per_token_ms/update_actor:0.07229260947190316 - timing_per_token_ms/gen:0.5783807792101715 - timing_per_token_ms/adv:0.0006868656065849226 - perf/total_num_tokens:177381 - perf/time_per_step:27.683567823842168 - perf/throughput:800.9309038881929 (TaskRunner pid=2448026) Training Progress: 24%|██▍ | 24/100 [12:30<37:33, 29.65s/it] (TaskRunner pid=2448026) step:25 - global_seqlen/min:16196 - global_seqlen/max:25259 - global_seqlen/minmax_diff:9063 - global_seqlen/balanced_min:21469 - global_seqlen/balanced_max:21473 - global_seqlen/mean:21471.0 - actor/entropy:0.1768006682395935 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17585769295692444 - training/rollout_probs_diff_mean:0.0034979358315467834 - training/rollout_probs_diff_std:0.01094727125018835 - training/rollout_actor_probs_pearson_corr:0.9985765814781189 - rollout_corr/rollout_is_mean:0.9997981190681458 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.80769231216982e-05 - rollout_corr/rollout_is_max:1.5659326314926147 - rollout_corr/rollout_is_min:0.4187690317630768 - rollout_corr/rollout_is_std:0.031105661764740944 - rollout_corr/rollout_is_eff_sample_size:0.9990329566469225 - rollout_corr/rollout_is_seq_mean:0.9997290968894958 - rollout_corr/rollout_is_seq_std:0.003951251972466707 - rollout_corr/rollout_is_seq_max:1.0155400037765503 - rollout_corr/rollout_is_seq_min:0.9840248823165894 - rollout_corr/rollout_is_seq_max_deviation:0.015975117683410645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1946323746815324) - rollout_corr/training_log_ppl:np.float64(0.16932234348496422) - rollout_corr/kl:np.float64(0.0012471448507298533) - rollout_corr/k3_kl:np.float64(0.0007940559721237861) - rollout_corr/rollout_ppl:np.float64(1.1930478955619037) - rollout_corr/rollout_log_ppl:np.float64(0.16807519879512256) - rollout_corr/log_ppl_diff:np.float64(0.0012471446898416616) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036384987834026106) - rollout_corr/log_ppl_diff_max:np.float64(0.02983108162879944) - rollout_corr/log_ppl_diff_min:np.float64(-0.011248350143432617) - rollout_corr/ppl_ratio:np.float64(1.001261152792722) - rollout_corr/chi2_token:np.float64(0.000547436298802495) - rollout_corr/chi2_seq:np.float64(0.04785990156233311) - actor/pg_loss:np.float64(0.00023725267965346575) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00046602546353824437) - actor/ppo_kl:np.float64(0.0004585378105730342) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.41720741987228394) - perf/mfu/actor:np.float64(0.08887373438442872) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(108.89731979370117) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002938702004030347 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002938702004030347 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:81.25 - response_length/max:343.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:81.25 - response_length_non_aborted/max:343.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:589.71875 - prompt_length/max:969.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011132098734378815 - timing_s/agent_loop/generate_sequences/min:np.float64(0.47054541297256947) - timing_s/agent_loop/generate_sequences/max:np.float64(2.7069908641278744) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.037047138546768) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.7069908641278744) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:784 - timing_s/agent_loop/slowest/response_length:343 - timing_s/gen:9.545532915741205 - timing_s/reward:0.06563716568052769 - timing_s/old_log_prob:3.1229444965720177 - timing_s/adv:0.12703221291303635 - timing_s/update_actor:12.64083474315703 - timing_s/step:25.570132851600647 - timing_s/stop_profile:0.00011366792023181915 - timing_per_token_ms/update_actor:0.07359248953912853 - timing_per_token_ms/gen:0.45891985171832717 - timing_per_token_ms/adv:0.0007395569192925128 - perf/total_num_tokens:171768 - perf/time_per_step:25.570132851600647 - perf/throughput:839.6905923254111 (TaskRunner pid=2448026) Training Progress: 25%|██▌ | 25/100 [12:55<35:33, 28.44s/it] (TaskRunner pid=2448026) step:26 - global_seqlen/min:15893 - global_seqlen/max:22455 - global_seqlen/minmax_diff:6562 - global_seqlen/balanced_min:19811 - global_seqlen/balanced_max:19817 - global_seqlen/mean:19813.125 - actor/entropy:0.1914120465517044 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3008977174758911 - training/rollout_probs_diff_mean:0.003540771547704935 - training/rollout_probs_diff_std:0.011071168817579746 - training/rollout_actor_probs_pearson_corr:0.9986568689346313 - rollout_corr/rollout_is_mean:0.9997686147689819 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7992891073226929 - rollout_corr/rollout_is_min:0.5216240882873535 - rollout_corr/rollout_is_std:0.03364094719290733 - rollout_corr/rollout_is_eff_sample_size:0.9988692685872906 - rollout_corr/rollout_is_seq_mean:0.9996310472488403 - rollout_corr/rollout_is_seq_std:0.003798197489231825 - rollout_corr/rollout_is_seq_max:1.0180745124816895 - rollout_corr/rollout_is_seq_min:0.986883819103241 - rollout_corr/rollout_is_seq_max_deviation:0.018074512481689453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2065912880934775) - rollout_corr/training_log_ppl:np.float64(0.17927087494172156) - rollout_corr/kl:np.float64(0.0008006306109251682) - rollout_corr/k3_kl:np.float64(0.0006834625761955593) - rollout_corr/rollout_ppl:np.float64(1.205544595606625) - rollout_corr/rollout_log_ppl:np.float64(0.1784702426448348) - rollout_corr/log_ppl_diff:np.float64(0.0008006322968867607) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032080167802632786) - rollout_corr/log_ppl_diff_max:np.float64(0.014288939535617828) - rollout_corr/log_ppl_diff_min:np.float64(-0.014724373817443848) - rollout_corr/ppl_ratio:np.float64(1.0008104466833174) - rollout_corr/chi2_token:np.float64(0.0010801516473293304) - rollout_corr/chi2_seq:np.float64(0.0456055358517915) - actor/pg_loss:np.float64(8.355360478162766e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003188047476214706) - actor/ppo_kl:np.float64(-0.00011592237819968432) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1840866133570671) - perf/mfu/actor:np.float64(0.08164059032790122) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(108.90914154052734) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01072641834616661 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01072641834616661 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:83.94140625 - response_length/max:434.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:83.94140625 - response_length_non_aborted/max:434.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:535.21875 - prompt_length/max:907.0 - prompt_length/min:241.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015681609511375427 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4355869609862566) - timing_s/agent_loop/generate_sequences/max:np.float64(2.983456863090396) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.019950789734139) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.983456863090396) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:733 - timing_s/agent_loop/slowest/response_length:388 - timing_s/gen:9.800003515556455 - timing_s/reward:0.05858388729393482 - timing_s/old_log_prob:2.982895253226161 - timing_s/adv:0.1474037915468216 - timing_s/update_actor:12.716893495991826 - timing_s/step:25.748708793893456 - timing_s/stop_profile:0.00011250190436840057 - timing_per_token_ms/update_actor:0.08023023561396692 - timing_per_token_ms/gen:0.4560474436016778 - timing_per_token_ms/adv:0.0009299630393162461 - perf/total_num_tokens:158505 - perf/time_per_step:25.748708793893456 - perf/throughput:769.4803323380187 (TaskRunner pid=2448026) Training Progress: 26%|██▌ | 26/100 [13:21<34:06, 27.65s/it] (TaskRunner pid=2448026) step:27 - global_seqlen/min:17814 - global_seqlen/max:28973 - global_seqlen/minmax_diff:11159 - global_seqlen/balanced_min:22803 - global_seqlen/balanced_max:22808 - global_seqlen/mean:22806.375 - actor/entropy:0.17007948458194733 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3394762873649597 - training/rollout_probs_diff_mean:0.003320823423564434 - training/rollout_probs_diff_std:0.010907599702477455 - training/rollout_actor_probs_pearson_corr:0.998582661151886 - rollout_corr/rollout_is_mean:1.0001362562179565 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.520833969116211 - rollout_corr/rollout_is_min:0.5632621645927429 - rollout_corr/rollout_is_std:0.03154236450791359 - rollout_corr/rollout_is_eff_sample_size:0.9990063060977364 - rollout_corr/rollout_is_seq_mean:1.0002071857452393 - rollout_corr/rollout_is_seq_std:0.0039260718040168285 - rollout_corr/rollout_is_seq_max:1.0183439254760742 - rollout_corr/rollout_is_seq_min:0.9876532554626465 - rollout_corr/rollout_is_seq_max_deviation:0.01834392547607422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.164628612343222) - rollout_corr/training_log_ppl:np.float64(0.14505130331235705) - rollout_corr/kl:np.float64(0.0010262654820889505) - rollout_corr/k3_kl:np.float64(0.0006483714343001878) - rollout_corr/rollout_ppl:np.float64(1.1633458682335913) - rollout_corr/rollout_log_ppl:np.float64(0.1440250356499746) - rollout_corr/log_ppl_diff:np.float64(0.0010262676623824518) - rollout_corr/log_ppl_abs_diff:np.float64(0.003299875901575433) - rollout_corr/log_ppl_diff_max:np.float64(0.02381649613380432) - rollout_corr/log_ppl_diff_min:np.float64(-0.009337767958641052) - rollout_corr/ppl_ratio:np.float64(1.0010370963718742) - rollout_corr/chi2_token:np.float64(0.0005191501695662737) - rollout_corr/chi2_seq:np.float64(0.06721403636038303) - actor/pg_loss:np.float64(-6.523833144456148e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007159791020967532) - actor/ppo_kl:np.float64(0.0007539142534511711) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26175145618617535) - perf/mfu/actor:np.float64(0.08950987698069414) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(108.99040985107422) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.28125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009949836879968643 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009949836879968643 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:82.54296875 - response_length/max:481.0 - response_length/min:28.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:82.54296875 - response_length_non_aborted/max:481.0 - response_length_non_aborted/min:28.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:630.15625 - prompt_length/max:1261.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001657400280237198 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5644382853060961) - timing_s/agent_loop/generate_sequences/max:np.float64(3.478972926735878) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0982625883480068) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.478972926735878) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:481 - timing_s/gen:10.09317115880549 - timing_s/reward:0.07168988883495331 - timing_s/old_log_prob:3.065929276868701 - timing_s/adv:0.14469160325825214 - timing_s/update_actor:13.111994344741106 - timing_s/step:26.575086729601026 - timing_s/stop_profile:0.00011673197150230408 - timing_per_token_ms/update_actor:0.07186583984051119 - timing_per_token_ms/gen:0.4776475869010217 - timing_per_token_ms/adv:0.0007930436295676765 - perf/total_num_tokens:182451 - perf/time_per_step:26.575086729601026 - perf/throughput:858.1862867298494 (TaskRunner pid=2448026) Training Progress: 27%|██▋ | 27/100 [13:48<33:16, 27.34s/it] (TaskRunner pid=2448026) step:28 - global_seqlen/min:17718 - global_seqlen/max:24708 - global_seqlen/minmax_diff:6990 - global_seqlen/balanced_min:20765 - global_seqlen/balanced_max:20777 - global_seqlen/mean:20772.625 - actor/entropy:0.18555545806884766 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016391694545746 - training/rollout_probs_diff_mean:0.0037023790646344423 - training/rollout_probs_diff_std:0.011125861667096615 - training/rollout_actor_probs_pearson_corr:0.9986006617546082 - rollout_corr/rollout_is_mean:0.9997002482414246 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9402096271514893 - rollout_corr/rollout_is_min:0.6085872650146484 - rollout_corr/rollout_is_std:0.0329960361123085 - rollout_corr/rollout_is_eff_sample_size:0.9989117319204782 - rollout_corr/rollout_is_seq_mean:0.9997862577438354 - rollout_corr/rollout_is_seq_std:0.004189760889858007 - rollout_corr/rollout_is_seq_max:1.0140587091445923 - rollout_corr/rollout_is_seq_min:0.9888555407524109 - rollout_corr/rollout_is_seq_max_deviation:0.014058709144592285 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1885352972894907) - rollout_corr/training_log_ppl:np.float64(0.16547452992017497) - rollout_corr/kl:np.float64(0.0011552481839132156) - rollout_corr/k3_kl:np.float64(0.0008155073295803561) - rollout_corr/rollout_ppl:np.float64(1.187074895016849) - rollout_corr/rollout_log_ppl:np.float64(0.1643192819065007) - rollout_corr/log_ppl_diff:np.float64(0.0011552480136742815) - rollout_corr/log_ppl_abs_diff:np.float64(0.003746223170310259) - rollout_corr/log_ppl_diff_max:np.float64(0.02639336884021759) - rollout_corr/log_ppl_diff_min:np.float64(-0.014315783977508545) - rollout_corr/ppl_ratio:np.float64(1.0011693930719048) - rollout_corr/chi2_token:np.float64(0.0007965350523591042) - rollout_corr/chi2_seq:np.float64(0.023620666936039925) - actor/pg_loss:np.float64(5.106779281049967e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006407758210116299) - actor/ppo_kl:np.float64(0.0004109144039432522) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37027278542518616) - perf/mfu/actor:np.float64(0.08552006734884661) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(108.95157623291016) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.3203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0052786595188081264 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0052786595188081264 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:72.61328125 - response_length/max:323.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:72.61328125 - response_length_non_aborted/max:323.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:576.53125 - prompt_length/max:1090.0 - prompt_length/min:220.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001970101147890091 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5523437391966581) - timing_s/agent_loop/generate_sequences/max:np.float64(2.514882693067193) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9715858603012748) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.514882693067193) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1090 - timing_s/agent_loop/slowest/response_length:323 - timing_s/gen:9.067848157137632 - timing_s/reward:0.06796282529830933 - timing_s/old_log_prob:3.0990322437137365 - timing_s/adv:0.1646527647972107 - timing_s/update_actor:12.695804292336106 - timing_s/step:25.18487747013569 - timing_s/stop_profile:0.0001130085438489914 - timing_per_token_ms/update_actor:0.0763974479172475 - timing_per_token_ms/gen:0.4878072062584126 - timing_per_token_ms/adv:0.0009908037910303265 - perf/total_num_tokens:166181 - perf/time_per_step:25.18487747013569 - perf/throughput:824.8054819656061 (TaskRunner pid=2448026) Training Progress: 28%|██▊ | 28/100 [14:13<32:03, 26.72s/it] (TaskRunner pid=2448026) step:29 - global_seqlen/min:16947 - global_seqlen/max:23364 - global_seqlen/minmax_diff:6417 - global_seqlen/balanced_min:19887 - global_seqlen/balanced_max:19888 - global_seqlen/mean:19887.125 - actor/entropy:0.17226053774356842 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21718502044677734 - training/rollout_probs_diff_mean:0.003391335951164365 - training/rollout_probs_diff_std:0.010995929129421711 - training/rollout_actor_probs_pearson_corr:0.9985835552215576 - rollout_corr/rollout_is_mean:1.000108003616333 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.963408350944519 - rollout_corr/rollout_is_min:0.5055305361747742 - rollout_corr/rollout_is_std:0.03393642604351044 - rollout_corr/rollout_is_eff_sample_size:0.9988498817672742 - rollout_corr/rollout_is_seq_mean:1.000058889389038 - rollout_corr/rollout_is_seq_std:0.004101099446415901 - rollout_corr/rollout_is_seq_max:1.0116193294525146 - rollout_corr/rollout_is_seq_min:0.9851050972938538 - rollout_corr/rollout_is_seq_max_deviation:0.01489490270614624 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1834971481002867) - rollout_corr/training_log_ppl:np.float64(0.15577457194012823) - rollout_corr/kl:np.float64(0.0005616647986279588) - rollout_corr/k3_kl:np.float64(0.0007758079397159889) - rollout_corr/rollout_ppl:np.float64(1.182836175430566) - rollout_corr/rollout_log_ppl:np.float64(0.15521290769902407) - rollout_corr/log_ppl_diff:np.float64(0.0005616642411041539) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035927129465562757) - rollout_corr/log_ppl_diff_max:np.float64(0.021324068307876587) - rollout_corr/log_ppl_diff_min:np.float64(-0.014472469687461853) - rollout_corr/ppl_ratio:np.float64(1.0005749957635999) - rollout_corr/chi2_token:np.float64(0.0020505180582404137) - rollout_corr/chi2_seq:np.float64(0.1587328698951751) - actor/pg_loss:np.float64(0.0005443779518827796) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002119834789482411) - actor/ppo_kl:np.float64(4.689464234708396e-05) - actor/pg_clipfrac_lower:np.float64(5.278716344037093e-05) - actor/grad_norm:np.float64(0.4089682847261429) - perf/mfu/actor:np.float64(0.08139443714592147) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(108.92642974853516) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.3671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015356040559709072 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015356040559709072 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:74.66015625 - response_length/max:393.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:74.66015625 - response_length_non_aborted/max:393.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:546.8125 - prompt_length/max:883.0 - prompt_length/min:249.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.585117757320404e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5090045109391212) - timing_s/agent_loop/generate_sequences/max:np.float64(2.860810087993741) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9791860311524943) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.860810087993741) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:478 - timing_s/agent_loop/slowest/response_length:393 - timing_s/gen:9.595924274995923 - timing_s/reward:0.0676498468965292 - timing_s/old_log_prob:2.920492572709918 - timing_s/adv:0.11992739886045456 - timing_s/update_actor:12.859809501096606 - timing_s/step:25.652669947594404 - timing_s/stop_profile:0.00011185556650161743 - timing_per_token_ms/update_actor:0.08082999365856432 - timing_per_token_ms/gen:0.5020626942393095 - timing_per_token_ms/adv:0.0007538005044749717 - perf/total_num_tokens:159097 - perf/time_per_step:25.652669947594404 - perf/throughput:775.2458142028576 (TaskRunner pid=2448026) Training Progress: 29%|██▉ | 29/100 [14:39<31:15, 26.42s/it] (TaskRunner pid=2448026) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2448026) validation generation end (TaskRunner pid=2448026) [prompt] user (TaskRunner pid=2448026) Your task is to answer the user's question using available tools. (TaskRunner pid=2448026) You have access to the following tools: (TaskRunner pid=2448026) Name: Axolotl (TaskRunner pid=2448026) Description: Collection of axolotl pictures and facts (TaskRunner pid=2448026) Documentation: (TaskRunner pid=2448026) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2448026) Parameters: {} (TaskRunner pid=2448026) Output: Successful response. (TaskRunner pid=2448026) - Format: application/json (TaskRunner pid=2448026) - Structure: Object{url, source, description} (TaskRunner pid=2448026) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2448026) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2448026) Output: Successful response. (TaskRunner pid=2448026) - Format: application/json (TaskRunner pid=2448026) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2448026) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2448026) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2448026) Output: Successful response. (TaskRunner pid=2448026) - Format: application/json (TaskRunner pid=2448026) - Structure: Array[Object{fact, source}] (TaskRunner pid=2448026) (TaskRunner pid=2448026) Use the following format: (TaskRunner pid=2448026) Thought: you should always think about what to do (TaskRunner pid=2448026) Action: the action to take, should be one of the tool names. (TaskRunner pid=2448026) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2448026) (TaskRunner pid=2448026) Begin! (TaskRunner pid=2448026) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2448026) assistant (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) [response] Thought: The user wants to see a random picture of an axolotl. I should use the getRandomAxolotlImage tool to retrieve a random image. (TaskRunner pid=2448026) Action: getRandomAxolotlImage (TaskRunner pid=2448026) Action Input: {} (TaskRunner pid=2448026) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2448026) [score] 1.0 (TaskRunner pid=2448026) [acc] 1.0 (TaskRunner pid=2448026) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2448026) [incorrect_format] 0 (TaskRunner pid=2448026) [feedback] (TaskRunner pid=2448026) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (TaskRunner pid=2448026) step:30 - global_seqlen/min:17424 - global_seqlen/max:25428 - global_seqlen/minmax_diff:8004 - global_seqlen/balanced_min:21117 - global_seqlen/balanced_max:21120 - global_seqlen/mean:21119.0 - actor/entropy:0.174345001578331 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17359286546707153 - training/rollout_probs_diff_mean:0.0036206701770424843 - training/rollout_probs_diff_std:0.011443980038166046 - training/rollout_actor_probs_pearson_corr:0.9984093308448792 - rollout_corr/rollout_is_mean:0.9999167919158936 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6020864248275757 - rollout_corr/rollout_is_min:0.6147533655166626 - rollout_corr/rollout_is_std:0.03235393017530441 - rollout_corr/rollout_is_eff_sample_size:0.9989541988641709 - rollout_corr/rollout_is_seq_mean:0.9999988675117493 - rollout_corr/rollout_is_seq_std:0.0035622739233076572 - rollout_corr/rollout_is_seq_max:1.0128672122955322 - rollout_corr/rollout_is_seq_min:0.9876852035522461 - rollout_corr/rollout_is_seq_max_deviation:0.012867212295532227 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1746694282628596) - rollout_corr/training_log_ppl:np.float64(0.1514518390831654) - rollout_corr/kl:np.float64(0.00017997677664993716) - rollout_corr/k3_kl:np.float64(0.0007583029761732973) - rollout_corr/rollout_ppl:np.float64(1.1743799289688468) - rollout_corr/rollout_log_ppl:np.float64(0.15127186164681916) - rollout_corr/log_ppl_diff:np.float64(0.00017997743634623475) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034040063655993436) - rollout_corr/log_ppl_diff_max:np.float64(0.023737788200378418) - rollout_corr/log_ppl_diff_min:np.float64(-0.015207484364509583) - rollout_corr/ppl_ratio:np.float64(1.0001910293940455) - rollout_corr/chi2_token:np.float64(0.0027057090774178505) - rollout_corr/chi2_seq:np.float64(0.17368023446761072) - actor/pg_loss:np.float64(-0.00022321112919598818) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028334771559457295) - actor/ppo_kl:np.float64(-0.0002888851697377959) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3047818155027926) - perf/mfu/actor:np.float64(0.08610146036849498) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(108.90614318847656) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5790441176470589) - val-aux/tooluse/reward/std@16:np.float64(0.048026239874106626) - val-aux/tooluse/reward/best@2/mean:np.float64(0.59775) - val-aux/tooluse/reward/best@2/std:np.float64(0.04164443441624792) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5596911764705882) - val-aux/tooluse/reward/worst@2/std:np.float64(0.039881518791685694) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5785588235294117) - val-aux/tooluse/reward/maj@2/std:np.float64(0.04813759220654495) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6132352941176471) - val-aux/tooluse/reward/best@4/std:np.float64(0.032251294121326414) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5445735294117648) - val-aux/tooluse/reward/worst@4/std:np.float64(0.02937286523109775) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5798529411764706) - val-aux/tooluse/reward/maj@4/std:np.float64(0.036094533919497906) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6258529411764706) - val-aux/tooluse/reward/best@8/std:np.float64(0.02375543020083673) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5335294117647058) - val-aux/tooluse/reward/worst@8/std:np.float64(0.021254440243975468) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5800441176470588) - val-aux/tooluse/reward/maj@8/std:np.float64(0.024961800826741605) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6359411764705882) - val-aux/tooluse/reward/best@16/std:np.float64(0.017562533701702472) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5251911764705881) - val-aux/tooluse/reward/worst@16/std:np.float64(0.014735369749159885) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5819264705882352) - val-aux/tooluse/reward/maj@16/std:np.float64(0.018366574941199258) - val-aux/tooluse/score/mean@16:np.float64(0.5790441176470589) - val-aux/tooluse/score/std@16:np.float64(0.048026239874106626) - val-aux/tooluse/score/best@2/mean:np.float64(0.59775) - val-aux/tooluse/score/best@2/std:np.float64(0.04164443441624792) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5596911764705882) - val-aux/tooluse/score/worst@2/std:np.float64(0.039881518791685694) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5785588235294117) - val-aux/tooluse/score/maj@2/std:np.float64(0.04813759220654495) - val-aux/tooluse/score/best@4/mean:np.float64(0.6132352941176471) - val-aux/tooluse/score/best@4/std:np.float64(0.032251294121326414) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5445735294117648) - val-aux/tooluse/score/worst@4/std:np.float64(0.02937286523109775) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5798529411764706) - val-aux/tooluse/score/maj@4/std:np.float64(0.036094533919497906) - val-aux/tooluse/score/best@8/mean:np.float64(0.6258529411764706) - val-aux/tooluse/score/best@8/std:np.float64(0.02375543020083673) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5335294117647058) - val-aux/tooluse/score/worst@8/std:np.float64(0.021254440243975468) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5800441176470588) - val-aux/tooluse/score/maj@8/std:np.float64(0.024961800826741605) - val-aux/tooluse/score/best@16/mean:np.float64(0.6359411764705882) - val-aux/tooluse/score/best@16/std:np.float64(0.017562533701702472) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5251911764705881) - val-aux/tooluse/score/worst@16/std:np.float64(0.014735369749159885) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5819264705882352) - val-aux/tooluse/score/maj@16/std:np.float64(0.018366574941199258) - val-core/tooluse/acc/mean@16:np.float64(0.5790441176470589) - val-aux/tooluse/acc/std@16:np.float64(0.048026239874106626) - val-aux/tooluse/acc/best@2/mean:np.float64(0.59775) - val-aux/tooluse/acc/best@2/std:np.float64(0.04164443441624792) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5596911764705882) - val-aux/tooluse/acc/worst@2/std:np.float64(0.039881518791685694) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5785588235294117) - val-aux/tooluse/acc/maj@2/std:np.float64(0.04813759220654495) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6132352941176471) - val-aux/tooluse/acc/best@4/std:np.float64(0.032251294121326414) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5445735294117648) - val-aux/tooluse/acc/worst@4/std:np.float64(0.02937286523109775) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5798529411764706) - val-aux/tooluse/acc/maj@4/std:np.float64(0.036094533919497906) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6258529411764706) - val-aux/tooluse/acc/best@8/std:np.float64(0.02375543020083673) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5335294117647058) - val-aux/tooluse/acc/worst@8/std:np.float64(0.021254440243975468) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5800441176470588) - val-aux/tooluse/acc/maj@8/std:np.float64(0.024961800826741605) - val-core/tooluse/acc/best@16/mean:np.float64(0.6359411764705882) - val-core/tooluse/acc/best@16/std:np.float64(0.017562533701702472) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5251911764705881) - val-aux/tooluse/acc/worst@16/std:np.float64(0.014735369749159885) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5819264705882352) - val-core/tooluse/acc/maj@16/std:np.float64(0.018366574941199258) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00010574243788141757 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.00010574243788141757 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:78.5 - response_length/max:236.0 - response_length/min:31.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:78.5 - response_length_non_aborted/max:236.0 - response_length_non_aborted/min:31.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:581.46875 - prompt_length/max:1128.0 - prompt_length/min:309.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010194815695285797 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5592606477439404) - timing_s/agent_loop/generate_sequences/max:np.float64(2.0239083636552095) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0303889402057393) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.0239083636552095) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:489 - timing_s/agent_loop/slowest/response_length:236 - timing_s/gen:9.051871843636036 - timing_s/reward:0.06795741245150566 - timing_s/old_log_prob:2.994393978267908 - timing_s/adv:0.1275482177734375 - timing_s/update_actor:12.933775318786502 - timing_s/step:25.222951328381896 - timing_s/testing:22.619584698230028 - timing_s/save_checkpoint:12.81557191349566 - timing_s/stop_profile:0.00014170631766319275 - timing_per_token_ms/update_actor:0.07655295775596917 - timing_per_token_ms/gen:0.45043152088157024 - timing_per_token_ms/adv:0.0007549376022387275 - perf/total_num_tokens:168952 - perf/time_per_step:25.222951328381896 - perf/throughput:837.2929767436073 (TaskRunner pid=2448026) Training Progress: 30%|███ | 30/100 [15:39<42:49, 36.70s/it] (TaskRunner pid=2448026) step:31 - global_seqlen/min:17997 - global_seqlen/max:25251 - global_seqlen/minmax_diff:7254 - global_seqlen/balanced_min:21386 - global_seqlen/balanced_max:21389 - global_seqlen/mean:21388.375 - actor/entropy:0.18397483229637146 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.14266479015350342 - training/rollout_probs_diff_mean:0.0036178564187139273 - training/rollout_probs_diff_std:0.011114903725683689 - training/rollout_actor_probs_pearson_corr:0.9985663294792175 - rollout_corr/rollout_is_mean:1.00025475025177 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6546589136123657 - rollout_corr/rollout_is_min:0.6647823452949524 - rollout_corr/rollout_is_std:0.03282942622900009 - rollout_corr/rollout_is_eff_sample_size:0.998923983917394 - rollout_corr/rollout_is_seq_mean:1.0002944469451904 - rollout_corr/rollout_is_seq_std:0.00432634400203824 - rollout_corr/rollout_is_seq_max:1.0189021825790405 - rollout_corr/rollout_is_seq_min:0.988476574420929 - rollout_corr/rollout_is_seq_max_deviation:0.018902182579040527 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.191180984955281) - rollout_corr/training_log_ppl:np.float64(0.16099891092744656) - rollout_corr/kl:np.float64(0.00029437409429178274) - rollout_corr/k3_kl:np.float64(0.0007001811105524225) - rollout_corr/rollout_ppl:np.float64(1.1907468978315592) - rollout_corr/rollout_log_ppl:np.float64(0.16070453578868182) - rollout_corr/log_ppl_diff:np.float64(0.0002943751387647353) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036897044119541533) - rollout_corr/log_ppl_diff_max:np.float64(0.018606245517730713) - rollout_corr/log_ppl_diff_min:np.float64(-0.020777031779289246) - rollout_corr/ppl_ratio:np.float64(1.0003068041987717) - rollout_corr/chi2_token:np.float64(0.002171088242903352) - rollout_corr/chi2_seq:np.float64(0.1989607021678239) - actor/pg_loss:np.float64(1.8237624317407608e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011160714348079637) - actor/ppo_kl:np.float64(9.014108547944488e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1598733365535736) - perf/mfu/actor:np.float64(0.08838498907262658) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(107.38927841186523) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.21484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.21484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013302849838510156 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013302849838510156 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:73.04296875 - response_length/max:291.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:73.04296875 - response_length_non_aborted/max:291.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:595.34375 - prompt_length/max:1091.0 - prompt_length/min:347.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016403570771217346 - timing_s/agent_loop/generate_sequences/min:np.float64(0.445566613227129) - timing_s/agent_loop/generate_sequences/max:np.float64(2.5103322584182024) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0082809509913204) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.5103322584182024) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:291 - timing_s/gen:8.808302007615566 - timing_s/reward:0.06877400167286396 - timing_s/old_log_prob:3.3566870391368866 - timing_s/adv:0.13912721537053585 - timing_s/update_actor:12.77087951079011 - timing_s/step:25.23591827042401 - timing_s/stop_profile:2.711080014705658e-05 - timing_per_token_ms/update_actor:0.07463680334989281 - timing_per_token_ms/gen:0.47105738315501183 - timing_per_token_ms/adv:0.0008131006643242874 - perf/total_num_tokens:171107 - perf/time_per_step:25.23591827042401 - perf/throughput:847.5370212728399 (TaskRunner pid=2448026) Training Progress: 31%|███ | 31/100 [16:05<38:15, 33.27s/it] (TaskRunner pid=2448026) step:32 - global_seqlen/min:15912 - global_seqlen/max:25671 - global_seqlen/minmax_diff:9759 - global_seqlen/balanced_min:20972 - global_seqlen/balanced_max:20974 - global_seqlen/mean:20972.875 - actor/entropy:0.10584317147731781 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29069283604621887 - training/rollout_probs_diff_mean:0.0030529669020324945 - training/rollout_probs_diff_std:0.011499609798192978 - training/rollout_actor_probs_pearson_corr:0.9975054264068604 - rollout_corr/rollout_is_mean:0.9999309778213501 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.494807555805892e-05 - rollout_corr/rollout_is_max:1.7156728506088257 - rollout_corr/rollout_is_min:0.46146276593208313 - rollout_corr/rollout_is_std:0.02996869757771492 - rollout_corr/rollout_is_eff_sample_size:0.999102683113268 - rollout_corr/rollout_is_seq_mean:0.999883234500885 - rollout_corr/rollout_is_seq_std:0.003848048159852624 - rollout_corr/rollout_is_seq_max:1.0096956491470337 - rollout_corr/rollout_is_seq_min:0.988899290561676 - rollout_corr/rollout_is_seq_max_deviation:0.011100709438323975 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1039182115346193) - rollout_corr/training_log_ppl:np.float64(0.09624250339857099) - rollout_corr/kl:np.float64(0.0005379400515481692) - rollout_corr/k3_kl:np.float64(0.0008697444509278007) - rollout_corr/rollout_ppl:np.float64(1.1032186252996325) - rollout_corr/rollout_log_ppl:np.float64(0.09570456246910908) - rollout_corr/log_ppl_diff:np.float64(0.0005379409294619109) - rollout_corr/log_ppl_abs_diff:np.float64(0.003852487786389247) - rollout_corr/log_ppl_diff_max:np.float64(0.05172038823366165) - rollout_corr/log_ppl_diff_min:np.float64(-0.026131346821784973) - rollout_corr/ppl_ratio:np.float64(1.000558052212) - rollout_corr/chi2_token:np.float64(0.0023909322917461395) - rollout_corr/chi2_seq:np.float64(0.16510920808650553) - actor/pg_loss:np.float64(-0.0008980103302747011) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012162026941950899) - actor/ppo_kl:np.float64(-3.784106627335859e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.41867247223854065) - perf/mfu/actor:np.float64(0.0865513851349042) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.2931752204895) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018407604657113552 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:0.0018407604657113552 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:71.08984375 - response_length/max:180.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:71.08984375 - response_length_non_aborted/max:180.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:584.3125 - prompt_length/max:981.0 - prompt_length/min:249.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.2201409339904785e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3350175488740206) - timing_s/agent_loop/generate_sequences/max:np.float64(1.6965950597077608) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9682859796230332) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.6965950597077608) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:714 - timing_s/agent_loop/slowest/response_length:161 - timing_s/gen:8.46879718825221 - timing_s/reward:0.06800480000674725 - timing_s/old_log_prob:3.0378013607114553 - timing_s/adv:0.1491863038390875 - timing_s/update_actor:12.706043066456914 - timing_s/step:24.517206877470016 - timing_s/stop_profile:6.631389260292053e-05 - timing_per_token_ms/update_actor:0.07572902538670136 - timing_per_token_ms/gen:0.4653440951839228 - timing_per_token_ms/adv:0.0008891622145216589 - perf/total_num_tokens:167783 - perf/time_per_step:24.517206877470016 - perf/throughput:855.4349239216533 (TaskRunner pid=2448026) Training Progress: 32%|███▏ | 32/100 [16:29<34:44, 30.65s/it] (TaskRunner pid=2448026) step:33 - global_seqlen/min:18013 - global_seqlen/max:24015 - global_seqlen/minmax_diff:6002 - global_seqlen/balanced_min:21339 - global_seqlen/balanced_max:21341 - global_seqlen/mean:21340.0 - actor/entropy:0.1172524020075798 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28219112753868103 - training/rollout_probs_diff_mean:0.0032827816903591156 - training/rollout_probs_diff_std:0.01186556275933981 - training/rollout_actor_probs_pearson_corr:0.9976831674575806 - rollout_corr/rollout_is_mean:0.9998560547828674 - rollout_corr/rollout_is_ratio_fraction_high:5.8768218877958134e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.8768218877958134e-05 - rollout_corr/rollout_is_max:2.075263738632202 - rollout_corr/rollout_is_min:0.25255388021469116 - rollout_corr/rollout_is_std:0.03248446807265282 - rollout_corr/rollout_is_eff_sample_size:0.9989455148537124 - rollout_corr/rollout_is_seq_mean:0.9997761249542236 - rollout_corr/rollout_is_seq_std:0.004250098019838333 - rollout_corr/rollout_is_seq_max:1.0144253969192505 - rollout_corr/rollout_is_seq_min:0.9874351024627686 - rollout_corr/rollout_is_seq_max_deviation:0.014425396919250488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.118108035530895) - rollout_corr/training_log_ppl:np.float64(0.10714274069505336) - rollout_corr/kl:np.float64(0.0009456591953025395) - rollout_corr/k3_kl:np.float64(0.0014957072714461006) - rollout_corr/rollout_ppl:np.float64(1.1169259711168706) - rollout_corr/rollout_log_ppl:np.float64(0.10619708061676647) - rollout_corr/log_ppl_diff:np.float64(0.0009456600782868918) - rollout_corr/log_ppl_abs_diff:np.float64(0.00484006918486557) - rollout_corr/log_ppl_diff_max:np.float64(0.05556429550051689) - rollout_corr/log_ppl_diff_min:np.float64(-0.031781356781721115) - rollout_corr/ppl_ratio:np.float64(1.0009749322198331) - rollout_corr/chi2_token:np.float64(0.004619225161150098) - rollout_corr/chi2_seq:np.float64(0.24745542067103088) - actor/pg_loss:np.float64(-0.00022274814546108246) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00066862295352621) - actor/ppo_kl:np.float64(0.00020936724136788598) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25179412215948105) - perf/mfu/actor:np.float64(0.08819196554294478) - perf/max_memory_allocated_gb:np.float64(128.6631588935852) - perf/max_memory_reserved_gb:np.float64(136.9375) - perf/cpu_memory_used_gb:np.float64(106.15742111206055) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.3984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01074723806232214 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:-0.01074723806232214 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:66.46875 - response_length/max:225.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:66.46875 - response_length_non_aborted/max:225.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:600.40625 - prompt_length/max:880.0 - prompt_length/min:247.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010926090180873871 - timing_s/agent_loop/generate_sequences/min:np.float64(0.504535797983408) - timing_s/agent_loop/generate_sequences/max:np.float64(1.918188774958253) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9498112941146246) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.918188774958253) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:705 - timing_s/agent_loop/slowest/response_length:225 - timing_s/gen:8.624560801312327 - timing_s/reward:0.0687526036053896 - timing_s/old_log_prob:3.0374930519610643 - timing_s/adv:0.13458062335848808 - timing_s/update_actor:12.59742221236229 - timing_s/step:24.5502154417336 - timing_s/stop_profile:0.0001124553382396698 - timing_per_token_ms/update_actor:0.07378996141261884 - timing_per_token_ms/gen:0.5068500705989849 - timing_per_token_ms/adv:0.0007883119924934869 - perf/total_num_tokens:170720 - perf/time_per_step:24.5502154417336 - perf/throughput:869.2388077264502 (TaskRunner pid=2448026) Training Progress: 33%|███▎ | 33/100 [16:54<32:11, 28.84s/it] (TaskRunner pid=2448026) step:34 - global_seqlen/min:16502 - global_seqlen/max:31010 - global_seqlen/minmax_diff:14508 - global_seqlen/balanced_min:21501 - global_seqlen/balanced_max:29293 - global_seqlen/mean:22475.75 - actor/entropy:0.06783878803253174 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22811102867126465 - training/rollout_probs_diff_mean:0.0018746056593954563 - training/rollout_probs_diff_std:0.009012462571263313 - training/rollout_actor_probs_pearson_corr:0.9977815747261047 - rollout_corr/rollout_is_mean:0.999635636806488 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.646081805229187 - rollout_corr/rollout_is_min:0.6306154131889343 - rollout_corr/rollout_is_std:0.024653367698192596 - rollout_corr/rollout_is_eff_sample_size:0.9993919257653825 - rollout_corr/rollout_is_seq_mean:0.9995258450508118 - rollout_corr/rollout_is_seq_std:0.0034325544256716967 - rollout_corr/rollout_is_seq_max:1.0116262435913086 - rollout_corr/rollout_is_seq_min:0.9900691509246826 - rollout_corr/rollout_is_seq_max_deviation:0.011626243591308594 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0971969454549253) - rollout_corr/training_log_ppl:np.float64(0.08819869507988187) - rollout_corr/kl:np.float64(0.00044117882201710756) - rollout_corr/k3_kl:np.float64(0.0006027336617609735) - rollout_corr/rollout_ppl:np.float64(1.0966038638725877) - rollout_corr/rollout_log_ppl:np.float64(0.0877575159920525) - rollout_corr/log_ppl_diff:np.float64(0.00044117908782936865) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029929660786365275) - rollout_corr/log_ppl_diff_max:np.float64(0.020411327481269836) - rollout_corr/log_ppl_diff_min:np.float64(-0.01272192969918251) - rollout_corr/ppl_ratio:np.float64(1.0004515319596976) - rollout_corr/chi2_token:np.float64(0.0014595773536711931) - rollout_corr/chi2_seq:np.float64(0.0659017851576209) - actor/pg_loss:np.float64(-0.00015708047430962324) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006597699975827709) - actor/ppo_kl:np.float64(-0.0004642985307334868) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.49082714691758156) - perf/mfu/actor:np.float64(0.08583402318488069) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(106.07360076904297) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0014964197762310505 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0014964197762310505 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:104.7421875 - response_length/max:8192.0 - response_length/min:23.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:104.7421875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:597.625 - prompt_length/max:862.0 - prompt_length/min:355.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016065873205661774 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4019253086298704) - timing_s/agent_loop/generate_sequences/max:np.float64(54.201869091019034) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.122731490082515) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(54.201869091019034) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:786 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:61.01498405262828 - timing_s/reward:0.07069508358836174 - timing_s/old_log_prob:3.3404365740716457 - timing_s/adv:0.10997999645769596 - timing_s/update_actor:13.697218215093017 - timing_s/step:78.3180810958147 - timing_s/stop_profile:3.905780613422394e-05 - timing_per_token_ms/update_actor:0.07617775944680943 - timing_per_token_ms/gen:2.2754898207141148 - timing_per_token_ms/adv:0.0006116592130279077 - perf/total_num_tokens:179806 - perf/time_per_step:78.3180810958147 - perf/throughput:286.9803458604031 (TaskRunner pid=2448026) Training Progress: 34%|███▍ | 34/100 [18:12<48:04, 43.70s/it] (TaskRunner pid=2448026) step:35 - global_seqlen/min:16659 - global_seqlen/max:27816 - global_seqlen/minmax_diff:11157 - global_seqlen/balanced_min:21266 - global_seqlen/balanced_max:21267 - global_seqlen/mean:21266.75 - actor/entropy:0.10790044069290161 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24962079524993896 - training/rollout_probs_diff_mean:0.0031485510990023613 - training/rollout_probs_diff_std:0.012108963914215565 - training/rollout_actor_probs_pearson_corr:0.9974215030670166 - rollout_corr/rollout_is_mean:0.9998846650123596 - rollout_corr/rollout_is_ratio_fraction_high:6.173602014314383e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.010491132736206 - rollout_corr/rollout_is_min:0.5510239601135254 - rollout_corr/rollout_is_std:0.035796377807855606 - rollout_corr/rollout_is_eff_sample_size:0.9987199024687595 - rollout_corr/rollout_is_seq_mean:0.9997034668922424 - rollout_corr/rollout_is_seq_std:0.004741272423416376 - rollout_corr/rollout_is_seq_max:1.0133460760116577 - rollout_corr/rollout_is_seq_min:0.9850015044212341 - rollout_corr/rollout_is_seq_max_deviation:0.01499849557876587 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1047534057870507) - rollout_corr/training_log_ppl:np.float64(0.09583136365654354) - rollout_corr/kl:np.float64(0.0019409403973020822) - rollout_corr/k3_kl:np.float64(0.0021721900752142117) - rollout_corr/rollout_ppl:np.float64(1.1024981015361845) - rollout_corr/rollout_log_ppl:np.float64(0.0938904222457495) - rollout_corr/log_ppl_diff:np.float64(0.0019409414089750499) - rollout_corr/log_ppl_abs_diff:np.float64(0.00557866579038091) - rollout_corr/log_ppl_diff_max:np.float64(0.0994080901145935) - rollout_corr/log_ppl_diff_min:np.float64(-0.027030080556869507) - rollout_corr/ppl_ratio:np.float64(1.0020217958372086) - rollout_corr/chi2_token:np.float64(0.0037718545645475388) - rollout_corr/chi2_seq:np.float64(0.18796940613538027) - actor/pg_loss:np.float64(8.712778799235821e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003900952015101211) - actor/ppo_kl:np.float64(0.0010225941209291989) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5841034082695842) - perf/mfu/actor:np.float64(0.08722799490918402) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(106.18114471435547) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0050469194538891315 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0050469194538891315 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:63.2734375 - response_length/max:168.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:63.2734375 - response_length_non_aborted/max:168.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:601.3125 - prompt_length/max:1518.0 - prompt_length/min:310.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.7131052017211914e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5234723575413227) - timing_s/agent_loop/generate_sequences/max:np.float64(1.6012352276593447) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9316267384274397) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.6012352276593447) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:698 - timing_s/agent_loop/slowest/response_length:168 - timing_s/gen:8.622623534873128 - timing_s/reward:0.07016462832689285 - timing_s/old_log_prob:3.0672460328787565 - timing_s/adv:0.14836185239255428 - timing_s/update_actor:12.907170491293073 - timing_s/step:24.904948886483908 - timing_s/stop_profile:0.00013680383563041687 - timing_per_token_ms/update_actor:0.07586473304156179 - timing_per_token_ms/gen:0.5323264313417168 - timing_per_token_ms/adv:0.0008720294144177783 - perf/total_num_tokens:170134 - perf/time_per_step:24.904948886483908 - perf/throughput:853.9166290576736 (TaskRunner pid=2448026) Training Progress: 35%|███▌ | 35/100 [18:37<41:14, 38.07s/it] (TaskRunner pid=2448026) step:36 - global_seqlen/min:18659 - global_seqlen/max:31770 - global_seqlen/minmax_diff:13111 - global_seqlen/balanced_min:23313 - global_seqlen/balanced_max:23316 - global_seqlen/mean:23315.125 - actor/entropy:0.1094154641032219 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016221523284912 - training/rollout_probs_diff_mean:0.002789558842778206 - training/rollout_probs_diff_std:0.011223123408854008 - training/rollout_actor_probs_pearson_corr:0.9977975487709045 - rollout_corr/rollout_is_mean:1.000141978263855 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.760192632675171 - rollout_corr/rollout_is_min:0.5040825605392456 - rollout_corr/rollout_is_std:0.030972199514508247 - rollout_corr/rollout_is_eff_sample_size:0.9990419991206019 - rollout_corr/rollout_is_seq_mean:1.0001487731933594 - rollout_corr/rollout_is_seq_std:0.004240594804286957 - rollout_corr/rollout_is_seq_max:1.0191733837127686 - rollout_corr/rollout_is_seq_min:0.9870032072067261 - rollout_corr/rollout_is_seq_max_deviation:0.019173383712768555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1061704652383924) - rollout_corr/training_log_ppl:np.float64(0.0955705170508736) - rollout_corr/kl:np.float64(0.000832762136276699) - rollout_corr/k3_kl:np.float64(0.0012156391768091482) - rollout_corr/rollout_ppl:np.float64(1.1050650174729526) - rollout_corr/rollout_log_ppl:np.float64(0.09473775420610764) - rollout_corr/log_ppl_diff:np.float64(0.0008327628447659663) - rollout_corr/log_ppl_abs_diff:np.float64(0.004004080658887688) - rollout_corr/log_ppl_diff_max:np.float64(0.054225899279117584) - rollout_corr/log_ppl_diff_min:np.float64(-0.05098208039999008) - rollout_corr/ppl_ratio:np.float64(1.0008584272582084) - rollout_corr/chi2_token:np.float64(0.005106565775349736) - rollout_corr/chi2_seq:np.float64(0.18091726955026388) - actor/pg_loss:np.float64(7.728044874966145e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011160714348079637) - actor/ppo_kl:np.float64(0.0004939473388176907) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.027728205546736717) - perf/mfu/actor:np.float64(0.09567257544434149) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(106.17373275756836) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.28515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:0.0 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:78.53515625 - response_length/max:557.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:78.53515625 - response_length_non_aborted/max:557.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:650.0625 - prompt_length/max:1250.0 - prompt_length/min:252.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015104562044143677 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5704701710492373) - timing_s/agent_loop/generate_sequences/max:np.float64(3.8988978303968906) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0769493748666719) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.8988978303968906) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:654 - timing_s/agent_loop/slowest/response_length:557 - timing_s/gen:10.752700073644519 - timing_s/reward:0.07211077399551868 - timing_s/old_log_prob:3.027080912142992 - timing_s/adv:0.12379146181046963 - timing_s/update_actor:12.903666820377111 - timing_s/step:26.96331188827753 - timing_s/stop_profile:0.00011024624109268188 - timing_per_token_ms/update_actor:0.06918077224750624 - timing_per_token_ms/gen:0.5348271610865217 - timing_per_token_ms/adv:0.0006636864578812553 - perf/total_num_tokens:186521 - perf/time_per_step:26.96331188827753 - perf/throughput:864.698116336977 (TaskRunner pid=2448026) Training Progress: 36%|███▌ | 36/100 [19:04<37:04, 34.76s/it] (TaskRunner pid=2448026) step:37 - global_seqlen/min:15938 - global_seqlen/max:25481 - global_seqlen/minmax_diff:9543 - global_seqlen/balanced_min:21031 - global_seqlen/balanced_max:21033 - global_seqlen/mean:21031.875 - actor/entropy:0.08763600140810013 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917799949645996 - training/rollout_probs_diff_mean:0.0025775693356990814 - training/rollout_probs_diff_std:0.010631529614329338 - training/rollout_actor_probs_pearson_corr:0.9974813461303711 - rollout_corr/rollout_is_mean:0.9998788833618164 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.148927059257403e-05 - rollout_corr/rollout_is_max:1.6584875583648682 - rollout_corr/rollout_is_min:0.39147502183914185 - rollout_corr/rollout_is_std:0.028494464233517647 - rollout_corr/rollout_is_eff_sample_size:0.999188605215701 - rollout_corr/rollout_is_seq_mean:0.9998454451560974 - rollout_corr/rollout_is_seq_std:0.0037512846756726503 - rollout_corr/rollout_is_seq_max:1.0161806344985962 - rollout_corr/rollout_is_seq_min:0.9882006049156189 - rollout_corr/rollout_is_seq_max_deviation:0.01618063449859619 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.083981916308403) - rollout_corr/training_log_ppl:np.float64(0.07810831124834294) - rollout_corr/kl:np.float64(0.0008820643163085506) - rollout_corr/k3_kl:np.float64(0.000904175102244836) - rollout_corr/rollout_ppl:np.float64(1.0829150243662298) - rollout_corr/rollout_log_ppl:np.float64(0.07722624591269778) - rollout_corr/log_ppl_diff:np.float64(0.0008820653356451658) - rollout_corr/log_ppl_abs_diff:np.float64(0.004066320769197773) - rollout_corr/log_ppl_diff_max:np.float64(0.02512640506029129) - rollout_corr/log_ppl_diff_min:np.float64(-0.021203629672527313) - rollout_corr/ppl_ratio:np.float64(1.000900134909898) - rollout_corr/chi2_token:np.float64(0.0018352963961660862) - rollout_corr/chi2_seq:np.float64(0.09340445604175329) - actor/pg_loss:np.float64(3.957573790103197e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0003062304746066502) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29184149391949177) - perf/mfu/actor:np.float64(0.08713190173492676) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(106.23775863647461) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-2.305847556272056e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-2.305847556272056e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:63.52734375 - response_length/max:139.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:63.52734375 - response_length_non_aborted/max:139.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:593.71875 - prompt_length/max:1108.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014193542301654816 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4855928272008896) - timing_s/agent_loop/generate_sequences/max:np.float64(1.40391980484128) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.921188939762942) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.40391980484128) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:490 - timing_s/agent_loop/slowest/response_length:139 - timing_s/gen:8.134130844846368 - timing_s/reward:0.06617194041609764 - timing_s/old_log_prob:2.9760467894375324 - timing_s/adv:0.12074950523674488 - timing_s/update_actor:12.541396103799343 - timing_s/step:23.9208531267941 - timing_s/stop_profile:3.744475543498993e-05 - timing_per_token_ms/update_actor:0.07453802920447739 - timing_per_token_ms/gen:0.5001617687294083 - timing_per_token_ms/adv:0.0007176577530340547 - perf/total_num_tokens:168255 - perf/time_per_step:23.9208531267941 - perf/throughput:879.2276299059705 (TaskRunner pid=2448026) Training Progress: 37%|███▋ | 37/100 [19:28<33:05, 31.51s/it] (TaskRunner pid=2448026) step:38 - global_seqlen/min:14200 - global_seqlen/max:26016 - global_seqlen/minmax_diff:11816 - global_seqlen/balanced_min:20867 - global_seqlen/balanced_max:20871 - global_seqlen/mean:20868.75 - actor/entropy:0.07100503891706467 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28970569372177124 - training/rollout_probs_diff_mean:0.0024669228587299585 - training/rollout_probs_diff_std:0.011315261013805866 - training/rollout_actor_probs_pearson_corr:0.9967005848884583 - rollout_corr/rollout_is_mean:0.9998286366462708 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.378364923875779e-05 - rollout_corr/rollout_is_max:1.6454116106033325 - rollout_corr/rollout_is_min:0.4618375897407532 - rollout_corr/rollout_is_std:0.02947334572672844 - rollout_corr/rollout_is_eff_sample_size:0.9991318378488921 - rollout_corr/rollout_is_seq_mean:0.9996922612190247 - rollout_corr/rollout_is_seq_std:0.0041647180914878845 - rollout_corr/rollout_is_seq_max:1.015931487083435 - rollout_corr/rollout_is_seq_min:0.9851982593536377 - rollout_corr/rollout_is_seq_max_deviation:0.01593148708343506 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.070449029095471) - rollout_corr/training_log_ppl:np.float64(0.06630817761470098) - rollout_corr/kl:np.float64(0.0006508337201012182) - rollout_corr/k3_kl:np.float64(0.0008507462097420999) - rollout_corr/rollout_ppl:np.float64(1.0697002364322543) - rollout_corr/rollout_log_ppl:np.float64(0.06565734352625441) - rollout_corr/log_ppl_diff:np.float64(0.0006508340884465724) - rollout_corr/log_ppl_abs_diff:np.float64(0.004023703037091764) - rollout_corr/log_ppl_diff_max:np.float64(0.023519545793533325) - rollout_corr/log_ppl_diff_min:np.float64(-0.017418062314391136) - rollout_corr/ppl_ratio:np.float64(1.0006675054319203) - rollout_corr/chi2_token:np.float64(0.0021650574635714293) - rollout_corr/chi2_seq:np.float64(0.1257851216942072) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(-0.00011679511070594728) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.08632270415665823) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(106.21752548217773) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:61.2421875 - response_length/max:272.0 - response_length/min:28.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:61.2421875 - response_length_non_aborted/max:272.0 - response_length_non_aborted/min:28.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:590.90625 - prompt_length/max:1166.0 - prompt_length/min:322.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.5140663385391235e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5159701481461525) - timing_s/agent_loop/generate_sequences/max:np.float64(2.143700459972024) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8794434285300667) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.143700459972024) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:758 - timing_s/agent_loop/slowest/response_length:272 - timing_s/gen:8.977025628089905 - timing_s/reward:0.06115909293293953 - timing_s/old_log_prob:2.98725507222116 - timing_s/adv:0.13001164980232716 - timing_s/update_actor:12.725646145641804 - timing_s/step:24.9925625808537 - timing_s/stop_profile:0.00011251494288444519 - timing_per_token_ms/update_actor:0.07622429557137948 - timing_per_token_ms/gen:0.5725874236567103 - timing_per_token_ms/adv:0.0007787460305620075 - perf/total_num_tokens:166950 - perf/time_per_step:24.9925625808537 - perf/throughput:834.9984093262661 (TaskRunner pid=2448026) Training Progress: 38%|███▊ | 38/100 [19:53<30:33, 29.57s/it] (TaskRunner pid=2448026) step:39 - global_seqlen/min:17415 - global_seqlen/max:27039 - global_seqlen/minmax_diff:9624 - global_seqlen/balanced_min:21493 - global_seqlen/balanced_max:21494 - global_seqlen/mean:21493.375 - actor/entropy:0.0934237614274025 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12288567423820496 - training/rollout_probs_diff_mean:0.00283451727591455 - training/rollout_probs_diff_std:0.010742252692580223 - training/rollout_actor_probs_pearson_corr:0.9977567195892334 - rollout_corr/rollout_is_mean:0.9998695254325867 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5471926927566528 - rollout_corr/rollout_is_min:0.5369587540626526 - rollout_corr/rollout_is_std:0.03140411525964737 - rollout_corr/rollout_is_eff_sample_size:0.99901439629272 - rollout_corr/rollout_is_seq_mean:0.9998050332069397 - rollout_corr/rollout_is_seq_std:0.004093321971595287 - rollout_corr/rollout_is_seq_max:1.0182945728302002 - rollout_corr/rollout_is_seq_min:0.9779058694839478 - rollout_corr/rollout_is_seq_max_deviation:0.022094130516052246 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0901494706049562) - rollout_corr/training_log_ppl:np.float64(0.08150277883214585) - rollout_corr/kl:np.float64(0.0002915315222065118) - rollout_corr/k3_kl:np.float64(0.0006384069817053401) - rollout_corr/rollout_ppl:np.float64(1.0896249385550618) - rollout_corr/rollout_log_ppl:np.float64(0.08121124641183997) - rollout_corr/log_ppl_diff:np.float64(0.00029153242030588444) - rollout_corr/log_ppl_abs_diff:np.float64(0.00334439935068076) - rollout_corr/log_ppl_diff_max:np.float64(0.024966895580291748) - rollout_corr/log_ppl_diff_min:np.float64(-0.019420363008975983) - rollout_corr/ppl_ratio:np.float64(1.0003041836898774) - rollout_corr/chi2_token:np.float64(0.0021346465218812227) - rollout_corr/chi2_seq:np.float64(0.07660992699675262) - actor/pg_loss:np.float64(-1.4666235074400902e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(-0.00035314361426763696) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2873027138411999) - perf/mfu/actor:np.float64(0.08932541978859179) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(106.16445541381836) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015977019211277366 - critic/advantages/max:0.25 - critic/advantages/min:-0.75 - critic/returns/mean:0.0015977019211277366 - critic/returns/max:0.25 - critic/returns/min:-0.75 - response_length/mean:60.51171875 - response_length/max:234.0 - response_length/min:10.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:60.51171875 - response_length_non_aborted/max:234.0 - response_length_non_aborted/min:10.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:611.15625 - prompt_length/max:970.0 - prompt_length/min:310.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010489486157894135 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3230640906840563) - timing_s/agent_loop/generate_sequences/max:np.float64(1.9211114905774593) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8448473537137033) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.9211114905774593) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:793 - timing_s/agent_loop/slowest/response_length:234 - timing_s/gen:8.729696227237582 - timing_s/reward:0.06448563002049923 - timing_s/old_log_prob:3.152145266532898 - timing_s/adv:0.13136174716055393 - timing_s/update_actor:12.75266346335411 - timing_s/step:24.92933896742761 - timing_s/stop_profile:0.00012088008224964142 - timing_per_token_ms/update_actor:0.07416624578128209 - timing_per_token_ms/gen:0.5635334211631 - timing_per_token_ms/adv:0.0007639664964236301 - perf/total_num_tokens:171947 - perf/time_per_step:24.92933896742761 - perf/throughput:862.1718782067586 (TaskRunner pid=2448026) Training Progress: 39%|███▉ | 39/100 [20:18<28:39, 28.19s/it] (TaskRunner pid=2448026) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2448026) validation generation end (TaskRunner pid=2448026) [prompt] user (TaskRunner pid=2448026) Your task is to answer the user's question using available tools. (TaskRunner pid=2448026) You have access to the following tools: (TaskRunner pid=2448026) Name: Axolotl (TaskRunner pid=2448026) Description: Collection of axolotl pictures and facts (TaskRunner pid=2448026) Documentation: (TaskRunner pid=2448026) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2448026) Parameters: {} (TaskRunner pid=2448026) Output: Successful response. (TaskRunner pid=2448026) - Format: application/json (TaskRunner pid=2448026) - Structure: Object{url, source, description} (TaskRunner pid=2448026) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2448026) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2448026) Output: Successful response. (TaskRunner pid=2448026) - Format: application/json (TaskRunner pid=2448026) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2448026) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2448026) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2448026) Output: Successful response. (TaskRunner pid=2448026) - Format: application/json (TaskRunner pid=2448026) - Structure: Array[Object{fact, source}] (TaskRunner pid=2448026) (TaskRunner pid=2448026) Use the following format: (TaskRunner pid=2448026) Thought: you should always think about what to do (TaskRunner pid=2448026) Action: the action to take, should be one of the tool names. (TaskRunner pid=2448026) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2448026) (TaskRunner pid=2448026) Begin! (TaskRunner pid=2448026) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2448026) assistant (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) (TaskRunner pid=2448026) [response] Thought: I need to retrieve a random axolotl image using the getRandomAxolotlImage tool. (TaskRunner pid=2448026) Action: getRandomAxolotlImage (TaskRunner pid=2448026) Action Input: {} (TaskRunner pid=2448026) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2448026) [score] 1.0 (TaskRunner pid=2448026) [acc] 1.0 (TaskRunner pid=2448026) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2448026) [incorrect_format] 0 (TaskRunner pid=2448026) [feedback] (TaskRunner pid=2448026) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2448026) step:40 - global_seqlen/min:14087 - global_seqlen/max:23671 - global_seqlen/minmax_diff:9584 - global_seqlen/balanced_min:20377 - global_seqlen/balanced_max:20379 - global_seqlen/mean:20378.0 - actor/entropy:0.06952592730522156 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16887664794921875 - training/rollout_probs_diff_mean:0.0022410580422729254 - training/rollout_probs_diff_std:0.010149526409804821 - training/rollout_actor_probs_pearson_corr:0.9972644448280334 - rollout_corr/rollout_is_mean:0.99969083070755 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.275100167840719e-05 - rollout_corr/rollout_is_max:1.6279504299163818 - rollout_corr/rollout_is_min:0.4736679792404175 - rollout_corr/rollout_is_std:0.027404718101024628 - rollout_corr/rollout_is_eff_sample_size:0.9992489499295585 - rollout_corr/rollout_is_seq_mean:0.9998679161071777 - rollout_corr/rollout_is_seq_std:0.004296263214200735 - rollout_corr/rollout_is_seq_max:1.014003038406372 - rollout_corr/rollout_is_seq_min:0.9804351925849915 - rollout_corr/rollout_is_seq_max_deviation:0.019564807415008545 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.067157625220716) - rollout_corr/training_log_ppl:np.float64(0.06211211615527645) - rollout_corr/kl:np.float64(0.0008139920691609959) - rollout_corr/k3_kl:np.float64(0.0006489804700305776) - rollout_corr/rollout_ppl:np.float64(1.066216129809618) - rollout_corr/rollout_log_ppl:np.float64(0.06129812395420231) - rollout_corr/log_ppl_diff:np.float64(0.0008139922010741429) - rollout_corr/log_ppl_abs_diff:np.float64(0.003510570852085948) - rollout_corr/log_ppl_diff_max:np.float64(0.02842288464307785) - rollout_corr/log_ppl_diff_min:np.float64(-0.01927696168422699) - rollout_corr/ppl_ratio:np.float64(1.000829858938232) - rollout_corr/chi2_token:np.float64(0.0009073852561414242) - rollout_corr/chi2_seq:np.float64(0.0720114151481539) - actor/pg_loss:np.float64(2.5476678274571896e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.00026795594617112783) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15786697901785374) - perf/mfu/actor:np.float64(0.08419554150323741) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(106.29495429992676) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5873161764705882) - val-aux/tooluse/reward/std@16:np.float64(0.021910529311605646) - val-aux/tooluse/reward/best@2/mean:np.float64(0.5942352941176471) - val-aux/tooluse/reward/best@2/std:np.float64(0.02086931120874627) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5805147058823529) - val-aux/tooluse/reward/worst@2/std:np.float64(0.015381239455687805) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.587) - val-aux/tooluse/reward/maj@2/std:np.float64(0.021694259411716987) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6008235294117648) - val-aux/tooluse/reward/best@4/std:np.float64(0.020572611678434754) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5749705882352941) - val-aux/tooluse/reward/worst@4/std:np.float64(0.012908827778193466) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5866911764705882) - val-aux/tooluse/reward/maj@4/std:np.float64(0.013814010301030237) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6094705882352942) - val-aux/tooluse/reward/best@8/std:np.float64(0.021449381508057596) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5687941176470588) - val-aux/tooluse/reward/worst@8/std:np.float64(0.011446718289705856) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5873529411764705) - val-aux/tooluse/reward/maj@8/std:np.float64(0.005259737778438937) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6202794117647059) - val-aux/tooluse/reward/best@16/std:np.float64(0.018664088571891148) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5639852941176471) - val-aux/tooluse/reward/worst@16/std:np.float64(0.008291113546095406) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5881323529411765) - val-aux/tooluse/reward/maj@16/std:np.float64(0.001226068491005571) - val-aux/tooluse/score/mean@16:np.float64(0.5873161764705882) - val-aux/tooluse/score/std@16:np.float64(0.021910529311605646) - val-aux/tooluse/score/best@2/mean:np.float64(0.5942352941176471) - val-aux/tooluse/score/best@2/std:np.float64(0.02086931120874627) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5805147058823529) - val-aux/tooluse/score/worst@2/std:np.float64(0.015381239455687805) - val-aux/tooluse/score/maj@2/mean:np.float64(0.587) - val-aux/tooluse/score/maj@2/std:np.float64(0.021694259411716987) - val-aux/tooluse/score/best@4/mean:np.float64(0.6008235294117648) - val-aux/tooluse/score/best@4/std:np.float64(0.020572611678434754) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5749705882352941) - val-aux/tooluse/score/worst@4/std:np.float64(0.012908827778193466) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5866911764705882) - val-aux/tooluse/score/maj@4/std:np.float64(0.013814010301030237) - val-aux/tooluse/score/best@8/mean:np.float64(0.6094705882352942) - val-aux/tooluse/score/best@8/std:np.float64(0.021449381508057596) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5687941176470588) - val-aux/tooluse/score/worst@8/std:np.float64(0.011446718289705856) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5873529411764705) - val-aux/tooluse/score/maj@8/std:np.float64(0.005259737778438937) - val-aux/tooluse/score/best@16/mean:np.float64(0.6202794117647059) - val-aux/tooluse/score/best@16/std:np.float64(0.018664088571891148) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5639852941176471) - val-aux/tooluse/score/worst@16/std:np.float64(0.008291113546095406) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5881323529411765) - val-aux/tooluse/score/maj@16/std:np.float64(0.001226068491005571) - val-core/tooluse/acc/mean@16:np.float64(0.5873161764705882) - val-aux/tooluse/acc/std@16:np.float64(0.021910529311605646) - val-aux/tooluse/acc/best@2/mean:np.float64(0.5942352941176471) - val-aux/tooluse/acc/best@2/std:np.float64(0.02086931120874627) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5805147058823529) - val-aux/tooluse/acc/worst@2/std:np.float64(0.015381239455687805) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.587) - val-aux/tooluse/acc/maj@2/std:np.float64(0.021694259411716987) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6008235294117648) - val-aux/tooluse/acc/best@4/std:np.float64(0.020572611678434754) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5749705882352941) - val-aux/tooluse/acc/worst@4/std:np.float64(0.012908827778193466) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5866911764705882) - val-aux/tooluse/acc/maj@4/std:np.float64(0.013814010301030237) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6094705882352942) - val-aux/tooluse/acc/best@8/std:np.float64(0.021449381508057596) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5687941176470588) - val-aux/tooluse/acc/worst@8/std:np.float64(0.011446718289705856) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5873529411764705) - val-aux/tooluse/acc/maj@8/std:np.float64(0.005259737778438937) - val-core/tooluse/acc/best@16/mean:np.float64(0.6202794117647059) - val-core/tooluse/acc/best@16/std:np.float64(0.018664088571891148) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5639852941176471) - val-aux/tooluse/acc/worst@16/std:np.float64(0.008291113546095406) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5881323529411765) - val-core/tooluse/acc/maj@16/std:np.float64(0.001226068491005571) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.33984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003019892144948244 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.003019892144948244 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:62.25 - response_length/max:293.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:62.25 - response_length_non_aborted/max:293.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:574.5625 - prompt_length/max:972.0 - prompt_length/min:256.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011559948325157166 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4116268940269947) - timing_s/agent_loop/generate_sequences/max:np.float64(2.187937757000327) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8542907030278002) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.187937757000327) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:537 - timing_s/agent_loop/slowest/response_length:293 - timing_s/gen:8.85091657564044 - timing_s/reward:0.06740262545645237 - timing_s/old_log_prob:2.8993595764040947 - timing_s/adv:0.14931634441018105 - timing_s/update_actor:12.851843036711216 - timing_s/step:24.906155655160546 - timing_s/testing:22.440025260671973 - timing_s/save_checkpoint:12.827179295942187 - timing_s/stop_profile:0.00014885887503623962 - timing_per_token_ms/update_actor:0.07883405533363931 - timing_per_token_ms/gen:0.555403901583863 - timing_per_token_ms/adv:0.0009159163338537948 - perf/total_num_tokens:163024 - perf/time_per_step:24.906155655160546 - perf/throughput:818.1913050791396 (TaskRunner pid=2448026) Training Progress: 40%|████ | 40/100 [21:18<37:48, 37.81s/it] (TaskRunner pid=2448026) step:41 - global_seqlen/min:18058 - global_seqlen/max:26320 - global_seqlen/minmax_diff:8262 - global_seqlen/balanced_min:22144 - global_seqlen/balanced_max:22268 - global_seqlen/mean:22176.125 - actor/entropy:0.07170713692903519 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3533320128917694 - training/rollout_probs_diff_mean:0.002316119149327278 - training/rollout_probs_diff_std:0.011487589217722416 - training/rollout_actor_probs_pearson_corr:0.9967785477638245 - rollout_corr/rollout_is_mean:1.0001319646835327 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012715366028714925 - rollout_corr/rollout_is_max:1.9126262664794922 - rollout_corr/rollout_is_min:0.43207812309265137 - rollout_corr/rollout_is_std:0.03275854140520096 - rollout_corr/rollout_is_eff_sample_size:0.9989285041474916 - rollout_corr/rollout_is_seq_mean:1.000069260597229 - rollout_corr/rollout_is_seq_std:0.004516740795224905 - rollout_corr/rollout_is_seq_max:1.0154341459274292 - rollout_corr/rollout_is_seq_min:0.9815228581428528 - rollout_corr/rollout_is_seq_max_deviation:0.018477141857147217 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.075913135893643) - rollout_corr/training_log_ppl:np.float64(0.07041687372816341) - rollout_corr/kl:np.float64(0.0015584639460186622) - rollout_corr/k3_kl:np.float64(0.001706473551221152) - rollout_corr/rollout_ppl:np.float64(1.0741174332797527) - rollout_corr/rollout_log_ppl:np.float64(0.06885840870222637) - rollout_corr/log_ppl_diff:np.float64(0.0015584650404889544) - rollout_corr/log_ppl_abs_diff:np.float64(0.005063267224159063) - rollout_corr/log_ppl_diff_max:np.float64(0.07176004350185394) - rollout_corr/log_ppl_diff_min:np.float64(-0.028573468327522278) - rollout_corr/ppl_ratio:np.float64(1.0016104993410408) - rollout_corr/chi2_token:np.float64(0.003230785485357046) - rollout_corr/chi2_seq:np.float64(0.2828303910791874) - actor/pg_loss:np.float64(-0.001014434965327382) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011876891294377856) - actor/ppo_kl:np.float64(0.0011012015019673527) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20685786753892899) - perf/mfu/actor:np.float64(0.09093338179356399) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(106.90082168579102) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020980355329811573 - critic/advantages/max:0.5 - critic/advantages/min:-0.75 - critic/returns/mean:0.0020980355329811573 - critic/returns/max:0.5 - critic/returns/min:-0.75 - response_length/mean:61.44140625 - response_length/max:374.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:61.44140625 - response_length_non_aborted/max:374.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:631.5625 - prompt_length/max:1106.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.844848394393921e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.46071535535156727) - timing_s/agent_loop/generate_sequences/max:np.float64(2.714626483619213) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8670289495785255) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.714626483619213) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1106 - timing_s/agent_loop/slowest/response_length:374 - timing_s/gen:8.90407563187182 - timing_s/reward:0.07462672330439091 - timing_s/old_log_prob:3.1442562006413937 - timing_s/adv:0.12245601043105125 - timing_s/update_actor:12.837922079488635 - timing_s/step:25.166267208755016 - timing_s/stop_profile:0.00012287870049476624 - timing_per_token_ms/update_actor:0.07236342056766362 - timing_per_token_ms/gen:0.5660929259248407 - timing_per_token_ms/adv:0.0006902468895662072 - perf/total_num_tokens:177409 - perf/time_per_step:25.166267208755016 - perf/throughput:881.1845164023855 (TaskRunner pid=2448026) Training Progress: 41%|████ | 41/100 [21:43<33:27, 34.03s/it] (TaskRunner pid=2448026) step:42 - global_seqlen/min:15565 - global_seqlen/max:27029 - global_seqlen/minmax_diff:11464 - global_seqlen/balanced_min:21749 - global_seqlen/balanced_max:21751 - global_seqlen/mean:21750.125 - actor/entropy:0.08173059672117233 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18461838364601135 - training/rollout_probs_diff_mean:0.002816988853737712 - training/rollout_probs_diff_std:0.011850755661725998 - training/rollout_actor_probs_pearson_corr:0.9967573881149292 - rollout_corr/rollout_is_mean:0.9998669624328613 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.602554665529169e-05 - rollout_corr/rollout_is_max:1.645333170890808 - rollout_corr/rollout_is_min:0.4728035032749176 - rollout_corr/rollout_is_std:0.0320819728076458 - rollout_corr/rollout_is_eff_sample_size:0.9989715673380457 - rollout_corr/rollout_is_seq_mean:0.9999083280563354 - rollout_corr/rollout_is_seq_std:0.0038275502156466246 - rollout_corr/rollout_is_seq_max:1.011169672012329 - rollout_corr/rollout_is_seq_min:0.9826750159263611 - rollout_corr/rollout_is_seq_max_deviation:0.017324984073638916 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0828497749753296) - rollout_corr/training_log_ppl:np.float64(0.07552901757799191) - rollout_corr/kl:np.float64(0.0023463610672469493) - rollout_corr/k3_kl:np.float64(0.0015220192083295666) - rollout_corr/rollout_ppl:np.float64(1.0801830082200468) - rollout_corr/rollout_log_ppl:np.float64(0.07318265644926214) - rollout_corr/log_ppl_diff:np.float64(0.002346361128729768) - rollout_corr/log_ppl_abs_diff:np.float64(0.005253741510387044) - rollout_corr/log_ppl_diff_max:np.float64(0.041754260659217834) - rollout_corr/log_ppl_diff_min:np.float64(-0.023880019783973694) - rollout_corr/ppl_ratio:np.float64(1.0023789517581463) - rollout_corr/chi2_token:np.float64(0.0015337318181991577) - rollout_corr/chi2_seq:np.float64(0.35352901672013104) - actor/pg_loss:np.float64(2.5707297027111053e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002604166729724966) - actor/ppo_kl:np.float64(0.0017713437161006595) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20906937494874) - perf/mfu/actor:np.float64(0.08658246577350039) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(164.91781759262085) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.26171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.26171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0003501596802379936 - critic/advantages/max:0.625 - critic/advantages/min:-0.375 - critic/returns/mean:-0.0003501596802379936 - critic/returns/max:0.625 - critic/returns/min:-0.375 - response_length/mean:69.72265625 - response_length/max:213.0 - response_length/min:30.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:69.72265625 - response_length_non_aborted/max:213.0 - response_length_non_aborted/min:30.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:609.96875 - prompt_length/max:1096.0 - prompt_length/min:247.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016233325004577637 - timing_s/agent_loop/generate_sequences/min:np.float64(0.45332893170416355) - timing_s/agent_loop/generate_sequences/max:np.float64(2.0104285944253206) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9639438584272284) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.0104285944253206) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:935 - timing_s/agent_loop/slowest/response_length:212 - timing_s/gen:11.125117583200336 - timing_s/reward:0.18361933156847954 - timing_s/old_log_prob:5.191113628447056 - timing_s/adv:0.1311605591326952 - timing_s/update_actor:13.074742700904608 - timing_s/step:29.810759907588363 - timing_s/stop_profile:0.00012407638132572174 - timing_per_token_ms/update_actor:0.07514176758124728 - timing_per_token_ms/gen:0.6232908052664202 - timing_per_token_ms/adv:0.0007537919847167269 - perf/total_num_tokens:174001 - perf/time_per_step:29.810759907588363 - perf/throughput:729.606526885733 (TaskRunner pid=2448026) Training Progress: 42%|████▏ | 42/100 [22:13<31:42, 32.80s/it] (TaskRunner pid=2448026) step:43 - global_seqlen/min:14730 - global_seqlen/max:26485 - global_seqlen/minmax_diff:11755 - global_seqlen/balanced_min:21987 - global_seqlen/balanced_max:21987 - global_seqlen/mean:21987.0 - actor/entropy:0.06117398664355278 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3394731283187866 - training/rollout_probs_diff_mean:0.0024877546820789576 - training/rollout_probs_diff_std:0.011749840341508389 - training/rollout_actor_probs_pearson_corr:0.9961252808570862 - rollout_corr/rollout_is_mean:0.9999476671218872 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00016411377873737365 - rollout_corr/rollout_is_max:1.9854850769042969 - rollout_corr/rollout_is_min:0.396138459444046 - rollout_corr/rollout_is_std:0.03346775472164154 - rollout_corr/rollout_is_eff_sample_size:0.9988811627143783 - rollout_corr/rollout_is_seq_mean:0.9999850988388062 - rollout_corr/rollout_is_seq_std:0.004011556506156921 - rollout_corr/rollout_is_seq_max:1.016424536705017 - rollout_corr/rollout_is_seq_min:0.9888381958007812 - rollout_corr/rollout_is_seq_max_deviation:0.01642453670501709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0628677867352962) - rollout_corr/training_log_ppl:np.float64(0.05901054110942994) - rollout_corr/kl:np.float64(0.005593629202129358) - rollout_corr/k3_kl:np.float64(0.004754803588778395) - rollout_corr/rollout_ppl:np.float64(1.0565789542160928) - rollout_corr/rollout_log_ppl:np.float64(0.05341691119326697) - rollout_corr/log_ppl_diff:np.float64(0.005593629850707771) - rollout_corr/log_ppl_abs_diff:np.float64(0.007763132470856249) - rollout_corr/log_ppl_diff_max:np.float64(0.2297414392232895) - rollout_corr/log_ppl_diff_min:np.float64(-0.023556970059871674) - rollout_corr/ppl_ratio:np.float64(1.006074825534597) - rollout_corr/chi2_token:np.float64(0.0030338026117533445) - rollout_corr/chi2_seq:np.float64(1.2322402724530548) - actor/pg_loss:np.float64(0.00012502633035182953) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000913327356101945) - actor/ppo_kl:np.float64(0.0051383633966037845) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27539026364684105) - perf/mfu/actor:np.float64(0.08929323413184496) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(119.96002674102783) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.31640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.31640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004718271375168115 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004718271375168115 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:71.40625 - response_length/max:343.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:71.40625 - response_length_non_aborted/max:343.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:615.6875 - prompt_length/max:912.0 - prompt_length/min:313.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012691877782344818 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4811346083879471) - timing_s/agent_loop/generate_sequences/max:np.float64(2.558582082390785) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9727964748381055) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.558582082390785) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:693 - timing_s/agent_loop/slowest/response_length:343 - timing_s/gen:12.371321195736527 - timing_s/reward:0.07264602370560169 - timing_s/old_log_prob:3.038275796920061 - timing_s/adv:0.12641285173594952 - timing_s/update_actor:13.036879435181618 - timing_s/step:28.691039629280567 - timing_s/stop_profile:0.00010772980749607086 - timing_per_token_ms/update_actor:0.0741169750033066 - timing_per_token_ms/gen:0.6767681179286941 - timing_per_token_ms/adv:0.0007186795136668799 - perf/total_num_tokens:175896 - perf/time_per_step:28.691039629280567 - perf/throughput:766.3368174906155 (TaskRunner pid=2448026) Training Progress: 43%|████▎ | 43/100 [22:42<30:00, 31.59s/it] (TaskRunner pid=2448026) step:44 - global_seqlen/min:19285 - global_seqlen/max:26064 - global_seqlen/minmax_diff:6779 - global_seqlen/balanced_min:22690 - global_seqlen/balanced_max:22692 - global_seqlen/mean:22691.125 - actor/entropy:0.07789044827222824 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23305201530456543 - training/rollout_probs_diff_mean:0.0026447956915944815 - training/rollout_probs_diff_std:0.011822449043393135 - training/rollout_actor_probs_pearson_corr:0.9968587756156921 - rollout_corr/rollout_is_mean:1.0000234842300415 - rollout_corr/rollout_is_ratio_fraction_high:5.413305916590616e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.413305916590616e-05 - rollout_corr/rollout_is_max:2.256159782409668 - rollout_corr/rollout_is_min:0.44322940707206726 - rollout_corr/rollout_is_std:0.03280762955546379 - rollout_corr/rollout_is_eff_sample_size:0.9989248165882855 - rollout_corr/rollout_is_seq_mean:1.0002392530441284 - rollout_corr/rollout_is_seq_std:0.003780314000323415 - rollout_corr/rollout_is_seq_max:1.0216200351715088 - rollout_corr/rollout_is_seq_min:0.9902517795562744 - rollout_corr/rollout_is_seq_max_deviation:0.02162003517150879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0688496143557131) - rollout_corr/training_log_ppl:np.float64(0.06418157074222108) - rollout_corr/kl:np.float64(0.0007993808062087826) - rollout_corr/k3_kl:np.float64(0.0006536556666558464) - rollout_corr/rollout_ppl:np.float64(1.0679748128168285) - rollout_corr/rollout_log_ppl:np.float64(0.06338218958717334) - rollout_corr/log_ppl_diff:np.float64(0.0007993811550477403) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032819734415170387) - rollout_corr/log_ppl_diff_max:np.float64(0.018435046076774597) - rollout_corr/log_ppl_diff_min:np.float64(-0.010797828435897827) - rollout_corr/ppl_ratio:np.float64(1.0008110620547086) - rollout_corr/chi2_token:np.float64(0.0009655207395553589) - rollout_corr/chi2_seq:np.float64(0.11449967278167605) - actor/pg_loss:np.float64(0.00017898401711136103) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036806522257393226) - actor/ppo_kl:np.float64(0.0005864073127005431) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07165075093507767) - perf/mfu/actor:np.float64(0.0889073321807116) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(121.13435363769531) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00016239917022176087 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:-0.00016239917022176087 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:72.16015625 - response_length/max:203.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:72.16015625 - response_length_non_aborted/max:203.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:636.9375 - prompt_length/max:971.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014416687190532684 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4092059787362814) - timing_s/agent_loop/generate_sequences/max:np.float64(1.997605787590146) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9633429685200099) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.997605787590146) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:485 - timing_s/agent_loop/slowest/response_length:203 - timing_s/gen:9.173241855576634 - timing_s/reward:0.0710380282253027 - timing_s/old_log_prob:3.019885452464223 - timing_s/adv:0.12925943545997143 - timing_s/update_actor:13.325038401409984 - timing_s/step:25.800612976774573 - timing_s/stop_profile:0.00011668168008327484 - timing_per_token_ms/update_actor:0.07340446100298015 - timing_per_token_ms/gen:0.4965756431319566 - timing_per_token_ms/adv:0.0007120594255461741 - perf/total_num_tokens:181529 - perf/time_per_step:25.800612976774573 - perf/throughput:879.4800736101231 (TaskRunner pid=2448026) Training Progress: 44%|████▍ | 44/100 [23:08<27:52, 29.87s/it] (TaskRunner pid=2448026) step:45 - global_seqlen/min:18899 - global_seqlen/max:28159 - global_seqlen/minmax_diff:9260 - global_seqlen/balanced_min:23369 - global_seqlen/balanced_max:23373 - global_seqlen/mean:23370.875 - actor/entropy:0.06806084513664246 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33707690238952637 - training/rollout_probs_diff_mean:0.002958549652248621 - training/rollout_probs_diff_std:0.014316844753921032 - training/rollout_actor_probs_pearson_corr:0.9945234656333923 - rollout_corr/rollout_is_mean:1.0000187158584595 - rollout_corr/rollout_is_ratio_fraction_high:8.735531446291134e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017471062892582268 - rollout_corr/rollout_is_max:4.589768886566162 - rollout_corr/rollout_is_min:0.38590171933174133 - rollout_corr/rollout_is_std:0.034757256507873535 - rollout_corr/rollout_is_eff_sample_size:0.9987930339592404 - rollout_corr/rollout_is_seq_mean:1.0001131296157837 - rollout_corr/rollout_is_seq_std:0.006319973152130842 - rollout_corr/rollout_is_seq_max:1.0614110231399536 - rollout_corr/rollout_is_seq_min:0.9823589324951172 - rollout_corr/rollout_is_seq_max_deviation:0.06141102313995361 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.05841612117365) - rollout_corr/training_log_ppl:np.float64(0.0554860308093339) - rollout_corr/kl:np.float64(0.0006691704598331683) - rollout_corr/k3_kl:np.float64(0.0007411395971654786) - rollout_corr/rollout_ppl:np.float64(1.0577177540399134) - rollout_corr/rollout_log_ppl:np.float64(0.05481686065013491) - rollout_corr/log_ppl_diff:np.float64(0.000669170159198984) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032387126602770877) - rollout_corr/log_ppl_diff_max:np.float64(0.031549446284770966) - rollout_corr/log_ppl_diff_min:np.float64(-0.020488116890192032) - rollout_corr/ppl_ratio:np.float64(1.0006849435158074) - rollout_corr/chi2_token:np.float64(0.0018247494008392096) - rollout_corr/chi2_seq:np.float64(0.13643001415766776) - actor/pg_loss:np.float64(4.271394573152065e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(-1.0351071232150844e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13614605367183685) - perf/mfu/actor:np.float64(0.08919438116670493) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(117.12595319747925) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.3046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016160734230652452 - critic/advantages/max:0.25 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0016160734230652452 - critic/returns/max:0.25 - critic/returns/min:-0.75 - response_length/mean:89.43359375 - response_length/max:471.0 - response_length/min:28.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:89.43359375 - response_length_non_aborted/max:471.0 - response_length_non_aborted/min:28.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:640.90625 - prompt_length/max:1168.0 - prompt_length/min:247.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012708641588687897 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4739733189344406) - timing_s/agent_loop/generate_sequences/max:np.float64(3.532186970114708) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0980966585557326) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.532186970114708) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:691 - timing_s/agent_loop/slowest/response_length:471 - timing_s/gen:10.99059029854834 - timing_s/reward:0.07888947799801826 - timing_s/old_log_prob:3.2237731143832207 - timing_s/adv:0.15304752811789513 - timing_s/update_actor:13.77125832438469 - timing_s/step:28.28988539800048 - timing_s/stop_profile:0.00012324191629886627 - timing_per_token_ms/update_actor:0.073656090777435 - timing_per_token_ms/gen:0.4800432539221813 - timing_per_token_ms/adv:0.0008185804346108945 - perf/total_num_tokens:186967 - perf/time_per_step:28.28988539800048 - perf/throughput:826.1212327728923 (TaskRunner pid=2448026) Training Progress: 45%|████▌ | 45/100 [23:36<26:57, 29.41s/it] (TaskRunner pid=2448026) step:46 - global_seqlen/min:18000 - global_seqlen/max:25776 - global_seqlen/minmax_diff:7776 - global_seqlen/balanced_min:20951 - global_seqlen/balanced_max:20958 - global_seqlen/mean:20952.625 - actor/entropy:0.06581872701644897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4830358028411865 - training/rollout_probs_diff_mean:0.0026606442406773567 - training/rollout_probs_diff_std:0.013761429116129875 - training/rollout_actor_probs_pearson_corr:0.9948807954788208 - rollout_corr/rollout_is_mean:0.9999655485153198 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00021761601965408772 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:0.3035470247268677 - rollout_corr/rollout_is_std:0.03511383756995201 - rollout_corr/rollout_is_eff_sample_size:0.998768417774027 - rollout_corr/rollout_is_seq_mean:1.0001786947250366 - rollout_corr/rollout_is_seq_std:0.005675902124494314 - rollout_corr/rollout_is_seq_max:1.0266809463500977 - rollout_corr/rollout_is_seq_min:0.9778066277503967 - rollout_corr/rollout_is_seq_max_deviation:0.026680946350097656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.063858987763524) - rollout_corr/training_log_ppl:np.float64(0.05976022402091985) - rollout_corr/kl:np.float64(0.0005586090809412969) - rollout_corr/k3_kl:np.float64(0.0010411815033048843) - rollout_corr/rollout_ppl:np.float64(1.0632656360976398) - rollout_corr/rollout_log_ppl:np.float64(0.05920161522408307) - rollout_corr/log_ppl_diff:np.float64(0.0005586087968367792) - rollout_corr/log_ppl_abs_diff:np.float64(0.004258156228843291) - rollout_corr/log_ppl_diff_max:np.float64(0.028902918100357056) - rollout_corr/log_ppl_diff_min:np.float64(-0.018556788563728333) - rollout_corr/ppl_ratio:np.float64(1.0005791690200567) - rollout_corr/chi2_token:np.float64(0.003366896416991949) - rollout_corr/chi2_seq:np.float64(0.1587341504637152) - actor/pg_loss:np.float64(8.248584344983101e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.985041469917633e-05) - actor/ppo_kl:np.float64(0.0001010096884810352) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5298027371463832) - perf/mfu/actor:np.float64(0.0806260994109171) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(109.15704345703125) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022509656846523285 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022509656846523285 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:71.80078125 - response_length/max:391.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:71.80078125 - response_length_non_aborted/max:391.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:582.96875 - prompt_length/max:1125.0 - prompt_length/min:341.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011851638555526733 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5265140607953072) - timing_s/agent_loop/generate_sequences/max:np.float64(2.753229435533285) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9350689940256416) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.753229435533285) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1010 - timing_s/agent_loop/slowest/response_length:391 - timing_s/gen:9.474367344751954 - timing_s/reward:0.06818493083119392 - timing_s/old_log_prob:2.9699992574751377 - timing_s/adv:0.12070733495056629 - timing_s/update_actor:13.60346839018166 - timing_s/step:26.319604463875294 - timing_s/stop_profile:2.785027027130127e-05 - timing_per_token_ms/update_actor:0.08115611045263815 - timing_per_token_ms/gen:0.5154435201975929 - timing_per_token_ms/adv:0.0007201205991526497 - perf/total_num_tokens:167621 - perf/time_per_step:26.319604463875294 - perf/throughput:796.0843419496791 (TaskRunner pid=2448026) Training Progress: 46%|████▌ | 46/100 [24:03<25:38, 28.49s/it] (TaskRunner pid=2448026) step:47 - global_seqlen/min:16854 - global_seqlen/max:24229 - global_seqlen/minmax_diff:7375 - global_seqlen/balanced_min:21444 - global_seqlen/balanced_max:21449 - global_seqlen/mean:21447.375 - actor/entropy:0.0633917897939682 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1953447461128235 - training/rollout_probs_diff_mean:0.0023635365068912506 - training/rollout_probs_diff_std:0.01099692564457655 - training/rollout_actor_probs_pearson_corr:0.9965986013412476 - rollout_corr/rollout_is_mean:1.0003154277801514 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5930992364883423 - rollout_corr/rollout_is_min:0.5718968510627747 - rollout_corr/rollout_is_std:0.03081592358648777 - rollout_corr/rollout_is_eff_sample_size:0.9990516366740662 - rollout_corr/rollout_is_seq_mean:1.0004730224609375 - rollout_corr/rollout_is_seq_std:0.004049126990139484 - rollout_corr/rollout_is_seq_max:1.016745924949646 - rollout_corr/rollout_is_seq_min:0.9858582615852356 - rollout_corr/rollout_is_seq_max_deviation:0.016745924949645996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0556640555150807) - rollout_corr/training_log_ppl:np.float64(0.052577974794076) - rollout_corr/kl:np.float64(0.0009086632947692408) - rollout_corr/k3_kl:np.float64(0.0009336453900709099) - rollout_corr/rollout_ppl:np.float64(1.0546888425014913) - rollout_corr/rollout_log_ppl:np.float64(0.051669311122623185) - rollout_corr/log_ppl_diff:np.float64(0.0009086636732718034) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035405598914621805) - rollout_corr/log_ppl_diff_max:np.float64(0.040301717817783356) - rollout_corr/log_ppl_diff_min:np.float64(-0.018452279269695282) - rollout_corr/ppl_ratio:np.float64(1.0009264468681067) - rollout_corr/chi2_token:np.float64(0.0023799219634383917) - rollout_corr/chi2_seq:np.float64(0.09051068848930299) - actor/pg_loss:np.float64(0.00029965233989059925) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002317767357453704) - actor/ppo_kl:np.float64(0.0009553659166385842) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4296541139483452) - perf/mfu/actor:np.float64(0.08866261548707131) - perf/max_memory_allocated_gb:np.float64(132.5369839668274) - perf/max_memory_reserved_gb:np.float64(145.24609375) - perf/cpu_memory_used_gb:np.float64(109.07061767578125) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002565960166975856 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:-0.0002565960166975856 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:64.69921875 - response_length/max:394.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:64.69921875 - response_length_non_aborted/max:394.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:605.53125 - prompt_length/max:975.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.858974039554596e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4013958405703306) - timing_s/agent_loop/generate_sequences/max:np.float64(2.7259990330785513) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8423455143347383) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.7259990330785513) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:776 - timing_s/agent_loop/slowest/response_length:392 - timing_s/gen:10.017162440344691 - timing_s/reward:0.06861354969441891 - timing_s/old_log_prob:3.0151512436568737 - timing_s/adv:0.15842614695429802 - timing_s/update_actor:12.766919892281294 - timing_s/step:26.111633388325572 - timing_s/stop_profile:0.00011288933455944061 - timing_per_token_ms/update_actor:0.07440840599537993 - timing_per_token_ms/gen:0.6047915498608157 - timing_per_token_ms/adv:0.0009233422910396845 - perf/total_num_tokens:171579 - perf/time_per_step:26.111633388325572 - perf/throughput:821.3724006093412 (TaskRunner pid=2448026) Training Progress: 47%|████▋ | 47/100 [24:29<24:32, 27.79s/it]