training_logs/report.md

SkyRL Standard-GRPO Training Metrics Analysis

Generated from 1 log file(s) (--format standard)

Overview

Log File	Max Step	Train Steps	Final Reward	Max Reward	Final Entropy
finelog	21	11	0.4062	0.5684	0.7739

Async Metrics

	Mean	Std	Min	Max	Count
async/discard_rate	0	0	0	0	11
async/discarded_count	0	0	0	0	11
async/effective_batch_groups	64	0	64	64	11
async/effective_batch_samples	512	0	512	512	11
async/staleness_max	3	1.78885	1	7	11
async/staleness_mean	0.919034	0.514099	0.03125	1.82812	11
async/staleness_min	0.272727	0.467099	0	1	11
async/staleness_ratio	0.721591	0.300863	0.03125	1	11

Generate Metrics

	Mean	Std	Min	Max	Count
generate/avg_num_tokens	2902.98	599.241	2234.61	3771.75	11
generate/avg_tokens_non_zero_rewards	2580.54	627.47	1941.57	3374.06	11
generate/avg_tokens_zero_rewards	3138.75	579.896	2427.73	4067.32	11
generate/max_num_tokens	23268.2	2191.1	20143	26854	11
generate/min_num_tokens	379.636	115.639	247	651	11
generate/std_num_tokens	2697.28	365.539	2265.94	3377.54	11
generate/tis/aligned_tokens	713531	149297	566081	955649	11
generate/tis/alignment_fail_count	0.0909091	0.301511	0	1	11
generate/tis/exact_match_fraction	0.999999	1.6768e-06	0.999994	1	11
generate/tis/lcs_fallback_alert	0	0	0	0	11
generate/tis/lcs_fallback_fraction	0	0	0	0	11
generate/tis/lcs_fallback_messages	0	0	0	0	11
generate/tis/unaligned_fraction	5.05574e-07	1.6768e-06	0	5.56131e-06	11

Loss Metrics

	Mean	Std	Min	Max	Count
loss/avg_final_rewards	0.407138	0.0823485	0.257812	0.568359	11
loss/avg_raw_advantages	-0.0153546	0.0118293	-0.0296332	0.00130317	11
loss/avg_raw_advantages_abs	0.117088	0.0249498	0.0684489	0.153624	11

Policy Metrics

	Mean	Std	Min	Max	Count
policy/final_loss	-9.3325e-05	0.000147465	-0.000340611	0.000204686	11
policy/policy_entropy	0.907788	0.0857953	0.773902	1.01393	11
policy/policy_loss	-9.3325e-05	0.000147465	-0.000340611	0.000204686	11
policy/policy_lr	8e-06	0	8e-06	8e-06	11
policy/policy_update_steps	1	0	1	1	11
policy/ppo_clip_ratio	0	0	0	0	11
policy/raw_grad_norm	0.0641336	0.00992899	0.0436851	0.0750673	11
policy/rollout_train_prob_diff_mean	497.177	1578.13	1.0394	5253.32	11
policy/rollout_train_prob_diff_std	455497	1.46949e+06	0.722785	4.88546e+06	11

Reward Metrics

	Mean	Std	Min	Max	Count
reward/avg_pass_at_8	0.53267	0.055663	0.453125	0.625	11
reward/avg_raw_reward	0.407138	0.0823485	0.257812	0.568359	11

System Metrics

	Mean	Std	Min	Max	Count
system/process_rss_gb	20.8444	1.41878	17.5353	22.0891	11
system/process_vms_gb	138.577	9.70567	120.923	149.985	11
system/ram_available_gb	1865.23	3.1764	1862.12	1872.6	11
system/ram_percent	7.4	0.173205	7	7.6	11
system/ram_total_gb	2014.43	0	2014.43	2014.43	11
system/ram_used_gb	149.202	3.1764	141.826	152.307	11

Timing Metrics

	Mean	Std	Min	Max	Count
timing/cleanup_old_checkpoints	0.518482	1.69636	0.00617218	5.63319	11
timing/compute_advantages_and_returns	0.0781249	0.00819244	0.0662737	0.0914588	11
timing/convert_to_training_input	3.14866	0.373154	2.52558	3.63984	11
timing/fwd_logprobs_values_reward	46.9547	7.59618	41.8199	69.1615	11
timing/policy_train	148.799	13.4953	135.556	183.641	11
timing/run_training	195.926	20.8822	177.516	253.015	11
timing/save_checkpoints	464.877	6.8193	455.88	479.929	11
timing/step	789.831	479.948	293.359	1965.64	11
timing/sync_weights	123.259	67.5995	81.1427	264.975	11
timing/train_critic_and_policy	148.872	13.5064	135.61	183.741	11
timing/wait_for_generation_buffer	467.458	507.172	0.000506878	1703.85	11
timing/save_hf_model	37.1397	2.46155	35.3991	38.8803	2

Tis Metrics

	Mean	Std	Min	Max	Count
tis/batch_skipped_no_logprobs	0	0	0	0	11
tis/skipped_fraction	0	0	0	0	11

Trainer Metrics

	Mean	Std	Min	Max	Count
trainer/epoch	0	0	0	0	11
trainer/global_step	16	3.31662	11	21	11

Training Progression (collapse signals)

finelog

Step	Epoch	Reward	avg_pass_at_8	Entropy	GradNorm	PPOClip	PolicyLoss	logRatio_mean	Step Time (s)
11	0	0.4492	0.5156	0.9170	0.0738	0.00000	-0.00011	nan	877.4
12	0	0.2578	0.4531	0.9270	0.0723	0.00000	-0.00001	nan	462.7
13	0	0.3828	0.5000	0.9662	0.0748	0.00000	0.00020	nan	293.4
14	0	0.3965	0.5312	1.0139	0.0654	0.00000	0.00004	nan	312.2
15	0	0.3730	0.5312	1.0018	0.0640	0.00000	-0.00017	nan	672.2
16	0	0.4941	0.6250	1.0002	0.0586	0.00000	-0.00034	nan	985.8
17	0	0.3184	0.4688	0.9017	0.0541	0.00000	-0.00023	nan	428.3
18	0	0.4043	0.5625	0.8676	0.0751	0.00000	-0.00001	nan	693.6
19	0	0.4277	0.5469	0.8370	0.0580	0.00000	-0.00007	nan	836.6
20	0	0.5684	0.6250	0.7793	0.0437	0.00000	-0.00018	nan	1160.3
21	0	0.4062	0.5000	0.7739	0.0657	0.00000	-0.00015	nan	1965.6

vLLM Inference Engine Analysis (per-engine)

Log	Avg Running/Engine	Avg Gen Throughput/Engine	Avg KV Cache %	Avg Prefix Hit %
finelog	2.0	232.1 tok/s	0.7%	90.2%

Best Checkpoint (trailing-5 EMA of reward/avg_raw_reward)

No step chosen: No saved-aligned checkpoint eligible (after cap + exports intersection). available_exports=[], cap_step=None, save_every=20

• available exports: []
• cap (latest_ckpt_global_step.txt): None

Step	Reward	EMA	Eligible
11	0.4492	0.4492
12	0.2578	0.3854
13	0.3828	0.3845
14	0.3965	0.3885
15	0.3730	0.3834
16	0.4941	0.4203
17	0.3184	0.3863
18	0.4043	0.3923
19	0.4277	0.4041
20	0.5684	0.4589
21	0.4062	0.4413