LorMolf commited on
Commit
0c8c99f
·
verified ·
1 Parent(s): 8af4d8b

Upload Connect4 checkpoint envstep_90000.pth.tar

Browse files
README.md CHANGED
@@ -1,16 +1,22 @@
1
  ---
2
- library_name: efficientzero
3
  tags:
4
- - muzero
5
  - efficientzero
6
- - connect4
 
 
7
  - crpt
8
  ---
9
 
10
  # Connect4-CK
11
 
12
- Intermediate EfficientZero Connect4 checkpoint from `simplified5_balanced_1day_wandb_online_20260518`.
13
 
14
- Uploaded checkpoint: `envstep_5000.pth.tar`.
 
 
 
 
 
15
 
16
- This is a recovery/export test from an ephemeral pod. The resolved training config is included at `configs/connect4.yaml`.
 
1
  ---
2
+ library_name: pytorch
3
  tags:
 
4
  - efficientzero
5
+ - muzero
6
+ - board-game
7
+ - combinatorial-reasoning
8
  - crpt
9
  ---
10
 
11
  # Connect4-CK
12
 
13
+ Latest exported EfficientZero checkpoint for `connect4` from the simplified-five CRPT training runs.
14
 
15
+ - Checkpoint: `checkpoints/envstep_90000.pth.tar`
16
+ - Source path: `/workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459/ckpt/envstep_90000.pth.tar`
17
+ - Source attempt directory: `/workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459`
18
+ - W&B project: `crpt-simplified5-constant-lr-continuation`
19
+ - W&B run: `constant_lr_20260519__connect4__a01`
20
+ - Uploaded at: `2026-05-19T15:04:12Z`
21
 
22
+ Companion metadata is stored under `metadata/`, including the resolved LightZero config when available.
checkpoints/envstep_90000.pth.tar ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bda258a15a1dbab2a7d8112baeb6cfdfd59ea1a34e2345eaef9b5db8d4373211
3
+ size 139564171
metadata/checkpoint_index.yaml ADDED
@@ -0,0 +1,2234 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ checkpoints:
2
+ ckpt_best.pth.tar:
3
+ checkpoint_name: ckpt_best.pth.tar
4
+ checkpoint_path: /workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459/ckpt/ckpt_best.pth.tar
5
+ saved_at: '2026-05-19T10:13:05.573846+00:00'
6
+ train_iter: 910
7
+ envstep: 10031
8
+ trigger: null
9
+ latest_scalars:
10
+ evaluator_iter/episode_count:
11
+ value: 10
12
+ step: 910
13
+ walltime: null
14
+ evaluator_step/episode_count:
15
+ value: 10
16
+ step: 10031
17
+ walltime: null
18
+ evaluator_iter/envstep_count:
19
+ value: 83
20
+ step: 910
21
+ walltime: null
22
+ evaluator_step/envstep_count:
23
+ value: 83
24
+ step: 10031
25
+ walltime: null
26
+ evaluator_iter/avg_envstep_per_episode:
27
+ value: 8.3
28
+ step: 910
29
+ walltime: null
30
+ evaluator_step/avg_envstep_per_episode:
31
+ value: 8.3
32
+ step: 10031
33
+ walltime: null
34
+ evaluator_iter/evaluate_time:
35
+ value: 8.95365625
36
+ step: 910
37
+ walltime: null
38
+ evaluator_step/evaluate_time:
39
+ value: 8.95365625
40
+ step: 10031
41
+ walltime: null
42
+ evaluator_iter/avg_envstep_per_sec:
43
+ value: 9.269956058453774
44
+ step: 910
45
+ walltime: null
46
+ evaluator_step/avg_envstep_per_sec:
47
+ value: 9.269956058453774
48
+ step: 10031
49
+ walltime: null
50
+ evaluator_iter/avg_time_per_episode:
51
+ value: 1.1168621757173223
52
+ step: 910
53
+ walltime: null
54
+ evaluator_step/avg_time_per_episode:
55
+ value: 1.1168621757173223
56
+ step: 10031
57
+ walltime: null
58
+ evaluator_iter/reward_mean:
59
+ value: -0.8
60
+ step: 910
61
+ walltime: null
62
+ evaluator_step/reward_mean:
63
+ value: -0.8
64
+ step: 10031
65
+ walltime: null
66
+ evaluator_iter/reward_std:
67
+ value: 0.6000000000000001
68
+ step: 910
69
+ walltime: null
70
+ evaluator_step/reward_std:
71
+ value: 0.6000000000000001
72
+ step: 10031
73
+ walltime: null
74
+ evaluator_iter/reward_max:
75
+ value: 1.0
76
+ step: 910
77
+ walltime: null
78
+ evaluator_step/reward_max:
79
+ value: 1.0
80
+ step: 10031
81
+ walltime: null
82
+ evaluator_iter/reward_min:
83
+ value: -1.0
84
+ step: 910
85
+ walltime: null
86
+ evaluator_step/reward_min:
87
+ value: -1.0
88
+ step: 10031
89
+ walltime: null
90
+ evaluator_iter/eval_episode_return_mean:
91
+ value: -0.8
92
+ step: 910
93
+ walltime: null
94
+ evaluator_step/eval_episode_return_mean:
95
+ value: -0.8
96
+ step: 10031
97
+ walltime: null
98
+ Buffer/Task_0/num_collected_episodes:
99
+ value: 368
100
+ step: 910
101
+ walltime: null
102
+ Buffer/Task_0/num_game_segments:
103
+ value: 368
104
+ step: 910
105
+ walltime: null
106
+ Buffer/Task_0/num_transitions:
107
+ value: 10031
108
+ step: 910
109
+ walltime: null
110
+ Buffer/Task_0/memory_usage_mb/game_segment_buffer:
111
+ value: 6.2646942138671875
112
+ step: 910
113
+ walltime: null
114
+ Buffer/Task_0/memory_usage_mb/process:
115
+ value: 2058.0
116
+ step: 910
117
+ walltime: null
118
+ learner_iter/collect_mcts_temperature_avg:
119
+ value: 0.25
120
+ step: 900
121
+ walltime: null
122
+ learner_step/collect_mcts_temperature_avg:
123
+ value: 0.25
124
+ step: 10031
125
+ walltime: null
126
+ learner_iter/cur_lr_avg:
127
+ value: 0.0029999999999999996
128
+ step: 900
129
+ walltime: null
130
+ learner_step/cur_lr_avg:
131
+ value: 0.0029999999999999996
132
+ step: 10031
133
+ walltime: null
134
+ learner_iter/weighted_total_loss_avg:
135
+ value: 0.010909480466084047
136
+ step: 900
137
+ walltime: null
138
+ learner_step/weighted_total_loss_avg:
139
+ value: 0.010909480466084047
140
+ step: 10031
141
+ walltime: null
142
+ learner_iter/total_loss_avg:
143
+ value: 0.010909480466084047
144
+ step: 900
145
+ walltime: null
146
+ learner_step/total_loss_avg:
147
+ value: 0.010909480466084047
148
+ step: 10031
149
+ walltime: null
150
+ learner_iter/policy_loss_avg:
151
+ value: 8.115459442138672
152
+ step: 900
153
+ walltime: null
154
+ learner_step/policy_loss_avg:
155
+ value: 8.115459442138672
156
+ step: 10031
157
+ walltime: null
158
+ learner_iter/policy_entropy_avg:
159
+ value: 1.429311665621671
160
+ step: 900
161
+ walltime: null
162
+ learner_step/policy_entropy_avg:
163
+ value: 1.429311665621671
164
+ step: 10031
165
+ walltime: null
166
+ learner_iter/target_policy_entropy_avg:
167
+ value: 1.4289432727929319
168
+ step: 900
169
+ walltime: null
170
+ learner_step/target_policy_entropy_avg:
171
+ value: 1.4289432727929319
172
+ step: 10031
173
+ walltime: null
174
+ learner_iter/value_prefix_loss_avg:
175
+ value: 0.21515980769287457
176
+ step: 900
177
+ walltime: null
178
+ learner_step/value_prefix_loss_avg:
179
+ value: 0.21515980769287457
180
+ step: 10031
181
+ walltime: null
182
+ learner_iter/value_loss_avg:
183
+ value: 4.965807264501398
184
+ step: 900
185
+ walltime: null
186
+ learner_step/value_loss_avg:
187
+ value: 4.965807264501398
188
+ step: 10031
189
+ walltime: null
190
+ learner_iter/consistency_loss_avg:
191
+ value: -0.9561161474748091
192
+ step: 900
193
+ walltime: null
194
+ learner_step/consistency_loss_avg:
195
+ value: -0.9561161474748091
196
+ step: 10031
197
+ walltime: null
198
+ learner_iter/value_priority_avg:
199
+ value: 0.5874042294242166
200
+ step: 900
201
+ walltime: null
202
+ learner_step/value_priority_avg:
203
+ value: 0.5874042294242166
204
+ step: 10031
205
+ walltime: null
206
+ learner_iter/target_value_prefix_avg:
207
+ value: 0.060487691313028336
208
+ step: 900
209
+ walltime: null
210
+ learner_step/target_value_prefix_avg:
211
+ value: 0.060487691313028336
212
+ step: 10031
213
+ walltime: null
214
+ learner_iter/target_value_avg:
215
+ value: -0.001775568192401393
216
+ step: 900
217
+ walltime: null
218
+ learner_step/target_value_avg:
219
+ value: -0.001775568192401393
220
+ step: 10031
221
+ walltime: null
222
+ learner_iter/predicted_value_prefixs_avg:
223
+ value: 0.04929109527306123
224
+ step: 900
225
+ walltime: null
226
+ learner_step/predicted_value_prefixs_avg:
227
+ value: 0.04929109527306123
228
+ step: 10031
229
+ walltime: null
230
+ learner_iter/predicted_values_avg:
231
+ value: -0.001578690177253024
232
+ step: 900
233
+ walltime: null
234
+ learner_step/predicted_values_avg:
235
+ value: -0.001578690177253024
236
+ step: 10031
237
+ walltime: null
238
+ learner_iter/transformed_target_value_prefix_avg:
239
+ value: 0.02511530691249804
240
+ step: 900
241
+ walltime: null
242
+ learner_step/transformed_target_value_prefix_avg:
243
+ value: 0.02511530691249804
244
+ step: 10031
245
+ walltime: null
246
+ learner_iter/transformed_target_value_avg:
247
+ value: -0.0007372399453412403
248
+ step: 900
249
+ walltime: null
250
+ learner_step/transformed_target_value_avg:
251
+ value: -0.0007372399453412403
252
+ step: 10031
253
+ walltime: null
254
+ learner_iter/total_grad_norm_before_clip_avg:
255
+ value: 0.3114241280338981
256
+ step: 900
257
+ walltime: null
258
+ learner_step/total_grad_norm_before_clip_avg:
259
+ value: 0.3114241280338981
260
+ step: 10031
261
+ walltime: null
262
+ collector_iter/episode_count:
263
+ value: 40
264
+ step: 900
265
+ walltime: null
266
+ collector_step/episode_count:
267
+ value: 40
268
+ step: 10031
269
+ walltime: null
270
+ collector_iter/envstep_count:
271
+ value: 1010
272
+ step: 900
273
+ walltime: null
274
+ collector_step/envstep_count:
275
+ value: 1010
276
+ step: 10031
277
+ walltime: null
278
+ collector_iter/avg_envstep_per_episode:
279
+ value: 25.25
280
+ step: 900
281
+ walltime: null
282
+ collector_step/avg_envstep_per_episode:
283
+ value: 25.25
284
+ step: 10031
285
+ walltime: null
286
+ collector_iter/avg_envstep_per_sec:
287
+ value: 9.44826405658907
288
+ step: 900
289
+ walltime: null
290
+ collector_step/avg_envstep_per_sec:
291
+ value: 9.44826405658907
292
+ step: 10031
293
+ walltime: null
294
+ collector_iter/avg_episode_per_sec:
295
+ value: 0.374188675508478
296
+ step: 900
297
+ walltime: null
298
+ collector_step/avg_episode_per_sec:
299
+ value: 0.374188675508478
300
+ step: 10031
301
+ walltime: null
302
+ collector_iter/collect_time:
303
+ value: 106.89794378636591
304
+ step: 900
305
+ walltime: null
306
+ collector_step/collect_time:
307
+ value: 106.89794378636591
308
+ step: 10031
309
+ walltime: null
310
+ collector_iter/reward_mean:
311
+ value: 0.4000000059604645
312
+ step: 900
313
+ walltime: null
314
+ collector_step/reward_mean:
315
+ value: 0.4000000059604645
316
+ step: 10031
317
+ walltime: null
318
+ collector_iter/reward_std:
319
+ value: 0.9165151715278625
320
+ step: 900
321
+ walltime: null
322
+ collector_step/reward_std:
323
+ value: 0.9165151715278625
324
+ step: 10031
325
+ walltime: null
326
+ collector_iter/reward_max:
327
+ value: 1.0
328
+ step: 900
329
+ walltime: null
330
+ collector_step/reward_max:
331
+ value: 1.0
332
+ step: 10031
333
+ walltime: null
334
+ collector_iter/reward_min:
335
+ value: -1.0
336
+ step: 900
337
+ walltime: null
338
+ collector_step/reward_min:
339
+ value: -1.0
340
+ step: 10031
341
+ walltime: null
342
+ collector_iter/total_envstep_count:
343
+ value: 10031
344
+ step: 900
345
+ walltime: null
346
+ collector_step/total_envstep_count:
347
+ value: 10031
348
+ step: 10031
349
+ walltime: null
350
+ collector_iter/total_episode_count:
351
+ value: 368
352
+ step: 900
353
+ walltime: null
354
+ collector_step/total_episode_count:
355
+ value: 368
356
+ step: 10031
357
+ walltime: null
358
+ collector_iter/total_duration:
359
+ value: 5777.1309334583675
360
+ step: 900
361
+ walltime: null
362
+ collector_step/total_duration:
363
+ value: 5777.1309334583675
364
+ step: 10031
365
+ walltime: null
366
+ collector_iter/visit_entropy_mean:
367
+ value: 0.970061481600583
368
+ step: 900
369
+ walltime: null
370
+ collector_step/visit_entropy_mean:
371
+ value: 0.970061481600583
372
+ step: 10031
373
+ walltime: null
374
+ envstep_70000.pth.tar:
375
+ checkpoint_name: envstep_70000.pth.tar
376
+ checkpoint_path: /workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459/ckpt/envstep_70000.pth.tar
377
+ saved_at: '2026-05-19T13:50:05.614710+00:00'
378
+ train_iter: 6180
379
+ envstep: 70000
380
+ trigger: envstep
381
+ latest_scalars:
382
+ evaluator_iter/episode_count:
383
+ value: 10
384
+ step: 5770
385
+ walltime: null
386
+ evaluator_step/episode_count:
387
+ value: 10
388
+ step: 65076
389
+ walltime: null
390
+ evaluator_iter/envstep_count:
391
+ value: 56
392
+ step: 5770
393
+ walltime: null
394
+ evaluator_step/envstep_count:
395
+ value: 56
396
+ step: 65076
397
+ walltime: null
398
+ evaluator_iter/avg_envstep_per_episode:
399
+ value: 5.6
400
+ step: 5770
401
+ walltime: null
402
+ evaluator_step/avg_envstep_per_episode:
403
+ value: 5.6
404
+ step: 65076
405
+ walltime: null
406
+ evaluator_iter/evaluate_time:
407
+ value: 7.194775390625
408
+ step: 5770
409
+ walltime: null
410
+ evaluator_step/evaluate_time:
411
+ value: 7.194775390625
412
+ step: 65076
413
+ walltime: null
414
+ evaluator_iter/avg_envstep_per_sec:
415
+ value: 7.783425744321305
416
+ step: 5770
417
+ walltime: null
418
+ evaluator_step/avg_envstep_per_sec:
419
+ value: 7.783425744321305
420
+ step: 65076
421
+ walltime: null
422
+ evaluator_iter/avg_time_per_episode:
423
+ value: 1.3898974543430902
424
+ step: 5770
425
+ walltime: null
426
+ evaluator_step/avg_time_per_episode:
427
+ value: 1.3898974543430902
428
+ step: 65076
429
+ walltime: null
430
+ evaluator_iter/reward_mean:
431
+ value: -1.0
432
+ step: 5770
433
+ walltime: null
434
+ evaluator_step/reward_mean:
435
+ value: -1.0
436
+ step: 65076
437
+ walltime: null
438
+ evaluator_iter/reward_std:
439
+ value: 0.0
440
+ step: 5770
441
+ walltime: null
442
+ evaluator_step/reward_std:
443
+ value: 0.0
444
+ step: 65076
445
+ walltime: null
446
+ evaluator_iter/reward_max:
447
+ value: -1.0
448
+ step: 5770
449
+ walltime: null
450
+ evaluator_step/reward_max:
451
+ value: -1.0
452
+ step: 65076
453
+ walltime: null
454
+ evaluator_iter/reward_min:
455
+ value: -1.0
456
+ step: 5770
457
+ walltime: null
458
+ evaluator_step/reward_min:
459
+ value: -1.0
460
+ step: 65076
461
+ walltime: null
462
+ evaluator_iter/eval_episode_return_mean:
463
+ value: -1.0
464
+ step: 5770
465
+ walltime: null
466
+ evaluator_step/eval_episode_return_mean:
467
+ value: -1.0
468
+ step: 65076
469
+ walltime: null
470
+ Buffer/Task_0/num_collected_episodes:
471
+ value: 2476
472
+ step: 6180
473
+ walltime: null
474
+ Buffer/Task_0/num_game_segments:
475
+ value: 1764
476
+ step: 6180
477
+ walltime: null
478
+ Buffer/Task_0/num_transitions:
479
+ value: 49968
480
+ step: 6180
481
+ walltime: null
482
+ Buffer/Task_0/memory_usage_mb/game_segment_buffer:
483
+ value: 30.9970703125
484
+ step: 6180
485
+ walltime: null
486
+ Buffer/Task_0/memory_usage_mb/process:
487
+ value: 2081.33984375
488
+ step: 6180
489
+ walltime: null
490
+ learner_iter/collect_mcts_temperature_avg:
491
+ value: 0.25
492
+ step: 6100
493
+ walltime: null
494
+ learner_step/collect_mcts_temperature_avg:
495
+ value: 0.25
496
+ step: 69076
497
+ walltime: null
498
+ learner_iter/cur_lr_avg:
499
+ value: 0.0029999999999999996
500
+ step: 6100
501
+ walltime: null
502
+ learner_step/cur_lr_avg:
503
+ value: 0.0029999999999999996
504
+ step: 69076
505
+ walltime: null
506
+ learner_iter/weighted_total_loss_avg:
507
+ value: 0.8979335264726118
508
+ step: 6100
509
+ walltime: null
510
+ learner_step/weighted_total_loss_avg:
511
+ value: 0.8979335264726118
512
+ step: 69076
513
+ walltime: null
514
+ learner_iter/total_loss_avg:
515
+ value: 0.8979335264726118
516
+ step: 6100
517
+ walltime: null
518
+ learner_step/total_loss_avg:
519
+ value: 0.8979335264726118
520
+ step: 69076
521
+ walltime: null
522
+ learner_iter/policy_loss_avg:
523
+ value: 8.947579123757102
524
+ step: 6100
525
+ walltime: null
526
+ learner_step/policy_loss_avg:
527
+ value: 8.947579123757102
528
+ step: 69076
529
+ walltime: null
530
+ learner_iter/policy_entropy_avg:
531
+ value: 1.5767489346590908
532
+ step: 6100
533
+ walltime: null
534
+ learner_step/policy_entropy_avg:
535
+ value: 1.5767489346590908
536
+ step: 69076
537
+ walltime: null
538
+ learner_iter/target_policy_entropy_avg:
539
+ value: 1.4807274847319634
540
+ step: 6100
541
+ walltime: null
542
+ learner_step/target_policy_entropy_avg:
543
+ value: 1.4807274847319634
544
+ step: 69076
545
+ walltime: null
546
+ learner_iter/value_prefix_loss_avg:
547
+ value: 0.21087754585526206
548
+ step: 6100
549
+ walltime: null
550
+ learner_step/value_prefix_loss_avg:
551
+ value: 0.21087754585526206
552
+ step: 69076
553
+ walltime: null
554
+ learner_iter/value_loss_avg:
555
+ value: 5.24790130962025
556
+ step: 6100
557
+ walltime: null
558
+ learner_step/value_loss_avg:
559
+ value: 5.24790130962025
560
+ step: 69076
561
+ walltime: null
562
+ learner_iter/consistency_loss_avg:
563
+ value: -0.9572498668323862
564
+ step: 6100
565
+ walltime: null
566
+ learner_step/consistency_loss_avg:
567
+ value: -0.9572498668323862
568
+ step: 69076
569
+ walltime: null
570
+ learner_iter/value_priority_avg:
571
+ value: 0.8108745000579141
572
+ step: 6100
573
+ walltime: null
574
+ learner_step/value_priority_avg:
575
+ value: 0.8108745000579141
576
+ step: 69076
577
+ walltime: null
578
+ learner_iter/target_value_prefix_avg:
579
+ value: 0.061316289346326484
580
+ step: 6100
581
+ walltime: null
582
+ learner_step/target_value_prefix_avg:
583
+ value: 0.061316289346326484
584
+ step: 69076
585
+ walltime: null
586
+ learner_iter/target_value_avg:
587
+ value: -0.000118371194482527
588
+ step: 6100
589
+ walltime: null
590
+ learner_step/target_value_avg:
591
+ value: -0.000118371194482527
592
+ step: 69076
593
+ walltime: null
594
+ learner_iter/predicted_value_prefixs_avg:
595
+ value: 0.05231790278445591
596
+ step: 6100
597
+ walltime: null
598
+ learner_step/predicted_value_prefixs_avg:
599
+ value: 0.05231790278445591
600
+ step: 69076
601
+ walltime: null
602
+ learner_iter/predicted_values_avg:
603
+ value: 0.0001337626811370931
604
+ step: 6100
605
+ walltime: null
606
+ learner_step/predicted_values_avg:
607
+ value: 0.0001337626811370931
608
+ step: 69076
609
+ walltime: null
610
+ learner_iter/transformed_target_value_prefix_avg:
611
+ value: 0.025459352880716324
612
+ step: 6100
613
+ walltime: null
614
+ learner_step/transformed_target_value_prefix_avg:
615
+ value: 0.025459352880716324
616
+ step: 69076
617
+ walltime: null
618
+ learner_iter/transformed_target_value_avg:
619
+ value: -4.914934238926931e-05
620
+ step: 6100
621
+ walltime: null
622
+ learner_step/transformed_target_value_avg:
623
+ value: -4.914934238926931e-05
624
+ step: 69076
625
+ walltime: null
626
+ learner_iter/total_grad_norm_before_clip_avg:
627
+ value: 0.3467647677118128
628
+ step: 6100
629
+ walltime: null
630
+ learner_step/total_grad_norm_before_clip_avg:
631
+ value: 0.3467647677118128
632
+ step: 69076
633
+ walltime: null
634
+ collector_iter/episode_count:
635
+ value: 40
636
+ step: 6100
637
+ walltime: null
638
+ collector_step/episode_count:
639
+ value: 40
640
+ step: 69076
641
+ walltime: null
642
+ collector_iter/envstep_count:
643
+ value: 1181
644
+ step: 6100
645
+ walltime: null
646
+ collector_step/envstep_count:
647
+ value: 1181
648
+ step: 69076
649
+ walltime: null
650
+ collector_iter/avg_envstep_per_episode:
651
+ value: 29.525
652
+ step: 6100
653
+ walltime: null
654
+ collector_step/avg_envstep_per_episode:
655
+ value: 29.525
656
+ step: 69076
657
+ walltime: null
658
+ collector_iter/avg_envstep_per_sec:
659
+ value: 9.84222987476683
660
+ step: 6100
661
+ walltime: null
662
+ collector_step/avg_envstep_per_sec:
663
+ value: 9.84222987476683
664
+ step: 69076
665
+ walltime: null
666
+ collector_iter/avg_episode_per_sec:
667
+ value: 0.3333524089675471
668
+ step: 6100
669
+ walltime: null
670
+ collector_step/avg_episode_per_sec:
671
+ value: 0.3333524089675471
672
+ step: 69076
673
+ walltime: null
674
+ collector_iter/collect_time:
675
+ value: 119.99313316465076
676
+ step: 6100
677
+ walltime: null
678
+ collector_step/collect_time:
679
+ value: 119.99313316465076
680
+ step: 69076
681
+ walltime: null
682
+ collector_iter/reward_mean:
683
+ value: 0.4749999940395355
684
+ step: 6100
685
+ walltime: null
686
+ collector_step/reward_mean:
687
+ value: 0.4749999940395355
688
+ step: 69076
689
+ walltime: null
690
+ collector_iter/reward_std:
691
+ value: 0.8656644821166992
692
+ step: 6100
693
+ walltime: null
694
+ collector_step/reward_std:
695
+ value: 0.8656644821166992
696
+ step: 69076
697
+ walltime: null
698
+ collector_iter/reward_max:
699
+ value: 1.0
700
+ step: 6100
701
+ walltime: null
702
+ collector_step/reward_max:
703
+ value: 1.0
704
+ step: 69076
705
+ walltime: null
706
+ collector_iter/reward_min:
707
+ value: -1.0
708
+ step: 6100
709
+ walltime: null
710
+ collector_step/reward_min:
711
+ value: -1.0
712
+ step: 69076
713
+ walltime: null
714
+ collector_iter/total_envstep_count:
715
+ value: 69076
716
+ step: 6100
717
+ walltime: null
718
+ collector_step/total_envstep_count:
719
+ value: 69076
720
+ step: 69076
721
+ walltime: null
722
+ collector_iter/total_episode_count:
723
+ value: 2448
724
+ step: 6100
725
+ walltime: null
726
+ collector_step/total_episode_count:
727
+ value: 2448
728
+ step: 69076
729
+ walltime: null
730
+ collector_iter/total_duration:
731
+ value: 38851.81019043435
732
+ step: 6100
733
+ walltime: null
734
+ collector_step/total_duration:
735
+ value: 38851.81019043435
736
+ step: 69076
737
+ walltime: null
738
+ collector_iter/visit_entropy_mean:
739
+ value: 0.9780714071106607
740
+ step: 6100
741
+ walltime: null
742
+ collector_step/visit_entropy_mean:
743
+ value: 0.9780714071106607
744
+ step: 69076
745
+ walltime: null
746
+ envstep_75000.pth.tar:
747
+ checkpoint_name: envstep_75000.pth.tar
748
+ checkpoint_path: /workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459/ckpt/envstep_75000.pth.tar
749
+ saved_at: '2026-05-19T14:07:06.562230+00:00'
750
+ train_iter: 6610
751
+ envstep: 75000
752
+ trigger: envstep
753
+ latest_scalars:
754
+ evaluator_iter/episode_count:
755
+ value: 10
756
+ step: 6190
757
+ walltime: null
758
+ evaluator_step/episode_count:
759
+ value: 10
760
+ step: 70062
761
+ walltime: null
762
+ evaluator_iter/envstep_count:
763
+ value: 69
764
+ step: 6190
765
+ walltime: null
766
+ evaluator_step/envstep_count:
767
+ value: 69
768
+ step: 70062
769
+ walltime: null
770
+ evaluator_iter/avg_envstep_per_episode:
771
+ value: 6.9
772
+ step: 6190
773
+ walltime: null
774
+ evaluator_step/avg_envstep_per_episode:
775
+ value: 6.9
776
+ step: 70062
777
+ walltime: null
778
+ evaluator_iter/evaluate_time:
779
+ value: 12.7152763671875
780
+ step: 6190
781
+ walltime: null
782
+ evaluator_step/evaluate_time:
783
+ value: 12.7152763671875
784
+ step: 70062
785
+ walltime: null
786
+ evaluator_iter/avg_envstep_per_sec:
787
+ value: 5.426543474749669
788
+ step: 6190
789
+ walltime: null
790
+ evaluator_step/avg_envstep_per_sec:
791
+ value: 5.426543474749669
792
+ step: 70062
793
+ walltime: null
794
+ evaluator_iter/avg_time_per_episode:
795
+ value: 0.7864555760506766
796
+ step: 6190
797
+ walltime: null
798
+ evaluator_step/avg_time_per_episode:
799
+ value: 0.7864555760506766
800
+ step: 70062
801
+ walltime: null
802
+ evaluator_iter/reward_mean:
803
+ value: -1.0
804
+ step: 6190
805
+ walltime: null
806
+ evaluator_step/reward_mean:
807
+ value: -1.0
808
+ step: 70062
809
+ walltime: null
810
+ evaluator_iter/reward_std:
811
+ value: 0.0
812
+ step: 6190
813
+ walltime: null
814
+ evaluator_step/reward_std:
815
+ value: 0.0
816
+ step: 70062
817
+ walltime: null
818
+ evaluator_iter/reward_max:
819
+ value: -1.0
820
+ step: 6190
821
+ walltime: null
822
+ evaluator_step/reward_max:
823
+ value: -1.0
824
+ step: 70062
825
+ walltime: null
826
+ evaluator_iter/reward_min:
827
+ value: -1.0
828
+ step: 6190
829
+ walltime: null
830
+ evaluator_step/reward_min:
831
+ value: -1.0
832
+ step: 70062
833
+ walltime: null
834
+ evaluator_iter/eval_episode_return_mean:
835
+ value: -1.0
836
+ step: 6190
837
+ walltime: null
838
+ evaluator_step/eval_episode_return_mean:
839
+ value: -1.0
840
+ step: 70062
841
+ walltime: null
842
+ Buffer/Task_0/num_collected_episodes:
843
+ value: 2648
844
+ step: 6610
845
+ walltime: null
846
+ Buffer/Task_0/num_game_segments:
847
+ value: 1763
848
+ step: 6610
849
+ walltime: null
850
+ Buffer/Task_0/num_transitions:
851
+ value: 49982
852
+ step: 6610
853
+ walltime: null
854
+ Buffer/Task_0/memory_usage_mb/game_segment_buffer:
855
+ value: 31.00725555419922
856
+ step: 6610
857
+ walltime: null
858
+ Buffer/Task_0/memory_usage_mb/process:
859
+ value: 2078.375
860
+ step: 6610
861
+ walltime: null
862
+ learner_iter/collect_mcts_temperature_avg:
863
+ value: 0.25
864
+ step: 6600
865
+ walltime: null
866
+ learner_step/collect_mcts_temperature_avg:
867
+ value: 0.25
868
+ step: 74963
869
+ walltime: null
870
+ learner_iter/cur_lr_avg:
871
+ value: 0.0029999999999999996
872
+ step: 6600
873
+ walltime: null
874
+ learner_step/cur_lr_avg:
875
+ value: 0.0029999999999999996
876
+ step: 74963
877
+ walltime: null
878
+ learner_iter/weighted_total_loss_avg:
879
+ value: 0.9603883515704762
880
+ step: 6600
881
+ walltime: null
882
+ learner_step/weighted_total_loss_avg:
883
+ value: 0.9603883515704762
884
+ step: 74963
885
+ walltime: null
886
+ learner_iter/total_loss_avg:
887
+ value: 0.9603883515704762
888
+ step: 6600
889
+ walltime: null
890
+ learner_step/total_loss_avg:
891
+ value: 0.9603883515704762
892
+ step: 74963
893
+ walltime: null
894
+ learner_iter/policy_loss_avg:
895
+ value: 9.02350971915505
896
+ step: 6600
897
+ walltime: null
898
+ learner_step/policy_loss_avg:
899
+ value: 9.02350971915505
900
+ step: 74963
901
+ walltime: null
902
+ learner_iter/policy_entropy_avg:
903
+ value: 1.580617500073982
904
+ step: 6600
905
+ walltime: null
906
+ learner_step/policy_entropy_avg:
907
+ value: 1.580617500073982
908
+ step: 74963
909
+ walltime: null
910
+ learner_iter/target_policy_entropy_avg:
911
+ value: 1.484574101187966
912
+ step: 6600
913
+ walltime: null
914
+ learner_step/target_policy_entropy_avg:
915
+ value: 1.484574101187966
916
+ step: 74963
917
+ walltime: null
918
+ learner_iter/value_prefix_loss_avg:
919
+ value: 0.2109665017236363
920
+ step: 6600
921
+ walltime: null
922
+ learner_step/value_prefix_loss_avg:
923
+ value: 0.2109665017236363
924
+ step: 74963
925
+ walltime: null
926
+ learner_iter/value_loss_avg:
927
+ value: 5.287646987221458
928
+ step: 6600
929
+ walltime: null
930
+ learner_step/value_loss_avg:
931
+ value: 5.287646987221458
932
+ step: 74963
933
+ walltime: null
934
+ learner_iter/consistency_loss_avg:
935
+ value: -0.9595999804410067
936
+ step: 6600
937
+ walltime: null
938
+ learner_step/consistency_loss_avg:
939
+ value: -0.9595999804410067
940
+ step: 74963
941
+ walltime: null
942
+ learner_iter/value_priority_avg:
943
+ value: 0.8126288869164207
944
+ step: 6600
945
+ walltime: null
946
+ learner_step/value_priority_avg:
947
+ value: 0.8126288869164207
948
+ step: 74963
949
+ walltime: null
950
+ learner_iter/target_value_prefix_avg:
951
+ value: 0.06214488839561289
952
+ step: 6600
953
+ walltime: null
954
+ learner_step/target_value_prefix_avg:
955
+ value: 0.06214488839561289
956
+ step: 74963
957
+ walltime: null
958
+ learner_iter/target_value_avg:
959
+ value: -0.0002367424312978983
960
+ step: 6600
961
+ walltime: null
962
+ learner_step/target_value_avg:
963
+ value: -0.0002367424312978983
964
+ step: 74963
965
+ walltime: null
966
+ learner_iter/predicted_value_prefixs_avg:
967
+ value: 0.04919351637363434
968
+ step: 6600
969
+ walltime: null
970
+ learner_step/predicted_value_prefixs_avg:
971
+ value: 0.04919351637363434
972
+ step: 74963
973
+ walltime: null
974
+ learner_iter/predicted_values_avg:
975
+ value: -0.00015135957668958739
976
+ step: 6600
977
+ walltime: null
978
+ learner_step/predicted_values_avg:
979
+ value: -0.00015135957668958739
980
+ step: 74963
981
+ walltime: null
982
+ learner_iter/transformed_target_value_prefix_avg:
983
+ value: 0.02580339749428359
984
+ step: 6600
985
+ walltime: null
986
+ learner_step/transformed_target_value_prefix_avg:
987
+ value: 0.02580339749428359
988
+ step: 74963
989
+ walltime: null
990
+ learner_iter/transformed_target_value_avg:
991
+ value: -9.829866537598518e-05
992
+ step: 6600
993
+ walltime: null
994
+ learner_step/transformed_target_value_avg:
995
+ value: -9.829866537598518e-05
996
+ step: 74963
997
+ walltime: null
998
+ learner_iter/total_grad_norm_before_clip_avg:
999
+ value: 0.3910723734985698
1000
+ step: 6600
1001
+ walltime: null
1002
+ learner_step/total_grad_norm_before_clip_avg:
1003
+ value: 0.3910723734985698
1004
+ step: 74963
1005
+ walltime: null
1006
+ collector_iter/episode_count:
1007
+ value: 40
1008
+ step: 6600
1009
+ walltime: null
1010
+ collector_step/episode_count:
1011
+ value: 40
1012
+ step: 74963
1013
+ walltime: null
1014
+ collector_iter/envstep_count:
1015
+ value: 1119
1016
+ step: 6600
1017
+ walltime: null
1018
+ collector_step/envstep_count:
1019
+ value: 1119
1020
+ step: 74963
1021
+ walltime: null
1022
+ collector_iter/avg_envstep_per_episode:
1023
+ value: 27.975
1024
+ step: 6600
1025
+ walltime: null
1026
+ collector_step/avg_envstep_per_episode:
1027
+ value: 27.975
1028
+ step: 74963
1029
+ walltime: null
1030
+ collector_iter/avg_envstep_per_sec:
1031
+ value: 10.896117848612953
1032
+ step: 6600
1033
+ walltime: null
1034
+ collector_step/avg_envstep_per_sec:
1035
+ value: 10.896117848612953
1036
+ step: 74963
1037
+ walltime: null
1038
+ collector_iter/avg_episode_per_sec:
1039
+ value: 0.38949482926230394
1040
+ step: 6600
1041
+ walltime: null
1042
+ collector_step/avg_episode_per_sec:
1043
+ value: 0.38949482926230394
1044
+ step: 74963
1045
+ walltime: null
1046
+ collector_iter/collect_time:
1047
+ value: 102.6971271371157
1048
+ step: 6600
1049
+ walltime: null
1050
+ collector_step/collect_time:
1051
+ value: 102.6971271371157
1052
+ step: 74963
1053
+ walltime: null
1054
+ collector_iter/reward_mean:
1055
+ value: 0.25
1056
+ step: 6600
1057
+ walltime: null
1058
+ collector_step/reward_mean:
1059
+ value: 0.25
1060
+ step: 74963
1061
+ walltime: null
1062
+ collector_iter/reward_std:
1063
+ value: 0.9682458639144897
1064
+ step: 6600
1065
+ walltime: null
1066
+ collector_step/reward_std:
1067
+ value: 0.9682458639144897
1068
+ step: 74963
1069
+ walltime: null
1070
+ collector_iter/reward_max:
1071
+ value: 1.0
1072
+ step: 6600
1073
+ walltime: null
1074
+ collector_step/reward_max:
1075
+ value: 1.0
1076
+ step: 74963
1077
+ walltime: null
1078
+ collector_iter/reward_min:
1079
+ value: -1.0
1080
+ step: 6600
1081
+ walltime: null
1082
+ collector_step/reward_min:
1083
+ value: -1.0
1084
+ step: 74963
1085
+ walltime: null
1086
+ collector_iter/total_envstep_count:
1087
+ value: 74963
1088
+ step: 6600
1089
+ walltime: null
1090
+ collector_step/total_envstep_count:
1091
+ value: 74963
1092
+ step: 74963
1093
+ walltime: null
1094
+ collector_iter/total_episode_count:
1095
+ value: 2648
1096
+ step: 6600
1097
+ walltime: null
1098
+ collector_step/total_episode_count:
1099
+ value: 2648
1100
+ step: 74963
1101
+ walltime: null
1102
+ collector_iter/total_duration:
1103
+ value: 41918.13410943206
1104
+ step: 6600
1105
+ walltime: null
1106
+ collector_step/total_duration:
1107
+ value: 41918.13410943206
1108
+ step: 74963
1109
+ walltime: null
1110
+ collector_iter/visit_entropy_mean:
1111
+ value: 1.107473344376133
1112
+ step: 6600
1113
+ walltime: null
1114
+ collector_step/visit_entropy_mean:
1115
+ value: 1.107473344376133
1116
+ step: 74963
1117
+ walltime: null
1118
+ envstep_80000.pth.tar:
1119
+ checkpoint_name: envstep_80000.pth.tar
1120
+ checkpoint_path: /workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459/ckpt/envstep_80000.pth.tar
1121
+ saved_at: '2026-05-19T14:25:15.596441+00:00'
1122
+ train_iter: 7070
1123
+ envstep: 80000
1124
+ trigger: envstep
1125
+ latest_scalars:
1126
+ evaluator_iter/episode_count:
1127
+ value: 10
1128
+ step: 6620
1129
+ walltime: null
1130
+ evaluator_step/episode_count:
1131
+ value: 10
1132
+ step: 75082
1133
+ walltime: null
1134
+ evaluator_iter/envstep_count:
1135
+ value: 50
1136
+ step: 6620
1137
+ walltime: null
1138
+ evaluator_step/envstep_count:
1139
+ value: 50
1140
+ step: 75082
1141
+ walltime: null
1142
+ evaluator_iter/avg_envstep_per_episode:
1143
+ value: 5.0
1144
+ step: 6620
1145
+ walltime: null
1146
+ evaluator_step/avg_envstep_per_episode:
1147
+ value: 5.0
1148
+ step: 75082
1149
+ walltime: null
1150
+ evaluator_iter/evaluate_time:
1151
+ value: 6.18236572265625
1152
+ step: 6620
1153
+ walltime: null
1154
+ evaluator_step/evaluate_time:
1155
+ value: 6.18236572265625
1156
+ step: 75082
1157
+ walltime: null
1158
+ evaluator_iter/avg_envstep_per_sec:
1159
+ value: 8.087518960058793
1160
+ step: 6620
1161
+ walltime: null
1162
+ evaluator_step/avg_envstep_per_sec:
1163
+ value: 8.087518960058793
1164
+ step: 75082
1165
+ walltime: null
1166
+ evaluator_iter/avg_time_per_episode:
1167
+ value: 1.6175037920117583
1168
+ step: 6620
1169
+ walltime: null
1170
+ evaluator_step/avg_time_per_episode:
1171
+ value: 1.6175037920117583
1172
+ step: 75082
1173
+ walltime: null
1174
+ evaluator_iter/reward_mean:
1175
+ value: -1.0
1176
+ step: 6620
1177
+ walltime: null
1178
+ evaluator_step/reward_mean:
1179
+ value: -1.0
1180
+ step: 75082
1181
+ walltime: null
1182
+ evaluator_iter/reward_std:
1183
+ value: 0.0
1184
+ step: 6620
1185
+ walltime: null
1186
+ evaluator_step/reward_std:
1187
+ value: 0.0
1188
+ step: 75082
1189
+ walltime: null
1190
+ evaluator_iter/reward_max:
1191
+ value: -1.0
1192
+ step: 6620
1193
+ walltime: null
1194
+ evaluator_step/reward_max:
1195
+ value: -1.0
1196
+ step: 75082
1197
+ walltime: null
1198
+ evaluator_iter/reward_min:
1199
+ value: -1.0
1200
+ step: 6620
1201
+ walltime: null
1202
+ evaluator_step/reward_min:
1203
+ value: -1.0
1204
+ step: 75082
1205
+ walltime: null
1206
+ evaluator_iter/eval_episode_return_mean:
1207
+ value: -1.0
1208
+ step: 6620
1209
+ walltime: null
1210
+ evaluator_step/eval_episode_return_mean:
1211
+ value: -1.0
1212
+ step: 75082
1213
+ walltime: null
1214
+ Buffer/Task_0/num_collected_episodes:
1215
+ value: 2832
1216
+ step: 7070
1217
+ walltime: null
1218
+ Buffer/Task_0/num_game_segments:
1219
+ value: 1769
1220
+ step: 7070
1221
+ walltime: null
1222
+ Buffer/Task_0/num_transitions:
1223
+ value: 49988
1224
+ step: 7070
1225
+ walltime: null
1226
+ Buffer/Task_0/memory_usage_mb/game_segment_buffer:
1227
+ value: 31.030921936035156
1228
+ step: 7070
1229
+ walltime: null
1230
+ Buffer/Task_0/memory_usage_mb/process:
1231
+ value: 2077.1328125
1232
+ step: 7070
1233
+ walltime: null
1234
+ learner_iter/collect_mcts_temperature_avg:
1235
+ value: 0.25
1236
+ step: 7000
1237
+ walltime: null
1238
+ learner_step/collect_mcts_temperature_avg:
1239
+ value: 0.25
1240
+ step: 79326
1241
+ walltime: null
1242
+ learner_iter/cur_lr_avg:
1243
+ value: 0.0029999999999999996
1244
+ step: 7000
1245
+ walltime: null
1246
+ learner_step/cur_lr_avg:
1247
+ value: 0.0029999999999999996
1248
+ step: 79326
1249
+ walltime: null
1250
+ learner_iter/weighted_total_loss_avg:
1251
+ value: 0.9828287308866327
1252
+ step: 7000
1253
+ walltime: null
1254
+ learner_step/weighted_total_loss_avg:
1255
+ value: 0.9828287308866327
1256
+ step: 79326
1257
+ walltime: null
1258
+ learner_iter/total_loss_avg:
1259
+ value: 0.9828287308866327
1260
+ step: 7000
1261
+ walltime: null
1262
+ learner_step/total_loss_avg:
1263
+ value: 0.9828287308866327
1264
+ step: 79326
1265
+ walltime: null
1266
+ learner_iter/policy_loss_avg:
1267
+ value: 9.123100020668723
1268
+ step: 7000
1269
+ walltime: null
1270
+ learner_step/policy_loss_avg:
1271
+ value: 9.123100020668723
1272
+ step: 79326
1273
+ walltime: null
1274
+ learner_iter/policy_entropy_avg:
1275
+ value: 1.5887374444441362
1276
+ step: 7000
1277
+ walltime: null
1278
+ learner_step/policy_entropy_avg:
1279
+ value: 1.5887374444441362
1280
+ step: 79326
1281
+ walltime: null
1282
+ learner_iter/target_policy_entropy_avg:
1283
+ value: 1.4960533488880505
1284
+ step: 7000
1285
+ walltime: null
1286
+ learner_step/target_policy_entropy_avg:
1287
+ value: 1.4960533488880505
1288
+ step: 79326
1289
+ walltime: null
1290
+ learner_iter/value_prefix_loss_avg:
1291
+ value: 0.18883771787990222
1292
+ step: 7000
1293
+ walltime: null
1294
+ learner_step/value_prefix_loss_avg:
1295
+ value: 0.18883771787990222
1296
+ step: 79326
1297
+ walltime: null
1298
+ learner_iter/value_loss_avg:
1299
+ value: 5.286637089469216
1300
+ step: 7000
1301
+ walltime: null
1302
+ learner_step/value_loss_avg:
1303
+ value: 5.286637089469216
1304
+ step: 79326
1305
+ walltime: null
1306
+ learner_iter/consistency_loss_avg:
1307
+ value: -0.9650768540122292
1308
+ step: 7000
1309
+ walltime: null
1310
+ learner_step/consistency_loss_avg:
1311
+ value: -0.9650768540122292
1312
+ step: 79326
1313
+ walltime: null
1314
+ learner_iter/value_priority_avg:
1315
+ value: 0.824982453476299
1316
+ step: 7000
1317
+ walltime: null
1318
+ learner_step/value_priority_avg:
1319
+ value: 0.824982453476299
1320
+ step: 79326
1321
+ walltime: null
1322
+ learner_iter/target_value_prefix_avg:
1323
+ value: 0.057054925709962845
1324
+ step: 7000
1325
+ walltime: null
1326
+ learner_step/target_value_prefix_avg:
1327
+ value: 0.057054925709962845
1328
+ step: 79326
1329
+ walltime: null
1330
+ learner_iter/target_value_avg:
1331
+ value: -0.0005918560676615347
1332
+ step: 7000
1333
+ walltime: null
1334
+ learner_step/target_value_avg:
1335
+ value: -0.0005918560676615347
1336
+ step: 79326
1337
+ walltime: null
1338
+ learner_iter/predicted_value_prefixs_avg:
1339
+ value: 0.04756909812038595
1340
+ step: 7000
1341
+ walltime: null
1342
+ learner_step/predicted_value_prefixs_avg:
1343
+ value: 0.04756909812038595
1344
+ step: 79326
1345
+ walltime: null
1346
+ learner_iter/predicted_values_avg:
1347
+ value: -0.00309789812176446
1348
+ step: 7000
1349
+ walltime: null
1350
+ learner_step/predicted_values_avg:
1351
+ value: -0.00309789812176446
1352
+ step: 79326
1353
+ walltime: null
1354
+ learner_iter/transformed_target_value_prefix_avg:
1355
+ value: 0.02368997579271143
1356
+ step: 7000
1357
+ walltime: null
1358
+ learner_step/transformed_target_value_prefix_avg:
1359
+ value: 0.02368997579271143
1360
+ step: 79326
1361
+ walltime: null
1362
+ learner_iter/transformed_target_value_avg:
1363
+ value: -0.00024574662728065795
1364
+ step: 7000
1365
+ walltime: null
1366
+ learner_step/transformed_target_value_avg:
1367
+ value: -0.00024574662728065795
1368
+ step: 79326
1369
+ walltime: null
1370
+ learner_iter/total_grad_norm_before_clip_avg:
1371
+ value: 0.3728354573249817
1372
+ step: 7000
1373
+ walltime: null
1374
+ learner_step/total_grad_norm_before_clip_avg:
1375
+ value: 0.3728354573249817
1376
+ step: 79326
1377
+ walltime: null
1378
+ collector_iter/episode_count:
1379
+ value: 40
1380
+ step: 7000
1381
+ walltime: null
1382
+ collector_step/episode_count:
1383
+ value: 40
1384
+ step: 79326
1385
+ walltime: null
1386
+ collector_iter/envstep_count:
1387
+ value: 1078
1388
+ step: 7000
1389
+ walltime: null
1390
+ collector_step/envstep_count:
1391
+ value: 1078
1392
+ step: 79326
1393
+ walltime: null
1394
+ collector_iter/avg_envstep_per_episode:
1395
+ value: 26.95
1396
+ step: 7000
1397
+ walltime: null
1398
+ collector_step/avg_envstep_per_episode:
1399
+ value: 26.95
1400
+ step: 79326
1401
+ walltime: null
1402
+ collector_iter/avg_envstep_per_sec:
1403
+ value: 10.314620505059771
1404
+ step: 7000
1405
+ walltime: null
1406
+ collector_step/avg_envstep_per_sec:
1407
+ value: 10.314620505059771
1408
+ step: 79326
1409
+ walltime: null
1410
+ collector_iter/avg_episode_per_sec:
1411
+ value: 0.38273174415806205
1412
+ step: 7000
1413
+ walltime: null
1414
+ collector_step/avg_episode_per_sec:
1415
+ value: 0.38273174415806205
1416
+ step: 79326
1417
+ walltime: null
1418
+ collector_iter/collect_time:
1419
+ value: 104.5118431134906
1420
+ step: 7000
1421
+ walltime: null
1422
+ collector_step/collect_time:
1423
+ value: 104.5118431134906
1424
+ step: 79326
1425
+ walltime: null
1426
+ collector_iter/reward_mean:
1427
+ value: 0.10000000149011612
1428
+ step: 7000
1429
+ walltime: null
1430
+ collector_step/reward_mean:
1431
+ value: 0.10000000149011612
1432
+ step: 79326
1433
+ walltime: null
1434
+ collector_iter/reward_std:
1435
+ value: 0.994987428188324
1436
+ step: 7000
1437
+ walltime: null
1438
+ collector_step/reward_std:
1439
+ value: 0.994987428188324
1440
+ step: 79326
1441
+ walltime: null
1442
+ collector_iter/reward_max:
1443
+ value: 1.0
1444
+ step: 7000
1445
+ walltime: null
1446
+ collector_step/reward_max:
1447
+ value: 1.0
1448
+ step: 79326
1449
+ walltime: null
1450
+ collector_iter/reward_min:
1451
+ value: -1.0
1452
+ step: 7000
1453
+ walltime: null
1454
+ collector_step/reward_min:
1455
+ value: -1.0
1456
+ step: 79326
1457
+ walltime: null
1458
+ collector_iter/total_envstep_count:
1459
+ value: 79326
1460
+ step: 7000
1461
+ walltime: null
1462
+ collector_step/total_envstep_count:
1463
+ value: 79326
1464
+ step: 79326
1465
+ walltime: null
1466
+ collector_iter/total_episode_count:
1467
+ value: 2808
1468
+ step: 7000
1469
+ walltime: null
1470
+ collector_step/total_episode_count:
1471
+ value: 2808
1472
+ step: 79326
1473
+ walltime: null
1474
+ collector_iter/total_duration:
1475
+ value: 44287.69188188307
1476
+ step: 7000
1477
+ walltime: null
1478
+ collector_step/total_duration:
1479
+ value: 44287.69188188307
1480
+ step: 79326
1481
+ walltime: null
1482
+ collector_iter/visit_entropy_mean:
1483
+ value: 1.0988913002798388
1484
+ step: 7000
1485
+ walltime: null
1486
+ collector_step/visit_entropy_mean:
1487
+ value: 1.0988913002798388
1488
+ step: 79326
1489
+ walltime: null
1490
+ envstep_85000.pth.tar:
1491
+ checkpoint_name: envstep_85000.pth.tar
1492
+ checkpoint_path: /workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459/ckpt/envstep_85000.pth.tar
1493
+ saved_at: '2026-05-19T14:44:43.675350+00:00'
1494
+ train_iter: 7580
1495
+ envstep: 85000
1496
+ trigger: envstep
1497
+ latest_scalars:
1498
+ evaluator_iter/episode_count:
1499
+ value: 10
1500
+ step: 7080
1501
+ walltime: null
1502
+ evaluator_step/episode_count:
1503
+ value: 10
1504
+ step: 80045
1505
+ walltime: null
1506
+ evaluator_iter/envstep_count:
1507
+ value: 73
1508
+ step: 7080
1509
+ walltime: null
1510
+ evaluator_step/envstep_count:
1511
+ value: 73
1512
+ step: 80045
1513
+ walltime: null
1514
+ evaluator_iter/avg_envstep_per_episode:
1515
+ value: 7.3
1516
+ step: 7080
1517
+ walltime: null
1518
+ evaluator_step/avg_envstep_per_episode:
1519
+ value: 7.3
1520
+ step: 80045
1521
+ walltime: null
1522
+ evaluator_iter/evaluate_time:
1523
+ value: 10.966330078125
1524
+ step: 7080
1525
+ walltime: null
1526
+ evaluator_step/evaluate_time:
1527
+ value: 10.966330078125
1528
+ step: 80045
1529
+ walltime: null
1530
+ evaluator_iter/avg_envstep_per_sec:
1531
+ value: 6.656739262811008
1532
+ step: 7080
1533
+ walltime: null
1534
+ evaluator_step/avg_envstep_per_sec:
1535
+ value: 6.656739262811008
1536
+ step: 80045
1537
+ walltime: null
1538
+ evaluator_iter/avg_time_per_episode:
1539
+ value: 0.9118820907960284
1540
+ step: 7080
1541
+ walltime: null
1542
+ evaluator_step/avg_time_per_episode:
1543
+ value: 0.9118820907960284
1544
+ step: 80045
1545
+ walltime: null
1546
+ evaluator_iter/reward_mean:
1547
+ value: -1.0
1548
+ step: 7080
1549
+ walltime: null
1550
+ evaluator_step/reward_mean:
1551
+ value: -1.0
1552
+ step: 80045
1553
+ walltime: null
1554
+ evaluator_iter/reward_std:
1555
+ value: 0.0
1556
+ step: 7080
1557
+ walltime: null
1558
+ evaluator_step/reward_std:
1559
+ value: 0.0
1560
+ step: 80045
1561
+ walltime: null
1562
+ evaluator_iter/reward_max:
1563
+ value: -1.0
1564
+ step: 7080
1565
+ walltime: null
1566
+ evaluator_step/reward_max:
1567
+ value: -1.0
1568
+ step: 80045
1569
+ walltime: null
1570
+ evaluator_iter/reward_min:
1571
+ value: -1.0
1572
+ step: 7080
1573
+ walltime: null
1574
+ evaluator_step/reward_min:
1575
+ value: -1.0
1576
+ step: 80045
1577
+ walltime: null
1578
+ evaluator_iter/eval_episode_return_mean:
1579
+ value: -1.0
1580
+ step: 7080
1581
+ walltime: null
1582
+ evaluator_step/eval_episode_return_mean:
1583
+ value: -1.0
1584
+ step: 80045
1585
+ walltime: null
1586
+ Buffer/Task_0/num_collected_episodes:
1587
+ value: 3036
1588
+ step: 7580
1589
+ walltime: null
1590
+ Buffer/Task_0/num_game_segments:
1591
+ value: 1795
1592
+ step: 7580
1593
+ walltime: null
1594
+ Buffer/Task_0/num_transitions:
1595
+ value: 49996
1596
+ step: 7580
1597
+ walltime: null
1598
+ Buffer/Task_0/memory_usage_mb/game_segment_buffer:
1599
+ value: 31.115081787109375
1600
+ step: 7580
1601
+ walltime: null
1602
+ Buffer/Task_0/memory_usage_mb/process:
1603
+ value: 2077.03515625
1604
+ step: 7580
1605
+ walltime: null
1606
+ learner_iter/collect_mcts_temperature_avg:
1607
+ value: 0.25
1608
+ step: 7500
1609
+ walltime: null
1610
+ learner_step/collect_mcts_temperature_avg:
1611
+ value: 0.25
1612
+ step: 84336
1613
+ walltime: null
1614
+ learner_iter/cur_lr_avg:
1615
+ value: 0.0029999999999999996
1616
+ step: 7500
1617
+ walltime: null
1618
+ learner_step/cur_lr_avg:
1619
+ value: 0.0029999999999999996
1620
+ step: 84336
1621
+ walltime: null
1622
+ learner_iter/weighted_total_loss_avg:
1623
+ value: 1.1162791360508313
1624
+ step: 7500
1625
+ walltime: null
1626
+ learner_step/weighted_total_loss_avg:
1627
+ value: 1.1162791360508313
1628
+ step: 84336
1629
+ walltime: null
1630
+ learner_iter/total_loss_avg:
1631
+ value: 1.1162791360508313
1632
+ step: 7500
1633
+ walltime: null
1634
+ learner_step/total_loss_avg:
1635
+ value: 1.1162791360508313
1636
+ step: 84336
1637
+ walltime: null
1638
+ learner_iter/policy_loss_avg:
1639
+ value: 9.150714440779252
1640
+ step: 7500
1641
+ walltime: null
1642
+ learner_step/policy_loss_avg:
1643
+ value: 9.150714440779252
1644
+ step: 84336
1645
+ walltime: null
1646
+ learner_iter/policy_entropy_avg:
1647
+ value: 1.615344033096776
1648
+ step: 7500
1649
+ walltime: null
1650
+ learner_step/policy_entropy_avg:
1651
+ value: 1.615344033096776
1652
+ step: 84336
1653
+ walltime: null
1654
+ learner_iter/target_policy_entropy_avg:
1655
+ value: 1.5126509810938977
1656
+ step: 7500
1657
+ walltime: null
1658
+ learner_step/target_policy_entropy_avg:
1659
+ value: 1.5126509810938977
1660
+ step: 84336
1661
+ walltime: null
1662
+ learner_iter/value_prefix_loss_avg:
1663
+ value: 0.22831940650939941
1664
+ step: 7500
1665
+ walltime: null
1666
+ learner_step/value_prefix_loss_avg:
1667
+ value: 0.22831940650939941
1668
+ step: 84336
1669
+ walltime: null
1670
+ learner_iter/value_loss_avg:
1671
+ value: 5.274801861156117
1672
+ step: 7500
1673
+ walltime: null
1674
+ learner_step/value_loss_avg:
1675
+ value: 5.274801861156117
1676
+ step: 84336
1677
+ walltime: null
1678
+ learner_iter/consistency_loss_avg:
1679
+ value: -0.9581454797224566
1680
+ step: 7500
1681
+ walltime: null
1682
+ learner_step/consistency_loss_avg:
1683
+ value: -0.9581454797224566
1684
+ step: 84336
1685
+ walltime: null
1686
+ learner_iter/value_priority_avg:
1687
+ value: 0.818570922721516
1688
+ step: 7500
1689
+ walltime: null
1690
+ learner_step/value_priority_avg:
1691
+ value: 0.818570922721516
1692
+ step: 84336
1693
+ walltime: null
1694
+ learner_iter/target_value_prefix_avg:
1695
+ value: 0.05906723676757379
1696
+ step: 7500
1697
+ walltime: null
1698
+ learner_step/target_value_prefix_avg:
1699
+ value: 0.05906723676757379
1700
+ step: 84336
1701
+ walltime: null
1702
+ learner_iter/target_value_avg:
1703
+ value: 0.0010653409514237535
1704
+ step: 7500
1705
+ walltime: null
1706
+ learner_step/target_value_avg:
1707
+ value: 0.0010653409514237535
1708
+ step: 84336
1709
+ walltime: null
1710
+ learner_iter/predicted_value_prefixs_avg:
1711
+ value: 0.049273560162295
1712
+ step: 7500
1713
+ walltime: null
1714
+ learner_step/predicted_value_prefixs_avg:
1715
+ value: 0.049273560162295
1716
+ step: 84336
1717
+ walltime: null
1718
+ learner_iter/predicted_values_avg:
1719
+ value: -0.001351071257059547
1720
+ step: 7500
1721
+ walltime: null
1722
+ learner_step/predicted_values_avg:
1723
+ value: -0.001351071257059547
1724
+ step: 84336
1725
+ walltime: null
1726
+ learner_iter/transformed_target_value_prefix_avg:
1727
+ value: 0.02452551421116699
1728
+ step: 7500
1729
+ walltime: null
1730
+ learner_step/transformed_target_value_prefix_avg:
1731
+ value: 0.02452551421116699
1732
+ step: 84336
1733
+ walltime: null
1734
+ learner_iter/transformed_target_value_avg:
1735
+ value: 0.0004423439650881019
1736
+ step: 7500
1737
+ walltime: null
1738
+ learner_step/transformed_target_value_avg:
1739
+ value: 0.0004423439650881019
1740
+ step: 84336
1741
+ walltime: null
1742
+ learner_iter/total_grad_norm_before_clip_avg:
1743
+ value: 0.45741088823838666
1744
+ step: 7500
1745
+ walltime: null
1746
+ learner_step/total_grad_norm_before_clip_avg:
1747
+ value: 0.45741088823838666
1748
+ step: 84336
1749
+ walltime: null
1750
+ collector_iter/episode_count:
1751
+ value: 40
1752
+ step: 7500
1753
+ walltime: null
1754
+ collector_step/episode_count:
1755
+ value: 40
1756
+ step: 84336
1757
+ walltime: null
1758
+ collector_iter/envstep_count:
1759
+ value: 1013
1760
+ step: 7500
1761
+ walltime: null
1762
+ collector_step/envstep_count:
1763
+ value: 1013
1764
+ step: 84336
1765
+ walltime: null
1766
+ collector_iter/avg_envstep_per_episode:
1767
+ value: 25.325
1768
+ step: 7500
1769
+ walltime: null
1770
+ collector_step/avg_envstep_per_episode:
1771
+ value: 25.325
1772
+ step: 84336
1773
+ walltime: null
1774
+ collector_iter/avg_envstep_per_sec:
1775
+ value: 9.130103441466435
1776
+ step: 7500
1777
+ walltime: null
1778
+ collector_step/avg_envstep_per_sec:
1779
+ value: 9.130103441466435
1780
+ step: 84336
1781
+ walltime: null
1782
+ collector_iter/avg_episode_per_sec:
1783
+ value: 0.3605174113116065
1784
+ step: 7500
1785
+ walltime: null
1786
+ collector_step/avg_episode_per_sec:
1787
+ value: 0.3605174113116065
1788
+ step: 84336
1789
+ walltime: null
1790
+ collector_iter/collect_time:
1791
+ value: 110.95164545444588
1792
+ step: 7500
1793
+ walltime: null
1794
+ collector_step/collect_time:
1795
+ value: 110.95164545444588
1796
+ step: 84336
1797
+ walltime: null
1798
+ collector_iter/reward_mean:
1799
+ value: 0.375
1800
+ step: 7500
1801
+ walltime: null
1802
+ collector_step/reward_mean:
1803
+ value: 0.375
1804
+ step: 84336
1805
+ walltime: null
1806
+ collector_iter/reward_std:
1807
+ value: 0.9134413003921509
1808
+ step: 7500
1809
+ walltime: null
1810
+ collector_step/reward_std:
1811
+ value: 0.9134413003921509
1812
+ step: 84336
1813
+ walltime: null
1814
+ collector_iter/reward_max:
1815
+ value: 1.0
1816
+ step: 7500
1817
+ walltime: null
1818
+ collector_step/reward_max:
1819
+ value: 1.0
1820
+ step: 84336
1821
+ walltime: null
1822
+ collector_iter/reward_min:
1823
+ value: -1.0
1824
+ step: 7500
1825
+ walltime: null
1826
+ collector_step/reward_min:
1827
+ value: -1.0
1828
+ step: 84336
1829
+ walltime: null
1830
+ collector_iter/total_envstep_count:
1831
+ value: 84336
1832
+ step: 7500
1833
+ walltime: null
1834
+ collector_step/total_envstep_count:
1835
+ value: 84336
1836
+ step: 84336
1837
+ walltime: null
1838
+ collector_iter/total_episode_count:
1839
+ value: 3008
1840
+ step: 7500
1841
+ walltime: null
1842
+ collector_step/total_episode_count:
1843
+ value: 3008
1844
+ step: 84336
1845
+ walltime: null
1846
+ collector_iter/total_duration:
1847
+ value: 47114.3674302446
1848
+ step: 7500
1849
+ walltime: null
1850
+ collector_step/total_duration:
1851
+ value: 47114.3674302446
1852
+ step: 84336
1853
+ walltime: null
1854
+ collector_iter/visit_entropy_mean:
1855
+ value: 1.27042619265444
1856
+ step: 7500
1857
+ walltime: null
1858
+ collector_step/visit_entropy_mean:
1859
+ value: 1.27042619265444
1860
+ step: 84336
1861
+ walltime: null
1862
+ envstep_90000.pth.tar:
1863
+ checkpoint_name: envstep_90000.pth.tar
1864
+ checkpoint_path: /workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459/ckpt/envstep_90000.pth.tar
1865
+ saved_at: '2026-05-19T15:02:38.872182+00:00'
1866
+ train_iter: 8050
1867
+ envstep: 90000
1868
+ trigger: envstep
1869
+ latest_scalars:
1870
+ evaluator_iter/episode_count:
1871
+ value: 10
1872
+ step: 7590
1873
+ walltime: null
1874
+ evaluator_step/episode_count:
1875
+ value: 10
1876
+ step: 85111
1877
+ walltime: null
1878
+ evaluator_iter/envstep_count:
1879
+ value: 55
1880
+ step: 7590
1881
+ walltime: null
1882
+ evaluator_step/envstep_count:
1883
+ value: 55
1884
+ step: 85111
1885
+ walltime: null
1886
+ evaluator_iter/avg_envstep_per_episode:
1887
+ value: 5.5
1888
+ step: 7590
1889
+ walltime: null
1890
+ evaluator_step/avg_envstep_per_episode:
1891
+ value: 5.5
1892
+ step: 85111
1893
+ walltime: null
1894
+ evaluator_iter/evaluate_time:
1895
+ value: 5.41702392578125
1896
+ step: 7590
1897
+ walltime: null
1898
+ evaluator_step/evaluate_time:
1899
+ value: 5.41702392578125
1900
+ step: 85111
1901
+ walltime: null
1902
+ evaluator_iter/avg_envstep_per_sec:
1903
+ value: 10.15317649572091
1904
+ step: 7590
1905
+ walltime: null
1906
+ evaluator_step/avg_envstep_per_sec:
1907
+ value: 10.15317649572091
1908
+ step: 85111
1909
+ walltime: null
1910
+ evaluator_iter/avg_time_per_episode:
1911
+ value: 1.8460320901310745
1912
+ step: 7590
1913
+ walltime: null
1914
+ evaluator_step/avg_time_per_episode:
1915
+ value: 1.8460320901310745
1916
+ step: 85111
1917
+ walltime: null
1918
+ evaluator_iter/reward_mean:
1919
+ value: -1.0
1920
+ step: 7590
1921
+ walltime: null
1922
+ evaluator_step/reward_mean:
1923
+ value: -1.0
1924
+ step: 85111
1925
+ walltime: null
1926
+ evaluator_iter/reward_std:
1927
+ value: 0.0
1928
+ step: 7590
1929
+ walltime: null
1930
+ evaluator_step/reward_std:
1931
+ value: 0.0
1932
+ step: 85111
1933
+ walltime: null
1934
+ evaluator_iter/reward_max:
1935
+ value: -1.0
1936
+ step: 7590
1937
+ walltime: null
1938
+ evaluator_step/reward_max:
1939
+ value: -1.0
1940
+ step: 85111
1941
+ walltime: null
1942
+ evaluator_iter/reward_min:
1943
+ value: -1.0
1944
+ step: 7590
1945
+ walltime: null
1946
+ evaluator_step/reward_min:
1947
+ value: -1.0
1948
+ step: 85111
1949
+ walltime: null
1950
+ evaluator_iter/eval_episode_return_mean:
1951
+ value: -1.0
1952
+ step: 7590
1953
+ walltime: null
1954
+ evaluator_step/eval_episode_return_mean:
1955
+ value: -1.0
1956
+ step: 85111
1957
+ walltime: null
1958
+ Buffer/Task_0/num_collected_episodes:
1959
+ value: 3224
1960
+ step: 8050
1961
+ walltime: null
1962
+ Buffer/Task_0/num_game_segments:
1963
+ value: 1800
1964
+ step: 8050
1965
+ walltime: null
1966
+ Buffer/Task_0/num_transitions:
1967
+ value: 49996
1968
+ step: 8050
1969
+ walltime: null
1970
+ Buffer/Task_0/memory_usage_mb/game_segment_buffer:
1971
+ value: 31.134323120117188
1972
+ step: 8050
1973
+ walltime: null
1974
+ Buffer/Task_0/memory_usage_mb/process:
1975
+ value: 2076.12109375
1976
+ step: 8050
1977
+ walltime: null
1978
+ learner_iter/collect_mcts_temperature_avg:
1979
+ value: 0.25
1980
+ step: 8000
1981
+ walltime: null
1982
+ learner_step/collect_mcts_temperature_avg:
1983
+ value: 0.25
1984
+ step: 89525
1985
+ walltime: null
1986
+ learner_iter/cur_lr_avg:
1987
+ value: 0.0029999999999999996
1988
+ step: 8000
1989
+ walltime: null
1990
+ learner_step/cur_lr_avg:
1991
+ value: 0.0029999999999999996
1992
+ step: 89525
1993
+ walltime: null
1994
+ learner_iter/weighted_total_loss_avg:
1995
+ value: 1.1734138293699785
1996
+ step: 8000
1997
+ walltime: null
1998
+ learner_step/weighted_total_loss_avg:
1999
+ value: 1.1734138293699785
2000
+ step: 89525
2001
+ walltime: null
2002
+ learner_iter/total_loss_avg:
2003
+ value: 1.1734138293699785
2004
+ step: 8000
2005
+ walltime: null
2006
+ learner_step/total_loss_avg:
2007
+ value: 1.1734138293699785
2008
+ step: 89525
2009
+ walltime: null
2010
+ learner_iter/policy_loss_avg:
2011
+ value: 9.202515255321156
2012
+ step: 8000
2013
+ walltime: null
2014
+ learner_step/policy_loss_avg:
2015
+ value: 9.202515255321156
2016
+ step: 89525
2017
+ walltime: null
2018
+ learner_iter/policy_entropy_avg:
2019
+ value: 1.6213162884567722
2020
+ step: 8000
2021
+ walltime: null
2022
+ learner_step/policy_entropy_avg:
2023
+ value: 1.6213162884567722
2024
+ step: 89525
2025
+ walltime: null
2026
+ learner_iter/target_policy_entropy_avg:
2027
+ value: 1.5226185683048135
2028
+ step: 8000
2029
+ walltime: null
2030
+ learner_step/target_policy_entropy_avg:
2031
+ value: 1.5226185683048135
2032
+ step: 89525
2033
+ walltime: null
2034
+ learner_iter/value_prefix_loss_avg:
2035
+ value: 0.2290004247968847
2036
+ step: 8000
2037
+ walltime: null
2038
+ learner_step/value_prefix_loss_avg:
2039
+ value: 0.2290004247968847
2040
+ step: 89525
2041
+ walltime: null
2042
+ learner_iter/value_loss_avg:
2043
+ value: 5.280927354639227
2044
+ step: 8000
2045
+ walltime: null
2046
+ learner_step/value_loss_avg:
2047
+ value: 5.280927354639227
2048
+ step: 89525
2049
+ walltime: null
2050
+ learner_iter/consistency_loss_avg:
2051
+ value: -0.9578333854675293
2052
+ step: 8000
2053
+ walltime: null
2054
+ learner_step/consistency_loss_avg:
2055
+ value: -0.9578333854675293
2056
+ step: 89525
2057
+ walltime: null
2058
+ learner_iter/value_priority_avg:
2059
+ value: 0.8299497799439863
2060
+ step: 8000
2061
+ walltime: null
2062
+ learner_step/value_priority_avg:
2063
+ value: 0.8299497799439863
2064
+ step: 89525
2065
+ walltime: null
2066
+ learner_iter/target_value_prefix_avg:
2067
+ value: 0.06202651695771651
2068
+ step: 8000
2069
+ walltime: null
2070
+ learner_step/target_value_prefix_avg:
2071
+ value: 0.06202651695771651
2072
+ step: 89525
2073
+ walltime: null
2074
+ learner_iter/target_value_avg:
2075
+ value: 0.00011837120506573807
2076
+ step: 8000
2077
+ walltime: null
2078
+ learner_step/target_value_avg:
2079
+ value: 0.00011837120506573807
2080
+ step: 89525
2081
+ walltime: null
2082
+ learner_iter/predicted_value_prefixs_avg:
2083
+ value: 0.05285426567901264
2084
+ step: 8000
2085
+ walltime: null
2086
+ learner_step/predicted_value_prefixs_avg:
2087
+ value: 0.05285426567901264
2088
+ step: 89525
2089
+ walltime: null
2090
+ learner_iter/predicted_values_avg:
2091
+ value: -0.0015143624815126796
2092
+ step: 8000
2093
+ walltime: null
2094
+ learner_step/predicted_values_avg:
2095
+ value: -0.0015143624815126796
2096
+ step: 89525
2097
+ walltime: null
2098
+ learner_iter/transformed_target_value_prefix_avg:
2099
+ value: 0.025754247707399456
2100
+ step: 8000
2101
+ walltime: null
2102
+ learner_step/transformed_target_value_prefix_avg:
2103
+ value: 0.025754247707399456
2104
+ step: 89525
2105
+ walltime: null
2106
+ learner_iter/transformed_target_value_avg:
2107
+ value: 4.914934238926931e-05
2108
+ step: 8000
2109
+ walltime: null
2110
+ learner_step/transformed_target_value_avg:
2111
+ value: 4.914934238926931e-05
2112
+ step: 89525
2113
+ walltime: null
2114
+ learner_iter/total_grad_norm_before_clip_avg:
2115
+ value: 0.4895417636091059
2116
+ step: 8000
2117
+ walltime: null
2118
+ learner_step/total_grad_norm_before_clip_avg:
2119
+ value: 0.4895417636091059
2120
+ step: 89525
2121
+ walltime: null
2122
+ collector_iter/episode_count:
2123
+ value: 40
2124
+ step: 8000
2125
+ walltime: null
2126
+ collector_step/episode_count:
2127
+ value: 40
2128
+ step: 89525
2129
+ walltime: null
2130
+ collector_iter/envstep_count:
2131
+ value: 1117
2132
+ step: 8000
2133
+ walltime: null
2134
+ collector_step/envstep_count:
2135
+ value: 1117
2136
+ step: 89525
2137
+ walltime: null
2138
+ collector_iter/avg_envstep_per_episode:
2139
+ value: 27.925
2140
+ step: 8000
2141
+ walltime: null
2142
+ collector_step/avg_envstep_per_episode:
2143
+ value: 27.925
2144
+ step: 89525
2145
+ walltime: null
2146
+ collector_iter/avg_envstep_per_sec:
2147
+ value: 10.071015591584729
2148
+ step: 8000
2149
+ walltime: null
2150
+ collector_step/avg_envstep_per_sec:
2151
+ value: 10.071015591584729
2152
+ step: 89525
2153
+ walltime: null
2154
+ collector_iter/avg_episode_per_sec:
2155
+ value: 0.3606451420442159
2156
+ step: 8000
2157
+ walltime: null
2158
+ collector_step/avg_episode_per_sec:
2159
+ value: 0.3606451420442159
2160
+ step: 89525
2161
+ walltime: null
2162
+ collector_iter/collect_time:
2163
+ value: 110.91234938940592
2164
+ step: 8000
2165
+ walltime: null
2166
+ collector_step/collect_time:
2167
+ value: 110.91234938940592
2168
+ step: 89525
2169
+ walltime: null
2170
+ collector_iter/reward_mean:
2171
+ value: 0.07500000298023224
2172
+ step: 8000
2173
+ walltime: null
2174
+ collector_step/reward_mean:
2175
+ value: 0.07500000298023224
2176
+ step: 89525
2177
+ walltime: null
2178
+ collector_iter/reward_std:
2179
+ value: 0.9845684170722961
2180
+ step: 8000
2181
+ walltime: null
2182
+ collector_step/reward_std:
2183
+ value: 0.9845684170722961
2184
+ step: 89525
2185
+ walltime: null
2186
+ collector_iter/reward_max:
2187
+ value: 1.0
2188
+ step: 8000
2189
+ walltime: null
2190
+ collector_step/reward_max:
2191
+ value: 1.0
2192
+ step: 89525
2193
+ walltime: null
2194
+ collector_iter/reward_min:
2195
+ value: -1.0
2196
+ step: 8000
2197
+ walltime: null
2198
+ collector_step/reward_min:
2199
+ value: -1.0
2200
+ step: 89525
2201
+ walltime: null
2202
+ collector_iter/total_envstep_count:
2203
+ value: 89525
2204
+ step: 8000
2205
+ walltime: null
2206
+ collector_step/total_envstep_count:
2207
+ value: 89525
2208
+ step: 89525
2209
+ walltime: null
2210
+ collector_iter/total_episode_count:
2211
+ value: 3208
2212
+ step: 8000
2213
+ walltime: null
2214
+ collector_step/total_episode_count:
2215
+ value: 3208
2216
+ step: 89525
2217
+ walltime: null
2218
+ collector_iter/total_duration:
2219
+ value: 49916.304257843265
2220
+ step: 8000
2221
+ walltime: null
2222
+ collector_step/total_duration:
2223
+ value: 49916.304257843265
2224
+ step: 89525
2225
+ walltime: null
2226
+ collector_iter/visit_entropy_mean:
2227
+ value: 1.024226404569014
2228
+ step: 8000
2229
+ walltime: null
2230
+ collector_step/visit_entropy_mean:
2231
+ value: 1.024226404569014
2232
+ step: 89525
2233
+ walltime: null
2234
+ metadata_version: 1
metadata/total_config.py ADDED
@@ -0,0 +1,236 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ exp_config = {
2
+ 'env': {
3
+ 'manager': {
4
+ 'episode_num': float("inf"),
5
+ 'max_retry': 1,
6
+ 'step_timeout': None,
7
+ 'auto_reset': True,
8
+ 'reset_timeout': None,
9
+ 'retry_type': 'reset',
10
+ 'retry_waiting_time': 0.1,
11
+ 'shared_memory': False,
12
+ 'copy_on_get': True,
13
+ 'context': 'fork',
14
+ 'wait_num': float("inf"),
15
+ 'step_wait_timeout': None,
16
+ 'connect_timeout': 60,
17
+ 'reset_inplace': False,
18
+ 'cfg_type': 'SyncSubprocessEnvManagerDict',
19
+ 'type': 'subprocess'
20
+ },
21
+ 'stop_value': 2,
22
+ 'n_evaluator_episode': 10,
23
+ 'env_id': 'Connect4',
24
+ 'battle_mode': 'self_play_mode',
25
+ 'battle_mode_in_simulation_env': 'self_play_mode',
26
+ 'render_mode': None,
27
+ 'replay_path': None,
28
+ 'bot_action_type': 'rule',
29
+ 'agent_vs_human': False,
30
+ 'prob_random_agent': 0,
31
+ 'prob_expert_agent': 0,
32
+ 'prob_random_action_in_bot': 0.0,
33
+ 'screen_scaling': 9,
34
+ 'channel_last': False,
35
+ 'scale': False,
36
+ 'cfg_type': 'Connect4EnvDict',
37
+ 'type': 'connect4',
38
+ 'import_names': ['zoo.board_games.connect4.envs.connect4_env'],
39
+ 'collector_env_num': 4,
40
+ 'evaluator_env_num': 10
41
+ },
42
+ 'policy': {
43
+ 'model': {
44
+ 'model_type': 'conv',
45
+ 'continuous_action_space': False,
46
+ 'observation_shape': (3, 6, 7),
47
+ 'self_supervised_learning_loss': True,
48
+ 'categorical_distribution': True,
49
+ 'image_channel': 3,
50
+ 'frame_stack_num': 1,
51
+ 'num_res_blocks': 1,
52
+ 'num_channels': 64,
53
+ 'reward_support_range': (-300.0, 301.0, 1.0),
54
+ 'value_support_range': (-300.0, 301.0, 1.0),
55
+ 'bias': True,
56
+ 'discrete_action_encoding_type': 'one_hot',
57
+ 'res_connection_in_dynamics': True,
58
+ 'norm_type': 'BN',
59
+ 'analysis_sim_norm': False,
60
+ 'analysis_dormant_ratio': False,
61
+ 'harmony_balance': False,
62
+ 'lstm_hidden_size': 512,
63
+ 'action_space_size': 7
64
+ },
65
+ 'learn': {
66
+ 'learner': {
67
+ 'train_iterations': 1000000000,
68
+ 'dataloader': {
69
+ 'num_workers': 0
70
+ },
71
+ 'log_policy': True,
72
+ 'hook': {
73
+ 'load_ckpt_before_run': '',
74
+ 'log_show_after_iter': 100,
75
+ 'save_ckpt_after_iter': 10000,
76
+ 'save_ckpt_after_run': True
77
+ },
78
+ 'cfg_type': 'BaseLearnerDict'
79
+ },
80
+ 'resume_training': False
81
+ },
82
+ 'collect': {
83
+ 'collector': {
84
+ 'deepcopy_obs': False,
85
+ 'transform_obs': False,
86
+ 'collect_print_freq': 100,
87
+ 'cfg_type': 'SampleSerialCollectorDict',
88
+ 'type': 'sample'
89
+ }
90
+ },
91
+ 'eval': {
92
+ 'evaluator': {
93
+ 'eval_freq': 1000,
94
+ 'render': {
95
+ 'render_freq': -1,
96
+ 'mode': 'train_iter'
97
+ },
98
+ 'figure_path': None,
99
+ 'cfg_type': 'InteractionSerialEvaluatorDict',
100
+ 'stop_value': 2,
101
+ 'n_episode': 10
102
+ }
103
+ },
104
+ 'other': {
105
+ 'replay_buffer': {
106
+ 'type': 'advanced',
107
+ 'replay_buffer_size': 4096,
108
+ 'max_use': float("inf"),
109
+ 'max_staleness': float("inf"),
110
+ 'alpha': 0.6,
111
+ 'beta': 0.4,
112
+ 'anneal_step': 100000,
113
+ 'enable_track_used_data': False,
114
+ 'deepcopy': False,
115
+ 'thruput_controller': {
116
+ 'push_sample_rate_limit': {
117
+ 'max': float("inf"),
118
+ 'min': 0
119
+ },
120
+ 'window_seconds': 30,
121
+ 'sample_min_limit_ratio': 1
122
+ },
123
+ 'monitor': {
124
+ 'sampled_data_attr': {
125
+ 'average_range': 5,
126
+ 'print_freq': 200
127
+ },
128
+ 'periodic_thruput': {
129
+ 'seconds': 60
130
+ }
131
+ },
132
+ 'cfg_type': 'AdvancedReplayBufferDict'
133
+ },
134
+ 'commander': {
135
+ 'cfg_type': 'BaseSerialCommanderDict'
136
+ }
137
+ },
138
+ 'on_policy': False,
139
+ 'cuda': True,
140
+ 'multi_gpu': False,
141
+ 'bp_update_sync': True,
142
+ 'traj_len_inf': False,
143
+ 'use_wandb': True,
144
+ 'use_rnd_model': False,
145
+ 'sampled_algo': False,
146
+ 'gumbel_algo': False,
147
+ 'mcts_ctree': True,
148
+ 'collector_env_num': 4,
149
+ 'evaluator_env_num': 10,
150
+ 'env_type': 'board_games',
151
+ 'action_type': 'varied_action_space',
152
+ 'battle_mode': 'self_play_mode',
153
+ 'monitor_extra_statistics': True,
154
+ 'game_segment_length': 42,
155
+ 'eval_offline': False,
156
+ 'calculate_dormant_ratio': False,
157
+ 'analysis_sim_norm': False,
158
+ 'analysis_dormant_ratio': False,
159
+ 'transform2string': False,
160
+ 'gray_scale': False,
161
+ 'use_augmentation': False,
162
+ 'augmentation': ['shift', 'intensity'],
163
+ 'ignore_done': False,
164
+ 'update_per_collect': 10,
165
+ 'replay_ratio': 0.25,
166
+ 'batch_size': 128,
167
+ 'optim_type': 'Adam',
168
+ 'learning_rate': 0.003,
169
+ 'target_update_freq': 100,
170
+ 'target_update_freq_for_intrinsic_reward': 1000,
171
+ 'weight_decay': 0.0001,
172
+ 'momentum': 0.9,
173
+ 'grad_clip_value': 0.5,
174
+ 'n_episode': 4,
175
+ 'num_segments': 8,
176
+ 'num_simulations': 20,
177
+ 'discount_factor': 1,
178
+ 'td_steps': 42,
179
+ 'num_unroll_steps': 5,
180
+ 'reward_loss_weight': 1,
181
+ 'value_loss_weight': 0.25,
182
+ 'policy_loss_weight': 1,
183
+ 'policy_entropy_weight': 0,
184
+ 'ssl_loss_weight': 2,
185
+ 'piecewise_decay_lr_scheduler': False,
186
+ 'threshold_training_steps_for_final_lr': 50000,
187
+ 'manual_temperature_decay': False,
188
+ 'threshold_training_steps_for_final_temperature': 100000,
189
+ 'fixed_temperature_value': 0.25,
190
+ 'use_ture_chance_label_in_chance_encoder': False,
191
+ 'reanalyze_noise': True,
192
+ 'reuse_search': False,
193
+ 'collect_with_pure_policy': False,
194
+ 'use_priority': False,
195
+ 'priority_prob_alpha': 0.6,
196
+ 'priority_prob_beta': 0.4,
197
+ 'root_dirichlet_alpha': 0.3,
198
+ 'root_noise_weight': 0.25,
199
+ 'random_collect_episode_num': 0,
200
+ 'eps': {
201
+ 'eps_greedy_exploration_in_collect': False,
202
+ 'type': 'linear',
203
+ 'start': 1.0,
204
+ 'end': 0.05,
205
+ 'decay': 100000
206
+ },
207
+ 'cfg_type': 'EfficientZeroPolicyDict',
208
+ 'lstm_horizon_len': 5,
209
+ 'type': 'efficientzero',
210
+ 'import_names': ['lzero.policy.efficientzero'],
211
+ 'model_path':
212
+ '/workspace/combinatorial_reasoning_post_training/models/simplified5_immediate_300k_continuation_20260518/round-01/connect4/attempt-01_260518_231735/ckpt/envstep_165000.pth.tar',
213
+ 'reanalyze_ratio': 0.0,
214
+ 'eval_freq': 300001,
215
+ 'replay_buffer_size': 50000,
216
+ 'best_ckpt_strategy': 'raw',
217
+ 'best_ckpt_ema_alpha': 0.3,
218
+ 'best_ckpt_min_episodes': 20,
219
+ 'battle_mode_in_simulation_env': 'self_play_mode',
220
+ 'eval_opponent_type': 'env_bot',
221
+ 'previous_best_checkpoint': {
222
+ 'path': None,
223
+ 'selector': 'best',
224
+ 'update_policy': 'on_new_best',
225
+ 'num_simulations': None,
226
+ 'n_evaluator_episode': None,
227
+ 'evaluator_env_num': None,
228
+ 'promotion_threshold': 0.0,
229
+ 'fallback_to_env_bot': False
230
+ },
231
+ 'device': 'cuda'
232
+ },
233
+ 'exp_name':
234
+ '/workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459',
235
+ 'seed': 0
236
+ }
metadata/upload_manifest.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "checkpoint_mtime_utc": "2026-05-19T15:02:37Z",
3
+ "checkpoint_size_bytes": 139564171,
4
+ "display_name": "Connect4",
5
+ "game_id": "connect4",
6
+ "path_in_repo": "checkpoints/envstep_90000.pth.tar",
7
+ "repo_id": "LorMolf/Connect4-CK",
8
+ "source_attempt_dir": "/workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459",
9
+ "source_checkpoint": "/workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459/ckpt/envstep_90000.pth.tar",
10
+ "source_checkpoint_index": "/workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459/ckpt/checkpoint_index.yaml",
11
+ "source_total_config": "/workspace/combinatorial_reasoning_post_training/models/simplified5_constant_lr_fixed_20260519/round-01/connect4/attempt-01_260519_093459/total_config.py",
12
+ "uploaded_at_utc": "2026-05-19T15:04:12Z",
13
+ "wandb_project": "crpt-simplified5-constant-lr-continuation",
14
+ "wandb_run": "constant_lr_20260519__connect4__a01"
15
+ }