Hugging Face's logo

joyrl
/
joyrl-benchmarks

TensorBoard
Model card Files
xet
Metrics Community
15
johnjim0816 commited on
Commit
acaf9b2
·
1 Parent(s): 43d89c4

update CliffWalking-v0 Sarsa

Browse files
Files changed (38) hide show
  1. ToyText/CliffWalking-v0/Test_CliffWalking-v0_Sarsa_20230311-001530/logs/log.txt +0 -46
  2. ToyText/CliffWalking-v0/Test_CliffWalking-v0_Sarsa_20230311-001530/results/learning_curve.png +0 -0
  3. ToyText/CliffWalking-v0/Test_CliffWalking-v0_Sarsa_20230311-001530/results/res.csv +0 -11
  4. ToyText/CliffWalking-v0/{Test_CliffWalking-v0_Sarsa_20230311-001530 → Test_single_CliffWalking-v0_Sarsa_20230519-010907}/config.yaml +20 -13
  5. ToyText/CliffWalking-v0/Test_single_CliffWalking-v0_Sarsa_20230519-010907/logs/log.txt +52 -0
  6. ToyText/CliffWalking-v0/{Train_CliffWalking-v0_Sarsa_20230311-001504/tb_logs/events.out.tfevents.1678464904.DESKTOP-H34HQIQ.27128.0 → Test_single_CliffWalking-v0_Sarsa_20230519-010907/tb_logs/interact/events.out.tfevents.1684429747.DESKTOP-H34HQIQ.88652.0} +2 -2
  7. ToyText/CliffWalking-v0/{Test_CliffWalking-v0_Sarsa_20230311-001530/tb_logs/events.out.tfevents.1678464930.DESKTOP-H34HQIQ.51560.0 → Test_single_CliffWalking-v0_Sarsa_20230519-010907/tb_logs/model/events.out.tfevents.1684429747.DESKTOP-H34HQIQ.88652.1} +1 -1
  8. ToyText/CliffWalking-v0/Train_CliffWalking-v0_Sarsa_20230311-001504/logs/log.txt +0 -475
  9. ToyText/CliffWalking-v0/Train_CliffWalking-v0_Sarsa_20230311-001504/results/learning_curve.png +0 -0
  10. ToyText/CliffWalking-v0/Train_CliffWalking-v0_Sarsa_20230311-001504/results/res.csv +0 -401
  11. ToyText/CliffWalking-v0/{Train_CliffWalking-v0_Sarsa_20230311-001504 → Train_single_CliffWalking-v0_Sarsa_20230519-010804}/config.yaml +20 -13
  12. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/logs/log.txt +342 -0
  13. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/1000 +0 -0
  14. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/10000 +0 -0
  15. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/10500 +0 -0
  16. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/11000 +0 -0
  17. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/11500 +0 -0
  18. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/12000 +0 -0
  19. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/1500 +0 -0
  20. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/2000 +0 -0
  21. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/2500 +0 -0
  22. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/3000 +0 -0
  23. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/3500 +0 -0
  24. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/4000 +0 -0
  25. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/4500 +0 -0
  26. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/500 +0 -0
  27. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/5000 +0 -0
  28. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/5500 +0 -0
  29. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/6000 +0 -0
  30. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/6500 +0 -0
  31. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/7000 +0 -0
  32. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/7500 +0 -0
  33. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/8000 +0 -0
  34. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/8500 +0 -0
  35. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/9000 +0 -0
  36. ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/9500 +0 -0
  37. ToyText/CliffWalking-v0/{Train_CliffWalking-v0_Sarsa_20230311-001504/models/checkpoint.pkl → Train_single_CliffWalking-v0_Sarsa_20230519-010804/tb_logs/interact/events.out.tfevents.1684429684.DESKTOP-H34HQIQ.87440.0} +2 -2
  38. ToyText/CliffWalking-v0/{Test_CliffWalking-v0_Sarsa_20230311-001530/models/checkpoint.pkl → Train_single_CliffWalking-v0_Sarsa_20230519-010804/tb_logs/model/events.out.tfevents.1684429684.DESKTOP-H34HQIQ.87440.1} +2 -2
ToyText/CliffWalking-v0/Test_CliffWalking-v0_Sarsa_20230311-001530/logs/log.txt DELETED
@@ -1,46 +0,0 @@
1
- 2023-03-11 00:15:30 - r - INFO: - Hyperparameters:
2
- 2023-03-11 00:15:30 - r - INFO: - ================================================================================
3
- 2023-03-11 00:15:30 - r - INFO: - Name Value Type
4
- 2023-03-11 00:15:30 - r - INFO: - env_name CliffWalking-v0 <class 'str'>
5
- 2023-03-11 00:15:30 - r - INFO: - new_step_api 1 <class 'bool'>
6
- 2023-03-11 00:15:30 - r - INFO: - wrapper envs.wrappers.CliffWalkingWapper <class 'str'>
7
- 2023-03-11 00:15:30 - r - INFO: - render 1 <class 'bool'>
8
- 2023-03-11 00:15:30 - r - INFO: - algo_name Sarsa <class 'str'>
9
- 2023-03-11 00:15:30 - r - INFO: - mode test <class 'str'>
10
- 2023-03-11 00:15:30 - r - INFO: - seed 1 <class 'int'>
11
- 2023-03-11 00:15:30 - r - INFO: - device cpu <class 'str'>
12
- 2023-03-11 00:15:30 - r - INFO: - train_eps 300 <class 'int'>
13
- 2023-03-11 00:15:30 - r - INFO: - test_eps 10 <class 'int'>
14
- 2023-03-11 00:15:30 - r - INFO: - eval_eps 10 <class 'int'>
15
- 2023-03-11 00:15:30 - r - INFO: - eval_per_episode 5 <class 'int'>
16
- 2023-03-11 00:15:30 - r - INFO: - max_steps 100 <class 'int'>
17
- 2023-03-11 00:15:30 - r - INFO: - load_checkpoint 1 <class 'bool'>
18
- 2023-03-11 00:15:30 - r - INFO: - load_path Train_CliffWalking-v0_Sarsa_20230311-001504 <class 'str'>
19
- 2023-03-11 00:15:30 - r - INFO: - show_fig 0 <class 'bool'>
20
- 2023-03-11 00:15:30 - r - INFO: - save_fig 1 <class 'bool'>
21
- 2023-03-11 00:15:30 - r - INFO: - render_mode human <class 'str'>
22
- 2023-03-11 00:15:30 - r - INFO: - epsilon_start 0.95 <class 'float'>
23
- 2023-03-11 00:15:30 - r - INFO: - epsilon_end 0.01 <class 'float'>
24
- 2023-03-11 00:15:30 - r - INFO: - epsilon_decay 300 <class 'int'>
25
- 2023-03-11 00:15:30 - r - INFO: - gamma 0.99 <class 'float'>
26
- 2023-03-11 00:15:30 - r - INFO: - lr 0.1 <class 'float'>
27
- 2023-03-11 00:15:30 - r - INFO: - task_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Test_CliffWalking-v0_Sarsa_20230311-001530 <class 'str'>
28
- 2023-03-11 00:15:30 - r - INFO: - res_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Test_CliffWalking-v0_Sarsa_20230311-001530/results <class 'str'>
29
- 2023-03-11 00:15:30 - r - INFO: - log_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Test_CliffWalking-v0_Sarsa_20230311-001530/logs <class 'str'>
30
- 2023-03-11 00:15:30 - r - INFO: - traj_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Test_CliffWalking-v0_Sarsa_20230311-001530/traj <class 'str'>
31
- 2023-03-11 00:15:30 - r - INFO: - tb_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Test_CliffWalking-v0_Sarsa_20230311-001530/tb_logs <class 'str'>
32
- 2023-03-11 00:15:30 - r - INFO: - ================================================================================
33
- 2023-03-11 00:15:31 - r - INFO: - n_states: 48, n_actions: 4
34
- 2023-03-11 00:15:31 - r - INFO: - Start testing!
35
- 2023-03-11 00:15:31 - r - INFO: - Env: CliffWalking-v0, Algorithm: Sarsa, Device: cpu
36
- 2023-03-11 00:15:38 - r - INFO: - Episode: 1/10, Reward: -13.000, Step: 13
37
- 2023-03-11 00:15:42 - r - INFO: - Episode: 2/10, Reward: -13.000, Step: 13
38
- 2023-03-11 00:15:45 - r - INFO: - Episode: 3/10, Reward: -13.000, Step: 13
39
- 2023-03-11 00:15:49 - r - INFO: - Episode: 4/10, Reward: -13.000, Step: 13
40
- 2023-03-11 00:15:52 - r - INFO: - Episode: 5/10, Reward: -13.000, Step: 13
41
- 2023-03-11 00:15:56 - r - INFO: - Episode: 6/10, Reward: -13.000, Step: 13
42
- 2023-03-11 00:15:59 - r - INFO: - Episode: 7/10, Reward: -13.000, Step: 13
43
- 2023-03-11 00:16:03 - r - INFO: - Episode: 8/10, Reward: -13.000, Step: 13
44
- 2023-03-11 00:16:06 - r - INFO: - Episode: 9/10, Reward: -13.000, Step: 13
45
- 2023-03-11 00:16:10 - r - INFO: - Episode: 10/10, Reward: -13.000, Step: 13
46
- 2023-03-11 00:16:10 - r - INFO: - Finish testing!
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ToyText/CliffWalking-v0/Test_CliffWalking-v0_Sarsa_20230311-001530/results/learning_curve.png DELETED
Binary file (22.4 kB)
 
ToyText/CliffWalking-v0/Test_CliffWalking-v0_Sarsa_20230311-001530/results/res.csv DELETED
@@ -1,11 +0,0 @@
1
- episodes,rewards,steps
2
- 0,-13,13
3
- 1,-13,13
4
- 2,-13,13
5
- 3,-13,13
6
- 4,-13,13
7
- 5,-13,13
8
- 6,-13,13
9
- 7,-13,13
10
- 8,-13,13
11
- 9,-13,13
 
 
 
 
 
 
 
 
 
 
 
 
ToyText/CliffWalking-v0/{Test_CliffWalking-v0_Sarsa_20230311-001530 → Test_single_CliffWalking-v0_Sarsa_20230519-010907}/config.yaml RENAMED
@@ -1,25 +1,32 @@
1
  general_cfg:
2
  algo_name: Sarsa
 
3
  device: cpu
4
- env_name: CliffWalking-v0
5
- eval_eps: 10
6
- eval_per_episode: 5
7
  load_checkpoint: true
8
- load_path: Train_CliffWalking-v0_Sarsa_20230311-001504
9
- max_steps: 100
 
 
10
  mode: test
11
- new_step_api: true
12
- render: true
13
- render_mode: human
14
- save_fig: true
 
 
15
  seed: 1
16
- show_fig: false
17
- test_eps: 10
18
- train_eps: 300
19
- wrapper: envs.wrappers.CliffWalkingWapper
20
  algo_cfg:
21
  epsilon_decay: 300
22
  epsilon_end: 0.01
23
  epsilon_start: 0.95
24
  gamma: 0.99
25
  lr: 0.1
 
 
 
 
 
 
 
 
1
  general_cfg:
2
  algo_name: Sarsa
3
+ collect_traj: false
4
  device: cpu
5
+ env_name: gym
 
 
6
  load_checkpoint: true
7
+ load_model_step: 12000
8
+ load_path: Train_single_CliffWalking-v0_Sarsa_20230519-010804
9
+ max_episode: 10
10
+ max_step: 100
11
  mode: test
12
+ model_save_fre: 500
13
+ mp_backend: single
14
+ n_learners: 1
15
+ n_workers: 1
16
+ online_eval: false
17
+ online_eval_episode: 10
18
  seed: 1
19
+ share_buffer: true
 
 
 
20
  algo_cfg:
21
  epsilon_decay: 300
22
  epsilon_end: 0.01
23
  epsilon_start: 0.95
24
  gamma: 0.99
25
  lr: 0.1
26
+ env_cfg:
27
+ id: CliffWalking-v0
28
+ ignore_params:
29
+ - wrapper
30
+ - ignore_params
31
+ render_mode: null
32
+ wrapper: null
ToyText/CliffWalking-v0/Test_single_CliffWalking-v0_Sarsa_20230519-010907/logs/log.txt ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - General Configs:
2
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - ================================================================================
3
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - Name Value Type
4
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - env_name gym <class 'str'>
5
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - algo_name Sarsa <class 'str'>
6
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - mode test <class 'str'>
7
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - device cpu <class 'str'>
8
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - seed 1 <class 'int'>
9
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - max_episode 10 <class 'int'>
10
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - max_step 100 <class 'int'>
11
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - collect_traj 0 <class 'bool'>
12
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - mp_backend single <class 'str'>
13
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - n_workers 1 <class 'int'>
14
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - n_learners 1 <class 'int'>
15
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - share_buffer 1 <class 'bool'>
16
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - online_eval 0 <class 'bool'>
17
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'>
18
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - model_save_fre 500 <class 'int'>
19
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - load_checkpoint 1 <class 'bool'>
20
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - load_path Train_single_CliffWalking-v0_Sarsa_20230519-010804 <class 'str'>
21
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - load_model_step 12000 <class 'int'>
22
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - ================================================================================
23
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - Algo Configs:
24
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - ================================================================================
25
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - Name Value Type
26
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - epsilon_start 0.95 <class 'float'>
27
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - epsilon_end 0.01 <class 'float'>
28
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - epsilon_decay 300 <class 'int'>
29
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - gamma 0.99 <class 'float'>
30
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - lr 0.1 <class 'float'>
31
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - ================================================================================
32
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - Env Configs:
33
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - ================================================================================
34
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - Name Value Type
35
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - id CliffWalking-v0 <class 'str'>
36
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - render_mode None <class 'str'>
37
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - wrapper None <class 'str'>
38
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'>
39
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - ================================================================================
40
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - obs_space: Discrete(48), n_actions: Discrete(4)
41
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - Start testing!
42
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - episode: 0, ep_reward: -13, ep_step: 13
43
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - episode: 1, ep_reward: -13, ep_step: 13
44
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - episode: 2, ep_reward: -13, ep_step: 13
45
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - episode: 3, ep_reward: -13, ep_step: 13
46
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - episode: 4, ep_reward: -13, ep_step: 13
47
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - episode: 5, ep_reward: -13, ep_step: 13
48
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - episode: 6, ep_reward: -13, ep_step: 13
49
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - episode: 7, ep_reward: -13, ep_step: 13
50
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - episode: 8, ep_reward: -13, ep_step: 13
51
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - episode: 9, ep_reward: -13, ep_step: 13
52
+ 2023-05-19 01:09:07 - SimpleLog - INFO: - Finish testing! total time consumed: 0.03s
ToyText/CliffWalking-v0/{Train_CliffWalking-v0_Sarsa_20230311-001504/tb_logs/events.out.tfevents.1678464904.DESKTOP-H34HQIQ.27128.0 → Test_single_CliffWalking-v0_Sarsa_20230519-010907/tb_logs/interact/events.out.tfevents.1684429747.DESKTOP-H34HQIQ.88652.0} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:406bc6b455783f40d4d9b3be306d06875ce10f5931c052b9b696a29baff18a63
3
- size 40
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abcc228bd876c82e1c1891506fe5699a275a2109e7c58ca6cbd9fbc317de514b
3
+ size 750
ToyText/CliffWalking-v0/{Test_CliffWalking-v0_Sarsa_20230311-001530/tb_logs/events.out.tfevents.1678464930.DESKTOP-H34HQIQ.51560.0 → Test_single_CliffWalking-v0_Sarsa_20230519-010907/tb_logs/model/events.out.tfevents.1684429747.DESKTOP-H34HQIQ.88652.1} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c2e83cf950f12dc53dd89a40339f0b381110e98b72b10b044c3d17d454d4301a
3
  size 40
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:992abe939d5e30fa284031fba59c9536bfae2eab4660e9bf3469a75f71614fb9
3
  size 40
ToyText/CliffWalking-v0/Train_CliffWalking-v0_Sarsa_20230311-001504/logs/log.txt DELETED
@@ -1,475 +0,0 @@
1
- 2023-03-11 00:15:04 - r - INFO: - Hyperparameters:
2
- 2023-03-11 00:15:04 - r - INFO: - ================================================================================
3
- 2023-03-11 00:15:04 - r - INFO: - Name Value Type
4
- 2023-03-11 00:15:04 - r - INFO: - env_name CliffWalking-v0 <class 'str'>
5
- 2023-03-11 00:15:04 - r - INFO: - new_step_api 1 <class 'bool'>
6
- 2023-03-11 00:15:04 - r - INFO: - wrapper envs.wrappers.CliffWalkingWapper <class 'str'>
7
- 2023-03-11 00:15:04 - r - INFO: - render 0 <class 'bool'>
8
- 2023-03-11 00:15:04 - r - INFO: - algo_name Sarsa <class 'str'>
9
- 2023-03-11 00:15:04 - r - INFO: - mode train <class 'str'>
10
- 2023-03-11 00:15:04 - r - INFO: - seed 1 <class 'int'>
11
- 2023-03-11 00:15:04 - r - INFO: - device cpu <class 'str'>
12
- 2023-03-11 00:15:04 - r - INFO: - train_eps 400 <class 'int'>
13
- 2023-03-11 00:15:04 - r - INFO: - test_eps 10 <class 'int'>
14
- 2023-03-11 00:15:04 - r - INFO: - eval_eps 10 <class 'int'>
15
- 2023-03-11 00:15:04 - r - INFO: - eval_per_episode 5 <class 'int'>
16
- 2023-03-11 00:15:04 - r - INFO: - max_steps 100 <class 'int'>
17
- 2023-03-11 00:15:04 - r - INFO: - load_checkpoint 0 <class 'bool'>
18
- 2023-03-11 00:15:04 - r - INFO: - load_path Train_CartPole-v1_DQN_20221026-054757 <class 'str'>
19
- 2023-03-11 00:15:04 - r - INFO: - show_fig 0 <class 'bool'>
20
- 2023-03-11 00:15:04 - r - INFO: - save_fig 1 <class 'bool'>
21
- 2023-03-11 00:15:04 - r - INFO: - epsilon_start 0.95 <class 'float'>
22
- 2023-03-11 00:15:04 - r - INFO: - epsilon_end 0.01 <class 'float'>
23
- 2023-03-11 00:15:04 - r - INFO: - epsilon_decay 300 <class 'int'>
24
- 2023-03-11 00:15:04 - r - INFO: - gamma 0.99 <class 'float'>
25
- 2023-03-11 00:15:04 - r - INFO: - lr 0.1 <class 'float'>
26
- 2023-03-11 00:15:04 - r - INFO: - exploration_type e-greedy <class 'str'>
27
- 2023-03-11 00:15:04 - r - INFO: - task_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CliffWalking-v0_Sarsa_20230311-001504 <class 'str'>
28
- 2023-03-11 00:15:04 - r - INFO: - res_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CliffWalking-v0_Sarsa_20230311-001504/results <class 'str'>
29
- 2023-03-11 00:15:04 - r - INFO: - log_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CliffWalking-v0_Sarsa_20230311-001504/logs <class 'str'>
30
- 2023-03-11 00:15:04 - r - INFO: - traj_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CliffWalking-v0_Sarsa_20230311-001504/traj <class 'str'>
31
- 2023-03-11 00:15:04 - r - INFO: - tb_dir C:\Users\24438\Desktop\joyrl-offline/tasks/Train_CliffWalking-v0_Sarsa_20230311-001504/tb_logs <class 'str'>
32
- 2023-03-11 00:15:04 - r - INFO: - ================================================================================
33
- 2023-03-11 00:15:04 - r - INFO: - n_states: 48, n_actions: 4
34
- 2023-03-11 00:15:04 - r - INFO: - Start training!
35
- 2023-03-11 00:15:04 - r - INFO: - Env: CliffWalking-v0, Algorithm: Sarsa, Device: cpu
36
- 2023-03-11 00:15:04 - r - INFO: - Episode: 1/400, Reward: -595.000, Step: 100
37
- 2023-03-11 00:15:04 - r - INFO: - Episode: 2/400, Reward: -397.000, Step: 100
38
- 2023-03-11 00:15:04 - r - INFO: - Episode: 3/400, Reward: -397.000, Step: 100
39
- 2023-03-11 00:15:04 - r - INFO: - Episode: 4/400, Reward: -199.000, Step: 100
40
- 2023-03-11 00:15:04 - r - INFO: - Episode: 5/400, Reward: -100.000, Step: 100
41
- 2023-03-11 00:15:04 - r - INFO: - Current episode 5 has the best eval reward: -100.000
42
- 2023-03-11 00:15:04 - r - INFO: - Episode: 6/400, Reward: -36.000, Step: 36
43
- 2023-03-11 00:15:04 - r - INFO: - Episode: 7/400, Reward: -100.000, Step: 100
44
- 2023-03-11 00:15:04 - r - INFO: - Episode: 8/400, Reward: -199.000, Step: 100
45
- 2023-03-11 00:15:04 - r - INFO: - Episode: 9/400, Reward: -199.000, Step: 100
46
- 2023-03-11 00:15:04 - r - INFO: - Episode: 10/400, Reward: -199.000, Step: 100
47
- 2023-03-11 00:15:04 - r - INFO: - Current episode 10 has the best eval reward: -100.000
48
- 2023-03-11 00:15:04 - r - INFO: - Episode: 11/400, Reward: -199.000, Step: 100
49
- 2023-03-11 00:15:04 - r - INFO: - Episode: 12/400, Reward: -100.000, Step: 100
50
- 2023-03-11 00:15:04 - r - INFO: - Episode: 13/400, Reward: -48.000, Step: 48
51
- 2023-03-11 00:15:04 - r - INFO: - Episode: 14/400, Reward: -100.000, Step: 100
52
- 2023-03-11 00:15:04 - r - INFO: - Episode: 15/400, Reward: -100.000, Step: 100
53
- 2023-03-11 00:15:04 - r - INFO: - Current episode 15 has the best eval reward: -100.000
54
- 2023-03-11 00:15:04 - r - INFO: - Episode: 16/400, Reward: -100.000, Step: 100
55
- 2023-03-11 00:15:04 - r - INFO: - Episode: 17/400, Reward: -46.000, Step: 46
56
- 2023-03-11 00:15:04 - r - INFO: - Episode: 18/400, Reward: -100.000, Step: 100
57
- 2023-03-11 00:15:04 - r - INFO: - Episode: 19/400, Reward: -100.000, Step: 100
58
- 2023-03-11 00:15:04 - r - INFO: - Episode: 20/400, Reward: -64.000, Step: 64
59
- 2023-03-11 00:15:04 - r - INFO: - Current episode 20 has the best eval reward: -100.000
60
- 2023-03-11 00:15:04 - r - INFO: - Episode: 21/400, Reward: -100.000, Step: 100
61
- 2023-03-11 00:15:04 - r - INFO: - Episode: 22/400, Reward: -74.000, Step: 74
62
- 2023-03-11 00:15:04 - r - INFO: - Episode: 23/400, Reward: -89.000, Step: 89
63
- 2023-03-11 00:15:04 - r - INFO: - Episode: 24/400, Reward: -100.000, Step: 100
64
- 2023-03-11 00:15:04 - r - INFO: - Episode: 25/400, Reward: -100.000, Step: 100
65
- 2023-03-11 00:15:04 - r - INFO: - Current episode 25 has the best eval reward: -100.000
66
- 2023-03-11 00:15:04 - r - INFO: - Episode: 26/400, Reward: -68.000, Step: 68
67
- 2023-03-11 00:15:04 - r - INFO: - Episode: 27/400, Reward: -100.000, Step: 100
68
- 2023-03-11 00:15:04 - r - INFO: - Episode: 28/400, Reward: -52.000, Step: 52
69
- 2023-03-11 00:15:04 - r - INFO: - Episode: 29/400, Reward: -86.000, Step: 86
70
- 2023-03-11 00:15:04 - r - INFO: - Episode: 30/400, Reward: -66.000, Step: 66
71
- 2023-03-11 00:15:04 - r - INFO: - Current episode 30 has the best eval reward: -100.000
72
- 2023-03-11 00:15:05 - r - INFO: - Episode: 31/400, Reward: -100.000, Step: 100
73
- 2023-03-11 00:15:05 - r - INFO: - Episode: 32/400, Reward: -45.000, Step: 45
74
- 2023-03-11 00:15:05 - r - INFO: - Episode: 33/400, Reward: -77.000, Step: 77
75
- 2023-03-11 00:15:05 - r - INFO: - Episode: 34/400, Reward: -100.000, Step: 100
76
- 2023-03-11 00:15:05 - r - INFO: - Episode: 35/400, Reward: -56.000, Step: 56
77
- 2023-03-11 00:15:05 - r - INFO: - Current episode 35 has the best eval reward: -100.000
78
- 2023-03-11 00:15:05 - r - INFO: - Episode: 36/400, Reward: -100.000, Step: 100
79
- 2023-03-11 00:15:05 - r - INFO: - Episode: 37/400, Reward: -43.000, Step: 43
80
- 2023-03-11 00:15:05 - r - INFO: - Episode: 38/400, Reward: -81.000, Step: 81
81
- 2023-03-11 00:15:05 - r - INFO: - Episode: 39/400, Reward: -79.000, Step: 79
82
- 2023-03-11 00:15:05 - r - INFO: - Episode: 40/400, Reward: -73.000, Step: 73
83
- 2023-03-11 00:15:05 - r - INFO: - Current episode 40 has the best eval reward: -100.000
84
- 2023-03-11 00:15:05 - r - INFO: - Episode: 41/400, Reward: -100.000, Step: 100
85
- 2023-03-11 00:15:05 - r - INFO: - Episode: 42/400, Reward: -35.000, Step: 35
86
- 2023-03-11 00:15:05 - r - INFO: - Episode: 43/400, Reward: -100.000, Step: 100
87
- 2023-03-11 00:15:05 - r - INFO: - Episode: 44/400, Reward: -82.000, Step: 82
88
- 2023-03-11 00:15:05 - r - INFO: - Episode: 45/400, Reward: -44.000, Step: 44
89
- 2023-03-11 00:15:05 - r - INFO: - Current episode 45 has the best eval reward: -100.000
90
- 2023-03-11 00:15:05 - r - INFO: - Episode: 46/400, Reward: -100.000, Step: 100
91
- 2023-03-11 00:15:05 - r - INFO: - Episode: 47/400, Reward: -42.000, Step: 42
92
- 2023-03-11 00:15:05 - r - INFO: - Episode: 48/400, Reward: -61.000, Step: 61
93
- 2023-03-11 00:15:05 - r - INFO: - Episode: 49/400, Reward: -71.000, Step: 71
94
- 2023-03-11 00:15:05 - r - INFO: - Episode: 50/400, Reward: -96.000, Step: 96
95
- 2023-03-11 00:15:05 - r - INFO: - Current episode 50 has the best eval reward: -100.000
96
- 2023-03-11 00:15:05 - r - INFO: - Episode: 51/400, Reward: -76.000, Step: 76
97
- 2023-03-11 00:15:05 - r - INFO: - Episode: 52/400, Reward: -100.000, Step: 100
98
- 2023-03-11 00:15:05 - r - INFO: - Episode: 53/400, Reward: -43.000, Step: 43
99
- 2023-03-11 00:15:05 - r - INFO: - Episode: 54/400, Reward: -99.000, Step: 99
100
- 2023-03-11 00:15:05 - r - INFO: - Episode: 55/400, Reward: -41.000, Step: 41
101
- 2023-03-11 00:15:05 - r - INFO: - Current episode 55 has the best eval reward: -100.000
102
- 2023-03-11 00:15:05 - r - INFO: - Episode: 56/400, Reward: -100.000, Step: 100
103
- 2023-03-11 00:15:05 - r - INFO: - Episode: 57/400, Reward: -34.000, Step: 34
104
- 2023-03-11 00:15:05 - r - INFO: - Episode: 58/400, Reward: -61.000, Step: 61
105
- 2023-03-11 00:15:05 - r - INFO: - Episode: 59/400, Reward: -55.000, Step: 55
106
- 2023-03-11 00:15:05 - r - INFO: - Episode: 60/400, Reward: -168.000, Step: 69
107
- 2023-03-11 00:15:05 - r - INFO: - Current episode 60 has the best eval reward: -100.000
108
- 2023-03-11 00:15:05 - r - INFO: - Episode: 61/400, Reward: -100.000, Step: 100
109
- 2023-03-11 00:15:05 - r - INFO: - Episode: 62/400, Reward: -76.000, Step: 76
110
- 2023-03-11 00:15:05 - r - INFO: - Episode: 63/400, Reward: -22.000, Step: 22
111
- 2023-03-11 00:15:05 - r - INFO: - Episode: 64/400, Reward: -54.000, Step: 54
112
- 2023-03-11 00:15:05 - r - INFO: - Episode: 65/400, Reward: -54.000, Step: 54
113
- 2023-03-11 00:15:05 - r - INFO: - Current episode 65 has the best eval reward: -100.000
114
- 2023-03-11 00:15:05 - r - INFO: - Episode: 66/400, Reward: -65.000, Step: 65
115
- 2023-03-11 00:15:05 - r - INFO: - Episode: 67/400, Reward: -42.000, Step: 42
116
- 2023-03-11 00:15:05 - r - INFO: - Episode: 68/400, Reward: -67.000, Step: 67
117
- 2023-03-11 00:15:05 - r - INFO: - Episode: 69/400, Reward: -61.000, Step: 61
118
- 2023-03-11 00:15:05 - r - INFO: - Episode: 70/400, Reward: -54.000, Step: 54
119
- 2023-03-11 00:15:05 - r - INFO: - Current episode 70 has the best eval reward: -100.000
120
- 2023-03-11 00:15:05 - r - INFO: - Episode: 71/400, Reward: -42.000, Step: 42
121
- 2023-03-11 00:15:05 - r - INFO: - Episode: 72/400, Reward: -64.000, Step: 64
122
- 2023-03-11 00:15:05 - r - INFO: - Episode: 73/400, Reward: -52.000, Step: 52
123
- 2023-03-11 00:15:05 - r - INFO: - Episode: 74/400, Reward: -45.000, Step: 45
124
- 2023-03-11 00:15:05 - r - INFO: - Episode: 75/400, Reward: -57.000, Step: 57
125
- 2023-03-11 00:15:05 - r - INFO: - Current episode 75 has the best eval reward: -100.000
126
- 2023-03-11 00:15:05 - r - INFO: - Episode: 76/400, Reward: -59.000, Step: 59
127
- 2023-03-11 00:15:05 - r - INFO: - Episode: 77/400, Reward: -67.000, Step: 67
128
- 2023-03-11 00:15:05 - r - INFO: - Episode: 78/400, Reward: -46.000, Step: 46
129
- 2023-03-11 00:15:05 - r - INFO: - Episode: 79/400, Reward: -48.000, Step: 48
130
- 2023-03-11 00:15:05 - r - INFO: - Episode: 80/400, Reward: -63.000, Step: 63
131
- 2023-03-11 00:15:05 - r - INFO: - Current episode 80 has the best eval reward: -100.000
132
- 2023-03-11 00:15:05 - r - INFO: - Episode: 81/400, Reward: -60.000, Step: 60
133
- 2023-03-11 00:15:05 - r - INFO: - Episode: 82/400, Reward: -47.000, Step: 47
134
- 2023-03-11 00:15:05 - r - INFO: - Episode: 83/400, Reward: -34.000, Step: 34
135
- 2023-03-11 00:15:05 - r - INFO: - Episode: 84/400, Reward: -188.000, Step: 89
136
- 2023-03-11 00:15:05 - r - INFO: - Episode: 85/400, Reward: -23.000, Step: 23
137
- 2023-03-11 00:15:05 - r - INFO: - Current episode 85 has the best eval reward: -100.000
138
- 2023-03-11 00:15:05 - r - INFO: - Episode: 86/400, Reward: -35.000, Step: 35
139
- 2023-03-11 00:15:05 - r - INFO: - Episode: 87/400, Reward: -43.000, Step: 43
140
- 2023-03-11 00:15:05 - r - INFO: - Episode: 88/400, Reward: -61.000, Step: 61
141
- 2023-03-11 00:15:05 - r - INFO: - Episode: 89/400, Reward: -39.000, Step: 39
142
- 2023-03-11 00:15:05 - r - INFO: - Episode: 90/400, Reward: -57.000, Step: 57
143
- 2023-03-11 00:15:05 - r - INFO: - Current episode 90 has the best eval reward: -100.000
144
- 2023-03-11 00:15:05 - r - INFO: - Episode: 91/400, Reward: -43.000, Step: 43
145
- 2023-03-11 00:15:05 - r - INFO: - Episode: 92/400, Reward: -56.000, Step: 56
146
- 2023-03-11 00:15:05 - r - INFO: - Episode: 93/400, Reward: -60.000, Step: 60
147
- 2023-03-11 00:15:05 - r - INFO: - Episode: 94/400, Reward: -24.000, Step: 24
148
- 2023-03-11 00:15:05 - r - INFO: - Episode: 95/400, Reward: -61.000, Step: 61
149
- 2023-03-11 00:15:05 - r - INFO: - Current episode 95 has the best eval reward: -100.000
150
- 2023-03-11 00:15:05 - r - INFO: - Episode: 96/400, Reward: -22.000, Step: 22
151
- 2023-03-11 00:15:05 - r - INFO: - Episode: 97/400, Reward: -27.000, Step: 27
152
- 2023-03-11 00:15:05 - r - INFO: - Episode: 98/400, Reward: -73.000, Step: 73
153
- 2023-03-11 00:15:05 - r - INFO: - Episode: 99/400, Reward: -19.000, Step: 19
154
- 2023-03-11 00:15:05 - r - INFO: - Episode: 100/400, Reward: -44.000, Step: 44
155
- 2023-03-11 00:15:05 - r - INFO: - Current episode 100 has the best eval reward: -100.000
156
- 2023-03-11 00:15:05 - r - INFO: - Episode: 101/400, Reward: -55.000, Step: 55
157
- 2023-03-11 00:15:05 - r - INFO: - Episode: 102/400, Reward: -56.000, Step: 56
158
- 2023-03-11 00:15:05 - r - INFO: - Episode: 103/400, Reward: -42.000, Step: 42
159
- 2023-03-11 00:15:05 - r - INFO: - Episode: 104/400, Reward: -48.000, Step: 48
160
- 2023-03-11 00:15:05 - r - INFO: - Episode: 105/400, Reward: -24.000, Step: 24
161
- 2023-03-11 00:15:05 - r - INFO: - Current episode 105 has the best eval reward: -100.000
162
- 2023-03-11 00:15:05 - r - INFO: - Episode: 106/400, Reward: -49.000, Step: 49
163
- 2023-03-11 00:15:05 - r - INFO: - Episode: 107/400, Reward: -42.000, Step: 42
164
- 2023-03-11 00:15:05 - r - INFO: - Episode: 108/400, Reward: -21.000, Step: 21
165
- 2023-03-11 00:15:05 - r - INFO: - Episode: 109/400, Reward: -49.000, Step: 49
166
- 2023-03-11 00:15:05 - r - INFO: - Episode: 110/400, Reward: -25.000, Step: 25
167
- 2023-03-11 00:15:05 - r - INFO: - Current episode 110 has the best eval reward: -100.000
168
- 2023-03-11 00:15:05 - r - INFO: - Episode: 111/400, Reward: -37.000, Step: 37
169
- 2023-03-11 00:15:05 - r - INFO: - Episode: 112/400, Reward: -34.000, Step: 34
170
- 2023-03-11 00:15:05 - r - INFO: - Episode: 113/400, Reward: -64.000, Step: 64
171
- 2023-03-11 00:15:05 - r - INFO: - Episode: 114/400, Reward: -32.000, Step: 32
172
- 2023-03-11 00:15:05 - r - INFO: - Episode: 115/400, Reward: -40.000, Step: 40
173
- 2023-03-11 00:15:05 - r - INFO: - Current episode 115 has the best eval reward: -100.000
174
- 2023-03-11 00:15:05 - r - INFO: - Episode: 116/400, Reward: -29.000, Step: 29
175
- 2023-03-11 00:15:05 - r - INFO: - Episode: 117/400, Reward: -41.000, Step: 41
176
- 2023-03-11 00:15:05 - r - INFO: - Episode: 118/400, Reward: -44.000, Step: 44
177
- 2023-03-11 00:15:05 - r - INFO: - Episode: 119/400, Reward: -29.000, Step: 29
178
- 2023-03-11 00:15:05 - r - INFO: - Episode: 120/400, Reward: -39.000, Step: 39
179
- 2023-03-11 00:15:05 - r - INFO: - Current episode 120 has the best eval reward: -100.000
180
- 2023-03-11 00:15:05 - r - INFO: - Episode: 121/400, Reward: -25.000, Step: 25
181
- 2023-03-11 00:15:05 - r - INFO: - Episode: 122/400, Reward: -43.000, Step: 43
182
- 2023-03-11 00:15:05 - r - INFO: - Episode: 123/400, Reward: -32.000, Step: 32
183
- 2023-03-11 00:15:05 - r - INFO: - Episode: 124/400, Reward: -20.000, Step: 20
184
- 2023-03-11 00:15:05 - r - INFO: - Episode: 125/400, Reward: -45.000, Step: 45
185
- 2023-03-11 00:15:05 - r - INFO: - Current episode 125 has the best eval reward: -100.000
186
- 2023-03-11 00:15:05 - r - INFO: - Episode: 126/400, Reward: -44.000, Step: 44
187
- 2023-03-11 00:15:05 - r - INFO: - Episode: 127/400, Reward: -50.000, Step: 50
188
- 2023-03-11 00:15:05 - r - INFO: - Episode: 128/400, Reward: -21.000, Step: 21
189
- 2023-03-11 00:15:05 - r - INFO: - Episode: 129/400, Reward: -25.000, Step: 25
190
- 2023-03-11 00:15:05 - r - INFO: - Episode: 130/400, Reward: -43.000, Step: 43
191
- 2023-03-11 00:15:05 - r - INFO: - Current episode 130 has the best eval reward: -100.000
192
- 2023-03-11 00:15:05 - r - INFO: - Episode: 131/400, Reward: -48.000, Step: 48
193
- 2023-03-11 00:15:05 - r - INFO: - Episode: 132/400, Reward: -55.000, Step: 55
194
- 2023-03-11 00:15:05 - r - INFO: - Episode: 133/400, Reward: -30.000, Step: 30
195
- 2023-03-11 00:15:05 - r - INFO: - Episode: 134/400, Reward: -25.000, Step: 25
196
- 2023-03-11 00:15:05 - r - INFO: - Episode: 135/400, Reward: -49.000, Step: 49
197
- 2023-03-11 00:15:05 - r - INFO: - Current episode 135 has the best eval reward: -100.000
198
- 2023-03-11 00:15:05 - r - INFO: - Episode: 136/400, Reward: -34.000, Step: 34
199
- 2023-03-11 00:15:05 - r - INFO: - Episode: 137/400, Reward: -41.000, Step: 41
200
- 2023-03-11 00:15:05 - r - INFO: - Episode: 138/400, Reward: -41.000, Step: 41
201
- 2023-03-11 00:15:05 - r - INFO: - Episode: 139/400, Reward: -24.000, Step: 24
202
- 2023-03-11 00:15:05 - r - INFO: - Episode: 140/400, Reward: -48.000, Step: 48
203
- 2023-03-11 00:15:05 - r - INFO: - Current episode 140 has the best eval reward: -100.000
204
- 2023-03-11 00:15:05 - r - INFO: - Episode: 141/400, Reward: -174.000, Step: 75
205
- 2023-03-11 00:15:05 - r - INFO: - Episode: 142/400, Reward: -20.000, Step: 20
206
- 2023-03-11 00:15:05 - r - INFO: - Episode: 143/400, Reward: -39.000, Step: 39
207
- 2023-03-11 00:15:05 - r - INFO: - Episode: 144/400, Reward: -25.000, Step: 25
208
- 2023-03-11 00:15:05 - r - INFO: - Episode: 145/400, Reward: -47.000, Step: 47
209
- 2023-03-11 00:15:05 - r - INFO: - Current episode 145 has the best eval reward: -100.000
210
- 2023-03-11 00:15:05 - r - INFO: - Episode: 146/400, Reward: -47.000, Step: 47
211
- 2023-03-11 00:15:05 - r - INFO: - Episode: 147/400, Reward: -22.000, Step: 22
212
- 2023-03-11 00:15:05 - r - INFO: - Episode: 148/400, Reward: -32.000, Step: 32
213
- 2023-03-11 00:15:05 - r - INFO: - Episode: 149/400, Reward: -49.000, Step: 49
214
- 2023-03-11 00:15:05 - r - INFO: - Episode: 150/400, Reward: -38.000, Step: 38
215
- 2023-03-11 00:15:05 - r - INFO: - Current episode 150 has the best eval reward: -100.000
216
- 2023-03-11 00:15:05 - r - INFO: - Episode: 151/400, Reward: -38.000, Step: 38
217
- 2023-03-11 00:15:05 - r - INFO: - Episode: 152/400, Reward: -37.000, Step: 37
218
- 2023-03-11 00:15:05 - r - INFO: - Episode: 153/400, Reward: -57.000, Step: 57
219
- 2023-03-11 00:15:05 - r - INFO: - Episode: 154/400, Reward: -27.000, Step: 27
220
- 2023-03-11 00:15:05 - r - INFO: - Episode: 155/400, Reward: -38.000, Step: 38
221
- 2023-03-11 00:15:05 - r - INFO: - Current episode 155 has the best eval reward: -15.000
222
- 2023-03-11 00:15:05 - r - INFO: - Episode: 156/400, Reward: -15.000, Step: 15
223
- 2023-03-11 00:15:05 - r - INFO: - Episode: 157/400, Reward: -43.000, Step: 43
224
- 2023-03-11 00:15:05 - r - INFO: - Episode: 158/400, Reward: -35.000, Step: 35
225
- 2023-03-11 00:15:05 - r - INFO: - Episode: 159/400, Reward: -45.000, Step: 45
226
- 2023-03-11 00:15:05 - r - INFO: - Episode: 160/400, Reward: -40.000, Step: 40
227
- 2023-03-11 00:15:05 - r - INFO: - Episode: 161/400, Reward: -35.000, Step: 35
228
- 2023-03-11 00:15:05 - r - INFO: - Episode: 162/400, Reward: -28.000, Step: 28
229
- 2023-03-11 00:15:05 - r - INFO: - Episode: 163/400, Reward: -22.000, Step: 22
230
- 2023-03-11 00:15:05 - r - INFO: - Episode: 164/400, Reward: -26.000, Step: 26
231
- 2023-03-11 00:15:05 - r - INFO: - Episode: 165/400, Reward: -30.000, Step: 30
232
- 2023-03-11 00:15:05 - r - INFO: - Episode: 166/400, Reward: -29.000, Step: 29
233
- 2023-03-11 00:15:05 - r - INFO: - Episode: 167/400, Reward: -35.000, Step: 35
234
- 2023-03-11 00:15:05 - r - INFO: - Episode: 168/400, Reward: -19.000, Step: 19
235
- 2023-03-11 00:15:05 - r - INFO: - Episode: 169/400, Reward: -20.000, Step: 20
236
- 2023-03-11 00:15:05 - r - INFO: - Episode: 170/400, Reward: -20.000, Step: 20
237
- 2023-03-11 00:15:05 - r - INFO: - Episode: 171/400, Reward: -38.000, Step: 38
238
- 2023-03-11 00:15:05 - r - INFO: - Episode: 172/400, Reward: -23.000, Step: 23
239
- 2023-03-11 00:15:05 - r - INFO: - Episode: 173/400, Reward: -35.000, Step: 35
240
- 2023-03-11 00:15:05 - r - INFO: - Episode: 174/400, Reward: -24.000, Step: 24
241
- 2023-03-11 00:15:05 - r - INFO: - Episode: 175/400, Reward: -38.000, Step: 38
242
- 2023-03-11 00:15:05 - r - INFO: - Episode: 176/400, Reward: -24.000, Step: 24
243
- 2023-03-11 00:15:05 - r - INFO: - Episode: 177/400, Reward: -44.000, Step: 44
244
- 2023-03-11 00:15:05 - r - INFO: - Episode: 178/400, Reward: -22.000, Step: 22
245
- 2023-03-11 00:15:05 - r - INFO: - Episode: 179/400, Reward: -27.000, Step: 27
246
- 2023-03-11 00:15:05 - r - INFO: - Episode: 180/400, Reward: -21.000, Step: 21
247
- 2023-03-11 00:15:05 - r - INFO: - Episode: 181/400, Reward: -31.000, Step: 31
248
- 2023-03-11 00:15:05 - r - INFO: - Episode: 182/400, Reward: -26.000, Step: 26
249
- 2023-03-11 00:15:05 - r - INFO: - Episode: 183/400, Reward: -22.000, Step: 22
250
- 2023-03-11 00:15:05 - r - INFO: - Episode: 184/400, Reward: -37.000, Step: 37
251
- 2023-03-11 00:15:05 - r - INFO: - Episode: 185/400, Reward: -25.000, Step: 25
252
- 2023-03-11 00:15:05 - r - INFO: - Episode: 186/400, Reward: -33.000, Step: 33
253
- 2023-03-11 00:15:05 - r - INFO: - Episode: 187/400, Reward: -15.000, Step: 15
254
- 2023-03-11 00:15:05 - r - INFO: - Episode: 188/400, Reward: -31.000, Step: 31
255
- 2023-03-11 00:15:05 - r - INFO: - Episode: 189/400, Reward: -29.000, Step: 29
256
- 2023-03-11 00:15:05 - r - INFO: - Episode: 190/400, Reward: -25.000, Step: 25
257
- 2023-03-11 00:15:05 - r - INFO: - Episode: 191/400, Reward: -18.000, Step: 18
258
- 2023-03-11 00:15:05 - r - INFO: - Episode: 192/400, Reward: -19.000, Step: 19
259
- 2023-03-11 00:15:05 - r - INFO: - Episode: 193/400, Reward: -43.000, Step: 43
260
- 2023-03-11 00:15:05 - r - INFO: - Episode: 194/400, Reward: -30.000, Step: 30
261
- 2023-03-11 00:15:05 - r - INFO: - Episode: 195/400, Reward: -21.000, Step: 21
262
- 2023-03-11 00:15:05 - r - INFO: - Episode: 196/400, Reward: -21.000, Step: 21
263
- 2023-03-11 00:15:05 - r - INFO: - Episode: 197/400, Reward: -27.000, Step: 27
264
- 2023-03-11 00:15:05 - r - INFO: - Episode: 198/400, Reward: -23.000, Step: 23
265
- 2023-03-11 00:15:05 - r - INFO: - Episode: 199/400, Reward: -31.000, Step: 31
266
- 2023-03-11 00:15:05 - r - INFO: - Episode: 200/400, Reward: -33.000, Step: 33
267
- 2023-03-11 00:15:05 - r - INFO: - Episode: 201/400, Reward: -35.000, Step: 35
268
- 2023-03-11 00:15:05 - r - INFO: - Episode: 202/400, Reward: -24.000, Step: 24
269
- 2023-03-11 00:15:05 - r - INFO: - Episode: 203/400, Reward: -22.000, Step: 22
270
- 2023-03-11 00:15:05 - r - INFO: - Episode: 204/400, Reward: -25.000, Step: 25
271
- 2023-03-11 00:15:05 - r - INFO: - Episode: 205/400, Reward: -35.000, Step: 35
272
- 2023-03-11 00:15:05 - r - INFO: - Episode: 206/400, Reward: -20.000, Step: 20
273
- 2023-03-11 00:15:05 - r - INFO: - Episode: 207/400, Reward: -23.000, Step: 23
274
- 2023-03-11 00:15:05 - r - INFO: - Episode: 208/400, Reward: -29.000, Step: 29
275
- 2023-03-11 00:15:05 - r - INFO: - Episode: 209/400, Reward: -26.000, Step: 26
276
- 2023-03-11 00:15:05 - r - INFO: - Episode: 210/400, Reward: -25.000, Step: 25
277
- 2023-03-11 00:15:05 - r - INFO: - Episode: 211/400, Reward: -31.000, Step: 31
278
- 2023-03-11 00:15:05 - r - INFO: - Episode: 212/400, Reward: -42.000, Step: 42
279
- 2023-03-11 00:15:05 - r - INFO: - Episode: 213/400, Reward: -22.000, Step: 22
280
- 2023-03-11 00:15:05 - r - INFO: - Episode: 214/400, Reward: -29.000, Step: 29
281
- 2023-03-11 00:15:05 - r - INFO: - Episode: 215/400, Reward: -25.000, Step: 25
282
- 2023-03-11 00:15:05 - r - INFO: - Episode: 216/400, Reward: -30.000, Step: 30
283
- 2023-03-11 00:15:05 - r - INFO: - Episode: 217/400, Reward: -16.000, Step: 16
284
- 2023-03-11 00:15:05 - r - INFO: - Episode: 218/400, Reward: -26.000, Step: 26
285
- 2023-03-11 00:15:05 - r - INFO: - Episode: 219/400, Reward: -23.000, Step: 23
286
- 2023-03-11 00:15:05 - r - INFO: - Episode: 220/400, Reward: -28.000, Step: 28
287
- 2023-03-11 00:15:05 - r - INFO: - Episode: 221/400, Reward: -17.000, Step: 17
288
- 2023-03-11 00:15:05 - r - INFO: - Episode: 222/400, Reward: -38.000, Step: 38
289
- 2023-03-11 00:15:05 - r - INFO: - Episode: 223/400, Reward: -21.000, Step: 21
290
- 2023-03-11 00:15:05 - r - INFO: - Episode: 224/400, Reward: -31.000, Step: 31
291
- 2023-03-11 00:15:05 - r - INFO: - Episode: 225/400, Reward: -23.000, Step: 23
292
- 2023-03-11 00:15:05 - r - INFO: - Episode: 226/400, Reward: -27.000, Step: 27
293
- 2023-03-11 00:15:05 - r - INFO: - Episode: 227/400, Reward: -19.000, Step: 19
294
- 2023-03-11 00:15:05 - r - INFO: - Episode: 228/400, Reward: -21.000, Step: 21
295
- 2023-03-11 00:15:05 - r - INFO: - Episode: 229/400, Reward: -25.000, Step: 25
296
- 2023-03-11 00:15:05 - r - INFO: - Episode: 230/400, Reward: -30.000, Step: 30
297
- 2023-03-11 00:15:05 - r - INFO: - Episode: 231/400, Reward: -22.000, Step: 22
298
- 2023-03-11 00:15:05 - r - INFO: - Episode: 232/400, Reward: -33.000, Step: 33
299
- 2023-03-11 00:15:05 - r - INFO: - Episode: 233/400, Reward: -21.000, Step: 21
300
- 2023-03-11 00:15:05 - r - INFO: - Episode: 234/400, Reward: -35.000, Step: 35
301
- 2023-03-11 00:15:05 - r - INFO: - Episode: 235/400, Reward: -25.000, Step: 25
302
- 2023-03-11 00:15:05 - r - INFO: - Episode: 236/400, Reward: -22.000, Step: 22
303
- 2023-03-11 00:15:05 - r - INFO: - Episode: 237/400, Reward: -25.000, Step: 25
304
- 2023-03-11 00:15:05 - r - INFO: - Episode: 238/400, Reward: -15.000, Step: 15
305
- 2023-03-11 00:15:05 - r - INFO: - Episode: 239/400, Reward: -33.000, Step: 33
306
- 2023-03-11 00:15:05 - r - INFO: - Episode: 240/400, Reward: -25.000, Step: 25
307
- 2023-03-11 00:15:05 - r - INFO: - Episode: 241/400, Reward: -18.000, Step: 18
308
- 2023-03-11 00:15:05 - r - INFO: - Episode: 242/400, Reward: -23.000, Step: 23
309
- 2023-03-11 00:15:05 - r - INFO: - Episode: 243/400, Reward: -21.000, Step: 21
310
- 2023-03-11 00:15:05 - r - INFO: - Episode: 244/400, Reward: -22.000, Step: 22
311
- 2023-03-11 00:15:05 - r - INFO: - Episode: 245/400, Reward: -26.000, Step: 26
312
- 2023-03-11 00:15:05 - r - INFO: - Episode: 246/400, Reward: -126.000, Step: 27
313
- 2023-03-11 00:15:05 - r - INFO: - Episode: 247/400, Reward: -27.000, Step: 27
314
- 2023-03-11 00:15:05 - r - INFO: - Episode: 248/400, Reward: -25.000, Step: 25
315
- 2023-03-11 00:15:05 - r - INFO: - Episode: 249/400, Reward: -26.000, Step: 26
316
- 2023-03-11 00:15:05 - r - INFO: - Episode: 250/400, Reward: -16.000, Step: 16
317
- 2023-03-11 00:15:05 - r - INFO: - Episode: 251/400, Reward: -26.000, Step: 26
318
- 2023-03-11 00:15:05 - r - INFO: - Episode: 252/400, Reward: -19.000, Step: 19
319
- 2023-03-11 00:15:05 - r - INFO: - Episode: 253/400, Reward: -25.000, Step: 25
320
- 2023-03-11 00:15:05 - r - INFO: - Episode: 254/400, Reward: -28.000, Step: 28
321
- 2023-03-11 00:15:05 - r - INFO: - Episode: 255/400, Reward: -31.000, Step: 31
322
- 2023-03-11 00:15:06 - r - INFO: - Episode: 256/400, Reward: -25.000, Step: 25
323
- 2023-03-11 00:15:06 - r - INFO: - Episode: 257/400, Reward: -25.000, Step: 25
324
- 2023-03-11 00:15:06 - r - INFO: - Episode: 258/400, Reward: -15.000, Step: 15
325
- 2023-03-11 00:15:06 - r - INFO: - Episode: 259/400, Reward: -19.000, Step: 19
326
- 2023-03-11 00:15:06 - r - INFO: - Episode: 260/400, Reward: -31.000, Step: 31
327
- 2023-03-11 00:15:06 - r - INFO: - Current episode 260 has the best eval reward: -15.000
328
- 2023-03-11 00:15:06 - r - INFO: - Episode: 261/400, Reward: -15.000, Step: 15
329
- 2023-03-11 00:15:06 - r - INFO: - Episode: 262/400, Reward: -27.000, Step: 27
330
- 2023-03-11 00:15:06 - r - INFO: - Episode: 263/400, Reward: -20.000, Step: 20
331
- 2023-03-11 00:15:06 - r - INFO: - Episode: 264/400, Reward: -23.000, Step: 23
332
- 2023-03-11 00:15:06 - r - INFO: - Episode: 265/400, Reward: -23.000, Step: 23
333
- 2023-03-11 00:15:06 - r - INFO: - Episode: 266/400, Reward: -24.000, Step: 24
334
- 2023-03-11 00:15:06 - r - INFO: - Episode: 267/400, Reward: -20.000, Step: 20
335
- 2023-03-11 00:15:06 - r - INFO: - Episode: 268/400, Reward: -24.000, Step: 24
336
- 2023-03-11 00:15:06 - r - INFO: - Episode: 269/400, Reward: -24.000, Step: 24
337
- 2023-03-11 00:15:06 - r - INFO: - Episode: 270/400, Reward: -17.000, Step: 17
338
- 2023-03-11 00:15:06 - r - INFO: - Episode: 271/400, Reward: -29.000, Step: 29
339
- 2023-03-11 00:15:06 - r - INFO: - Episode: 272/400, Reward: -15.000, Step: 15
340
- 2023-03-11 00:15:06 - r - INFO: - Episode: 273/400, Reward: -23.000, Step: 23
341
- 2023-03-11 00:15:06 - r - INFO: - Episode: 274/400, Reward: -30.000, Step: 30
342
- 2023-03-11 00:15:06 - r - INFO: - Episode: 275/400, Reward: -17.000, Step: 17
343
- 2023-03-11 00:15:06 - r - INFO: - Episode: 276/400, Reward: -19.000, Step: 19
344
- 2023-03-11 00:15:06 - r - INFO: - Episode: 277/400, Reward: -23.000, Step: 23
345
- 2023-03-11 00:15:06 - r - INFO: - Episode: 278/400, Reward: -19.000, Step: 19
346
- 2023-03-11 00:15:06 - r - INFO: - Episode: 279/400, Reward: -27.000, Step: 27
347
- 2023-03-11 00:15:06 - r - INFO: - Episode: 280/400, Reward: -20.000, Step: 20
348
- 2023-03-11 00:15:06 - r - INFO: - Episode: 281/400, Reward: -28.000, Step: 28
349
- 2023-03-11 00:15:06 - r - INFO: - Episode: 282/400, Reward: -19.000, Step: 19
350
- 2023-03-11 00:15:06 - r - INFO: - Episode: 283/400, Reward: -22.000, Step: 22
351
- 2023-03-11 00:15:06 - r - INFO: - Episode: 284/400, Reward: -28.000, Step: 28
352
- 2023-03-11 00:15:06 - r - INFO: - Episode: 285/400, Reward: -37.000, Step: 37
353
- 2023-03-11 00:15:06 - r - INFO: - Episode: 286/400, Reward: -19.000, Step: 19
354
- 2023-03-11 00:15:06 - r - INFO: - Episode: 287/400, Reward: -15.000, Step: 15
355
- 2023-03-11 00:15:06 - r - INFO: - Episode: 288/400, Reward: -19.000, Step: 19
356
- 2023-03-11 00:15:06 - r - INFO: - Episode: 289/400, Reward: -19.000, Step: 19
357
- 2023-03-11 00:15:06 - r - INFO: - Episode: 290/400, Reward: -19.000, Step: 19
358
- 2023-03-11 00:15:06 - r - INFO: - Episode: 291/400, Reward: -20.000, Step: 20
359
- 2023-03-11 00:15:06 - r - INFO: - Episode: 292/400, Reward: -15.000, Step: 15
360
- 2023-03-11 00:15:06 - r - INFO: - Episode: 293/400, Reward: -20.000, Step: 20
361
- 2023-03-11 00:15:06 - r - INFO: - Episode: 294/400, Reward: -23.000, Step: 23
362
- 2023-03-11 00:15:06 - r - INFO: - Episode: 295/400, Reward: -20.000, Step: 20
363
- 2023-03-11 00:15:06 - r - INFO: - Episode: 296/400, Reward: -21.000, Step: 21
364
- 2023-03-11 00:15:06 - r - INFO: - Episode: 297/400, Reward: -22.000, Step: 22
365
- 2023-03-11 00:15:06 - r - INFO: - Episode: 298/400, Reward: -17.000, Step: 17
366
- 2023-03-11 00:15:06 - r - INFO: - Episode: 299/400, Reward: -19.000, Step: 19
367
- 2023-03-11 00:15:06 - r - INFO: - Episode: 300/400, Reward: -30.000, Step: 30
368
- 2023-03-11 00:15:06 - r - INFO: - Episode: 301/400, Reward: -16.000, Step: 16
369
- 2023-03-11 00:15:06 - r - INFO: - Episode: 302/400, Reward: -29.000, Step: 29
370
- 2023-03-11 00:15:06 - r - INFO: - Episode: 303/400, Reward: -15.000, Step: 15
371
- 2023-03-11 00:15:06 - r - INFO: - Episode: 304/400, Reward: -18.000, Step: 18
372
- 2023-03-11 00:15:06 - r - INFO: - Episode: 305/400, Reward: -21.000, Step: 21
373
- 2023-03-11 00:15:06 - r - INFO: - Current episode 305 has the best eval reward: -15.000
374
- 2023-03-11 00:15:06 - r - INFO: - Episode: 306/400, Reward: -15.000, Step: 15
375
- 2023-03-11 00:15:06 - r - INFO: - Episode: 307/400, Reward: -20.000, Step: 20
376
- 2023-03-11 00:15:06 - r - INFO: - Episode: 308/400, Reward: -22.000, Step: 22
377
- 2023-03-11 00:15:06 - r - INFO: - Episode: 309/400, Reward: -32.000, Step: 32
378
- 2023-03-11 00:15:06 - r - INFO: - Episode: 310/400, Reward: -24.000, Step: 24
379
- 2023-03-11 00:15:06 - r - INFO: - Episode: 311/400, Reward: -21.000, Step: 21
380
- 2023-03-11 00:15:06 - r - INFO: - Episode: 312/400, Reward: -17.000, Step: 17
381
- 2023-03-11 00:15:06 - r - INFO: - Episode: 313/400, Reward: -17.000, Step: 17
382
- 2023-03-11 00:15:06 - r - INFO: - Episode: 314/400, Reward: -20.000, Step: 20
383
- 2023-03-11 00:15:06 - r - INFO: - Episode: 315/400, Reward: -19.000, Step: 19
384
- 2023-03-11 00:15:06 - r - INFO: - Current episode 315 has the best eval reward: -15.000
385
- 2023-03-11 00:15:06 - r - INFO: - Episode: 316/400, Reward: -15.000, Step: 15
386
- 2023-03-11 00:15:06 - r - INFO: - Episode: 317/400, Reward: -23.000, Step: 23
387
- 2023-03-11 00:15:06 - r - INFO: - Episode: 318/400, Reward: -17.000, Step: 17
388
- 2023-03-11 00:15:06 - r - INFO: - Episode: 319/400, Reward: -14.000, Step: 14
389
- 2023-03-11 00:15:06 - r - INFO: - Episode: 320/400, Reward: -19.000, Step: 19
390
- 2023-03-11 00:15:06 - r - INFO: - Episode: 321/400, Reward: -23.000, Step: 23
391
- 2023-03-11 00:15:06 - r - INFO: - Episode: 322/400, Reward: -24.000, Step: 24
392
- 2023-03-11 00:15:06 - r - INFO: - Episode: 323/400, Reward: -22.000, Step: 22
393
- 2023-03-11 00:15:06 - r - INFO: - Episode: 324/400, Reward: -21.000, Step: 21
394
- 2023-03-11 00:15:06 - r - INFO: - Episode: 325/400, Reward: -18.000, Step: 18
395
- 2023-03-11 00:15:06 - r - INFO: - Episode: 326/400, Reward: -26.000, Step: 26
396
- 2023-03-11 00:15:06 - r - INFO: - Episode: 327/400, Reward: -17.000, Step: 17
397
- 2023-03-11 00:15:06 - r - INFO: - Episode: 328/400, Reward: -13.000, Step: 13
398
- 2023-03-11 00:15:06 - r - INFO: - Episode: 329/400, Reward: -18.000, Step: 18
399
- 2023-03-11 00:15:06 - r - INFO: - Episode: 330/400, Reward: -15.000, Step: 15
400
- 2023-03-11 00:15:06 - r - INFO: - Episode: 331/400, Reward: -19.000, Step: 19
401
- 2023-03-11 00:15:06 - r - INFO: - Episode: 332/400, Reward: -20.000, Step: 20
402
- 2023-03-11 00:15:06 - r - INFO: - Episode: 333/400, Reward: -13.000, Step: 13
403
- 2023-03-11 00:15:06 - r - INFO: - Episode: 334/400, Reward: -20.000, Step: 20
404
- 2023-03-11 00:15:06 - r - INFO: - Episode: 335/400, Reward: -17.000, Step: 17
405
- 2023-03-11 00:15:06 - r - INFO: - Episode: 336/400, Reward: -18.000, Step: 18
406
- 2023-03-11 00:15:06 - r - INFO: - Episode: 337/400, Reward: -17.000, Step: 17
407
- 2023-03-11 00:15:06 - r - INFO: - Episode: 338/400, Reward: -24.000, Step: 24
408
- 2023-03-11 00:15:06 - r - INFO: - Episode: 339/400, Reward: -25.000, Step: 25
409
- 2023-03-11 00:15:06 - r - INFO: - Episode: 340/400, Reward: -13.000, Step: 13
410
- 2023-03-11 00:15:06 - r - INFO: - Episode: 341/400, Reward: -24.000, Step: 24
411
- 2023-03-11 00:15:06 - r - INFO: - Episode: 342/400, Reward: -16.000, Step: 16
412
- 2023-03-11 00:15:06 - r - INFO: - Episode: 343/400, Reward: -17.000, Step: 17
413
- 2023-03-11 00:15:06 - r - INFO: - Episode: 344/400, Reward: -14.000, Step: 14
414
- 2023-03-11 00:15:06 - r - INFO: - Episode: 345/400, Reward: -15.000, Step: 15
415
- 2023-03-11 00:15:06 - r - INFO: - Episode: 346/400, Reward: -20.000, Step: 20
416
- 2023-03-11 00:15:06 - r - INFO: - Episode: 347/400, Reward: -13.000, Step: 13
417
- 2023-03-11 00:15:06 - r - INFO: - Episode: 348/400, Reward: -16.000, Step: 16
418
- 2023-03-11 00:15:06 - r - INFO: - Episode: 349/400, Reward: -15.000, Step: 15
419
- 2023-03-11 00:15:06 - r - INFO: - Episode: 350/400, Reward: -13.000, Step: 13
420
- 2023-03-11 00:15:06 - r - INFO: - Episode: 351/400, Reward: -24.000, Step: 24
421
- 2023-03-11 00:15:06 - r - INFO: - Episode: 352/400, Reward: -19.000, Step: 19
422
- 2023-03-11 00:15:06 - r - INFO: - Episode: 353/400, Reward: -13.000, Step: 13
423
- 2023-03-11 00:15:06 - r - INFO: - Episode: 354/400, Reward: -16.000, Step: 16
424
- 2023-03-11 00:15:06 - r - INFO: - Episode: 355/400, Reward: -16.000, Step: 16
425
- 2023-03-11 00:15:06 - r - INFO: - Current episode 355 has the best eval reward: -15.000
426
- 2023-03-11 00:15:06 - r - INFO: - Episode: 356/400, Reward: -15.000, Step: 15
427
- 2023-03-11 00:15:06 - r - INFO: - Episode: 357/400, Reward: -26.000, Step: 26
428
- 2023-03-11 00:15:06 - r - INFO: - Episode: 358/400, Reward: -15.000, Step: 15
429
- 2023-03-11 00:15:06 - r - INFO: - Episode: 359/400, Reward: -23.000, Step: 23
430
- 2023-03-11 00:15:06 - r - INFO: - Episode: 360/400, Reward: -13.000, Step: 13
431
- 2023-03-11 00:15:06 - r - INFO: - Episode: 361/400, Reward: -17.000, Step: 17
432
- 2023-03-11 00:15:06 - r - INFO: - Episode: 362/400, Reward: -22.000, Step: 22
433
- 2023-03-11 00:15:06 - r - INFO: - Episode: 363/400, Reward: -17.000, Step: 17
434
- 2023-03-11 00:15:06 - r - INFO: - Episode: 364/400, Reward: -17.000, Step: 17
435
- 2023-03-11 00:15:06 - r - INFO: - Episode: 365/400, Reward: -16.000, Step: 16
436
- 2023-03-11 00:15:06 - r - INFO: - Current episode 365 has the best eval reward: -15.000
437
- 2023-03-11 00:15:06 - r - INFO: - Episode: 366/400, Reward: -15.000, Step: 15
438
- 2023-03-11 00:15:06 - r - INFO: - Episode: 367/400, Reward: -13.000, Step: 13
439
- 2023-03-11 00:15:06 - r - INFO: - Episode: 368/400, Reward: -19.000, Step: 19
440
- 2023-03-11 00:15:06 - r - INFO: - Episode: 369/400, Reward: -15.000, Step: 15
441
- 2023-03-11 00:15:06 - r - INFO: - Episode: 370/400, Reward: -13.000, Step: 13
442
- 2023-03-11 00:15:06 - r - INFO: - Episode: 371/400, Reward: -17.000, Step: 17
443
- 2023-03-11 00:15:06 - r - INFO: - Episode: 372/400, Reward: -15.000, Step: 15
444
- 2023-03-11 00:15:06 - r - INFO: - Episode: 373/400, Reward: -13.000, Step: 13
445
- 2023-03-11 00:15:06 - r - INFO: - Episode: 374/400, Reward: -22.000, Step: 22
446
- 2023-03-11 00:15:06 - r - INFO: - Episode: 375/400, Reward: -14.000, Step: 14
447
- 2023-03-11 00:15:06 - r - INFO: - Current episode 375 has the best eval reward: -13.000
448
- 2023-03-11 00:15:06 - r - INFO: - Episode: 376/400, Reward: -13.000, Step: 13
449
- 2023-03-11 00:15:06 - r - INFO: - Episode: 377/400, Reward: -16.000, Step: 16
450
- 2023-03-11 00:15:06 - r - INFO: - Episode: 378/400, Reward: -13.000, Step: 13
451
- 2023-03-11 00:15:06 - r - INFO: - Episode: 379/400, Reward: -13.000, Step: 13
452
- 2023-03-11 00:15:06 - r - INFO: - Episode: 380/400, Reward: -17.000, Step: 17
453
- 2023-03-11 00:15:06 - r - INFO: - Current episode 380 has the best eval reward: -13.000
454
- 2023-03-11 00:15:06 - r - INFO: - Episode: 381/400, Reward: -13.000, Step: 13
455
- 2023-03-11 00:15:06 - r - INFO: - Episode: 382/400, Reward: -14.000, Step: 14
456
- 2023-03-11 00:15:06 - r - INFO: - Episode: 383/400, Reward: -21.000, Step: 21
457
- 2023-03-11 00:15:06 - r - INFO: - Episode: 384/400, Reward: -15.000, Step: 15
458
- 2023-03-11 00:15:06 - r - INFO: - Episode: 385/400, Reward: -15.000, Step: 15
459
- 2023-03-11 00:15:06 - r - INFO: - Episode: 386/400, Reward: -18.000, Step: 18
460
- 2023-03-11 00:15:06 - r - INFO: - Episode: 387/400, Reward: -14.000, Step: 14
461
- 2023-03-11 00:15:06 - r - INFO: - Episode: 388/400, Reward: -13.000, Step: 13
462
- 2023-03-11 00:15:06 - r - INFO: - Episode: 389/400, Reward: -13.000, Step: 13
463
- 2023-03-11 00:15:06 - r - INFO: - Episode: 390/400, Reward: -13.000, Step: 13
464
- 2023-03-11 00:15:06 - r - INFO: - Episode: 391/400, Reward: -18.000, Step: 18
465
- 2023-03-11 00:15:06 - r - INFO: - Episode: 392/400, Reward: -13.000, Step: 13
466
- 2023-03-11 00:15:06 - r - INFO: - Episode: 393/400, Reward: -13.000, Step: 13
467
- 2023-03-11 00:15:06 - r - INFO: - Episode: 394/400, Reward: -13.000, Step: 13
468
- 2023-03-11 00:15:06 - r - INFO: - Episode: 395/400, Reward: -14.000, Step: 14
469
- 2023-03-11 00:15:06 - r - INFO: - Current episode 395 has the best eval reward: -13.000
470
- 2023-03-11 00:15:06 - r - INFO: - Episode: 396/400, Reward: -13.000, Step: 13
471
- 2023-03-11 00:15:06 - r - INFO: - Episode: 397/400, Reward: -13.000, Step: 13
472
- 2023-03-11 00:15:06 - r - INFO: - Episode: 398/400, Reward: -13.000, Step: 13
473
- 2023-03-11 00:15:06 - r - INFO: - Episode: 399/400, Reward: -13.000, Step: 13
474
- 2023-03-11 00:15:06 - r - INFO: - Episode: 400/400, Reward: -13.000, Step: 13
475
- 2023-03-11 00:15:06 - r - INFO: - Finish training!
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ToyText/CliffWalking-v0/Train_CliffWalking-v0_Sarsa_20230311-001504/results/learning_curve.png DELETED
Binary file (44.7 kB)
 
ToyText/CliffWalking-v0/Train_CliffWalking-v0_Sarsa_20230311-001504/results/res.csv DELETED
@@ -1,401 +0,0 @@
1
- episodes,rewards,steps
2
- 0,-595,100
3
- 1,-397,100
4
- 2,-397,100
5
- 3,-199,100
6
- 4,-100,100
7
- 5,-36,36
8
- 6,-100,100
9
- 7,-199,100
10
- 8,-199,100
11
- 9,-199,100
12
- 10,-199,100
13
- 11,-100,100
14
- 12,-48,48
15
- 13,-100,100
16
- 14,-100,100
17
- 15,-100,100
18
- 16,-46,46
19
- 17,-100,100
20
- 18,-100,100
21
- 19,-64,64
22
- 20,-100,100
23
- 21,-74,74
24
- 22,-89,89
25
- 23,-100,100
26
- 24,-100,100
27
- 25,-68,68
28
- 26,-100,100
29
- 27,-52,52
30
- 28,-86,86
31
- 29,-66,66
32
- 30,-100,100
33
- 31,-45,45
34
- 32,-77,77
35
- 33,-100,100
36
- 34,-56,56
37
- 35,-100,100
38
- 36,-43,43
39
- 37,-81,81
40
- 38,-79,79
41
- 39,-73,73
42
- 40,-100,100
43
- 41,-35,35
44
- 42,-100,100
45
- 43,-82,82
46
- 44,-44,44
47
- 45,-100,100
48
- 46,-42,42
49
- 47,-61,61
50
- 48,-71,71
51
- 49,-96,96
52
- 50,-76,76
53
- 51,-100,100
54
- 52,-43,43
55
- 53,-99,99
56
- 54,-41,41
57
- 55,-100,100
58
- 56,-34,34
59
- 57,-61,61
60
- 58,-55,55
61
- 59,-168,69
62
- 60,-100,100
63
- 61,-76,76
64
- 62,-22,22
65
- 63,-54,54
66
- 64,-54,54
67
- 65,-65,65
68
- 66,-42,42
69
- 67,-67,67
70
- 68,-61,61
71
- 69,-54,54
72
- 70,-42,42
73
- 71,-64,64
74
- 72,-52,52
75
- 73,-45,45
76
- 74,-57,57
77
- 75,-59,59
78
- 76,-67,67
79
- 77,-46,46
80
- 78,-48,48
81
- 79,-63,63
82
- 80,-60,60
83
- 81,-47,47
84
- 82,-34,34
85
- 83,-188,89
86
- 84,-23,23
87
- 85,-35,35
88
- 86,-43,43
89
- 87,-61,61
90
- 88,-39,39
91
- 89,-57,57
92
- 90,-43,43
93
- 91,-56,56
94
- 92,-60,60
95
- 93,-24,24
96
- 94,-61,61
97
- 95,-22,22
98
- 96,-27,27
99
- 97,-73,73
100
- 98,-19,19
101
- 99,-44,44
102
- 100,-55,55
103
- 101,-56,56
104
- 102,-42,42
105
- 103,-48,48
106
- 104,-24,24
107
- 105,-49,49
108
- 106,-42,42
109
- 107,-21,21
110
- 108,-49,49
111
- 109,-25,25
112
- 110,-37,37
113
- 111,-34,34
114
- 112,-64,64
115
- 113,-32,32
116
- 114,-40,40
117
- 115,-29,29
118
- 116,-41,41
119
- 117,-44,44
120
- 118,-29,29
121
- 119,-39,39
122
- 120,-25,25
123
- 121,-43,43
124
- 122,-32,32
125
- 123,-20,20
126
- 124,-45,45
127
- 125,-44,44
128
- 126,-50,50
129
- 127,-21,21
130
- 128,-25,25
131
- 129,-43,43
132
- 130,-48,48
133
- 131,-55,55
134
- 132,-30,30
135
- 133,-25,25
136
- 134,-49,49
137
- 135,-34,34
138
- 136,-41,41
139
- 137,-41,41
140
- 138,-24,24
141
- 139,-48,48
142
- 140,-174,75
143
- 141,-20,20
144
- 142,-39,39
145
- 143,-25,25
146
- 144,-47,47
147
- 145,-47,47
148
- 146,-22,22
149
- 147,-32,32
150
- 148,-49,49
151
- 149,-38,38
152
- 150,-38,38
153
- 151,-37,37
154
- 152,-57,57
155
- 153,-27,27
156
- 154,-38,38
157
- 155,-15,15
158
- 156,-43,43
159
- 157,-35,35
160
- 158,-45,45
161
- 159,-40,40
162
- 160,-35,35
163
- 161,-28,28
164
- 162,-22,22
165
- 163,-26,26
166
- 164,-30,30
167
- 165,-29,29
168
- 166,-35,35
169
- 167,-19,19
170
- 168,-20,20
171
- 169,-20,20
172
- 170,-38,38
173
- 171,-23,23
174
- 172,-35,35
175
- 173,-24,24
176
- 174,-38,38
177
- 175,-24,24
178
- 176,-44,44
179
- 177,-22,22
180
- 178,-27,27
181
- 179,-21,21
182
- 180,-31,31
183
- 181,-26,26
184
- 182,-22,22
185
- 183,-37,37
186
- 184,-25,25
187
- 185,-33,33
188
- 186,-15,15
189
- 187,-31,31
190
- 188,-29,29
191
- 189,-25,25
192
- 190,-18,18
193
- 191,-19,19
194
- 192,-43,43
195
- 193,-30,30
196
- 194,-21,21
197
- 195,-21,21
198
- 196,-27,27
199
- 197,-23,23
200
- 198,-31,31
201
- 199,-33,33
202
- 200,-35,35
203
- 201,-24,24
204
- 202,-22,22
205
- 203,-25,25
206
- 204,-35,35
207
- 205,-20,20
208
- 206,-23,23
209
- 207,-29,29
210
- 208,-26,26
211
- 209,-25,25
212
- 210,-31,31
213
- 211,-42,42
214
- 212,-22,22
215
- 213,-29,29
216
- 214,-25,25
217
- 215,-30,30
218
- 216,-16,16
219
- 217,-26,26
220
- 218,-23,23
221
- 219,-28,28
222
- 220,-17,17
223
- 221,-38,38
224
- 222,-21,21
225
- 223,-31,31
226
- 224,-23,23
227
- 225,-27,27
228
- 226,-19,19
229
- 227,-21,21
230
- 228,-25,25
231
- 229,-30,30
232
- 230,-22,22
233
- 231,-33,33
234
- 232,-21,21
235
- 233,-35,35
236
- 234,-25,25
237
- 235,-22,22
238
- 236,-25,25
239
- 237,-15,15
240
- 238,-33,33
241
- 239,-25,25
242
- 240,-18,18
243
- 241,-23,23
244
- 242,-21,21
245
- 243,-22,22
246
- 244,-26,26
247
- 245,-126,27
248
- 246,-27,27
249
- 247,-25,25
250
- 248,-26,26
251
- 249,-16,16
252
- 250,-26,26
253
- 251,-19,19
254
- 252,-25,25
255
- 253,-28,28
256
- 254,-31,31
257
- 255,-25,25
258
- 256,-25,25
259
- 257,-15,15
260
- 258,-19,19
261
- 259,-31,31
262
- 260,-15,15
263
- 261,-27,27
264
- 262,-20,20
265
- 263,-23,23
266
- 264,-23,23
267
- 265,-24,24
268
- 266,-20,20
269
- 267,-24,24
270
- 268,-24,24
271
- 269,-17,17
272
- 270,-29,29
273
- 271,-15,15
274
- 272,-23,23
275
- 273,-30,30
276
- 274,-17,17
277
- 275,-19,19
278
- 276,-23,23
279
- 277,-19,19
280
- 278,-27,27
281
- 279,-20,20
282
- 280,-28,28
283
- 281,-19,19
284
- 282,-22,22
285
- 283,-28,28
286
- 284,-37,37
287
- 285,-19,19
288
- 286,-15,15
289
- 287,-19,19
290
- 288,-19,19
291
- 289,-19,19
292
- 290,-20,20
293
- 291,-15,15
294
- 292,-20,20
295
- 293,-23,23
296
- 294,-20,20
297
- 295,-21,21
298
- 296,-22,22
299
- 297,-17,17
300
- 298,-19,19
301
- 299,-30,30
302
- 300,-16,16
303
- 301,-29,29
304
- 302,-15,15
305
- 303,-18,18
306
- 304,-21,21
307
- 305,-15,15
308
- 306,-20,20
309
- 307,-22,22
310
- 308,-32,32
311
- 309,-24,24
312
- 310,-21,21
313
- 311,-17,17
314
- 312,-17,17
315
- 313,-20,20
316
- 314,-19,19
317
- 315,-15,15
318
- 316,-23,23
319
- 317,-17,17
320
- 318,-14,14
321
- 319,-19,19
322
- 320,-23,23
323
- 321,-24,24
324
- 322,-22,22
325
- 323,-21,21
326
- 324,-18,18
327
- 325,-26,26
328
- 326,-17,17
329
- 327,-13,13
330
- 328,-18,18
331
- 329,-15,15
332
- 330,-19,19
333
- 331,-20,20
334
- 332,-13,13
335
- 333,-20,20
336
- 334,-17,17
337
- 335,-18,18
338
- 336,-17,17
339
- 337,-24,24
340
- 338,-25,25
341
- 339,-13,13
342
- 340,-24,24
343
- 341,-16,16
344
- 342,-17,17
345
- 343,-14,14
346
- 344,-15,15
347
- 345,-20,20
348
- 346,-13,13
349
- 347,-16,16
350
- 348,-15,15
351
- 349,-13,13
352
- 350,-24,24
353
- 351,-19,19
354
- 352,-13,13
355
- 353,-16,16
356
- 354,-16,16
357
- 355,-15,15
358
- 356,-26,26
359
- 357,-15,15
360
- 358,-23,23
361
- 359,-13,13
362
- 360,-17,17
363
- 361,-22,22
364
- 362,-17,17
365
- 363,-17,17
366
- 364,-16,16
367
- 365,-15,15
368
- 366,-13,13
369
- 367,-19,19
370
- 368,-15,15
371
- 369,-13,13
372
- 370,-17,17
373
- 371,-15,15
374
- 372,-13,13
375
- 373,-22,22
376
- 374,-14,14
377
- 375,-13,13
378
- 376,-16,16
379
- 377,-13,13
380
- 378,-13,13
381
- 379,-17,17
382
- 380,-13,13
383
- 381,-14,14
384
- 382,-21,21
385
- 383,-15,15
386
- 384,-15,15
387
- 385,-18,18
388
- 386,-14,14
389
- 387,-13,13
390
- 388,-13,13
391
- 389,-13,13
392
- 390,-18,18
393
- 391,-13,13
394
- 392,-13,13
395
- 393,-13,13
396
- 394,-14,14
397
- 395,-13,13
398
- 396,-13,13
399
- 397,-13,13
400
- 398,-13,13
401
- 399,-13,13
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ToyText/CliffWalking-v0/{Train_CliffWalking-v0_Sarsa_20230311-001504 → Train_single_CliffWalking-v0_Sarsa_20230519-010804}/config.yaml RENAMED
@@ -1,25 +1,32 @@
1
  general_cfg:
2
  algo_name: Sarsa
 
3
  device: cpu
4
- env_name: CliffWalking-v0
5
- eval_eps: 10
6
- eval_per_episode: 5
7
  load_checkpoint: false
8
- load_path: Train_CartPole-v1_DQN_20221026-054757
9
- max_steps: 100
 
 
10
  mode: train
11
- new_step_api: true
12
- render: false
13
- save_fig: true
 
 
 
14
  seed: 1
15
- show_fig: false
16
- test_eps: 10
17
- train_eps: 400
18
- wrapper: envs.wrappers.CliffWalkingWapper
19
  algo_cfg:
20
  epsilon_decay: 300
21
  epsilon_end: 0.01
22
  epsilon_start: 0.95
23
- exploration_type: e-greedy
24
  gamma: 0.99
25
  lr: 0.1
 
 
 
 
 
 
 
 
1
  general_cfg:
2
  algo_name: Sarsa
3
+ collect_traj: false
4
  device: cpu
5
+ env_name: gym
 
 
6
  load_checkpoint: false
7
+ load_model_step: 12000
8
+ load_path: Train_single_CliffWalking-v0_QLearning_20230519-005837
9
+ max_episode: 300
10
+ max_step: 100
11
  mode: train
12
+ model_save_fre: 500
13
+ mp_backend: single
14
+ n_learners: 1
15
+ n_workers: 1
16
+ online_eval: false
17
+ online_eval_episode: 10
18
  seed: 1
19
+ share_buffer: true
 
 
 
20
  algo_cfg:
21
  epsilon_decay: 300
22
  epsilon_end: 0.01
23
  epsilon_start: 0.95
 
24
  gamma: 0.99
25
  lr: 0.1
26
+ env_cfg:
27
+ id: CliffWalking-v0
28
+ ignore_params:
29
+ - wrapper
30
+ - ignore_params
31
+ render_mode: null
32
+ wrapper: null
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/logs/log.txt ADDED
@@ -0,0 +1,342 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - General Configs:
2
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - ================================================================================
3
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - Name Value Type
4
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - env_name gym <class 'str'>
5
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - algo_name Sarsa <class 'str'>
6
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - mode train <class 'str'>
7
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - device cpu <class 'str'>
8
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - seed 1 <class 'int'>
9
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - max_episode 300 <class 'int'>
10
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - max_step 100 <class 'int'>
11
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - collect_traj 0 <class 'bool'>
12
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - mp_backend single <class 'str'>
13
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - n_workers 1 <class 'int'>
14
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - n_learners 1 <class 'int'>
15
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - share_buffer 1 <class 'bool'>
16
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - online_eval 0 <class 'bool'>
17
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - online_eval_episode 10 <class 'int'>
18
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - model_save_fre 500 <class 'int'>
19
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - load_checkpoint 0 <class 'bool'>
20
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - load_path Train_single_CliffWalking-v0_QLearning_20230519-005837 <class 'str'>
21
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - load_model_step 12000 <class 'int'>
22
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - ================================================================================
23
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - Algo Configs:
24
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - ================================================================================
25
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - Name Value Type
26
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - epsilon_start 0.95 <class 'float'>
27
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - epsilon_end 0.01 <class 'float'>
28
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - epsilon_decay 300 <class 'int'>
29
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - gamma 0.99 <class 'float'>
30
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - lr 0.1 <class 'float'>
31
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - ================================================================================
32
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - Env Configs:
33
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - ================================================================================
34
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - Name Value Type
35
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - id CliffWalking-v0 <class 'str'>
36
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - render_mode None <class 'str'>
37
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - wrapper None <class 'str'>
38
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] <class 'str'>
39
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - ================================================================================
40
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - obs_space: Discrete(48), n_actions: Discrete(4)
41
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - Start training!
42
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - episode: 0, ep_reward: -496, ep_step: 100
43
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - episode: 1, ep_reward: -496, ep_step: 100
44
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - episode: 2, ep_reward: -298, ep_step: 100
45
+ 2023-05-19 01:08:04 - SimpleLog - INFO: - episode: 3, ep_reward: -298, ep_step: 100
46
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 4, ep_reward: -298, ep_step: 100
47
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 5, ep_reward: -298, ep_step: 100
48
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 6, ep_reward: -199, ep_step: 100
49
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 7, ep_reward: -73, ep_step: 73
50
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 8, ep_reward: -100, ep_step: 100
51
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 9, ep_reward: -199, ep_step: 100
52
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 10, ep_reward: -199, ep_step: 100
53
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 11, ep_reward: -199, ep_step: 100
54
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 12, ep_reward: -100, ep_step: 100
55
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 13, ep_reward: -71, ep_step: 71
56
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 14, ep_reward: -89, ep_step: 89
57
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 15, ep_reward: -100, ep_step: 100
58
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 16, ep_reward: -100, ep_step: 100
59
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 17, ep_reward: -100, ep_step: 100
60
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 18, ep_reward: -81, ep_step: 81
61
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 19, ep_reward: -100, ep_step: 100
62
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 20, ep_reward: -58, ep_step: 58
63
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 21, ep_reward: -185, ep_step: 86
64
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 22, ep_reward: -100, ep_step: 100
65
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 23, ep_reward: -71, ep_step: 71
66
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 24, ep_reward: -97, ep_step: 97
67
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 25, ep_reward: -100, ep_step: 100
68
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 26, ep_reward: -55, ep_step: 55
69
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 27, ep_reward: -100, ep_step: 100
70
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 28, ep_reward: -98, ep_step: 98
71
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 29, ep_reward: -73, ep_step: 73
72
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 30, ep_reward: -74, ep_step: 74
73
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 31, ep_reward: -99, ep_step: 99
74
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 32, ep_reward: -85, ep_step: 85
75
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 33, ep_reward: -78, ep_step: 78
76
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 34, ep_reward: -98, ep_step: 98
77
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 35, ep_reward: -81, ep_step: 81
78
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 36, ep_reward: -57, ep_step: 57
79
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 37, ep_reward: -100, ep_step: 100
80
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 38, ep_reward: -27, ep_step: 27
81
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 39, ep_reward: -63, ep_step: 63
82
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 40, ep_reward: -60, ep_step: 60
83
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 41, ep_reward: -75, ep_step: 75
84
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 42, ep_reward: -93, ep_step: 93
85
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 43, ep_reward: -60, ep_step: 60
86
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 44, ep_reward: -100, ep_step: 100
87
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 45, ep_reward: -83, ep_step: 83
88
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 46, ep_reward: -70, ep_step: 70
89
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 47, ep_reward: -74, ep_step: 74
90
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 48, ep_reward: -86, ep_step: 86
91
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 49, ep_reward: -67, ep_step: 67
92
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 50, ep_reward: -93, ep_step: 93
93
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 51, ep_reward: -45, ep_step: 45
94
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 52, ep_reward: -100, ep_step: 100
95
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 53, ep_reward: -75, ep_step: 75
96
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 54, ep_reward: -51, ep_step: 51
97
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 55, ep_reward: -55, ep_step: 55
98
+ 2023-05-19 01:08:05 - SimpleLog - INFO: - episode: 56, ep_reward: -61, ep_step: 61
99
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 57, ep_reward: -74, ep_step: 74
100
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 58, ep_reward: -25, ep_step: 25
101
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 59, ep_reward: -100, ep_step: 100
102
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 60, ep_reward: -82, ep_step: 82
103
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 61, ep_reward: -42, ep_step: 42
104
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 62, ep_reward: -53, ep_step: 53
105
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 63, ep_reward: -48, ep_step: 48
106
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 64, ep_reward: -65, ep_step: 65
107
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 65, ep_reward: -91, ep_step: 91
108
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 66, ep_reward: -87, ep_step: 87
109
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 67, ep_reward: -20, ep_step: 20
110
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 68, ep_reward: -38, ep_step: 38
111
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 69, ep_reward: -75, ep_step: 75
112
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 70, ep_reward: -65, ep_step: 65
113
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 71, ep_reward: -61, ep_step: 61
114
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 72, ep_reward: -54, ep_step: 54
115
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 73, ep_reward: -47, ep_step: 47
116
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 74, ep_reward: -77, ep_step: 77
117
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 75, ep_reward: -48, ep_step: 48
118
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 76, ep_reward: -60, ep_step: 60
119
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 77, ep_reward: -50, ep_step: 50
120
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 78, ep_reward: -62, ep_step: 62
121
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 79, ep_reward: -40, ep_step: 40
122
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 80, ep_reward: -28, ep_step: 28
123
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 81, ep_reward: -64, ep_step: 64
124
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 82, ep_reward: -58, ep_step: 58
125
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 83, ep_reward: -37, ep_step: 37
126
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 84, ep_reward: -56, ep_step: 56
127
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 85, ep_reward: -46, ep_step: 46
128
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 86, ep_reward: -50, ep_step: 50
129
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 87, ep_reward: -38, ep_step: 38
130
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 88, ep_reward: -46, ep_step: 46
131
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 89, ep_reward: -41, ep_step: 41
132
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 90, ep_reward: -43, ep_step: 43
133
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 91, ep_reward: -45, ep_step: 45
134
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 92, ep_reward: -61, ep_step: 61
135
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 93, ep_reward: -48, ep_step: 48
136
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 94, ep_reward: -37, ep_step: 37
137
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 95, ep_reward: -28, ep_step: 28
138
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 96, ep_reward: -78, ep_step: 78
139
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 97, ep_reward: -33, ep_step: 33
140
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 98, ep_reward: -133, ep_step: 34
141
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 99, ep_reward: -48, ep_step: 48
142
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 100, ep_reward: -44, ep_step: 44
143
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 101, ep_reward: -31, ep_step: 31
144
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 102, ep_reward: -49, ep_step: 49
145
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 103, ep_reward: -39, ep_step: 39
146
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 104, ep_reward: -62, ep_step: 62
147
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 105, ep_reward: -42, ep_step: 42
148
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 106, ep_reward: -25, ep_step: 25
149
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 107, ep_reward: -30, ep_step: 30
150
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 108, ep_reward: -63, ep_step: 63
151
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 109, ep_reward: -16, ep_step: 16
152
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 110, ep_reward: -33, ep_step: 33
153
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 111, ep_reward: -45, ep_step: 45
154
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 112, ep_reward: -55, ep_step: 55
155
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 113, ep_reward: -26, ep_step: 26
156
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 114, ep_reward: -35, ep_step: 35
157
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 115, ep_reward: -30, ep_step: 30
158
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 116, ep_reward: -51, ep_step: 51
159
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 117, ep_reward: -37, ep_step: 37
160
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 118, ep_reward: -27, ep_step: 27
161
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 119, ep_reward: -28, ep_step: 28
162
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 120, ep_reward: -37, ep_step: 37
163
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 121, ep_reward: -54, ep_step: 54
164
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 122, ep_reward: -37, ep_step: 37
165
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 123, ep_reward: -40, ep_step: 40
166
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 124, ep_reward: -21, ep_step: 21
167
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 125, ep_reward: -38, ep_step: 38
168
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 126, ep_reward: -40, ep_step: 40
169
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 127, ep_reward: -37, ep_step: 37
170
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 128, ep_reward: -31, ep_step: 31
171
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 129, ep_reward: -73, ep_step: 73
172
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 130, ep_reward: -37, ep_step: 37
173
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 131, ep_reward: -41, ep_step: 41
174
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 132, ep_reward: -26, ep_step: 26
175
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 133, ep_reward: -26, ep_step: 26
176
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 134, ep_reward: -48, ep_step: 48
177
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 135, ep_reward: -24, ep_step: 24
178
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 136, ep_reward: -32, ep_step: 32
179
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 137, ep_reward: -26, ep_step: 26
180
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 138, ep_reward: -30, ep_step: 30
181
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 139, ep_reward: -32, ep_step: 32
182
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 140, ep_reward: -32, ep_step: 32
183
+ 2023-05-19 01:08:06 - SimpleLog - INFO: - episode: 141, ep_reward: -32, ep_step: 32
184
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 142, ep_reward: -36, ep_step: 36
185
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 143, ep_reward: -17, ep_step: 17
186
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 144, ep_reward: -41, ep_step: 41
187
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 145, ep_reward: -132, ep_step: 33
188
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 146, ep_reward: -21, ep_step: 21
189
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 147, ep_reward: -34, ep_step: 34
190
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 148, ep_reward: -21, ep_step: 21
191
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 149, ep_reward: -29, ep_step: 29
192
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 150, ep_reward: -38, ep_step: 38
193
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 151, ep_reward: -23, ep_step: 23
194
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 152, ep_reward: -19, ep_step: 19
195
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 153, ep_reward: -37, ep_step: 37
196
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 154, ep_reward: -22, ep_step: 22
197
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 155, ep_reward: -21, ep_step: 21
198
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 156, ep_reward: -33, ep_step: 33
199
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 157, ep_reward: -46, ep_step: 46
200
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 158, ep_reward: -29, ep_step: 29
201
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 159, ep_reward: -22, ep_step: 22
202
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 160, ep_reward: -32, ep_step: 32
203
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 161, ep_reward: -21, ep_step: 21
204
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 162, ep_reward: -23, ep_step: 23
205
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 163, ep_reward: -22, ep_step: 22
206
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 164, ep_reward: -25, ep_step: 25
207
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 165, ep_reward: -39, ep_step: 39
208
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 166, ep_reward: -27, ep_step: 27
209
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 167, ep_reward: -24, ep_step: 24
210
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 168, ep_reward: -14, ep_step: 14
211
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 169, ep_reward: -37, ep_step: 37
212
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 170, ep_reward: -33, ep_step: 33
213
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 171, ep_reward: -34, ep_step: 34
214
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 172, ep_reward: -19, ep_step: 19
215
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 173, ep_reward: -41, ep_step: 41
216
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 174, ep_reward: -20, ep_step: 20
217
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 175, ep_reward: -23, ep_step: 23
218
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 176, ep_reward: -17, ep_step: 17
219
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 177, ep_reward: -24, ep_step: 24
220
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 178, ep_reward: -32, ep_step: 32
221
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 179, ep_reward: -16, ep_step: 16
222
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 180, ep_reward: -31, ep_step: 31
223
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 181, ep_reward: -17, ep_step: 17
224
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 182, ep_reward: -23, ep_step: 23
225
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 183, ep_reward: -31, ep_step: 31
226
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 184, ep_reward: -15, ep_step: 15
227
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 185, ep_reward: -40, ep_step: 40
228
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 186, ep_reward: -15, ep_step: 15
229
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 187, ep_reward: -19, ep_step: 19
230
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 188, ep_reward: -29, ep_step: 29
231
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 189, ep_reward: -13, ep_step: 13
232
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 190, ep_reward: -49, ep_step: 49
233
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 191, ep_reward: -13, ep_step: 13
234
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 192, ep_reward: -14, ep_step: 14
235
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 193, ep_reward: -33, ep_step: 33
236
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 194, ep_reward: -19, ep_step: 19
237
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 195, ep_reward: -20, ep_step: 20
238
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 196, ep_reward: -27, ep_step: 27
239
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 197, ep_reward: -28, ep_step: 28
240
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 198, ep_reward: -26, ep_step: 26
241
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 199, ep_reward: -37, ep_step: 37
242
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 200, ep_reward: -19, ep_step: 19
243
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 201, ep_reward: -23, ep_step: 23
244
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 202, ep_reward: -27, ep_step: 27
245
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 203, ep_reward: -26, ep_step: 26
246
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 204, ep_reward: -23, ep_step: 23
247
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 205, ep_reward: -24, ep_step: 24
248
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 206, ep_reward: -28, ep_step: 28
249
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 207, ep_reward: -18, ep_step: 18
250
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 208, ep_reward: -22, ep_step: 22
251
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 209, ep_reward: -25, ep_step: 25
252
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 210, ep_reward: -26, ep_step: 26
253
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 211, ep_reward: -16, ep_step: 16
254
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 212, ep_reward: -23, ep_step: 23
255
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 213, ep_reward: -20, ep_step: 20
256
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 214, ep_reward: -31, ep_step: 31
257
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 215, ep_reward: -15, ep_step: 15
258
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 216, ep_reward: -19, ep_step: 19
259
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 217, ep_reward: -35, ep_step: 35
260
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 218, ep_reward: -13, ep_step: 13
261
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 219, ep_reward: -21, ep_step: 21
262
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 220, ep_reward: -21, ep_step: 21
263
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 221, ep_reward: -29, ep_step: 29
264
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 222, ep_reward: -15, ep_step: 15
265
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 223, ep_reward: -25, ep_step: 25
266
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 224, ep_reward: -21, ep_step: 21
267
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 225, ep_reward: -26, ep_step: 26
268
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 226, ep_reward: -27, ep_step: 27
269
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 227, ep_reward: -22, ep_step: 22
270
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 228, ep_reward: -17, ep_step: 17
271
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 229, ep_reward: -22, ep_step: 22
272
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 230, ep_reward: -15, ep_step: 15
273
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 231, ep_reward: -20, ep_step: 20
274
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 232, ep_reward: -28, ep_step: 28
275
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 233, ep_reward: -18, ep_step: 18
276
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 234, ep_reward: -21, ep_step: 21
277
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 235, ep_reward: -21, ep_step: 21
278
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 236, ep_reward: -13, ep_step: 13
279
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 237, ep_reward: -30, ep_step: 30
280
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 238, ep_reward: -13, ep_step: 13
281
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 239, ep_reward: -21, ep_step: 21
282
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 240, ep_reward: -13, ep_step: 13
283
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 241, ep_reward: -21, ep_step: 21
284
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 242, ep_reward: -19, ep_step: 19
285
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 243, ep_reward: -16, ep_step: 16
286
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 244, ep_reward: -18, ep_step: 18
287
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 245, ep_reward: -15, ep_step: 15
288
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 246, ep_reward: -35, ep_step: 35
289
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 247, ep_reward: -18, ep_step: 18
290
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 248, ep_reward: -19, ep_step: 19
291
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 249, ep_reward: -17, ep_step: 17
292
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 250, ep_reward: -19, ep_step: 19
293
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 251, ep_reward: -15, ep_step: 15
294
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 252, ep_reward: -13, ep_step: 13
295
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 253, ep_reward: -22, ep_step: 22
296
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 254, ep_reward: -15, ep_step: 15
297
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 255, ep_reward: -13, ep_step: 13
298
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 256, ep_reward: -18, ep_step: 18
299
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 257, ep_reward: -22, ep_step: 22
300
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 258, ep_reward: -25, ep_step: 25
301
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 259, ep_reward: -24, ep_step: 24
302
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 260, ep_reward: -13, ep_step: 13
303
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 261, ep_reward: -15, ep_step: 15
304
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 262, ep_reward: -21, ep_step: 21
305
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 263, ep_reward: -17, ep_step: 17
306
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 264, ep_reward: -15, ep_step: 15
307
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 265, ep_reward: -13, ep_step: 13
308
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 266, ep_reward: -24, ep_step: 24
309
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 267, ep_reward: -23, ep_step: 23
310
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 268, ep_reward: -13, ep_step: 13
311
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 269, ep_reward: -27, ep_step: 27
312
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 270, ep_reward: -118, ep_step: 19
313
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 271, ep_reward: -20, ep_step: 20
314
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 272, ep_reward: -15, ep_step: 15
315
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 273, ep_reward: -15, ep_step: 15
316
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 274, ep_reward: -17, ep_step: 17
317
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 275, ep_reward: -13, ep_step: 13
318
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 276, ep_reward: -21, ep_step: 21
319
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 277, ep_reward: -14, ep_step: 14
320
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 278, ep_reward: -15, ep_step: 15
321
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 279, ep_reward: -23, ep_step: 23
322
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 280, ep_reward: -13, ep_step: 13
323
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 281, ep_reward: -13, ep_step: 13
324
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 282, ep_reward: -24, ep_step: 24
325
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 283, ep_reward: -13, ep_step: 13
326
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 284, ep_reward: -25, ep_step: 25
327
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 285, ep_reward: -13, ep_step: 13
328
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 286, ep_reward: -23, ep_step: 23
329
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 287, ep_reward: -13, ep_step: 13
330
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 288, ep_reward: -13, ep_step: 13
331
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 289, ep_reward: -13, ep_step: 13
332
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 290, ep_reward: -13, ep_step: 13
333
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 291, ep_reward: -18, ep_step: 18
334
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 292, ep_reward: -14, ep_step: 14
335
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 293, ep_reward: -15, ep_step: 15
336
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 294, ep_reward: -15, ep_step: 15
337
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 295, ep_reward: -15, ep_step: 15
338
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 296, ep_reward: -19, ep_step: 19
339
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 297, ep_reward: -16, ep_step: 16
340
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 298, ep_reward: -13, ep_step: 13
341
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - episode: 299, ep_reward: -21, ep_step: 21
342
+ 2023-05-19 01:08:07 - SimpleLog - INFO: - Finish training! total time consumed: 3.05s
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/1000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/10000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/10500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/11000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/11500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/12000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/1500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/2000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/2500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/3000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/3500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/4000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/4500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/500 ADDED
Binary file (2.46 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/5000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/5500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/6000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/6500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/7000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/7500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/8000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/8500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/9000 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/Train_single_CliffWalking-v0_Sarsa_20230519-010804/models/9500 ADDED
Binary file (2.66 kB). View file
 
ToyText/CliffWalking-v0/{Train_CliffWalking-v0_Sarsa_20230311-001504/models/checkpoint.pkl → Train_single_CliffWalking-v0_Sarsa_20230519-010804/tb_logs/interact/events.out.tfevents.1684429684.DESKTOP-H34HQIQ.87440.0} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b49e03abcf8f10412bffd219b9d2a522e02d39fce0365e81456762229100643d
3
- size 6127
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:025841db3721291cda19950ea71ff36b489c2b1c6a74d921c28a1d8f0afb6129
3
+ size 31580
ToyText/CliffWalking-v0/{Test_CliffWalking-v0_Sarsa_20230311-001530/models/checkpoint.pkl → Train_single_CliffWalking-v0_Sarsa_20230519-010804/tb_logs/model/events.out.tfevents.1684429684.DESKTOP-H34HQIQ.87440.1} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b49e03abcf8f10412bffd219b9d2a522e02d39fce0365e81456762229100643d
3
- size 6127
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62720e5c99c8fe77addcb70a6088704899605eaeda7e957b3bf7f7db36fdcb8f
3
+ size 591735