Hugging Face's logo

garvitsachdeva
/
spindleflow-rl

Reinforcement Learning
stable-baselines3
sb3-contrib
gymnasium
multi-agent
openenv
Model card Files
xet
 Community
garvitsachdeva commited on
Commit
01cb591
·
verified ·
1 Parent(s): 901dc66

Checkpoint at step 5,000

Browse files
Files changed (5) hide show
  1. reward_curve.json +1 -1
  2. reward_curve.png +0 -0
  3. spindleflow_model_latest.zip +2 -2
  4. training_log.txt +81 -577
  5. vec_normalize_latest.pkl +1 -1
reward_curve.json CHANGED
@@ -1 +1 @@
1
- {"episodes": [0, 67, 134, 201, 268, 335, 402, 469, 536, 603, 670, 737, 804, 871, 938, 1005, 1072, 1139, 1206, 1273, 1340, 1407, 1474, 1541, 1608, 1675, 1742, 1809, 1876, 1943, 2010, 2077, 2144, 2211, 2278, 2345, 2412, 2479, 2546, 2613, 2680, 2747, 2814, 2881, 2948, 3015, 3082, 3149, 3216, 3283, 3350, 3417, 3484, 3551, 3618, 3685, 3752, 3819, 3886, 3953, 4020, 4087, 4154, 4221, 4288, 4355, 4422, 4489, 4556, 4623, 4690, 4757, 4824, 4891, 4958, 5025, 5092, 5159, 5226, 5293, 5360, 5427, 5494, 5561, 5628, 5695, 5762, 5829, 5896, 5963, 6030, 6097, 6164, 6231, 6298, 6365, 6432, 6499, 6566, 6633, 6700, 6767, 6834, 6901, 6968, 7035, 7102, 7169, 7236, 7303, 7370, 7437, 7504, 7571, 7638, 7705, 7772, 7839, 7906, 7973, 8040, 8107, 8174, 8241, 8308, 8375, 8442, 8509, 8576, 8643, 8710, 8777, 8844, 8911, 8978, 9045, 9112, 9179, 9246, 9313, 9380, 9447, 9514, 9581, 9648, 9715, 9782, 9849, 9916, 9983, 10050, 10117, 10184, 10251, 10318, 10385, 10452, 10519, 10586, 10653, 10720, 10787, 10854, 10921, 10988, 11055, 11122, 11189, 11256, 11323, 11390, 11457, 11524, 11591, 11658, 11725, 11792, 11859, 11926, 11993, 12060, 12127, 12194, 12261, 12328, 12395, 12462, 12529, 12596, 12663, 12730, 12797, 12864, 12931, 12998, 13065, 13132, 13199, 13266, 13333, 13400, 13467], "mean_rewards": [7.869385480880737, -0.4680378864902784, -0.48984615507501145, -0.5551410619040379, -0.4882915579094153, -0.4724334484614831, -0.4706858817982504, -0.49536512678172046, -0.488181437265121, -0.49218673040220723, -0.4815726703055059, -0.4820088524676782, -0.43784926100558375, -0.38558788626885354, -0.3533458443251239, -0.3040934353599124, -0.26831979214033574, -0.19140849363449494, -0.14203320211143491, -0.03154456203850192, 0.036356829438342037, 0.10923927782606911, 0.18537376434205202, 0.23818426830707837, 0.2971860210863843, 0.351879875832877, 0.41131269029737827, 0.49316205722733814, 0.54418244560461, 0.5529096998612051, 0.5803396979899199, 0.647491346514733, 0.6767948179212062, 0.7278867457176142, 0.7801977058292352, 0.8324114113203805, 0.8846536156650867, 0.9135522807000074, 0.9906498014824725, 1.0747783365352257, 1.118064828834938, 1.1254296454959651, 1.1729068560170468, 1.223296715608557, 1.250830467714188, 1.2658592661535932, 1.3026064717935804, 1.3210587293828564, 1.3291228487317284, 1.3327255037611838, 1.3617869722608757, 1.4155327298567029, 1.43804301345443, 1.4533835403824795, 1.4490587244572413, 1.4650314738802017, 1.4970641783053438, 1.5264675998406834, 1.5249301680553555, 1.5104940346199014, 1.5259575240262075, 1.5392910738888663, 1.5554828937112133, 1.5631932486042976, 1.577678209906162, 1.578992944977183, 1.6025471957099515, 1.5807723147838604, 1.5952618680505644, 1.6292361341937072, 1.6326987860754143, 1.6245606879480392, 1.6551997169478658, 1.6609219776152042, 1.6956165906986145, 1.7281981333328353, 1.7382102399385817, 1.7609851391969418, 1.7657828451398392, 1.7754888206320893, 1.7645542004733432, 1.7736322304512135, 1.7687026667687216, 1.7580264367949372, 1.7511381573787308, 1.738839277673473, 1.722614758535346, 1.736745614141461, 1.7516905580270015, 1.7799245542025521, 1.7700678259391585, 1.7663207697615921, 1.7562163755505256, 1.7587610971747853, 1.7777937432416644, 1.7704596316888854, 1.8123356010955924, 1.8279217910185117, 1.8466502024516873, 1.8298634645036476, 1.864942445150443, 1.8738156217091924, 1.866000112849446, 1.8781902273140842, 1.8416169193525835, 1.8387443213458785, 1.8485462165543736, 1.8278536795119362, 1.8172942939169872, 1.8037021071630313, 1.7578254819032049, 1.789927663865958, 1.8083126330250383, 1.8363163345167517, 1.8767271982310745, 1.8823451488214322, 1.8882187248528846, 1.879882797724712, 1.8682739835334294, 1.8768755783047664, 1.8853264416580362, 1.8954450143514918, 1.9106359317722665, 1.9006636210578318, 1.8993877949828675, 1.9237823593464614, 1.9344475316007566, 1.9503340135305085, 1.9860419661475228, 1.9742594653096717, 2.001614794778903, 1.9832609987142256, 1.9856477235292567, 1.9926783719954344, 1.9750121485619383, 1.9528438657518075, 1.9334705030714798, 1.9176346147756205, 1.9197328341457625, 1.929320812971638, 1.935611099781306, 1.9301533763026026, 1.9524929260035577, 1.9774023529251636, 1.9670411776322891, 1.9650732083037827, 1.9756527485480824, 1.9960569345849135, 1.9994612800762286, 2.0023671949464004, 1.968471810134468, 1.9351308788716286, 1.9281577662479867, 1.9180797577689113, 1.941868303399712, 1.9551983473480623, 1.9582474042363658, 1.9446796976185323, 1.9599162018053136, 1.9677243051538726, 1.9905415299682818, 2.0126337819571773, 2.0055850803634727, 1.9728688634663787, 1.942376174849831, 1.961291219857542, 1.9382520422395841, 1.9331532626002175, 1.9315321168749136, 1.9510658689675868, 1.93959757961361, 1.9583952448716349, 1.9957628746242215, 2.0171623558352496, 2.045212839087822, 2.027753655145466, 2.023227911412426, 2.0044681133162108, 1.9950296741726627, 1.9736256399910699, 1.9867524237810816, 1.9778640382239483, 1.9617218394162603, 1.9373436287414156, 1.9262697115193839, 1.9367491794142053, 1.9615602505914702, 1.9952850935889883, 1.9757408594996002, 1.9619378888624635, 1.9616936761638262, 1.9569105818823662, 1.966847003359166, 1.995661675253198, 2.011519161810991, 2.030814020499653, 2.015335946653363, 2.010846167103975, 2.036694835887966, 2.063149935127053, 2.0468815370260613, 2.0691288812374875]}
 
1
+ {"episodes": [0, 7, 14, 21, 28, 35, 42, 49, 56, 63, 70, 77, 84, 91, 98, 105, 112, 119, 126, 133, 140, 147, 154, 161, 168, 175, 182, 189, 196, 203, 210, 217, 224, 231, 238, 245, 252, 259, 266, 273, 280, 287, 294, 301, 308, 315, 322, 329, 336, 343, 350, 357, 364, 371, 378, 385, 392, 399, 406, 413, 420, 427, 434, 441, 448, 455, 462, 469, 476, 483, 490, 497, 504, 511, 518, 525, 532, 539, 546, 553, 560, 567, 574, 581, 588, 595, 602, 609, 616, 623, 630, 637, 644, 651, 658, 665, 672, 679, 686, 693, 700, 707, 714, 721, 728, 735, 742, 749, 756, 763, 770, 777, 784, 791, 798, 805, 812, 819, 826, 833, 840, 847, 854, 861, 868, 875, 882, 889, 896, 903, 910, 917, 924, 931, 938, 945, 952, 959, 966, 973, 980, 987, 994, 1001, 1008, 1015, 1022, 1029, 1036, 1043, 1050, 1057, 1064, 1071, 1078, 1085, 1092, 1099, 1106, 1113, 1120, 1127, 1134, 1141, 1148, 1155, 1162, 1169, 1176, 1183, 1190, 1197, 1204, 1211, 1218, 1225, 1232, 1239, 1246, 1253, 1260, 1267, 1274, 1281, 1288, 1295, 1302, 1309, 1316, 1323, 1330, 1337, 1344, 1351, 1358, 1365, 1372, 1379, 1386, 1393, 1400, 1407, 1414, 1421, 1428, 1435, 1442, 1449, 1456, 1463, 1470, 1477, 1484, 1491, 1498, 1505, 1512], "mean_rewards": [2.011087656021118, -0.005159212742000818, -0.1737367016573747, -0.16313832066953182, -0.3787871667280279, -0.48352695473780233, -0.5125730392024961, -0.5108446795493364, -0.591070648936326, -0.548012486717198, -0.5592205684474656, -0.648504812123352, -0.6919027066936618, -0.6955952520825361, -0.7180035078015766, -0.6574326691854941, -0.5642441839077755, -0.4426701495816049, -0.42057076522982434, -0.39051416334941197, -0.34610133123044906, -0.37051013518909093, -0.34348269900012957, -0.4140826832797182, -0.3751730122358391, -0.4399783388270359, -0.47836029264879854, -0.49714391801114144, -0.5542931245443853, -0.5445789469798145, -0.5709554771157471, -0.6067243746334785, -0.6017522437890109, -0.6369883727870489, -0.5803498540465769, -0.5577858655076278, -0.5325302795359963, -0.48865976008145434, -0.5554399732990485, -0.5570450848164527, -0.5919859797056568, -0.5692832094960307, -0.5678086980295024, -0.5382031025364995, -0.47720208489581156, -0.3598009528298127, -0.40263426323470314, -0.362639883925256, -0.43668582216885526, -0.44475414255928053, -0.4044125404110865, -0.38040371361727776, -0.4100033921728793, -0.4820523009586491, -0.44493751438628687, -0.43020405066444684, -0.39207222337197317, -0.35661065328474106, -0.3769962504800213, -0.38675586683185476, -0.38980928779040513, -0.37443856669491843, -0.4122140955081896, -0.39305691680821936, -0.2994311030856089, -0.2579036255514151, -0.28448272915557027, -0.43960564032098964, -0.4300747401767263, -0.4077215094462429, -0.3728640738041385, -0.27169732158807547, -0.2620353500456794, -0.18189748504052036, -0.1810223087668419, -0.20924439489547358, -0.2289466186622648, -0.2507834549209005, -0.15905087524534842, -0.13111531837402204, -0.15006807980764852, -0.18575034001351973, -0.2604610369491734, -0.2928624492089607, -0.3075465619269955, -0.24049769925247683, -0.22923651928278177, -0.2999157253897896, -0.31141990337422804, -0.33937625506108526, -0.42456708659761044, -0.43582449468636986, -0.5326400213678809, -0.5070261559939306, -0.4317533519658211, -0.33543822365371806, -0.44631047938999374, -0.3564543535834865, -0.26606688962170955, -0.21719291747400635, -0.27412202130807073, -0.2125598310345882, -0.07335185637011339, -0.0011550859106998694, 0.018151712015663322, -0.0774569240349688, -0.14214420813674988, -0.11551256982707664, -0.16595394688805468, -0.23713979571077384, -0.16795892655653388, -0.06744629551509493, -0.11995828043865531, -0.12436404804650106, -0.14944038054856815, -0.23180527205726034, -0.2659248520963286, -0.2523376572583067, -0.19898790969072203, -0.20628121068799182, -0.16211953068054036, -0.1457904805490551, -0.16110725120616784, -0.200850348215011, -0.22606607534506015, -0.2517521789505784, -0.17527172132395208, -0.07480273323476706, -0.0609145848889296, -0.04951209085993469, -0.021120717859287794, -0.016507455736006562, -0.032915668678469956, -0.07218090113325927, -0.044014879569754396, 0.02370264898912099, 0.1031873529887219, 0.10730368968736577, 0.0996610147903036, 0.050899871637260445, 0.09100137392402087, 0.015319903932609841, -0.10438356234839088, -0.0165395066632252, -0.004070776514708996, -0.07922083899182708, -0.13058845507667252, -0.12486932470806335, -0.09150928616719811, -0.1680179141755951, -0.13921746273378008, -0.09083811982877944, -0.03548207682998557, -0.04591693949738616, 0.0030767117675982022, 0.07675606031951152, 0.10681556083114915, 0.07763870174408351, 0.11034342362300346, 0.06843050755560398, 0.06757143875093836, 0.01234445064083526, 0.02761492897805415, -0.03187786944602665, -0.032681759593910294, -0.16471537766291908, -0.24135849561150136, -0.23454911743388757, -0.1854007679503411, -0.18656958046516306, -0.18510495003704963, -0.10837438682976522, -0.030741750409728603, -0.02555339254046741, 0.013502296648527446, 0.19569833522760555, 0.3064040861729729, 0.29516894110527475, 0.33559935329187857, 0.3795126823120211, 0.3801108545280601, 0.3302895774280554, 0.369674541350258, 0.31977684952710805, 0.29791844812662976, 0.30011620813686596, 0.308363941056948, 0.2674849516663112, 0.3455620188462107, 0.34136716354834407, 0.3282235764751309, 0.4036587957096727, 0.4394211591662545, 0.5384622680905619, 0.5320534381623331, 0.47804044314513083, 0.5449790290899967, 0.41111522383595767, 0.5577673092484474, 0.5171146543794557, 0.5383412212280458, 0.5662813418740896, 0.48640087474823784, 0.5216721576932621, 0.5288327510566696, 0.5378777498232299, 0.6355279753425795, 0.6394623351087304, 0.8364699263076641, 0.7872928960101777, 0.753283737594948, 0.8241279435981261, 0.8046827170213586, 0.8140687913584866, 0.7523131846499286, 0.6723473424974241, 0.6993183063735303], "raw_rewards": [2.011087656021118, -1.6073665618896484, 0.7498452365398407, 0.2025524079799652, -1.6622264385223389, -1.5905271768569946, -0.38584309816360474, 1.0968736708164215, -1.2686316668987274, -1.5422813892364502, -1.5154712200164795, -1.494507908821106, -1.4630639553070068, -0.7266715168952942, 1.3441461846232414, -1.4742072820663452, 1.3427048027515411, -1.486713171005249, 0.37279677391052246, -0.08876031637191772, 1.0587169826030731, -1.384795069694519, -1.5081825256347656, -1.4929274320602417, 2.797714591026306, -1.47134530544281, 1.7549763023853302, -1.4590506553649902, -0.3829669952392578, 0.158383309841156, -1.9421470761299133, -1.2414164543151855, -1.3137173354625702, -1.481404185295105, -0.9551828801631927, -0.2936665415763855, -2.085797905921936, 1.0033707320690155, 0.8391484022140503, 0.653633639216423, -1.8033610582351685, 0.8675942420959473, -1.5180444717407227, -1.5836696326732635, 0.6177796721458435, 0.7355930209159851, 0.877733588218689, 1.3888882398605347, 0.3279541917145252, -0.8971331119537354, 1.4678781032562256, -0.8665207028388977, -1.5461901426315308, -1.3819766342639923, -0.6615731120109558, -1.552343487739563, -2.1833449825644493, -1.5418636798858643, 2.6514883637428284, -3.489000581204891, -1.532758355140686, -1.525796890258789, -2.4441225975751877, -2.568617880344391, -2.0896497666835785, 0.02511417865753174, 0.9480623602867126, 1.7094351649284363, 1.4755454063415527, 1.8405121266841888, 0.42442914843559265, -0.7997810021042824, 1.0508716404438019, 1.2704868093132973, -3.3804259300231934, 0.980677954852581, -1.4998018816113472, -1.544227957725525, -3.4981040954589844, 1.5663873851299286, -0.6641147136688232, 0.27374209463596344, -2.0531827434897423, -1.657704919576645, -1.8326283991336823, 1.6819107234477997, -0.3654499650001526, -0.6246205270290375, 1.4166848957538605, 2.029248148202896, -0.29613277316093445, -0.588357999920845, -0.25787171721458435, 0.7583343088626862, 0.562331348657608, -1.557992935180664, -1.1823661029338837, 0.3907746523618698, -0.4797786474227905, -1.5714150667190552, -0.3134910464286804, 0.8237113058567047, -0.9736100733280182, -1.211975336074829, -1.5755164623260498, -0.9478763937950134, -1.5776058435440063, 2.1475477814674377, -0.8160175681114197, -3.0692324340343475, -0.7797359824180603, 1.4694251716136932, -0.8319140374660492, 2.2456377148628235, -2.098494417965412, -1.287732794880867, -1.4832541942596436, -1.5874537229537964, -1.588030457496643, -1.2483505010604858, -0.2727857828140259, 0.8989545307122171, -1.5966110229492188, 2.2053601145744324, -0.717426210641861, -2.831665724515915, 1.736989140510559, -1.188669204711914, -0.19181019067764282, -1.1811168491840363, -0.4813724160194397, 0.6415759027004242, 1.293091356754303, -0.0336492657661438, -0.44722092151641846, -1.9554425179958344, 0.1328943967819214, -0.7047230452299118, 1.6609033942222595, -1.382057100534439, 0.07072758674621582, 0.12381599843502045, 0.25281383097171783, 1.180851012468338, -0.23011904954910278, -1.0367010533809662, -0.6570718288421631, -1.613729476928711, 2.0679474472999573, -1.5930011868476868, -1.0900622457265854, -0.1502295732498169, 0.48140305280685425, 2.2967147827148438, 2.0166079998016357, 1.2933931052684784, 1.0831645131111145, -1.3386335968971252, 0.8910820409655571, -0.5921838283538818, -0.7030514478683472, -1.617857813835144, 0.7024455070495605, -1.753804087638855, -0.3411940596997738, -1.7523786127567291, -0.8349017053842545, -1.6273200511932373, 1.2151376754045486, -1.0690277218818665, -1.3782235980033875, 1.008091401308775, 2.0703495740890503, 0.4502451419830322, -1.6283951997756958, 0.477810263633728, 0.9067497551441193, 1.4728646278381348, -1.6322773694992065, 1.5017837882041931, 1.5531878471374512, 1.020134299993515, 0.8596443384885788, -0.8000854253768921, 1.0358693972229958, -1.3619660139083862, 1.6163530200719833, -1.6371392011642456, 0.16331470012664795, 0.5777678936719894, 0.47991079092025757, 0.7756211757659912, 1.0058870911598206, 1.6384191513061523, -1.2334475964307785, -1.3401591777801514, 0.2782462537288666, 0.10824376344680786, 0.5431969165802002, 0.7870588600635529, -0.8065369129180908, 0.5831526145339012, 2.441283255815506, 2.3787267208099365, 0.2636760473251343, 2.3850573301315308, 2.1936150193214417, 0.1838168501853943, 0.938224121928215, 2.5081106424331665, 0.312656968832016, 3.3604268431663513, 1.5672679245471954, 1.7494555711746216, -0.3077901601791382, 1.4622975587844849, -2.4785069823265076], "step": 5000}
reward_curve.png CHANGED
spindleflow_model_latest.zip CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f6f19b14daaf52c171779e401edb5c73ca27af9785c0cbb20e0ef84ceca08999
3
- size 143819554
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd0cfcc124e6f9dfce8be6c8f89f16976d75318017c28e8e144f8fd969303e74
3
+ size 143819905
training_log.txt CHANGED
@@ -1,578 +1,82 @@
1
- [07:52:57] OpenAI key found — finetuner + spawn self-learning enabled.
2
- [07:52:58] Model will be pushed to: https://huggingface.co/garvitsachdeva/spindleflow-rl
3
- [07:52:58] Working directory: /home/user/app
4
- [07:52:58] Patching SentenceTransformer to CUDA...
5
- [07:53:04] SentenceTransformer → cuda ✓
6
- [07:53:04] Loading environment...
7
- [07:53:04] TieredRewardScorer → Tier-1 only (LLM judge disabled for speed) ✓
8
- [07:53:04] Generalist baseline → static simulation (0 API calls per episode) ✓
9
- [07:53:04] Running smoke test...
10
- [07:53:07] Smoke test OK — obs shape (5490,)
11
- [07:53:07] Benchmarking SentenceTransformer encode speed...
12
- [07:53:07] Encode speed : 8.0 ms/call [CUDA ✓ fast]
13
- [07:53:07] Benchmarking full env.step() speed...
14
- [07:53:09] Step speed : 33.4 ms/step [fast ✓]
15
- [07:53:09] Projected 100k steps: 56 min
16
- [07:53:10] Training on : cuda
17
- [07:53:10] Curriculum : Phase 1 — Phase 1/3 | Rolling mean: 0.000 / 0.6 | Episodes in phase: 0
18
- [07:53:10] Total steps : 30,000
19
- [07:53:10] Training started...
20
 
21
- [07:53:12] Ep 25 | reward +2.018 | Phase 1/3 | Rolling mean: 0.237 / 0.6 | Episodes in phase: 25
22
- [07:53:14] Ep 50 | reward +0.209 | Phase 1/3 | Rolling mean: -0.374 / 0.6 | Episodes in phase: 50
23
- [07:53:16] Ep 75 | reward -1.476 | Phase 1/3 | Rolling mean: -0.471 / 0.6 | Episodes in phase: 75
24
- [07:53:17] Ep 100 | reward -1.467 | Phase 1/3 | Rolling mean: -0.517 / 0.6 | Episodes in phase: 100
25
- [07:53:32] Ep 125 | reward +0.543 | Phase 1/3 | Rolling mean: -0.513 / 0.6 | Episodes in phase: 125
26
- [07:53:35] Ep 150 | reward -2.948 | Phase 1/3 | Rolling mean: -0.557 / 0.6 | Episodes in phase: 150
27
- [07:53:36] Ep 175 | reward +1.288 | Phase 1/3 | Rolling mean: -0.583 / 0.6 | Episodes in phase: 175
28
- [07:53:41] Ep 200 | reward +0.220 | Phase 1/3 | Rolling mean: -0.545 / 0.6 | Episodes in phase: 200
29
- [07:53:57] Ep 225 | reward -1.473 | Phase 1/3 | Rolling mean: -0.648 / 0.6 | Episodes in phase: 225
30
- [07:54:00] Ep 250 | reward -0.720 | Phase 1/3 | Rolling mean: -0.548 / 0.6 | Episodes in phase: 250
31
- [07:54:02] Ep 275 | reward -3.039 | Phase 1/3 | Rolling mean: -0.512 / 0.6 | Episodes in phase: 275
32
- [07:54:04] Ep 300 | reward -0.902 | Phase 1/3 | Rolling mean: -0.439 / 0.6 | Episodes in phase: 300
33
- [07:54:21] Ep 325 | reward -1.453 | Phase 1/3 | Rolling mean: -0.447 / 0.6 | Episodes in phase: 325
34
- [07:54:25] Ep 350 | reward +1.876 | Phase 1/3 | Rolling mean: -0.410 / 0.6 | Episodes in phase: 350
35
- [07:54:27] Ep 375 | reward +0.403 | Phase 1/3 | Rolling mean: -0.379 / 0.6 | Episodes in phase: 375
36
- [07:54:29] Ep 400 | reward -0.602 | Phase 1/3 | Rolling mean: -0.394 / 0.6 | Episodes in phase: 400
37
- [07:54:44] Ep 425 | reward -2.030 | Phase 1/3 | Rolling mean: -0.444 / 0.6 | Episodes in phase: 425
38
- [07:54:46] Ep 450 | reward +0.240 | Phase 1/3 | Rolling mean: -0.449 / 0.6 | Episodes in phase: 450
39
- [07:54:49] Ep 475 | reward -1.267 | Phase 1/3 | Rolling mean: -0.487 / 0.6 | Episodes in phase: 475
40
- [07:54:54] Ep 500 | reward -2.088 | Phase 1/3 | Rolling mean: -0.564 / 0.6 | Episodes in phase: 500
41
- [07:55:11] Ep 525 | reward -2.098 | Phase 1/3 | Rolling mean: -0.550 / 0.6 | Episodes in phase: 525
42
- [07:55:14] Ep 550 | reward +0.230 | Phase 1/3 | Rolling mean: -0.511 / 0.6 | Episodes in phase: 550
43
- [07:55:17] Ep 575 | reward -1.291 | Phase 1/3 | Rolling mean: -0.472 / 0.6 | Episodes in phase: 575
44
- [07:55:19] Ep 600 | reward -2.534 | Phase 1/3 | Rolling mean: -0.551 / 0.6 | Episodes in phase: 600
45
- [07:55:36] Ep 625 | reward +0.415 | Phase 1/3 | Rolling mean: -0.478 / 0.6 | Episodes in phase: 625
46
- [07:55:38] Ep 650 | reward -1.307 | Phase 1/3 | Rolling mean: -0.453 / 0.6 | Episodes in phase: 650
47
- [07:55:41] Ep 675 | reward +1.887 | Phase 1/3 | Rolling mean: -0.420 / 0.6 | Episodes in phase: 675
48
- [07:55:43] Ep 700 | reward -0.268 | Phase 1/3 | Rolling mean: -0.380 / 0.6 | Episodes in phase: 700
49
- [07:55:58] Ep 725 | reward -1.548 | Phase 1/3 | Rolling mean: -0.372 / 0.6 | Episodes in phase: 725
50
- [07:56:02] Ep 750 | reward +3.379 | Phase 1/3 | Rolling mean: -0.462 / 0.6 | Episodes in phase: 750
51
- [07:56:04] Ep 775 | reward -0.134 | Phase 1/3 | Rolling mean: -0.422 / 0.6 | Episodes in phase: 775
52
- [07:56:07] Ep 800 | reward +0.350 | Phase 1/3 | Rolling mean: -0.320 / 0.6 | Episodes in phase: 800
53
- [07:56:23] Ep 825 | reward -0.756 | Phase 1/3 | Rolling mean: -0.284 / 0.6 | Episodes in phase: 825
54
- [07:56:25] Ep 850 | reward +0.532 | Phase 1/3 | Rolling mean: -0.308 / 0.6 | Episodes in phase: 850
55
- [07:56:28] Ep 875 | reward +0.814 | Phase 1/3 | Rolling mean: -0.246 / 0.6 | Episodes in phase: 875
56
- [07:56:32] Ep 900 | reward -0.429 | Phase 1/3 | Rolling mean: -0.187 / 0.6 | Episodes in phase: 900
57
- [07:56:46] Ep 925 | reward +3.362 | Phase 1/3 | Rolling mean: -0.150 / 0.6 | Episodes in phase: 925
58
- [07:56:49] Ep 950 | reward -1.139 | Phase 1/3 | Rolling mean: -0.037 / 0.6 | Episodes in phase: 950
59
- [07:56:52] Ep 975 | reward +0.267 | Phase 1/3 | Rolling mean: -0.090 / 0.6 | Episodes in phase: 975
60
- [07:56:55] Ep 1000 | reward +0.900 | Phase 1/3 | Rolling mean: -0.015 / 0.6 | Episodes in phase: 1000
61
- [07:57:10] Ep 1025 | reward -0.576 | Phase 1/3 | Rolling mean: 0.031 / 0.6 | Episodes in phase: 1025
62
- [07:57:13] Ep 1050 | reward -1.113 | Phase 1/3 | Rolling mean: 0.062 / 0.6 | Episodes in phase: 1050
63
- [07:57:17] Ep 1075 | reward -1.574 | Phase 1/3 | Rolling mean: -0.009 / 0.6 | Episodes in phase: 1075
64
- [07:57:19] Ep 1100 | reward -0.291 | Phase 1/3 | Rolling mean: 0.049 / 0.6 | Episodes in phase: 1100
65
- [07:57:36] Ep 1125 | reward +1.694 | Phase 1/3 | Rolling mean: 0.050 / 0.6 | Episodes in phase: 1125
66
- [07:57:38] Ep 1150 | reward +0.781 | Phase 1/3 | Rolling mean: 0.078 / 0.6 | Episodes in phase: 1150
67
- [07:57:40] Ep 1175 | reward -0.938 | Phase 1/3 | Rolling mean: 0.077 / 0.6 | Episodes in phase: 1175
68
- [07:57:42] Ep 1200 | reward -0.997 | Phase 1/3 | Rolling mean: 0.031 / 0.6 | Episodes in phase: 1200
69
- [07:58:00] Ep 1225 | reward +0.129 | Phase 1/3 | Rolling mean: 0.039 / 0.6 | Episodes in phase: 1225
70
- [07:58:02] Ep 1250 | reward +0.765 | Phase 1/3 | Rolling mean: 0.134 / 0.6 | Episodes in phase: 1250
71
- [07:58:04] Ep 1275 | reward +0.867 | Phase 1/3 | Rolling mean: 0.273 / 0.6 | Episodes in phase: 1275
72
- [07:58:07] Ep 1300 | reward -0.775 | Phase 1/3 | Rolling mean: 0.221 / 0.6 | Episodes in phase: 1300
73
- [07:58:23] Ep 1325 | reward -1.674 | Phase 1/3 | Rolling mean: 0.282 / 0.6 | Episodes in phase: 1325
74
- [07:58:26] Ep 1350 | reward -0.136 | Phase 1/3 | Rolling mean: 0.298 / 0.6 | Episodes in phase: 1350
75
- [07:58:28] Ep 1375 | reward -0.152 | Phase 1/3 | Rolling mean: 0.378 / 0.6 | Episodes in phase: 1375
76
- [07:58:32] Ep 1400 | reward -0.021 | Phase 1/3 | Rolling mean: 0.404 / 0.6 | Episodes in phase: 1400
77
- [07:58:47] Ep 1425 | reward +0.736 | Phase 1/3 | Rolling mean: 0.461 / 0.6 | Episodes in phase: 1425
78
- [07:58:49] Ep 1450 | reward -0.544 | Phase 1/3 | Rolling mean: 0.417 / 0.6 | Episodes in phase: 1450
79
- [07:58:51] Ep 1475 | reward +0.547 | Phase 1/3 | Rolling mean: 0.386 / 0.6 | Episodes in phase: 1475
80
- [07:58:53] Ep 1500 | reward +0.621 | Phase 1/3 | Rolling mean: 0.475 / 0.6 | Episodes in phase: 1500
81
- [07:59:08] Periodic save at step 5,000 ...
82
- [07:59:12] Periodic push done — 5 files at step 5,000
83
- [07:59:13] Ep 1525 | reward +0.139 | Phase 1/3 | Rolling mean: 0.434 / 0.6 | Episodes in phase: 1525
84
- [07:59:18] Ep 1550 | reward +1.843 | Phase 1/3 | Rolling mean: 0.449 / 0.6 | Episodes in phase: 1550
85
- [07:59:21] Ep 1575 | reward +1.560 | Phase 1/3 | Rolling mean: 0.460 / 0.6 | Episodes in phase: 1575
86
- [07:59:23] Ep 1600 | reward +0.935 | Phase 1/3 | Rolling mean: 0.513 / 0.6 | Episodes in phase: 1600
87
- [07:59:40] Ep 1625 | reward -2.150 | Phase 1/3 | Rolling mean: 0.468 / 0.6 | Episodes in phase: 1625
88
- [07:59:42] Ep 1650 | reward +1.466 | Phase 1/3 | Rolling mean: 0.548 / 0.6 | Episodes in phase: 1650
89
- [07:59:44] Ep 1675 | reward +0.112 | Phase 1/3 | Rolling mean: 0.557 / 0.6 | Episodes in phase: 1675
90
- [07:59:44] Ep 1680 | reward +1.979 | Phase 2/3 | Rolling mean: 0.000 / 1.0 | Episodes in phase: 0
91
- [07:59:46] Ep 1700 | reward +2.247 | Phase 2/3 | Rolling mean: 0.741 / 1.0 | Episodes in phase: 20
92
- [08:00:01] Ep 1725 | reward -0.777 | Phase 2/3 | Rolling mean: 0.624 / 1.0 | Episodes in phase: 45
93
- [08:00:06] Ep 1750 | reward -1.629 | Phase 2/3 | Rolling mean: 0.511 / 1.0 | Episodes in phase: 70
94
- [08:00:08] Ep 1775 | reward +1.412 | Phase 2/3 | Rolling mean: 0.674 / 1.0 | Episodes in phase: 95
95
- [08:00:10] Ep 1800 | reward +0.696 | Phase 2/3 | Rolling mean: 0.725 / 1.0 | Episodes in phase: 120
96
- [08:00:32] Ep 1825 | reward +3.791 | Phase 2/3 | Rolling mean: 0.763 / 1.0 | Episodes in phase: 145
97
- [08:00:35] Ep 1850 | reward +0.643 | Phase 2/3 | Rolling mean: 0.697 / 1.0 | Episodes in phase: 170
98
- [08:00:37] Ep 1875 | reward -0.231 | Phase 2/3 | Rolling mean: 0.663 / 1.0 | Episodes in phase: 195
99
- [08:00:39] Ep 1900 | reward +0.922 | Phase 2/3 | Rolling mean: 0.645 / 1.0 | Episodes in phase: 220
100
- [08:00:58] Ep 1925 | reward +1.702 | Phase 2/3 | Rolling mean: 0.613 / 1.0 | Episodes in phase: 245
101
- [08:01:03] Ep 1950 | reward +0.046 | Phase 2/3 | Rolling mean: 0.702 / 1.0 | Episodes in phase: 270
102
- [08:01:05] Ep 1975 | reward +4.085 | Phase 2/3 | Rolling mean: 0.690 / 1.0 | Episodes in phase: 295
103
- [08:01:07] Ep 2000 | reward +0.513 | Phase 2/3 | Rolling mean: 0.630 / 1.0 | Episodes in phase: 320
104
- [08:01:25] Ep 2025 | reward +1.310 | Phase 2/3 | Rolling mean: 0.559 / 1.0 | Episodes in phase: 345
105
- [08:01:27] Ep 2050 | reward +1.680 | Phase 2/3 | Rolling mean: 0.656 / 1.0 | Episodes in phase: 370
106
- [08:01:28] Ep 2075 | reward -1.329 | Phase 2/3 | Rolling mean: 0.732 / 1.0 | Episodes in phase: 395
107
- [08:01:30] Ep 2100 | reward -0.745 | Phase 2/3 | Rolling mean: 0.743 / 1.0 | Episodes in phase: 420
108
- [08:01:48] Ep 2125 | reward -1.805 | Phase 2/3 | Rolling mean: 0.776 / 1.0 | Episodes in phase: 445
109
- [08:01:52] Ep 2150 | reward -0.513 | Phase 2/3 | Rolling mean: 0.838 / 1.0 | Episodes in phase: 470
110
- [08:01:53] Ep 2175 | reward +1.576 | Phase 2/3 | Rolling mean: 0.832 / 1.0 | Episodes in phase: 495
111
- [08:01:55] Ep 2200 | reward -0.518 | Phase 2/3 | Rolling mean: 0.924 / 1.0 | Episodes in phase: 520
112
- [08:02:13] Ep 2225 | reward +2.084 | Phase 3/3 | Rolling mean: 0.000 / — | Episodes in phase: 0
113
- [08:02:14] Ep 2250 | reward +2.102 | Phase 3/3 | Rolling mean: 0.763 / — | Episodes in phase: 25
114
- [08:02:16] Ep 2275 | reward +2.070 | Phase 3/3 | Rolling mean: 0.972 / — | Episodes in phase: 50
115
- [08:02:18] Ep 2300 | reward +1.526 | Phase 3/3 | Rolling mean: 1.075 / — | Episodes in phase: 75
116
- [08:02:41] Ep 2325 | reward -0.972 | Phase 3/3 | Rolling mean: 1.124 / — | Episodes in phase: 100
117
- [08:02:42] Ep 2350 | reward +2.646 | Phase 3/3 | Rolling mean: 1.119 / — | Episodes in phase: 125
118
- [08:02:46] Ep 2375 | reward +0.804 | Phase 3/3 | Rolling mean: 1.073 / — | Episodes in phase: 150
119
- [08:02:47] Ep 2400 | reward +0.805 | Phase 3/3 | Rolling mean: 1.053 / — | Episodes in phase: 175
120
- [08:03:03] Ep 2425 | reward +2.599 | Phase 3/3 | Rolling mean: 1.063 / — | Episodes in phase: 200
121
- [08:03:05] Ep 2450 | reward +1.555 | Phase 3/3 | Rolling mean: 1.132 / — | Episodes in phase: 225
122
- [08:03:07] Ep 2475 | reward +1.189 | Phase 3/3 | Rolling mean: 1.136 / — | Episodes in phase: 250
123
- [08:03:08] Ep 2500 | reward -0.174 | Phase 3/3 | Rolling mean: 1.126 / — | Episodes in phase: 275
124
- [08:03:26] Ep 2525 | reward +0.991 | Phase 3/3 | Rolling mean: 1.184 / — | Episodes in phase: 300
125
- [08:03:28] Ep 2550 | reward +0.487 | Phase 3/3 | Rolling mean: 1.187 / — | Episodes in phase: 325
126
- [08:03:29] Ep 2575 | reward +2.330 | Phase 3/3 | Rolling mean: 1.248 / — | Episodes in phase: 350
127
- [08:03:32] Ep 2600 | reward +0.458 | Phase 3/3 | Rolling mean: 1.285 / — | Episodes in phase: 375
128
- [08:03:53] Ep 2625 | reward +2.538 | Phase 3/3 | Rolling mean: 1.341 / — | Episodes in phase: 400
129
- [08:03:55] Ep 2650 | reward +0.220 | Phase 3/3 | Rolling mean: 1.298 / — | Episodes in phase: 425
130
- [08:03:56] Ep 2675 | reward +0.329 | Phase 3/3 | Rolling mean: 1.297 / — | Episodes in phase: 450
131
- [08:03:58] Ep 2700 | reward -0.493 | Phase 3/3 | Rolling mean: 1.248 / — | Episodes in phase: 475
132
- [08:04:14] Ep 2725 | reward -1.109 | Phase 3/3 | Rolling mean: 1.134 / — | Episodes in phase: 500
133
- [08:04:16] Ep 2750 | reward +1.337 | Phase 3/3 | Rolling mean: 1.200 / — | Episodes in phase: 525
134
- [08:04:17] Ep 2775 | reward +2.999 | Phase 3/3 | Rolling mean: 1.185 / — | Episodes in phase: 550
135
- [08:04:18] Ep 2800 | reward +0.851 | Phase 3/3 | Rolling mean: 1.198 / — | Episodes in phase: 575
136
- [08:04:36] Ep 2825 | reward +2.942 | Phase 3/3 | Rolling mean: 1.153 / — | Episodes in phase: 600
137
- [08:04:38] Ep 2850 | reward +1.594 | Phase 3/3 | Rolling mean: 1.208 / — | Episodes in phase: 625
138
- [08:04:39] Ep 2875 | reward +1.437 | Phase 3/3 | Rolling mean: 1.266 / — | Episodes in phase: 650
139
- [08:04:41] Ep 2900 | reward +1.378 | Phase 3/3 | Rolling mean: 1.376 / — | Episodes in phase: 675
140
- [08:05:01] Ep 2925 | reward +1.841 | Phase 3/3 | Rolling mean: 1.466 / — | Episodes in phase: 700
141
- [08:05:03] Ep 2950 | reward +1.378 | Phase 3/3 | Rolling mean: 1.383 / — | Episodes in phase: 725
142
- [08:05:04] Ep 2975 | reward +2.754 | Phase 3/3 | Rolling mean: 1.417 / — | Episodes in phase: 750
143
- [08:05:06] Ep 3000 | reward +1.674 | Phase 3/3 | Rolling mean: 1.441 / — | Episodes in phase: 775
144
- [08:05:24] Ep 3025 | reward +1.687 | Phase 3/3 | Rolling mean: 1.461 / — | Episodes in phase: 800
145
- [08:05:27] Ep 3050 | reward +3.082 | Phase 3/3 | Rolling mean: 1.481 / — | Episodes in phase: 825
146
- [08:05:29] Ep 3075 | reward +1.691 | Phase 3/3 | Rolling mean: 1.365 / — | Episodes in phase: 850
147
- [08:05:31] Ep 3100 | reward +1.570 | Phase 3/3 | Rolling mean: 1.296 / — | Episodes in phase: 875
148
- [08:05:46] Ep 3125 | reward +2.934 | Phase 3/3 | Rolling mean: 1.288 / — | Episodes in phase: 900
149
- [08:05:47] Ep 3150 | reward +2.934 | Phase 3/3 | Rolling mean: 1.372 / — | Episodes in phase: 925
150
- [08:05:48] Ep 3175 | reward +1.372 | Phase 3/3 | Rolling mean: 1.368 / — | Episodes in phase: 950
151
- [08:05:50] Ep 3200 | reward +0.083 | Phase 3/3 | Rolling mean: 1.343 / — | Episodes in phase: 975
152
- [08:06:11] Ep 3225 | reward -0.211 | Phase 3/3 | Rolling mean: 1.375 / — | Episodes in phase: 1000
153
- [08:06:12] Ep 3250 | reward +2.469 | Phase 3/3 | Rolling mean: 1.360 / — | Episodes in phase: 1025
154
- [08:06:16] Ep 3275 | reward -0.052 | Phase 3/3 | Rolling mean: 1.423 / — | Episodes in phase: 1050
155
- [08:06:17] Ep 3300 | reward +0.193 | Phase 3/3 | Rolling mean: 1.398 / — | Episodes in phase: 1075
156
- [08:06:35] Ep 3325 | reward +1.664 | Phase 3/3 | Rolling mean: 1.411 / — | Episodes in phase: 1100
157
- [08:06:36] Ep 3350 | reward +2.164 | Phase 3/3 | Rolling mean: 1.429 / — | Episodes in phase: 1125
158
- [08:06:37] Ep 3375 | reward +0.881 | Phase 3/3 | Rolling mean: 1.460 / — | Episodes in phase: 1150
159
- [08:06:39] Ep 3400 | reward -1.587 | Phase 3/3 | Rolling mean: 1.476 / — | Episodes in phase: 1175
160
- [08:07:00] Ep 3425 | reward +0.563 | Phase 3/3 | Rolling mean: 1.499 / — | Episodes in phase: 1200
161
- [08:07:01] Ep 3450 | reward +0.873 | Phase 3/3 | Rolling mean: 1.544 / — | Episodes in phase: 1225
162
- [08:07:03] Ep 3475 | reward +0.235 | Phase 3/3 | Rolling mean: 1.536 / — | Episodes in phase: 1250
163
- [08:07:04] Ep 3500 | reward +1.512 | Phase 3/3 | Rolling mean: 1.570 / — | Episodes in phase: 1275
164
- [08:07:23] Ep 3525 | reward +2.749 | Phase 3/3 | Rolling mean: 1.605 / — | Episodes in phase: 1300
165
- [08:07:25] Ep 3550 | reward -0.013 | Phase 3/3 | Rolling mean: 1.583 / — | Episodes in phase: 1325
166
- [08:07:26] Ep 3575 | reward +3.445 | Phase 3/3 | Rolling mean: 1.559 / — | Episodes in phase: 1350
167
- [08:07:27] Ep 3600 | reward -0.091 | Phase 3/3 | Rolling mean: 1.554 / — | Episodes in phase: 1375
168
- [08:07:43] Ep 3625 | reward +0.999 | Phase 3/3 | Rolling mean: 1.488 / — | Episodes in phase: 1400
169
- [08:07:44] Periodic save at step 10,000 ...
170
- [08:07:47] Periodic push done — 5 files at step 10,000
171
- [08:07:48] Ep 3650 | reward +3.507 | Phase 3/3 | Rolling mean: 1.491 / — | Episodes in phase: 1425
172
- [08:07:50] Ep 3675 | reward +1.569 | Phase 3/3 | Rolling mean: 1.523 / — | Episodes in phase: 1450
173
- [08:07:51] Ep 3700 | reward +1.832 | Phase 3/3 | Rolling mean: 1.531 / — | Episodes in phase: 1475
174
- [08:08:05] Ep 3725 | reward +1.000 | Phase 3/3 | Rolling mean: 1.487 / — | Episodes in phase: 1500
175
- [08:08:06] Ep 3750 | reward +0.162 | Phase 3/3 | Rolling mean: 1.503 / — | Episodes in phase: 1525
176
- [08:08:09] Ep 3775 | reward +0.485 | Phase 3/3 | Rolling mean: 1.498 / — | Episodes in phase: 1550
177
- [08:08:11] Ep 3800 | reward +2.840 | Phase 3/3 | Rolling mean: 1.526 / — | Episodes in phase: 1575
178
- [08:08:25] Ep 3825 | reward +0.482 | Phase 3/3 | Rolling mean: 1.608 / — | Episodes in phase: 1600
179
- [08:08:26] Ep 3850 | reward +2.716 | Phase 3/3 | Rolling mean: 1.512 / — | Episodes in phase: 1625
180
- [08:08:28] Ep 3875 | reward +2.714 | Phase 3/3 | Rolling mean: 1.549 / — | Episodes in phase: 1650
181
- [08:08:29] Ep 3900 | reward +1.553 | Phase 3/3 | Rolling mean: 1.577 / — | Episodes in phase: 1675
182
- [08:08:45] Ep 3925 | reward -0.296 | Phase 3/3 | Rolling mean: 1.540 / — | Episodes in phase: 1700
183
- [08:08:46] Ep 3950 | reward +1.561 | Phase 3/3 | Rolling mean: 1.466 / — | Episodes in phase: 1725
184
- [08:08:48] Ep 3975 | reward +4.074 | Phase 3/3 | Rolling mean: 1.487 / — | Episodes in phase: 1750
185
- [08:08:51] Ep 4000 | reward +1.566 | Phase 3/3 | Rolling mean: 1.503 / — | Episodes in phase: 1775
186
- [08:09:06] Ep 4025 | reward +3.457 | Phase 3/3 | Rolling mean: 1.455 / — | Episodes in phase: 1800
187
- [08:09:08] Ep 4050 | reward +2.916 | Phase 3/3 | Rolling mean: 1.559 / — | Episodes in phase: 1825
188
- [08:09:09] Ep 4075 | reward -1.564 | Phase 3/3 | Rolling mean: 1.541 / — | Episodes in phase: 1850
189
- [08:09:10] Ep 4100 | reward +3.629 | Phase 3/3 | Rolling mean: 1.564 / — | Episodes in phase: 1875
190
- [08:09:31] Ep 4125 | reward -0.041 | Phase 3/3 | Rolling mean: 1.616 / — | Episodes in phase: 1900
191
- [08:09:33] Ep 4150 | reward +2.066 | Phase 3/3 | Rolling mean: 1.656 / — | Episodes in phase: 1925
192
- [08:09:34] Ep 4175 | reward +2.140 | Phase 3/3 | Rolling mean: 1.644 / — | Episodes in phase: 1950
193
- [08:09:35] Ep 4200 | reward +3.126 | Phase 3/3 | Rolling mean: 1.636 / — | Episodes in phase: 1975
194
- [08:09:51] Ep 4225 | reward +1.709 | Phase 3/3 | Rolling mean: 1.699 / — | Episodes in phase: 2000
195
- [08:09:54] Ep 4250 | reward -0.055 | Phase 3/3 | Rolling mean: 1.668 / — | Episodes in phase: 2025
196
- [08:09:56] Ep 4275 | reward +0.543 | Phase 3/3 | Rolling mean: 1.624 / — | Episodes in phase: 2050
197
- [08:09:57] Ep 4300 | reward +2.236 | Phase 3/3 | Rolling mean: 1.643 / — | Episodes in phase: 2075
198
- [08:10:12] Ep 4325 | reward +0.196 | Phase 3/3 | Rolling mean: 1.628 / — | Episodes in phase: 2100
199
- [08:10:13] Ep 4350 | reward +1.938 | Phase 3/3 | Rolling mean: 1.650 / — | Episodes in phase: 2125
200
- [08:10:14] Ep 4375 | reward +1.430 | Phase 3/3 | Rolling mean: 1.648 / — | Episodes in phase: 2150
201
- [08:10:16] Ep 4400 | reward +1.555 | Phase 3/3 | Rolling mean: 1.652 / — | Episodes in phase: 2175
202
- [08:10:32] Ep 4425 | reward +2.752 | Phase 3/3 | Rolling mean: 1.617 / — | Episodes in phase: 2200
203
- [08:10:34] Ep 4450 | reward +2.292 | Phase 3/3 | Rolling mean: 1.567 / — | Episodes in phase: 2225
204
- [08:10:38] Ep 4475 | reward -0.754 | Phase 3/3 | Rolling mean: 1.611 / — | Episodes in phase: 2250
205
- [08:10:39] Ep 4500 | reward +2.128 | Phase 3/3 | Rolling mean: 1.563 / — | Episodes in phase: 2275
206
- [08:10:55] Ep 4525 | reward +0.792 | Phase 3/3 | Rolling mean: 1.522 / — | Episodes in phase: 2300
207
- [08:10:57] Ep 4550 | reward +2.848 | Phase 3/3 | Rolling mean: 1.561 / — | Episodes in phase: 2325
208
- [08:10:58] Ep 4575 | reward +1.605 | Phase 3/3 | Rolling mean: 1.600 / — | Episodes in phase: 2350
209
- [08:10:59] Ep 4600 | reward +3.546 | Phase 3/3 | Rolling mean: 1.579 / — | Episodes in phase: 2375
210
- [08:11:16] Ep 4625 | reward +0.029 | Phase 3/3 | Rolling mean: 1.570 / — | Episodes in phase: 2400
211
- [08:11:18] Ep 4650 | reward +2.394 | Phase 3/3 | Rolling mean: 1.676 / — | Episodes in phase: 2425
212
- [08:11:19] Ep 4675 | reward +2.261 | Phase 3/3 | Rolling mean: 1.612 / — | Episodes in phase: 2450
213
- [08:11:21] Ep 4700 | reward +0.827 | Phase 3/3 | Rolling mean: 1.650 / — | Episodes in phase: 2475
214
- [08:11:39] Ep 4725 | reward +3.394 | Phase 3/3 | Rolling mean: 1.728 / — | Episodes in phase: 2500
215
- [08:11:40] Ep 4750 | reward +1.881 | Phase 3/3 | Rolling mean: 1.688 / — | Episodes in phase: 2525
216
- [08:11:42] Ep 4775 | reward +2.768 | Phase 3/3 | Rolling mean: 1.665 / — | Episodes in phase: 2550
217
- [08:11:43] Ep 4800 | reward +2.454 | Phase 3/3 | Rolling mean: 1.714 / — | Episodes in phase: 2575
218
- [08:12:01] Ep 4825 | reward +2.767 | Phase 3/3 | Rolling mean: 1.745 / — | Episodes in phase: 2600
219
- [08:12:02] Ep 4850 | reward -0.418 | Phase 3/3 | Rolling mean: 1.665 / — | Episodes in phase: 2625
220
- [08:12:04] Ep 4875 | reward +1.551 | Phase 3/3 | Rolling mean: 1.779 / — | Episodes in phase: 2650
221
- [08:12:06] Ep 4900 | reward +3.347 | Phase 3/3 | Rolling mean: 1.793 / — | Episodes in phase: 2675
222
- [08:12:22] Ep 4925 | reward +1.148 | Phase 3/3 | Rolling mean: 1.783 / — | Episodes in phase: 2700
223
- [08:12:23] Ep 4950 | reward +2.564 | Phase 3/3 | Rolling mean: 1.854 / — | Episodes in phase: 2725
224
- [08:12:27] Ep 4975 | reward +1.749 | Phase 3/3 | Rolling mean: 1.908 / — | Episodes in phase: 2750
225
- [08:12:28] Ep 5000 | reward +3.049 | Phase 3/3 | Rolling mean: 1.911 / — | Episodes in phase: 2775
226
- [08:12:44] Ep 5025 | reward +3.009 | Phase 3/3 | Rolling mean: 1.889 / — | Episodes in phase: 2800
227
- [08:12:45] Ep 5050 | reward +2.506 | Phase 3/3 | Rolling mean: 1.943 / — | Episodes in phase: 2825
228
- [08:12:47] Ep 5075 | reward +1.562 | Phase 3/3 | Rolling mean: 1.932 / — | Episodes in phase: 2850
229
- [08:12:48] Ep 5100 | reward +1.220 | Phase 3/3 | Rolling mean: 1.894 / — | Episodes in phase: 2875
230
- [08:13:06] Ep 5125 | reward +2.966 | Phase 3/3 | Rolling mean: 1.891 / — | Episodes in phase: 2900
231
- [08:13:07] Ep 5150 | reward +2.490 | Phase 3/3 | Rolling mean: 1.806 / — | Episodes in phase: 2925
232
- [08:13:08] Ep 5175 | reward +3.160 | Phase 3/3 | Rolling mean: 1.779 / — | Episodes in phase: 2950
233
- [08:13:11] Ep 5200 | reward +2.764 | Phase 3/3 | Rolling mean: 1.741 / — | Episodes in phase: 2975
234
- [08:13:28] Ep 5225 | reward -1.548 | Phase 3/3 | Rolling mean: 1.686 / — | Episodes in phase: 3000
235
- [08:13:29] Ep 5250 | reward +3.422 | Phase 3/3 | Rolling mean: 1.688 / — | Episodes in phase: 3025
236
- [08:13:30] Ep 5275 | reward +1.713 | Phase 3/3 | Rolling mean: 1.656 / — | Episodes in phase: 3050
237
- [08:13:32] Ep 5300 | reward +1.742 | Phase 3/3 | Rolling mean: 1.694 / — | Episodes in phase: 3075
238
- [08:13:46] Ep 5325 | reward +1.142 | Phase 3/3 | Rolling mean: 1.640 / — | Episodes in phase: 3100
239
- [08:13:48] Ep 5350 | reward +0.820 | Phase 3/3 | Rolling mean: 1.695 / — | Episodes in phase: 3125
240
- [08:13:49] Ep 5375 | reward +1.741 | Phase 3/3 | Rolling mean: 1.693 / — | Episodes in phase: 3150
241
- [08:13:50] Ep 5400 | reward +0.858 | Phase 3/3 | Rolling mean: 1.678 / — | Episodes in phase: 3175
242
- [08:14:06] Ep 5425 | reward +0.858 | Phase 3/3 | Rolling mean: 1.710 / — | Episodes in phase: 3200
243
- [08:14:09] Ep 5450 | reward +0.983 | Phase 3/3 | Rolling mean: 1.734 / — | Episodes in phase: 3225
244
- [08:14:11] Ep 5475 | reward +0.324 | Phase 3/3 | Rolling mean: 1.743 / — | Episodes in phase: 3250
245
- [08:14:12] Ep 5500 | reward +0.666 | Phase 3/3 | Rolling mean: 1.652 / — | Episodes in phase: 3275
246
- [08:14:28] Ep 5525 | reward +0.981 | Phase 3/3 | Rolling mean: 1.719 / — | Episodes in phase: 3300
247
- [08:14:29] Ep 5550 | reward +2.169 | Phase 3/3 | Rolling mean: 1.706 / — | Episodes in phase: 3325
248
- [08:14:30] Ep 5575 | reward +0.982 | Phase 3/3 | Rolling mean: 1.679 / — | Episodes in phase: 3350
249
- [08:14:32] Ep 5600 | reward +1.002 | Phase 3/3 | Rolling mean: 1.733 / — | Episodes in phase: 3375
250
- [08:14:47] Ep 5625 | reward +3.085 | Phase 3/3 | Rolling mean: 1.797 / — | Episodes in phase: 3400
251
- [08:14:48] Ep 5650 | reward +0.114 | Phase 3/3 | Rolling mean: 1.749 / — | Episodes in phase: 3425
252
- [08:14:49] Ep 5675 | reward +2.483 | Phase 3/3 | Rolling mean: 1.784 / — | Episodes in phase: 3450
253
- [08:14:53] Ep 5700 | reward +0.842 | Phase 3/3 | Rolling mean: 1.832 / — | Episodes in phase: 3475
254
- [08:15:09] Ep 5725 | reward -1.542 | Phase 3/3 | Rolling mean: 1.815 / — | Episodes in phase: 3500
255
- [08:15:11] Ep 5750 | reward +1.748 | Phase 3/3 | Rolling mean: 1.753 / — | Episodes in phase: 3525
256
- [08:15:12] Ep 5775 | reward +1.872 | Phase 3/3 | Rolling mean: 1.796 / — | Episodes in phase: 3550
257
- [08:15:13] Ep 5800 | reward +2.027 | Phase 3/3 | Rolling mean: 1.785 / — | Episodes in phase: 3575
258
- [08:15:28] Ep 5825 | reward +2.915 | Phase 3/3 | Rolling mean: 1.741 / — | Episodes in phase: 3600
259
- [08:15:30] Ep 5850 | reward +3.336 | Phase 3/3 | Rolling mean: 1.799 / — | Episodes in phase: 3625
260
- [08:15:31] Ep 5875 | reward +0.619 | Phase 3/3 | Rolling mean: 1.748 / — | Episodes in phase: 3650
261
- [08:15:32] Ep 5900 | reward +0.004 | Phase 3/3 | Rolling mean: 1.753 / — | Episodes in phase: 3675
262
- [08:15:47] Ep 5925 | reward +2.854 | Phase 3/3 | Rolling mean: 1.796 / — | Episodes in phase: 3700
263
- [08:15:49] Ep 5950 | reward +2.325 | Phase 3/3 | Rolling mean: 1.873 / — | Episodes in phase: 3725
264
- [08:15:52] Ep 5975 | reward +2.218 | Phase 3/3 | Rolling mean: 1.895 / — | Episodes in phase: 3750
265
- [08:15:53] Ep 6000 | reward +3.170 | Phase 3/3 | Rolling mean: 1.835 / — | Episodes in phase: 3775
266
- [08:16:08] Ep 6025 | reward +0.068 | Phase 3/3 | Rolling mean: 1.834 / — | Episodes in phase: 3800
267
- [08:16:08] Periodic save at step 15,000 ...
268
- [08:16:11] Periodic push done — 5 files at step 15,000
269
- [08:16:12] Ep 6050 | reward +1.294 | Phase 3/3 | Rolling mean: 1.760 / — | Episodes in phase: 3825
270
- [08:16:13] Ep 6075 | reward -0.963 | Phase 3/3 | Rolling mean: 1.749 / — | Episodes in phase: 3850
271
- [08:16:15] Ep 6100 | reward +1.004 | Phase 3/3 | Rolling mean: 1.748 / — | Episodes in phase: 3875
272
- [08:16:30] Ep 6125 | reward +1.414 | Phase 3/3 | Rolling mean: 1.696 / — | Episodes in phase: 3900
273
- [08:16:32] Ep 6150 | reward +1.373 | Phase 3/3 | Rolling mean: 1.669 / — | Episodes in phase: 3925
274
- [08:16:33] Ep 6175 | reward +0.593 | Phase 3/3 | Rolling mean: 1.626 / — | Episodes in phase: 3950
275
- [08:16:34] Ep 6200 | reward +2.699 | Phase 3/3 | Rolling mean: 1.643 / — | Episodes in phase: 3975
276
- [08:16:54] Ep 6225 | reward +2.586 | Phase 3/3 | Rolling mean: 1.619 / — | Episodes in phase: 4000
277
- [08:16:55] Ep 6250 | reward +1.268 | Phase 3/3 | Rolling mean: 1.669 / — | Episodes in phase: 4025
278
- [08:16:57] Ep 6275 | reward +3.113 | Phase 3/3 | Rolling mean: 1.792 / — | Episodes in phase: 4050
279
- [08:16:58] Ep 6300 | reward +2.942 | Phase 3/3 | Rolling mean: 1.832 / — | Episodes in phase: 4075
280
- [08:17:16] Ep 6325 | reward +2.941 | Phase 3/3 | Rolling mean: 1.865 / — | Episodes in phase: 4100
281
- [08:17:17] Ep 6350 | reward +1.657 | Phase 3/3 | Rolling mean: 1.826 / — | Episodes in phase: 4125
282
- [08:17:19] Ep 6375 | reward +1.085 | Phase 3/3 | Rolling mean: 1.824 / — | Episodes in phase: 4150
283
- [08:17:20] Ep 6400 | reward +0.326 | Phase 3/3 | Rolling mean: 1.899 / — | Episodes in phase: 4175
284
- [08:17:34] Ep 6425 | reward -0.289 | Phase 3/3 | Rolling mean: 1.936 / — | Episodes in phase: 4200
285
- [08:17:35] Ep 6450 | reward +0.661 | Phase 3/3 | Rolling mean: 1.959 / — | Episodes in phase: 4225
286
- [08:17:39] Ep 6475 | reward +3.628 | Phase 3/3 | Rolling mean: 1.923 / — | Episodes in phase: 4250
287
- [08:17:41] Ep 6500 | reward +0.684 | Phase 3/3 | Rolling mean: 1.907 / — | Episodes in phase: 4275
288
- [08:17:58] Ep 6525 | reward +2.853 | Phase 3/3 | Rolling mean: 1.924 / — | Episodes in phase: 4300
289
- [08:17:59] Ep 6550 | reward +3.239 | Phase 3/3 | Rolling mean: 1.995 / — | Episodes in phase: 4325
290
- [08:18:01] Ep 6575 | reward -1.630 | Phase 3/3 | Rolling mean: 2.006 / — | Episodes in phase: 4350
291
- [08:18:02] Ep 6600 | reward +2.893 | Phase 3/3 | Rolling mean: 1.982 / — | Episodes in phase: 4375
292
- [08:18:20] Ep 6625 | reward +1.059 | Phase 3/3 | Rolling mean: 1.976 / — | Episodes in phase: 4400
293
- [08:18:21] Ep 6650 | reward +3.015 | Phase 3/3 | Rolling mean: 1.982 / — | Episodes in phase: 4425
294
- [08:18:22] Ep 6675 | reward +1.059 | Phase 3/3 | Rolling mean: 1.907 / — | Episodes in phase: 4450
295
- [08:18:23] Ep 6700 | reward +0.000 | Phase 3/3 | Rolling mean: 1.970 / — | Episodes in phase: 4475
296
- [08:18:41] Ep 6725 | reward +2.527 | Phase 3/3 | Rolling mean: 1.933 / — | Episodes in phase: 4500
297
- [08:18:43] Ep 6750 | reward +2.207 | Phase 3/3 | Rolling mean: 1.882 / — | Episodes in phase: 4525
298
- [08:18:44] Ep 6775 | reward +0.991 | Phase 3/3 | Rolling mean: 1.876 / — | Episodes in phase: 4550
299
- [08:18:45] Ep 6800 | reward +1.038 | Phase 3/3 | Rolling mean: 1.838 / — | Episodes in phase: 4575
300
- [08:19:00] Ep 6825 | reward +1.530 | Phase 3/3 | Rolling mean: 1.793 / — | Episodes in phase: 4600
301
- [08:19:01] Ep 6850 | reward -0.560 | Phase 3/3 | Rolling mean: 1.732 / — | Episodes in phase: 4625
302
- [08:19:02] Ep 6875 | reward +2.855 | Phase 3/3 | Rolling mean: 1.745 / — | Episodes in phase: 4650
303
- [08:19:04] Ep 6900 | reward +2.213 | Phase 3/3 | Rolling mean: 1.663 / — | Episodes in phase: 4675
304
- [08:19:18] Ep 6925 | reward +2.214 | Phase 3/3 | Rolling mean: 1.676 / — | Episodes in phase: 4700
305
- [08:19:20] Ep 6950 | reward +0.990 | Phase 3/3 | Rolling mean: 1.641 / — | Episodes in phase: 4725
306
- [08:19:23] Ep 6975 | reward +0.355 | Phase 3/3 | Rolling mean: 1.665 / — | Episodes in phase: 4750
307
- [08:19:24] Ep 7000 | reward +2.411 | Phase 3/3 | Rolling mean: 1.658 / — | Episodes in phase: 4775
308
- [08:19:39] Ep 7025 | reward +0.505 | Phase 3/3 | Rolling mean: 1.714 / — | Episodes in phase: 4800
309
- [08:19:40] Ep 7050 | reward +2.409 | Phase 3/3 | Rolling mean: 1.794 / — | Episodes in phase: 4825
310
- [08:19:41] Ep 7075 | reward +2.886 | Phase 3/3 | Rolling mean: 1.792 / — | Episodes in phase: 4850
311
- [08:19:43] Ep 7100 | reward +2.091 | Phase 3/3 | Rolling mean: 1.843 / — | Episodes in phase: 4875
312
- [08:19:59] Ep 7125 | reward +0.874 | Phase 3/3 | Rolling mean: 1.878 / — | Episodes in phase: 4900
313
- [08:20:00] Ep 7150 | reward +2.551 | Phase 3/3 | Rolling mean: 1.926 / — | Episodes in phase: 4925
314
- [08:20:01] Ep 7175 | reward +2.947 | Phase 3/3 | Rolling mean: 1.910 / — | Episodes in phase: 4950
315
- [08:20:02] Ep 7200 | reward +1.366 | Phase 3/3 | Rolling mean: 1.940 / — | Episodes in phase: 4975
316
- [08:20:20] Ep 7225 | reward +0.658 | Phase 3/3 | Rolling mean: 1.939 / — | Episodes in phase: 5000
317
- [08:20:21] Ep 7250 | reward +0.482 | Phase 3/3 | Rolling mean: 1.899 / — | Episodes in phase: 5025
318
- [08:20:22] Ep 7275 | reward +1.392 | Phase 3/3 | Rolling mean: 1.878 / — | Episodes in phase: 5050
319
- [08:20:24] Ep 7300 | reward +0.991 | Phase 3/3 | Rolling mean: 1.802 / — | Episodes in phase: 5075
320
- [08:20:40] Ep 7325 | reward +1.144 | Phase 3/3 | Rolling mean: 1.774 / — | Episodes in phase: 5100
321
- [08:20:41] Ep 7350 | reward +1.261 | Phase 3/3 | Rolling mean: 1.749 / — | Episodes in phase: 5125
322
- [08:20:42] Ep 7375 | reward +2.112 | Phase 3/3 | Rolling mean: 1.724 / — | Episodes in phase: 5150
323
- [08:20:43] Ep 7400 | reward +2.036 | Phase 3/3 | Rolling mean: 1.751 / — | Episodes in phase: 5175
324
- [08:20:59] Ep 7425 | reward +2.308 | Phase 3/3 | Rolling mean: 1.775 / — | Episodes in phase: 5200
325
- [08:21:02] Ep 7450 | reward +1.764 | Phase 3/3 | Rolling mean: 1.743 / — | Episodes in phase: 5225
326
- [08:21:03] Ep 7475 | reward +1.355 | Phase 3/3 | Rolling mean: 1.790 / — | Episodes in phase: 5250
327
- [08:21:04] Ep 7500 | reward +1.752 | Phase 3/3 | Rolling mean: 1.847 / — | Episodes in phase: 5275
328
- [08:21:20] Ep 7525 | reward +1.202 | Phase 3/3 | Rolling mean: 1.866 / — | Episodes in phase: 5300
329
- [08:21:21] Ep 7550 | reward +1.265 | Phase 3/3 | Rolling mean: 1.907 / — | Episodes in phase: 5325
330
- [08:21:22] Ep 7575 | reward +2.410 | Phase 3/3 | Rolling mean: 1.918 / — | Episodes in phase: 5350
331
- [08:21:24] Ep 7600 | reward +2.552 | Phase 3/3 | Rolling mean: 1.963 / — | Episodes in phase: 5375
332
- [08:21:37] Ep 7625 | reward +3.110 | Phase 3/3 | Rolling mean: 1.961 / — | Episodes in phase: 5400
333
- [08:21:38] Ep 7650 | reward +0.200 | Phase 3/3 | Rolling mean: 1.953 / — | Episodes in phase: 5425
334
- [08:21:40] Ep 7675 | reward +2.411 | Phase 3/3 | Rolling mean: 1.982 / — | Episodes in phase: 5450
335
- [08:21:41] Ep 7700 | reward +1.059 | Phase 3/3 | Rolling mean: 1.966 / — | Episodes in phase: 5475
336
- [08:21:57] Ep 7725 | reward +3.306 | Phase 3/3 | Rolling mean: 1.974 / — | Episodes in phase: 5500
337
- [08:21:58] Ep 7750 | reward +2.317 | Phase 3/3 | Rolling mean: 1.970 / — | Episodes in phase: 5525
338
- [08:22:00] Ep 7775 | reward +2.925 | Phase 3/3 | Rolling mean: 2.017 / — | Episodes in phase: 5550
339
- [08:22:01] Ep 7800 | reward +2.278 | Phase 3/3 | Rolling mean: 1.931 / — | Episodes in phase: 5575
340
- [08:22:20] Ep 7825 | reward +3.443 | Phase 3/3 | Rolling mean: 1.886 / — | Episodes in phase: 5600
341
- [08:22:21] Ep 7850 | reward +3.094 | Phase 3/3 | Rolling mean: 1.889 / — | Episodes in phase: 5625
342
- [08:22:22] Ep 7875 | reward +2.675 | Phase 3/3 | Rolling mean: 1.833 / — | Episodes in phase: 5650
343
- [08:22:23] Ep 7900 | reward +2.489 | Phase 3/3 | Rolling mean: 1.845 / — | Episodes in phase: 5675
344
- [08:22:37] Ep 7925 | reward +2.072 | Phase 3/3 | Rolling mean: 1.816 / — | Episodes in phase: 5700
345
- [08:22:39] Ep 7950 | reward +0.401 | Phase 3/3 | Rolling mean: 1.803 / — | Episodes in phase: 5725
346
- [08:22:42] Ep 7975 | reward +0.619 | Phase 3/3 | Rolling mean: 1.770 / — | Episodes in phase: 5750
347
- [08:22:43] Ep 8000 | reward +2.239 | Phase 3/3 | Rolling mean: 1.808 / — | Episodes in phase: 5775
348
- [08:22:59] Ep 8025 | reward +1.391 | Phase 3/3 | Rolling mean: 1.785 / — | Episodes in phase: 5800
349
- [08:23:00] Ep 8050 | reward +2.563 | Phase 3/3 | Rolling mean: 1.821 / — | Episodes in phase: 5825
350
- [08:23:02] Ep 8075 | reward -0.242 | Phase 3/3 | Rolling mean: 1.838 / — | Episodes in phase: 5850
351
- [08:23:03] Ep 8100 | reward +1.609 | Phase 3/3 | Rolling mean: 1.892 / — | Episodes in phase: 5875
352
- [08:23:18] Ep 8125 | reward +1.544 | Phase 3/3 | Rolling mean: 1.920 / — | Episodes in phase: 5900
353
- [08:23:20] Ep 8150 | reward +2.318 | Phase 3/3 | Rolling mean: 1.953 / — | Episodes in phase: 5925
354
- [08:23:21] Ep 8175 | reward +3.053 | Phase 3/3 | Rolling mean: 1.952 / — | Episodes in phase: 5950
355
- [08:23:22] Ep 8200 | reward +0.620 | Phase 3/3 | Rolling mean: 1.922 / — | Episodes in phase: 5975
356
- [08:23:38] Ep 8225 | reward +2.690 | Phase 3/3 | Rolling mean: 1.960 / — | Episodes in phase: 6000
357
- [08:23:40] Ep 8250 | reward +1.656 | Phase 3/3 | Rolling mean: 1.986 / — | Episodes in phase: 6025
358
- [08:23:41] Ep 8275 | reward +2.964 | Phase 3/3 | Rolling mean: 1.998 / — | Episodes in phase: 6050
359
- [08:23:42] Ep 8300 | reward +3.911 | Phase 3/3 | Rolling mean: 1.953 / — | Episodes in phase: 6075
360
- [08:23:57] Ep 8325 | reward +1.108 | Phase 3/3 | Rolling mean: 1.979 / — | Episodes in phase: 6100
361
- [08:23:58] Ep 8350 | reward +3.940 | Phase 3/3 | Rolling mean: 2.029 / — | Episodes in phase: 6125
362
- [08:24:00] Ep 8375 | reward +2.965 | Phase 3/3 | Rolling mean: 2.016 / — | Episodes in phase: 6150
363
- [08:24:01] Ep 8400 | reward +2.909 | Phase 3/3 | Rolling mean: 2.081 / — | Episodes in phase: 6175
364
- [08:24:16] Ep 8425 | reward -0.087 | Phase 3/3 | Rolling mean: 2.094 / — | Episodes in phase: 6200
365
- [08:24:17] Ep 8450 | reward +1.111 | Phase 3/3 | Rolling mean: 2.066 / — | Episodes in phase: 6225
366
- [08:24:20] Ep 8475 | reward +3.093 | Phase 3/3 | Rolling mean: 2.109 / — | Episodes in phase: 6250
367
- [08:24:20] Periodic save at step 20,000 ...
368
- [08:24:24] Periodic push done — 5 files at step 20,000
369
- [08:24:25] Ep 8500 | reward +3.116 | Phase 3/3 | Rolling mean: 2.121 / — | Episodes in phase: 6275
370
- [08:24:39] Ep 8525 | reward +1.944 | Phase 3/3 | Rolling mean: 2.051 / — | Episodes in phase: 6300
371
- [08:24:40] Ep 8550 | reward +2.910 | Phase 3/3 | Rolling mean: 1.994 / — | Episodes in phase: 6325
372
- [08:24:42] Ep 8575 | reward +2.345 | Phase 3/3 | Rolling mean: 2.057 / — | Episodes in phase: 6350
373
- [08:24:43] Ep 8600 | reward +1.379 | Phase 3/3 | Rolling mean: 1.980 / — | Episodes in phase: 6375
374
- [08:24:57] Ep 8625 | reward +0.658 | Phase 3/3 | Rolling mean: 1.964 / — | Episodes in phase: 6400
375
- [08:24:58] Ep 8650 | reward +1.380 | Phase 3/3 | Rolling mean: 1.911 / — | Episodes in phase: 6425
376
- [08:25:00] Ep 8675 | reward +1.496 | Phase 3/3 | Rolling mean: 1.918 / — | Episodes in phase: 6450
377
- [08:25:01] Ep 8700 | reward +1.379 | Phase 3/3 | Rolling mean: 1.919 / — | Episodes in phase: 6475
378
- [08:25:17] Ep 8725 | reward +1.979 | Phase 3/3 | Rolling mean: 1.922 / — | Episodes in phase: 6500
379
- [08:25:18] Ep 8750 | reward +1.654 | Phase 3/3 | Rolling mean: 1.968 / — | Episodes in phase: 6525
380
- [08:25:19] Ep 8775 | reward +3.113 | Phase 3/3 | Rolling mean: 1.884 / — | Episodes in phase: 6550
381
- [08:25:20] Ep 8800 | reward +2.273 | Phase 3/3 | Rolling mean: 1.943 / — | Episodes in phase: 6575
382
- [08:25:34] Ep 8825 | reward +2.554 | Phase 3/3 | Rolling mean: 1.984 / — | Episodes in phase: 6600
383
- [08:25:36] Ep 8850 | reward +2.646 | Phase 3/3 | Rolling mean: 1.994 / — | Episodes in phase: 6625
384
- [08:25:37] Ep 8875 | reward +2.192 | Phase 3/3 | Rolling mean: 1.889 / — | Episodes in phase: 6650
385
- [08:25:38] Ep 8900 | reward +1.617 | Phase 3/3 | Rolling mean: 1.906 / — | Episodes in phase: 6675
386
- [08:25:51] Ep 8925 | reward +2.647 | Phase 3/3 | Rolling mean: 1.952 / — | Episodes in phase: 6700
387
- [08:25:53] Ep 8950 | reward +2.496 | Phase 3/3 | Rolling mean: 1.884 / — | Episodes in phase: 6725
388
- [08:25:56] Ep 8975 | reward +2.195 | Phase 3/3 | Rolling mean: 1.936 / — | Episodes in phase: 6750
389
- [08:25:57] Ep 9000 | reward +2.345 | Phase 3/3 | Rolling mean: 1.899 / — | Episodes in phase: 6775
390
- [08:26:10] Ep 9025 | reward +2.477 | Phase 3/3 | Rolling mean: 1.844 / — | Episodes in phase: 6800
391
- [08:26:11] Ep 9050 | reward +2.954 | Phase 3/3 | Rolling mean: 1.885 / — | Episodes in phase: 6825
392
- [08:26:13] Ep 9075 | reward +1.034 | Phase 3/3 | Rolling mean: 1.946 / — | Episodes in phase: 6850
393
- [08:26:14] Ep 9100 | reward +2.091 | Phase 3/3 | Rolling mean: 1.933 / — | Episodes in phase: 6875
394
- [08:26:26] Ep 9125 | reward +2.799 | Phase 3/3 | Rolling mean: 1.908 / — | Episodes in phase: 6900
395
- [08:26:28] Ep 9150 | reward +1.280 | Phase 3/3 | Rolling mean: 1.920 / — | Episodes in phase: 6925
396
- [08:26:29] Ep 9175 | reward +2.092 | Phase 3/3 | Rolling mean: 1.903 / — | Episodes in phase: 6950
397
- [08:26:30] Ep 9200 | reward +2.657 | Phase 3/3 | Rolling mean: 1.945 / — | Episodes in phase: 6975
398
- [08:26:45] Ep 9225 | reward -1.512 | Phase 3/3 | Rolling mean: 1.969 / — | Episodes in phase: 7000
399
- [08:26:46] Ep 9250 | reward +1.509 | Phase 3/3 | Rolling mean: 1.948 / — | Episodes in phase: 7025
400
- [08:26:48] Ep 9275 | reward +1.571 | Phase 3/3 | Rolling mean: 1.916 / — | Episodes in phase: 7050
401
- [08:26:49] Ep 9300 | reward +2.625 | Phase 3/3 | Rolling mean: 1.896 / — | Episodes in phase: 7075
402
- [08:27:03] Ep 9325 | reward +2.590 | Phase 3/3 | Rolling mean: 1.909 / — | Episodes in phase: 7100
403
- [08:27:04] Ep 9350 | reward +1.605 | Phase 3/3 | Rolling mean: 1.943 / — | Episodes in phase: 7125
404
- [08:27:05] Ep 9375 | reward +0.890 | Phase 3/3 | Rolling mean: 1.975 / — | Episodes in phase: 7150
405
- [08:27:06] Ep 9400 | reward +2.478 | Phase 3/3 | Rolling mean: 1.925 / — | Episodes in phase: 7175
406
- [08:27:20] Ep 9425 | reward +1.982 | Phase 3/3 | Rolling mean: 1.950 / — | Episodes in phase: 7200
407
- [08:27:22] Ep 9450 | reward +1.982 | Phase 3/3 | Rolling mean: 1.954 / — | Episodes in phase: 7225
408
- [08:27:24] Ep 9475 | reward +3.113 | Phase 3/3 | Rolling mean: 2.030 / — | Episodes in phase: 7250
409
- [08:27:26] Ep 9500 | reward +1.654 | Phase 3/3 | Rolling mean: 2.060 / — | Episodes in phase: 7275
410
- [08:27:40] Ep 9525 | reward +0.484 | Phase 3/3 | Rolling mean: 2.067 / — | Episodes in phase: 7300
411
- [08:27:41] Ep 9550 | reward +3.141 | Phase 3/3 | Rolling mean: 2.053 / — | Episodes in phase: 7325
412
- [08:27:42] Ep 9575 | reward +2.562 | Phase 3/3 | Rolling mean: 2.070 / — | Episodes in phase: 7350
413
- [08:27:43] Ep 9600 | reward +2.478 | Phase 3/3 | Rolling mean: 2.065 / — | Episodes in phase: 7375
414
- [08:27:58] Ep 9625 | reward +0.659 | Phase 3/3 | Rolling mean: 2.022 / — | Episodes in phase: 7400
415
- [08:27:59] Ep 9650 | reward +3.233 | Phase 3/3 | Rolling mean: 2.038 / — | Episodes in phase: 7425
416
- [08:28:00] Ep 9675 | reward +0.474 | Phase 3/3 | Rolling mean: 1.934 / — | Episodes in phase: 7450
417
- [08:28:01] Ep 9700 | reward +0.866 | Phase 3/3 | Rolling mean: 1.898 / — | Episodes in phase: 7475
418
- [08:28:20] Ep 9725 | reward +2.296 | Phase 3/3 | Rolling mean: 1.938 / — | Episodes in phase: 7500
419
- [08:28:21] Ep 9750 | reward +1.855 | Phase 3/3 | Rolling mean: 1.926 / — | Episodes in phase: 7525
420
- [08:28:22] Ep 9775 | reward +0.782 | Phase 3/3 | Rolling mean: 1.905 / — | Episodes in phase: 7550
421
- [08:28:24] Ep 9800 | reward +2.960 | Phase 3/3 | Rolling mean: 1.954 / — | Episodes in phase: 7575
422
- [08:28:37] Ep 9825 | reward +2.961 | Phase 3/3 | Rolling mean: 1.974 / — | Episodes in phase: 7600
423
- [08:28:38] Ep 9850 | reward +2.306 | Phase 3/3 | Rolling mean: 1.992 / — | Episodes in phase: 7625
424
- [08:28:39] Ep 9875 | reward +2.611 | Phase 3/3 | Rolling mean: 2.085 / — | Episodes in phase: 7650
425
- [08:28:40] Ep 9900 | reward +0.457 | Phase 3/3 | Rolling mean: 2.099 / — | Episodes in phase: 7675
426
- [08:28:53] Ep 9925 | reward +0.458 | Phase 3/3 | Rolling mean: 2.034 / — | Episodes in phase: 7700
427
- [08:28:54] Ep 9950 | reward +0.803 | Phase 3/3 | Rolling mean: 2.032 / — | Episodes in phase: 7725
428
- [08:28:57] Ep 9975 | reward +1.350 | Phase 3/3 | Rolling mean: 2.003 / — | Episodes in phase: 7750
429
- [08:28:58] Ep 10000 | reward +3.037 | Phase 3/3 | Rolling mean: 1.991 / — | Episodes in phase: 7775
430
- [08:29:12] Ep 10025 | reward +0.658 | Phase 3/3 | Rolling mean: 1.935 / — | Episodes in phase: 7800
431
- [08:29:13] Ep 10050 | reward +0.985 | Phase 3/3 | Rolling mean: 1.886 / — | Episodes in phase: 7825
432
- [08:29:14] Ep 10075 | reward +1.687 | Phase 3/3 | Rolling mean: 1.832 / — | Episodes in phase: 7850
433
- [08:29:15] Ep 10100 | reward +1.688 | Phase 3/3 | Rolling mean: 1.776 / — | Episodes in phase: 7875
434
- [08:29:29] Ep 10125 | reward +0.801 | Phase 3/3 | Rolling mean: 1.762 / — | Episodes in phase: 7900
435
- [08:29:31] Ep 10150 | reward +0.985 | Phase 3/3 | Rolling mean: 1.787 / — | Episodes in phase: 7925
436
- [08:29:32] Ep 10175 | reward +0.660 | Phase 3/3 | Rolling mean: 1.784 / — | Episodes in phase: 7950
437
- [08:29:33] Ep 10200 | reward +0.632 | Phase 3/3 | Rolling mean: 1.787 / — | Episodes in phase: 7975
438
- [08:29:47] Ep 10225 | reward +1.526 | Phase 3/3 | Rolling mean: 1.852 / — | Episodes in phase: 8000
439
- [08:29:50] Ep 10250 | reward +1.526 | Phase 3/3 | Rolling mean: 1.900 / — | Episodes in phase: 8025
440
- [08:29:51] Ep 10275 | reward +3.283 | Phase 3/3 | Rolling mean: 1.906 / — | Episodes in phase: 8050
441
- [08:29:52] Ep 10300 | reward +2.186 | Phase 3/3 | Rolling mean: 1.991 / — | Episodes in phase: 8075
442
- [08:30:05] Ep 10325 | reward +1.118 | Phase 3/3 | Rolling mean: 2.017 / — | Episodes in phase: 8100
443
- [08:30:06] Ep 10350 | reward +3.127 | Phase 3/3 | Rolling mean: 2.023 / — | Episodes in phase: 8125
444
- [08:30:07] Ep 10375 | reward +2.992 | Phase 3/3 | Rolling mean: 2.033 / — | Episodes in phase: 8150
445
- [08:30:09] Ep 10400 | reward +3.099 | Phase 3/3 | Rolling mean: 2.033 / — | Episodes in phase: 8175
446
- [08:30:22] Ep 10425 | reward +1.935 | Phase 3/3 | Rolling mean: 2.025 / — | Episodes in phase: 8200
447
- [08:30:23] Ep 10450 | reward +0.764 | Phase 3/3 | Rolling mean: 2.032 / — | Episodes in phase: 8225
448
- [08:30:24] Ep 10475 | reward +1.935 | Phase 3/3 | Rolling mean: 2.103 / — | Episodes in phase: 8250
449
- [08:30:27] Ep 10500 | reward +2.951 | Phase 3/3 | Rolling mean: 2.061 / — | Episodes in phase: 8275
450
- [08:30:40] Ep 10525 | reward +1.709 | Phase 3/3 | Rolling mean: 2.034 / — | Episodes in phase: 8300
451
- [08:30:41] Ep 10550 | reward +2.188 | Phase 3/3 | Rolling mean: 1.996 / — | Episodes in phase: 8325
452
- [08:30:42] Ep 10575 | reward +1.201 | Phase 3/3 | Rolling mean: 2.052 / — | Episodes in phase: 8350
453
- [08:30:44] Ep 10600 | reward +0.480 | Phase 3/3 | Rolling mean: 2.033 / — | Episodes in phase: 8375
454
- [08:30:58] Ep 10625 | reward +2.980 | Phase 3/3 | Rolling mean: 2.070 / — | Episodes in phase: 8400
455
- [08:30:59] Ep 10650 | reward +1.055 | Phase 3/3 | Rolling mean: 2.034 / — | Episodes in phase: 8425
456
- [08:31:00] Ep 10675 | reward +1.155 | Phase 3/3 | Rolling mean: 1.964 / — | Episodes in phase: 8450
457
- [08:31:01] Ep 10700 | reward +0.480 | Phase 3/3 | Rolling mean: 2.021 / — | Episodes in phase: 8475
458
- [08:31:15] Ep 10725 | reward +2.531 | Phase 3/3 | Rolling mean: 2.078 / — | Episodes in phase: 8500
459
- [08:31:18] Ep 10750 | reward +2.950 | Phase 3/3 | Rolling mean: 2.079 / — | Episodes in phase: 8525
460
- [08:31:19] Ep 10775 | reward +1.978 | Phase 3/3 | Rolling mean: 1.999 / — | Episodes in phase: 8550
461
- [08:31:20] Ep 10800 | reward +2.013 | Phase 3/3 | Rolling mean: 1.963 / — | Episodes in phase: 8575
462
- [08:31:33] Ep 10825 | reward +1.748 | Phase 3/3 | Rolling mean: 1.908 / — | Episodes in phase: 8600
463
- [08:31:34] Ep 10850 | reward +2.468 | Phase 3/3 | Rolling mean: 1.910 / — | Episodes in phase: 8625
464
- [08:31:35] Ep 10875 | reward +2.013 | Phase 3/3 | Rolling mean: 1.912 / — | Episodes in phase: 8650
465
- [08:31:36] Ep 10900 | reward +2.468 | Phase 3/3 | Rolling mean: 1.867 / — | Episodes in phase: 8675
466
- [08:31:50] Ep 10925 | reward +1.010 | Phase 3/3 | Rolling mean: 1.803 / — | Episodes in phase: 8700
467
- [08:31:50] Periodic save at step 25,000 ...
468
- [08:31:53] Periodic push done — 5 files at step 25,000
469
- [08:31:54] Ep 10950 | reward -0.240 | Phase 3/3 | Rolling mean: 1.819 / — | Episodes in phase: 8725
470
- [08:31:55] Ep 10975 | reward +2.557 | Phase 3/3 | Rolling mean: 1.794 / — | Episodes in phase: 8750
471
- [08:31:58] Ep 11000 | reward +1.679 | Phase 3/3 | Rolling mean: 1.837 / — | Episodes in phase: 8775
472
- [08:32:12] Ep 11025 | reward +1.643 | Phase 3/3 | Rolling mean: 1.874 / — | Episodes in phase: 8800
473
- [08:32:13] Ep 11050 | reward +1.746 | Phase 3/3 | Rolling mean: 1.874 / — | Episodes in phase: 8825
474
- [08:32:14] Ep 11075 | reward +2.555 | Phase 3/3 | Rolling mean: 1.932 / — | Episodes in phase: 8850
475
- [08:32:15] Ep 11100 | reward +1.653 | Phase 3/3 | Rolling mean: 1.908 / — | Episodes in phase: 8875
476
- [08:32:28] Ep 11125 | reward +0.585 | Phase 3/3 | Rolling mean: 1.906 / — | Episodes in phase: 8900
477
- [08:32:30] Ep 11150 | reward -0.566 | Phase 3/3 | Rolling mean: 1.899 / — | Episodes in phase: 8925
478
- [08:32:31] Ep 11175 | reward +3.657 | Phase 3/3 | Rolling mean: 1.977 / — | Episodes in phase: 8950
479
- [08:32:32] Ep 11200 | reward +3.456 | Phase 3/3 | Rolling mean: 1.963 / — | Episodes in phase: 8975
480
- [08:32:47] Ep 11225 | reward +3.456 | Phase 3/3 | Rolling mean: 2.016 / — | Episodes in phase: 9000
481
- [08:32:50] Ep 11250 | reward +2.387 | Phase 3/3 | Rolling mean: 1.988 / — | Episodes in phase: 9025
482
- [08:32:52] Ep 11275 | reward +3.258 | Phase 3/3 | Rolling mean: 1.953 / — | Episodes in phase: 9050
483
- [08:32:53] Ep 11300 | reward +3.181 | Phase 3/3 | Rolling mean: 2.048 / — | Episodes in phase: 9075
484
- [08:33:07] Ep 11325 | reward +2.864 | Phase 3/3 | Rolling mean: 2.096 / — | Episodes in phase: 9100
485
- [08:33:08] Ep 11350 | reward +1.978 | Phase 3/3 | Rolling mean: 2.099 / — | Episodes in phase: 9125
486
- [08:33:09] Ep 11375 | reward +0.465 | Phase 3/3 | Rolling mean: 2.069 / — | Episodes in phase: 9150
487
- [08:33:10] Ep 11400 | reward +3.197 | Phase 3/3 | Rolling mean: 2.076 / — | Episodes in phase: 9175
488
- [08:33:23] Ep 11425 | reward +1.710 | Phase 3/3 | Rolling mean: 2.003 / — | Episodes in phase: 9200
489
- [08:33:24] Ep 11450 | reward +2.868 | Phase 3/3 | Rolling mean: 2.058 / — | Episodes in phase: 9225
490
- [08:33:26] Ep 11475 | reward +1.904 | Phase 3/3 | Rolling mean: 2.068 / — | Episodes in phase: 9250
491
- [08:33:28] Ep 11500 | reward +1.711 | Phase 3/3 | Rolling mean: 2.019 / — | Episodes in phase: 9275
492
- [08:33:41] Ep 11525 | reward +1.751 | Phase 3/3 | Rolling mean: 2.053 / — | Episodes in phase: 9300
493
- [08:33:42] Ep 11550 | reward +1.387 | Phase 3/3 | Rolling mean: 2.094 / — | Episodes in phase: 9325
494
- [08:33:44] Ep 11575 | reward +1.035 | Phase 3/3 | Rolling mean: 2.112 / — | Episodes in phase: 9350
495
- [08:33:45] Ep 11600 | reward +2.211 | Phase 3/3 | Rolling mean: 2.132 / — | Episodes in phase: 9375
496
- [08:33:57] Ep 11625 | reward +3.292 | Phase 3/3 | Rolling mean: 2.128 / — | Episodes in phase: 9400
497
- [08:33:58] Ep 11650 | reward +0.802 | Phase 3/3 | Rolling mean: 2.109 / — | Episodes in phase: 9425
498
- [08:33:59] Ep 11675 | reward +3.068 | Phase 3/3 | Rolling mean: 2.110 / — | Episodes in phase: 9450
499
- [08:34:00] Ep 11700 | reward +2.431 | Phase 3/3 | Rolling mean: 2.051 / — | Episodes in phase: 9475
500
- [08:34:14] Ep 11725 | reward +1.524 | Phase 3/3 | Rolling mean: 1.986 / — | Episodes in phase: 9500
501
- [08:34:17] Ep 11750 | reward +0.803 | Phase 3/3 | Rolling mean: 1.906 / — | Episodes in phase: 9525
502
- [08:34:18] Ep 11775 | reward +1.523 | Phase 3/3 | Rolling mean: 1.945 / — | Episodes in phase: 9550
503
- [08:34:19] Ep 11800 | reward +3.072 | Phase 3/3 | Rolling mean: 1.877 / — | Episodes in phase: 9575
504
- [08:34:33] Ep 11825 | reward -0.071 | Phase 3/3 | Rolling mean: 1.839 / — | Episodes in phase: 9600
505
- [08:34:35] Ep 11850 | reward +3.872 | Phase 3/3 | Rolling mean: 1.807 / — | Episodes in phase: 9625
506
- [08:34:36] Ep 11875 | reward +0.767 | Phase 3/3 | Rolling mean: 1.798 / — | Episodes in phase: 9650
507
- [08:34:37] Ep 11900 | reward +2.220 | Phase 3/3 | Rolling mean: 1.846 / — | Episodes in phase: 9675
508
- [08:34:52] Ep 11925 | reward -0.071 | Phase 3/3 | Rolling mean: 1.883 / — | Episodes in phase: 9700
509
- [08:34:54] Ep 11950 | reward +0.199 | Phase 3/3 | Rolling mean: 1.926 / — | Episodes in phase: 9725
510
- [08:34:55] Ep 11975 | reward +0.804 | Phase 3/3 | Rolling mean: 1.835 / — | Episodes in phase: 9750
511
- [08:34:58] Ep 12000 | reward +0.978 | Phase 3/3 | Rolling mean: 1.918 / — | Episodes in phase: 9775
512
- [08:35:11] Ep 12025 | reward +3.038 | Phase 3/3 | Rolling mean: 2.018 / — | Episodes in phase: 9800
513
- [08:35:13] Ep 12050 | reward +1.684 | Phase 3/3 | Rolling mean: 2.013 / — | Episodes in phase: 9825
514
- [08:35:14] Ep 12075 | reward +2.189 | Phase 3/3 | Rolling mean: 2.020 / — | Episodes in phase: 9850
515
- [08:35:16] Ep 12100 | reward +1.327 | Phase 3/3 | Rolling mean: 1.993 / — | Episodes in phase: 9875
516
- [08:35:29] Ep 12125 | reward +1.684 | Phase 3/3 | Rolling mean: 1.987 / — | Episodes in phase: 9900
517
- [08:35:31] Ep 12150 | reward +0.586 | Phase 3/3 | Rolling mean: 1.976 / — | Episodes in phase: 9925
518
- [08:35:32] Ep 12175 | reward +2.470 | Phase 3/3 | Rolling mean: 2.025 / — | Episodes in phase: 9950
519
- [08:35:34] Ep 12200 | reward +1.239 | Phase 3/3 | Rolling mean: 2.021 / — | Episodes in phase: 9975
520
- [08:35:46] Ep 12225 | reward +2.182 | Phase 3/3 | Rolling mean: 1.990 / — | Episodes in phase: 10000
521
- [08:35:49] Ep 12250 | reward +2.294 | Phase 3/3 | Rolling mean: 1.973 / — | Episodes in phase: 10025
522
- [08:35:50] Ep 12275 | reward +3.076 | Phase 3/3 | Rolling mean: 1.959 / — | Episodes in phase: 10050
523
- [08:35:52] Ep 12300 | reward +2.295 | Phase 3/3 | Rolling mean: 1.911 / — | Episodes in phase: 10075
524
- [08:36:07] Ep 12325 | reward +2.958 | Phase 3/3 | Rolling mean: 1.919 / — | Episodes in phase: 10100
525
- [08:36:09] Ep 12350 | reward +3.208 | Phase 3/3 | Rolling mean: 1.923 / — | Episodes in phase: 10125
526
- [08:36:10] Ep 12375 | reward +2.612 | Phase 3/3 | Rolling mean: 1.879 / — | Episodes in phase: 10150
527
- [08:36:11] Ep 12400 | reward +2.650 | Phase 3/3 | Rolling mean: 1.900 / — | Episodes in phase: 10175
528
- [08:36:23] Ep 12425 | reward +3.601 | Phase 3/3 | Rolling mean: 1.897 / — | Episodes in phase: 10200
529
- [08:36:25] Ep 12450 | reward +3.106 | Phase 3/3 | Rolling mean: 1.974 / — | Episodes in phase: 10225
530
- [08:36:26] Ep 12475 | reward +1.523 | Phase 3/3 | Rolling mean: 1.975 / — | Episodes in phase: 10250
531
- [08:36:27] Ep 12500 | reward +3.107 | Phase 3/3 | Rolling mean: 2.040 / — | Episodes in phase: 10275
532
- [08:36:42] Ep 12525 | reward +2.442 | Phase 3/3 | Rolling mean: 2.050 / — | Episodes in phase: 10300
533
- [08:36:43] Ep 12550 | reward +0.632 | Phase 3/3 | Rolling mean: 2.060 / — | Episodes in phase: 10325
534
- [08:36:45] Ep 12575 | reward +1.523 | Phase 3/3 | Rolling mean: 2.061 / — | Episodes in phase: 10350
535
- [08:36:46] Ep 12600 | reward +2.288 | Phase 3/3 | Rolling mean: 2.032 / — | Episodes in phase: 10375
536
- [08:36:58] Ep 12625 | reward +1.717 | Phase 3/3 | Rolling mean: 1.994 / — | Episodes in phase: 10400
537
- [08:36:59] Ep 12650 | reward +2.290 | Phase 3/3 | Rolling mean: 1.946 / — | Episodes in phase: 10425
538
- [08:37:00] Ep 12675 | reward +2.676 | Phase 3/3 | Rolling mean: 1.909 / — | Episodes in phase: 10450
539
- [08:37:02] Ep 12700 | reward +1.032 | Phase 3/3 | Rolling mean: 1.874 / — | Episodes in phase: 10475
540
- [08:37:14] Ep 12725 | reward +1.160 | Phase 3/3 | Rolling mean: 1.875 / — | Episodes in phase: 10500
541
- [08:37:15] Ep 12750 | reward +0.979 | Phase 3/3 | Rolling mean: 1.908 / — | Episodes in phase: 10525
542
- [08:37:18] Ep 12775 | reward +1.161 | Phase 3/3 | Rolling mean: 1.910 / — | Episodes in phase: 10550
543
- [08:37:19] Ep 12800 | reward +3.096 | Phase 3/3 | Rolling mean: 1.908 / — | Episodes in phase: 10575
544
- [08:37:32] Ep 12825 | reward +3.052 | Phase 3/3 | Rolling mean: 1.965 / — | Episodes in phase: 10600
545
- [08:37:33] Ep 12850 | reward +1.685 | Phase 3/3 | Rolling mean: 1.986 / — | Episodes in phase: 10625
546
- [08:37:34] Ep 12875 | reward +2.534 | Phase 3/3 | Rolling mean: 2.051 / — | Episodes in phase: 10650
547
- [08:37:35] Ep 12900 | reward +1.032 | Phase 3/3 | Rolling mean: 2.098 / — | Episodes in phase: 10675
548
- [08:37:48] Ep 12925 | reward +2.050 | Phase 3/3 | Rolling mean: 2.111 / — | Episodes in phase: 10700
549
- [08:37:50] Ep 12950 | reward +2.943 | Phase 3/3 | Rolling mean: 2.070 / — | Episodes in phase: 10725
550
- [08:37:51] Ep 12975 | reward +1.045 | Phase 3/3 | Rolling mean: 2.084 / — | Episodes in phase: 10750
551
- [08:37:52] Ep 13000 | reward +1.650 | Phase 3/3 | Rolling mean: 2.082 / — | Episodes in phase: 10775
552
- [08:38:09] Ep 13025 | reward +2.441 | Phase 3/3 | Rolling mean: 2.071 / — | Episodes in phase: 10800
553
- [08:38:10] Ep 13050 | reward +0.963 | Phase 3/3 | Rolling mean: 2.100 / — | Episodes in phase: 10825
554
- [08:38:11] Ep 13075 | reward +1.686 | Phase 3/3 | Rolling mean: 2.095 / — | Episodes in phase: 10850
555
- [08:38:13] Ep 13100 | reward +1.686 | Phase 3/3 | Rolling mean: 2.062 / — | Episodes in phase: 10875
556
- [08:38:27] Ep 13125 | reward +3.282 | Phase 3/3 | Rolling mean: 2.038 / — | Episodes in phase: 10900
557
- [08:38:28] Ep 13150 | reward +0.731 | Phase 3/3 | Rolling mean: 2.049 / — | Episodes in phase: 10925
558
- [08:38:29] Ep 13175 | reward +2.536 | Phase 3/3 | Rolling mean: 2.073 / — | Episodes in phase: 10950
559
- [08:38:30] Ep 13200 | reward +3.398 | Phase 3/3 | Rolling mean: 2.072 / — | Episodes in phase: 10975
560
- [08:38:43] Ep 13225 | reward +1.518 | Phase 3/3 | Rolling mean: 2.098 / — | Episodes in phase: 11000
561
- [08:38:44] Ep 13250 | reward +2.680 | Phase 3/3 | Rolling mean: 2.090 / — | Episodes in phase: 11025
562
- [08:38:47] Ep 13275 | reward +1.597 | Phase 3/3 | Rolling mean: 2.038 / — | Episodes in phase: 11050
563
- [08:38:48] Ep 13300 | reward +1.098 | Phase 3/3 | Rolling mean: 2.072 / — | Episodes in phase: 11075
564
- [08:39:02] Ep 13325 | reward +2.680 | Phase 3/3 | Rolling mean: 2.070 / — | Episodes in phase: 11100
565
- [08:39:03] Ep 13350 | reward +2.517 | Phase 3/3 | Rolling mean: 2.073 / — | Episodes in phase: 11125
566
- [08:39:04] Ep 13375 | reward +0.413 | Phase 3/3 | Rolling mean: 2.023 / — | Episodes in phase: 11150
567
- [08:39:05] Ep 13400 | reward +3.291 | Phase 3/3 | Rolling mean: 1.999 / — | Episodes in phase: 11175
568
- [08:39:19] Periodic save at step 30,000 ...
569
- [08:39:22] Periodic push done — 5 files at step 30,000
570
- [08:39:22] Ep 13425 | reward +1.596 | Phase 3/3 | Rolling mean: 1.962 / — | Episodes in phase: 11200
571
- [08:39:23] Ep 13450 | reward +3.219 | Phase 3/3 | Rolling mean: 1.964 / — | Episodes in phase: 11225
572
- [08:39:25] Ep 13475 | reward +3.019 | Phase 3/3 | Rolling mean: 2.014 / — | Episodes in phase: 11250
573
- [08:39:26] Ep 13500 | reward +0.549 | Phase 3/3 | Rolling mean: 2.003 / — | Episodes in phase: 11275
574
- [08:39:40] Ep 13525 | reward +3.228 | Phase 3/3 | Rolling mean: 2.011 / — | Episodes in phase: 11300
575
- [08:39:42] Model saved — 13526 episodes completed.
576
- [08:39:42] Final curriculum: Phase 3/3 | Rolling mean: 2.007 / — | Episodes in phase: 11301
577
- [08:39:43] Reward curve saved.
578
- [08:39:43] Pushing to https://huggingface.co/garvitsachdeva/spindleflow-rl ...
 
1
+ [11:05:13] OpenAI key found — finetuner + spawn self-learning enabled.
2
+ [11:05:13] Model will be pushed to: https://huggingface.co/garvitsachdeva/spindleflow-rl
3
+ [11:05:13] Working directory: /home/user/app
4
+ [11:05:13] Patching SentenceTransformer to CUDA...
5
+ [11:05:15] WARNING: CUDA not available for SentenceTransformer — CPU mode (slow)
6
+ [11:05:15] Loading environment...
7
+ [11:05:20] TieredRewardScorer → Tier-1 only (LLM judge disabled for speed) ✓
8
+ [11:05:20] Generalist baseline → static simulation (0 API calls per episode) ✓
9
+ [11:05:20] Running smoke test...
10
+ [11:05:22] Smoke test OK — obs shape (5490,)
11
+ [11:05:22] Benchmarking SentenceTransformer encode speed...
12
+ [11:05:22] Encode speed : 10.2 ms/call [CUDA ✓ fast]
13
+ [11:05:22] Benchmarking full env.step() speed...
14
+ [11:05:24] Step speed : 67.4 ms/step [fast ✓]
15
+ [11:05:24] Projected 100k steps: 112 min
16
+ [11:05:25] Training on : cpu
17
+ [11:05:25] Curriculum : Phase 1 — Phase 1/3 | Rolling mean: 0.000 / 0.6 | Episodes in phase: 0
18
+ [11:05:25] Total steps : 30,000
19
+ [11:05:25] Training started...
20
 
21
+ [11:05:32] Ep 25 | reward +0.736 | Phase 1/3 | Rolling mean: -0.185 / 0.6 | Episodes in phase: 25
22
+ [11:05:37] Ep 50 | reward +1.097 | Phase 1/3 | Rolling mean: -0.511 / 0.6 | Episodes in phase: 50
23
+ [11:05:43] Ep 75 | reward -1.508 | Phase 1/3 | Rolling mean: -0.597 / 0.6 | Episodes in phase: 75
24
+ [11:05:48] Ep 100 | reward -0.701 | Phase 1/3 | Rolling mean: -0.595 / 0.6 | Episodes in phase: 100
25
+ [11:06:10] Ep 125 | reward +0.805 | Phase 1/3 | Rolling mean: -0.463 / 0.6 | Episodes in phase: 125
26
+ [11:06:16] Ep 150 | reward -1.505 | Phase 1/3 | Rolling mean: -0.476 / 0.6 | Episodes in phase: 150
27
+ [11:07:36] Ep 175 | reward -1.475 | Phase 1/3 | Rolling mean: -0.522 / 0.6 | Episodes in phase: 175
28
+ [11:07:42] Ep 200 | reward -1.879 | Phase 1/3 | Rolling mean: -0.517 / 0.6 | Episodes in phase: 200
29
+ [11:08:06] Ep 225 | reward -1.314 | Phase 1/3 | Rolling mean: -0.555 / 0.6 | Episodes in phase: 225
30
+ [11:08:13] Ep 250 | reward -1.912 | Phase 1/3 | Rolling mean: -0.541 / 0.6 | Episodes in phase: 250
31
+ [11:08:21] Ep 275 | reward -0.073 | Phase 1/3 | Rolling mean: -0.492 / 0.6 | Episodes in phase: 275
32
+ [11:08:27] Ep 300 | reward +0.842 | Phase 1/3 | Rolling mean: -0.483 / 0.6 | Episodes in phase: 300
33
+ [11:09:50] Ep 325 | reward -1.541 | Phase 1/3 | Rolling mean: -0.526 / 0.6 | Episodes in phase: 325
34
+ [11:09:56] Ep 350 | reward -1.545 | Phase 1/3 | Rolling mean: -0.520 / 0.6 | Episodes in phase: 350
35
+ [11:10:03] Ep 375 | reward -1.545 | Phase 1/3 | Rolling mean: -0.498 / 0.6 | Episodes in phase: 375
36
+ [11:10:09] Ep 400 | reward -1.542 | Phase 1/3 | Rolling mean: -0.427 / 0.6 | Episodes in phase: 400
37
+ [11:10:32] Ep 425 | reward +2.306 | Phase 1/3 | Rolling mean: -0.443 / 0.6 | Episodes in phase: 425
38
+ [11:10:39] Ep 450 | reward -1.521 | Phase 1/3 | Rolling mean: -0.405 / 0.6 | Episodes in phase: 450
39
+ [11:11:56] Ep 475 | reward +1.361 | Phase 1/3 | Rolling mean: -0.392 / 0.6 | Episodes in phase: 475
40
+ [11:12:03] Ep 500 | reward +1.262 | Phase 1/3 | Rolling mean: -0.330 / 0.6 | Episodes in phase: 500
41
+ [11:12:25] Ep 525 | reward -1.370 | Phase 1/3 | Rolling mean: -0.343 / 0.6 | Episodes in phase: 525
42
+ [11:12:32] Ep 550 | reward -0.791 | Phase 1/3 | Rolling mean: -0.304 / 0.6 | Episodes in phase: 550
43
+ [11:12:39] Ep 575 | reward -2.053 | Phase 1/3 | Rolling mean: -0.261 / 0.6 | Episodes in phase: 575
44
+ [11:12:46] Ep 600 | reward -1.163 | Phase 1/3 | Rolling mean: -0.303 / 0.6 | Episodes in phase: 600
45
+ [11:14:12] Ep 625 | reward +0.563 | Phase 1/3 | Rolling mean: -0.280 / 0.6 | Episodes in phase: 625
46
+ [11:14:19] Ep 650 | reward -1.620 | Phase 1/3 | Rolling mean: -0.335 / 0.6 | Episodes in phase: 650
47
+ [11:14:27] Ep 675 | reward +0.994 | Phase 1/3 | Rolling mean: -0.274 / 0.6 | Episodes in phase: 675
48
+ [11:14:34] Ep 700 | reward -0.728 | Phase 1/3 | Rolling mean: -0.293 / 0.6 | Episodes in phase: 700
49
+ [11:14:55] Ep 725 | reward -0.023 | Phase 1/3 | Rolling mean: -0.249 / 0.6 | Episodes in phase: 725
50
+ [11:15:02] Ep 750 | reward +2.148 | Phase 1/3 | Rolling mean: -0.249 / 0.6 | Episodes in phase: 750
51
+ [11:16:20] Ep 775 | reward -1.583 | Phase 1/3 | Rolling mean: -0.238 / 0.6 | Episodes in phase: 775
52
+ [11:16:28] Ep 800 | reward -0.096 | Phase 1/3 | Rolling mean: -0.213 / 0.6 | Episodes in phase: 800
53
+ [11:16:50] Ep 825 | reward -0.967 | Phase 1/3 | Rolling mean: -0.204 / 0.6 | Episodes in phase: 825
54
+ [11:16:56] Ep 850 | reward +0.192 | Phase 1/3 | Rolling mean: -0.094 / 0.6 | Episodes in phase: 850
55
+ [11:17:04] Ep 875 | reward -1.050 | Phase 1/3 | Rolling mean: -0.160 / 0.6 | Episodes in phase: 875
56
+ [11:18:25] Ep 900 | reward +3.062 | Phase 1/3 | Rolling mean: -0.128 / 0.6 | Episodes in phase: 900
57
+ [11:18:47] Ep 925 | reward +1.293 | Phase 1/3 | Rolling mean: -0.108 / 0.6 | Episodes in phase: 925
58
+ [11:18:54] Ep 950 | reward +0.422 | Phase 1/3 | Rolling mean: -0.084 / 0.6 | Episodes in phase: 950
59
+ [11:19:01] Ep 975 | reward +1.644 | Phase 1/3 | Rolling mean: -0.055 / 0.6 | Episodes in phase: 975
60
+ [11:19:11] Ep 1000 | reward +1.232 | Phase 1/3 | Rolling mean: -0.082 / 0.6 | Episodes in phase: 1000
61
+ [11:19:32] Ep 1025 | reward -0.330 | Phase 1/3 | Rolling mean: -0.049 / 0.6 | Episodes in phase: 1025
62
+ [11:20:58] Ep 1050 | reward +0.055 | Phase 1/3 | Rolling mean: -0.055 / 0.6 | Episodes in phase: 1050
63
+ [11:21:05] Ep 1075 | reward -0.634 | Phase 1/3 | Rolling mean: -0.011 / 0.6 | Episodes in phase: 1075
64
+ [11:21:11] Ep 1100 | reward -1.339 | Phase 1/3 | Rolling mean: 0.009 / 0.6 | Episodes in phase: 1100
65
+ [11:21:35] Ep 1125 | reward +0.593 | Phase 1/3 | Rolling mean: -0.020 / 0.6 | Episodes in phase: 1125
66
+ [11:21:43] Ep 1150 | reward -1.622 | Phase 1/3 | Rolling mean: -0.062 / 0.6 | Episodes in phase: 1150
67
+ [11:21:50] Ep 1175 | reward -0.510 | Phase 1/3 | Rolling mean: -0.104 / 0.6 | Episodes in phase: 1175
68
+ [11:23:03] Ep 1200 | reward -2.411 | Phase 1/3 | Rolling mean: -0.054 / 0.6 | Episodes in phase: 1200
69
+ [11:23:24] Ep 1225 | reward +1.149 | Phase 1/3 | Rolling mean: 0.051 / 0.6 | Episodes in phase: 1225
70
+ [11:23:30] Ep 1250 | reward +0.250 | Phase 1/3 | Rolling mean: 0.083 / 0.6 | Episodes in phase: 1250
71
+ [11:23:37] Ep 1275 | reward +0.860 | Phase 1/3 | Rolling mean: 0.109 / 0.6 | Episodes in phase: 1275
72
+ [11:23:43] Ep 1300 | reward +2.844 | Phase 1/3 | Rolling mean: 0.121 / 0.6 | Episodes in phase: 1300
73
+ [11:24:05] Ep 1325 | reward +0.399 | Phase 1/3 | Rolling mean: 0.196 / 0.6 | Episodes in phase: 1325
74
+ [11:24:10] Ep 1350 | reward -0.790 | Phase 1/3 | Rolling mean: 0.311 / 0.6 | Episodes in phase: 1350
75
+ [11:24:15] Ep 1375 | reward -1.641 | Phase 1/3 | Rolling mean: 0.390 / 0.6 | Episodes in phase: 1375
76
+ [11:25:23] Ep 1400 | reward +0.059 | Phase 1/3 | Rolling mean: 0.465 / 0.6 | Episodes in phase: 1400
77
+ [11:25:41] Ep 1425 | reward +0.962 | Phase 1/3 | Rolling mean: 0.427 / 0.6 | Episodes in phase: 1425
78
+ [11:25:48] Ep 1450 | reward +0.184 | Phase 1/3 | Rolling mean: 0.502 / 0.6 | Episodes in phase: 1450
79
+ [11:25:53] Ep 1475 | reward +2.529 | Phase 1/3 | Rolling mean: 0.597 / 0.6 | Episodes in phase: 1475
80
+ [11:25:54] Ep 1478 | reward +3.360 | Phase 2/3 | Rolling mean: 0.000 / 1.0 | Episodes in phase: 0
81
+ [11:25:59] Ep 1500 | reward -0.227 | Phase 2/3 | Rolling mean: 0.121 / 1.0 | Episodes in phase: 22
82
+ [11:26:17] Periodic save at step 5,000 ...
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
vec_normalize_latest.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:dfbb011443883d8e840fa1ee5aabaa1811de6c62e2b52edccfc92451d8294c05
3
  size 166596
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9817d38d65e129b7b9bf6bcb2fe97feb9ada55153d9febb2577f3534ce20cb00
3
  size 166596