Logging to experiments/gym_fwalker2d/Wa01/Mon-07-Nov-2022-10-29-40-AM-CST_gym_fwalker2d_trpo_iteration_20_seed2531
Print configuration .....
{'env_name': 'gym_fwalker2d', 'random_seeds': [3214, 2431, 2531, 2231], 'save_variables': False, 'model_save_dir': '/tmp/gym_fwalker2d_models/', 'restore_variables': False, 'start_onpol_iter': 0, 'onpol_iters': 33, 'num_path_random': 6, 'num_path_onpol': 6, 'env_horizon': 1000, 'max_train_data': 200000, 'max_val_data': 100000, 'discard_ratio': 0.0, 'dynamics': {'pre_training': {'mode': 'intrinsic_reward', 'itr': 0, 'policy_itr': 20}, 'model': 'nn', 'ensemble': False, 'ensemble_model_count': 5, 'enable_particle_ensemble': True, 'particles': 5, 'obs_var': 1.0, 'intrinsic_reward_coeff': 1.0, 'ita': 1.0, 'mode': 'random', 'val': True, 'n_layers': 4, 'hidden_size': 1000, 'activation': 'relu', 'batch_size': 1000, 'learning_rate': 0.001, 'reg_coeff': 0.0, 'epochs': 200, 'kfac_params': {'learning_rate': 0.1, 'damping': 0.001, 'momentum': 0.9, 'kl_clip': 0.0001, 'cov_ema_decay': 0.99}}, 'policy': {'network_shape': [64, 64], 'init_logstd': 0.0, 'activation': 'tanh', 'reinitialize_every_itr': False}, 'trpo': {'horizon': 1000, 'gamma': 0.99, 'step_size': 0.01, 'iterations': 20, 'batch_size': 50000, 'gae': 0.95, 'visualization': False, 'visualize_iterations': [0]}, 'algo': 'trpo'}
Generating random rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 27.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 62.
Path 5 | total_timesteps 76.
Path 6 | total_timesteps 95.
Path 7 | total_timesteps 107.
Path 8 | total_timesteps 130.
Path 9 | total_timesteps 143.
Path 10 | total_timesteps 163.
Path 11 | total_timesteps 180.
Path 12 | total_timesteps 191.
Path 13 | total_timesteps 203.
Path 14 | total_timesteps 243.
Path 15 | total_timesteps 262.
Path 16 | total_timesteps 277.
Path 17 | total_timesteps 293.
Path 18 | total_timesteps 333.
Path 19 | total_timesteps 349.
Path 20 | total_timesteps 370.
Path 21 | total_timesteps 394.
Path 22 | total_timesteps 418.
Path 23 | total_timesteps 438.
Path 24 | total_timesteps 458.
Path 25 | total_timesteps 482.
Path 26 | total_timesteps 495.
Path 27 | total_timesteps 506.
Path 28 | total_timesteps 540.
Path 29 | total_timesteps 560.
Path 30 | total_timesteps 576.
Path 31 | total_timesteps 606.
Path 32 | total_timesteps 654.
Path 33 | total_timesteps 667.
Path 34 | total_timesteps 689.
Path 35 | total_timesteps 707.
Path 36 | total_timesteps 718.
Path 37 | total_timesteps 753.
Path 38 | total_timesteps 766.
Path 39 | total_timesteps 790.
Path 40 | total_timesteps 806.
Path 41 | total_timesteps 819.
Path 42 | total_timesteps 841.
Path 43 | total_timesteps 863.
Path 44 | total_timesteps 883.
Path 45 | total_timesteps 894.
Path 46 | total_timesteps 905.
Path 47 | total_timesteps 923.
Path 48 | total_timesteps 935.
Path 49 | total_timesteps 945.
Path 50 | total_timesteps 969.
Path 51 | total_timesteps 986.
Path 52 | total_timesteps 1008.
Path 53 | total_timesteps 1046.
Path 54 | total_timesteps 1063.
Path 55 | total_timesteps 1085.
Path 56 | total_timesteps 1115.
Path 57 | total_timesteps 1130.
Path 58 | total_timesteps 1150.
Path 59 | total_timesteps 1184.
Path 60 | total_timesteps 1201.
Path 61 | total_timesteps 1222.
Path 62 | total_timesteps 1240.
Path 63 | total_timesteps 1256.
Path 64 | total_timesteps 1284.
Path 65 | total_timesteps 1310.
Path 66 | total_timesteps 1332.
Path 67 | total_timesteps 1351.
Path 68 | total_timesteps 1364.
Path 69 | total_timesteps 1383.
Path 70 | total_timesteps 1402.
Path 71 | total_timesteps 1417.
Path 72 | total_timesteps 1432.
Path 73 | total_timesteps 1456.
Path 74 | total_timesteps 1484.
Path 75 | total_timesteps 1506.
Path 76 | total_timesteps 1515.
Path 77 | total_timesteps 1531.
Path 78 | total_timesteps 1545.
Path 79 | total_timesteps 1572.
Path 80 | total_timesteps 1589.
Path 81 | total_timesteps 1606.
Path 82 | total_timesteps 1641.
Path 83 | total_timesteps 1653.
Path 84 | total_timesteps 1668.
Path 85 | total_timesteps 1694.
Path 86 | total_timesteps 1707.
Path 87 | total_timesteps 1725.
Path 88 | total_timesteps 1736.
Path 89 | total_timesteps 1748.
Path 90 | total_timesteps 1782.
Path 91 | total_timesteps 1797.
Path 92 | total_timesteps 1818.
Path 93 | total_timesteps 1832.
Path 94 | total_timesteps 1865.
Path 95 | total_timesteps 1886.
Path 96 | total_timesteps 1915.
Path 97 | total_timesteps 1925.
Path 98 | total_timesteps 1937.
Path 99 | total_timesteps 1959.
Path 100 | total_timesteps 1989.
Path 101 | total_timesteps 2003.
Path 102 | total_timesteps 2033.
Path 103 | total_timesteps 2051.
Path 104 | total_timesteps 2069.
Path 105 | total_timesteps 2086.
Path 106 | total_timesteps 2115.
Path 107 | total_timesteps 2129.
Path 108 | total_timesteps 2152.
Path 109 | total_timesteps 2176.
Path 110 | total_timesteps 2188.
Path 111 | total_timesteps 2202.
Path 112 | total_timesteps 2222.
Path 113 | total_timesteps 2239.
Path 114 | total_timesteps 2254.
Path 115 | total_timesteps 2269.
Path 116 | total_timesteps 2302.
Path 117 | total_timesteps 2345.
Path 118 | total_timesteps 2358.
Path 119 | total_timesteps 2379.
Path 120 | total_timesteps 2396.
Path 121 | total_timesteps 2421.
Path 122 | total_timesteps 2444.
Path 123 | total_timesteps 2469.
Path 124 | total_timesteps 2481.
Path 125 | total_timesteps 2494.
Path 126 | total_timesteps 2537.
Path 127 | total_timesteps 2556.
Path 128 | total_timesteps 2575.
Path 129 | total_timesteps 2608.
Path 130 | total_timesteps 2622.
Path 131 | total_timesteps 2635.
Path 132 | total_timesteps 2652.
Path 133 | total_timesteps 2665.
Path 134 | total_timesteps 2688.
Path 135 | total_timesteps 2700.
Path 136 | total_timesteps 2721.
Path 137 | total_timesteps 2737.
Path 138 | total_timesteps 2751.
Path 139 | total_timesteps 2779.
Path 140 | total_timesteps 2810.
Path 141 | total_timesteps 2835.
Path 142 | total_timesteps 2868.
Path 143 | total_timesteps 2879.
Path 144 | total_timesteps 2895.
Path 145 | total_timesteps 2918.
Path 146 | total_timesteps 2938.
Path 147 | total_timesteps 2951.
Path 148 | total_timesteps 2969.
Path 149 | total_timesteps 2989.
Path 150 | total_timesteps 3010.
Path 151 | total_timesteps 3027.
Path 152 | total_timesteps 3038.
Path 153 | total_timesteps 3069.
Path 154 | total_timesteps 3080.
Path 155 | total_timesteps 3092.
Path 156 | total_timesteps 3105.
Path 157 | total_timesteps 3132.
Path 158 | total_timesteps 3153.
Path 159 | total_timesteps 3164.
Path 160 | total_timesteps 3183.
Path 161 | total_timesteps 3234.
Path 162 | total_timesteps 3278.
Path 163 | total_timesteps 3294.
Path 164 | total_timesteps 3333.
Path 165 | total_timesteps 3355.
Path 166 | total_timesteps 3376.
Path 167 | total_timesteps 3400.
Path 168 | total_timesteps 3431.
Path 169 | total_timesteps 3455.
Path 170 | total_timesteps 3482.
Path 171 | total_timesteps 3499.
Path 172 | total_timesteps 3515.
Path 173 | total_timesteps 3541.
Path 174 | total_timesteps 3551.
Path 175 | total_timesteps 3585.
Path 176 | total_timesteps 3623.
Path 177 | total_timesteps 3658.
Path 178 | total_timesteps 3692.
Path 179 | total_timesteps 3713.
Path 180 | total_timesteps 3742.
Path 181 | total_timesteps 3771.
Path 182 | total_timesteps 3781.
Path 183 | total_timesteps 3799.
Path 184 | total_timesteps 3816.
Path 185 | total_timesteps 3880.
Path 186 | total_timesteps 3916.
Path 187 | total_timesteps 3930.
Path 188 | total_timesteps 3948.
Path 189 | total_timesteps 3965.
Path 190 | total_timesteps 4005.
Path 191 | total_timesteps 4021.
Path 192 | total_timesteps 4059.
Path 193 | total_timesteps 4075.
Path 194 | total_timesteps 4093.
Path 195 | total_timesteps 4108.
Path 196 | total_timesteps 4124.
Path 197 | total_timesteps 4150.
Path 198 | total_timesteps 4164.
Path 199 | total_timesteps 4202.
Path 200 | total_timesteps 4224.
Path 201 | total_timesteps 4240.
Path 202 | total_timesteps 4255.
Path 203 | total_timesteps 4269.
Path 204 | total_timesteps 4303.
Path 205 | total_timesteps 4330.
Path 206 | total_timesteps 4358.
Path 207 | total_timesteps 4374.
Path 208 | total_timesteps 4420.
Path 209 | total_timesteps 4443.
Path 210 | total_timesteps 4458.
Path 211 | total_timesteps 4481.
Path 212 | total_timesteps 4499.
Path 213 | total_timesteps 4524.
Path 214 | total_timesteps 4546.
Path 215 | total_timesteps 4563.
Path 216 | total_timesteps 4581.
Path 217 | total_timesteps 4606.
Path 218 | total_timesteps 4622.
Path 219 | total_timesteps 4633.
Path 220 | total_timesteps 4686.
Path 221 | total_timesteps 4699.
Path 222 | total_timesteps 4716.
Path 223 | total_timesteps 4733.
Path 224 | total_timesteps 4748.
Path 225 | total_timesteps 4765.
Path 226 | total_timesteps 4785.
Path 227 | total_timesteps 4807.
Path 228 | total_timesteps 4831.
Path 229 | total_timesteps 4848.
Path 230 | total_timesteps 4871.
Path 231 | total_timesteps 4894.
Path 232 | total_timesteps 4907.
Path 233 | total_timesteps 4924.
Path 234 | total_timesteps 4937.
Path 235 | total_timesteps 4953.
Path 236 | total_timesteps 4984.
Path 237 | total_timesteps 4992.
Path 238 | total_timesteps 5012.
Path 239 | total_timesteps 5032.
Path 240 | total_timesteps 5053.
Path 241 | total_timesteps 5076.
Path 242 | total_timesteps 5096.
Path 243 | total_timesteps 5113.
Path 244 | total_timesteps 5125.
Path 245 | total_timesteps 5139.
Path 246 | total_timesteps 5175.
Path 247 | total_timesteps 5196.
Path 248 | total_timesteps 5208.
Path 249 | total_timesteps 5232.
Path 250 | total_timesteps 5252.
Path 251 | total_timesteps 5267.
Path 252 | total_timesteps 5284.
Path 253 | total_timesteps 5309.
Path 254 | total_timesteps 5363.
Path 255 | total_timesteps 5386.
Path 256 | total_timesteps 5408.
Path 257 | total_timesteps 5419.
Path 258 | total_timesteps 5453.
Path 259 | total_timesteps 5469.
Path 260 | total_timesteps 5490.
Path 261 | total_timesteps 5548.
Path 262 | total_timesteps 5569.
Path 263 | total_timesteps 5593.
Path 264 | total_timesteps 5618.
Path 265 | total_timesteps 5632.
Path 266 | total_timesteps 5653.
Path 267 | total_timesteps 5676.
Path 268 | total_timesteps 5700.
Path 269 | total_timesteps 5711.
Path 270 | total_timesteps 5723.
Path 271 | total_timesteps 5739.
Path 272 | total_timesteps 5751.
Path 273 | total_timesteps 5760.
Path 274 | total_timesteps 5782.
Path 275 | total_timesteps 5797.
Path 276 | total_timesteps 5818.
Path 277 | total_timesteps 5840.
Path 278 | total_timesteps 5856.
Path 279 | total_timesteps 5874.
Path 280 | total_timesteps 5888.
Path 281 | total_timesteps 5909.
Path 282 | total_timesteps 5935.
Path 283 | total_timesteps 5951.
Path 284 | total_timesteps 5970.
Path 285 | total_timesteps 5999.
Done generating random rollouts.
Creating normalization for training data.
Done creating normalization for training data.
Train dynamics model with intrinsic reward only? False
Pre-training enabled. Using only intrinsic reward.
Pre-training dynamics model for 0 iterations...
Done pre-training dynamics model.
Using external reward only.
itr #0 | 
Fitting dynamics.
Validation loss = 0.5097161531448364
Validation loss = 0.13215892016887665
Validation loss = 0.0991649180650711
Validation loss = 0.08560658991336823
Validation loss = 0.07787848263978958
Validation loss = 0.07013669610023499
Validation loss = 0.06970594823360443
Validation loss = 0.06209917366504669
Validation loss = 0.06145311892032623
Validation loss = 0.057358600199222565
Validation loss = 0.0588708333671093
Validation loss = 0.05520766228437424
Validation loss = 0.05080099776387215
Validation loss = 0.052702270448207855
Validation loss = 0.05266589671373367
Validation loss = 0.06405968964099884
Validation loss = 0.04908766224980354
Validation loss = 0.0528903603553772
Validation loss = 0.04787009209394455
Validation loss = 0.05388951301574707
Validation loss = 0.049026668071746826
Validation loss = 0.04975567013025284
Validation loss = 0.04719332233071327
Validation loss = 0.05349283665418625
Validation loss = 0.04768304526805878
Validation loss = 0.05712835490703583
Validation loss = 0.04678318649530411
Validation loss = 0.05842461809515953
Validation loss = 0.04729752242565155
Validation loss = 0.05859935283660889
Validation loss = 0.04563703387975693
Validation loss = 0.04416162148118019
Validation loss = 0.04638083279132843
Validation loss = 0.04414055123925209
Validation loss = 0.056582845747470856
Validation loss = 0.044671401381492615
Validation loss = 0.04376562684774399
Validation loss = 0.0467253252863884
Validation loss = 0.04421275109052658
Validation loss = 0.041967324912548065
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 29.
Path 2 | total_timesteps 49.
Path 3 | total_timesteps 58.
Path 4 | total_timesteps 75.
Path 5 | total_timesteps 99.
Path 6 | total_timesteps 114.
Path 7 | total_timesteps 140.
Path 8 | total_timesteps 156.
Path 9 | total_timesteps 175.
Path 10 | total_timesteps 197.
Path 11 | total_timesteps 218.
Path 12 | total_timesteps 279.
Path 13 | total_timesteps 303.
Path 14 | total_timesteps 324.
Path 15 | total_timesteps 337.
Path 16 | total_timesteps 356.
Path 17 | total_timesteps 382.
Path 18 | total_timesteps 400.
Path 19 | total_timesteps 412.
Path 20 | total_timesteps 429.
Path 21 | total_timesteps 440.
Path 22 | total_timesteps 465.
Path 23 | total_timesteps 501.
Path 24 | total_timesteps 522.
Path 25 | total_timesteps 552.
Path 26 | total_timesteps 585.
Path 27 | total_timesteps 598.
Path 28 | total_timesteps 623.
Path 29 | total_timesteps 636.
Path 30 | total_timesteps 648.
Path 31 | total_timesteps 664.
Path 32 | total_timesteps 671.
Path 33 | total_timesteps 690.
Path 34 | total_timesteps 700.
Path 35 | total_timesteps 722.
Path 36 | total_timesteps 744.
Path 37 | total_timesteps 761.
Path 38 | total_timesteps 792.
Path 39 | total_timesteps 812.
Path 40 | total_timesteps 833.
Path 41 | total_timesteps 862.
Path 42 | total_timesteps 877.
Path 43 | total_timesteps 897.
Path 44 | total_timesteps 920.
Path 45 | total_timesteps 936.
Path 46 | total_timesteps 943.
Path 47 | total_timesteps 952.
Path 48 | total_timesteps 970.
Path 49 | total_timesteps 983.
Path 50 | total_timesteps 1001.
Path 51 | total_timesteps 1018.
Path 52 | total_timesteps 1043.
Path 53 | total_timesteps 1060.
Path 54 | total_timesteps 1093.
Path 55 | total_timesteps 1105.
Path 56 | total_timesteps 1125.
Path 57 | total_timesteps 1139.
Path 58 | total_timesteps 1147.
Path 59 | total_timesteps 1162.
Path 60 | total_timesteps 1174.
Path 61 | total_timesteps 1191.
Path 62 | total_timesteps 1201.
Path 63 | total_timesteps 1210.
Path 64 | total_timesteps 1225.
Path 65 | total_timesteps 1237.
Path 66 | total_timesteps 1259.
Path 67 | total_timesteps 1269.
Path 68 | total_timesteps 1278.
Path 69 | total_timesteps 1294.
Path 70 | total_timesteps 1302.
Path 71 | total_timesteps 1313.
Path 72 | total_timesteps 1324.
Path 73 | total_timesteps 1362.
Path 74 | total_timesteps 1386.
Path 75 | total_timesteps 1402.
Path 76 | total_timesteps 1413.
Path 77 | total_timesteps 1425.
Path 78 | total_timesteps 1456.
Path 79 | total_timesteps 1475.
Path 80 | total_timesteps 1497.
Path 81 | total_timesteps 1546.
Path 82 | total_timesteps 1564.
Path 83 | total_timesteps 1581.
Path 84 | total_timesteps 1599.
Path 85 | total_timesteps 1609.
Path 86 | total_timesteps 1621.
Path 87 | total_timesteps 1639.
Path 88 | total_timesteps 1661.
Path 89 | total_timesteps 1671.
Path 90 | total_timesteps 1690.
Path 91 | total_timesteps 1711.
Path 92 | total_timesteps 1723.
Path 93 | total_timesteps 1734.
Path 94 | total_timesteps 1743.
Path 95 | total_timesteps 1761.
Path 96 | total_timesteps 1774.
Path 97 | total_timesteps 1792.
Path 98 | total_timesteps 1819.
Path 99 | total_timesteps 1842.
Path 100 | total_timesteps 1867.
Path 101 | total_timesteps 1900.
Path 102 | total_timesteps 1910.
Path 103 | total_timesteps 1920.
Path 104 | total_timesteps 1948.
Path 105 | total_timesteps 1967.
Path 106 | total_timesteps 1979.
Path 107 | total_timesteps 1998.
Path 108 | total_timesteps 2018.
Path 109 | total_timesteps 2026.
Path 110 | total_timesteps 2052.
Path 111 | total_timesteps 2064.
Path 112 | total_timesteps 2088.
Path 113 | total_timesteps 2143.
Path 114 | total_timesteps 2169.
Path 115 | total_timesteps 2190.
Path 116 | total_timesteps 2221.
Path 117 | total_timesteps 2242.
Path 118 | total_timesteps 2256.
Path 119 | total_timesteps 2274.
Path 120 | total_timesteps 2288.
Path 121 | total_timesteps 2298.
Path 122 | total_timesteps 2317.
Path 123 | total_timesteps 2339.
Path 124 | total_timesteps 2355.
Path 125 | total_timesteps 2369.
Path 126 | total_timesteps 2384.
Path 127 | total_timesteps 2396.
Path 128 | total_timesteps 2411.
Path 129 | total_timesteps 2436.
Path 130 | total_timesteps 2447.
Path 131 | total_timesteps 2465.
Path 132 | total_timesteps 2494.
Path 133 | total_timesteps 2527.
Path 134 | total_timesteps 2543.
Path 135 | total_timesteps 2566.
Path 136 | total_timesteps 2587.
Path 137 | total_timesteps 2602.
Path 138 | total_timesteps 2628.
Path 139 | total_timesteps 2645.
Path 140 | total_timesteps 2666.
Path 141 | total_timesteps 2682.
Path 142 | total_timesteps 2695.
Path 143 | total_timesteps 2713.
Path 144 | total_timesteps 2729.
Path 145 | total_timesteps 2751.
Path 146 | total_timesteps 2766.
Path 147 | total_timesteps 2788.
Path 148 | total_timesteps 2811.
Path 149 | total_timesteps 2831.
Path 150 | total_timesteps 2847.
Path 151 | total_timesteps 2871.
Path 152 | total_timesteps 2887.
Path 153 | total_timesteps 2912.
Path 154 | total_timesteps 2933.
Path 155 | total_timesteps 2953.
Path 156 | total_timesteps 2992.
Path 157 | total_timesteps 3015.
Path 158 | total_timesteps 3024.
Path 159 | total_timesteps 3051.
Path 160 | total_timesteps 3067.
Path 161 | total_timesteps 3083.
Path 162 | total_timesteps 3100.
Path 163 | total_timesteps 3126.
Path 164 | total_timesteps 3155.
Path 165 | total_timesteps 3170.
Path 166 | total_timesteps 3180.
Path 167 | total_timesteps 3207.
Path 168 | total_timesteps 3226.
Path 169 | total_timesteps 3255.
Path 170 | total_timesteps 3266.
Path 171 | total_timesteps 3279.
Path 172 | total_timesteps 3308.
Path 173 | total_timesteps 3328.
Path 174 | total_timesteps 3350.
Path 175 | total_timesteps 3369.
Path 176 | total_timesteps 3430.
Path 177 | total_timesteps 3454.
Path 178 | total_timesteps 3468.
Path 179 | total_timesteps 3483.
Path 180 | total_timesteps 3492.
Path 181 | total_timesteps 3519.
Path 182 | total_timesteps 3537.
Path 183 | total_timesteps 3571.
Path 184 | total_timesteps 3587.
Path 185 | total_timesteps 3609.
Path 186 | total_timesteps 3618.
Path 187 | total_timesteps 3642.
Path 188 | total_timesteps 3657.
Path 189 | total_timesteps 3684.
Path 190 | total_timesteps 3704.
Path 191 | total_timesteps 3721.
Path 192 | total_timesteps 3743.
Path 193 | total_timesteps 3762.
Path 194 | total_timesteps 3784.
Path 195 | total_timesteps 3812.
Path 196 | total_timesteps 3829.
Path 197 | total_timesteps 3857.
Path 198 | total_timesteps 3877.
Path 199 | total_timesteps 3909.
Path 200 | total_timesteps 3921.
Path 201 | total_timesteps 3936.
Path 202 | total_timesteps 3943.
Path 203 | total_timesteps 3964.
Path 204 | total_timesteps 3984.
Path 205 | total_timesteps 4025.
Path 206 | total_timesteps 4041.
Path 207 | total_timesteps 4071.
Path 208 | total_timesteps 4091.
Path 209 | total_timesteps 4112.
Path 210 | total_timesteps 4127.
Path 211 | total_timesteps 4146.
Path 212 | total_timesteps 4166.
Path 213 | total_timesteps 4183.
Path 214 | total_timesteps 4196.
Path 215 | total_timesteps 4221.
Path 216 | total_timesteps 4241.
Path 217 | total_timesteps 4252.
Path 218 | total_timesteps 4263.
Path 219 | total_timesteps 4291.
Path 220 | total_timesteps 4301.
Path 221 | total_timesteps 4319.
Path 222 | total_timesteps 4340.
Path 223 | total_timesteps 4352.
Path 224 | total_timesteps 4364.
Path 225 | total_timesteps 4377.
Path 226 | total_timesteps 4395.
Path 227 | total_timesteps 4407.
Path 228 | total_timesteps 4438.
Path 229 | total_timesteps 4448.
Path 230 | total_timesteps 4475.
Path 231 | total_timesteps 4503.
Path 232 | total_timesteps 4520.
Path 233 | total_timesteps 4532.
Path 234 | total_timesteps 4559.
Path 235 | total_timesteps 4570.
Path 236 | total_timesteps 4585.
Path 237 | total_timesteps 4599.
Path 238 | total_timesteps 4630.
Path 239 | total_timesteps 4662.
Path 240 | total_timesteps 4675.
Path 241 | total_timesteps 4684.
Path 242 | total_timesteps 4706.
Path 243 | total_timesteps 4732.
Path 244 | total_timesteps 4749.
Path 245 | total_timesteps 4763.
Path 246 | total_timesteps 4801.
Path 247 | total_timesteps 4810.
Path 248 | total_timesteps 4832.
Path 249 | total_timesteps 4856.
Path 250 | total_timesteps 4869.
Path 251 | total_timesteps 4889.
Path 252 | total_timesteps 4898.
Path 253 | total_timesteps 4909.
Path 254 | total_timesteps 4932.
Path 255 | total_timesteps 4947.
Path 256 | total_timesteps 4969.
Path 257 | total_timesteps 4982.
Path 258 | total_timesteps 4997.
Path 259 | total_timesteps 5026.
Path 260 | total_timesteps 5059.
Path 261 | total_timesteps 5072.
Path 262 | total_timesteps 5085.
Path 263 | total_timesteps 5114.
Path 264 | total_timesteps 5132.
Path 265 | total_timesteps 5151.
Path 266 | total_timesteps 5170.
Path 267 | total_timesteps 5185.
Path 268 | total_timesteps 5195.
Path 269 | total_timesteps 5216.
Path 270 | total_timesteps 5242.
Path 271 | total_timesteps 5259.
Path 272 | total_timesteps 5276.
Path 273 | total_timesteps 5287.
Path 274 | total_timesteps 5305.
Path 275 | total_timesteps 5320.
Path 276 | total_timesteps 5343.
Path 277 | total_timesteps 5356.
Path 278 | total_timesteps 5380.
Path 279 | total_timesteps 5389.
Path 280 | total_timesteps 5416.
Path 281 | total_timesteps 5435.
Path 282 | total_timesteps 5453.
Path 283 | total_timesteps 5470.
Path 284 | total_timesteps 5508.
Path 285 | total_timesteps 5544.
Path 286 | total_timesteps 5573.
Path 287 | total_timesteps 5584.
Path 288 | total_timesteps 5597.
Path 289 | total_timesteps 5614.
Path 290 | total_timesteps 5636.
Path 291 | total_timesteps 5651.
Path 292 | total_timesteps 5665.
Path 293 | total_timesteps 5684.
Path 294 | total_timesteps 5710.
Path 295 | total_timesteps 5737.
Path 296 | total_timesteps 5766.
Path 297 | total_timesteps 5774.
Path 298 | total_timesteps 5788.
Path 299 | total_timesteps 5804.
Path 300 | total_timesteps 5831.
Path 301 | total_timesteps 5852.
Path 302 | total_timesteps 5878.
Path 303 | total_timesteps 5902.
Path 304 | total_timesteps 5920.
Path 305 | total_timesteps 5934.
Path 306 | total_timesteps 5960.
Path 307 | total_timesteps 5980.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.43    |
| Iteration     | 0        |
| MaximumReturn | 8.71     |
| MinimumReturn | -26.4    |
| TotalSamples  | 8009     |
----------------------------
itr #1 | 
Fitting dynamics.
Validation loss = 0.08125370740890503
Validation loss = 0.05806147679686546
Validation loss = 0.05869334191083908
Validation loss = 0.041910506784915924
Validation loss = 0.03957894444465637
Validation loss = 0.04369555413722992
Validation loss = 0.039993446320295334
Validation loss = 0.040078505873680115
Validation loss = 0.0356857106089592
Validation loss = 0.035994138568639755
Validation loss = 0.037234362214803696
Validation loss = 0.03598063066601753
Validation loss = 0.039173953235149384
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 18.
Path 2 | total_timesteps 47.
Path 3 | total_timesteps 66.
Path 4 | total_timesteps 97.
Path 5 | total_timesteps 111.
Path 6 | total_timesteps 133.
Path 7 | total_timesteps 154.
Path 8 | total_timesteps 178.
Path 9 | total_timesteps 200.
Path 10 | total_timesteps 213.
Path 11 | total_timesteps 243.
Path 12 | total_timesteps 264.
Path 13 | total_timesteps 286.
Path 14 | total_timesteps 316.
Path 15 | total_timesteps 326.
Path 16 | total_timesteps 348.
Path 17 | total_timesteps 367.
Path 18 | total_timesteps 382.
Path 19 | total_timesteps 416.
Path 20 | total_timesteps 430.
Path 21 | total_timesteps 440.
Path 22 | total_timesteps 456.
Path 23 | total_timesteps 467.
Path 24 | total_timesteps 493.
Path 25 | total_timesteps 505.
Path 26 | total_timesteps 529.
Path 27 | total_timesteps 551.
Path 28 | total_timesteps 562.
Path 29 | total_timesteps 576.
Path 30 | total_timesteps 596.
Path 31 | total_timesteps 611.
Path 32 | total_timesteps 622.
Path 33 | total_timesteps 641.
Path 34 | total_timesteps 668.
Path 35 | total_timesteps 682.
Path 36 | total_timesteps 696.
Path 37 | total_timesteps 746.
Path 38 | total_timesteps 769.
Path 39 | total_timesteps 794.
Path 40 | total_timesteps 813.
Path 41 | total_timesteps 837.
Path 42 | total_timesteps 856.
Path 43 | total_timesteps 869.
Path 44 | total_timesteps 879.
Path 45 | total_timesteps 896.
Path 46 | total_timesteps 920.
Path 47 | total_timesteps 940.
Path 48 | total_timesteps 953.
Path 49 | total_timesteps 970.
Path 50 | total_timesteps 981.
Path 51 | total_timesteps 1001.
Path 52 | total_timesteps 1015.
Path 53 | total_timesteps 1026.
Path 54 | total_timesteps 1038.
Path 55 | total_timesteps 1056.
Path 56 | total_timesteps 1074.
Path 57 | total_timesteps 1091.
Path 58 | total_timesteps 1106.
Path 59 | total_timesteps 1144.
Path 60 | total_timesteps 1154.
Path 61 | total_timesteps 1174.
Path 62 | total_timesteps 1188.
Path 63 | total_timesteps 1197.
Path 64 | total_timesteps 1208.
Path 65 | total_timesteps 1237.
Path 66 | total_timesteps 1252.
Path 67 | total_timesteps 1263.
Path 68 | total_timesteps 1286.
Path 69 | total_timesteps 1302.
Path 70 | total_timesteps 1318.
Path 71 | total_timesteps 1333.
Path 72 | total_timesteps 1352.
Path 73 | total_timesteps 1389.
Path 74 | total_timesteps 1401.
Path 75 | total_timesteps 1429.
Path 76 | total_timesteps 1450.
Path 77 | total_timesteps 1462.
Path 78 | total_timesteps 1472.
Path 79 | total_timesteps 1483.
Path 80 | total_timesteps 1498.
Path 81 | total_timesteps 1513.
Path 82 | total_timesteps 1535.
Path 83 | total_timesteps 1546.
Path 84 | total_timesteps 1569.
Path 85 | total_timesteps 1608.
Path 86 | total_timesteps 1624.
Path 87 | total_timesteps 1643.
Path 88 | total_timesteps 1671.
Path 89 | total_timesteps 1693.
Path 90 | total_timesteps 1712.
Path 91 | total_timesteps 1729.
Path 92 | total_timesteps 1758.
Path 93 | total_timesteps 1774.
Path 94 | total_timesteps 1792.
Path 95 | total_timesteps 1803.
Path 96 | total_timesteps 1819.
Path 97 | total_timesteps 1829.
Path 98 | total_timesteps 1837.
Path 99 | total_timesteps 1850.
Path 100 | total_timesteps 1867.
Path 101 | total_timesteps 1880.
Path 102 | total_timesteps 1910.
Path 103 | total_timesteps 1942.
Path 104 | total_timesteps 1968.
Path 105 | total_timesteps 1979.
Path 106 | total_timesteps 2012.
Path 107 | total_timesteps 2027.
Path 108 | total_timesteps 2046.
Path 109 | total_timesteps 2067.
Path 110 | total_timesteps 2090.
Path 111 | total_timesteps 2106.
Path 112 | total_timesteps 2116.
Path 113 | total_timesteps 2136.
Path 114 | total_timesteps 2202.
Path 115 | total_timesteps 2212.
Path 116 | total_timesteps 2232.
Path 117 | total_timesteps 2265.
Path 118 | total_timesteps 2277.
Path 119 | total_timesteps 2293.
Path 120 | total_timesteps 2309.
Path 121 | total_timesteps 2331.
Path 122 | total_timesteps 2349.
Path 123 | total_timesteps 2368.
Path 124 | total_timesteps 2392.
Path 125 | total_timesteps 2444.
Path 126 | total_timesteps 2459.
Path 127 | total_timesteps 2492.
Path 128 | total_timesteps 2508.
Path 129 | total_timesteps 2520.
Path 130 | total_timesteps 2540.
Path 131 | total_timesteps 2563.
Path 132 | total_timesteps 2576.
Path 133 | total_timesteps 2594.
Path 134 | total_timesteps 2607.
Path 135 | total_timesteps 2615.
Path 136 | total_timesteps 2631.
Path 137 | total_timesteps 2652.
Path 138 | total_timesteps 2669.
Path 139 | total_timesteps 2681.
Path 140 | total_timesteps 2705.
Path 141 | total_timesteps 2716.
Path 142 | total_timesteps 2728.
Path 143 | total_timesteps 2766.
Path 144 | total_timesteps 2781.
Path 145 | total_timesteps 2795.
Path 146 | total_timesteps 2817.
Path 147 | total_timesteps 2831.
Path 148 | total_timesteps 2851.
Path 149 | total_timesteps 2877.
Path 150 | total_timesteps 2891.
Path 151 | total_timesteps 2918.
Path 152 | total_timesteps 2956.
Path 153 | total_timesteps 2976.
Path 154 | total_timesteps 3015.
Path 155 | total_timesteps 3027.
Path 156 | total_timesteps 3052.
Path 157 | total_timesteps 3070.
Path 158 | total_timesteps 3104.
Path 159 | total_timesteps 3121.
Path 160 | total_timesteps 3130.
Path 161 | total_timesteps 3143.
Path 162 | total_timesteps 3163.
Path 163 | total_timesteps 3188.
Path 164 | total_timesteps 3204.
Path 165 | total_timesteps 3225.
Path 166 | total_timesteps 3258.
Path 167 | total_timesteps 3299.
Path 168 | total_timesteps 3310.
Path 169 | total_timesteps 3324.
Path 170 | total_timesteps 3346.
Path 171 | total_timesteps 3356.
Path 172 | total_timesteps 3365.
Path 173 | total_timesteps 3396.
Path 174 | total_timesteps 3407.
Path 175 | total_timesteps 3420.
Path 176 | total_timesteps 3439.
Path 177 | total_timesteps 3451.
Path 178 | total_timesteps 3462.
Path 179 | total_timesteps 3488.
Path 180 | total_timesteps 3499.
Path 181 | total_timesteps 3523.
Path 182 | total_timesteps 3541.
Path 183 | total_timesteps 3566.
Path 184 | total_timesteps 3577.
Path 185 | total_timesteps 3590.
Path 186 | total_timesteps 3606.
Path 187 | total_timesteps 3622.
Path 188 | total_timesteps 3637.
Path 189 | total_timesteps 3655.
Path 190 | total_timesteps 3676.
Path 191 | total_timesteps 3688.
Path 192 | total_timesteps 3714.
Path 193 | total_timesteps 3726.
Path 194 | total_timesteps 3753.
Path 195 | total_timesteps 3796.
Path 196 | total_timesteps 3817.
Path 197 | total_timesteps 3831.
Path 198 | total_timesteps 3875.
Path 199 | total_timesteps 3895.
Path 200 | total_timesteps 3907.
Path 201 | total_timesteps 3916.
Path 202 | total_timesteps 3940.
Path 203 | total_timesteps 3961.
Path 204 | total_timesteps 3980.
Path 205 | total_timesteps 3993.
Path 206 | total_timesteps 4005.
Path 207 | total_timesteps 4023.
Path 208 | total_timesteps 4036.
Path 209 | total_timesteps 4050.
Path 210 | total_timesteps 4059.
Path 211 | total_timesteps 4069.
Path 212 | total_timesteps 4091.
Path 213 | total_timesteps 4123.
Path 214 | total_timesteps 4141.
Path 215 | total_timesteps 4171.
Path 216 | total_timesteps 4188.
Path 217 | total_timesteps 4206.
Path 218 | total_timesteps 4220.
Path 219 | total_timesteps 4238.
Path 220 | total_timesteps 4255.
Path 221 | total_timesteps 4280.
Path 222 | total_timesteps 4304.
Path 223 | total_timesteps 4317.
Path 224 | total_timesteps 4334.
Path 225 | total_timesteps 4349.
Path 226 | total_timesteps 4375.
Path 227 | total_timesteps 4390.
Path 228 | total_timesteps 4415.
Path 229 | total_timesteps 4443.
Path 230 | total_timesteps 4455.
Path 231 | total_timesteps 4478.
Path 232 | total_timesteps 4493.
Path 233 | total_timesteps 4505.
Path 234 | total_timesteps 4528.
Path 235 | total_timesteps 4554.
Path 236 | total_timesteps 4593.
Path 237 | total_timesteps 4618.
Path 238 | total_timesteps 4633.
Path 239 | total_timesteps 4655.
Path 240 | total_timesteps 4673.
Path 241 | total_timesteps 4696.
Path 242 | total_timesteps 4728.
Path 243 | total_timesteps 4748.
Path 244 | total_timesteps 4761.
Path 245 | total_timesteps 4790.
Path 246 | total_timesteps 4807.
Path 247 | total_timesteps 4815.
Path 248 | total_timesteps 4828.
Path 249 | total_timesteps 4839.
Path 250 | total_timesteps 4867.
Path 251 | total_timesteps 4889.
Path 252 | total_timesteps 4904.
Path 253 | total_timesteps 4918.
Path 254 | total_timesteps 4931.
Path 255 | total_timesteps 4941.
Path 256 | total_timesteps 4958.
Path 257 | total_timesteps 4995.
Path 258 | total_timesteps 5014.
Path 259 | total_timesteps 5027.
Path 260 | total_timesteps 5044.
Path 261 | total_timesteps 5052.
Path 262 | total_timesteps 5069.
Path 263 | total_timesteps 5092.
Path 264 | total_timesteps 5104.
Path 265 | total_timesteps 5118.
Path 266 | total_timesteps 5137.
Path 267 | total_timesteps 5147.
Path 268 | total_timesteps 5164.
Path 269 | total_timesteps 5178.
Path 270 | total_timesteps 5192.
Path 271 | total_timesteps 5207.
Path 272 | total_timesteps 5220.
Path 273 | total_timesteps 5235.
Path 274 | total_timesteps 5248.
Path 275 | total_timesteps 5263.
Path 276 | total_timesteps 5301.
Path 277 | total_timesteps 5323.
Path 278 | total_timesteps 5342.
Path 279 | total_timesteps 5360.
Path 280 | total_timesteps 5375.
Path 281 | total_timesteps 5407.
Path 282 | total_timesteps 5429.
Path 283 | total_timesteps 5446.
Path 284 | total_timesteps 5471.
Path 285 | total_timesteps 5485.
Path 286 | total_timesteps 5500.
Path 287 | total_timesteps 5517.
Path 288 | total_timesteps 5538.
Path 289 | total_timesteps 5557.
Path 290 | total_timesteps 5592.
Path 291 | total_timesteps 5616.
Path 292 | total_timesteps 5630.
Path 293 | total_timesteps 5643.
Path 294 | total_timesteps 5661.
Path 295 | total_timesteps 5698.
Path 296 | total_timesteps 5714.
Path 297 | total_timesteps 5732.
Path 298 | total_timesteps 5746.
Path 299 | total_timesteps 5764.
Path 300 | total_timesteps 5780.
Path 301 | total_timesteps 5798.
Path 302 | total_timesteps 5809.
Path 303 | total_timesteps 5840.
Path 304 | total_timesteps 5861.
Path 305 | total_timesteps 5885.
Path 306 | total_timesteps 5898.
Path 307 | total_timesteps 5911.
Path 308 | total_timesteps 5929.
Path 309 | total_timesteps 5955.
Path 310 | total_timesteps 5976.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.63    |
| Iteration     | 1        |
| MaximumReturn | 12.6     |
| MinimumReturn | -23.9    |
| TotalSamples  | 12010    |
----------------------------
itr #2 | 
Fitting dynamics.
Validation loss = 0.04544718936085701
Validation loss = 0.030898744240403175
Validation loss = 0.03175440803170204
Validation loss = 0.03184017166495323
Validation loss = 0.029883520677685738
Validation loss = 0.035025134682655334
Validation loss = 0.03444700315594673
Validation loss = 0.02852550707757473
Validation loss = 0.027901597321033478
Validation loss = 0.028085097670555115
Validation loss = 0.026437530294060707
Validation loss = 0.02576136775314808
Validation loss = 0.02579331211745739
Validation loss = 0.03390352055430412
Validation loss = 0.02670862525701523
Validation loss = 0.029933461919426918
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 56.
Path 2 | total_timesteps 77.
Path 3 | total_timesteps 123.
Path 4 | total_timesteps 139.
Path 5 | total_timesteps 151.
Path 6 | total_timesteps 170.
Path 7 | total_timesteps 190.
Path 8 | total_timesteps 219.
Path 9 | total_timesteps 240.
Path 10 | total_timesteps 263.
Path 11 | total_timesteps 278.
Path 12 | total_timesteps 303.
Path 13 | total_timesteps 349.
Path 14 | total_timesteps 362.
Path 15 | total_timesteps 385.
Path 16 | total_timesteps 412.
Path 17 | total_timesteps 428.
Path 18 | total_timesteps 449.
Path 19 | total_timesteps 458.
Path 20 | total_timesteps 472.
Path 21 | total_timesteps 505.
Path 22 | total_timesteps 519.
Path 23 | total_timesteps 539.
Path 24 | total_timesteps 566.
Path 25 | total_timesteps 581.
Path 26 | total_timesteps 595.
Path 27 | total_timesteps 607.
Path 28 | total_timesteps 629.
Path 29 | total_timesteps 652.
Path 30 | total_timesteps 671.
Path 31 | total_timesteps 689.
Path 32 | total_timesteps 704.
Path 33 | total_timesteps 744.
Path 34 | total_timesteps 766.
Path 35 | total_timesteps 785.
Path 36 | total_timesteps 795.
Path 37 | total_timesteps 806.
Path 38 | total_timesteps 817.
Path 39 | total_timesteps 834.
Path 40 | total_timesteps 847.
Path 41 | total_timesteps 876.
Path 42 | total_timesteps 893.
Path 43 | total_timesteps 903.
Path 44 | total_timesteps 932.
Path 45 | total_timesteps 951.
Path 46 | total_timesteps 988.
Path 47 | total_timesteps 1002.
Path 48 | total_timesteps 1023.
Path 49 | total_timesteps 1048.
Path 50 | total_timesteps 1070.
Path 51 | total_timesteps 1081.
Path 52 | total_timesteps 1105.
Path 53 | total_timesteps 1115.
Path 54 | total_timesteps 1127.
Path 55 | total_timesteps 1146.
Path 56 | total_timesteps 1179.
Path 57 | total_timesteps 1197.
Path 58 | total_timesteps 1216.
Path 59 | total_timesteps 1229.
Path 60 | total_timesteps 1245.
Path 61 | total_timesteps 1265.
Path 62 | total_timesteps 1279.
Path 63 | total_timesteps 1293.
Path 64 | total_timesteps 1332.
Path 65 | total_timesteps 1379.
Path 66 | total_timesteps 1406.
Path 67 | total_timesteps 1434.
Path 68 | total_timesteps 1449.
Path 69 | total_timesteps 1463.
Path 70 | total_timesteps 1484.
Path 71 | total_timesteps 1513.
Path 72 | total_timesteps 1539.
Path 73 | total_timesteps 1562.
Path 74 | total_timesteps 1581.
Path 75 | total_timesteps 1598.
Path 76 | total_timesteps 1620.
Path 77 | total_timesteps 1639.
Path 78 | total_timesteps 1654.
Path 79 | total_timesteps 1674.
Path 80 | total_timesteps 1696.
Path 81 | total_timesteps 1712.
Path 82 | total_timesteps 1733.
Path 83 | total_timesteps 1747.
Path 84 | total_timesteps 1764.
Path 85 | total_timesteps 1776.
Path 86 | total_timesteps 1795.
Path 87 | total_timesteps 1813.
Path 88 | total_timesteps 1823.
Path 89 | total_timesteps 1834.
Path 90 | total_timesteps 1846.
Path 91 | total_timesteps 1863.
Path 92 | total_timesteps 1881.
Path 93 | total_timesteps 1891.
Path 94 | total_timesteps 1908.
Path 95 | total_timesteps 1941.
Path 96 | total_timesteps 1954.
Path 97 | total_timesteps 1969.
Path 98 | total_timesteps 1980.
Path 99 | total_timesteps 1999.
Path 100 | total_timesteps 2022.
Path 101 | total_timesteps 2048.
Path 102 | total_timesteps 2071.
Path 103 | total_timesteps 2100.
Path 104 | total_timesteps 2115.
Path 105 | total_timesteps 2132.
Path 106 | total_timesteps 2151.
Path 107 | total_timesteps 2203.
Path 108 | total_timesteps 2245.
Path 109 | total_timesteps 2262.
Path 110 | total_timesteps 2288.
Path 111 | total_timesteps 2307.
Path 112 | total_timesteps 2318.
Path 113 | total_timesteps 2330.
Path 114 | total_timesteps 2354.
Path 115 | total_timesteps 2381.
Path 116 | total_timesteps 2397.
Path 117 | total_timesteps 2420.
Path 118 | total_timesteps 2456.
Path 119 | total_timesteps 2469.
Path 120 | total_timesteps 2482.
Path 121 | total_timesteps 2499.
Path 122 | total_timesteps 2511.
Path 123 | total_timesteps 2525.
Path 124 | total_timesteps 2547.
Path 125 | total_timesteps 2564.
Path 126 | total_timesteps 2580.
Path 127 | total_timesteps 2592.
Path 128 | total_timesteps 2608.
Path 129 | total_timesteps 2623.
Path 130 | total_timesteps 2634.
Path 131 | total_timesteps 2645.
Path 132 | total_timesteps 2674.
Path 133 | total_timesteps 2727.
Path 134 | total_timesteps 2750.
Path 135 | total_timesteps 2761.
Path 136 | total_timesteps 2774.
Path 137 | total_timesteps 2789.
Path 138 | total_timesteps 2803.
Path 139 | total_timesteps 2826.
Path 140 | total_timesteps 2838.
Path 141 | total_timesteps 2874.
Path 142 | total_timesteps 2902.
Path 143 | total_timesteps 2914.
Path 144 | total_timesteps 2934.
Path 145 | total_timesteps 2951.
Path 146 | total_timesteps 2977.
Path 147 | total_timesteps 2987.
Path 148 | total_timesteps 3008.
Path 149 | total_timesteps 3028.
Path 150 | total_timesteps 3049.
Path 151 | total_timesteps 3068.
Path 152 | total_timesteps 3086.
Path 153 | total_timesteps 3106.
Path 154 | total_timesteps 3138.
Path 155 | total_timesteps 3160.
Path 156 | total_timesteps 3175.
Path 157 | total_timesteps 3198.
Path 158 | total_timesteps 3241.
Path 159 | total_timesteps 3261.
Path 160 | total_timesteps 3296.
Path 161 | total_timesteps 3313.
Path 162 | total_timesteps 3324.
Path 163 | total_timesteps 3338.
Path 164 | total_timesteps 3360.
Path 165 | total_timesteps 3393.
Path 166 | total_timesteps 3415.
Path 167 | total_timesteps 3424.
Path 168 | total_timesteps 3433.
Path 169 | total_timesteps 3449.
Path 170 | total_timesteps 3464.
Path 171 | total_timesteps 3485.
Path 172 | total_timesteps 3508.
Path 173 | total_timesteps 3536.
Path 174 | total_timesteps 3582.
Path 175 | total_timesteps 3598.
Path 176 | total_timesteps 3619.
Path 177 | total_timesteps 3642.
Path 178 | total_timesteps 3651.
Path 179 | total_timesteps 3672.
Path 180 | total_timesteps 3696.
Path 181 | total_timesteps 3711.
Path 182 | total_timesteps 3748.
Path 183 | total_timesteps 3764.
Path 184 | total_timesteps 3779.
Path 185 | total_timesteps 3792.
Path 186 | total_timesteps 3821.
Path 187 | total_timesteps 3835.
Path 188 | total_timesteps 3877.
Path 189 | total_timesteps 3908.
Path 190 | total_timesteps 3923.
Path 191 | total_timesteps 3946.
Path 192 | total_timesteps 3962.
Path 193 | total_timesteps 3979.
Path 194 | total_timesteps 4000.
Path 195 | total_timesteps 4021.
Path 196 | total_timesteps 4052.
Path 197 | total_timesteps 4066.
Path 198 | total_timesteps 4081.
Path 199 | total_timesteps 4097.
Path 200 | total_timesteps 4150.
Path 201 | total_timesteps 4167.
Path 202 | total_timesteps 4191.
Path 203 | total_timesteps 4207.
Path 204 | total_timesteps 4224.
Path 205 | total_timesteps 4254.
Path 206 | total_timesteps 4265.
Path 207 | total_timesteps 4291.
Path 208 | total_timesteps 4321.
Path 209 | total_timesteps 4350.
Path 210 | total_timesteps 4377.
Path 211 | total_timesteps 4388.
Path 212 | total_timesteps 4403.
Path 213 | total_timesteps 4417.
Path 214 | total_timesteps 4451.
Path 215 | total_timesteps 4466.
Path 216 | total_timesteps 4493.
Path 217 | total_timesteps 4513.
Path 218 | total_timesteps 4538.
Path 219 | total_timesteps 4575.
Path 220 | total_timesteps 4591.
Path 221 | total_timesteps 4610.
Path 222 | total_timesteps 4620.
Path 223 | total_timesteps 4639.
Path 224 | total_timesteps 4671.
Path 225 | total_timesteps 4704.
Path 226 | total_timesteps 4719.
Path 227 | total_timesteps 4734.
Path 228 | total_timesteps 4748.
Path 229 | total_timesteps 4762.
Path 230 | total_timesteps 4783.
Path 231 | total_timesteps 4796.
Path 232 | total_timesteps 4825.
Path 233 | total_timesteps 4841.
Path 234 | total_timesteps 4855.
Path 235 | total_timesteps 4872.
Path 236 | total_timesteps 4883.
Path 237 | total_timesteps 4917.
Path 238 | total_timesteps 4938.
Path 239 | total_timesteps 4965.
Path 240 | total_timesteps 4988.
Path 241 | total_timesteps 4999.
Path 242 | total_timesteps 5021.
Path 243 | total_timesteps 5060.
Path 244 | total_timesteps 5087.
Path 245 | total_timesteps 5112.
Path 246 | total_timesteps 5130.
Path 247 | total_timesteps 5147.
Path 248 | total_timesteps 5169.
Path 249 | total_timesteps 5181.
Path 250 | total_timesteps 5200.
Path 251 | total_timesteps 5215.
Path 252 | total_timesteps 5232.
Path 253 | total_timesteps 5261.
Path 254 | total_timesteps 5299.
Path 255 | total_timesteps 5311.
Path 256 | total_timesteps 5338.
Path 257 | total_timesteps 5354.
Path 258 | total_timesteps 5383.
Path 259 | total_timesteps 5403.
Path 260 | total_timesteps 5417.
Path 261 | total_timesteps 5446.
Path 262 | total_timesteps 5471.
Path 263 | total_timesteps 5496.
Path 264 | total_timesteps 5504.
Path 265 | total_timesteps 5525.
Path 266 | total_timesteps 5572.
Path 267 | total_timesteps 5582.
Path 268 | total_timesteps 5608.
Path 269 | total_timesteps 5619.
Path 270 | total_timesteps 5634.
Path 271 | total_timesteps 5644.
Path 272 | total_timesteps 5660.
Path 273 | total_timesteps 5693.
Path 274 | total_timesteps 5703.
Path 275 | total_timesteps 5724.
Path 276 | total_timesteps 5737.
Path 277 | total_timesteps 5770.
Path 278 | total_timesteps 5784.
Path 279 | total_timesteps 5804.
Path 280 | total_timesteps 5822.
Path 281 | total_timesteps 5848.
Path 282 | total_timesteps 5888.
Path 283 | total_timesteps 5961.
Path 284 | total_timesteps 5984.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.67    |
| Iteration     | 2        |
| MaximumReturn | 10.7     |
| MinimumReturn | -26.1    |
| TotalSamples  | 16014    |
----------------------------
itr #3 | 
Fitting dynamics.
Validation loss = 0.03164652734994888
Validation loss = 0.025972038507461548
Validation loss = 0.02458031475543976
Validation loss = 0.024181421846151352
Validation loss = 0.024278871715068817
Validation loss = 0.02523931860923767
Validation loss = 0.02754475176334381
Validation loss = 0.022598806768655777
Validation loss = 0.02338486909866333
Validation loss = 0.022844523191452026
Validation loss = 0.02333856374025345
Validation loss = 0.02444618195295334
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 44.
Path 2 | total_timesteps 62.
Path 3 | total_timesteps 84.
Path 4 | total_timesteps 99.
Path 5 | total_timesteps 110.
Path 6 | total_timesteps 132.
Path 7 | total_timesteps 143.
Path 8 | total_timesteps 170.
Path 9 | total_timesteps 205.
Path 10 | total_timesteps 243.
Path 11 | total_timesteps 256.
Path 12 | total_timesteps 282.
Path 13 | total_timesteps 309.
Path 14 | total_timesteps 327.
Path 15 | total_timesteps 362.
Path 16 | total_timesteps 376.
Path 17 | total_timesteps 391.
Path 18 | total_timesteps 417.
Path 19 | total_timesteps 439.
Path 20 | total_timesteps 450.
Path 21 | total_timesteps 464.
Path 22 | total_timesteps 477.
Path 23 | total_timesteps 492.
Path 24 | total_timesteps 537.
Path 25 | total_timesteps 561.
Path 26 | total_timesteps 579.
Path 27 | total_timesteps 599.
Path 28 | total_timesteps 633.
Path 29 | total_timesteps 650.
Path 30 | total_timesteps 677.
Path 31 | total_timesteps 708.
Path 32 | total_timesteps 718.
Path 33 | total_timesteps 751.
Path 34 | total_timesteps 766.
Path 35 | total_timesteps 791.
Path 36 | total_timesteps 807.
Path 37 | total_timesteps 844.
Path 38 | total_timesteps 860.
Path 39 | total_timesteps 872.
Path 40 | total_timesteps 922.
Path 41 | total_timesteps 949.
Path 42 | total_timesteps 961.
Path 43 | total_timesteps 972.
Path 44 | total_timesteps 993.
Path 45 | total_timesteps 1010.
Path 46 | total_timesteps 1029.
Path 47 | total_timesteps 1037.
Path 48 | total_timesteps 1066.
Path 49 | total_timesteps 1089.
Path 50 | total_timesteps 1107.
Path 51 | total_timesteps 1127.
Path 52 | total_timesteps 1141.
Path 53 | total_timesteps 1168.
Path 54 | total_timesteps 1188.
Path 55 | total_timesteps 1218.
Path 56 | total_timesteps 1239.
Path 57 | total_timesteps 1253.
Path 58 | total_timesteps 1279.
Path 59 | total_timesteps 1311.
Path 60 | total_timesteps 1324.
Path 61 | total_timesteps 1333.
Path 62 | total_timesteps 1347.
Path 63 | total_timesteps 1362.
Path 64 | total_timesteps 1388.
Path 65 | total_timesteps 1415.
Path 66 | total_timesteps 1432.
Path 67 | total_timesteps 1453.
Path 68 | total_timesteps 1467.
Path 69 | total_timesteps 1485.
Path 70 | total_timesteps 1496.
Path 71 | total_timesteps 1518.
Path 72 | total_timesteps 1539.
Path 73 | total_timesteps 1552.
Path 74 | total_timesteps 1569.
Path 75 | total_timesteps 1595.
Path 76 | total_timesteps 1609.
Path 77 | total_timesteps 1621.
Path 78 | total_timesteps 1634.
Path 79 | total_timesteps 1664.
Path 80 | total_timesteps 1686.
Path 81 | total_timesteps 1704.
Path 82 | total_timesteps 1723.
Path 83 | total_timesteps 1744.
Path 84 | total_timesteps 1756.
Path 85 | total_timesteps 1774.
Path 86 | total_timesteps 1791.
Path 87 | total_timesteps 1805.
Path 88 | total_timesteps 1816.
Path 89 | total_timesteps 1848.
Path 90 | total_timesteps 1870.
Path 91 | total_timesteps 1881.
Path 92 | total_timesteps 1899.
Path 93 | total_timesteps 1919.
Path 94 | total_timesteps 1932.
Path 95 | total_timesteps 1958.
Path 96 | total_timesteps 1968.
Path 97 | total_timesteps 1985.
Path 98 | total_timesteps 1998.
Path 99 | total_timesteps 2017.
Path 100 | total_timesteps 2040.
Path 101 | total_timesteps 2054.
Path 102 | total_timesteps 2088.
Path 103 | total_timesteps 2111.
Path 104 | total_timesteps 2142.
Path 105 | total_timesteps 2159.
Path 106 | total_timesteps 2169.
Path 107 | total_timesteps 2189.
Path 108 | total_timesteps 2212.
Path 109 | total_timesteps 2226.
Path 110 | total_timesteps 2238.
Path 111 | total_timesteps 2261.
Path 112 | total_timesteps 2288.
Path 113 | total_timesteps 2307.
Path 114 | total_timesteps 2327.
Path 115 | total_timesteps 2349.
Path 116 | total_timesteps 2366.
Path 117 | total_timesteps 2379.
Path 118 | total_timesteps 2396.
Path 119 | total_timesteps 2423.
Path 120 | total_timesteps 2445.
Path 121 | total_timesteps 2479.
Path 122 | total_timesteps 2498.
Path 123 | total_timesteps 2529.
Path 124 | total_timesteps 2559.
Path 125 | total_timesteps 2572.
Path 126 | total_timesteps 2589.
Path 127 | total_timesteps 2621.
Path 128 | total_timesteps 2636.
Path 129 | total_timesteps 2653.
Path 130 | total_timesteps 2675.
Path 131 | total_timesteps 2696.
Path 132 | total_timesteps 2711.
Path 133 | total_timesteps 2728.
Path 134 | total_timesteps 2760.
Path 135 | total_timesteps 2771.
Path 136 | total_timesteps 2792.
Path 137 | total_timesteps 2805.
Path 138 | total_timesteps 2815.
Path 139 | total_timesteps 2835.
Path 140 | total_timesteps 2846.
Path 141 | total_timesteps 2878.
Path 142 | total_timesteps 2888.
Path 143 | total_timesteps 2899.
Path 144 | total_timesteps 2914.
Path 145 | total_timesteps 2943.
Path 146 | total_timesteps 2955.
Path 147 | total_timesteps 2978.
Path 148 | total_timesteps 2999.
Path 149 | total_timesteps 3010.
Path 150 | total_timesteps 3039.
Path 151 | total_timesteps 3055.
Path 152 | total_timesteps 3071.
Path 153 | total_timesteps 3092.
Path 154 | total_timesteps 3110.
Path 155 | total_timesteps 3141.
Path 156 | total_timesteps 3171.
Path 157 | total_timesteps 3192.
Path 158 | total_timesteps 3223.
Path 159 | total_timesteps 3236.
Path 160 | total_timesteps 3258.
Path 161 | total_timesteps 3300.
Path 162 | total_timesteps 3346.
Path 163 | total_timesteps 3367.
Path 164 | total_timesteps 3383.
Path 165 | total_timesteps 3399.
Path 166 | total_timesteps 3421.
Path 167 | total_timesteps 3439.
Path 168 | total_timesteps 3465.
Path 169 | total_timesteps 3497.
Path 170 | total_timesteps 3508.
Path 171 | total_timesteps 3527.
Path 172 | total_timesteps 3536.
Path 173 | total_timesteps 3547.
Path 174 | total_timesteps 3574.
Path 175 | total_timesteps 3581.
Path 176 | total_timesteps 3596.
Path 177 | total_timesteps 3606.
Path 178 | total_timesteps 3631.
Path 179 | total_timesteps 3668.
Path 180 | total_timesteps 3688.
Path 181 | total_timesteps 3709.
Path 182 | total_timesteps 3720.
Path 183 | total_timesteps 3738.
Path 184 | total_timesteps 3752.
Path 185 | total_timesteps 3773.
Path 186 | total_timesteps 3787.
Path 187 | total_timesteps 3813.
Path 188 | total_timesteps 3843.
Path 189 | total_timesteps 3859.
Path 190 | total_timesteps 3872.
Path 191 | total_timesteps 3890.
Path 192 | total_timesteps 3899.
Path 193 | total_timesteps 3920.
Path 194 | total_timesteps 3942.
Path 195 | total_timesteps 3956.
Path 196 | total_timesteps 3989.
Path 197 | total_timesteps 4009.
Path 198 | total_timesteps 4030.
Path 199 | total_timesteps 4048.
Path 200 | total_timesteps 4080.
Path 201 | total_timesteps 4097.
Path 202 | total_timesteps 4114.
Path 203 | total_timesteps 4141.
Path 204 | total_timesteps 4154.
Path 205 | total_timesteps 4169.
Path 206 | total_timesteps 4181.
Path 207 | total_timesteps 4204.
Path 208 | total_timesteps 4220.
Path 209 | total_timesteps 4250.
Path 210 | total_timesteps 4269.
Path 211 | total_timesteps 4320.
Path 212 | total_timesteps 4333.
Path 213 | total_timesteps 4354.
Path 214 | total_timesteps 4372.
Path 215 | total_timesteps 4382.
Path 216 | total_timesteps 4400.
Path 217 | total_timesteps 4423.
Path 218 | total_timesteps 4459.
Path 219 | total_timesteps 4475.
Path 220 | total_timesteps 4495.
Path 221 | total_timesteps 4514.
Path 222 | total_timesteps 4534.
Path 223 | total_timesteps 4560.
Path 224 | total_timesteps 4579.
Path 225 | total_timesteps 4594.
Path 226 | total_timesteps 4633.
Path 227 | total_timesteps 4653.
Path 228 | total_timesteps 4688.
Path 229 | total_timesteps 4737.
Path 230 | total_timesteps 4752.
Path 231 | total_timesteps 4767.
Path 232 | total_timesteps 4790.
Path 233 | total_timesteps 4808.
Path 234 | total_timesteps 4828.
Path 235 | total_timesteps 4842.
Path 236 | total_timesteps 4863.
Path 237 | total_timesteps 4908.
Path 238 | total_timesteps 4923.
Path 239 | total_timesteps 4936.
Path 240 | total_timesteps 4951.
Path 241 | total_timesteps 4975.
Path 242 | total_timesteps 4988.
Path 243 | total_timesteps 5012.
Path 244 | total_timesteps 5029.
Path 245 | total_timesteps 5042.
Path 246 | total_timesteps 5061.
Path 247 | total_timesteps 5081.
Path 248 | total_timesteps 5097.
Path 249 | total_timesteps 5116.
Path 250 | total_timesteps 5126.
Path 251 | total_timesteps 5155.
Path 252 | total_timesteps 5177.
Path 253 | total_timesteps 5191.
Path 254 | total_timesteps 5203.
Path 255 | total_timesteps 5219.
Path 256 | total_timesteps 5233.
Path 257 | total_timesteps 5245.
Path 258 | total_timesteps 5270.
Path 259 | total_timesteps 5286.
Path 260 | total_timesteps 5306.
Path 261 | total_timesteps 5326.
Path 262 | total_timesteps 5342.
Path 263 | total_timesteps 5364.
Path 264 | total_timesteps 5407.
Path 265 | total_timesteps 5429.
Path 266 | total_timesteps 5444.
Path 267 | total_timesteps 5479.
Path 268 | total_timesteps 5494.
Path 269 | total_timesteps 5509.
Path 270 | total_timesteps 5524.
Path 271 | total_timesteps 5543.
Path 272 | total_timesteps 5570.
Path 273 | total_timesteps 5587.
Path 274 | total_timesteps 5607.
Path 275 | total_timesteps 5635.
Path 276 | total_timesteps 5661.
Path 277 | total_timesteps 5693.
Path 278 | total_timesteps 5714.
Path 279 | total_timesteps 5728.
Path 280 | total_timesteps 5741.
Path 281 | total_timesteps 5758.
Path 282 | total_timesteps 5784.
Path 283 | total_timesteps 5804.
Path 284 | total_timesteps 5819.
Path 285 | total_timesteps 5856.
Path 286 | total_timesteps 5882.
Path 287 | total_timesteps 5898.
Path 288 | total_timesteps 5913.
Path 289 | total_timesteps 5935.
Path 290 | total_timesteps 5953.
Path 291 | total_timesteps 5973.
Path 292 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.55    |
| Iteration     | 3        |
| MaximumReturn | 7.36     |
| MinimumReturn | -24      |
| TotalSamples  | 20016    |
----------------------------
itr #4 | 
Fitting dynamics.
Validation loss = 0.029006097465753555
Validation loss = 0.020198237150907516
Validation loss = 0.020383307710289955
Validation loss = 0.021448291838169098
Validation loss = 0.018921643495559692
Validation loss = 0.021335789933800697
Validation loss = 0.01921863481402397
Validation loss = 0.021213453263044357
Validation loss = 0.018677575513720512
Validation loss = 0.02133404091000557
Validation loss = 0.019960355013608932
Validation loss = 0.018968287855386734
Validation loss = 0.019980601966381073
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 36.
Path 4 | total_timesteps 48.
Path 5 | total_timesteps 68.
Path 6 | total_timesteps 97.
Path 7 | total_timesteps 111.
Path 8 | total_timesteps 142.
Path 9 | total_timesteps 153.
Path 10 | total_timesteps 168.
Path 11 | total_timesteps 188.
Path 12 | total_timesteps 201.
Path 13 | total_timesteps 225.
Path 14 | total_timesteps 237.
Path 15 | total_timesteps 252.
Path 16 | total_timesteps 263.
Path 17 | total_timesteps 281.
Path 18 | total_timesteps 290.
Path 19 | total_timesteps 308.
Path 20 | total_timesteps 319.
Path 21 | total_timesteps 337.
Path 22 | total_timesteps 356.
Path 23 | total_timesteps 364.
Path 24 | total_timesteps 379.
Path 25 | total_timesteps 406.
Path 26 | total_timesteps 420.
Path 27 | total_timesteps 438.
Path 28 | total_timesteps 449.
Path 29 | total_timesteps 469.
Path 30 | total_timesteps 483.
Path 31 | total_timesteps 497.
Path 32 | total_timesteps 509.
Path 33 | total_timesteps 518.
Path 34 | total_timesteps 531.
Path 35 | total_timesteps 551.
Path 36 | total_timesteps 562.
Path 37 | total_timesteps 579.
Path 38 | total_timesteps 588.
Path 39 | total_timesteps 600.
Path 40 | total_timesteps 617.
Path 41 | total_timesteps 629.
Path 42 | total_timesteps 651.
Path 43 | total_timesteps 665.
Path 44 | total_timesteps 679.
Path 45 | total_timesteps 686.
Path 46 | total_timesteps 700.
Path 47 | total_timesteps 721.
Path 48 | total_timesteps 740.
Path 49 | total_timesteps 754.
Path 50 | total_timesteps 783.
Path 51 | total_timesteps 794.
Path 52 | total_timesteps 808.
Path 53 | total_timesteps 818.
Path 54 | total_timesteps 832.
Path 55 | total_timesteps 853.
Path 56 | total_timesteps 872.
Path 57 | total_timesteps 888.
Path 58 | total_timesteps 897.
Path 59 | total_timesteps 906.
Path 60 | total_timesteps 920.
Path 61 | total_timesteps 929.
Path 62 | total_timesteps 944.
Path 63 | total_timesteps 968.
Path 64 | total_timesteps 977.
Path 65 | total_timesteps 991.
Path 66 | total_timesteps 1016.
Path 67 | total_timesteps 1032.
Path 68 | total_timesteps 1044.
Path 69 | total_timesteps 1058.
Path 70 | total_timesteps 1066.
Path 71 | total_timesteps 1075.
Path 72 | total_timesteps 1096.
Path 73 | total_timesteps 1118.
Path 74 | total_timesteps 1132.
Path 75 | total_timesteps 1151.
Path 76 | total_timesteps 1163.
Path 77 | total_timesteps 1178.
Path 78 | total_timesteps 1188.
Path 79 | total_timesteps 1207.
Path 80 | total_timesteps 1229.
Path 81 | total_timesteps 1244.
Path 82 | total_timesteps 1263.
Path 83 | total_timesteps 1308.
Path 84 | total_timesteps 1321.
Path 85 | total_timesteps 1337.
Path 86 | total_timesteps 1360.
Path 87 | total_timesteps 1381.
Path 88 | total_timesteps 1390.
Path 89 | total_timesteps 1402.
Path 90 | total_timesteps 1414.
Path 91 | total_timesteps 1430.
Path 92 | total_timesteps 1442.
Path 93 | total_timesteps 1452.
Path 94 | total_timesteps 1462.
Path 95 | total_timesteps 1479.
Path 96 | total_timesteps 1492.
Path 97 | total_timesteps 1505.
Path 98 | total_timesteps 1515.
Path 99 | total_timesteps 1524.
Path 100 | total_timesteps 1539.
Path 101 | total_timesteps 1559.
Path 102 | total_timesteps 1573.
Path 103 | total_timesteps 1596.
Path 104 | total_timesteps 1615.
Path 105 | total_timesteps 1633.
Path 106 | total_timesteps 1653.
Path 107 | total_timesteps 1673.
Path 108 | total_timesteps 1687.
Path 109 | total_timesteps 1697.
Path 110 | total_timesteps 1715.
Path 111 | total_timesteps 1750.
Path 112 | total_timesteps 1765.
Path 113 | total_timesteps 1789.
Path 114 | total_timesteps 1805.
Path 115 | total_timesteps 1819.
Path 116 | total_timesteps 1842.
Path 117 | total_timesteps 1859.
Path 118 | total_timesteps 1871.
Path 119 | total_timesteps 1886.
Path 120 | total_timesteps 1907.
Path 121 | total_timesteps 1926.
Path 122 | total_timesteps 1959.
Path 123 | total_timesteps 1982.
Path 124 | total_timesteps 1995.
Path 125 | total_timesteps 2021.
Path 126 | total_timesteps 2033.
Path 127 | total_timesteps 2041.
Path 128 | total_timesteps 2068.
Path 129 | total_timesteps 2084.
Path 130 | total_timesteps 2098.
Path 131 | total_timesteps 2108.
Path 132 | total_timesteps 2137.
Path 133 | total_timesteps 2157.
Path 134 | total_timesteps 2176.
Path 135 | total_timesteps 2187.
Path 136 | total_timesteps 2216.
Path 137 | total_timesteps 2230.
Path 138 | total_timesteps 2244.
Path 139 | total_timesteps 2255.
Path 140 | total_timesteps 2289.
Path 141 | total_timesteps 2309.
Path 142 | total_timesteps 2317.
Path 143 | total_timesteps 2328.
Path 144 | total_timesteps 2338.
Path 145 | total_timesteps 2355.
Path 146 | total_timesteps 2367.
Path 147 | total_timesteps 2380.
Path 148 | total_timesteps 2403.
Path 149 | total_timesteps 2417.
Path 150 | total_timesteps 2428.
Path 151 | total_timesteps 2441.
Path 152 | total_timesteps 2456.
Path 153 | total_timesteps 2476.
Path 154 | total_timesteps 2489.
Path 155 | total_timesteps 2502.
Path 156 | total_timesteps 2511.
Path 157 | total_timesteps 2527.
Path 158 | total_timesteps 2550.
Path 159 | total_timesteps 2562.
Path 160 | total_timesteps 2571.
Path 161 | total_timesteps 2589.
Path 162 | total_timesteps 2605.
Path 163 | total_timesteps 2616.
Path 164 | total_timesteps 2629.
Path 165 | total_timesteps 2642.
Path 166 | total_timesteps 2658.
Path 167 | total_timesteps 2672.
Path 168 | total_timesteps 2685.
Path 169 | total_timesteps 2696.
Path 170 | total_timesteps 2711.
Path 171 | total_timesteps 2728.
Path 172 | total_timesteps 2741.
Path 173 | total_timesteps 2748.
Path 174 | total_timesteps 2759.
Path 175 | total_timesteps 2768.
Path 176 | total_timesteps 2780.
Path 177 | total_timesteps 2804.
Path 178 | total_timesteps 2815.
Path 179 | total_timesteps 2827.
Path 180 | total_timesteps 2836.
Path 181 | total_timesteps 2845.
Path 182 | total_timesteps 2859.
Path 183 | total_timesteps 2883.
Path 184 | total_timesteps 2898.
Path 185 | total_timesteps 2907.
Path 186 | total_timesteps 2921.
Path 187 | total_timesteps 2949.
Path 188 | total_timesteps 2962.
Path 189 | total_timesteps 2972.
Path 190 | total_timesteps 2992.
Path 191 | total_timesteps 3005.
Path 192 | total_timesteps 3025.
Path 193 | total_timesteps 3037.
Path 194 | total_timesteps 3045.
Path 195 | total_timesteps 3074.
Path 196 | total_timesteps 3091.
Path 197 | total_timesteps 3133.
Path 198 | total_timesteps 3153.
Path 199 | total_timesteps 3169.
Path 200 | total_timesteps 3197.
Path 201 | total_timesteps 3223.
Path 202 | total_timesteps 3236.
Path 203 | total_timesteps 3247.
Path 204 | total_timesteps 3256.
Path 205 | total_timesteps 3265.
Path 206 | total_timesteps 3281.
Path 207 | total_timesteps 3300.
Path 208 | total_timesteps 3317.
Path 209 | total_timesteps 3345.
Path 210 | total_timesteps 3363.
Path 211 | total_timesteps 3384.
Path 212 | total_timesteps 3416.
Path 213 | total_timesteps 3435.
Path 214 | total_timesteps 3471.
Path 215 | total_timesteps 3479.
Path 216 | total_timesteps 3492.
Path 217 | total_timesteps 3517.
Path 218 | total_timesteps 3535.
Path 219 | total_timesteps 3550.
Path 220 | total_timesteps 3569.
Path 221 | total_timesteps 3589.
Path 222 | total_timesteps 3600.
Path 223 | total_timesteps 3618.
Path 224 | total_timesteps 3633.
Path 225 | total_timesteps 3648.
Path 226 | total_timesteps 3666.
Path 227 | total_timesteps 3684.
Path 228 | total_timesteps 3701.
Path 229 | total_timesteps 3722.
Path 230 | total_timesteps 3739.
Path 231 | total_timesteps 3780.
Path 232 | total_timesteps 3788.
Path 233 | total_timesteps 3811.
Path 234 | total_timesteps 3825.
Path 235 | total_timesteps 3840.
Path 236 | total_timesteps 3864.
Path 237 | total_timesteps 3877.
Path 238 | total_timesteps 3890.
Path 239 | total_timesteps 3913.
Path 240 | total_timesteps 3928.
Path 241 | total_timesteps 3937.
Path 242 | total_timesteps 3951.
Path 243 | total_timesteps 3970.
Path 244 | total_timesteps 3989.
Path 245 | total_timesteps 4001.
Path 246 | total_timesteps 4021.
Path 247 | total_timesteps 4029.
Path 248 | total_timesteps 4058.
Path 249 | total_timesteps 4072.
Path 250 | total_timesteps 4091.
Path 251 | total_timesteps 4102.
Path 252 | total_timesteps 4117.
Path 253 | total_timesteps 4125.
Path 254 | total_timesteps 4151.
Path 255 | total_timesteps 4165.
Path 256 | total_timesteps 4181.
Path 257 | total_timesteps 4196.
Path 258 | total_timesteps 4221.
Path 259 | total_timesteps 4233.
Path 260 | total_timesteps 4244.
Path 261 | total_timesteps 4265.
Path 262 | total_timesteps 4279.
Path 263 | total_timesteps 4289.
Path 264 | total_timesteps 4303.
Path 265 | total_timesteps 4316.
Path 266 | total_timesteps 4329.
Path 267 | total_timesteps 4336.
Path 268 | total_timesteps 4356.
Path 269 | total_timesteps 4363.
Path 270 | total_timesteps 4375.
Path 271 | total_timesteps 4386.
Path 272 | total_timesteps 4397.
Path 273 | total_timesteps 4409.
Path 274 | total_timesteps 4422.
Path 275 | total_timesteps 4434.
Path 276 | total_timesteps 4448.
Path 277 | total_timesteps 4462.
Path 278 | total_timesteps 4473.
Path 279 | total_timesteps 4500.
Path 280 | total_timesteps 4520.
Path 281 | total_timesteps 4533.
Path 282 | total_timesteps 4549.
Path 283 | total_timesteps 4582.
Path 284 | total_timesteps 4593.
Path 285 | total_timesteps 4605.
Path 286 | total_timesteps 4615.
Path 287 | total_timesteps 4623.
Path 288 | total_timesteps 4634.
Path 289 | total_timesteps 4648.
Path 290 | total_timesteps 4666.
Path 291 | total_timesteps 4685.
Path 292 | total_timesteps 4706.
Path 293 | total_timesteps 4733.
Path 294 | total_timesteps 4743.
Path 295 | total_timesteps 4765.
Path 296 | total_timesteps 4783.
Path 297 | total_timesteps 4804.
Path 298 | total_timesteps 4829.
Path 299 | total_timesteps 4849.
Path 300 | total_timesteps 4862.
Path 301 | total_timesteps 4890.
Path 302 | total_timesteps 4931.
Path 303 | total_timesteps 4941.
Path 304 | total_timesteps 4950.
Path 305 | total_timesteps 4964.
Path 306 | total_timesteps 4976.
Path 307 | total_timesteps 4999.
Path 308 | total_timesteps 5017.
Path 309 | total_timesteps 5028.
Path 310 | total_timesteps 5045.
Path 311 | total_timesteps 5053.
Path 312 | total_timesteps 5061.
Path 313 | total_timesteps 5079.
Path 314 | total_timesteps 5095.
Path 315 | total_timesteps 5116.
Path 316 | total_timesteps 5132.
Path 317 | total_timesteps 5145.
Path 318 | total_timesteps 5160.
Path 319 | total_timesteps 5175.
Path 320 | total_timesteps 5187.
Path 321 | total_timesteps 5203.
Path 322 | total_timesteps 5218.
Path 323 | total_timesteps 5243.
Path 324 | total_timesteps 5263.
Path 325 | total_timesteps 5274.
Path 326 | total_timesteps 5299.
Path 327 | total_timesteps 5317.
Path 328 | total_timesteps 5331.
Path 329 | total_timesteps 5346.
Path 330 | total_timesteps 5368.
Path 331 | total_timesteps 5387.
Path 332 | total_timesteps 5412.
Path 333 | total_timesteps 5439.
Path 334 | total_timesteps 5454.
Path 335 | total_timesteps 5463.
Path 336 | total_timesteps 5476.
Path 337 | total_timesteps 5495.
Path 338 | total_timesteps 5522.
Path 339 | total_timesteps 5544.
Path 340 | total_timesteps 5557.
Path 341 | total_timesteps 5578.
Path 342 | total_timesteps 5589.
Path 343 | total_timesteps 5611.
Path 344 | total_timesteps 5622.
Path 345 | total_timesteps 5642.
Path 346 | total_timesteps 5664.
Path 347 | total_timesteps 5679.
Path 348 | total_timesteps 5702.
Path 349 | total_timesteps 5710.
Path 350 | total_timesteps 5731.
Path 351 | total_timesteps 5742.
Path 352 | total_timesteps 5751.
Path 353 | total_timesteps 5771.
Path 354 | total_timesteps 5796.
Path 355 | total_timesteps 5805.
Path 356 | total_timesteps 5817.
Path 357 | total_timesteps 5831.
Path 358 | total_timesteps 5841.
Path 359 | total_timesteps 5867.
Path 360 | total_timesteps 5886.
Path 361 | total_timesteps 5896.
Path 362 | total_timesteps 5905.
Path 363 | total_timesteps 5919.
Path 364 | total_timesteps 5941.
Path 365 | total_timesteps 5961.
Path 366 | total_timesteps 5973.
Path 367 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.99    |
| Iteration     | 4        |
| MaximumReturn | 2.81     |
| MinimumReturn | -23.7    |
| TotalSamples  | 24024    |
----------------------------
itr #5 | 
Fitting dynamics.
Validation loss = 0.017691051587462425
Validation loss = 0.016409816220402718
Validation loss = 0.016017211601138115
Validation loss = 0.01923520304262638
Validation loss = 0.017274441197514534
Validation loss = 0.018074212595820427
Validation loss = 0.017045466229319572
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 29.
Path 3 | total_timesteps 46.
Path 4 | total_timesteps 57.
Path 5 | total_timesteps 67.
Path 6 | total_timesteps 79.
Path 7 | total_timesteps 98.
Path 8 | total_timesteps 124.
Path 9 | total_timesteps 141.
Path 10 | total_timesteps 148.
Path 11 | total_timesteps 163.
Path 12 | total_timesteps 177.
Path 13 | total_timesteps 189.
Path 14 | total_timesteps 200.
Path 15 | total_timesteps 218.
Path 16 | total_timesteps 233.
Path 17 | total_timesteps 242.
Path 18 | total_timesteps 254.
Path 19 | total_timesteps 287.
Path 20 | total_timesteps 308.
Path 21 | total_timesteps 321.
Path 22 | total_timesteps 334.
Path 23 | total_timesteps 350.
Path 24 | total_timesteps 358.
Path 25 | total_timesteps 379.
Path 26 | total_timesteps 402.
Path 27 | total_timesteps 421.
Path 28 | total_timesteps 440.
Path 29 | total_timesteps 449.
Path 30 | total_timesteps 485.
Path 31 | total_timesteps 506.
Path 32 | total_timesteps 520.
Path 33 | total_timesteps 531.
Path 34 | total_timesteps 549.
Path 35 | total_timesteps 579.
Path 36 | total_timesteps 595.
Path 37 | total_timesteps 621.
Path 38 | total_timesteps 631.
Path 39 | total_timesteps 642.
Path 40 | total_timesteps 650.
Path 41 | total_timesteps 669.
Path 42 | total_timesteps 682.
Path 43 | total_timesteps 692.
Path 44 | total_timesteps 705.
Path 45 | total_timesteps 717.
Path 46 | total_timesteps 729.
Path 47 | total_timesteps 745.
Path 48 | total_timesteps 766.
Path 49 | total_timesteps 779.
Path 50 | total_timesteps 788.
Path 51 | total_timesteps 808.
Path 52 | total_timesteps 829.
Path 53 | total_timesteps 844.
Path 54 | total_timesteps 854.
Path 55 | total_timesteps 865.
Path 56 | total_timesteps 883.
Path 57 | total_timesteps 897.
Path 58 | total_timesteps 910.
Path 59 | total_timesteps 927.
Path 60 | total_timesteps 939.
Path 61 | total_timesteps 966.
Path 62 | total_timesteps 975.
Path 63 | total_timesteps 993.
Path 64 | total_timesteps 1011.
Path 65 | total_timesteps 1020.
Path 66 | total_timesteps 1031.
Path 67 | total_timesteps 1056.
Path 68 | total_timesteps 1071.
Path 69 | total_timesteps 1084.
Path 70 | total_timesteps 1100.
Path 71 | total_timesteps 1112.
Path 72 | total_timesteps 1148.
Path 73 | total_timesteps 1165.
Path 74 | total_timesteps 1191.
Path 75 | total_timesteps 1224.
Path 76 | total_timesteps 1236.
Path 77 | total_timesteps 1246.
Path 78 | total_timesteps 1258.
Path 79 | total_timesteps 1274.
Path 80 | total_timesteps 1292.
Path 81 | total_timesteps 1310.
Path 82 | total_timesteps 1322.
Path 83 | total_timesteps 1349.
Path 84 | total_timesteps 1358.
Path 85 | total_timesteps 1373.
Path 86 | total_timesteps 1390.
Path 87 | total_timesteps 1418.
Path 88 | total_timesteps 1429.
Path 89 | total_timesteps 1453.
Path 90 | total_timesteps 1465.
Path 91 | total_timesteps 1488.
Path 92 | total_timesteps 1507.
Path 93 | total_timesteps 1521.
Path 94 | total_timesteps 1539.
Path 95 | total_timesteps 1554.
Path 96 | total_timesteps 1578.
Path 97 | total_timesteps 1588.
Path 98 | total_timesteps 1599.
Path 99 | total_timesteps 1608.
Path 100 | total_timesteps 1624.
Path 101 | total_timesteps 1636.
Path 102 | total_timesteps 1658.
Path 103 | total_timesteps 1668.
Path 104 | total_timesteps 1679.
Path 105 | total_timesteps 1686.
Path 106 | total_timesteps 1700.
Path 107 | total_timesteps 1710.
Path 108 | total_timesteps 1723.
Path 109 | total_timesteps 1732.
Path 110 | total_timesteps 1747.
Path 111 | total_timesteps 1757.
Path 112 | total_timesteps 1768.
Path 113 | total_timesteps 1788.
Path 114 | total_timesteps 1800.
Path 115 | total_timesteps 1824.
Path 116 | total_timesteps 1841.
Path 117 | total_timesteps 1853.
Path 118 | total_timesteps 1867.
Path 119 | total_timesteps 1876.
Path 120 | total_timesteps 1885.
Path 121 | total_timesteps 1902.
Path 122 | total_timesteps 1922.
Path 123 | total_timesteps 1944.
Path 124 | total_timesteps 1961.
Path 125 | total_timesteps 1973.
Path 126 | total_timesteps 1988.
Path 127 | total_timesteps 2018.
Path 128 | total_timesteps 2052.
Path 129 | total_timesteps 2066.
Path 130 | total_timesteps 2082.
Path 131 | total_timesteps 2091.
Path 132 | total_timesteps 2120.
Path 133 | total_timesteps 2132.
Path 134 | total_timesteps 2142.
Path 135 | total_timesteps 2154.
Path 136 | total_timesteps 2174.
Path 137 | total_timesteps 2188.
Path 138 | total_timesteps 2202.
Path 139 | total_timesteps 2219.
Path 140 | total_timesteps 2232.
Path 141 | total_timesteps 2250.
Path 142 | total_timesteps 2269.
Path 143 | total_timesteps 2282.
Path 144 | total_timesteps 2295.
Path 145 | total_timesteps 2325.
Path 146 | total_timesteps 2345.
Path 147 | total_timesteps 2359.
Path 148 | total_timesteps 2379.
Path 149 | total_timesteps 2400.
Path 150 | total_timesteps 2430.
Path 151 | total_timesteps 2449.
Path 152 | total_timesteps 2461.
Path 153 | total_timesteps 2474.
Path 154 | total_timesteps 2489.
Path 155 | total_timesteps 2507.
Path 156 | total_timesteps 2525.
Path 157 | total_timesteps 2542.
Path 158 | total_timesteps 2557.
Path 159 | total_timesteps 2571.
Path 160 | total_timesteps 2591.
Path 161 | total_timesteps 2605.
Path 162 | total_timesteps 2617.
Path 163 | total_timesteps 2629.
Path 164 | total_timesteps 2658.
Path 165 | total_timesteps 2673.
Path 166 | total_timesteps 2689.
Path 167 | total_timesteps 2704.
Path 168 | total_timesteps 2715.
Path 169 | total_timesteps 2725.
Path 170 | total_timesteps 2739.
Path 171 | total_timesteps 2751.
Path 172 | total_timesteps 2761.
Path 173 | total_timesteps 2805.
Path 174 | total_timesteps 2819.
Path 175 | total_timesteps 2833.
Path 176 | total_timesteps 2845.
Path 177 | total_timesteps 2873.
Path 178 | total_timesteps 2890.
Path 179 | total_timesteps 2919.
Path 180 | total_timesteps 2938.
Path 181 | total_timesteps 2954.
Path 182 | total_timesteps 2967.
Path 183 | total_timesteps 2977.
Path 184 | total_timesteps 2999.
Path 185 | total_timesteps 3011.
Path 186 | total_timesteps 3019.
Path 187 | total_timesteps 3036.
Path 188 | total_timesteps 3049.
Path 189 | total_timesteps 3066.
Path 190 | total_timesteps 3078.
Path 191 | total_timesteps 3090.
Path 192 | total_timesteps 3100.
Path 193 | total_timesteps 3111.
Path 194 | total_timesteps 3121.
Path 195 | total_timesteps 3150.
Path 196 | total_timesteps 3166.
Path 197 | total_timesteps 3179.
Path 198 | total_timesteps 3198.
Path 199 | total_timesteps 3220.
Path 200 | total_timesteps 3236.
Path 201 | total_timesteps 3256.
Path 202 | total_timesteps 3274.
Path 203 | total_timesteps 3308.
Path 204 | total_timesteps 3316.
Path 205 | total_timesteps 3328.
Path 206 | total_timesteps 3349.
Path 207 | total_timesteps 3362.
Path 208 | total_timesteps 3377.
Path 209 | total_timesteps 3388.
Path 210 | total_timesteps 3402.
Path 211 | total_timesteps 3423.
Path 212 | total_timesteps 3440.
Path 213 | total_timesteps 3464.
Path 214 | total_timesteps 3487.
Path 215 | total_timesteps 3497.
Path 216 | total_timesteps 3509.
Path 217 | total_timesteps 3524.
Path 218 | total_timesteps 3535.
Path 219 | total_timesteps 3554.
Path 220 | total_timesteps 3565.
Path 221 | total_timesteps 3593.
Path 222 | total_timesteps 3602.
Path 223 | total_timesteps 3618.
Path 224 | total_timesteps 3636.
Path 225 | total_timesteps 3649.
Path 226 | total_timesteps 3663.
Path 227 | total_timesteps 3674.
Path 228 | total_timesteps 3684.
Path 229 | total_timesteps 3696.
Path 230 | total_timesteps 3714.
Path 231 | total_timesteps 3734.
Path 232 | total_timesteps 3746.
Path 233 | total_timesteps 3777.
Path 234 | total_timesteps 3804.
Path 235 | total_timesteps 3820.
Path 236 | total_timesteps 3841.
Path 237 | total_timesteps 3853.
Path 238 | total_timesteps 3873.
Path 239 | total_timesteps 3889.
Path 240 | total_timesteps 3907.
Path 241 | total_timesteps 3929.
Path 242 | total_timesteps 3940.
Path 243 | total_timesteps 3950.
Path 244 | total_timesteps 3968.
Path 245 | total_timesteps 3987.
Path 246 | total_timesteps 3995.
Path 247 | total_timesteps 4007.
Path 248 | total_timesteps 4019.
Path 249 | total_timesteps 4043.
Path 250 | total_timesteps 4057.
Path 251 | total_timesteps 4082.
Path 252 | total_timesteps 4090.
Path 253 | total_timesteps 4100.
Path 254 | total_timesteps 4114.
Path 255 | total_timesteps 4131.
Path 256 | total_timesteps 4151.
Path 257 | total_timesteps 4165.
Path 258 | total_timesteps 4180.
Path 259 | total_timesteps 4189.
Path 260 | total_timesteps 4211.
Path 261 | total_timesteps 4221.
Path 262 | total_timesteps 4233.
Path 263 | total_timesteps 4252.
Path 264 | total_timesteps 4261.
Path 265 | total_timesteps 4270.
Path 266 | total_timesteps 4283.
Path 267 | total_timesteps 4295.
Path 268 | total_timesteps 4312.
Path 269 | total_timesteps 4324.
Path 270 | total_timesteps 4338.
Path 271 | total_timesteps 4353.
Path 272 | total_timesteps 4379.
Path 273 | total_timesteps 4388.
Path 274 | total_timesteps 4397.
Path 275 | total_timesteps 4406.
Path 276 | total_timesteps 4417.
Path 277 | total_timesteps 4433.
Path 278 | total_timesteps 4445.
Path 279 | total_timesteps 4457.
Path 280 | total_timesteps 4477.
Path 281 | total_timesteps 4486.
Path 282 | total_timesteps 4497.
Path 283 | total_timesteps 4513.
Path 284 | total_timesteps 4565.
Path 285 | total_timesteps 4578.
Path 286 | total_timesteps 4590.
Path 287 | total_timesteps 4609.
Path 288 | total_timesteps 4623.
Path 289 | total_timesteps 4642.
Path 290 | total_timesteps 4658.
Path 291 | total_timesteps 4680.
Path 292 | total_timesteps 4693.
Path 293 | total_timesteps 4725.
Path 294 | total_timesteps 4738.
Path 295 | total_timesteps 4751.
Path 296 | total_timesteps 4760.
Path 297 | total_timesteps 4776.
Path 298 | total_timesteps 4795.
Path 299 | total_timesteps 4821.
Path 300 | total_timesteps 4836.
Path 301 | total_timesteps 4859.
Path 302 | total_timesteps 4881.
Path 303 | total_timesteps 4891.
Path 304 | total_timesteps 4915.
Path 305 | total_timesteps 4939.
Path 306 | total_timesteps 4953.
Path 307 | total_timesteps 4973.
Path 308 | total_timesteps 4987.
Path 309 | total_timesteps 5010.
Path 310 | total_timesteps 5030.
Path 311 | total_timesteps 5044.
Path 312 | total_timesteps 5056.
Path 313 | total_timesteps 5068.
Path 314 | total_timesteps 5080.
Path 315 | total_timesteps 5092.
Path 316 | total_timesteps 5108.
Path 317 | total_timesteps 5121.
Path 318 | total_timesteps 5146.
Path 319 | total_timesteps 5158.
Path 320 | total_timesteps 5168.
Path 321 | total_timesteps 5180.
Path 322 | total_timesteps 5204.
Path 323 | total_timesteps 5219.
Path 324 | total_timesteps 5239.
Path 325 | total_timesteps 5273.
Path 326 | total_timesteps 5286.
Path 327 | total_timesteps 5302.
Path 328 | total_timesteps 5317.
Path 329 | total_timesteps 5327.
Path 330 | total_timesteps 5342.
Path 331 | total_timesteps 5361.
Path 332 | total_timesteps 5372.
Path 333 | total_timesteps 5413.
Path 334 | total_timesteps 5425.
Path 335 | total_timesteps 5437.
Path 336 | total_timesteps 5454.
Path 337 | total_timesteps 5469.
Path 338 | total_timesteps 5483.
Path 339 | total_timesteps 5492.
Path 340 | total_timesteps 5525.
Path 341 | total_timesteps 5539.
Path 342 | total_timesteps 5555.
Path 343 | total_timesteps 5566.
Path 344 | total_timesteps 5577.
Path 345 | total_timesteps 5610.
Path 346 | total_timesteps 5637.
Path 347 | total_timesteps 5657.
Path 348 | total_timesteps 5674.
Path 349 | total_timesteps 5695.
Path 350 | total_timesteps 5706.
Path 351 | total_timesteps 5720.
Path 352 | total_timesteps 5734.
Path 353 | total_timesteps 5760.
Path 354 | total_timesteps 5783.
Path 355 | total_timesteps 5790.
Path 356 | total_timesteps 5804.
Path 357 | total_timesteps 5823.
Path 358 | total_timesteps 5833.
Path 359 | total_timesteps 5844.
Path 360 | total_timesteps 5862.
Path 361 | total_timesteps 5871.
Path 362 | total_timesteps 5880.
Path 363 | total_timesteps 5904.
Path 364 | total_timesteps 5921.
Path 365 | total_timesteps 5931.
Path 366 | total_timesteps 5949.
Path 367 | total_timesteps 5964.
Path 368 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.2     |
| Iteration     | 5        |
| MaximumReturn | 7.09     |
| MinimumReturn | -20.6    |
| TotalSamples  | 28028    |
----------------------------
itr #6 | 
Fitting dynamics.
Validation loss = 0.0172348003834486
Validation loss = 0.015559002757072449
Validation loss = 0.018446877598762512
Validation loss = 0.01528247632086277
Validation loss = 0.015016933903098106
Validation loss = 0.01557070016860962
Validation loss = 0.014942856505513191
Validation loss = 0.014733877964317799
Validation loss = 0.014247951097786427
Validation loss = 0.015196370892226696
Validation loss = 0.014795003458857536
Validation loss = 0.015577760525047779
Validation loss = 0.014675376936793327
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 20.
Path 2 | total_timesteps 53.
Path 3 | total_timesteps 72.
Path 4 | total_timesteps 86.
Path 5 | total_timesteps 98.
Path 6 | total_timesteps 108.
Path 7 | total_timesteps 121.
Path 8 | total_timesteps 143.
Path 9 | total_timesteps 161.
Path 10 | total_timesteps 174.
Path 11 | total_timesteps 193.
Path 12 | total_timesteps 203.
Path 13 | total_timesteps 223.
Path 14 | total_timesteps 236.
Path 15 | total_timesteps 244.
Path 16 | total_timesteps 251.
Path 17 | total_timesteps 268.
Path 18 | total_timesteps 289.
Path 19 | total_timesteps 304.
Path 20 | total_timesteps 325.
Path 21 | total_timesteps 334.
Path 22 | total_timesteps 348.
Path 23 | total_timesteps 360.
Path 24 | total_timesteps 372.
Path 25 | total_timesteps 395.
Path 26 | total_timesteps 405.
Path 27 | total_timesteps 423.
Path 28 | total_timesteps 438.
Path 29 | total_timesteps 452.
Path 30 | total_timesteps 465.
Path 31 | total_timesteps 486.
Path 32 | total_timesteps 494.
Path 33 | total_timesteps 510.
Path 34 | total_timesteps 521.
Path 35 | total_timesteps 542.
Path 36 | total_timesteps 553.
Path 37 | total_timesteps 563.
Path 38 | total_timesteps 583.
Path 39 | total_timesteps 594.
Path 40 | total_timesteps 610.
Path 41 | total_timesteps 627.
Path 42 | total_timesteps 641.
Path 43 | total_timesteps 660.
Path 44 | total_timesteps 669.
Path 45 | total_timesteps 682.
Path 46 | total_timesteps 696.
Path 47 | total_timesteps 720.
Path 48 | total_timesteps 734.
Path 49 | total_timesteps 752.
Path 50 | total_timesteps 760.
Path 51 | total_timesteps 772.
Path 52 | total_timesteps 788.
Path 53 | total_timesteps 805.
Path 54 | total_timesteps 832.
Path 55 | total_timesteps 845.
Path 56 | total_timesteps 867.
Path 57 | total_timesteps 889.
Path 58 | total_timesteps 900.
Path 59 | total_timesteps 919.
Path 60 | total_timesteps 935.
Path 61 | total_timesteps 949.
Path 62 | total_timesteps 967.
Path 63 | total_timesteps 979.
Path 64 | total_timesteps 992.
Path 65 | total_timesteps 1009.
Path 66 | total_timesteps 1031.
Path 67 | total_timesteps 1042.
Path 68 | total_timesteps 1051.
Path 69 | total_timesteps 1058.
Path 70 | total_timesteps 1070.
Path 71 | total_timesteps 1088.
Path 72 | total_timesteps 1108.
Path 73 | total_timesteps 1119.
Path 74 | total_timesteps 1136.
Path 75 | total_timesteps 1150.
Path 76 | total_timesteps 1160.
Path 77 | total_timesteps 1175.
Path 78 | total_timesteps 1189.
Path 79 | total_timesteps 1200.
Path 80 | total_timesteps 1220.
Path 81 | total_timesteps 1234.
Path 82 | total_timesteps 1246.
Path 83 | total_timesteps 1272.
Path 84 | total_timesteps 1282.
Path 85 | total_timesteps 1293.
Path 86 | total_timesteps 1308.
Path 87 | total_timesteps 1322.
Path 88 | total_timesteps 1353.
Path 89 | total_timesteps 1364.
Path 90 | total_timesteps 1383.
Path 91 | total_timesteps 1393.
Path 92 | total_timesteps 1406.
Path 93 | total_timesteps 1419.
Path 94 | total_timesteps 1428.
Path 95 | total_timesteps 1445.
Path 96 | total_timesteps 1457.
Path 97 | total_timesteps 1464.
Path 98 | total_timesteps 1475.
Path 99 | total_timesteps 1485.
Path 100 | total_timesteps 1511.
Path 101 | total_timesteps 1525.
Path 102 | total_timesteps 1536.
Path 103 | total_timesteps 1560.
Path 104 | total_timesteps 1572.
Path 105 | total_timesteps 1591.
Path 106 | total_timesteps 1616.
Path 107 | total_timesteps 1627.
Path 108 | total_timesteps 1643.
Path 109 | total_timesteps 1661.
Path 110 | total_timesteps 1670.
Path 111 | total_timesteps 1685.
Path 112 | total_timesteps 1707.
Path 113 | total_timesteps 1721.
Path 114 | total_timesteps 1746.
Path 115 | total_timesteps 1758.
Path 116 | total_timesteps 1768.
Path 117 | total_timesteps 1778.
Path 118 | total_timesteps 1785.
Path 119 | total_timesteps 1797.
Path 120 | total_timesteps 1814.
Path 121 | total_timesteps 1828.
Path 122 | total_timesteps 1846.
Path 123 | total_timesteps 1859.
Path 124 | total_timesteps 1873.
Path 125 | total_timesteps 1893.
Path 126 | total_timesteps 1931.
Path 127 | total_timesteps 1949.
Path 128 | total_timesteps 1963.
Path 129 | total_timesteps 1975.
Path 130 | total_timesteps 1999.
Path 131 | total_timesteps 2032.
Path 132 | total_timesteps 2049.
Path 133 | total_timesteps 2068.
Path 134 | total_timesteps 2081.
Path 135 | total_timesteps 2090.
Path 136 | total_timesteps 2101.
Path 137 | total_timesteps 2112.
Path 138 | total_timesteps 2125.
Path 139 | total_timesteps 2138.
Path 140 | total_timesteps 2154.
Path 141 | total_timesteps 2178.
Path 142 | total_timesteps 2198.
Path 143 | total_timesteps 2215.
Path 144 | total_timesteps 2230.
Path 145 | total_timesteps 2245.
Path 146 | total_timesteps 2257.
Path 147 | total_timesteps 2267.
Path 148 | total_timesteps 2283.
Path 149 | total_timesteps 2300.
Path 150 | total_timesteps 2313.
Path 151 | total_timesteps 2330.
Path 152 | total_timesteps 2343.
Path 153 | total_timesteps 2364.
Path 154 | total_timesteps 2373.
Path 155 | total_timesteps 2388.
Path 156 | total_timesteps 2398.
Path 157 | total_timesteps 2416.
Path 158 | total_timesteps 2430.
Path 159 | total_timesteps 2447.
Path 160 | total_timesteps 2458.
Path 161 | total_timesteps 2468.
Path 162 | total_timesteps 2482.
Path 163 | total_timesteps 2498.
Path 164 | total_timesteps 2506.
Path 165 | total_timesteps 2535.
Path 166 | total_timesteps 2545.
Path 167 | total_timesteps 2554.
Path 168 | total_timesteps 2562.
Path 169 | total_timesteps 2575.
Path 170 | total_timesteps 2593.
Path 171 | total_timesteps 2605.
Path 172 | total_timesteps 2613.
Path 173 | total_timesteps 2633.
Path 174 | total_timesteps 2647.
Path 175 | total_timesteps 2661.
Path 176 | total_timesteps 2675.
Path 177 | total_timesteps 2695.
Path 178 | total_timesteps 2704.
Path 179 | total_timesteps 2716.
Path 180 | total_timesteps 2739.
Path 181 | total_timesteps 2753.
Path 182 | total_timesteps 2767.
Path 183 | total_timesteps 2794.
Path 184 | total_timesteps 2803.
Path 185 | total_timesteps 2823.
Path 186 | total_timesteps 2839.
Path 187 | total_timesteps 2859.
Path 188 | total_timesteps 2875.
Path 189 | total_timesteps 2886.
Path 190 | total_timesteps 2898.
Path 191 | total_timesteps 2938.
Path 192 | total_timesteps 2950.
Path 193 | total_timesteps 2959.
Path 194 | total_timesteps 2967.
Path 195 | total_timesteps 2984.
Path 196 | total_timesteps 2997.
Path 197 | total_timesteps 3010.
Path 198 | total_timesteps 3024.
Path 199 | total_timesteps 3037.
Path 200 | total_timesteps 3048.
Path 201 | total_timesteps 3068.
Path 202 | total_timesteps 3094.
Path 203 | total_timesteps 3120.
Path 204 | total_timesteps 3137.
Path 205 | total_timesteps 3149.
Path 206 | total_timesteps 3182.
Path 207 | total_timesteps 3191.
Path 208 | total_timesteps 3230.
Path 209 | total_timesteps 3245.
Path 210 | total_timesteps 3256.
Path 211 | total_timesteps 3268.
Path 212 | total_timesteps 3283.
Path 213 | total_timesteps 3300.
Path 214 | total_timesteps 3309.
Path 215 | total_timesteps 3329.
Path 216 | total_timesteps 3341.
Path 217 | total_timesteps 3360.
Path 218 | total_timesteps 3383.
Path 219 | total_timesteps 3395.
Path 220 | total_timesteps 3408.
Path 221 | total_timesteps 3417.
Path 222 | total_timesteps 3447.
Path 223 | total_timesteps 3474.
Path 224 | total_timesteps 3493.
Path 225 | total_timesteps 3506.
Path 226 | total_timesteps 3515.
Path 227 | total_timesteps 3528.
Path 228 | total_timesteps 3551.
Path 229 | total_timesteps 3570.
Path 230 | total_timesteps 3582.
Path 231 | total_timesteps 3599.
Path 232 | total_timesteps 3608.
Path 233 | total_timesteps 3622.
Path 234 | total_timesteps 3635.
Path 235 | total_timesteps 3650.
Path 236 | total_timesteps 3670.
Path 237 | total_timesteps 3688.
Path 238 | total_timesteps 3706.
Path 239 | total_timesteps 3731.
Path 240 | total_timesteps 3746.
Path 241 | total_timesteps 3760.
Path 242 | total_timesteps 3774.
Path 243 | total_timesteps 3788.
Path 244 | total_timesteps 3798.
Path 245 | total_timesteps 3819.
Path 246 | total_timesteps 3834.
Path 247 | total_timesteps 3856.
Path 248 | total_timesteps 3869.
Path 249 | total_timesteps 3895.
Path 250 | total_timesteps 3923.
Path 251 | total_timesteps 3936.
Path 252 | total_timesteps 3947.
Path 253 | total_timesteps 3969.
Path 254 | total_timesteps 3992.
Path 255 | total_timesteps 4003.
Path 256 | total_timesteps 4017.
Path 257 | total_timesteps 4029.
Path 258 | total_timesteps 4040.
Path 259 | total_timesteps 4075.
Path 260 | total_timesteps 4103.
Path 261 | total_timesteps 4116.
Path 262 | total_timesteps 4141.
Path 263 | total_timesteps 4164.
Path 264 | total_timesteps 4187.
Path 265 | total_timesteps 4205.
Path 266 | total_timesteps 4219.
Path 267 | total_timesteps 4230.
Path 268 | total_timesteps 4246.
Path 269 | total_timesteps 4272.
Path 270 | total_timesteps 4283.
Path 271 | total_timesteps 4296.
Path 272 | total_timesteps 4308.
Path 273 | total_timesteps 4320.
Path 274 | total_timesteps 4335.
Path 275 | total_timesteps 4351.
Path 276 | total_timesteps 4367.
Path 277 | total_timesteps 4388.
Path 278 | total_timesteps 4413.
Path 279 | total_timesteps 4429.
Path 280 | total_timesteps 4438.
Path 281 | total_timesteps 4447.
Path 282 | total_timesteps 4469.
Path 283 | total_timesteps 4478.
Path 284 | total_timesteps 4485.
Path 285 | total_timesteps 4495.
Path 286 | total_timesteps 4519.
Path 287 | total_timesteps 4534.
Path 288 | total_timesteps 4554.
Path 289 | total_timesteps 4567.
Path 290 | total_timesteps 4587.
Path 291 | total_timesteps 4596.
Path 292 | total_timesteps 4604.
Path 293 | total_timesteps 4619.
Path 294 | total_timesteps 4628.
Path 295 | total_timesteps 4642.
Path 296 | total_timesteps 4652.
Path 297 | total_timesteps 4664.
Path 298 | total_timesteps 4678.
Path 299 | total_timesteps 4696.
Path 300 | total_timesteps 4707.
Path 301 | total_timesteps 4717.
Path 302 | total_timesteps 4731.
Path 303 | total_timesteps 4748.
Path 304 | total_timesteps 4766.
Path 305 | total_timesteps 4781.
Path 306 | total_timesteps 4810.
Path 307 | total_timesteps 4830.
Path 308 | total_timesteps 4838.
Path 309 | total_timesteps 4846.
Path 310 | total_timesteps 4855.
Path 311 | total_timesteps 4867.
Path 312 | total_timesteps 4886.
Path 313 | total_timesteps 4901.
Path 314 | total_timesteps 4921.
Path 315 | total_timesteps 4932.
Path 316 | total_timesteps 4950.
Path 317 | total_timesteps 4958.
Path 318 | total_timesteps 4969.
Path 319 | total_timesteps 4991.
Path 320 | total_timesteps 5003.
Path 321 | total_timesteps 5022.
Path 322 | total_timesteps 5044.
Path 323 | total_timesteps 5059.
Path 324 | total_timesteps 5075.
Path 325 | total_timesteps 5100.
Path 326 | total_timesteps 5112.
Path 327 | total_timesteps 5132.
Path 328 | total_timesteps 5149.
Path 329 | total_timesteps 5161.
Path 330 | total_timesteps 5176.
Path 331 | total_timesteps 5186.
Path 332 | total_timesteps 5200.
Path 333 | total_timesteps 5221.
Path 334 | total_timesteps 5233.
Path 335 | total_timesteps 5256.
Path 336 | total_timesteps 5273.
Path 337 | total_timesteps 5286.
Path 338 | total_timesteps 5297.
Path 339 | total_timesteps 5309.
Path 340 | total_timesteps 5319.
Path 341 | total_timesteps 5341.
Path 342 | total_timesteps 5352.
Path 343 | total_timesteps 5370.
Path 344 | total_timesteps 5379.
Path 345 | total_timesteps 5397.
Path 346 | total_timesteps 5410.
Path 347 | total_timesteps 5428.
Path 348 | total_timesteps 5447.
Path 349 | total_timesteps 5461.
Path 350 | total_timesteps 5473.
Path 351 | total_timesteps 5488.
Path 352 | total_timesteps 5498.
Path 353 | total_timesteps 5514.
Path 354 | total_timesteps 5527.
Path 355 | total_timesteps 5540.
Path 356 | total_timesteps 5558.
Path 357 | total_timesteps 5574.
Path 358 | total_timesteps 5588.
Path 359 | total_timesteps 5606.
Path 360 | total_timesteps 5617.
Path 361 | total_timesteps 5629.
Path 362 | total_timesteps 5645.
Path 363 | total_timesteps 5655.
Path 364 | total_timesteps 5666.
Path 365 | total_timesteps 5676.
Path 366 | total_timesteps 5704.
Path 367 | total_timesteps 5721.
Path 368 | total_timesteps 5734.
Path 369 | total_timesteps 5753.
Path 370 | total_timesteps 5777.
Path 371 | total_timesteps 5793.
Path 372 | total_timesteps 5812.
Path 373 | total_timesteps 5844.
Path 374 | total_timesteps 5858.
Path 375 | total_timesteps 5866.
Path 376 | total_timesteps 5878.
Path 377 | total_timesteps 5891.
Path 378 | total_timesteps 5901.
Path 379 | total_timesteps 5918.
Path 380 | total_timesteps 5955.
Path 381 | total_timesteps 5966.
Path 382 | total_timesteps 5987.
Path 383 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.69    |
| Iteration     | 6        |
| MaximumReturn | 7.81     |
| MinimumReturn | -20.4    |
| TotalSamples  | 32038    |
----------------------------
itr #7 | 
Fitting dynamics.
Validation loss = 0.016357410699129105
Validation loss = 0.013295098207890987
Validation loss = 0.013245603069663048
Validation loss = 0.013207390904426575
Validation loss = 0.013448519632220268
Validation loss = 0.015857595950365067
Validation loss = 0.01477776188403368
Validation loss = 0.01439102366566658
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 29.
Path 3 | total_timesteps 39.
Path 4 | total_timesteps 54.
Path 5 | total_timesteps 72.
Path 6 | total_timesteps 90.
Path 7 | total_timesteps 110.
Path 8 | total_timesteps 129.
Path 9 | total_timesteps 139.
Path 10 | total_timesteps 160.
Path 11 | total_timesteps 171.
Path 12 | total_timesteps 195.
Path 13 | total_timesteps 222.
Path 14 | total_timesteps 239.
Path 15 | total_timesteps 259.
Path 16 | total_timesteps 268.
Path 17 | total_timesteps 290.
Path 18 | total_timesteps 305.
Path 19 | total_timesteps 323.
Path 20 | total_timesteps 336.
Path 21 | total_timesteps 345.
Path 22 | total_timesteps 357.
Path 23 | total_timesteps 378.
Path 24 | total_timesteps 387.
Path 25 | total_timesteps 403.
Path 26 | total_timesteps 416.
Path 27 | total_timesteps 428.
Path 28 | total_timesteps 439.
Path 29 | total_timesteps 455.
Path 30 | total_timesteps 473.
Path 31 | total_timesteps 488.
Path 32 | total_timesteps 509.
Path 33 | total_timesteps 525.
Path 34 | total_timesteps 543.
Path 35 | total_timesteps 560.
Path 36 | total_timesteps 569.
Path 37 | total_timesteps 596.
Path 38 | total_timesteps 610.
Path 39 | total_timesteps 621.
Path 40 | total_timesteps 632.
Path 41 | total_timesteps 646.
Path 42 | total_timesteps 659.
Path 43 | total_timesteps 674.
Path 44 | total_timesteps 691.
Path 45 | total_timesteps 701.
Path 46 | total_timesteps 723.
Path 47 | total_timesteps 734.
Path 48 | total_timesteps 745.
Path 49 | total_timesteps 767.
Path 50 | total_timesteps 779.
Path 51 | total_timesteps 790.
Path 52 | total_timesteps 806.
Path 53 | total_timesteps 818.
Path 54 | total_timesteps 851.
Path 55 | total_timesteps 863.
Path 56 | total_timesteps 878.
Path 57 | total_timesteps 889.
Path 58 | total_timesteps 907.
Path 59 | total_timesteps 934.
Path 60 | total_timesteps 954.
Path 61 | total_timesteps 972.
Path 62 | total_timesteps 985.
Path 63 | total_timesteps 993.
Path 64 | total_timesteps 1005.
Path 65 | total_timesteps 1020.
Path 66 | total_timesteps 1028.
Path 67 | total_timesteps 1050.
Path 68 | total_timesteps 1067.
Path 69 | total_timesteps 1084.
Path 70 | total_timesteps 1106.
Path 71 | total_timesteps 1118.
Path 72 | total_timesteps 1139.
Path 73 | total_timesteps 1158.
Path 74 | total_timesteps 1175.
Path 75 | total_timesteps 1189.
Path 76 | total_timesteps 1203.
Path 77 | total_timesteps 1225.
Path 78 | total_timesteps 1242.
Path 79 | total_timesteps 1265.
Path 80 | total_timesteps 1275.
Path 81 | total_timesteps 1291.
Path 82 | total_timesteps 1317.
Path 83 | total_timesteps 1336.
Path 84 | total_timesteps 1352.
Path 85 | total_timesteps 1369.
Path 86 | total_timesteps 1380.
Path 87 | total_timesteps 1399.
Path 88 | total_timesteps 1414.
Path 89 | total_timesteps 1437.
Path 90 | total_timesteps 1459.
Path 91 | total_timesteps 1471.
Path 92 | total_timesteps 1483.
Path 93 | total_timesteps 1493.
Path 94 | total_timesteps 1506.
Path 95 | total_timesteps 1516.
Path 96 | total_timesteps 1531.
Path 97 | total_timesteps 1543.
Path 98 | total_timesteps 1558.
Path 99 | total_timesteps 1572.
Path 100 | total_timesteps 1587.
Path 101 | total_timesteps 1598.
Path 102 | total_timesteps 1608.
Path 103 | total_timesteps 1621.
Path 104 | total_timesteps 1638.
Path 105 | total_timesteps 1654.
Path 106 | total_timesteps 1673.
Path 107 | total_timesteps 1689.
Path 108 | total_timesteps 1704.
Path 109 | total_timesteps 1717.
Path 110 | total_timesteps 1734.
Path 111 | total_timesteps 1761.
Path 112 | total_timesteps 1771.
Path 113 | total_timesteps 1791.
Path 114 | total_timesteps 1806.
Path 115 | total_timesteps 1817.
Path 116 | total_timesteps 1832.
Path 117 | total_timesteps 1855.
Path 118 | total_timesteps 1869.
Path 119 | total_timesteps 1882.
Path 120 | total_timesteps 1890.
Path 121 | total_timesteps 1906.
Path 122 | total_timesteps 1923.
Path 123 | total_timesteps 1944.
Path 124 | total_timesteps 1954.
Path 125 | total_timesteps 1975.
Path 126 | total_timesteps 1985.
Path 127 | total_timesteps 2003.
Path 128 | total_timesteps 2017.
Path 129 | total_timesteps 2040.
Path 130 | total_timesteps 2051.
Path 131 | total_timesteps 2062.
Path 132 | total_timesteps 2076.
Path 133 | total_timesteps 2091.
Path 134 | total_timesteps 2102.
Path 135 | total_timesteps 2113.
Path 136 | total_timesteps 2132.
Path 137 | total_timesteps 2143.
Path 138 | total_timesteps 2154.
Path 139 | total_timesteps 2177.
Path 140 | total_timesteps 2191.
Path 141 | total_timesteps 2216.
Path 142 | total_timesteps 2225.
Path 143 | total_timesteps 2241.
Path 144 | total_timesteps 2260.
Path 145 | total_timesteps 2270.
Path 146 | total_timesteps 2281.
Path 147 | total_timesteps 2292.
Path 148 | total_timesteps 2307.
Path 149 | total_timesteps 2320.
Path 150 | total_timesteps 2346.
Path 151 | total_timesteps 2378.
Path 152 | total_timesteps 2402.
Path 153 | total_timesteps 2413.
Path 154 | total_timesteps 2436.
Path 155 | total_timesteps 2457.
Path 156 | total_timesteps 2469.
Path 157 | total_timesteps 2486.
Path 158 | total_timesteps 2497.
Path 159 | total_timesteps 2509.
Path 160 | total_timesteps 2525.
Path 161 | total_timesteps 2552.
Path 162 | total_timesteps 2565.
Path 163 | total_timesteps 2582.
Path 164 | total_timesteps 2598.
Path 165 | total_timesteps 2617.
Path 166 | total_timesteps 2633.
Path 167 | total_timesteps 2646.
Path 168 | total_timesteps 2665.
Path 169 | total_timesteps 2677.
Path 170 | total_timesteps 2691.
Path 171 | total_timesteps 2703.
Path 172 | total_timesteps 2713.
Path 173 | total_timesteps 2730.
Path 174 | total_timesteps 2742.
Path 175 | total_timesteps 2757.
Path 176 | total_timesteps 2776.
Path 177 | total_timesteps 2784.
Path 178 | total_timesteps 2795.
Path 179 | total_timesteps 2822.
Path 180 | total_timesteps 2835.
Path 181 | total_timesteps 2854.
Path 182 | total_timesteps 2875.
Path 183 | total_timesteps 2897.
Path 184 | total_timesteps 2910.
Path 185 | total_timesteps 2921.
Path 186 | total_timesteps 2937.
Path 187 | total_timesteps 2946.
Path 188 | total_timesteps 2965.
Path 189 | total_timesteps 2980.
Path 190 | total_timesteps 2989.
Path 191 | total_timesteps 3006.
Path 192 | total_timesteps 3024.
Path 193 | total_timesteps 3036.
Path 194 | total_timesteps 3058.
Path 195 | total_timesteps 3081.
Path 196 | total_timesteps 3092.
Path 197 | total_timesteps 3107.
Path 198 | total_timesteps 3120.
Path 199 | total_timesteps 3134.
Path 200 | total_timesteps 3152.
Path 201 | total_timesteps 3170.
Path 202 | total_timesteps 3184.
Path 203 | total_timesteps 3202.
Path 204 | total_timesteps 3221.
Path 205 | total_timesteps 3243.
Path 206 | total_timesteps 3257.
Path 207 | total_timesteps 3267.
Path 208 | total_timesteps 3280.
Path 209 | total_timesteps 3293.
Path 210 | total_timesteps 3302.
Path 211 | total_timesteps 3316.
Path 212 | total_timesteps 3332.
Path 213 | total_timesteps 3361.
Path 214 | total_timesteps 3380.
Path 215 | total_timesteps 3390.
Path 216 | total_timesteps 3398.
Path 217 | total_timesteps 3416.
Path 218 | total_timesteps 3429.
Path 219 | total_timesteps 3440.
Path 220 | total_timesteps 3462.
Path 221 | total_timesteps 3470.
Path 222 | total_timesteps 3488.
Path 223 | total_timesteps 3501.
Path 224 | total_timesteps 3521.
Path 225 | total_timesteps 3547.
Path 226 | total_timesteps 3559.
Path 227 | total_timesteps 3582.
Path 228 | total_timesteps 3597.
Path 229 | total_timesteps 3603.
Path 230 | total_timesteps 3615.
Path 231 | total_timesteps 3635.
Path 232 | total_timesteps 3661.
Path 233 | total_timesteps 3688.
Path 234 | total_timesteps 3702.
Path 235 | total_timesteps 3721.
Path 236 | total_timesteps 3733.
Path 237 | total_timesteps 3748.
Path 238 | total_timesteps 3764.
Path 239 | total_timesteps 3781.
Path 240 | total_timesteps 3790.
Path 241 | total_timesteps 3800.
Path 242 | total_timesteps 3816.
Path 243 | total_timesteps 3830.
Path 244 | total_timesteps 3854.
Path 245 | total_timesteps 3867.
Path 246 | total_timesteps 3880.
Path 247 | total_timesteps 3897.
Path 248 | total_timesteps 3918.
Path 249 | total_timesteps 3929.
Path 250 | total_timesteps 3947.
Path 251 | total_timesteps 3961.
Path 252 | total_timesteps 3975.
Path 253 | total_timesteps 3984.
Path 254 | total_timesteps 4006.
Path 255 | total_timesteps 4015.
Path 256 | total_timesteps 4033.
Path 257 | total_timesteps 4056.
Path 258 | total_timesteps 4072.
Path 259 | total_timesteps 4085.
Path 260 | total_timesteps 4096.
Path 261 | total_timesteps 4116.
Path 262 | total_timesteps 4135.
Path 263 | total_timesteps 4148.
Path 264 | total_timesteps 4170.
Path 265 | total_timesteps 4183.
Path 266 | total_timesteps 4195.
Path 267 | total_timesteps 4211.
Path 268 | total_timesteps 4225.
Path 269 | total_timesteps 4234.
Path 270 | total_timesteps 4242.
Path 271 | total_timesteps 4250.
Path 272 | total_timesteps 4263.
Path 273 | total_timesteps 4278.
Path 274 | total_timesteps 4290.
Path 275 | total_timesteps 4301.
Path 276 | total_timesteps 4314.
Path 277 | total_timesteps 4324.
Path 278 | total_timesteps 4343.
Path 279 | total_timesteps 4360.
Path 280 | total_timesteps 4370.
Path 281 | total_timesteps 4379.
Path 282 | total_timesteps 4393.
Path 283 | total_timesteps 4405.
Path 284 | total_timesteps 4430.
Path 285 | total_timesteps 4447.
Path 286 | total_timesteps 4464.
Path 287 | total_timesteps 4475.
Path 288 | total_timesteps 4488.
Path 289 | total_timesteps 4501.
Path 290 | total_timesteps 4516.
Path 291 | total_timesteps 4526.
Path 292 | total_timesteps 4547.
Path 293 | total_timesteps 4559.
Path 294 | total_timesteps 4577.
Path 295 | total_timesteps 4586.
Path 296 | total_timesteps 4606.
Path 297 | total_timesteps 4619.
Path 298 | total_timesteps 4642.
Path 299 | total_timesteps 4655.
Path 300 | total_timesteps 4674.
Path 301 | total_timesteps 4694.
Path 302 | total_timesteps 4707.
Path 303 | total_timesteps 4719.
Path 304 | total_timesteps 4738.
Path 305 | total_timesteps 4761.
Path 306 | total_timesteps 4774.
Path 307 | total_timesteps 4789.
Path 308 | total_timesteps 4798.
Path 309 | total_timesteps 4818.
Path 310 | total_timesteps 4830.
Path 311 | total_timesteps 4844.
Path 312 | total_timesteps 4863.
Path 313 | total_timesteps 4875.
Path 314 | total_timesteps 4894.
Path 315 | total_timesteps 4913.
Path 316 | total_timesteps 4924.
Path 317 | total_timesteps 4941.
Path 318 | total_timesteps 4962.
Path 319 | total_timesteps 4985.
Path 320 | total_timesteps 5005.
Path 321 | total_timesteps 5024.
Path 322 | total_timesteps 5045.
Path 323 | total_timesteps 5057.
Path 324 | total_timesteps 5065.
Path 325 | total_timesteps 5095.
Path 326 | total_timesteps 5115.
Path 327 | total_timesteps 5132.
Path 328 | total_timesteps 5156.
Path 329 | total_timesteps 5178.
Path 330 | total_timesteps 5192.
Path 331 | total_timesteps 5221.
Path 332 | total_timesteps 5228.
Path 333 | total_timesteps 5241.
Path 334 | total_timesteps 5260.
Path 335 | total_timesteps 5276.
Path 336 | total_timesteps 5291.
Path 337 | total_timesteps 5316.
Path 338 | total_timesteps 5342.
Path 339 | total_timesteps 5351.
Path 340 | total_timesteps 5361.
Path 341 | total_timesteps 5371.
Path 342 | total_timesteps 5390.
Path 343 | total_timesteps 5401.
Path 344 | total_timesteps 5413.
Path 345 | total_timesteps 5424.
Path 346 | total_timesteps 5441.
Path 347 | total_timesteps 5451.
Path 348 | total_timesteps 5464.
Path 349 | total_timesteps 5475.
Path 350 | total_timesteps 5485.
Path 351 | total_timesteps 5498.
Path 352 | total_timesteps 5527.
Path 353 | total_timesteps 5543.
Path 354 | total_timesteps 5559.
Path 355 | total_timesteps 5574.
Path 356 | total_timesteps 5594.
Path 357 | total_timesteps 5604.
Path 358 | total_timesteps 5616.
Path 359 | total_timesteps 5632.
Path 360 | total_timesteps 5642.
Path 361 | total_timesteps 5656.
Path 362 | total_timesteps 5667.
Path 363 | total_timesteps 5684.
Path 364 | total_timesteps 5702.
Path 365 | total_timesteps 5716.
Path 366 | total_timesteps 5739.
Path 367 | total_timesteps 5749.
Path 368 | total_timesteps 5760.
Path 369 | total_timesteps 5780.
Path 370 | total_timesteps 5801.
Path 371 | total_timesteps 5811.
Path 372 | total_timesteps 5829.
Path 373 | total_timesteps 5843.
Path 374 | total_timesteps 5868.
Path 375 | total_timesteps 5881.
Path 376 | total_timesteps 5892.
Path 377 | total_timesteps 5905.
Path 378 | total_timesteps 5919.
Path 379 | total_timesteps 5934.
Path 380 | total_timesteps 5942.
Path 381 | total_timesteps 5951.
Path 382 | total_timesteps 5972.
Path 383 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.17    |
| Iteration     | 7        |
| MaximumReturn | 0.981    |
| MinimumReturn | -20      |
| TotalSamples  | 36042    |
----------------------------
itr #8 | 
Fitting dynamics.
Validation loss = 0.013500047847628593
Validation loss = 0.012009180150926113
Validation loss = 0.0116634052246809
Validation loss = 0.012457172386348248
Validation loss = 0.012943913228809834
Validation loss = 0.012649663724005222
Validation loss = 0.01172146387398243
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 65.
Path 4 | total_timesteps 79.
Path 5 | total_timesteps 88.
Path 6 | total_timesteps 99.
Path 7 | total_timesteps 117.
Path 8 | total_timesteps 130.
Path 9 | total_timesteps 144.
Path 10 | total_timesteps 159.
Path 11 | total_timesteps 174.
Path 12 | total_timesteps 192.
Path 13 | total_timesteps 208.
Path 14 | total_timesteps 224.
Path 15 | total_timesteps 238.
Path 16 | total_timesteps 254.
Path 17 | total_timesteps 276.
Path 18 | total_timesteps 292.
Path 19 | total_timesteps 320.
Path 20 | total_timesteps 341.
Path 21 | total_timesteps 355.
Path 22 | total_timesteps 376.
Path 23 | total_timesteps 387.
Path 24 | total_timesteps 409.
Path 25 | total_timesteps 424.
Path 26 | total_timesteps 439.
Path 27 | total_timesteps 453.
Path 28 | total_timesteps 465.
Path 29 | total_timesteps 471.
Path 30 | total_timesteps 484.
Path 31 | total_timesteps 498.
Path 32 | total_timesteps 523.
Path 33 | total_timesteps 534.
Path 34 | total_timesteps 550.
Path 35 | total_timesteps 567.
Path 36 | total_timesteps 588.
Path 37 | total_timesteps 597.
Path 38 | total_timesteps 622.
Path 39 | total_timesteps 633.
Path 40 | total_timesteps 644.
Path 41 | total_timesteps 652.
Path 42 | total_timesteps 660.
Path 43 | total_timesteps 677.
Path 44 | total_timesteps 687.
Path 45 | total_timesteps 698.
Path 46 | total_timesteps 713.
Path 47 | total_timesteps 722.
Path 48 | total_timesteps 749.
Path 49 | total_timesteps 773.
Path 50 | total_timesteps 791.
Path 51 | total_timesteps 799.
Path 52 | total_timesteps 827.
Path 53 | total_timesteps 844.
Path 54 | total_timesteps 853.
Path 55 | total_timesteps 861.
Path 56 | total_timesteps 870.
Path 57 | total_timesteps 888.
Path 58 | total_timesteps 898.
Path 59 | total_timesteps 918.
Path 60 | total_timesteps 927.
Path 61 | total_timesteps 937.
Path 62 | total_timesteps 957.
Path 63 | total_timesteps 972.
Path 64 | total_timesteps 989.
Path 65 | total_timesteps 1001.
Path 66 | total_timesteps 1017.
Path 67 | total_timesteps 1031.
Path 68 | total_timesteps 1060.
Path 69 | total_timesteps 1070.
Path 70 | total_timesteps 1079.
Path 71 | total_timesteps 1090.
Path 72 | total_timesteps 1108.
Path 73 | total_timesteps 1125.
Path 74 | total_timesteps 1132.
Path 75 | total_timesteps 1149.
Path 76 | total_timesteps 1167.
Path 77 | total_timesteps 1187.
Path 78 | total_timesteps 1199.
Path 79 | total_timesteps 1215.
Path 80 | total_timesteps 1228.
Path 81 | total_timesteps 1243.
Path 82 | total_timesteps 1255.
Path 83 | total_timesteps 1272.
Path 84 | total_timesteps 1285.
Path 85 | total_timesteps 1300.
Path 86 | total_timesteps 1314.
Path 87 | total_timesteps 1328.
Path 88 | total_timesteps 1348.
Path 89 | total_timesteps 1364.
Path 90 | total_timesteps 1374.
Path 91 | total_timesteps 1393.
Path 92 | total_timesteps 1408.
Path 93 | total_timesteps 1419.
Path 94 | total_timesteps 1441.
Path 95 | total_timesteps 1456.
Path 96 | total_timesteps 1470.
Path 97 | total_timesteps 1481.
Path 98 | total_timesteps 1495.
Path 99 | total_timesteps 1504.
Path 100 | total_timesteps 1525.
Path 101 | total_timesteps 1538.
Path 102 | total_timesteps 1548.
Path 103 | total_timesteps 1561.
Path 104 | total_timesteps 1585.
Path 105 | total_timesteps 1609.
Path 106 | total_timesteps 1619.
Path 107 | total_timesteps 1633.
Path 108 | total_timesteps 1647.
Path 109 | total_timesteps 1662.
Path 110 | total_timesteps 1682.
Path 111 | total_timesteps 1693.
Path 112 | total_timesteps 1706.
Path 113 | total_timesteps 1719.
Path 114 | total_timesteps 1738.
Path 115 | total_timesteps 1749.
Path 116 | total_timesteps 1762.
Path 117 | total_timesteps 1775.
Path 118 | total_timesteps 1794.
Path 119 | total_timesteps 1804.
Path 120 | total_timesteps 1827.
Path 121 | total_timesteps 1835.
Path 122 | total_timesteps 1855.
Path 123 | total_timesteps 1868.
Path 124 | total_timesteps 1891.
Path 125 | total_timesteps 1907.
Path 126 | total_timesteps 1925.
Path 127 | total_timesteps 1939.
Path 128 | total_timesteps 1949.
Path 129 | total_timesteps 1971.
Path 130 | total_timesteps 1990.
Path 131 | total_timesteps 2002.
Path 132 | total_timesteps 2014.
Path 133 | total_timesteps 2024.
Path 134 | total_timesteps 2041.
Path 135 | total_timesteps 2051.
Path 136 | total_timesteps 2062.
Path 137 | total_timesteps 2078.
Path 138 | total_timesteps 2094.
Path 139 | total_timesteps 2109.
Path 140 | total_timesteps 2132.
Path 141 | total_timesteps 2163.
Path 142 | total_timesteps 2182.
Path 143 | total_timesteps 2195.
Path 144 | total_timesteps 2205.
Path 145 | total_timesteps 2222.
Path 146 | total_timesteps 2230.
Path 147 | total_timesteps 2242.
Path 148 | total_timesteps 2258.
Path 149 | total_timesteps 2268.
Path 150 | total_timesteps 2282.
Path 151 | total_timesteps 2295.
Path 152 | total_timesteps 2306.
Path 153 | total_timesteps 2318.
Path 154 | total_timesteps 2331.
Path 155 | total_timesteps 2344.
Path 156 | total_timesteps 2355.
Path 157 | total_timesteps 2368.
Path 158 | total_timesteps 2384.
Path 159 | total_timesteps 2404.
Path 160 | total_timesteps 2414.
Path 161 | total_timesteps 2422.
Path 162 | total_timesteps 2440.
Path 163 | total_timesteps 2450.
Path 164 | total_timesteps 2462.
Path 165 | total_timesteps 2488.
Path 166 | total_timesteps 2506.
Path 167 | total_timesteps 2525.
Path 168 | total_timesteps 2549.
Path 169 | total_timesteps 2570.
Path 170 | total_timesteps 2587.
Path 171 | total_timesteps 2604.
Path 172 | total_timesteps 2617.
Path 173 | total_timesteps 2640.
Path 174 | total_timesteps 2653.
Path 175 | total_timesteps 2662.
Path 176 | total_timesteps 2674.
Path 177 | total_timesteps 2688.
Path 178 | total_timesteps 2703.
Path 179 | total_timesteps 2714.
Path 180 | total_timesteps 2726.
Path 181 | total_timesteps 2740.
Path 182 | total_timesteps 2762.
Path 183 | total_timesteps 2780.
Path 184 | total_timesteps 2802.
Path 185 | total_timesteps 2810.
Path 186 | total_timesteps 2821.
Path 187 | total_timesteps 2847.
Path 188 | total_timesteps 2866.
Path 189 | total_timesteps 2877.
Path 190 | total_timesteps 2885.
Path 191 | total_timesteps 2902.
Path 192 | total_timesteps 2915.
Path 193 | total_timesteps 2924.
Path 194 | total_timesteps 2932.
Path 195 | total_timesteps 2949.
Path 196 | total_timesteps 2958.
Path 197 | total_timesteps 2968.
Path 198 | total_timesteps 2983.
Path 199 | total_timesteps 2995.
Path 200 | total_timesteps 3009.
Path 201 | total_timesteps 3029.
Path 202 | total_timesteps 3046.
Path 203 | total_timesteps 3058.
Path 204 | total_timesteps 3086.
Path 205 | total_timesteps 3097.
Path 206 | total_timesteps 3111.
Path 207 | total_timesteps 3123.
Path 208 | total_timesteps 3132.
Path 209 | total_timesteps 3143.
Path 210 | total_timesteps 3161.
Path 211 | total_timesteps 3170.
Path 212 | total_timesteps 3182.
Path 213 | total_timesteps 3191.
Path 214 | total_timesteps 3202.
Path 215 | total_timesteps 3211.
Path 216 | total_timesteps 3222.
Path 217 | total_timesteps 3243.
Path 218 | total_timesteps 3250.
Path 219 | total_timesteps 3272.
Path 220 | total_timesteps 3291.
Path 221 | total_timesteps 3302.
Path 222 | total_timesteps 3334.
Path 223 | total_timesteps 3347.
Path 224 | total_timesteps 3373.
Path 225 | total_timesteps 3385.
Path 226 | total_timesteps 3397.
Path 227 | total_timesteps 3414.
Path 228 | total_timesteps 3424.
Path 229 | total_timesteps 3435.
Path 230 | total_timesteps 3447.
Path 231 | total_timesteps 3465.
Path 232 | total_timesteps 3475.
Path 233 | total_timesteps 3487.
Path 234 | total_timesteps 3501.
Path 235 | total_timesteps 3518.
Path 236 | total_timesteps 3537.
Path 237 | total_timesteps 3549.
Path 238 | total_timesteps 3568.
Path 239 | total_timesteps 3586.
Path 240 | total_timesteps 3598.
Path 241 | total_timesteps 3621.
Path 242 | total_timesteps 3633.
Path 243 | total_timesteps 3648.
Path 244 | total_timesteps 3660.
Path 245 | total_timesteps 3676.
Path 246 | total_timesteps 3696.
Path 247 | total_timesteps 3725.
Path 248 | total_timesteps 3737.
Path 249 | total_timesteps 3754.
Path 250 | total_timesteps 3775.
Path 251 | total_timesteps 3796.
Path 252 | total_timesteps 3809.
Path 253 | total_timesteps 3822.
Path 254 | total_timesteps 3837.
Path 255 | total_timesteps 3849.
Path 256 | total_timesteps 3860.
Path 257 | total_timesteps 3881.
Path 258 | total_timesteps 3893.
Path 259 | total_timesteps 3910.
Path 260 | total_timesteps 3928.
Path 261 | total_timesteps 3943.
Path 262 | total_timesteps 3956.
Path 263 | total_timesteps 3979.
Path 264 | total_timesteps 3988.
Path 265 | total_timesteps 4001.
Path 266 | total_timesteps 4012.
Path 267 | total_timesteps 4035.
Path 268 | total_timesteps 4045.
Path 269 | total_timesteps 4058.
Path 270 | total_timesteps 4069.
Path 271 | total_timesteps 4082.
Path 272 | total_timesteps 4094.
Path 273 | total_timesteps 4107.
Path 274 | total_timesteps 4124.
Path 275 | total_timesteps 4138.
Path 276 | total_timesteps 4150.
Path 277 | total_timesteps 4174.
Path 278 | total_timesteps 4185.
Path 279 | total_timesteps 4195.
Path 280 | total_timesteps 4209.
Path 281 | total_timesteps 4222.
Path 282 | total_timesteps 4240.
Path 283 | total_timesteps 4261.
Path 284 | total_timesteps 4271.
Path 285 | total_timesteps 4289.
Path 286 | total_timesteps 4304.
Path 287 | total_timesteps 4323.
Path 288 | total_timesteps 4332.
Path 289 | total_timesteps 4350.
Path 290 | total_timesteps 4370.
Path 291 | total_timesteps 4384.
Path 292 | total_timesteps 4394.
Path 293 | total_timesteps 4408.
Path 294 | total_timesteps 4426.
Path 295 | total_timesteps 4443.
Path 296 | total_timesteps 4461.
Path 297 | total_timesteps 4472.
Path 298 | total_timesteps 4492.
Path 299 | total_timesteps 4501.
Path 300 | total_timesteps 4514.
Path 301 | total_timesteps 4525.
Path 302 | total_timesteps 4557.
Path 303 | total_timesteps 4580.
Path 304 | total_timesteps 4590.
Path 305 | total_timesteps 4604.
Path 306 | total_timesteps 4636.
Path 307 | total_timesteps 4658.
Path 308 | total_timesteps 4672.
Path 309 | total_timesteps 4682.
Path 310 | total_timesteps 4693.
Path 311 | total_timesteps 4711.
Path 312 | total_timesteps 4731.
Path 313 | total_timesteps 4747.
Path 314 | total_timesteps 4759.
Path 315 | total_timesteps 4792.
Path 316 | total_timesteps 4804.
Path 317 | total_timesteps 4816.
Path 318 | total_timesteps 4831.
Path 319 | total_timesteps 4857.
Path 320 | total_timesteps 4870.
Path 321 | total_timesteps 4886.
Path 322 | total_timesteps 4915.
Path 323 | total_timesteps 4934.
Path 324 | total_timesteps 4949.
Path 325 | total_timesteps 4964.
Path 326 | total_timesteps 4978.
Path 327 | total_timesteps 4988.
Path 328 | total_timesteps 4997.
Path 329 | total_timesteps 5008.
Path 330 | total_timesteps 5023.
Path 331 | total_timesteps 5038.
Path 332 | total_timesteps 5048.
Path 333 | total_timesteps 5064.
Path 334 | total_timesteps 5090.
Path 335 | total_timesteps 5104.
Path 336 | total_timesteps 5125.
Path 337 | total_timesteps 5136.
Path 338 | total_timesteps 5151.
Path 339 | total_timesteps 5166.
Path 340 | total_timesteps 5178.
Path 341 | total_timesteps 5188.
Path 342 | total_timesteps 5199.
Path 343 | total_timesteps 5222.
Path 344 | total_timesteps 5236.
Path 345 | total_timesteps 5255.
Path 346 | total_timesteps 5275.
Path 347 | total_timesteps 5290.
Path 348 | total_timesteps 5301.
Path 349 | total_timesteps 5315.
Path 350 | total_timesteps 5325.
Path 351 | total_timesteps 5341.
Path 352 | total_timesteps 5356.
Path 353 | total_timesteps 5371.
Path 354 | total_timesteps 5382.
Path 355 | total_timesteps 5406.
Path 356 | total_timesteps 5418.
Path 357 | total_timesteps 5440.
Path 358 | total_timesteps 5458.
Path 359 | total_timesteps 5474.
Path 360 | total_timesteps 5484.
Path 361 | total_timesteps 5507.
Path 362 | total_timesteps 5522.
Path 363 | total_timesteps 5543.
Path 364 | total_timesteps 5559.
Path 365 | total_timesteps 5577.
Path 366 | total_timesteps 5604.
Path 367 | total_timesteps 5623.
Path 368 | total_timesteps 5638.
Path 369 | total_timesteps 5650.
Path 370 | total_timesteps 5662.
Path 371 | total_timesteps 5685.
Path 372 | total_timesteps 5698.
Path 373 | total_timesteps 5714.
Path 374 | total_timesteps 5726.
Path 375 | total_timesteps 5741.
Path 376 | total_timesteps 5750.
Path 377 | total_timesteps 5776.
Path 378 | total_timesteps 5796.
Path 379 | total_timesteps 5812.
Path 380 | total_timesteps 5820.
Path 381 | total_timesteps 5836.
Path 382 | total_timesteps 5848.
Path 383 | total_timesteps 5863.
Path 384 | total_timesteps 5881.
Path 385 | total_timesteps 5896.
Path 386 | total_timesteps 5912.
Path 387 | total_timesteps 5924.
Path 388 | total_timesteps 5941.
Path 389 | total_timesteps 5960.
Path 390 | total_timesteps 5973.
Path 391 | total_timesteps 5985.
Path 392 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.8     |
| Iteration     | 8        |
| MaximumReturn | 0.364    |
| MinimumReturn | -23.2    |
| TotalSamples  | 40046    |
----------------------------
itr #9 | 
Fitting dynamics.
Validation loss = 0.012525762431323528
Validation loss = 0.01225954107940197
Validation loss = 0.012038620188832283
Validation loss = 0.010728590190410614
Validation loss = 0.011782074347138405
Validation loss = 0.01152639277279377
Validation loss = 0.011513909325003624
Validation loss = 0.011509221978485584
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 32.
Path 2 | total_timesteps 42.
Path 3 | total_timesteps 57.
Path 4 | total_timesteps 72.
Path 5 | total_timesteps 89.
Path 6 | total_timesteps 99.
Path 7 | total_timesteps 132.
Path 8 | total_timesteps 154.
Path 9 | total_timesteps 164.
Path 10 | total_timesteps 181.
Path 11 | total_timesteps 193.
Path 12 | total_timesteps 205.
Path 13 | total_timesteps 221.
Path 14 | total_timesteps 237.
Path 15 | total_timesteps 266.
Path 16 | total_timesteps 276.
Path 17 | total_timesteps 295.
Path 18 | total_timesteps 310.
Path 19 | total_timesteps 324.
Path 20 | total_timesteps 346.
Path 21 | total_timesteps 353.
Path 22 | total_timesteps 378.
Path 23 | total_timesteps 389.
Path 24 | total_timesteps 406.
Path 25 | total_timesteps 420.
Path 26 | total_timesteps 432.
Path 27 | total_timesteps 443.
Path 28 | total_timesteps 451.
Path 29 | total_timesteps 475.
Path 30 | total_timesteps 492.
Path 31 | total_timesteps 515.
Path 32 | total_timesteps 525.
Path 33 | total_timesteps 539.
Path 34 | total_timesteps 559.
Path 35 | total_timesteps 572.
Path 36 | total_timesteps 588.
Path 37 | total_timesteps 600.
Path 38 | total_timesteps 611.
Path 39 | total_timesteps 632.
Path 40 | total_timesteps 646.
Path 41 | total_timesteps 658.
Path 42 | total_timesteps 672.
Path 43 | total_timesteps 680.
Path 44 | total_timesteps 699.
Path 45 | total_timesteps 708.
Path 46 | total_timesteps 721.
Path 47 | total_timesteps 735.
Path 48 | total_timesteps 749.
Path 49 | total_timesteps 760.
Path 50 | total_timesteps 784.
Path 51 | total_timesteps 792.
Path 52 | total_timesteps 809.
Path 53 | total_timesteps 819.
Path 54 | total_timesteps 831.
Path 55 | total_timesteps 847.
Path 56 | total_timesteps 861.
Path 57 | total_timesteps 871.
Path 58 | total_timesteps 893.
Path 59 | total_timesteps 910.
Path 60 | total_timesteps 919.
Path 61 | total_timesteps 942.
Path 62 | total_timesteps 953.
Path 63 | total_timesteps 969.
Path 64 | total_timesteps 979.
Path 65 | total_timesteps 999.
Path 66 | total_timesteps 1015.
Path 67 | total_timesteps 1028.
Path 68 | total_timesteps 1052.
Path 69 | total_timesteps 1063.
Path 70 | total_timesteps 1088.
Path 71 | total_timesteps 1117.
Path 72 | total_timesteps 1135.
Path 73 | total_timesteps 1166.
Path 74 | total_timesteps 1177.
Path 75 | total_timesteps 1191.
Path 76 | total_timesteps 1202.
Path 77 | total_timesteps 1210.
Path 78 | total_timesteps 1225.
Path 79 | total_timesteps 1236.
Path 80 | total_timesteps 1252.
Path 81 | total_timesteps 1266.
Path 82 | total_timesteps 1277.
Path 83 | total_timesteps 1294.
Path 84 | total_timesteps 1323.
Path 85 | total_timesteps 1334.
Path 86 | total_timesteps 1344.
Path 87 | total_timesteps 1366.
Path 88 | total_timesteps 1380.
Path 89 | total_timesteps 1397.
Path 90 | total_timesteps 1415.
Path 91 | total_timesteps 1433.
Path 92 | total_timesteps 1461.
Path 93 | total_timesteps 1476.
Path 94 | total_timesteps 1498.
Path 95 | total_timesteps 1519.
Path 96 | total_timesteps 1535.
Path 97 | total_timesteps 1553.
Path 98 | total_timesteps 1567.
Path 99 | total_timesteps 1582.
Path 100 | total_timesteps 1595.
Path 101 | total_timesteps 1606.
Path 102 | total_timesteps 1632.
Path 103 | total_timesteps 1659.
Path 104 | total_timesteps 1683.
Path 105 | total_timesteps 1713.
Path 106 | total_timesteps 1722.
Path 107 | total_timesteps 1740.
Path 108 | total_timesteps 1752.
Path 109 | total_timesteps 1761.
Path 110 | total_timesteps 1780.
Path 111 | total_timesteps 1804.
Path 112 | total_timesteps 1818.
Path 113 | total_timesteps 1835.
Path 114 | total_timesteps 1849.
Path 115 | total_timesteps 1864.
Path 116 | total_timesteps 1883.
Path 117 | total_timesteps 1896.
Path 118 | total_timesteps 1916.
Path 119 | total_timesteps 1926.
Path 120 | total_timesteps 1934.
Path 121 | total_timesteps 1954.
Path 122 | total_timesteps 1962.
Path 123 | total_timesteps 1984.
Path 124 | total_timesteps 1998.
Path 125 | total_timesteps 2015.
Path 126 | total_timesteps 2031.
Path 127 | total_timesteps 2051.
Path 128 | total_timesteps 2063.
Path 129 | total_timesteps 2074.
Path 130 | total_timesteps 2085.
Path 131 | total_timesteps 2097.
Path 132 | total_timesteps 2116.
Path 133 | total_timesteps 2131.
Path 134 | total_timesteps 2154.
Path 135 | total_timesteps 2162.
Path 136 | total_timesteps 2177.
Path 137 | total_timesteps 2185.
Path 138 | total_timesteps 2202.
Path 139 | total_timesteps 2212.
Path 140 | total_timesteps 2220.
Path 141 | total_timesteps 2238.
Path 142 | total_timesteps 2247.
Path 143 | total_timesteps 2260.
Path 144 | total_timesteps 2276.
Path 145 | total_timesteps 2293.
Path 146 | total_timesteps 2315.
Path 147 | total_timesteps 2338.
Path 148 | total_timesteps 2356.
Path 149 | total_timesteps 2377.
Path 150 | total_timesteps 2388.
Path 151 | total_timesteps 2402.
Path 152 | total_timesteps 2415.
Path 153 | total_timesteps 2439.
Path 154 | total_timesteps 2452.
Path 155 | total_timesteps 2469.
Path 156 | total_timesteps 2485.
Path 157 | total_timesteps 2497.
Path 158 | total_timesteps 2516.
Path 159 | total_timesteps 2528.
Path 160 | total_timesteps 2546.
Path 161 | total_timesteps 2557.
Path 162 | total_timesteps 2582.
Path 163 | total_timesteps 2602.
Path 164 | total_timesteps 2621.
Path 165 | total_timesteps 2632.
Path 166 | total_timesteps 2642.
Path 167 | total_timesteps 2649.
Path 168 | total_timesteps 2667.
Path 169 | total_timesteps 2677.
Path 170 | total_timesteps 2693.
Path 171 | total_timesteps 2711.
Path 172 | total_timesteps 2723.
Path 173 | total_timesteps 2734.
Path 174 | total_timesteps 2746.
Path 175 | total_timesteps 2759.
Path 176 | total_timesteps 2775.
Path 177 | total_timesteps 2795.
Path 178 | total_timesteps 2828.
Path 179 | total_timesteps 2843.
Path 180 | total_timesteps 2862.
Path 181 | total_timesteps 2886.
Path 182 | total_timesteps 2901.
Path 183 | total_timesteps 2910.
Path 184 | total_timesteps 2931.
Path 185 | total_timesteps 2952.
Path 186 | total_timesteps 2968.
Path 187 | total_timesteps 2980.
Path 188 | total_timesteps 2995.
Path 189 | total_timesteps 3023.
Path 190 | total_timesteps 3033.
Path 191 | total_timesteps 3050.
Path 192 | total_timesteps 3064.
Path 193 | total_timesteps 3076.
Path 194 | total_timesteps 3092.
Path 195 | total_timesteps 3104.
Path 196 | total_timesteps 3120.
Path 197 | total_timesteps 3135.
Path 198 | total_timesteps 3148.
Path 199 | total_timesteps 3157.
Path 200 | total_timesteps 3167.
Path 201 | total_timesteps 3179.
Path 202 | total_timesteps 3188.
Path 203 | total_timesteps 3199.
Path 204 | total_timesteps 3214.
Path 205 | total_timesteps 3228.
Path 206 | total_timesteps 3239.
Path 207 | total_timesteps 3251.
Path 208 | total_timesteps 3267.
Path 209 | total_timesteps 3282.
Path 210 | total_timesteps 3296.
Path 211 | total_timesteps 3312.
Path 212 | total_timesteps 3329.
Path 213 | total_timesteps 3345.
Path 214 | total_timesteps 3366.
Path 215 | total_timesteps 3381.
Path 216 | total_timesteps 3391.
Path 217 | total_timesteps 3399.
Path 218 | total_timesteps 3415.
Path 219 | total_timesteps 3433.
Path 220 | total_timesteps 3472.
Path 221 | total_timesteps 3490.
Path 222 | total_timesteps 3504.
Path 223 | total_timesteps 3520.
Path 224 | total_timesteps 3532.
Path 225 | total_timesteps 3542.
Path 226 | total_timesteps 3565.
Path 227 | total_timesteps 3573.
Path 228 | total_timesteps 3591.
Path 229 | total_timesteps 3602.
Path 230 | total_timesteps 3621.
Path 231 | total_timesteps 3631.
Path 232 | total_timesteps 3642.
Path 233 | total_timesteps 3663.
Path 234 | total_timesteps 3671.
Path 235 | total_timesteps 3691.
Path 236 | total_timesteps 3700.
Path 237 | total_timesteps 3732.
Path 238 | total_timesteps 3745.
Path 239 | total_timesteps 3763.
Path 240 | total_timesteps 3787.
Path 241 | total_timesteps 3804.
Path 242 | total_timesteps 3816.
Path 243 | total_timesteps 3835.
Path 244 | total_timesteps 3857.
Path 245 | total_timesteps 3876.
Path 246 | total_timesteps 3889.
Path 247 | total_timesteps 3905.
Path 248 | total_timesteps 3918.
Path 249 | total_timesteps 3930.
Path 250 | total_timesteps 3957.
Path 251 | total_timesteps 3966.
Path 252 | total_timesteps 3979.
Path 253 | total_timesteps 3993.
Path 254 | total_timesteps 4001.
Path 255 | total_timesteps 4012.
Path 256 | total_timesteps 4022.
Path 257 | total_timesteps 4037.
Path 258 | total_timesteps 4047.
Path 259 | total_timesteps 4066.
Path 260 | total_timesteps 4083.
Path 261 | total_timesteps 4095.
Path 262 | total_timesteps 4105.
Path 263 | total_timesteps 4113.
Path 264 | total_timesteps 4131.
Path 265 | total_timesteps 4146.
Path 266 | total_timesteps 4167.
Path 267 | total_timesteps 4179.
Path 268 | total_timesteps 4189.
Path 269 | total_timesteps 4206.
Path 270 | total_timesteps 4221.
Path 271 | total_timesteps 4233.
Path 272 | total_timesteps 4250.
Path 273 | total_timesteps 4282.
Path 274 | total_timesteps 4296.
Path 275 | total_timesteps 4305.
Path 276 | total_timesteps 4323.
Path 277 | total_timesteps 4337.
Path 278 | total_timesteps 4348.
Path 279 | total_timesteps 4360.
Path 280 | total_timesteps 4387.
Path 281 | total_timesteps 4407.
Path 282 | total_timesteps 4429.
Path 283 | total_timesteps 4469.
Path 284 | total_timesteps 4480.
Path 285 | total_timesteps 4491.
Path 286 | total_timesteps 4508.
Path 287 | total_timesteps 4519.
Path 288 | total_timesteps 4542.
Path 289 | total_timesteps 4554.
Path 290 | total_timesteps 4570.
Path 291 | total_timesteps 4586.
Path 292 | total_timesteps 4604.
Path 293 | total_timesteps 4626.
Path 294 | total_timesteps 4637.
Path 295 | total_timesteps 4650.
Path 296 | total_timesteps 4665.
Path 297 | total_timesteps 4682.
Path 298 | total_timesteps 4706.
Path 299 | total_timesteps 4719.
Path 300 | total_timesteps 4732.
Path 301 | total_timesteps 4753.
Path 302 | total_timesteps 4770.
Path 303 | total_timesteps 4782.
Path 304 | total_timesteps 4798.
Path 305 | total_timesteps 4806.
Path 306 | total_timesteps 4818.
Path 307 | total_timesteps 4845.
Path 308 | total_timesteps 4858.
Path 309 | total_timesteps 4868.
Path 310 | total_timesteps 4886.
Path 311 | total_timesteps 4900.
Path 312 | total_timesteps 4924.
Path 313 | total_timesteps 4933.
Path 314 | total_timesteps 4949.
Path 315 | total_timesteps 4958.
Path 316 | total_timesteps 4967.
Path 317 | total_timesteps 4980.
Path 318 | total_timesteps 4999.
Path 319 | total_timesteps 5017.
Path 320 | total_timesteps 5043.
Path 321 | total_timesteps 5057.
Path 322 | total_timesteps 5085.
Path 323 | total_timesteps 5100.
Path 324 | total_timesteps 5110.
Path 325 | total_timesteps 5120.
Path 326 | total_timesteps 5127.
Path 327 | total_timesteps 5139.
Path 328 | total_timesteps 5154.
Path 329 | total_timesteps 5170.
Path 330 | total_timesteps 5183.
Path 331 | total_timesteps 5196.
Path 332 | total_timesteps 5208.
Path 333 | total_timesteps 5223.
Path 334 | total_timesteps 5241.
Path 335 | total_timesteps 5254.
Path 336 | total_timesteps 5265.
Path 337 | total_timesteps 5294.
Path 338 | total_timesteps 5309.
Path 339 | total_timesteps 5326.
Path 340 | total_timesteps 5343.
Path 341 | total_timesteps 5356.
Path 342 | total_timesteps 5379.
Path 343 | total_timesteps 5407.
Path 344 | total_timesteps 5422.
Path 345 | total_timesteps 5436.
Path 346 | total_timesteps 5447.
Path 347 | total_timesteps 5460.
Path 348 | total_timesteps 5476.
Path 349 | total_timesteps 5487.
Path 350 | total_timesteps 5505.
Path 351 | total_timesteps 5528.
Path 352 | total_timesteps 5541.
Path 353 | total_timesteps 5553.
Path 354 | total_timesteps 5572.
Path 355 | total_timesteps 5583.
Path 356 | total_timesteps 5599.
Path 357 | total_timesteps 5625.
Path 358 | total_timesteps 5636.
Path 359 | total_timesteps 5644.
Path 360 | total_timesteps 5662.
Path 361 | total_timesteps 5675.
Path 362 | total_timesteps 5683.
Path 363 | total_timesteps 5701.
Path 364 | total_timesteps 5715.
Path 365 | total_timesteps 5724.
Path 366 | total_timesteps 5750.
Path 367 | total_timesteps 5763.
Path 368 | total_timesteps 5778.
Path 369 | total_timesteps 5793.
Path 370 | total_timesteps 5808.
Path 371 | total_timesteps 5823.
Path 372 | total_timesteps 5838.
Path 373 | total_timesteps 5853.
Path 374 | total_timesteps 5864.
Path 375 | total_timesteps 5873.
Path 376 | total_timesteps 5887.
Path 377 | total_timesteps 5901.
Path 378 | total_timesteps 5911.
Path 379 | total_timesteps 5929.
Path 380 | total_timesteps 5943.
Path 381 | total_timesteps 5954.
Path 382 | total_timesteps 5971.
Path 383 | total_timesteps 5981.
Path 384 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.58    |
| Iteration     | 9        |
| MaximumReturn | 0.515    |
| MinimumReturn | -19.7    |
| TotalSamples  | 44052    |
----------------------------
itr #10 | 
Fitting dynamics.
Validation loss = 0.013200298883020878
Validation loss = 0.010363698936998844
Validation loss = 0.010425318963825703
Validation loss = 0.014411208219826221
Validation loss = 0.011632637120783329
Validation loss = 0.010522655211389065
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 34.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 59.
Path 5 | total_timesteps 68.
Path 6 | total_timesteps 78.
Path 7 | total_timesteps 98.
Path 8 | total_timesteps 115.
Path 9 | total_timesteps 125.
Path 10 | total_timesteps 136.
Path 11 | total_timesteps 158.
Path 12 | total_timesteps 175.
Path 13 | total_timesteps 194.
Path 14 | total_timesteps 212.
Path 15 | total_timesteps 220.
Path 16 | total_timesteps 241.
Path 17 | total_timesteps 256.
Path 18 | total_timesteps 278.
Path 19 | total_timesteps 301.
Path 20 | total_timesteps 314.
Path 21 | total_timesteps 334.
Path 22 | total_timesteps 352.
Path 23 | total_timesteps 366.
Path 24 | total_timesteps 380.
Path 25 | total_timesteps 401.
Path 26 | total_timesteps 416.
Path 27 | total_timesteps 424.
Path 28 | total_timesteps 436.
Path 29 | total_timesteps 445.
Path 30 | total_timesteps 468.
Path 31 | total_timesteps 478.
Path 32 | total_timesteps 489.
Path 33 | total_timesteps 513.
Path 34 | total_timesteps 527.
Path 35 | total_timesteps 543.
Path 36 | total_timesteps 562.
Path 37 | total_timesteps 586.
Path 38 | total_timesteps 606.
Path 39 | total_timesteps 631.
Path 40 | total_timesteps 649.
Path 41 | total_timesteps 662.
Path 42 | total_timesteps 672.
Path 43 | total_timesteps 700.
Path 44 | total_timesteps 712.
Path 45 | total_timesteps 725.
Path 46 | total_timesteps 740.
Path 47 | total_timesteps 764.
Path 48 | total_timesteps 781.
Path 49 | total_timesteps 802.
Path 50 | total_timesteps 811.
Path 51 | total_timesteps 828.
Path 52 | total_timesteps 851.
Path 53 | total_timesteps 868.
Path 54 | total_timesteps 882.
Path 55 | total_timesteps 894.
Path 56 | total_timesteps 905.
Path 57 | total_timesteps 917.
Path 58 | total_timesteps 935.
Path 59 | total_timesteps 945.
Path 60 | total_timesteps 964.
Path 61 | total_timesteps 979.
Path 62 | total_timesteps 999.
Path 63 | total_timesteps 1021.
Path 64 | total_timesteps 1033.
Path 65 | total_timesteps 1054.
Path 66 | total_timesteps 1062.
Path 67 | total_timesteps 1079.
Path 68 | total_timesteps 1091.
Path 69 | total_timesteps 1111.
Path 70 | total_timesteps 1133.
Path 71 | total_timesteps 1148.
Path 72 | total_timesteps 1160.
Path 73 | total_timesteps 1178.
Path 74 | total_timesteps 1190.
Path 75 | total_timesteps 1212.
Path 76 | total_timesteps 1222.
Path 77 | total_timesteps 1235.
Path 78 | total_timesteps 1246.
Path 79 | total_timesteps 1259.
Path 80 | total_timesteps 1273.
Path 81 | total_timesteps 1294.
Path 82 | total_timesteps 1314.
Path 83 | total_timesteps 1328.
Path 84 | total_timesteps 1344.
Path 85 | total_timesteps 1359.
Path 86 | total_timesteps 1376.
Path 87 | total_timesteps 1392.
Path 88 | total_timesteps 1407.
Path 89 | total_timesteps 1425.
Path 90 | total_timesteps 1443.
Path 91 | total_timesteps 1453.
Path 92 | total_timesteps 1472.
Path 93 | total_timesteps 1495.
Path 94 | total_timesteps 1521.
Path 95 | total_timesteps 1532.
Path 96 | total_timesteps 1543.
Path 97 | total_timesteps 1557.
Path 98 | total_timesteps 1569.
Path 99 | total_timesteps 1588.
Path 100 | total_timesteps 1604.
Path 101 | total_timesteps 1637.
Path 102 | total_timesteps 1645.
Path 103 | total_timesteps 1664.
Path 104 | total_timesteps 1673.
Path 105 | total_timesteps 1683.
Path 106 | total_timesteps 1701.
Path 107 | total_timesteps 1715.
Path 108 | total_timesteps 1727.
Path 109 | total_timesteps 1739.
Path 110 | total_timesteps 1751.
Path 111 | total_timesteps 1763.
Path 112 | total_timesteps 1778.
Path 113 | total_timesteps 1795.
Path 114 | total_timesteps 1813.
Path 115 | total_timesteps 1845.
Path 116 | total_timesteps 1854.
Path 117 | total_timesteps 1873.
Path 118 | total_timesteps 1896.
Path 119 | total_timesteps 1909.
Path 120 | total_timesteps 1920.
Path 121 | total_timesteps 1936.
Path 122 | total_timesteps 1948.
Path 123 | total_timesteps 1969.
Path 124 | total_timesteps 1987.
Path 125 | total_timesteps 2005.
Path 126 | total_timesteps 2016.
Path 127 | total_timesteps 2045.
Path 128 | total_timesteps 2061.
Path 129 | total_timesteps 2072.
Path 130 | total_timesteps 2086.
Path 131 | total_timesteps 2104.
Path 132 | total_timesteps 2112.
Path 133 | total_timesteps 2120.
Path 134 | total_timesteps 2148.
Path 135 | total_timesteps 2158.
Path 136 | total_timesteps 2175.
Path 137 | total_timesteps 2190.
Path 138 | total_timesteps 2210.
Path 139 | total_timesteps 2223.
Path 140 | total_timesteps 2241.
Path 141 | total_timesteps 2254.
Path 142 | total_timesteps 2267.
Path 143 | total_timesteps 2282.
Path 144 | total_timesteps 2292.
Path 145 | total_timesteps 2320.
Path 146 | total_timesteps 2335.
Path 147 | total_timesteps 2366.
Path 148 | total_timesteps 2380.
Path 149 | total_timesteps 2398.
Path 150 | total_timesteps 2416.
Path 151 | total_timesteps 2429.
Path 152 | total_timesteps 2443.
Path 153 | total_timesteps 2455.
Path 154 | total_timesteps 2465.
Path 155 | total_timesteps 2480.
Path 156 | total_timesteps 2495.
Path 157 | total_timesteps 2504.
Path 158 | total_timesteps 2514.
Path 159 | total_timesteps 2523.
Path 160 | total_timesteps 2535.
Path 161 | total_timesteps 2545.
Path 162 | total_timesteps 2555.
Path 163 | total_timesteps 2570.
Path 164 | total_timesteps 2580.
Path 165 | total_timesteps 2599.
Path 166 | total_timesteps 2609.
Path 167 | total_timesteps 2622.
Path 168 | total_timesteps 2635.
Path 169 | total_timesteps 2658.
Path 170 | total_timesteps 2675.
Path 171 | total_timesteps 2687.
Path 172 | total_timesteps 2699.
Path 173 | total_timesteps 2714.
Path 174 | total_timesteps 2736.
Path 175 | total_timesteps 2752.
Path 176 | total_timesteps 2766.
Path 177 | total_timesteps 2775.
Path 178 | total_timesteps 2804.
Path 179 | total_timesteps 2819.
Path 180 | total_timesteps 2831.
Path 181 | total_timesteps 2844.
Path 182 | total_timesteps 2852.
Path 183 | total_timesteps 2861.
Path 184 | total_timesteps 2879.
Path 185 | total_timesteps 2888.
Path 186 | total_timesteps 2915.
Path 187 | total_timesteps 2929.
Path 188 | total_timesteps 2944.
Path 189 | total_timesteps 2989.
Path 190 | total_timesteps 3006.
Path 191 | total_timesteps 3024.
Path 192 | total_timesteps 3037.
Path 193 | total_timesteps 3054.
Path 194 | total_timesteps 3065.
Path 195 | total_timesteps 3082.
Path 196 | total_timesteps 3092.
Path 197 | total_timesteps 3118.
Path 198 | total_timesteps 3132.
Path 199 | total_timesteps 3145.
Path 200 | total_timesteps 3158.
Path 201 | total_timesteps 3168.
Path 202 | total_timesteps 3192.
Path 203 | total_timesteps 3206.
Path 204 | total_timesteps 3222.
Path 205 | total_timesteps 3234.
Path 206 | total_timesteps 3258.
Path 207 | total_timesteps 3274.
Path 208 | total_timesteps 3286.
Path 209 | total_timesteps 3300.
Path 210 | total_timesteps 3321.
Path 211 | total_timesteps 3330.
Path 212 | total_timesteps 3345.
Path 213 | total_timesteps 3357.
Path 214 | total_timesteps 3370.
Path 215 | total_timesteps 3385.
Path 216 | total_timesteps 3406.
Path 217 | total_timesteps 3427.
Path 218 | total_timesteps 3446.
Path 219 | total_timesteps 3463.
Path 220 | total_timesteps 3471.
Path 221 | total_timesteps 3480.
Path 222 | total_timesteps 3495.
Path 223 | total_timesteps 3504.
Path 224 | total_timesteps 3522.
Path 225 | total_timesteps 3543.
Path 226 | total_timesteps 3558.
Path 227 | total_timesteps 3582.
Path 228 | total_timesteps 3599.
Path 229 | total_timesteps 3610.
Path 230 | total_timesteps 3637.
Path 231 | total_timesteps 3655.
Path 232 | total_timesteps 3671.
Path 233 | total_timesteps 3685.
Path 234 | total_timesteps 3698.
Path 235 | total_timesteps 3712.
Path 236 | total_timesteps 3730.
Path 237 | total_timesteps 3738.
Path 238 | total_timesteps 3753.
Path 239 | total_timesteps 3768.
Path 240 | total_timesteps 3782.
Path 241 | total_timesteps 3799.
Path 242 | total_timesteps 3816.
Path 243 | total_timesteps 3836.
Path 244 | total_timesteps 3849.
Path 245 | total_timesteps 3859.
Path 246 | total_timesteps 3868.
Path 247 | total_timesteps 3882.
Path 248 | total_timesteps 3902.
Path 249 | total_timesteps 3920.
Path 250 | total_timesteps 3941.
Path 251 | total_timesteps 3958.
Path 252 | total_timesteps 3974.
Path 253 | total_timesteps 3990.
Path 254 | total_timesteps 3999.
Path 255 | total_timesteps 4019.
Path 256 | total_timesteps 4038.
Path 257 | total_timesteps 4056.
Path 258 | total_timesteps 4072.
Path 259 | total_timesteps 4092.
Path 260 | total_timesteps 4108.
Path 261 | total_timesteps 4142.
Path 262 | total_timesteps 4170.
Path 263 | total_timesteps 4183.
Path 264 | total_timesteps 4196.
Path 265 | total_timesteps 4204.
Path 266 | total_timesteps 4228.
Path 267 | total_timesteps 4242.
Path 268 | total_timesteps 4255.
Path 269 | total_timesteps 4267.
Path 270 | total_timesteps 4284.
Path 271 | total_timesteps 4314.
Path 272 | total_timesteps 4328.
Path 273 | total_timesteps 4339.
Path 274 | total_timesteps 4370.
Path 275 | total_timesteps 4386.
Path 276 | total_timesteps 4398.
Path 277 | total_timesteps 4409.
Path 278 | total_timesteps 4420.
Path 279 | total_timesteps 4427.
Path 280 | total_timesteps 4438.
Path 281 | total_timesteps 4451.
Path 282 | total_timesteps 4473.
Path 283 | total_timesteps 4482.
Path 284 | total_timesteps 4496.
Path 285 | total_timesteps 4505.
Path 286 | total_timesteps 4518.
Path 287 | total_timesteps 4550.
Path 288 | total_timesteps 4568.
Path 289 | total_timesteps 4583.
Path 290 | total_timesteps 4595.
Path 291 | total_timesteps 4609.
Path 292 | total_timesteps 4627.
Path 293 | total_timesteps 4644.
Path 294 | total_timesteps 4662.
Path 295 | total_timesteps 4680.
Path 296 | total_timesteps 4694.
Path 297 | total_timesteps 4708.
Path 298 | total_timesteps 4717.
Path 299 | total_timesteps 4730.
Path 300 | total_timesteps 4740.
Path 301 | total_timesteps 4756.
Path 302 | total_timesteps 4771.
Path 303 | total_timesteps 4782.
Path 304 | total_timesteps 4799.
Path 305 | total_timesteps 4822.
Path 306 | total_timesteps 4831.
Path 307 | total_timesteps 4850.
Path 308 | total_timesteps 4857.
Path 309 | total_timesteps 4877.
Path 310 | total_timesteps 4904.
Path 311 | total_timesteps 4916.
Path 312 | total_timesteps 4933.
Path 313 | total_timesteps 4948.
Path 314 | total_timesteps 4963.
Path 315 | total_timesteps 4981.
Path 316 | total_timesteps 4988.
Path 317 | total_timesteps 4998.
Path 318 | total_timesteps 5008.
Path 319 | total_timesteps 5022.
Path 320 | total_timesteps 5037.
Path 321 | total_timesteps 5054.
Path 322 | total_timesteps 5072.
Path 323 | total_timesteps 5094.
Path 324 | total_timesteps 5111.
Path 325 | total_timesteps 5121.
Path 326 | total_timesteps 5142.
Path 327 | total_timesteps 5159.
Path 328 | total_timesteps 5174.
Path 329 | total_timesteps 5197.
Path 330 | total_timesteps 5208.
Path 331 | total_timesteps 5217.
Path 332 | total_timesteps 5226.
Path 333 | total_timesteps 5242.
Path 334 | total_timesteps 5250.
Path 335 | total_timesteps 5278.
Path 336 | total_timesteps 5297.
Path 337 | total_timesteps 5310.
Path 338 | total_timesteps 5325.
Path 339 | total_timesteps 5339.
Path 340 | total_timesteps 5356.
Path 341 | total_timesteps 5365.
Path 342 | total_timesteps 5379.
Path 343 | total_timesteps 5403.
Path 344 | total_timesteps 5416.
Path 345 | total_timesteps 5428.
Path 346 | total_timesteps 5442.
Path 347 | total_timesteps 5449.
Path 348 | total_timesteps 5482.
Path 349 | total_timesteps 5490.
Path 350 | total_timesteps 5503.
Path 351 | total_timesteps 5520.
Path 352 | total_timesteps 5531.
Path 353 | total_timesteps 5549.
Path 354 | total_timesteps 5561.
Path 355 | total_timesteps 5574.
Path 356 | total_timesteps 5586.
Path 357 | total_timesteps 5604.
Path 358 | total_timesteps 5615.
Path 359 | total_timesteps 5624.
Path 360 | total_timesteps 5653.
Path 361 | total_timesteps 5667.
Path 362 | total_timesteps 5679.
Path 363 | total_timesteps 5706.
Path 364 | total_timesteps 5727.
Path 365 | total_timesteps 5736.
Path 366 | total_timesteps 5754.
Path 367 | total_timesteps 5766.
Path 368 | total_timesteps 5779.
Path 369 | total_timesteps 5793.
Path 370 | total_timesteps 5805.
Path 371 | total_timesteps 5821.
Path 372 | total_timesteps 5834.
Path 373 | total_timesteps 5844.
Path 374 | total_timesteps 5862.
Path 375 | total_timesteps 5873.
Path 376 | total_timesteps 5898.
Path 377 | total_timesteps 5919.
Path 378 | total_timesteps 5931.
Path 379 | total_timesteps 5947.
Path 380 | total_timesteps 5956.
Path 381 | total_timesteps 5966.
Path 382 | total_timesteps 5983.
Path 383 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.3     |
| Iteration     | 10       |
| MaximumReturn | 2.92     |
| MinimumReturn | -19.7    |
| TotalSamples  | 48061    |
----------------------------
itr #11 | 
Fitting dynamics.
Validation loss = 0.010439434088766575
Validation loss = 0.0107627734541893
Validation loss = 0.01025394070893526
Validation loss = 0.010408084839582443
Validation loss = 0.010194451548159122
Validation loss = 0.010075614787638187
Validation loss = 0.01004979107528925
Validation loss = 0.010615120641887188
Validation loss = 0.009731600992381573
Validation loss = 0.0106785474345088
Validation loss = 0.010050271637737751
Validation loss = 0.010067186318337917
Validation loss = 0.00941613968461752
Validation loss = 0.009758071042597294
Validation loss = 0.01012793555855751
Validation loss = 0.009247023612260818
Validation loss = 0.009097895585000515
Validation loss = 0.009584545157849789
Validation loss = 0.009615282528102398
Validation loss = 0.009347609244287014
Validation loss = 0.009494810365140438
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 22.
Path 3 | total_timesteps 36.
Path 4 | total_timesteps 53.
Path 5 | total_timesteps 63.
Path 6 | total_timesteps 83.
Path 7 | total_timesteps 95.
Path 8 | total_timesteps 106.
Path 9 | total_timesteps 123.
Path 10 | total_timesteps 136.
Path 11 | total_timesteps 151.
Path 12 | total_timesteps 161.
Path 13 | total_timesteps 173.
Path 14 | total_timesteps 183.
Path 15 | total_timesteps 208.
Path 16 | total_timesteps 217.
Path 17 | total_timesteps 224.
Path 18 | total_timesteps 237.
Path 19 | total_timesteps 246.
Path 20 | total_timesteps 260.
Path 21 | total_timesteps 272.
Path 22 | total_timesteps 286.
Path 23 | total_timesteps 307.
Path 24 | total_timesteps 318.
Path 25 | total_timesteps 336.
Path 26 | total_timesteps 350.
Path 27 | total_timesteps 361.
Path 28 | total_timesteps 386.
Path 29 | total_timesteps 395.
Path 30 | total_timesteps 405.
Path 31 | total_timesteps 415.
Path 32 | total_timesteps 427.
Path 33 | total_timesteps 441.
Path 34 | total_timesteps 448.
Path 35 | total_timesteps 456.
Path 36 | total_timesteps 469.
Path 37 | total_timesteps 477.
Path 38 | total_timesteps 488.
Path 39 | total_timesteps 498.
Path 40 | total_timesteps 510.
Path 41 | total_timesteps 519.
Path 42 | total_timesteps 527.
Path 43 | total_timesteps 544.
Path 44 | total_timesteps 580.
Path 45 | total_timesteps 591.
Path 46 | total_timesteps 598.
Path 47 | total_timesteps 608.
Path 48 | total_timesteps 618.
Path 49 | total_timesteps 651.
Path 50 | total_timesteps 666.
Path 51 | total_timesteps 678.
Path 52 | total_timesteps 697.
Path 53 | total_timesteps 707.
Path 54 | total_timesteps 719.
Path 55 | total_timesteps 736.
Path 56 | total_timesteps 749.
Path 57 | total_timesteps 756.
Path 58 | total_timesteps 774.
Path 59 | total_timesteps 786.
Path 60 | total_timesteps 803.
Path 61 | total_timesteps 814.
Path 62 | total_timesteps 843.
Path 63 | total_timesteps 860.
Path 64 | total_timesteps 876.
Path 65 | total_timesteps 888.
Path 66 | total_timesteps 901.
Path 67 | total_timesteps 908.
Path 68 | total_timesteps 919.
Path 69 | total_timesteps 930.
Path 70 | total_timesteps 943.
Path 71 | total_timesteps 958.
Path 72 | total_timesteps 968.
Path 73 | total_timesteps 978.
Path 74 | total_timesteps 991.
Path 75 | total_timesteps 999.
Path 76 | total_timesteps 1015.
Path 77 | total_timesteps 1029.
Path 78 | total_timesteps 1040.
Path 79 | total_timesteps 1051.
Path 80 | total_timesteps 1072.
Path 81 | total_timesteps 1084.
Path 82 | total_timesteps 1095.
Path 83 | total_timesteps 1105.
Path 84 | total_timesteps 1117.
Path 85 | total_timesteps 1128.
Path 86 | total_timesteps 1136.
Path 87 | total_timesteps 1148.
Path 88 | total_timesteps 1158.
Path 89 | total_timesteps 1165.
Path 90 | total_timesteps 1175.
Path 91 | total_timesteps 1186.
Path 92 | total_timesteps 1197.
Path 93 | total_timesteps 1206.
Path 94 | total_timesteps 1228.
Path 95 | total_timesteps 1245.
Path 96 | total_timesteps 1267.
Path 97 | total_timesteps 1280.
Path 98 | total_timesteps 1297.
Path 99 | total_timesteps 1314.
Path 100 | total_timesteps 1328.
Path 101 | total_timesteps 1343.
Path 102 | total_timesteps 1358.
Path 103 | total_timesteps 1371.
Path 104 | total_timesteps 1385.
Path 105 | total_timesteps 1398.
Path 106 | total_timesteps 1418.
Path 107 | total_timesteps 1433.
Path 108 | total_timesteps 1443.
Path 109 | total_timesteps 1456.
Path 110 | total_timesteps 1467.
Path 111 | total_timesteps 1482.
Path 112 | total_timesteps 1493.
Path 113 | total_timesteps 1507.
Path 114 | total_timesteps 1517.
Path 115 | total_timesteps 1528.
Path 116 | total_timesteps 1543.
Path 117 | total_timesteps 1551.
Path 118 | total_timesteps 1560.
Path 119 | total_timesteps 1571.
Path 120 | total_timesteps 1583.
Path 121 | total_timesteps 1596.
Path 122 | total_timesteps 1604.
Path 123 | total_timesteps 1620.
Path 124 | total_timesteps 1632.
Path 125 | total_timesteps 1642.
Path 126 | total_timesteps 1658.
Path 127 | total_timesteps 1667.
Path 128 | total_timesteps 1677.
Path 129 | total_timesteps 1687.
Path 130 | total_timesteps 1695.
Path 131 | total_timesteps 1708.
Path 132 | total_timesteps 1722.
Path 133 | total_timesteps 1729.
Path 134 | total_timesteps 1738.
Path 135 | total_timesteps 1761.
Path 136 | total_timesteps 1768.
Path 137 | total_timesteps 1781.
Path 138 | total_timesteps 1793.
Path 139 | total_timesteps 1811.
Path 140 | total_timesteps 1825.
Path 141 | total_timesteps 1837.
Path 142 | total_timesteps 1848.
Path 143 | total_timesteps 1861.
Path 144 | total_timesteps 1881.
Path 145 | total_timesteps 1894.
Path 146 | total_timesteps 1902.
Path 147 | total_timesteps 1930.
Path 148 | total_timesteps 1944.
Path 149 | total_timesteps 1956.
Path 150 | total_timesteps 1964.
Path 151 | total_timesteps 1995.
Path 152 | total_timesteps 2010.
Path 153 | total_timesteps 2023.
Path 154 | total_timesteps 2032.
Path 155 | total_timesteps 2041.
Path 156 | total_timesteps 2056.
Path 157 | total_timesteps 2067.
Path 158 | total_timesteps 2083.
Path 159 | total_timesteps 2096.
Path 160 | total_timesteps 2109.
Path 161 | total_timesteps 2118.
Path 162 | total_timesteps 2136.
Path 163 | total_timesteps 2148.
Path 164 | total_timesteps 2159.
Path 165 | total_timesteps 2170.
Path 166 | total_timesteps 2181.
Path 167 | total_timesteps 2192.
Path 168 | total_timesteps 2215.
Path 169 | total_timesteps 2223.
Path 170 | total_timesteps 2232.
Path 171 | total_timesteps 2253.
Path 172 | total_timesteps 2265.
Path 173 | total_timesteps 2272.
Path 174 | total_timesteps 2289.
Path 175 | total_timesteps 2301.
Path 176 | total_timesteps 2327.
Path 177 | total_timesteps 2345.
Path 178 | total_timesteps 2357.
Path 179 | total_timesteps 2376.
Path 180 | total_timesteps 2385.
Path 181 | total_timesteps 2404.
Path 182 | total_timesteps 2414.
Path 183 | total_timesteps 2424.
Path 184 | total_timesteps 2437.
Path 185 | total_timesteps 2450.
Path 186 | total_timesteps 2461.
Path 187 | total_timesteps 2471.
Path 188 | total_timesteps 2483.
Path 189 | total_timesteps 2496.
Path 190 | total_timesteps 2506.
Path 191 | total_timesteps 2520.
Path 192 | total_timesteps 2531.
Path 193 | total_timesteps 2546.
Path 194 | total_timesteps 2559.
Path 195 | total_timesteps 2574.
Path 196 | total_timesteps 2584.
Path 197 | total_timesteps 2595.
Path 198 | total_timesteps 2607.
Path 199 | total_timesteps 2625.
Path 200 | total_timesteps 2634.
Path 201 | total_timesteps 2645.
Path 202 | total_timesteps 2654.
Path 203 | total_timesteps 2670.
Path 204 | total_timesteps 2685.
Path 205 | total_timesteps 2698.
Path 206 | total_timesteps 2709.
Path 207 | total_timesteps 2724.
Path 208 | total_timesteps 2734.
Path 209 | total_timesteps 2757.
Path 210 | total_timesteps 2772.
Path 211 | total_timesteps 2779.
Path 212 | total_timesteps 2791.
Path 213 | total_timesteps 2807.
Path 214 | total_timesteps 2816.
Path 215 | total_timesteps 2833.
Path 216 | total_timesteps 2848.
Path 217 | total_timesteps 2861.
Path 218 | total_timesteps 2874.
Path 219 | total_timesteps 2882.
Path 220 | total_timesteps 2891.
Path 221 | total_timesteps 2902.
Path 222 | total_timesteps 2912.
Path 223 | total_timesteps 2929.
Path 224 | total_timesteps 2944.
Path 225 | total_timesteps 2952.
Path 226 | total_timesteps 2961.
Path 227 | total_timesteps 2972.
Path 228 | total_timesteps 2986.
Path 229 | total_timesteps 2997.
Path 230 | total_timesteps 3009.
Path 231 | total_timesteps 3023.
Path 232 | total_timesteps 3041.
Path 233 | total_timesteps 3059.
Path 234 | total_timesteps 3078.
Path 235 | total_timesteps 3089.
Path 236 | total_timesteps 3100.
Path 237 | total_timesteps 3110.
Path 238 | total_timesteps 3123.
Path 239 | total_timesteps 3134.
Path 240 | total_timesteps 3146.
Path 241 | total_timesteps 3162.
Path 242 | total_timesteps 3183.
Path 243 | total_timesteps 3190.
Path 244 | total_timesteps 3199.
Path 245 | total_timesteps 3210.
Path 246 | total_timesteps 3228.
Path 247 | total_timesteps 3240.
Path 248 | total_timesteps 3251.
Path 249 | total_timesteps 3259.
Path 250 | total_timesteps 3276.
Path 251 | total_timesteps 3288.
Path 252 | total_timesteps 3299.
Path 253 | total_timesteps 3309.
Path 254 | total_timesteps 3324.
Path 255 | total_timesteps 3337.
Path 256 | total_timesteps 3357.
Path 257 | total_timesteps 3369.
Path 258 | total_timesteps 3381.
Path 259 | total_timesteps 3392.
Path 260 | total_timesteps 3402.
Path 261 | total_timesteps 3418.
Path 262 | total_timesteps 3430.
Path 263 | total_timesteps 3440.
Path 264 | total_timesteps 3455.
Path 265 | total_timesteps 3462.
Path 266 | total_timesteps 3480.
Path 267 | total_timesteps 3492.
Path 268 | total_timesteps 3517.
Path 269 | total_timesteps 3527.
Path 270 | total_timesteps 3539.
Path 271 | total_timesteps 3547.
Path 272 | total_timesteps 3556.
Path 273 | total_timesteps 3571.
Path 274 | total_timesteps 3580.
Path 275 | total_timesteps 3588.
Path 276 | total_timesteps 3599.
Path 277 | total_timesteps 3611.
Path 278 | total_timesteps 3619.
Path 279 | total_timesteps 3630.
Path 280 | total_timesteps 3642.
Path 281 | total_timesteps 3654.
Path 282 | total_timesteps 3666.
Path 283 | total_timesteps 3678.
Path 284 | total_timesteps 3687.
Path 285 | total_timesteps 3697.
Path 286 | total_timesteps 3706.
Path 287 | total_timesteps 3714.
Path 288 | total_timesteps 3726.
Path 289 | total_timesteps 3738.
Path 290 | total_timesteps 3748.
Path 291 | total_timesteps 3759.
Path 292 | total_timesteps 3770.
Path 293 | total_timesteps 3798.
Path 294 | total_timesteps 3806.
Path 295 | total_timesteps 3814.
Path 296 | total_timesteps 3825.
Path 297 | total_timesteps 3835.
Path 298 | total_timesteps 3844.
Path 299 | total_timesteps 3857.
Path 300 | total_timesteps 3867.
Path 301 | total_timesteps 3882.
Path 302 | total_timesteps 3900.
Path 303 | total_timesteps 3915.
Path 304 | total_timesteps 3927.
Path 305 | total_timesteps 3939.
Path 306 | total_timesteps 3951.
Path 307 | total_timesteps 3966.
Path 308 | total_timesteps 3975.
Path 309 | total_timesteps 3990.
Path 310 | total_timesteps 4002.
Path 311 | total_timesteps 4018.
Path 312 | total_timesteps 4029.
Path 313 | total_timesteps 4045.
Path 314 | total_timesteps 4054.
Path 315 | total_timesteps 4067.
Path 316 | total_timesteps 4088.
Path 317 | total_timesteps 4099.
Path 318 | total_timesteps 4108.
Path 319 | total_timesteps 4118.
Path 320 | total_timesteps 4132.
Path 321 | total_timesteps 4140.
Path 322 | total_timesteps 4154.
Path 323 | total_timesteps 4162.
Path 324 | total_timesteps 4174.
Path 325 | total_timesteps 4185.
Path 326 | total_timesteps 4202.
Path 327 | total_timesteps 4214.
Path 328 | total_timesteps 4223.
Path 329 | total_timesteps 4233.
Path 330 | total_timesteps 4241.
Path 331 | total_timesteps 4251.
Path 332 | total_timesteps 4258.
Path 333 | total_timesteps 4270.
Path 334 | total_timesteps 4290.
Path 335 | total_timesteps 4307.
Path 336 | total_timesteps 4321.
Path 337 | total_timesteps 4334.
Path 338 | total_timesteps 4346.
Path 339 | total_timesteps 4357.
Path 340 | total_timesteps 4371.
Path 341 | total_timesteps 4384.
Path 342 | total_timesteps 4397.
Path 343 | total_timesteps 4408.
Path 344 | total_timesteps 4417.
Path 345 | total_timesteps 4430.
Path 346 | total_timesteps 4441.
Path 347 | total_timesteps 4455.
Path 348 | total_timesteps 4469.
Path 349 | total_timesteps 4485.
Path 350 | total_timesteps 4495.
Path 351 | total_timesteps 4506.
Path 352 | total_timesteps 4523.
Path 353 | total_timesteps 4531.
Path 354 | total_timesteps 4547.
Path 355 | total_timesteps 4566.
Path 356 | total_timesteps 4576.
Path 357 | total_timesteps 4584.
Path 358 | total_timesteps 4592.
Path 359 | total_timesteps 4600.
Path 360 | total_timesteps 4613.
Path 361 | total_timesteps 4623.
Path 362 | total_timesteps 4637.
Path 363 | total_timesteps 4654.
Path 364 | total_timesteps 4667.
Path 365 | total_timesteps 4681.
Path 366 | total_timesteps 4696.
Path 367 | total_timesteps 4706.
Path 368 | total_timesteps 4718.
Path 369 | total_timesteps 4733.
Path 370 | total_timesteps 4742.
Path 371 | total_timesteps 4754.
Path 372 | total_timesteps 4762.
Path 373 | total_timesteps 4772.
Path 374 | total_timesteps 4787.
Path 375 | total_timesteps 4799.
Path 376 | total_timesteps 4813.
Path 377 | total_timesteps 4822.
Path 378 | total_timesteps 4834.
Path 379 | total_timesteps 4850.
Path 380 | total_timesteps 4858.
Path 381 | total_timesteps 4874.
Path 382 | total_timesteps 4882.
Path 383 | total_timesteps 4898.
Path 384 | total_timesteps 4915.
Path 385 | total_timesteps 4927.
Path 386 | total_timesteps 4940.
Path 387 | total_timesteps 4950.
Path 388 | total_timesteps 4963.
Path 389 | total_timesteps 4979.
Path 390 | total_timesteps 4992.
Path 391 | total_timesteps 5003.
Path 392 | total_timesteps 5018.
Path 393 | total_timesteps 5028.
Path 394 | total_timesteps 5044.
Path 395 | total_timesteps 5059.
Path 396 | total_timesteps 5071.
Path 397 | total_timesteps 5085.
Path 398 | total_timesteps 5101.
Path 399 | total_timesteps 5115.
Path 400 | total_timesteps 5123.
Path 401 | total_timesteps 5137.
Path 402 | total_timesteps 5152.
Path 403 | total_timesteps 5163.
Path 404 | total_timesteps 5174.
Path 405 | total_timesteps 5184.
Path 406 | total_timesteps 5202.
Path 407 | total_timesteps 5212.
Path 408 | total_timesteps 5223.
Path 409 | total_timesteps 5242.
Path 410 | total_timesteps 5251.
Path 411 | total_timesteps 5267.
Path 412 | total_timesteps 5280.
Path 413 | total_timesteps 5294.
Path 414 | total_timesteps 5302.
Path 415 | total_timesteps 5315.
Path 416 | total_timesteps 5325.
Path 417 | total_timesteps 5335.
Path 418 | total_timesteps 5346.
Path 419 | total_timesteps 5359.
Path 420 | total_timesteps 5373.
Path 421 | total_timesteps 5384.
Path 422 | total_timesteps 5398.
Path 423 | total_timesteps 5408.
Path 424 | total_timesteps 5417.
Path 425 | total_timesteps 5424.
Path 426 | total_timesteps 5441.
Path 427 | total_timesteps 5451.
Path 428 | total_timesteps 5463.
Path 429 | total_timesteps 5477.
Path 430 | total_timesteps 5490.
Path 431 | total_timesteps 5502.
Path 432 | total_timesteps 5510.
Path 433 | total_timesteps 5520.
Path 434 | total_timesteps 5539.
Path 435 | total_timesteps 5556.
Path 436 | total_timesteps 5568.
Path 437 | total_timesteps 5583.
Path 438 | total_timesteps 5600.
Path 439 | total_timesteps 5613.
Path 440 | total_timesteps 5626.
Path 441 | total_timesteps 5642.
Path 442 | total_timesteps 5666.
Path 443 | total_timesteps 5676.
Path 444 | total_timesteps 5691.
Path 445 | total_timesteps 5707.
Path 446 | total_timesteps 5719.
Path 447 | total_timesteps 5729.
Path 448 | total_timesteps 5746.
Path 449 | total_timesteps 5763.
Path 450 | total_timesteps 5775.
Path 451 | total_timesteps 5785.
Path 452 | total_timesteps 5807.
Path 453 | total_timesteps 5816.
Path 454 | total_timesteps 5830.
Path 455 | total_timesteps 5841.
Path 456 | total_timesteps 5856.
Path 457 | total_timesteps 5872.
Path 458 | total_timesteps 5891.
Path 459 | total_timesteps 5903.
Path 460 | total_timesteps 5915.
Path 461 | total_timesteps 5930.
Path 462 | total_timesteps 5946.
Path 463 | total_timesteps 5957.
Path 464 | total_timesteps 5982.
Path 465 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.02    |
| Iteration     | 11       |
| MaximumReturn | 6.11     |
| MinimumReturn | -21      |
| TotalSamples  | 52061    |
----------------------------
itr #12 | 
Fitting dynamics.
Validation loss = 0.009505772031843662
Validation loss = 0.008952881209552288
Validation loss = 0.009062083438038826
Validation loss = 0.009417236782610416
Validation loss = 0.009893794544041157
Validation loss = 0.008713902905583382
Validation loss = 0.009783623740077019
Validation loss = 0.008983489125967026
Validation loss = 0.009145509451627731
Validation loss = 0.008677543140947819
Validation loss = 0.008909082971513271
Validation loss = 0.008931312710046768
Validation loss = 0.008937952108681202
Validation loss = 0.009300816804170609
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 34.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 67.
Path 5 | total_timesteps 85.
Path 6 | total_timesteps 93.
Path 7 | total_timesteps 113.
Path 8 | total_timesteps 129.
Path 9 | total_timesteps 147.
Path 10 | total_timesteps 164.
Path 11 | total_timesteps 173.
Path 12 | total_timesteps 193.
Path 13 | total_timesteps 206.
Path 14 | total_timesteps 221.
Path 15 | total_timesteps 237.
Path 16 | total_timesteps 246.
Path 17 | total_timesteps 259.
Path 18 | total_timesteps 280.
Path 19 | total_timesteps 294.
Path 20 | total_timesteps 313.
Path 21 | total_timesteps 329.
Path 22 | total_timesteps 340.
Path 23 | total_timesteps 348.
Path 24 | total_timesteps 359.
Path 25 | total_timesteps 372.
Path 26 | total_timesteps 381.
Path 27 | total_timesteps 391.
Path 28 | total_timesteps 400.
Path 29 | total_timesteps 409.
Path 30 | total_timesteps 423.
Path 31 | total_timesteps 435.
Path 32 | total_timesteps 445.
Path 33 | total_timesteps 455.
Path 34 | total_timesteps 465.
Path 35 | total_timesteps 474.
Path 36 | total_timesteps 489.
Path 37 | total_timesteps 502.
Path 38 | total_timesteps 518.
Path 39 | total_timesteps 532.
Path 40 | total_timesteps 546.
Path 41 | total_timesteps 569.
Path 42 | total_timesteps 582.
Path 43 | total_timesteps 589.
Path 44 | total_timesteps 599.
Path 45 | total_timesteps 620.
Path 46 | total_timesteps 635.
Path 47 | total_timesteps 647.
Path 48 | total_timesteps 679.
Path 49 | total_timesteps 690.
Path 50 | total_timesteps 712.
Path 51 | total_timesteps 726.
Path 52 | total_timesteps 735.
Path 53 | total_timesteps 741.
Path 54 | total_timesteps 754.
Path 55 | total_timesteps 763.
Path 56 | total_timesteps 775.
Path 57 | total_timesteps 783.
Path 58 | total_timesteps 799.
Path 59 | total_timesteps 811.
Path 60 | total_timesteps 819.
Path 61 | total_timesteps 838.
Path 62 | total_timesteps 854.
Path 63 | total_timesteps 865.
Path 64 | total_timesteps 875.
Path 65 | total_timesteps 884.
Path 66 | total_timesteps 895.
Path 67 | total_timesteps 904.
Path 68 | total_timesteps 913.
Path 69 | total_timesteps 922.
Path 70 | total_timesteps 937.
Path 71 | total_timesteps 951.
Path 72 | total_timesteps 960.
Path 73 | total_timesteps 977.
Path 74 | total_timesteps 991.
Path 75 | total_timesteps 1000.
Path 76 | total_timesteps 1015.
Path 77 | total_timesteps 1028.
Path 78 | total_timesteps 1041.
Path 79 | total_timesteps 1054.
Path 80 | total_timesteps 1073.
Path 81 | total_timesteps 1089.
Path 82 | total_timesteps 1110.
Path 83 | total_timesteps 1123.
Path 84 | total_timesteps 1132.
Path 85 | total_timesteps 1144.
Path 86 | total_timesteps 1153.
Path 87 | total_timesteps 1162.
Path 88 | total_timesteps 1173.
Path 89 | total_timesteps 1187.
Path 90 | total_timesteps 1193.
Path 91 | total_timesteps 1206.
Path 92 | total_timesteps 1215.
Path 93 | total_timesteps 1229.
Path 94 | total_timesteps 1252.
Path 95 | total_timesteps 1278.
Path 96 | total_timesteps 1294.
Path 97 | total_timesteps 1303.
Path 98 | total_timesteps 1315.
Path 99 | total_timesteps 1332.
Path 100 | total_timesteps 1341.
Path 101 | total_timesteps 1357.
Path 102 | total_timesteps 1374.
Path 103 | total_timesteps 1384.
Path 104 | total_timesteps 1401.
Path 105 | total_timesteps 1413.
Path 106 | total_timesteps 1427.
Path 107 | total_timesteps 1441.
Path 108 | total_timesteps 1455.
Path 109 | total_timesteps 1470.
Path 110 | total_timesteps 1486.
Path 111 | total_timesteps 1499.
Path 112 | total_timesteps 1512.
Path 113 | total_timesteps 1528.
Path 114 | total_timesteps 1543.
Path 115 | total_timesteps 1556.
Path 116 | total_timesteps 1574.
Path 117 | total_timesteps 1595.
Path 118 | total_timesteps 1605.
Path 119 | total_timesteps 1617.
Path 120 | total_timesteps 1634.
Path 121 | total_timesteps 1654.
Path 122 | total_timesteps 1673.
Path 123 | total_timesteps 1682.
Path 124 | total_timesteps 1695.
Path 125 | total_timesteps 1728.
Path 126 | total_timesteps 1747.
Path 127 | total_timesteps 1756.
Path 128 | total_timesteps 1767.
Path 129 | total_timesteps 1785.
Path 130 | total_timesteps 1799.
Path 131 | total_timesteps 1809.
Path 132 | total_timesteps 1826.
Path 133 | total_timesteps 1840.
Path 134 | total_timesteps 1856.
Path 135 | total_timesteps 1867.
Path 136 | total_timesteps 1882.
Path 137 | total_timesteps 1904.
Path 138 | total_timesteps 1920.
Path 139 | total_timesteps 1931.
Path 140 | total_timesteps 1944.
Path 141 | total_timesteps 1954.
Path 142 | total_timesteps 1967.
Path 143 | total_timesteps 1987.
Path 144 | total_timesteps 1999.
Path 145 | total_timesteps 2010.
Path 146 | total_timesteps 2018.
Path 147 | total_timesteps 2026.
Path 148 | total_timesteps 2036.
Path 149 | total_timesteps 2049.
Path 150 | total_timesteps 2063.
Path 151 | total_timesteps 2075.
Path 152 | total_timesteps 2083.
Path 153 | total_timesteps 2095.
Path 154 | total_timesteps 2108.
Path 155 | total_timesteps 2125.
Path 156 | total_timesteps 2143.
Path 157 | total_timesteps 2162.
Path 158 | total_timesteps 2175.
Path 159 | total_timesteps 2183.
Path 160 | total_timesteps 2195.
Path 161 | total_timesteps 2204.
Path 162 | total_timesteps 2215.
Path 163 | total_timesteps 2229.
Path 164 | total_timesteps 2244.
Path 165 | total_timesteps 2253.
Path 166 | total_timesteps 2262.
Path 167 | total_timesteps 2274.
Path 168 | total_timesteps 2288.
Path 169 | total_timesteps 2296.
Path 170 | total_timesteps 2305.
Path 171 | total_timesteps 2325.
Path 172 | total_timesteps 2333.
Path 173 | total_timesteps 2345.
Path 174 | total_timesteps 2359.
Path 175 | total_timesteps 2374.
Path 176 | total_timesteps 2383.
Path 177 | total_timesteps 2392.
Path 178 | total_timesteps 2404.
Path 179 | total_timesteps 2419.
Path 180 | total_timesteps 2429.
Path 181 | total_timesteps 2437.
Path 182 | total_timesteps 2450.
Path 183 | total_timesteps 2462.
Path 184 | total_timesteps 2474.
Path 185 | total_timesteps 2483.
Path 186 | total_timesteps 2490.
Path 187 | total_timesteps 2505.
Path 188 | total_timesteps 2515.
Path 189 | total_timesteps 2542.
Path 190 | total_timesteps 2555.
Path 191 | total_timesteps 2564.
Path 192 | total_timesteps 2579.
Path 193 | total_timesteps 2593.
Path 194 | total_timesteps 2603.
Path 195 | total_timesteps 2617.
Path 196 | total_timesteps 2628.
Path 197 | total_timesteps 2642.
Path 198 | total_timesteps 2659.
Path 199 | total_timesteps 2672.
Path 200 | total_timesteps 2695.
Path 201 | total_timesteps 2708.
Path 202 | total_timesteps 2725.
Path 203 | total_timesteps 2739.
Path 204 | total_timesteps 2755.
Path 205 | total_timesteps 2768.
Path 206 | total_timesteps 2784.
Path 207 | total_timesteps 2798.
Path 208 | total_timesteps 2807.
Path 209 | total_timesteps 2821.
Path 210 | total_timesteps 2838.
Path 211 | total_timesteps 2851.
Path 212 | total_timesteps 2863.
Path 213 | total_timesteps 2873.
Path 214 | total_timesteps 2886.
Path 215 | total_timesteps 2896.
Path 216 | total_timesteps 2905.
Path 217 | total_timesteps 2916.
Path 218 | total_timesteps 2931.
Path 219 | total_timesteps 2946.
Path 220 | total_timesteps 2957.
Path 221 | total_timesteps 2969.
Path 222 | total_timesteps 2982.
Path 223 | total_timesteps 2996.
Path 224 | total_timesteps 3011.
Path 225 | total_timesteps 3021.
Path 226 | total_timesteps 3029.
Path 227 | total_timesteps 3041.
Path 228 | total_timesteps 3052.
Path 229 | total_timesteps 3066.
Path 230 | total_timesteps 3075.
Path 231 | total_timesteps 3087.
Path 232 | total_timesteps 3103.
Path 233 | total_timesteps 3119.
Path 234 | total_timesteps 3134.
Path 235 | total_timesteps 3148.
Path 236 | total_timesteps 3156.
Path 237 | total_timesteps 3165.
Path 238 | total_timesteps 3175.
Path 239 | total_timesteps 3192.
Path 240 | total_timesteps 3205.
Path 241 | total_timesteps 3215.
Path 242 | total_timesteps 3237.
Path 243 | total_timesteps 3245.
Path 244 | total_timesteps 3253.
Path 245 | total_timesteps 3270.
Path 246 | total_timesteps 3279.
Path 247 | total_timesteps 3290.
Path 248 | total_timesteps 3297.
Path 249 | total_timesteps 3310.
Path 250 | total_timesteps 3320.
Path 251 | total_timesteps 3339.
Path 252 | total_timesteps 3348.
Path 253 | total_timesteps 3362.
Path 254 | total_timesteps 3376.
Path 255 | total_timesteps 3392.
Path 256 | total_timesteps 3403.
Path 257 | total_timesteps 3412.
Path 258 | total_timesteps 3425.
Path 259 | total_timesteps 3440.
Path 260 | total_timesteps 3452.
Path 261 | total_timesteps 3459.
Path 262 | total_timesteps 3471.
Path 263 | total_timesteps 3491.
Path 264 | total_timesteps 3511.
Path 265 | total_timesteps 3524.
Path 266 | total_timesteps 3532.
Path 267 | total_timesteps 3549.
Path 268 | total_timesteps 3563.
Path 269 | total_timesteps 3574.
Path 270 | total_timesteps 3598.
Path 271 | total_timesteps 3615.
Path 272 | total_timesteps 3627.
Path 273 | total_timesteps 3640.
Path 274 | total_timesteps 3653.
Path 275 | total_timesteps 3662.
Path 276 | total_timesteps 3672.
Path 277 | total_timesteps 3681.
Path 278 | total_timesteps 3694.
Path 279 | total_timesteps 3704.
Path 280 | total_timesteps 3719.
Path 281 | total_timesteps 3730.
Path 282 | total_timesteps 3738.
Path 283 | total_timesteps 3753.
Path 284 | total_timesteps 3768.
Path 285 | total_timesteps 3787.
Path 286 | total_timesteps 3799.
Path 287 | total_timesteps 3810.
Path 288 | total_timesteps 3820.
Path 289 | total_timesteps 3834.
Path 290 | total_timesteps 3856.
Path 291 | total_timesteps 3865.
Path 292 | total_timesteps 3873.
Path 293 | total_timesteps 3886.
Path 294 | total_timesteps 3900.
Path 295 | total_timesteps 3912.
Path 296 | total_timesteps 3920.
Path 297 | total_timesteps 3937.
Path 298 | total_timesteps 3952.
Path 299 | total_timesteps 3968.
Path 300 | total_timesteps 3984.
Path 301 | total_timesteps 3994.
Path 302 | total_timesteps 4011.
Path 303 | total_timesteps 4024.
Path 304 | total_timesteps 4035.
Path 305 | total_timesteps 4050.
Path 306 | total_timesteps 4058.
Path 307 | total_timesteps 4077.
Path 308 | total_timesteps 4087.
Path 309 | total_timesteps 4099.
Path 310 | total_timesteps 4109.
Path 311 | total_timesteps 4118.
Path 312 | total_timesteps 4130.
Path 313 | total_timesteps 4153.
Path 314 | total_timesteps 4160.
Path 315 | total_timesteps 4168.
Path 316 | total_timesteps 4185.
Path 317 | total_timesteps 4198.
Path 318 | total_timesteps 4207.
Path 319 | total_timesteps 4219.
Path 320 | total_timesteps 4229.
Path 321 | total_timesteps 4240.
Path 322 | total_timesteps 4259.
Path 323 | total_timesteps 4268.
Path 324 | total_timesteps 4294.
Path 325 | total_timesteps 4305.
Path 326 | total_timesteps 4315.
Path 327 | total_timesteps 4324.
Path 328 | total_timesteps 4341.
Path 329 | total_timesteps 4352.
Path 330 | total_timesteps 4368.
Path 331 | total_timesteps 4383.
Path 332 | total_timesteps 4395.
Path 333 | total_timesteps 4407.
Path 334 | total_timesteps 4426.
Path 335 | total_timesteps 4437.
Path 336 | total_timesteps 4445.
Path 337 | total_timesteps 4456.
Path 338 | total_timesteps 4474.
Path 339 | total_timesteps 4485.
Path 340 | total_timesteps 4496.
Path 341 | total_timesteps 4510.
Path 342 | total_timesteps 4521.
Path 343 | total_timesteps 4539.
Path 344 | total_timesteps 4559.
Path 345 | total_timesteps 4570.
Path 346 | total_timesteps 4579.
Path 347 | total_timesteps 4586.
Path 348 | total_timesteps 4599.
Path 349 | total_timesteps 4611.
Path 350 | total_timesteps 4633.
Path 351 | total_timesteps 4641.
Path 352 | total_timesteps 4654.
Path 353 | total_timesteps 4666.
Path 354 | total_timesteps 4674.
Path 355 | total_timesteps 4687.
Path 356 | total_timesteps 4695.
Path 357 | total_timesteps 4706.
Path 358 | total_timesteps 4728.
Path 359 | total_timesteps 4739.
Path 360 | total_timesteps 4753.
Path 361 | total_timesteps 4762.
Path 362 | total_timesteps 4775.
Path 363 | total_timesteps 4787.
Path 364 | total_timesteps 4797.
Path 365 | total_timesteps 4808.
Path 366 | total_timesteps 4819.
Path 367 | total_timesteps 4837.
Path 368 | total_timesteps 4851.
Path 369 | total_timesteps 4862.
Path 370 | total_timesteps 4876.
Path 371 | total_timesteps 4887.
Path 372 | total_timesteps 4895.
Path 373 | total_timesteps 4908.
Path 374 | total_timesteps 4921.
Path 375 | total_timesteps 4934.
Path 376 | total_timesteps 4945.
Path 377 | total_timesteps 4957.
Path 378 | total_timesteps 4967.
Path 379 | total_timesteps 4979.
Path 380 | total_timesteps 4992.
Path 381 | total_timesteps 5001.
Path 382 | total_timesteps 5015.
Path 383 | total_timesteps 5031.
Path 384 | total_timesteps 5038.
Path 385 | total_timesteps 5055.
Path 386 | total_timesteps 5065.
Path 387 | total_timesteps 5083.
Path 388 | total_timesteps 5101.
Path 389 | total_timesteps 5115.
Path 390 | total_timesteps 5127.
Path 391 | total_timesteps 5137.
Path 392 | total_timesteps 5156.
Path 393 | total_timesteps 5172.
Path 394 | total_timesteps 5196.
Path 395 | total_timesteps 5207.
Path 396 | total_timesteps 5220.
Path 397 | total_timesteps 5232.
Path 398 | total_timesteps 5244.
Path 399 | total_timesteps 5255.
Path 400 | total_timesteps 5265.
Path 401 | total_timesteps 5283.
Path 402 | total_timesteps 5299.
Path 403 | total_timesteps 5312.
Path 404 | total_timesteps 5328.
Path 405 | total_timesteps 5344.
Path 406 | total_timesteps 5357.
Path 407 | total_timesteps 5367.
Path 408 | total_timesteps 5375.
Path 409 | total_timesteps 5387.
Path 410 | total_timesteps 5399.
Path 411 | total_timesteps 5409.
Path 412 | total_timesteps 5422.
Path 413 | total_timesteps 5435.
Path 414 | total_timesteps 5446.
Path 415 | total_timesteps 5460.
Path 416 | total_timesteps 5468.
Path 417 | total_timesteps 5479.
Path 418 | total_timesteps 5494.
Path 419 | total_timesteps 5511.
Path 420 | total_timesteps 5529.
Path 421 | total_timesteps 5539.
Path 422 | total_timesteps 5550.
Path 423 | total_timesteps 5558.
Path 424 | total_timesteps 5575.
Path 425 | total_timesteps 5585.
Path 426 | total_timesteps 5594.
Path 427 | total_timesteps 5602.
Path 428 | total_timesteps 5612.
Path 429 | total_timesteps 5632.
Path 430 | total_timesteps 5643.
Path 431 | total_timesteps 5653.
Path 432 | total_timesteps 5666.
Path 433 | total_timesteps 5677.
Path 434 | total_timesteps 5693.
Path 435 | total_timesteps 5708.
Path 436 | total_timesteps 5715.
Path 437 | total_timesteps 5724.
Path 438 | total_timesteps 5738.
Path 439 | total_timesteps 5748.
Path 440 | total_timesteps 5759.
Path 441 | total_timesteps 5772.
Path 442 | total_timesteps 5791.
Path 443 | total_timesteps 5799.
Path 444 | total_timesteps 5808.
Path 445 | total_timesteps 5820.
Path 446 | total_timesteps 5832.
Path 447 | total_timesteps 5842.
Path 448 | total_timesteps 5853.
Path 449 | total_timesteps 5870.
Path 450 | total_timesteps 5882.
Path 451 | total_timesteps 5897.
Path 452 | total_timesteps 5909.
Path 453 | total_timesteps 5921.
Path 454 | total_timesteps 5938.
Path 455 | total_timesteps 5953.
Path 456 | total_timesteps 5962.
Path 457 | total_timesteps 5972.
Path 458 | total_timesteps 5983.
Path 459 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.94    |
| Iteration     | 12       |
| MaximumReturn | 5.71     |
| MinimumReturn | -20.1    |
| TotalSamples  | 56064    |
----------------------------
itr #13 | 
Fitting dynamics.
Validation loss = 0.009329969063401222
Validation loss = 0.009404084645211697
Validation loss = 0.008643096312880516
Validation loss = 0.008181056007742882
Validation loss = 0.008903300389647484
Validation loss = 0.008230009116232395
Validation loss = 0.008430175483226776
Validation loss = 0.008444877341389656
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 22.
Path 3 | total_timesteps 35.
Path 4 | total_timesteps 43.
Path 5 | total_timesteps 52.
Path 6 | total_timesteps 64.
Path 7 | total_timesteps 87.
Path 8 | total_timesteps 96.
Path 9 | total_timesteps 106.
Path 10 | total_timesteps 124.
Path 11 | total_timesteps 140.
Path 12 | total_timesteps 152.
Path 13 | total_timesteps 172.
Path 14 | total_timesteps 182.
Path 15 | total_timesteps 196.
Path 16 | total_timesteps 210.
Path 17 | total_timesteps 223.
Path 18 | total_timesteps 240.
Path 19 | total_timesteps 248.
Path 20 | total_timesteps 261.
Path 21 | total_timesteps 273.
Path 22 | total_timesteps 290.
Path 23 | total_timesteps 305.
Path 24 | total_timesteps 320.
Path 25 | total_timesteps 337.
Path 26 | total_timesteps 351.
Path 27 | total_timesteps 363.
Path 28 | total_timesteps 382.
Path 29 | total_timesteps 394.
Path 30 | total_timesteps 406.
Path 31 | total_timesteps 413.
Path 32 | total_timesteps 429.
Path 33 | total_timesteps 440.
Path 34 | total_timesteps 450.
Path 35 | total_timesteps 463.
Path 36 | total_timesteps 476.
Path 37 | total_timesteps 486.
Path 38 | total_timesteps 495.
Path 39 | total_timesteps 510.
Path 40 | total_timesteps 531.
Path 41 | total_timesteps 546.
Path 42 | total_timesteps 559.
Path 43 | total_timesteps 571.
Path 44 | total_timesteps 580.
Path 45 | total_timesteps 594.
Path 46 | total_timesteps 612.
Path 47 | total_timesteps 622.
Path 48 | total_timesteps 638.
Path 49 | total_timesteps 645.
Path 50 | total_timesteps 654.
Path 51 | total_timesteps 664.
Path 52 | total_timesteps 678.
Path 53 | total_timesteps 688.
Path 54 | total_timesteps 698.
Path 55 | total_timesteps 710.
Path 56 | total_timesteps 724.
Path 57 | total_timesteps 735.
Path 58 | total_timesteps 747.
Path 59 | total_timesteps 759.
Path 60 | total_timesteps 770.
Path 61 | total_timesteps 784.
Path 62 | total_timesteps 793.
Path 63 | total_timesteps 814.
Path 64 | total_timesteps 825.
Path 65 | total_timesteps 842.
Path 66 | total_timesteps 854.
Path 67 | total_timesteps 876.
Path 68 | total_timesteps 883.
Path 69 | total_timesteps 896.
Path 70 | total_timesteps 906.
Path 71 | total_timesteps 919.
Path 72 | total_timesteps 929.
Path 73 | total_timesteps 944.
Path 74 | total_timesteps 972.
Path 75 | total_timesteps 994.
Path 76 | total_timesteps 1003.
Path 77 | total_timesteps 1009.
Path 78 | total_timesteps 1016.
Path 79 | total_timesteps 1033.
Path 80 | total_timesteps 1048.
Path 81 | total_timesteps 1061.
Path 82 | total_timesteps 1071.
Path 83 | total_timesteps 1082.
Path 84 | total_timesteps 1096.
Path 85 | total_timesteps 1109.
Path 86 | total_timesteps 1121.
Path 87 | total_timesteps 1135.
Path 88 | total_timesteps 1145.
Path 89 | total_timesteps 1158.
Path 90 | total_timesteps 1172.
Path 91 | total_timesteps 1186.
Path 92 | total_timesteps 1199.
Path 93 | total_timesteps 1213.
Path 94 | total_timesteps 1226.
Path 95 | total_timesteps 1241.
Path 96 | total_timesteps 1257.
Path 97 | total_timesteps 1271.
Path 98 | total_timesteps 1285.
Path 99 | total_timesteps 1295.
Path 100 | total_timesteps 1314.
Path 101 | total_timesteps 1324.
Path 102 | total_timesteps 1335.
Path 103 | total_timesteps 1344.
Path 104 | total_timesteps 1355.
Path 105 | total_timesteps 1367.
Path 106 | total_timesteps 1380.
Path 107 | total_timesteps 1389.
Path 108 | total_timesteps 1397.
Path 109 | total_timesteps 1407.
Path 110 | total_timesteps 1417.
Path 111 | total_timesteps 1429.
Path 112 | total_timesteps 1439.
Path 113 | total_timesteps 1459.
Path 114 | total_timesteps 1475.
Path 115 | total_timesteps 1490.
Path 116 | total_timesteps 1514.
Path 117 | total_timesteps 1543.
Path 118 | total_timesteps 1561.
Path 119 | total_timesteps 1576.
Path 120 | total_timesteps 1586.
Path 121 | total_timesteps 1599.
Path 122 | total_timesteps 1607.
Path 123 | total_timesteps 1620.
Path 124 | total_timesteps 1630.
Path 125 | total_timesteps 1651.
Path 126 | total_timesteps 1694.
Path 127 | total_timesteps 1706.
Path 128 | total_timesteps 1718.
Path 129 | total_timesteps 1728.
Path 130 | total_timesteps 1739.
Path 131 | total_timesteps 1746.
Path 132 | total_timesteps 1761.
Path 133 | total_timesteps 1775.
Path 134 | total_timesteps 1788.
Path 135 | total_timesteps 1798.
Path 136 | total_timesteps 1807.
Path 137 | total_timesteps 1815.
Path 138 | total_timesteps 1823.
Path 139 | total_timesteps 1835.
Path 140 | total_timesteps 1847.
Path 141 | total_timesteps 1855.
Path 142 | total_timesteps 1863.
Path 143 | total_timesteps 1875.
Path 144 | total_timesteps 1888.
Path 145 | total_timesteps 1900.
Path 146 | total_timesteps 1911.
Path 147 | total_timesteps 1930.
Path 148 | total_timesteps 1941.
Path 149 | total_timesteps 1956.
Path 150 | total_timesteps 1964.
Path 151 | total_timesteps 1970.
Path 152 | total_timesteps 1987.
Path 153 | total_timesteps 2002.
Path 154 | total_timesteps 2012.
Path 155 | total_timesteps 2023.
Path 156 | total_timesteps 2032.
Path 157 | total_timesteps 2041.
Path 158 | total_timesteps 2053.
Path 159 | total_timesteps 2066.
Path 160 | total_timesteps 2086.
Path 161 | total_timesteps 2098.
Path 162 | total_timesteps 2116.
Path 163 | total_timesteps 2124.
Path 164 | total_timesteps 2136.
Path 165 | total_timesteps 2149.
Path 166 | total_timesteps 2160.
Path 167 | total_timesteps 2171.
Path 168 | total_timesteps 2181.
Path 169 | total_timesteps 2199.
Path 170 | total_timesteps 2209.
Path 171 | total_timesteps 2232.
Path 172 | total_timesteps 2248.
Path 173 | total_timesteps 2258.
Path 174 | total_timesteps 2272.
Path 175 | total_timesteps 2286.
Path 176 | total_timesteps 2299.
Path 177 | total_timesteps 2309.
Path 178 | total_timesteps 2326.
Path 179 | total_timesteps 2337.
Path 180 | total_timesteps 2348.
Path 181 | total_timesteps 2362.
Path 182 | total_timesteps 2374.
Path 183 | total_timesteps 2391.
Path 184 | total_timesteps 2403.
Path 185 | total_timesteps 2414.
Path 186 | total_timesteps 2422.
Path 187 | total_timesteps 2435.
Path 188 | total_timesteps 2449.
Path 189 | total_timesteps 2459.
Path 190 | total_timesteps 2476.
Path 191 | total_timesteps 2494.
Path 192 | total_timesteps 2506.
Path 193 | total_timesteps 2514.
Path 194 | total_timesteps 2525.
Path 195 | total_timesteps 2542.
Path 196 | total_timesteps 2552.
Path 197 | total_timesteps 2573.
Path 198 | total_timesteps 2587.
Path 199 | total_timesteps 2606.
Path 200 | total_timesteps 2612.
Path 201 | total_timesteps 2622.
Path 202 | total_timesteps 2634.
Path 203 | total_timesteps 2642.
Path 204 | total_timesteps 2655.
Path 205 | total_timesteps 2663.
Path 206 | total_timesteps 2673.
Path 207 | total_timesteps 2681.
Path 208 | total_timesteps 2700.
Path 209 | total_timesteps 2716.
Path 210 | total_timesteps 2731.
Path 211 | total_timesteps 2742.
Path 212 | total_timesteps 2755.
Path 213 | total_timesteps 2770.
Path 214 | total_timesteps 2779.
Path 215 | total_timesteps 2789.
Path 216 | total_timesteps 2805.
Path 217 | total_timesteps 2819.
Path 218 | total_timesteps 2839.
Path 219 | total_timesteps 2853.
Path 220 | total_timesteps 2863.
Path 221 | total_timesteps 2886.
Path 222 | total_timesteps 2897.
Path 223 | total_timesteps 2909.
Path 224 | total_timesteps 2919.
Path 225 | total_timesteps 2938.
Path 226 | total_timesteps 2946.
Path 227 | total_timesteps 2963.
Path 228 | total_timesteps 2973.
Path 229 | total_timesteps 2988.
Path 230 | total_timesteps 3005.
Path 231 | total_timesteps 3013.
Path 232 | total_timesteps 3030.
Path 233 | total_timesteps 3041.
Path 234 | total_timesteps 3048.
Path 235 | total_timesteps 3059.
Path 236 | total_timesteps 3070.
Path 237 | total_timesteps 3083.
Path 238 | total_timesteps 3099.
Path 239 | total_timesteps 3116.
Path 240 | total_timesteps 3128.
Path 241 | total_timesteps 3144.
Path 242 | total_timesteps 3153.
Path 243 | total_timesteps 3169.
Path 244 | total_timesteps 3181.
Path 245 | total_timesteps 3196.
Path 246 | total_timesteps 3214.
Path 247 | total_timesteps 3225.
Path 248 | total_timesteps 3243.
Path 249 | total_timesteps 3254.
Path 250 | total_timesteps 3266.
Path 251 | total_timesteps 3281.
Path 252 | total_timesteps 3303.
Path 253 | total_timesteps 3314.
Path 254 | total_timesteps 3325.
Path 255 | total_timesteps 3338.
Path 256 | total_timesteps 3351.
Path 257 | total_timesteps 3365.
Path 258 | total_timesteps 3378.
Path 259 | total_timesteps 3393.
Path 260 | total_timesteps 3407.
Path 261 | total_timesteps 3415.
Path 262 | total_timesteps 3431.
Path 263 | total_timesteps 3441.
Path 264 | total_timesteps 3456.
Path 265 | total_timesteps 3469.
Path 266 | total_timesteps 3479.
Path 267 | total_timesteps 3489.
Path 268 | total_timesteps 3502.
Path 269 | total_timesteps 3513.
Path 270 | total_timesteps 3521.
Path 271 | total_timesteps 3539.
Path 272 | total_timesteps 3553.
Path 273 | total_timesteps 3574.
Path 274 | total_timesteps 3585.
Path 275 | total_timesteps 3604.
Path 276 | total_timesteps 3620.
Path 277 | total_timesteps 3635.
Path 278 | total_timesteps 3644.
Path 279 | total_timesteps 3656.
Path 280 | total_timesteps 3673.
Path 281 | total_timesteps 3691.
Path 282 | total_timesteps 3709.
Path 283 | total_timesteps 3721.
Path 284 | total_timesteps 3740.
Path 285 | total_timesteps 3754.
Path 286 | total_timesteps 3767.
Path 287 | total_timesteps 3775.
Path 288 | total_timesteps 3794.
Path 289 | total_timesteps 3806.
Path 290 | total_timesteps 3819.
Path 291 | total_timesteps 3836.
Path 292 | total_timesteps 3847.
Path 293 | total_timesteps 3862.
Path 294 | total_timesteps 3879.
Path 295 | total_timesteps 3893.
Path 296 | total_timesteps 3903.
Path 297 | total_timesteps 3919.
Path 298 | total_timesteps 3937.
Path 299 | total_timesteps 3948.
Path 300 | total_timesteps 3962.
Path 301 | total_timesteps 3984.
Path 302 | total_timesteps 3995.
Path 303 | total_timesteps 4004.
Path 304 | total_timesteps 4018.
Path 305 | total_timesteps 4034.
Path 306 | total_timesteps 4042.
Path 307 | total_timesteps 4053.
Path 308 | total_timesteps 4071.
Path 309 | total_timesteps 4083.
Path 310 | total_timesteps 4090.
Path 311 | total_timesteps 4108.
Path 312 | total_timesteps 4121.
Path 313 | total_timesteps 4137.
Path 314 | total_timesteps 4149.
Path 315 | total_timesteps 4160.
Path 316 | total_timesteps 4169.
Path 317 | total_timesteps 4186.
Path 318 | total_timesteps 4200.
Path 319 | total_timesteps 4207.
Path 320 | total_timesteps 4220.
Path 321 | total_timesteps 4229.
Path 322 | total_timesteps 4242.
Path 323 | total_timesteps 4259.
Path 324 | total_timesteps 4273.
Path 325 | total_timesteps 4286.
Path 326 | total_timesteps 4296.
Path 327 | total_timesteps 4318.
Path 328 | total_timesteps 4331.
Path 329 | total_timesteps 4344.
Path 330 | total_timesteps 4363.
Path 331 | total_timesteps 4378.
Path 332 | total_timesteps 4398.
Path 333 | total_timesteps 4414.
Path 334 | total_timesteps 4422.
Path 335 | total_timesteps 4440.
Path 336 | total_timesteps 4454.
Path 337 | total_timesteps 4464.
Path 338 | total_timesteps 4480.
Path 339 | total_timesteps 4494.
Path 340 | total_timesteps 4505.
Path 341 | total_timesteps 4514.
Path 342 | total_timesteps 4523.
Path 343 | total_timesteps 4533.
Path 344 | total_timesteps 4546.
Path 345 | total_timesteps 4557.
Path 346 | total_timesteps 4575.
Path 347 | total_timesteps 4583.
Path 348 | total_timesteps 4596.
Path 349 | total_timesteps 4605.
Path 350 | total_timesteps 4616.
Path 351 | total_timesteps 4628.
Path 352 | total_timesteps 4637.
Path 353 | total_timesteps 4650.
Path 354 | total_timesteps 4663.
Path 355 | total_timesteps 4676.
Path 356 | total_timesteps 4694.
Path 357 | total_timesteps 4716.
Path 358 | total_timesteps 4725.
Path 359 | total_timesteps 4736.
Path 360 | total_timesteps 4753.
Path 361 | total_timesteps 4767.
Path 362 | total_timesteps 4782.
Path 363 | total_timesteps 4796.
Path 364 | total_timesteps 4807.
Path 365 | total_timesteps 4818.
Path 366 | total_timesteps 4828.
Path 367 | total_timesteps 4838.
Path 368 | total_timesteps 4853.
Path 369 | total_timesteps 4865.
Path 370 | total_timesteps 4880.
Path 371 | total_timesteps 4890.
Path 372 | total_timesteps 4901.
Path 373 | total_timesteps 4911.
Path 374 | total_timesteps 4924.
Path 375 | total_timesteps 4935.
Path 376 | total_timesteps 4951.
Path 377 | total_timesteps 4960.
Path 378 | total_timesteps 4975.
Path 379 | total_timesteps 4984.
Path 380 | total_timesteps 4996.
Path 381 | total_timesteps 5005.
Path 382 | total_timesteps 5016.
Path 383 | total_timesteps 5032.
Path 384 | total_timesteps 5043.
Path 385 | total_timesteps 5053.
Path 386 | total_timesteps 5065.
Path 387 | total_timesteps 5080.
Path 388 | total_timesteps 5089.
Path 389 | total_timesteps 5104.
Path 390 | total_timesteps 5114.
Path 391 | total_timesteps 5132.
Path 392 | total_timesteps 5144.
Path 393 | total_timesteps 5156.
Path 394 | total_timesteps 5164.
Path 395 | total_timesteps 5179.
Path 396 | total_timesteps 5192.
Path 397 | total_timesteps 5210.
Path 398 | total_timesteps 5224.
Path 399 | total_timesteps 5235.
Path 400 | total_timesteps 5253.
Path 401 | total_timesteps 5267.
Path 402 | total_timesteps 5282.
Path 403 | total_timesteps 5293.
Path 404 | total_timesteps 5305.
Path 405 | total_timesteps 5316.
Path 406 | total_timesteps 5327.
Path 407 | total_timesteps 5346.
Path 408 | total_timesteps 5359.
Path 409 | total_timesteps 5372.
Path 410 | total_timesteps 5388.
Path 411 | total_timesteps 5399.
Path 412 | total_timesteps 5414.
Path 413 | total_timesteps 5438.
Path 414 | total_timesteps 5453.
Path 415 | total_timesteps 5471.
Path 416 | total_timesteps 5489.
Path 417 | total_timesteps 5500.
Path 418 | total_timesteps 5506.
Path 419 | total_timesteps 5528.
Path 420 | total_timesteps 5542.
Path 421 | total_timesteps 5557.
Path 422 | total_timesteps 5567.
Path 423 | total_timesteps 5574.
Path 424 | total_timesteps 5591.
Path 425 | total_timesteps 5608.
Path 426 | total_timesteps 5627.
Path 427 | total_timesteps 5646.
Path 428 | total_timesteps 5663.
Path 429 | total_timesteps 5672.
Path 430 | total_timesteps 5683.
Path 431 | total_timesteps 5693.
Path 432 | total_timesteps 5710.
Path 433 | total_timesteps 5724.
Path 434 | total_timesteps 5741.
Path 435 | total_timesteps 5751.
Path 436 | total_timesteps 5764.
Path 437 | total_timesteps 5776.
Path 438 | total_timesteps 5789.
Path 439 | total_timesteps 5799.
Path 440 | total_timesteps 5823.
Path 441 | total_timesteps 5845.
Path 442 | total_timesteps 5854.
Path 443 | total_timesteps 5867.
Path 444 | total_timesteps 5878.
Path 445 | total_timesteps 5886.
Path 446 | total_timesteps 5904.
Path 447 | total_timesteps 5917.
Path 448 | total_timesteps 5927.
Path 449 | total_timesteps 5938.
Path 450 | total_timesteps 5953.
Path 451 | total_timesteps 5972.
Path 452 | total_timesteps 5982.
Path 453 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.15    |
| Iteration     | 13       |
| MaximumReturn | 3.37     |
| MinimumReturn | -20.2    |
| TotalSamples  | 60071    |
----------------------------
itr #14 | 
Fitting dynamics.
Validation loss = 0.009076828137040138
Validation loss = 0.008167668245732784
Validation loss = 0.009153525345027447
Validation loss = 0.008342946879565716
Validation loss = 0.007902598939836025
Validation loss = 0.009012366645038128
Validation loss = 0.008080835454165936
Validation loss = 0.008172068744897842
Validation loss = 0.008128106594085693
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 25.
Path 3 | total_timesteps 59.
Path 4 | total_timesteps 69.
Path 5 | total_timesteps 86.
Path 6 | total_timesteps 102.
Path 7 | total_timesteps 122.
Path 8 | total_timesteps 132.
Path 9 | total_timesteps 140.
Path 10 | total_timesteps 162.
Path 11 | total_timesteps 170.
Path 12 | total_timesteps 182.
Path 13 | total_timesteps 192.
Path 14 | total_timesteps 215.
Path 15 | total_timesteps 225.
Path 16 | total_timesteps 234.
Path 17 | total_timesteps 242.
Path 18 | total_timesteps 250.
Path 19 | total_timesteps 260.
Path 20 | total_timesteps 269.
Path 21 | total_timesteps 279.
Path 22 | total_timesteps 296.
Path 23 | total_timesteps 305.
Path 24 | total_timesteps 314.
Path 25 | total_timesteps 323.
Path 26 | total_timesteps 333.
Path 27 | total_timesteps 348.
Path 28 | total_timesteps 362.
Path 29 | total_timesteps 372.
Path 30 | total_timesteps 384.
Path 31 | total_timesteps 401.
Path 32 | total_timesteps 423.
Path 33 | total_timesteps 440.
Path 34 | total_timesteps 463.
Path 35 | total_timesteps 473.
Path 36 | total_timesteps 483.
Path 37 | total_timesteps 495.
Path 38 | total_timesteps 505.
Path 39 | total_timesteps 516.
Path 40 | total_timesteps 529.
Path 41 | total_timesteps 540.
Path 42 | total_timesteps 551.
Path 43 | total_timesteps 560.
Path 44 | total_timesteps 573.
Path 45 | total_timesteps 589.
Path 46 | total_timesteps 606.
Path 47 | total_timesteps 617.
Path 48 | total_timesteps 628.
Path 49 | total_timesteps 642.
Path 50 | total_timesteps 659.
Path 51 | total_timesteps 675.
Path 52 | total_timesteps 684.
Path 53 | total_timesteps 694.
Path 54 | total_timesteps 712.
Path 55 | total_timesteps 721.
Path 56 | total_timesteps 731.
Path 57 | total_timesteps 743.
Path 58 | total_timesteps 755.
Path 59 | total_timesteps 766.
Path 60 | total_timesteps 779.
Path 61 | total_timesteps 793.
Path 62 | total_timesteps 803.
Path 63 | total_timesteps 811.
Path 64 | total_timesteps 822.
Path 65 | total_timesteps 831.
Path 66 | total_timesteps 842.
Path 67 | total_timesteps 858.
Path 68 | total_timesteps 875.
Path 69 | total_timesteps 887.
Path 70 | total_timesteps 909.
Path 71 | total_timesteps 919.
Path 72 | total_timesteps 933.
Path 73 | total_timesteps 949.
Path 74 | total_timesteps 958.
Path 75 | total_timesteps 974.
Path 76 | total_timesteps 983.
Path 77 | total_timesteps 994.
Path 78 | total_timesteps 1006.
Path 79 | total_timesteps 1015.
Path 80 | total_timesteps 1030.
Path 81 | total_timesteps 1040.
Path 82 | total_timesteps 1055.
Path 83 | total_timesteps 1069.
Path 84 | total_timesteps 1082.
Path 85 | total_timesteps 1101.
Path 86 | total_timesteps 1112.
Path 87 | total_timesteps 1122.
Path 88 | total_timesteps 1132.
Path 89 | total_timesteps 1140.
Path 90 | total_timesteps 1152.
Path 91 | total_timesteps 1163.
Path 92 | total_timesteps 1179.
Path 93 | total_timesteps 1195.
Path 94 | total_timesteps 1205.
Path 95 | total_timesteps 1212.
Path 96 | total_timesteps 1226.
Path 97 | total_timesteps 1238.
Path 98 | total_timesteps 1259.
Path 99 | total_timesteps 1268.
Path 100 | total_timesteps 1299.
Path 101 | total_timesteps 1319.
Path 102 | total_timesteps 1332.
Path 103 | total_timesteps 1353.
Path 104 | total_timesteps 1366.
Path 105 | total_timesteps 1378.
Path 106 | total_timesteps 1393.
Path 107 | total_timesteps 1404.
Path 108 | total_timesteps 1420.
Path 109 | total_timesteps 1430.
Path 110 | total_timesteps 1446.
Path 111 | total_timesteps 1462.
Path 112 | total_timesteps 1475.
Path 113 | total_timesteps 1490.
Path 114 | total_timesteps 1499.
Path 115 | total_timesteps 1508.
Path 116 | total_timesteps 1522.
Path 117 | total_timesteps 1529.
Path 118 | total_timesteps 1547.
Path 119 | total_timesteps 1557.
Path 120 | total_timesteps 1573.
Path 121 | total_timesteps 1590.
Path 122 | total_timesteps 1605.
Path 123 | total_timesteps 1617.
Path 124 | total_timesteps 1633.
Path 125 | total_timesteps 1642.
Path 126 | total_timesteps 1652.
Path 127 | total_timesteps 1662.
Path 128 | total_timesteps 1677.
Path 129 | total_timesteps 1690.
Path 130 | total_timesteps 1722.
Path 131 | total_timesteps 1735.
Path 132 | total_timesteps 1748.
Path 133 | total_timesteps 1767.
Path 134 | total_timesteps 1780.
Path 135 | total_timesteps 1789.
Path 136 | total_timesteps 1799.
Path 137 | total_timesteps 1806.
Path 138 | total_timesteps 1815.
Path 139 | total_timesteps 1822.
Path 140 | total_timesteps 1834.
Path 141 | total_timesteps 1843.
Path 142 | total_timesteps 1859.
Path 143 | total_timesteps 1875.
Path 144 | total_timesteps 1882.
Path 145 | total_timesteps 1891.
Path 146 | total_timesteps 1903.
Path 147 | total_timesteps 1914.
Path 148 | total_timesteps 1923.
Path 149 | total_timesteps 1937.
Path 150 | total_timesteps 1951.
Path 151 | total_timesteps 1962.
Path 152 | total_timesteps 1971.
Path 153 | total_timesteps 1986.
Path 154 | total_timesteps 2003.
Path 155 | total_timesteps 2014.
Path 156 | total_timesteps 2025.
Path 157 | total_timesteps 2033.
Path 158 | total_timesteps 2045.
Path 159 | total_timesteps 2062.
Path 160 | total_timesteps 2069.
Path 161 | total_timesteps 2084.
Path 162 | total_timesteps 2097.
Path 163 | total_timesteps 2112.
Path 164 | total_timesteps 2120.
Path 165 | total_timesteps 2130.
Path 166 | total_timesteps 2143.
Path 167 | total_timesteps 2154.
Path 168 | total_timesteps 2173.
Path 169 | total_timesteps 2184.
Path 170 | total_timesteps 2202.
Path 171 | total_timesteps 2217.
Path 172 | total_timesteps 2237.
Path 173 | total_timesteps 2252.
Path 174 | total_timesteps 2266.
Path 175 | total_timesteps 2279.
Path 176 | total_timesteps 2291.
Path 177 | total_timesteps 2303.
Path 178 | total_timesteps 2318.
Path 179 | total_timesteps 2330.
Path 180 | total_timesteps 2339.
Path 181 | total_timesteps 2352.
Path 182 | total_timesteps 2367.
Path 183 | total_timesteps 2383.
Path 184 | total_timesteps 2401.
Path 185 | total_timesteps 2416.
Path 186 | total_timesteps 2433.
Path 187 | total_timesteps 2442.
Path 188 | total_timesteps 2452.
Path 189 | total_timesteps 2463.
Path 190 | total_timesteps 2474.
Path 191 | total_timesteps 2488.
Path 192 | total_timesteps 2502.
Path 193 | total_timesteps 2516.
Path 194 | total_timesteps 2528.
Path 195 | total_timesteps 2538.
Path 196 | total_timesteps 2546.
Path 197 | total_timesteps 2557.
Path 198 | total_timesteps 2565.
Path 199 | total_timesteps 2575.
Path 200 | total_timesteps 2586.
Path 201 | total_timesteps 2600.
Path 202 | total_timesteps 2610.
Path 203 | total_timesteps 2624.
Path 204 | total_timesteps 2641.
Path 205 | total_timesteps 2652.
Path 206 | total_timesteps 2662.
Path 207 | total_timesteps 2671.
Path 208 | total_timesteps 2683.
Path 209 | total_timesteps 2693.
Path 210 | total_timesteps 2703.
Path 211 | total_timesteps 2718.
Path 212 | total_timesteps 2726.
Path 213 | total_timesteps 2737.
Path 214 | total_timesteps 2749.
Path 215 | total_timesteps 2772.
Path 216 | total_timesteps 2784.
Path 217 | total_timesteps 2803.
Path 218 | total_timesteps 2811.
Path 219 | total_timesteps 2826.
Path 220 | total_timesteps 2841.
Path 221 | total_timesteps 2858.
Path 222 | total_timesteps 2873.
Path 223 | total_timesteps 2884.
Path 224 | total_timesteps 2893.
Path 225 | total_timesteps 2903.
Path 226 | total_timesteps 2914.
Path 227 | total_timesteps 2925.
Path 228 | total_timesteps 2934.
Path 229 | total_timesteps 2950.
Path 230 | total_timesteps 2959.
Path 231 | total_timesteps 2971.
Path 232 | total_timesteps 2980.
Path 233 | total_timesteps 2990.
Path 234 | total_timesteps 3005.
Path 235 | total_timesteps 3018.
Path 236 | total_timesteps 3027.
Path 237 | total_timesteps 3039.
Path 238 | total_timesteps 3053.
Path 239 | total_timesteps 3068.
Path 240 | total_timesteps 3077.
Path 241 | total_timesteps 3086.
Path 242 | total_timesteps 3096.
Path 243 | total_timesteps 3111.
Path 244 | total_timesteps 3120.
Path 245 | total_timesteps 3135.
Path 246 | total_timesteps 3152.
Path 247 | total_timesteps 3166.
Path 248 | total_timesteps 3175.
Path 249 | total_timesteps 3189.
Path 250 | total_timesteps 3196.
Path 251 | total_timesteps 3207.
Path 252 | total_timesteps 3215.
Path 253 | total_timesteps 3230.
Path 254 | total_timesteps 3242.
Path 255 | total_timesteps 3250.
Path 256 | total_timesteps 3265.
Path 257 | total_timesteps 3275.
Path 258 | total_timesteps 3289.
Path 259 | total_timesteps 3303.
Path 260 | total_timesteps 3317.
Path 261 | total_timesteps 3328.
Path 262 | total_timesteps 3347.
Path 263 | total_timesteps 3361.
Path 264 | total_timesteps 3372.
Path 265 | total_timesteps 3386.
Path 266 | total_timesteps 3397.
Path 267 | total_timesteps 3412.
Path 268 | total_timesteps 3439.
Path 269 | total_timesteps 3456.
Path 270 | total_timesteps 3463.
Path 271 | total_timesteps 3478.
Path 272 | total_timesteps 3492.
Path 273 | total_timesteps 3498.
Path 274 | total_timesteps 3509.
Path 275 | total_timesteps 3518.
Path 276 | total_timesteps 3532.
Path 277 | total_timesteps 3546.
Path 278 | total_timesteps 3559.
Path 279 | total_timesteps 3571.
Path 280 | total_timesteps 3586.
Path 281 | total_timesteps 3596.
Path 282 | total_timesteps 3604.
Path 283 | total_timesteps 3611.
Path 284 | total_timesteps 3628.
Path 285 | total_timesteps 3639.
Path 286 | total_timesteps 3650.
Path 287 | total_timesteps 3660.
Path 288 | total_timesteps 3679.
Path 289 | total_timesteps 3699.
Path 290 | total_timesteps 3707.
Path 291 | total_timesteps 3720.
Path 292 | total_timesteps 3735.
Path 293 | total_timesteps 3746.
Path 294 | total_timesteps 3758.
Path 295 | total_timesteps 3767.
Path 296 | total_timesteps 3780.
Path 297 | total_timesteps 3791.
Path 298 | total_timesteps 3803.
Path 299 | total_timesteps 3818.
Path 300 | total_timesteps 3832.
Path 301 | total_timesteps 3846.
Path 302 | total_timesteps 3857.
Path 303 | total_timesteps 3881.
Path 304 | total_timesteps 3890.
Path 305 | total_timesteps 3907.
Path 306 | total_timesteps 3925.
Path 307 | total_timesteps 3936.
Path 308 | total_timesteps 3956.
Path 309 | total_timesteps 3966.
Path 310 | total_timesteps 3974.
Path 311 | total_timesteps 3983.
Path 312 | total_timesteps 3999.
Path 313 | total_timesteps 4008.
Path 314 | total_timesteps 4022.
Path 315 | total_timesteps 4034.
Path 316 | total_timesteps 4042.
Path 317 | total_timesteps 4049.
Path 318 | total_timesteps 4066.
Path 319 | total_timesteps 4074.
Path 320 | total_timesteps 4088.
Path 321 | total_timesteps 4101.
Path 322 | total_timesteps 4111.
Path 323 | total_timesteps 4127.
Path 324 | total_timesteps 4135.
Path 325 | total_timesteps 4145.
Path 326 | total_timesteps 4157.
Path 327 | total_timesteps 4173.
Path 328 | total_timesteps 4186.
Path 329 | total_timesteps 4193.
Path 330 | total_timesteps 4207.
Path 331 | total_timesteps 4215.
Path 332 | total_timesteps 4226.
Path 333 | total_timesteps 4241.
Path 334 | total_timesteps 4250.
Path 335 | total_timesteps 4259.
Path 336 | total_timesteps 4270.
Path 337 | total_timesteps 4286.
Path 338 | total_timesteps 4295.
Path 339 | total_timesteps 4304.
Path 340 | total_timesteps 4314.
Path 341 | total_timesteps 4324.
Path 342 | total_timesteps 4338.
Path 343 | total_timesteps 4353.
Path 344 | total_timesteps 4367.
Path 345 | total_timesteps 4378.
Path 346 | total_timesteps 4385.
Path 347 | total_timesteps 4394.
Path 348 | total_timesteps 4404.
Path 349 | total_timesteps 4426.
Path 350 | total_timesteps 4449.
Path 351 | total_timesteps 4465.
Path 352 | total_timesteps 4476.
Path 353 | total_timesteps 4495.
Path 354 | total_timesteps 4510.
Path 355 | total_timesteps 4520.
Path 356 | total_timesteps 4537.
Path 357 | total_timesteps 4549.
Path 358 | total_timesteps 4559.
Path 359 | total_timesteps 4572.
Path 360 | total_timesteps 4589.
Path 361 | total_timesteps 4604.
Path 362 | total_timesteps 4615.
Path 363 | total_timesteps 4629.
Path 364 | total_timesteps 4638.
Path 365 | total_timesteps 4650.
Path 366 | total_timesteps 4663.
Path 367 | total_timesteps 4675.
Path 368 | total_timesteps 4685.
Path 369 | total_timesteps 4693.
Path 370 | total_timesteps 4706.
Path 371 | total_timesteps 4716.
Path 372 | total_timesteps 4728.
Path 373 | total_timesteps 4736.
Path 374 | total_timesteps 4744.
Path 375 | total_timesteps 4754.
Path 376 | total_timesteps 4761.
Path 377 | total_timesteps 4776.
Path 378 | total_timesteps 4787.
Path 379 | total_timesteps 4803.
Path 380 | total_timesteps 4813.
Path 381 | total_timesteps 4823.
Path 382 | total_timesteps 4842.
Path 383 | total_timesteps 4854.
Path 384 | total_timesteps 4867.
Path 385 | total_timesteps 4877.
Path 386 | total_timesteps 4891.
Path 387 | total_timesteps 4900.
Path 388 | total_timesteps 4910.
Path 389 | total_timesteps 4923.
Path 390 | total_timesteps 4934.
Path 391 | total_timesteps 4949.
Path 392 | total_timesteps 4962.
Path 393 | total_timesteps 4974.
Path 394 | total_timesteps 4987.
Path 395 | total_timesteps 4995.
Path 396 | total_timesteps 5008.
Path 397 | total_timesteps 5019.
Path 398 | total_timesteps 5037.
Path 399 | total_timesteps 5049.
Path 400 | total_timesteps 5058.
Path 401 | total_timesteps 5069.
Path 402 | total_timesteps 5080.
Path 403 | total_timesteps 5093.
Path 404 | total_timesteps 5102.
Path 405 | total_timesteps 5109.
Path 406 | total_timesteps 5121.
Path 407 | total_timesteps 5132.
Path 408 | total_timesteps 5142.
Path 409 | total_timesteps 5160.
Path 410 | total_timesteps 5172.
Path 411 | total_timesteps 5182.
Path 412 | total_timesteps 5192.
Path 413 | total_timesteps 5214.
Path 414 | total_timesteps 5223.
Path 415 | total_timesteps 5234.
Path 416 | total_timesteps 5246.
Path 417 | total_timesteps 5258.
Path 418 | total_timesteps 5278.
Path 419 | total_timesteps 5289.
Path 420 | total_timesteps 5306.
Path 421 | total_timesteps 5319.
Path 422 | total_timesteps 5328.
Path 423 | total_timesteps 5338.
Path 424 | total_timesteps 5352.
Path 425 | total_timesteps 5365.
Path 426 | total_timesteps 5379.
Path 427 | total_timesteps 5389.
Path 428 | total_timesteps 5401.
Path 429 | total_timesteps 5412.
Path 430 | total_timesteps 5430.
Path 431 | total_timesteps 5445.
Path 432 | total_timesteps 5466.
Path 433 | total_timesteps 5477.
Path 434 | total_timesteps 5488.
Path 435 | total_timesteps 5499.
Path 436 | total_timesteps 5515.
Path 437 | total_timesteps 5524.
Path 438 | total_timesteps 5533.
Path 439 | total_timesteps 5546.
Path 440 | total_timesteps 5554.
Path 441 | total_timesteps 5569.
Path 442 | total_timesteps 5583.
Path 443 | total_timesteps 5592.
Path 444 | total_timesteps 5610.
Path 445 | total_timesteps 5624.
Path 446 | total_timesteps 5637.
Path 447 | total_timesteps 5653.
Path 448 | total_timesteps 5665.
Path 449 | total_timesteps 5686.
Path 450 | total_timesteps 5701.
Path 451 | total_timesteps 5712.
Path 452 | total_timesteps 5722.
Path 453 | total_timesteps 5737.
Path 454 | total_timesteps 5749.
Path 455 | total_timesteps 5757.
Path 456 | total_timesteps 5782.
Path 457 | total_timesteps 5793.
Path 458 | total_timesteps 5804.
Path 459 | total_timesteps 5814.
Path 460 | total_timesteps 5829.
Path 461 | total_timesteps 5838.
Path 462 | total_timesteps 5846.
Path 463 | total_timesteps 5868.
Path 464 | total_timesteps 5883.
Path 465 | total_timesteps 5901.
Path 466 | total_timesteps 5909.
Path 467 | total_timesteps 5924.
Path 468 | total_timesteps 5934.
Path 469 | total_timesteps 5945.
Path 470 | total_timesteps 5964.
Path 471 | total_timesteps 5984.
Path 472 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.45    |
| Iteration     | 14       |
| MaximumReturn | -1.46    |
| MinimumReturn | -20.6    |
| TotalSamples  | 64076    |
----------------------------
itr #15 | 
Fitting dynamics.
Validation loss = 0.010014694184064865
Validation loss = 0.0075037735514342785
Validation loss = 0.008717360906302929
Validation loss = 0.007856938056647778
Validation loss = 0.00840082298964262
Validation loss = 0.007919751107692719
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 38.
Path 4 | total_timesteps 47.
Path 5 | total_timesteps 55.
Path 6 | total_timesteps 75.
Path 7 | total_timesteps 92.
Path 8 | total_timesteps 101.
Path 9 | total_timesteps 110.
Path 10 | total_timesteps 123.
Path 11 | total_timesteps 140.
Path 12 | total_timesteps 162.
Path 13 | total_timesteps 177.
Path 14 | total_timesteps 186.
Path 15 | total_timesteps 199.
Path 16 | total_timesteps 214.
Path 17 | total_timesteps 232.
Path 18 | total_timesteps 244.
Path 19 | total_timesteps 252.
Path 20 | total_timesteps 266.
Path 21 | total_timesteps 279.
Path 22 | total_timesteps 299.
Path 23 | total_timesteps 309.
Path 24 | total_timesteps 325.
Path 25 | total_timesteps 336.
Path 26 | total_timesteps 353.
Path 27 | total_timesteps 367.
Path 28 | total_timesteps 380.
Path 29 | total_timesteps 393.
Path 30 | total_timesteps 408.
Path 31 | total_timesteps 426.
Path 32 | total_timesteps 435.
Path 33 | total_timesteps 449.
Path 34 | total_timesteps 456.
Path 35 | total_timesteps 468.
Path 36 | total_timesteps 477.
Path 37 | total_timesteps 488.
Path 38 | total_timesteps 498.
Path 39 | total_timesteps 511.
Path 40 | total_timesteps 523.
Path 41 | total_timesteps 531.
Path 42 | total_timesteps 555.
Path 43 | total_timesteps 568.
Path 44 | total_timesteps 585.
Path 45 | total_timesteps 597.
Path 46 | total_timesteps 613.
Path 47 | total_timesteps 630.
Path 48 | total_timesteps 638.
Path 49 | total_timesteps 647.
Path 50 | total_timesteps 661.
Path 51 | total_timesteps 682.
Path 52 | total_timesteps 697.
Path 53 | total_timesteps 713.
Path 54 | total_timesteps 730.
Path 55 | total_timesteps 746.
Path 56 | total_timesteps 758.
Path 57 | total_timesteps 770.
Path 58 | total_timesteps 783.
Path 59 | total_timesteps 793.
Path 60 | total_timesteps 802.
Path 61 | total_timesteps 811.
Path 62 | total_timesteps 820.
Path 63 | total_timesteps 830.
Path 64 | total_timesteps 837.
Path 65 | total_timesteps 848.
Path 66 | total_timesteps 859.
Path 67 | total_timesteps 869.
Path 68 | total_timesteps 880.
Path 69 | total_timesteps 892.
Path 70 | total_timesteps 907.
Path 71 | total_timesteps 921.
Path 72 | total_timesteps 937.
Path 73 | total_timesteps 957.
Path 74 | total_timesteps 966.
Path 75 | total_timesteps 980.
Path 76 | total_timesteps 990.
Path 77 | total_timesteps 1005.
Path 78 | total_timesteps 1015.
Path 79 | total_timesteps 1032.
Path 80 | total_timesteps 1043.
Path 81 | total_timesteps 1056.
Path 82 | total_timesteps 1066.
Path 83 | total_timesteps 1085.
Path 84 | total_timesteps 1098.
Path 85 | total_timesteps 1110.
Path 86 | total_timesteps 1123.
Path 87 | total_timesteps 1134.
Path 88 | total_timesteps 1148.
Path 89 | total_timesteps 1161.
Path 90 | total_timesteps 1177.
Path 91 | total_timesteps 1187.
Path 92 | total_timesteps 1201.
Path 93 | total_timesteps 1211.
Path 94 | total_timesteps 1222.
Path 95 | total_timesteps 1233.
Path 96 | total_timesteps 1256.
Path 97 | total_timesteps 1278.
Path 98 | total_timesteps 1296.
Path 99 | total_timesteps 1306.
Path 100 | total_timesteps 1318.
Path 101 | total_timesteps 1330.
Path 102 | total_timesteps 1341.
Path 103 | total_timesteps 1357.
Path 104 | total_timesteps 1367.
Path 105 | total_timesteps 1380.
Path 106 | total_timesteps 1388.
Path 107 | total_timesteps 1397.
Path 108 | total_timesteps 1412.
Path 109 | total_timesteps 1425.
Path 110 | total_timesteps 1432.
Path 111 | total_timesteps 1445.
Path 112 | total_timesteps 1457.
Path 113 | total_timesteps 1472.
Path 114 | total_timesteps 1496.
Path 115 | total_timesteps 1513.
Path 116 | total_timesteps 1526.
Path 117 | total_timesteps 1537.
Path 118 | total_timesteps 1556.
Path 119 | total_timesteps 1570.
Path 120 | total_timesteps 1576.
Path 121 | total_timesteps 1591.
Path 122 | total_timesteps 1609.
Path 123 | total_timesteps 1627.
Path 124 | total_timesteps 1644.
Path 125 | total_timesteps 1656.
Path 126 | total_timesteps 1663.
Path 127 | total_timesteps 1674.
Path 128 | total_timesteps 1685.
Path 129 | total_timesteps 1702.
Path 130 | total_timesteps 1714.
Path 131 | total_timesteps 1732.
Path 132 | total_timesteps 1744.
Path 133 | total_timesteps 1762.
Path 134 | total_timesteps 1781.
Path 135 | total_timesteps 1788.
Path 136 | total_timesteps 1795.
Path 137 | total_timesteps 1809.
Path 138 | total_timesteps 1821.
Path 139 | total_timesteps 1835.
Path 140 | total_timesteps 1850.
Path 141 | total_timesteps 1859.
Path 142 | total_timesteps 1867.
Path 143 | total_timesteps 1880.
Path 144 | total_timesteps 1887.
Path 145 | total_timesteps 1903.
Path 146 | total_timesteps 1913.
Path 147 | total_timesteps 1929.
Path 148 | total_timesteps 1961.
Path 149 | total_timesteps 1978.
Path 150 | total_timesteps 1988.
Path 151 | total_timesteps 1996.
Path 152 | total_timesteps 2003.
Path 153 | total_timesteps 2023.
Path 154 | total_timesteps 2031.
Path 155 | total_timesteps 2051.
Path 156 | total_timesteps 2066.
Path 157 | total_timesteps 2081.
Path 158 | total_timesteps 2090.
Path 159 | total_timesteps 2105.
Path 160 | total_timesteps 2117.
Path 161 | total_timesteps 2127.
Path 162 | total_timesteps 2141.
Path 163 | total_timesteps 2162.
Path 164 | total_timesteps 2182.
Path 165 | total_timesteps 2189.
Path 166 | total_timesteps 2209.
Path 167 | total_timesteps 2226.
Path 168 | total_timesteps 2243.
Path 169 | total_timesteps 2260.
Path 170 | total_timesteps 2271.
Path 171 | total_timesteps 2290.
Path 172 | total_timesteps 2302.
Path 173 | total_timesteps 2310.
Path 174 | total_timesteps 2327.
Path 175 | total_timesteps 2343.
Path 176 | total_timesteps 2369.
Path 177 | total_timesteps 2384.
Path 178 | total_timesteps 2394.
Path 179 | total_timesteps 2413.
Path 180 | total_timesteps 2428.
Path 181 | total_timesteps 2442.
Path 182 | total_timesteps 2453.
Path 183 | total_timesteps 2470.
Path 184 | total_timesteps 2481.
Path 185 | total_timesteps 2492.
Path 186 | total_timesteps 2509.
Path 187 | total_timesteps 2520.
Path 188 | total_timesteps 2529.
Path 189 | total_timesteps 2547.
Path 190 | total_timesteps 2560.
Path 191 | total_timesteps 2573.
Path 192 | total_timesteps 2584.
Path 193 | total_timesteps 2593.
Path 194 | total_timesteps 2604.
Path 195 | total_timesteps 2614.
Path 196 | total_timesteps 2627.
Path 197 | total_timesteps 2642.
Path 198 | total_timesteps 2657.
Path 199 | total_timesteps 2671.
Path 200 | total_timesteps 2681.
Path 201 | total_timesteps 2690.
Path 202 | total_timesteps 2710.
Path 203 | total_timesteps 2721.
Path 204 | total_timesteps 2736.
Path 205 | total_timesteps 2749.
Path 206 | total_timesteps 2759.
Path 207 | total_timesteps 2772.
Path 208 | total_timesteps 2789.
Path 209 | total_timesteps 2797.
Path 210 | total_timesteps 2806.
Path 211 | total_timesteps 2822.
Path 212 | total_timesteps 2832.
Path 213 | total_timesteps 2847.
Path 214 | total_timesteps 2859.
Path 215 | total_timesteps 2880.
Path 216 | total_timesteps 2887.
Path 217 | total_timesteps 2894.
Path 218 | total_timesteps 2907.
Path 219 | total_timesteps 2916.
Path 220 | total_timesteps 2931.
Path 221 | total_timesteps 2946.
Path 222 | total_timesteps 2959.
Path 223 | total_timesteps 2970.
Path 224 | total_timesteps 2984.
Path 225 | total_timesteps 2995.
Path 226 | total_timesteps 3010.
Path 227 | total_timesteps 3025.
Path 228 | total_timesteps 3041.
Path 229 | total_timesteps 3050.
Path 230 | total_timesteps 3061.
Path 231 | total_timesteps 3076.
Path 232 | total_timesteps 3086.
Path 233 | total_timesteps 3098.
Path 234 | total_timesteps 3109.
Path 235 | total_timesteps 3123.
Path 236 | total_timesteps 3133.
Path 237 | total_timesteps 3144.
Path 238 | total_timesteps 3154.
Path 239 | total_timesteps 3171.
Path 240 | total_timesteps 3192.
Path 241 | total_timesteps 3212.
Path 242 | total_timesteps 3232.
Path 243 | total_timesteps 3245.
Path 244 | total_timesteps 3254.
Path 245 | total_timesteps 3264.
Path 246 | total_timesteps 3283.
Path 247 | total_timesteps 3294.
Path 248 | total_timesteps 3310.
Path 249 | total_timesteps 3326.
Path 250 | total_timesteps 3337.
Path 251 | total_timesteps 3348.
Path 252 | total_timesteps 3359.
Path 253 | total_timesteps 3368.
Path 254 | total_timesteps 3379.
Path 255 | total_timesteps 3396.
Path 256 | total_timesteps 3406.
Path 257 | total_timesteps 3420.
Path 258 | total_timesteps 3440.
Path 259 | total_timesteps 3454.
Path 260 | total_timesteps 3466.
Path 261 | total_timesteps 3479.
Path 262 | total_timesteps 3488.
Path 263 | total_timesteps 3497.
Path 264 | total_timesteps 3510.
Path 265 | total_timesteps 3529.
Path 266 | total_timesteps 3538.
Path 267 | total_timesteps 3560.
Path 268 | total_timesteps 3572.
Path 269 | total_timesteps 3583.
Path 270 | total_timesteps 3596.
Path 271 | total_timesteps 3613.
Path 272 | total_timesteps 3623.
Path 273 | total_timesteps 3644.
Path 274 | total_timesteps 3656.
Path 275 | total_timesteps 3671.
Path 276 | total_timesteps 3690.
Path 277 | total_timesteps 3702.
Path 278 | total_timesteps 3717.
Path 279 | total_timesteps 3730.
Path 280 | total_timesteps 3742.
Path 281 | total_timesteps 3748.
Path 282 | total_timesteps 3761.
Path 283 | total_timesteps 3774.
Path 284 | total_timesteps 3783.
Path 285 | total_timesteps 3793.
Path 286 | total_timesteps 3808.
Path 287 | total_timesteps 3825.
Path 288 | total_timesteps 3832.
Path 289 | total_timesteps 3841.
Path 290 | total_timesteps 3851.
Path 291 | total_timesteps 3864.
Path 292 | total_timesteps 3874.
Path 293 | total_timesteps 3887.
Path 294 | total_timesteps 3901.
Path 295 | total_timesteps 3916.
Path 296 | total_timesteps 3931.
Path 297 | total_timesteps 3941.
Path 298 | total_timesteps 3950.
Path 299 | total_timesteps 3959.
Path 300 | total_timesteps 3972.
Path 301 | total_timesteps 3991.
Path 302 | total_timesteps 4000.
Path 303 | total_timesteps 4013.
Path 304 | total_timesteps 4022.
Path 305 | total_timesteps 4038.
Path 306 | total_timesteps 4052.
Path 307 | total_timesteps 4064.
Path 308 | total_timesteps 4078.
Path 309 | total_timesteps 4094.
Path 310 | total_timesteps 4103.
Path 311 | total_timesteps 4118.
Path 312 | total_timesteps 4129.
Path 313 | total_timesteps 4142.
Path 314 | total_timesteps 4162.
Path 315 | total_timesteps 4176.
Path 316 | total_timesteps 4191.
Path 317 | total_timesteps 4203.
Path 318 | total_timesteps 4213.
Path 319 | total_timesteps 4226.
Path 320 | total_timesteps 4243.
Path 321 | total_timesteps 4252.
Path 322 | total_timesteps 4268.
Path 323 | total_timesteps 4283.
Path 324 | total_timesteps 4294.
Path 325 | total_timesteps 4306.
Path 326 | total_timesteps 4317.
Path 327 | total_timesteps 4328.
Path 328 | total_timesteps 4342.
Path 329 | total_timesteps 4358.
Path 330 | total_timesteps 4369.
Path 331 | total_timesteps 4377.
Path 332 | total_timesteps 4399.
Path 333 | total_timesteps 4412.
Path 334 | total_timesteps 4426.
Path 335 | total_timesteps 4436.
Path 336 | total_timesteps 4444.
Path 337 | total_timesteps 4455.
Path 338 | total_timesteps 4470.
Path 339 | total_timesteps 4481.
Path 340 | total_timesteps 4488.
Path 341 | total_timesteps 4502.
Path 342 | total_timesteps 4512.
Path 343 | total_timesteps 4521.
Path 344 | total_timesteps 4533.
Path 345 | total_timesteps 4547.
Path 346 | total_timesteps 4558.
Path 347 | total_timesteps 4570.
Path 348 | total_timesteps 4582.
Path 349 | total_timesteps 4591.
Path 350 | total_timesteps 4612.
Path 351 | total_timesteps 4626.
Path 352 | total_timesteps 4634.
Path 353 | total_timesteps 4645.
Path 354 | total_timesteps 4664.
Path 355 | total_timesteps 4681.
Path 356 | total_timesteps 4698.
Path 357 | total_timesteps 4712.
Path 358 | total_timesteps 4724.
Path 359 | total_timesteps 4737.
Path 360 | total_timesteps 4752.
Path 361 | total_timesteps 4768.
Path 362 | total_timesteps 4779.
Path 363 | total_timesteps 4787.
Path 364 | total_timesteps 4803.
Path 365 | total_timesteps 4816.
Path 366 | total_timesteps 4827.
Path 367 | total_timesteps 4841.
Path 368 | total_timesteps 4856.
Path 369 | total_timesteps 4865.
Path 370 | total_timesteps 4876.
Path 371 | total_timesteps 4888.
Path 372 | total_timesteps 4895.
Path 373 | total_timesteps 4904.
Path 374 | total_timesteps 4915.
Path 375 | total_timesteps 4923.
Path 376 | total_timesteps 4938.
Path 377 | total_timesteps 4948.
Path 378 | total_timesteps 4966.
Path 379 | total_timesteps 4989.
Path 380 | total_timesteps 5000.
Path 381 | total_timesteps 5010.
Path 382 | total_timesteps 5021.
Path 383 | total_timesteps 5031.
Path 384 | total_timesteps 5043.
Path 385 | total_timesteps 5057.
Path 386 | total_timesteps 5069.
Path 387 | total_timesteps 5081.
Path 388 | total_timesteps 5091.
Path 389 | total_timesteps 5101.
Path 390 | total_timesteps 5113.
Path 391 | total_timesteps 5126.
Path 392 | total_timesteps 5137.
Path 393 | total_timesteps 5149.
Path 394 | total_timesteps 5167.
Path 395 | total_timesteps 5177.
Path 396 | total_timesteps 5188.
Path 397 | total_timesteps 5209.
Path 398 | total_timesteps 5225.
Path 399 | total_timesteps 5234.
Path 400 | total_timesteps 5245.
Path 401 | total_timesteps 5257.
Path 402 | total_timesteps 5269.
Path 403 | total_timesteps 5277.
Path 404 | total_timesteps 5291.
Path 405 | total_timesteps 5304.
Path 406 | total_timesteps 5314.
Path 407 | total_timesteps 5324.
Path 408 | total_timesteps 5343.
Path 409 | total_timesteps 5361.
Path 410 | total_timesteps 5371.
Path 411 | total_timesteps 5385.
Path 412 | total_timesteps 5400.
Path 413 | total_timesteps 5410.
Path 414 | total_timesteps 5425.
Path 415 | total_timesteps 5437.
Path 416 | total_timesteps 5454.
Path 417 | total_timesteps 5466.
Path 418 | total_timesteps 5478.
Path 419 | total_timesteps 5492.
Path 420 | total_timesteps 5507.
Path 421 | total_timesteps 5517.
Path 422 | total_timesteps 5528.
Path 423 | total_timesteps 5536.
Path 424 | total_timesteps 5547.
Path 425 | total_timesteps 5560.
Path 426 | total_timesteps 5573.
Path 427 | total_timesteps 5586.
Path 428 | total_timesteps 5598.
Path 429 | total_timesteps 5616.
Path 430 | total_timesteps 5632.
Path 431 | total_timesteps 5644.
Path 432 | total_timesteps 5658.
Path 433 | total_timesteps 5666.
Path 434 | total_timesteps 5677.
Path 435 | total_timesteps 5689.
Path 436 | total_timesteps 5698.
Path 437 | total_timesteps 5715.
Path 438 | total_timesteps 5723.
Path 439 | total_timesteps 5737.
Path 440 | total_timesteps 5747.
Path 441 | total_timesteps 5756.
Path 442 | total_timesteps 5777.
Path 443 | total_timesteps 5787.
Path 444 | total_timesteps 5799.
Path 445 | total_timesteps 5809.
Path 446 | total_timesteps 5818.
Path 447 | total_timesteps 5830.
Path 448 | total_timesteps 5847.
Path 449 | total_timesteps 5862.
Path 450 | total_timesteps 5882.
Path 451 | total_timesteps 5891.
Path 452 | total_timesteps 5927.
Path 453 | total_timesteps 5937.
Path 454 | total_timesteps 5953.
Path 455 | total_timesteps 5961.
Path 456 | total_timesteps 5972.
Path 457 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.36    |
| Iteration     | 15       |
| MaximumReturn | -0.659   |
| MinimumReturn | -19.5    |
| TotalSamples  | 68080    |
----------------------------
itr #16 | 
Fitting dynamics.
Validation loss = 0.008745625615119934
Validation loss = 0.008031763136386871
Validation loss = 0.009375517256557941
Validation loss = 0.007700345013290644
Validation loss = 0.007689137477427721
Validation loss = 0.008307446725666523
Validation loss = 0.007281414233148098
Validation loss = 0.008084498345851898
Validation loss = 0.007802954409271479
Validation loss = 0.007342300843447447
Validation loss = 0.007028763648122549
Validation loss = 0.007307292893528938
Validation loss = 0.007146238349378109
Validation loss = 0.007273745723068714
Validation loss = 0.009217986837029457
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 42.
Path 4 | total_timesteps 51.
Path 5 | total_timesteps 58.
Path 6 | total_timesteps 68.
Path 7 | total_timesteps 77.
Path 8 | total_timesteps 88.
Path 9 | total_timesteps 95.
Path 10 | total_timesteps 105.
Path 11 | total_timesteps 112.
Path 12 | total_timesteps 126.
Path 13 | total_timesteps 135.
Path 14 | total_timesteps 149.
Path 15 | total_timesteps 163.
Path 16 | total_timesteps 180.
Path 17 | total_timesteps 194.
Path 18 | total_timesteps 201.
Path 19 | total_timesteps 212.
Path 20 | total_timesteps 219.
Path 21 | total_timesteps 231.
Path 22 | total_timesteps 244.
Path 23 | total_timesteps 260.
Path 24 | total_timesteps 270.
Path 25 | total_timesteps 281.
Path 26 | total_timesteps 292.
Path 27 | total_timesteps 301.
Path 28 | total_timesteps 318.
Path 29 | total_timesteps 337.
Path 30 | total_timesteps 345.
Path 31 | total_timesteps 353.
Path 32 | total_timesteps 362.
Path 33 | total_timesteps 376.
Path 34 | total_timesteps 392.
Path 35 | total_timesteps 405.
Path 36 | total_timesteps 415.
Path 37 | total_timesteps 430.
Path 38 | total_timesteps 447.
Path 39 | total_timesteps 460.
Path 40 | total_timesteps 469.
Path 41 | total_timesteps 480.
Path 42 | total_timesteps 491.
Path 43 | total_timesteps 505.
Path 44 | total_timesteps 515.
Path 45 | total_timesteps 527.
Path 46 | total_timesteps 541.
Path 47 | total_timesteps 549.
Path 48 | total_timesteps 559.
Path 49 | total_timesteps 567.
Path 50 | total_timesteps 581.
Path 51 | total_timesteps 593.
Path 52 | total_timesteps 606.
Path 53 | total_timesteps 619.
Path 54 | total_timesteps 627.
Path 55 | total_timesteps 635.
Path 56 | total_timesteps 651.
Path 57 | total_timesteps 658.
Path 58 | total_timesteps 670.
Path 59 | total_timesteps 685.
Path 60 | total_timesteps 696.
Path 61 | total_timesteps 713.
Path 62 | total_timesteps 729.
Path 63 | total_timesteps 740.
Path 64 | total_timesteps 748.
Path 65 | total_timesteps 766.
Path 66 | total_timesteps 777.
Path 67 | total_timesteps 790.
Path 68 | total_timesteps 808.
Path 69 | total_timesteps 819.
Path 70 | total_timesteps 828.
Path 71 | total_timesteps 841.
Path 72 | total_timesteps 857.
Path 73 | total_timesteps 874.
Path 74 | total_timesteps 893.
Path 75 | total_timesteps 907.
Path 76 | total_timesteps 917.
Path 77 | total_timesteps 939.
Path 78 | total_timesteps 958.
Path 79 | total_timesteps 969.
Path 80 | total_timesteps 977.
Path 81 | total_timesteps 988.
Path 82 | total_timesteps 995.
Path 83 | total_timesteps 1004.
Path 84 | total_timesteps 1014.
Path 85 | total_timesteps 1034.
Path 86 | total_timesteps 1042.
Path 87 | total_timesteps 1050.
Path 88 | total_timesteps 1065.
Path 89 | total_timesteps 1074.
Path 90 | total_timesteps 1085.
Path 91 | total_timesteps 1096.
Path 92 | total_timesteps 1109.
Path 93 | total_timesteps 1119.
Path 94 | total_timesteps 1130.
Path 95 | total_timesteps 1143.
Path 96 | total_timesteps 1160.
Path 97 | total_timesteps 1173.
Path 98 | total_timesteps 1188.
Path 99 | total_timesteps 1195.
Path 100 | total_timesteps 1211.
Path 101 | total_timesteps 1223.
Path 102 | total_timesteps 1234.
Path 103 | total_timesteps 1241.
Path 104 | total_timesteps 1257.
Path 105 | total_timesteps 1268.
Path 106 | total_timesteps 1279.
Path 107 | total_timesteps 1307.
Path 108 | total_timesteps 1323.
Path 109 | total_timesteps 1331.
Path 110 | total_timesteps 1347.
Path 111 | total_timesteps 1362.
Path 112 | total_timesteps 1375.
Path 113 | total_timesteps 1383.
Path 114 | total_timesteps 1398.
Path 115 | total_timesteps 1406.
Path 116 | total_timesteps 1425.
Path 117 | total_timesteps 1432.
Path 118 | total_timesteps 1455.
Path 119 | total_timesteps 1467.
Path 120 | total_timesteps 1480.
Path 121 | total_timesteps 1498.
Path 122 | total_timesteps 1510.
Path 123 | total_timesteps 1523.
Path 124 | total_timesteps 1535.
Path 125 | total_timesteps 1546.
Path 126 | total_timesteps 1558.
Path 127 | total_timesteps 1569.
Path 128 | total_timesteps 1579.
Path 129 | total_timesteps 1588.
Path 130 | total_timesteps 1598.
Path 131 | total_timesteps 1608.
Path 132 | total_timesteps 1624.
Path 133 | total_timesteps 1634.
Path 134 | total_timesteps 1643.
Path 135 | total_timesteps 1652.
Path 136 | total_timesteps 1671.
Path 137 | total_timesteps 1683.
Path 138 | total_timesteps 1690.
Path 139 | total_timesteps 1700.
Path 140 | total_timesteps 1711.
Path 141 | total_timesteps 1719.
Path 142 | total_timesteps 1727.
Path 143 | total_timesteps 1735.
Path 144 | total_timesteps 1749.
Path 145 | total_timesteps 1760.
Path 146 | total_timesteps 1770.
Path 147 | total_timesteps 1777.
Path 148 | total_timesteps 1787.
Path 149 | total_timesteps 1795.
Path 150 | total_timesteps 1810.
Path 151 | total_timesteps 1822.
Path 152 | total_timesteps 1838.
Path 153 | total_timesteps 1847.
Path 154 | total_timesteps 1862.
Path 155 | total_timesteps 1871.
Path 156 | total_timesteps 1883.
Path 157 | total_timesteps 1893.
Path 158 | total_timesteps 1903.
Path 159 | total_timesteps 1917.
Path 160 | total_timesteps 1926.
Path 161 | total_timesteps 1938.
Path 162 | total_timesteps 1951.
Path 163 | total_timesteps 1958.
Path 164 | total_timesteps 1968.
Path 165 | total_timesteps 1982.
Path 166 | total_timesteps 1995.
Path 167 | total_timesteps 2013.
Path 168 | total_timesteps 2026.
Path 169 | total_timesteps 2037.
Path 170 | total_timesteps 2044.
Path 171 | total_timesteps 2053.
Path 172 | total_timesteps 2064.
Path 173 | total_timesteps 2086.
Path 174 | total_timesteps 2095.
Path 175 | total_timesteps 2105.
Path 176 | total_timesteps 2115.
Path 177 | total_timesteps 2128.
Path 178 | total_timesteps 2139.
Path 179 | total_timesteps 2164.
Path 180 | total_timesteps 2174.
Path 181 | total_timesteps 2189.
Path 182 | total_timesteps 2199.
Path 183 | total_timesteps 2207.
Path 184 | total_timesteps 2229.
Path 185 | total_timesteps 2239.
Path 186 | total_timesteps 2252.
Path 187 | total_timesteps 2262.
Path 188 | total_timesteps 2272.
Path 189 | total_timesteps 2287.
Path 190 | total_timesteps 2296.
Path 191 | total_timesteps 2305.
Path 192 | total_timesteps 2313.
Path 193 | total_timesteps 2332.
Path 194 | total_timesteps 2340.
Path 195 | total_timesteps 2351.
Path 196 | total_timesteps 2366.
Path 197 | total_timesteps 2380.
Path 198 | total_timesteps 2388.
Path 199 | total_timesteps 2402.
Path 200 | total_timesteps 2410.
Path 201 | total_timesteps 2417.
Path 202 | total_timesteps 2432.
Path 203 | total_timesteps 2446.
Path 204 | total_timesteps 2470.
Path 205 | total_timesteps 2479.
Path 206 | total_timesteps 2493.
Path 207 | total_timesteps 2503.
Path 208 | total_timesteps 2516.
Path 209 | total_timesteps 2528.
Path 210 | total_timesteps 2536.
Path 211 | total_timesteps 2550.
Path 212 | total_timesteps 2562.
Path 213 | total_timesteps 2575.
Path 214 | total_timesteps 2594.
Path 215 | total_timesteps 2614.
Path 216 | total_timesteps 2625.
Path 217 | total_timesteps 2638.
Path 218 | total_timesteps 2657.
Path 219 | total_timesteps 2671.
Path 220 | total_timesteps 2684.
Path 221 | total_timesteps 2696.
Path 222 | total_timesteps 2711.
Path 223 | total_timesteps 2724.
Path 224 | total_timesteps 2734.
Path 225 | total_timesteps 2748.
Path 226 | total_timesteps 2766.
Path 227 | total_timesteps 2773.
Path 228 | total_timesteps 2785.
Path 229 | total_timesteps 2795.
Path 230 | total_timesteps 2812.
Path 231 | total_timesteps 2824.
Path 232 | total_timesteps 2834.
Path 233 | total_timesteps 2846.
Path 234 | total_timesteps 2868.
Path 235 | total_timesteps 2878.
Path 236 | total_timesteps 2891.
Path 237 | total_timesteps 2907.
Path 238 | total_timesteps 2917.
Path 239 | total_timesteps 2932.
Path 240 | total_timesteps 2947.
Path 241 | total_timesteps 2955.
Path 242 | total_timesteps 2975.
Path 243 | total_timesteps 2986.
Path 244 | total_timesteps 2998.
Path 245 | total_timesteps 3008.
Path 246 | total_timesteps 3017.
Path 247 | total_timesteps 3034.
Path 248 | total_timesteps 3046.
Path 249 | total_timesteps 3066.
Path 250 | total_timesteps 3082.
Path 251 | total_timesteps 3092.
Path 252 | total_timesteps 3103.
Path 253 | total_timesteps 3113.
Path 254 | total_timesteps 3123.
Path 255 | total_timesteps 3140.
Path 256 | total_timesteps 3150.
Path 257 | total_timesteps 3160.
Path 258 | total_timesteps 3182.
Path 259 | total_timesteps 3196.
Path 260 | total_timesteps 3209.
Path 261 | total_timesteps 3217.
Path 262 | total_timesteps 3225.
Path 263 | total_timesteps 3232.
Path 264 | total_timesteps 3242.
Path 265 | total_timesteps 3256.
Path 266 | total_timesteps 3270.
Path 267 | total_timesteps 3279.
Path 268 | total_timesteps 3307.
Path 269 | total_timesteps 3326.
Path 270 | total_timesteps 3334.
Path 271 | total_timesteps 3350.
Path 272 | total_timesteps 3364.
Path 273 | total_timesteps 3373.
Path 274 | total_timesteps 3391.
Path 275 | total_timesteps 3400.
Path 276 | total_timesteps 3418.
Path 277 | total_timesteps 3433.
Path 278 | total_timesteps 3444.
Path 279 | total_timesteps 3458.
Path 280 | total_timesteps 3480.
Path 281 | total_timesteps 3497.
Path 282 | total_timesteps 3506.
Path 283 | total_timesteps 3514.
Path 284 | total_timesteps 3526.
Path 285 | total_timesteps 3534.
Path 286 | total_timesteps 3548.
Path 287 | total_timesteps 3558.
Path 288 | total_timesteps 3574.
Path 289 | total_timesteps 3585.
Path 290 | total_timesteps 3592.
Path 291 | total_timesteps 3606.
Path 292 | total_timesteps 3615.
Path 293 | total_timesteps 3632.
Path 294 | total_timesteps 3647.
Path 295 | total_timesteps 3660.
Path 296 | total_timesteps 3673.
Path 297 | total_timesteps 3687.
Path 298 | total_timesteps 3701.
Path 299 | total_timesteps 3719.
Path 300 | total_timesteps 3729.
Path 301 | total_timesteps 3741.
Path 302 | total_timesteps 3749.
Path 303 | total_timesteps 3756.
Path 304 | total_timesteps 3765.
Path 305 | total_timesteps 3781.
Path 306 | total_timesteps 3790.
Path 307 | total_timesteps 3803.
Path 308 | total_timesteps 3819.
Path 309 | total_timesteps 3830.
Path 310 | total_timesteps 3846.
Path 311 | total_timesteps 3859.
Path 312 | total_timesteps 3871.
Path 313 | total_timesteps 3885.
Path 314 | total_timesteps 3897.
Path 315 | total_timesteps 3908.
Path 316 | total_timesteps 3915.
Path 317 | total_timesteps 3930.
Path 318 | total_timesteps 3948.
Path 319 | total_timesteps 3960.
Path 320 | total_timesteps 3967.
Path 321 | total_timesteps 3976.
Path 322 | total_timesteps 3990.
Path 323 | total_timesteps 4000.
Path 324 | total_timesteps 4010.
Path 325 | total_timesteps 4020.
Path 326 | total_timesteps 4029.
Path 327 | total_timesteps 4039.
Path 328 | total_timesteps 4057.
Path 329 | total_timesteps 4070.
Path 330 | total_timesteps 4080.
Path 331 | total_timesteps 4087.
Path 332 | total_timesteps 4096.
Path 333 | total_timesteps 4110.
Path 334 | total_timesteps 4123.
Path 335 | total_timesteps 4132.
Path 336 | total_timesteps 4143.
Path 337 | total_timesteps 4151.
Path 338 | total_timesteps 4164.
Path 339 | total_timesteps 4175.
Path 340 | total_timesteps 4196.
Path 341 | total_timesteps 4209.
Path 342 | total_timesteps 4225.
Path 343 | total_timesteps 4237.
Path 344 | total_timesteps 4252.
Path 345 | total_timesteps 4261.
Path 346 | total_timesteps 4273.
Path 347 | total_timesteps 4284.
Path 348 | total_timesteps 4296.
Path 349 | total_timesteps 4308.
Path 350 | total_timesteps 4320.
Path 351 | total_timesteps 4333.
Path 352 | total_timesteps 4350.
Path 353 | total_timesteps 4358.
Path 354 | total_timesteps 4367.
Path 355 | total_timesteps 4373.
Path 356 | total_timesteps 4390.
Path 357 | total_timesteps 4401.
Path 358 | total_timesteps 4414.
Path 359 | total_timesteps 4425.
Path 360 | total_timesteps 4435.
Path 361 | total_timesteps 4450.
Path 362 | total_timesteps 4458.
Path 363 | total_timesteps 4469.
Path 364 | total_timesteps 4479.
Path 365 | total_timesteps 4497.
Path 366 | total_timesteps 4508.
Path 367 | total_timesteps 4521.
Path 368 | total_timesteps 4528.
Path 369 | total_timesteps 4549.
Path 370 | total_timesteps 4562.
Path 371 | total_timesteps 4570.
Path 372 | total_timesteps 4582.
Path 373 | total_timesteps 4595.
Path 374 | total_timesteps 4606.
Path 375 | total_timesteps 4616.
Path 376 | total_timesteps 4625.
Path 377 | total_timesteps 4641.
Path 378 | total_timesteps 4652.
Path 379 | total_timesteps 4670.
Path 380 | total_timesteps 4680.
Path 381 | total_timesteps 4689.
Path 382 | total_timesteps 4706.
Path 383 | total_timesteps 4716.
Path 384 | total_timesteps 4728.
Path 385 | total_timesteps 4738.
Path 386 | total_timesteps 4753.
Path 387 | total_timesteps 4763.
Path 388 | total_timesteps 4771.
Path 389 | total_timesteps 4785.
Path 390 | total_timesteps 4805.
Path 391 | total_timesteps 4811.
Path 392 | total_timesteps 4827.
Path 393 | total_timesteps 4835.
Path 394 | total_timesteps 4847.
Path 395 | total_timesteps 4860.
Path 396 | total_timesteps 4868.
Path 397 | total_timesteps 4875.
Path 398 | total_timesteps 4881.
Path 399 | total_timesteps 4893.
Path 400 | total_timesteps 4906.
Path 401 | total_timesteps 4921.
Path 402 | total_timesteps 4929.
Path 403 | total_timesteps 4945.
Path 404 | total_timesteps 4961.
Path 405 | total_timesteps 4977.
Path 406 | total_timesteps 4995.
Path 407 | total_timesteps 5009.
Path 408 | total_timesteps 5020.
Path 409 | total_timesteps 5031.
Path 410 | total_timesteps 5047.
Path 411 | total_timesteps 5057.
Path 412 | total_timesteps 5067.
Path 413 | total_timesteps 5077.
Path 414 | total_timesteps 5086.
Path 415 | total_timesteps 5095.
Path 416 | total_timesteps 5106.
Path 417 | total_timesteps 5116.
Path 418 | total_timesteps 5136.
Path 419 | total_timesteps 5153.
Path 420 | total_timesteps 5167.
Path 421 | total_timesteps 5188.
Path 422 | total_timesteps 5201.
Path 423 | total_timesteps 5215.
Path 424 | total_timesteps 5233.
Path 425 | total_timesteps 5241.
Path 426 | total_timesteps 5252.
Path 427 | total_timesteps 5266.
Path 428 | total_timesteps 5275.
Path 429 | total_timesteps 5292.
Path 430 | total_timesteps 5303.
Path 431 | total_timesteps 5312.
Path 432 | total_timesteps 5327.
Path 433 | total_timesteps 5340.
Path 434 | total_timesteps 5351.
Path 435 | total_timesteps 5361.
Path 436 | total_timesteps 5374.
Path 437 | total_timesteps 5388.
Path 438 | total_timesteps 5400.
Path 439 | total_timesteps 5411.
Path 440 | total_timesteps 5425.
Path 441 | total_timesteps 5435.
Path 442 | total_timesteps 5447.
Path 443 | total_timesteps 5461.
Path 444 | total_timesteps 5481.
Path 445 | total_timesteps 5489.
Path 446 | total_timesteps 5501.
Path 447 | total_timesteps 5517.
Path 448 | total_timesteps 5525.
Path 449 | total_timesteps 5541.
Path 450 | total_timesteps 5568.
Path 451 | total_timesteps 5577.
Path 452 | total_timesteps 5595.
Path 453 | total_timesteps 5604.
Path 454 | total_timesteps 5614.
Path 455 | total_timesteps 5625.
Path 456 | total_timesteps 5637.
Path 457 | total_timesteps 5651.
Path 458 | total_timesteps 5664.
Path 459 | total_timesteps 5678.
Path 460 | total_timesteps 5689.
Path 461 | total_timesteps 5706.
Path 462 | total_timesteps 5719.
Path 463 | total_timesteps 5727.
Path 464 | total_timesteps 5743.
Path 465 | total_timesteps 5754.
Path 466 | total_timesteps 5769.
Path 467 | total_timesteps 5780.
Path 468 | total_timesteps 5793.
Path 469 | total_timesteps 5802.
Path 470 | total_timesteps 5820.
Path 471 | total_timesteps 5831.
Path 472 | total_timesteps 5839.
Path 473 | total_timesteps 5854.
Path 474 | total_timesteps 5867.
Path 475 | total_timesteps 5876.
Path 476 | total_timesteps 5882.
Path 477 | total_timesteps 5895.
Path 478 | total_timesteps 5911.
Path 479 | total_timesteps 5919.
Path 480 | total_timesteps 5930.
Path 481 | total_timesteps 5947.
Path 482 | total_timesteps 5961.
Path 483 | total_timesteps 5982.
Path 484 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.35    |
| Iteration     | 16       |
| MaximumReturn | 1.1      |
| MinimumReturn | -21.6    |
| TotalSamples  | 72082    |
----------------------------
itr #17 | 
Fitting dynamics.
Validation loss = 0.007406659424304962
Validation loss = 0.007311924360692501
Validation loss = 0.007879588752985
Validation loss = 0.007157097104936838
Validation loss = 0.007795811630785465
Validation loss = 0.006937775760889053
Validation loss = 0.007265963591635227
Validation loss = 0.007175908423960209
Validation loss = 0.00728317117318511
Validation loss = 0.007364796940237284
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 42.
Path 2 | total_timesteps 79.
Path 3 | total_timesteps 121.
Path 4 | total_timesteps 156.
Path 5 | total_timesteps 192.
Path 6 | total_timesteps 222.
Path 7 | total_timesteps 249.
Path 8 | total_timesteps 290.
Path 9 | total_timesteps 319.
Path 10 | total_timesteps 357.
Path 11 | total_timesteps 389.
Path 12 | total_timesteps 422.
Path 13 | total_timesteps 455.
Path 14 | total_timesteps 494.
Path 15 | total_timesteps 533.
Path 16 | total_timesteps 556.
Path 17 | total_timesteps 595.
Path 18 | total_timesteps 660.
Path 19 | total_timesteps 709.
Path 20 | total_timesteps 800.
Path 21 | total_timesteps 868.
Path 22 | total_timesteps 897.
Path 23 | total_timesteps 926.
Path 24 | total_timesteps 983.
Path 25 | total_timesteps 1020.
Path 26 | total_timesteps 1042.
Path 27 | total_timesteps 1102.
Path 28 | total_timesteps 1163.
Path 29 | total_timesteps 1193.
Path 30 | total_timesteps 1221.
Path 31 | total_timesteps 1255.
Path 32 | total_timesteps 1284.
Path 33 | total_timesteps 1325.
Path 34 | total_timesteps 1380.
Path 35 | total_timesteps 1426.
Path 36 | total_timesteps 1462.
Path 37 | total_timesteps 1501.
Path 38 | total_timesteps 1587.
Path 39 | total_timesteps 1622.
Path 40 | total_timesteps 1680.
Path 41 | total_timesteps 1709.
Path 42 | total_timesteps 1776.
Path 43 | total_timesteps 1809.
Path 44 | total_timesteps 1862.
Path 45 | total_timesteps 1906.
Path 46 | total_timesteps 1929.
Path 47 | total_timesteps 1958.
Path 48 | total_timesteps 2002.
Path 49 | total_timesteps 2046.
Path 50 | total_timesteps 2066.
Path 51 | total_timesteps 2096.
Path 52 | total_timesteps 2121.
Path 53 | total_timesteps 2166.
Path 54 | total_timesteps 2225.
Path 55 | total_timesteps 2288.
Path 56 | total_timesteps 2364.
Path 57 | total_timesteps 2410.
Path 58 | total_timesteps 2454.
Path 59 | total_timesteps 2482.
Path 60 | total_timesteps 2521.
Path 61 | total_timesteps 2553.
Path 62 | total_timesteps 2592.
Path 63 | total_timesteps 2626.
Path 64 | total_timesteps 2648.
Path 65 | total_timesteps 2694.
Path 66 | total_timesteps 2724.
Path 67 | total_timesteps 2757.
Path 68 | total_timesteps 2822.
Path 69 | total_timesteps 2911.
Path 70 | total_timesteps 2951.
Path 71 | total_timesteps 2979.
Path 72 | total_timesteps 3027.
Path 73 | total_timesteps 3047.
Path 74 | total_timesteps 3109.
Path 75 | total_timesteps 3125.
Path 76 | total_timesteps 3182.
Path 77 | total_timesteps 3202.
Path 78 | total_timesteps 3225.
Path 79 | total_timesteps 3246.
Path 80 | total_timesteps 3266.
Path 81 | total_timesteps 3310.
Path 82 | total_timesteps 3347.
Path 83 | total_timesteps 3379.
Path 84 | total_timesteps 3412.
Path 85 | total_timesteps 3456.
Path 86 | total_timesteps 3480.
Path 87 | total_timesteps 3525.
Path 88 | total_timesteps 3548.
Path 89 | total_timesteps 3572.
Path 90 | total_timesteps 3599.
Path 91 | total_timesteps 3643.
Path 92 | total_timesteps 3710.
Path 93 | total_timesteps 3741.
Path 94 | total_timesteps 3779.
Path 95 | total_timesteps 3810.
Path 96 | total_timesteps 3848.
Path 97 | total_timesteps 3884.
Path 98 | total_timesteps 3915.
Path 99 | total_timesteps 3942.
Path 100 | total_timesteps 3971.
Path 101 | total_timesteps 4044.
Path 102 | total_timesteps 4076.
Path 103 | total_timesteps 4109.
Path 104 | total_timesteps 4155.
Path 105 | total_timesteps 4199.
Path 106 | total_timesteps 4219.
Path 107 | total_timesteps 4255.
Path 108 | total_timesteps 4308.
Path 109 | total_timesteps 4374.
Path 110 | total_timesteps 4420.
Path 111 | total_timesteps 4526.
Path 112 | total_timesteps 4555.
Path 113 | total_timesteps 4607.
Path 114 | total_timesteps 4655.
Path 115 | total_timesteps 4704.
Path 116 | total_timesteps 4722.
Path 117 | total_timesteps 4747.
Path 118 | total_timesteps 4814.
Path 119 | total_timesteps 4844.
Path 120 | total_timesteps 4885.
Path 121 | total_timesteps 4913.
Path 122 | total_timesteps 4936.
Path 123 | total_timesteps 4963.
Path 124 | total_timesteps 5016.
Path 125 | total_timesteps 5079.
Path 126 | total_timesteps 5106.
Path 127 | total_timesteps 5132.
Path 128 | total_timesteps 5173.
Path 129 | total_timesteps 5199.
Path 130 | total_timesteps 5226.
Path 131 | total_timesteps 5257.
Path 132 | total_timesteps 5300.
Path 133 | total_timesteps 5332.
Path 134 | total_timesteps 5378.
Path 135 | total_timesteps 5426.
Path 136 | total_timesteps 5465.
Path 137 | total_timesteps 5530.
Path 138 | total_timesteps 5592.
Path 139 | total_timesteps 5622.
Path 140 | total_timesteps 5687.
Path 141 | total_timesteps 5729.
Path 142 | total_timesteps 5768.
Path 143 | total_timesteps 5817.
Path 144 | total_timesteps 5910.
Path 145 | total_timesteps 5955.
Path 146 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.92    |
| Iteration     | 17       |
| MaximumReturn | 62       |
| MinimumReturn | -37.2    |
| TotalSamples  | 76092    |
----------------------------
itr #18 | 
Fitting dynamics.
Validation loss = 0.0077796820551157
Validation loss = 0.007302859798073769
Validation loss = 0.0076231807470321655
Validation loss = 0.007530400529503822
Validation loss = 0.0071101863868534565
Validation loss = 0.00679516326636076
Validation loss = 0.007212815340608358
Validation loss = 0.00714826351031661
Validation loss = 0.006913449615240097
Validation loss = 0.006822124123573303
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 83.
Path 2 | total_timesteps 132.
Path 3 | total_timesteps 163.
Path 4 | total_timesteps 258.
Path 5 | total_timesteps 310.
Path 6 | total_timesteps 341.
Path 7 | total_timesteps 380.
Path 8 | total_timesteps 455.
Path 9 | total_timesteps 488.
Path 10 | total_timesteps 520.
Path 11 | total_timesteps 556.
Path 12 | total_timesteps 631.
Path 13 | total_timesteps 693.
Path 14 | total_timesteps 752.
Path 15 | total_timesteps 784.
Path 16 | total_timesteps 818.
Path 17 | total_timesteps 854.
Path 18 | total_timesteps 870.
Path 19 | total_timesteps 899.
Path 20 | total_timesteps 976.
Path 21 | total_timesteps 1021.
Path 22 | total_timesteps 1090.
Path 23 | total_timesteps 1122.
Path 24 | total_timesteps 1180.
Path 25 | total_timesteps 1216.
Path 26 | total_timesteps 1284.
Path 27 | total_timesteps 1340.
Path 28 | total_timesteps 1375.
Path 29 | total_timesteps 1427.
Path 30 | total_timesteps 1500.
Path 31 | total_timesteps 1543.
Path 32 | total_timesteps 1576.
Path 33 | total_timesteps 1602.
Path 34 | total_timesteps 1650.
Path 35 | total_timesteps 1701.
Path 36 | total_timesteps 1755.
Path 37 | total_timesteps 1787.
Path 38 | total_timesteps 1871.
Path 39 | total_timesteps 1916.
Path 40 | total_timesteps 1965.
Path 41 | total_timesteps 2015.
Path 42 | total_timesteps 2047.
Path 43 | total_timesteps 2125.
Path 44 | total_timesteps 2218.
Path 45 | total_timesteps 2266.
Path 46 | total_timesteps 2276.
Path 47 | total_timesteps 2298.
Path 48 | total_timesteps 2379.
Path 49 | total_timesteps 2437.
Path 50 | total_timesteps 2522.
Path 51 | total_timesteps 2568.
Path 52 | total_timesteps 2618.
Path 53 | total_timesteps 2659.
Path 54 | total_timesteps 2708.
Path 55 | total_timesteps 2779.
Path 56 | total_timesteps 2835.
Path 57 | total_timesteps 2903.
Path 58 | total_timesteps 2944.
Path 59 | total_timesteps 2974.
Path 60 | total_timesteps 2990.
Path 61 | total_timesteps 3039.
Path 62 | total_timesteps 3097.
Path 63 | total_timesteps 3144.
Path 64 | total_timesteps 3183.
Path 65 | total_timesteps 3239.
Path 66 | total_timesteps 3254.
Path 67 | total_timesteps 3272.
Path 68 | total_timesteps 3331.
Path 69 | total_timesteps 3390.
Path 70 | total_timesteps 3419.
Path 71 | total_timesteps 3459.
Path 72 | total_timesteps 3505.
Path 73 | total_timesteps 3536.
Path 74 | total_timesteps 3593.
Path 75 | total_timesteps 3669.
Path 76 | total_timesteps 3768.
Path 77 | total_timesteps 3810.
Path 78 | total_timesteps 3838.
Path 79 | total_timesteps 3897.
Path 80 | total_timesteps 3935.
Path 81 | total_timesteps 3989.
Path 82 | total_timesteps 4062.
Path 83 | total_timesteps 4087.
Path 84 | total_timesteps 4126.
Path 85 | total_timesteps 4196.
Path 86 | total_timesteps 4271.
Path 87 | total_timesteps 4331.
Path 88 | total_timesteps 4361.
Path 89 | total_timesteps 4412.
Path 90 | total_timesteps 4444.
Path 91 | total_timesteps 4500.
Path 92 | total_timesteps 4569.
Path 93 | total_timesteps 4619.
Path 94 | total_timesteps 4658.
Path 95 | total_timesteps 4693.
Path 96 | total_timesteps 4732.
Path 97 | total_timesteps 4793.
Path 98 | total_timesteps 4839.
Path 99 | total_timesteps 4890.
Path 100 | total_timesteps 4955.
Path 101 | total_timesteps 5059.
Path 102 | total_timesteps 5148.
Path 103 | total_timesteps 5175.
Path 104 | total_timesteps 5224.
Path 105 | total_timesteps 5264.
Path 106 | total_timesteps 5302.
Path 107 | total_timesteps 5345.
Path 108 | total_timesteps 5390.
Path 109 | total_timesteps 5419.
Path 110 | total_timesteps 5477.
Path 111 | total_timesteps 5503.
Path 112 | total_timesteps 5548.
Path 113 | total_timesteps 5591.
Path 114 | total_timesteps 5622.
Path 115 | total_timesteps 5651.
Path 116 | total_timesteps 5713.
Path 117 | total_timesteps 5787.
Path 118 | total_timesteps 5858.
Path 119 | total_timesteps 5937.
Path 120 | total_timesteps 5986.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.7    |
| Iteration     | 18       |
| MaximumReturn | 62.6     |
| MinimumReturn | -65.3    |
| TotalSamples  | 80108    |
----------------------------
itr #19 | 
Fitting dynamics.
Validation loss = 0.008396652527153492
Validation loss = 0.00750480592250824
Validation loss = 0.00730444211512804
Validation loss = 0.0073082344606518745
Validation loss = 0.006995684467256069
Validation loss = 0.0070981355383992195
Validation loss = 0.007089757826179266
Validation loss = 0.007035139016807079
Validation loss = 0.007058925926685333
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 69.
Path 2 | total_timesteps 105.
Path 3 | total_timesteps 152.
Path 4 | total_timesteps 207.
Path 5 | total_timesteps 235.
Path 6 | total_timesteps 279.
Path 7 | total_timesteps 341.
Path 8 | total_timesteps 463.
Path 9 | total_timesteps 498.
Path 10 | total_timesteps 547.
Path 11 | total_timesteps 581.
Path 12 | total_timesteps 619.
Path 13 | total_timesteps 690.
Path 14 | total_timesteps 786.
Path 15 | total_timesteps 814.
Path 16 | total_timesteps 872.
Path 17 | total_timesteps 923.
Path 18 | total_timesteps 1047.
Path 19 | total_timesteps 1116.
Path 20 | total_timesteps 1179.
Path 21 | total_timesteps 1235.
Path 22 | total_timesteps 1291.
Path 23 | total_timesteps 1329.
Path 24 | total_timesteps 1371.
Path 25 | total_timesteps 1405.
Path 26 | total_timesteps 1444.
Path 27 | total_timesteps 1490.
Path 28 | total_timesteps 1549.
Path 29 | total_timesteps 1586.
Path 30 | total_timesteps 1606.
Path 31 | total_timesteps 1646.
Path 32 | total_timesteps 1675.
Path 33 | total_timesteps 1726.
Path 34 | total_timesteps 1763.
Path 35 | total_timesteps 1786.
Path 36 | total_timesteps 1816.
Path 37 | total_timesteps 1851.
Path 38 | total_timesteps 1893.
Path 39 | total_timesteps 1987.
Path 40 | total_timesteps 2034.
Path 41 | total_timesteps 2120.
Path 42 | total_timesteps 2179.
Path 43 | total_timesteps 2198.
Path 44 | total_timesteps 2244.
Path 45 | total_timesteps 2301.
Path 46 | total_timesteps 2325.
Path 47 | total_timesteps 2350.
Path 48 | total_timesteps 2380.
Path 49 | total_timesteps 2457.
Path 50 | total_timesteps 2496.
Path 51 | total_timesteps 2542.
Path 52 | total_timesteps 2602.
Path 53 | total_timesteps 2654.
Path 54 | total_timesteps 2710.
Path 55 | total_timesteps 2778.
Path 56 | total_timesteps 2822.
Path 57 | total_timesteps 2856.
Path 58 | total_timesteps 2896.
Path 59 | total_timesteps 2939.
Path 60 | total_timesteps 2990.
Path 61 | total_timesteps 3022.
Path 62 | total_timesteps 3078.
Path 63 | total_timesteps 3105.
Path 64 | total_timesteps 3155.
Path 65 | total_timesteps 3184.
Path 66 | total_timesteps 3232.
Path 67 | total_timesteps 3293.
Path 68 | total_timesteps 3339.
Path 69 | total_timesteps 3384.
Path 70 | total_timesteps 3439.
Path 71 | total_timesteps 3473.
Path 72 | total_timesteps 3518.
Path 73 | total_timesteps 3589.
Path 74 | total_timesteps 3634.
Path 75 | total_timesteps 3658.
Path 76 | total_timesteps 3683.
Path 77 | total_timesteps 3740.
Path 78 | total_timesteps 3762.
Path 79 | total_timesteps 3807.
Path 80 | total_timesteps 3894.
Path 81 | total_timesteps 3959.
Path 82 | total_timesteps 3990.
Path 83 | total_timesteps 4046.
Path 84 | total_timesteps 4103.
Path 85 | total_timesteps 4159.
Path 86 | total_timesteps 4204.
Path 87 | total_timesteps 4274.
Path 88 | total_timesteps 4324.
Path 89 | total_timesteps 4373.
Path 90 | total_timesteps 4424.
Path 91 | total_timesteps 4478.
Path 92 | total_timesteps 4533.
Path 93 | total_timesteps 4585.
Path 94 | total_timesteps 4629.
Path 95 | total_timesteps 4667.
Path 96 | total_timesteps 4703.
Path 97 | total_timesteps 4749.
Path 98 | total_timesteps 4773.
Path 99 | total_timesteps 4824.
Path 100 | total_timesteps 4867.
Path 101 | total_timesteps 4907.
Path 102 | total_timesteps 4959.
Path 103 | total_timesteps 5019.
Path 104 | total_timesteps 5054.
Path 105 | total_timesteps 5078.
Path 106 | total_timesteps 5120.
Path 107 | total_timesteps 5151.
Path 108 | total_timesteps 5185.
Path 109 | total_timesteps 5222.
Path 110 | total_timesteps 5257.
Path 111 | total_timesteps 5282.
Path 112 | total_timesteps 5306.
Path 113 | total_timesteps 5388.
Path 114 | total_timesteps 5429.
Path 115 | total_timesteps 5483.
Path 116 | total_timesteps 5527.
Path 117 | total_timesteps 5586.
Path 118 | total_timesteps 5649.
Path 119 | total_timesteps 5704.
Path 120 | total_timesteps 5761.
Path 121 | total_timesteps 5820.
Path 122 | total_timesteps 5896.
Path 123 | total_timesteps 5930.
Path 124 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.3    |
| Iteration     | 19       |
| MaximumReturn | 64.9     |
| MinimumReturn | -41.9    |
| TotalSamples  | 84132    |
----------------------------
itr #20 | 
Fitting dynamics.
Validation loss = 0.0074911536648869514
Validation loss = 0.007003074046224356
Validation loss = 0.00671576801687479
Validation loss = 0.007142617367208004
Validation loss = 0.007069770246744156
Validation loss = 0.0069943019188940525
Validation loss = 0.00725336279720068
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 62.
Path 2 | total_timesteps 129.
Path 3 | total_timesteps 243.
Path 4 | total_timesteps 315.
Path 5 | total_timesteps 349.
Path 6 | total_timesteps 383.
Path 7 | total_timesteps 421.
Path 8 | total_timesteps 457.
Path 9 | total_timesteps 507.
Path 10 | total_timesteps 527.
Path 11 | total_timesteps 580.
Path 12 | total_timesteps 622.
Path 13 | total_timesteps 692.
Path 14 | total_timesteps 769.
Path 15 | total_timesteps 818.
Path 16 | total_timesteps 860.
Path 17 | total_timesteps 914.
Path 18 | total_timesteps 955.
Path 19 | total_timesteps 1022.
Path 20 | total_timesteps 1052.
Path 21 | total_timesteps 1114.
Path 22 | total_timesteps 1156.
Path 23 | total_timesteps 1208.
Path 24 | total_timesteps 1303.
Path 25 | total_timesteps 1336.
Path 26 | total_timesteps 1378.
Path 27 | total_timesteps 1417.
Path 28 | total_timesteps 1460.
Path 29 | total_timesteps 1497.
Path 30 | total_timesteps 1578.
Path 31 | total_timesteps 1658.
Path 32 | total_timesteps 1707.
Path 33 | total_timesteps 1796.
Path 34 | total_timesteps 1822.
Path 35 | total_timesteps 1906.
Path 36 | total_timesteps 1961.
Path 37 | total_timesteps 2045.
Path 38 | total_timesteps 2129.
Path 39 | total_timesteps 2175.
Path 40 | total_timesteps 2231.
Path 41 | total_timesteps 2269.
Path 42 | total_timesteps 2348.
Path 43 | total_timesteps 2391.
Path 44 | total_timesteps 2436.
Path 45 | total_timesteps 2508.
Path 46 | total_timesteps 2562.
Path 47 | total_timesteps 2633.
Path 48 | total_timesteps 2683.
Path 49 | total_timesteps 2722.
Path 50 | total_timesteps 2758.
Path 51 | total_timesteps 2827.
Path 52 | total_timesteps 2866.
Path 53 | total_timesteps 2923.
Path 54 | total_timesteps 2977.
Path 55 | total_timesteps 3021.
Path 56 | total_timesteps 3074.
Path 57 | total_timesteps 3135.
Path 58 | total_timesteps 3203.
Path 59 | total_timesteps 3235.
Path 60 | total_timesteps 3293.
Path 61 | total_timesteps 3327.
Path 62 | total_timesteps 3391.
Path 63 | total_timesteps 3426.
Path 64 | total_timesteps 3481.
Path 65 | total_timesteps 3577.
Path 66 | total_timesteps 3616.
Path 67 | total_timesteps 3649.
Path 68 | total_timesteps 3706.
Path 69 | total_timesteps 3742.
Path 70 | total_timesteps 3850.
Path 71 | total_timesteps 3893.
Path 72 | total_timesteps 3943.
Path 73 | total_timesteps 3984.
Path 74 | total_timesteps 4031.
Path 75 | total_timesteps 4075.
Path 76 | total_timesteps 4103.
Path 77 | total_timesteps 4149.
Path 78 | total_timesteps 4216.
Path 79 | total_timesteps 4260.
Path 80 | total_timesteps 4312.
Path 81 | total_timesteps 4392.
Path 82 | total_timesteps 4440.
Path 83 | total_timesteps 4497.
Path 84 | total_timesteps 4577.
Path 85 | total_timesteps 4631.
Path 86 | total_timesteps 4709.
Path 87 | total_timesteps 4759.
Path 88 | total_timesteps 4807.
Path 89 | total_timesteps 4835.
Path 90 | total_timesteps 4875.
Path 91 | total_timesteps 4921.
Path 92 | total_timesteps 4997.
Path 93 | total_timesteps 5039.
Path 94 | total_timesteps 5091.
Path 95 | total_timesteps 5125.
Path 96 | total_timesteps 5189.
Path 97 | total_timesteps 5214.
Path 98 | total_timesteps 5281.
Path 99 | total_timesteps 5329.
Path 100 | total_timesteps 5411.
Path 101 | total_timesteps 5445.
Path 102 | total_timesteps 5513.
Path 103 | total_timesteps 5554.
Path 104 | total_timesteps 5596.
Path 105 | total_timesteps 5635.
Path 106 | total_timesteps 5671.
Path 107 | total_timesteps 5697.
Path 108 | total_timesteps 5735.
Path 109 | total_timesteps 5770.
Path 110 | total_timesteps 5840.
Path 111 | total_timesteps 5929.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -10.8    |
| Iteration     | 20       |
| MaximumReturn | 68.8     |
| MinimumReturn | -49.2    |
| TotalSamples  | 88132    |
----------------------------
itr #21 | 
Fitting dynamics.
Validation loss = 0.007787875831127167
Validation loss = 0.007072985637933016
Validation loss = 0.007159191649407148
Validation loss = 0.0069234948605299
Validation loss = 0.0068259951658546925
Validation loss = 0.006600555963814259
Validation loss = 0.00660704867914319
Validation loss = 0.006894673220813274
Validation loss = 0.007352237589657307
Validation loss = 0.006637622602283955
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 53.
Path 2 | total_timesteps 100.
Path 3 | total_timesteps 164.
Path 4 | total_timesteps 179.
Path 5 | total_timesteps 203.
Path 6 | total_timesteps 303.
Path 7 | total_timesteps 365.
Path 8 | total_timesteps 426.
Path 9 | total_timesteps 486.
Path 10 | total_timesteps 501.
Path 11 | total_timesteps 572.
Path 12 | total_timesteps 616.
Path 13 | total_timesteps 718.
Path 14 | total_timesteps 758.
Path 15 | total_timesteps 786.
Path 16 | total_timesteps 837.
Path 17 | total_timesteps 882.
Path 18 | total_timesteps 912.
Path 19 | total_timesteps 951.
Path 20 | total_timesteps 1003.
Path 21 | total_timesteps 1053.
Path 22 | total_timesteps 1148.
Path 23 | total_timesteps 1182.
Path 24 | total_timesteps 1272.
Path 25 | total_timesteps 1310.
Path 26 | total_timesteps 1364.
Path 27 | total_timesteps 1406.
Path 28 | total_timesteps 1452.
Path 29 | total_timesteps 1488.
Path 30 | total_timesteps 1563.
Path 31 | total_timesteps 1618.
Path 32 | total_timesteps 1652.
Path 33 | total_timesteps 1733.
Path 34 | total_timesteps 1779.
Path 35 | total_timesteps 1836.
Path 36 | total_timesteps 1903.
Path 37 | total_timesteps 1951.
Path 38 | total_timesteps 1995.
Path 39 | total_timesteps 2028.
Path 40 | total_timesteps 2071.
Path 41 | total_timesteps 2111.
Path 42 | total_timesteps 2154.
Path 43 | total_timesteps 2205.
Path 44 | total_timesteps 2259.
Path 45 | total_timesteps 2308.
Path 46 | total_timesteps 2338.
Path 47 | total_timesteps 2385.
Path 48 | total_timesteps 2410.
Path 49 | total_timesteps 2451.
Path 50 | total_timesteps 2488.
Path 51 | total_timesteps 2550.
Path 52 | total_timesteps 2581.
Path 53 | total_timesteps 2620.
Path 54 | total_timesteps 2662.
Path 55 | total_timesteps 2724.
Path 56 | total_timesteps 2782.
Path 57 | total_timesteps 2873.
Path 58 | total_timesteps 2977.
Path 59 | total_timesteps 3012.
Path 60 | total_timesteps 3091.
Path 61 | total_timesteps 3165.
Path 62 | total_timesteps 3213.
Path 63 | total_timesteps 3286.
Path 64 | total_timesteps 3332.
Path 65 | total_timesteps 3367.
Path 66 | total_timesteps 3389.
Path 67 | total_timesteps 3420.
Path 68 | total_timesteps 3490.
Path 69 | total_timesteps 3554.
Path 70 | total_timesteps 3581.
Path 71 | total_timesteps 3630.
Path 72 | total_timesteps 3659.
Path 73 | total_timesteps 3689.
Path 74 | total_timesteps 3742.
Path 75 | total_timesteps 3782.
Path 76 | total_timesteps 3848.
Path 77 | total_timesteps 3871.
Path 78 | total_timesteps 3935.
Path 79 | total_timesteps 3985.
Path 80 | total_timesteps 4056.
Path 81 | total_timesteps 4106.
Path 82 | total_timesteps 4145.
Path 83 | total_timesteps 4204.
Path 84 | total_timesteps 4256.
Path 85 | total_timesteps 4315.
Path 86 | total_timesteps 4337.
Path 87 | total_timesteps 4402.
Path 88 | total_timesteps 4431.
Path 89 | total_timesteps 4487.
Path 90 | total_timesteps 4536.
Path 91 | total_timesteps 4604.
Path 92 | total_timesteps 4664.
Path 93 | total_timesteps 4703.
Path 94 | total_timesteps 4756.
Path 95 | total_timesteps 4824.
Path 96 | total_timesteps 4874.
Path 97 | total_timesteps 4937.
Path 98 | total_timesteps 5108.
Path 99 | total_timesteps 5156.
Path 100 | total_timesteps 5187.
Path 101 | total_timesteps 5282.
Path 102 | total_timesteps 5314.
Path 103 | total_timesteps 5356.
Path 104 | total_timesteps 5419.
Path 105 | total_timesteps 5470.
Path 106 | total_timesteps 5548.
Path 107 | total_timesteps 5593.
Path 108 | total_timesteps 5650.
Path 109 | total_timesteps 5689.
Path 110 | total_timesteps 5762.
Path 111 | total_timesteps 5798.
Path 112 | total_timesteps 5854.
Path 113 | total_timesteps 5895.
Path 114 | total_timesteps 5929.
Path 115 | total_timesteps 5945.
Path 116 | total_timesteps 5979.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.52    |
| Iteration     | 21       |
| MaximumReturn | 73.7     |
| MinimumReturn | -56.4    |
| TotalSamples  | 92139    |
----------------------------
itr #22 | 
Fitting dynamics.
Validation loss = 0.006884949281811714
Validation loss = 0.006746227387338877
Validation loss = 0.006751397158950567
Validation loss = 0.006491354200989008
Validation loss = 0.006710840854793787
Validation loss = 0.006318490952253342
Validation loss = 0.006287073250859976
Validation loss = 0.006713087670505047
Validation loss = 0.0064996047876775265
Validation loss = 0.007148346398025751
Validation loss = 0.00650158803910017
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 77.
Path 2 | total_timesteps 118.
Path 3 | total_timesteps 180.
Path 4 | total_timesteps 244.
Path 5 | total_timesteps 283.
Path 6 | total_timesteps 328.
Path 7 | total_timesteps 365.
Path 8 | total_timesteps 416.
Path 9 | total_timesteps 472.
Path 10 | total_timesteps 519.
Path 11 | total_timesteps 570.
Path 12 | total_timesteps 610.
Path 13 | total_timesteps 638.
Path 14 | total_timesteps 673.
Path 15 | total_timesteps 707.
Path 16 | total_timesteps 795.
Path 17 | total_timesteps 823.
Path 18 | total_timesteps 872.
Path 19 | total_timesteps 931.
Path 20 | total_timesteps 977.
Path 21 | total_timesteps 1099.
Path 22 | total_timesteps 1183.
Path 23 | total_timesteps 1229.
Path 24 | total_timesteps 1287.
Path 25 | total_timesteps 1385.
Path 26 | total_timesteps 1441.
Path 27 | total_timesteps 1481.
Path 28 | total_timesteps 1533.
Path 29 | total_timesteps 1617.
Path 30 | total_timesteps 1664.
Path 31 | total_timesteps 1700.
Path 32 | total_timesteps 1757.
Path 33 | total_timesteps 1812.
Path 34 | total_timesteps 1861.
Path 35 | total_timesteps 1947.
Path 36 | total_timesteps 1981.
Path 37 | total_timesteps 2060.
Path 38 | total_timesteps 2116.
Path 39 | total_timesteps 2167.
Path 40 | total_timesteps 2214.
Path 41 | total_timesteps 2323.
Path 42 | total_timesteps 2357.
Path 43 | total_timesteps 2438.
Path 44 | total_timesteps 2488.
Path 45 | total_timesteps 2533.
Path 46 | total_timesteps 2594.
Path 47 | total_timesteps 2648.
Path 48 | total_timesteps 2706.
Path 49 | total_timesteps 2747.
Path 50 | total_timesteps 2814.
Path 51 | total_timesteps 2901.
Path 52 | total_timesteps 2948.
Path 53 | total_timesteps 3017.
Path 54 | total_timesteps 3073.
Path 55 | total_timesteps 3120.
Path 56 | total_timesteps 3162.
Path 57 | total_timesteps 3187.
Path 58 | total_timesteps 3243.
Path 59 | total_timesteps 3285.
Path 60 | total_timesteps 3358.
Path 61 | total_timesteps 3429.
Path 62 | total_timesteps 3500.
Path 63 | total_timesteps 3527.
Path 64 | total_timesteps 3577.
Path 65 | total_timesteps 3620.
Path 66 | total_timesteps 3661.
Path 67 | total_timesteps 3729.
Path 68 | total_timesteps 3759.
Path 69 | total_timesteps 3866.
Path 70 | total_timesteps 3903.
Path 71 | total_timesteps 4023.
Path 72 | total_timesteps 4090.
Path 73 | total_timesteps 4139.
Path 74 | total_timesteps 4177.
Path 75 | total_timesteps 4222.
Path 76 | total_timesteps 4254.
Path 77 | total_timesteps 4342.
Path 78 | total_timesteps 4408.
Path 79 | total_timesteps 4464.
Path 80 | total_timesteps 4526.
Path 81 | total_timesteps 4573.
Path 82 | total_timesteps 4608.
Path 83 | total_timesteps 4650.
Path 84 | total_timesteps 4724.
Path 85 | total_timesteps 4771.
Path 86 | total_timesteps 4826.
Path 87 | total_timesteps 4908.
Path 88 | total_timesteps 4950.
Path 89 | total_timesteps 4999.
Path 90 | total_timesteps 5030.
Path 91 | total_timesteps 5053.
Path 92 | total_timesteps 5103.
Path 93 | total_timesteps 5205.
Path 94 | total_timesteps 5297.
Path 95 | total_timesteps 5330.
Path 96 | total_timesteps 5408.
Path 97 | total_timesteps 5447.
Path 98 | total_timesteps 5475.
Path 99 | total_timesteps 5509.
Path 100 | total_timesteps 5551.
Path 101 | total_timesteps 5580.
Path 102 | total_timesteps 5632.
Path 103 | total_timesteps 5669.
Path 104 | total_timesteps 5712.
Path 105 | total_timesteps 5782.
Path 106 | total_timesteps 5827.
Path 107 | total_timesteps 5855.
Path 108 | total_timesteps 5903.
Path 109 | total_timesteps 5944.
Path 110 | total_timesteps 5978.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.3    |
| Iteration     | 22       |
| MaximumReturn | 80.7     |
| MinimumReturn | -68.3    |
| TotalSamples  | 96179    |
----------------------------
itr #23 | 
Fitting dynamics.
Validation loss = 0.007067948579788208
Validation loss = 0.007192134857177734
Validation loss = 0.006400085985660553
Validation loss = 0.006390926893800497
Validation loss = 0.006532568950206041
Validation loss = 0.0064629134722054005
Validation loss = 0.0066776275634765625
Validation loss = 0.006460403557866812
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 44.
Path 2 | total_timesteps 101.
Path 3 | total_timesteps 134.
Path 4 | total_timesteps 213.
Path 5 | total_timesteps 283.
Path 6 | total_timesteps 336.
Path 7 | total_timesteps 374.
Path 8 | total_timesteps 410.
Path 9 | total_timesteps 483.
Path 10 | total_timesteps 527.
Path 11 | total_timesteps 594.
Path 12 | total_timesteps 632.
Path 13 | total_timesteps 679.
Path 14 | total_timesteps 740.
Path 15 | total_timesteps 772.
Path 16 | total_timesteps 797.
Path 17 | total_timesteps 866.
Path 18 | total_timesteps 907.
Path 19 | total_timesteps 964.
Path 20 | total_timesteps 1013.
Path 21 | total_timesteps 1097.
Path 22 | total_timesteps 1136.
Path 23 | total_timesteps 1195.
Path 24 | total_timesteps 1238.
Path 25 | total_timesteps 1282.
Path 26 | total_timesteps 1332.
Path 27 | total_timesteps 1384.
Path 28 | total_timesteps 1438.
Path 29 | total_timesteps 1485.
Path 30 | total_timesteps 1533.
Path 31 | total_timesteps 1607.
Path 32 | total_timesteps 1655.
Path 33 | total_timesteps 1713.
Path 34 | total_timesteps 1770.
Path 35 | total_timesteps 1802.
Path 36 | total_timesteps 1835.
Path 37 | total_timesteps 1909.
Path 38 | total_timesteps 1963.
Path 39 | total_timesteps 2033.
Path 40 | total_timesteps 2079.
Path 41 | total_timesteps 2113.
Path 42 | total_timesteps 2143.
Path 43 | total_timesteps 2205.
Path 44 | total_timesteps 2257.
Path 45 | total_timesteps 2362.
Path 46 | total_timesteps 2398.
Path 47 | total_timesteps 2460.
Path 48 | total_timesteps 2534.
Path 49 | total_timesteps 2605.
Path 50 | total_timesteps 2694.
Path 51 | total_timesteps 2737.
Path 52 | total_timesteps 2798.
Path 53 | total_timesteps 2861.
Path 54 | total_timesteps 2889.
Path 55 | total_timesteps 2936.
Path 56 | total_timesteps 2975.
Path 57 | total_timesteps 3012.
Path 58 | total_timesteps 3045.
Path 59 | total_timesteps 3078.
Path 60 | total_timesteps 3119.
Path 61 | total_timesteps 3186.
Path 62 | total_timesteps 3255.
Path 63 | total_timesteps 3321.
Path 64 | total_timesteps 3371.
Path 65 | total_timesteps 3455.
Path 66 | total_timesteps 3508.
Path 67 | total_timesteps 3535.
Path 68 | total_timesteps 3564.
Path 69 | total_timesteps 3687.
Path 70 | total_timesteps 3718.
Path 71 | total_timesteps 3757.
Path 72 | total_timesteps 3806.
Path 73 | total_timesteps 3859.
Path 74 | total_timesteps 3913.
Path 75 | total_timesteps 4002.
Path 76 | total_timesteps 4047.
Path 77 | total_timesteps 4114.
Path 78 | total_timesteps 4139.
Path 79 | total_timesteps 4188.
Path 80 | total_timesteps 4249.
Path 81 | total_timesteps 4277.
Path 82 | total_timesteps 4326.
Path 83 | total_timesteps 4407.
Path 84 | total_timesteps 4484.
Path 85 | total_timesteps 4533.
Path 86 | total_timesteps 4559.
Path 87 | total_timesteps 4594.
Path 88 | total_timesteps 4647.
Path 89 | total_timesteps 4724.
Path 90 | total_timesteps 4792.
Path 91 | total_timesteps 4816.
Path 92 | total_timesteps 4856.
Path 93 | total_timesteps 4931.
Path 94 | total_timesteps 4993.
Path 95 | total_timesteps 5068.
Path 96 | total_timesteps 5095.
Path 97 | total_timesteps 5125.
Path 98 | total_timesteps 5175.
Path 99 | total_timesteps 5215.
Path 100 | total_timesteps 5261.
Path 101 | total_timesteps 5317.
Path 102 | total_timesteps 5343.
Path 103 | total_timesteps 5397.
Path 104 | total_timesteps 5444.
Path 105 | total_timesteps 5522.
Path 106 | total_timesteps 5586.
Path 107 | total_timesteps 5634.
Path 108 | total_timesteps 5689.
Path 109 | total_timesteps 5741.
Path 110 | total_timesteps 5765.
Path 111 | total_timesteps 5813.
Path 112 | total_timesteps 5854.
Path 113 | total_timesteps 5893.
Path 114 | total_timesteps 5931.
Path 115 | total_timesteps 5958.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -10.7    |
| Iteration     | 23       |
| MaximumReturn | 49.4     |
| MinimumReturn | -47.8    |
| TotalSamples  | 100194   |
----------------------------
itr #24 | 
Fitting dynamics.
Validation loss = 0.006566615775227547
Validation loss = 0.006774993147701025
Validation loss = 0.006281131878495216
Validation loss = 0.0065034013241529465
Validation loss = 0.006230424158275127
Validation loss = 0.006441752891987562
Validation loss = 0.006315522361546755
Validation loss = 0.006299612578004599
Validation loss = 0.006229713559150696
Validation loss = 0.006256648804992437
Validation loss = 0.006294765509665012
Validation loss = 0.006098397541791201
Validation loss = 0.006098252721130848
Validation loss = 0.006106431595981121
Validation loss = 0.0063637979328632355
Validation loss = 0.0063913301564753056
Validation loss = 0.006046903319656849
Validation loss = 0.006121441721916199
Validation loss = 0.005924329161643982
Validation loss = 0.006145617458969355
Validation loss = 0.005971936974674463
Validation loss = 0.0060712541453540325
Validation loss = 0.006309498101472855
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 47.
Path 2 | total_timesteps 96.
Path 3 | total_timesteps 146.
Path 4 | total_timesteps 201.
Path 5 | total_timesteps 265.
Path 6 | total_timesteps 330.
Path 7 | total_timesteps 362.
Path 8 | total_timesteps 467.
Path 9 | total_timesteps 521.
Path 10 | total_timesteps 564.
Path 11 | total_timesteps 639.
Path 12 | total_timesteps 695.
Path 13 | total_timesteps 744.
Path 14 | total_timesteps 804.
Path 15 | total_timesteps 862.
Path 16 | total_timesteps 914.
Path 17 | total_timesteps 971.
Path 18 | total_timesteps 997.
Path 19 | total_timesteps 1074.
Path 20 | total_timesteps 1109.
Path 21 | total_timesteps 1168.
Path 22 | total_timesteps 1208.
Path 23 | total_timesteps 1238.
Path 24 | total_timesteps 1321.
Path 25 | total_timesteps 1390.
Path 26 | total_timesteps 1455.
Path 27 | total_timesteps 1496.
Path 28 | total_timesteps 1549.
Path 29 | total_timesteps 1607.
Path 30 | total_timesteps 1661.
Path 31 | total_timesteps 1705.
Path 32 | total_timesteps 1746.
Path 33 | total_timesteps 1778.
Path 34 | total_timesteps 1828.
Path 35 | total_timesteps 1858.
Path 36 | total_timesteps 1927.
Path 37 | total_timesteps 1979.
Path 38 | total_timesteps 2031.
Path 39 | total_timesteps 2083.
Path 40 | total_timesteps 2153.
Path 41 | total_timesteps 2177.
Path 42 | total_timesteps 2248.
Path 43 | total_timesteps 2288.
Path 44 | total_timesteps 2351.
Path 45 | total_timesteps 2407.
Path 46 | total_timesteps 2458.
Path 47 | total_timesteps 2484.
Path 48 | total_timesteps 2527.
Path 49 | total_timesteps 2566.
Path 50 | total_timesteps 2668.
Path 51 | total_timesteps 2703.
Path 52 | total_timesteps 2735.
Path 53 | total_timesteps 2780.
Path 54 | total_timesteps 2804.
Path 55 | total_timesteps 2895.
Path 56 | total_timesteps 2954.
Path 57 | total_timesteps 3003.
Path 58 | total_timesteps 3049.
Path 59 | total_timesteps 3082.
Path 60 | total_timesteps 3124.
Path 61 | total_timesteps 3158.
Path 62 | total_timesteps 3215.
Path 63 | total_timesteps 3251.
Path 64 | total_timesteps 3296.
Path 65 | total_timesteps 3362.
Path 66 | total_timesteps 3418.
Path 67 | total_timesteps 3472.
Path 68 | total_timesteps 3514.
Path 69 | total_timesteps 3541.
Path 70 | total_timesteps 3587.
Path 71 | total_timesteps 3620.
Path 72 | total_timesteps 3666.
Path 73 | total_timesteps 3706.
Path 74 | total_timesteps 3733.
Path 75 | total_timesteps 3776.
Path 76 | total_timesteps 3828.
Path 77 | total_timesteps 3878.
Path 78 | total_timesteps 3933.
Path 79 | total_timesteps 3988.
Path 80 | total_timesteps 4018.
Path 81 | total_timesteps 4059.
Path 82 | total_timesteps 4121.
Path 83 | total_timesteps 4157.
Path 84 | total_timesteps 4214.
Path 85 | total_timesteps 4327.
Path 86 | total_timesteps 4378.
Path 87 | total_timesteps 4428.
Path 88 | total_timesteps 4473.
Path 89 | total_timesteps 4523.
Path 90 | total_timesteps 4585.
Path 91 | total_timesteps 4641.
Path 92 | total_timesteps 4678.
Path 93 | total_timesteps 4736.
Path 94 | total_timesteps 4786.
Path 95 | total_timesteps 4849.
Path 96 | total_timesteps 4875.
Path 97 | total_timesteps 4931.
Path 98 | total_timesteps 5016.
Path 99 | total_timesteps 5064.
Path 100 | total_timesteps 5169.
Path 101 | total_timesteps 5231.
Path 102 | total_timesteps 5291.
Path 103 | total_timesteps 5333.
Path 104 | total_timesteps 5382.
Path 105 | total_timesteps 5421.
Path 106 | total_timesteps 5444.
Path 107 | total_timesteps 5509.
Path 108 | total_timesteps 5531.
Path 109 | total_timesteps 5571.
Path 110 | total_timesteps 5613.
Path 111 | total_timesteps 5682.
Path 112 | total_timesteps 5706.
Path 113 | total_timesteps 5810.
Path 114 | total_timesteps 5882.
Path 115 | total_timesteps 5920.
Path 116 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -10.6    |
| Iteration     | 24       |
| MaximumReturn | 82.6     |
| MinimumReturn | -50.4    |
| TotalSamples  | 104234   |
----------------------------
itr #25 | 
Fitting dynamics.
Validation loss = 0.006312203593552113
Validation loss = 0.0063848416320979595
Validation loss = 0.006132533773779869
Validation loss = 0.006115383468568325
Validation loss = 0.006351962685585022
Validation loss = 0.006995154079049826
Validation loss = 0.005846145562827587
Validation loss = 0.006192052736878395
Validation loss = 0.00606881408020854
Validation loss = 0.006015291437506676
Validation loss = 0.005988001357764006
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 73.
Path 2 | total_timesteps 143.
Path 3 | total_timesteps 241.
Path 4 | total_timesteps 314.
Path 5 | total_timesteps 364.
Path 6 | total_timesteps 432.
Path 7 | total_timesteps 479.
Path 8 | total_timesteps 524.
Path 9 | total_timesteps 557.
Path 10 | total_timesteps 579.
Path 11 | total_timesteps 602.
Path 12 | total_timesteps 665.
Path 13 | total_timesteps 729.
Path 14 | total_timesteps 763.
Path 15 | total_timesteps 794.
Path 16 | total_timesteps 866.
Path 17 | total_timesteps 1013.
Path 18 | total_timesteps 1036.
Path 19 | total_timesteps 1073.
Path 20 | total_timesteps 1119.
Path 21 | total_timesteps 1152.
Path 22 | total_timesteps 1213.
Path 23 | total_timesteps 1278.
Path 24 | total_timesteps 1326.
Path 25 | total_timesteps 1370.
Path 26 | total_timesteps 1424.
Path 27 | total_timesteps 1477.
Path 28 | total_timesteps 1529.
Path 29 | total_timesteps 1585.
Path 30 | total_timesteps 1655.
Path 31 | total_timesteps 1711.
Path 32 | total_timesteps 1751.
Path 33 | total_timesteps 1807.
Path 34 | total_timesteps 1863.
Path 35 | total_timesteps 1929.
Path 36 | total_timesteps 2009.
Path 37 | total_timesteps 2058.
Path 38 | total_timesteps 2099.
Path 39 | total_timesteps 2128.
Path 40 | total_timesteps 2195.
Path 41 | total_timesteps 2289.
Path 42 | total_timesteps 2337.
Path 43 | total_timesteps 2375.
Path 44 | total_timesteps 2403.
Path 45 | total_timesteps 2442.
Path 46 | total_timesteps 2499.
Path 47 | total_timesteps 2570.
Path 48 | total_timesteps 2610.
Path 49 | total_timesteps 2649.
Path 50 | total_timesteps 2675.
Path 51 | total_timesteps 2706.
Path 52 | total_timesteps 2762.
Path 53 | total_timesteps 2800.
Path 54 | total_timesteps 2890.
Path 55 | total_timesteps 2940.
Path 56 | total_timesteps 2994.
Path 57 | total_timesteps 3042.
Path 58 | total_timesteps 3104.
Path 59 | total_timesteps 3162.
Path 60 | total_timesteps 3221.
Path 61 | total_timesteps 3283.
Path 62 | total_timesteps 3320.
Path 63 | total_timesteps 3407.
Path 64 | total_timesteps 3453.
Path 65 | total_timesteps 3497.
Path 66 | total_timesteps 3524.
Path 67 | total_timesteps 3570.
Path 68 | total_timesteps 3608.
Path 69 | total_timesteps 3645.
Path 70 | total_timesteps 3681.
Path 71 | total_timesteps 3725.
Path 72 | total_timesteps 3771.
Path 73 | total_timesteps 3811.
Path 74 | total_timesteps 3871.
Path 75 | total_timesteps 3911.
Path 76 | total_timesteps 3940.
Path 77 | total_timesteps 3967.
Path 78 | total_timesteps 4024.
Path 79 | total_timesteps 4080.
Path 80 | total_timesteps 4128.
Path 81 | total_timesteps 4153.
Path 82 | total_timesteps 4214.
Path 83 | total_timesteps 4259.
Path 84 | total_timesteps 4304.
Path 85 | total_timesteps 4338.
Path 86 | total_timesteps 4407.
Path 87 | total_timesteps 4475.
Path 88 | total_timesteps 4511.
Path 89 | total_timesteps 4562.
Path 90 | total_timesteps 4605.
Path 91 | total_timesteps 4632.
Path 92 | total_timesteps 4677.
Path 93 | total_timesteps 4718.
Path 94 | total_timesteps 4760.
Path 95 | total_timesteps 4810.
Path 96 | total_timesteps 4861.
Path 97 | total_timesteps 4915.
Path 98 | total_timesteps 4942.
Path 99 | total_timesteps 4979.
Path 100 | total_timesteps 5027.
Path 101 | total_timesteps 5137.
Path 102 | total_timesteps 5186.
Path 103 | total_timesteps 5212.
Path 104 | total_timesteps 5236.
Path 105 | total_timesteps 5309.
Path 106 | total_timesteps 5368.
Path 107 | total_timesteps 5406.
Path 108 | total_timesteps 5457.
Path 109 | total_timesteps 5512.
Path 110 | total_timesteps 5541.
Path 111 | total_timesteps 5582.
Path 112 | total_timesteps 5631.
Path 113 | total_timesteps 5691.
Path 114 | total_timesteps 5733.
Path 115 | total_timesteps 5777.
Path 116 | total_timesteps 5855.
Path 117 | total_timesteps 5877.
Path 118 | total_timesteps 5938.
Path 119 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -13.2    |
| Iteration     | 25       |
| MaximumReturn | 85.6     |
| MinimumReturn | -53.3    |
| TotalSamples  | 108269   |
----------------------------
itr #26 | 
Fitting dynamics.
Validation loss = 0.006131506524980068
Validation loss = 0.00631789630278945
Validation loss = 0.006329333409667015
Validation loss = 0.005988584831357002
Validation loss = 0.006050819996744394
Validation loss = 0.0058838683180511
Validation loss = 0.005758075974881649
Validation loss = 0.006119809113442898
Validation loss = 0.00575482239946723
Validation loss = 0.005987474229186773
Validation loss = 0.005808471702039242
Validation loss = 0.005952565930783749
Validation loss = 0.005893021356314421
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 37.
Path 2 | total_timesteps 82.
Path 3 | total_timesteps 140.
Path 4 | total_timesteps 162.
Path 5 | total_timesteps 222.
Path 6 | total_timesteps 274.
Path 7 | total_timesteps 328.
Path 8 | total_timesteps 386.
Path 9 | total_timesteps 423.
Path 10 | total_timesteps 454.
Path 11 | total_timesteps 492.
Path 12 | total_timesteps 529.
Path 13 | total_timesteps 568.
Path 14 | total_timesteps 639.
Path 15 | total_timesteps 679.
Path 16 | total_timesteps 762.
Path 17 | total_timesteps 828.
Path 18 | total_timesteps 883.
Path 19 | total_timesteps 920.
Path 20 | total_timesteps 979.
Path 21 | total_timesteps 1015.
Path 22 | total_timesteps 1055.
Path 23 | total_timesteps 1126.
Path 24 | total_timesteps 1218.
Path 25 | total_timesteps 1302.
Path 26 | total_timesteps 1395.
Path 27 | total_timesteps 1448.
Path 28 | total_timesteps 1504.
Path 29 | total_timesteps 1545.
Path 30 | total_timesteps 1588.
Path 31 | total_timesteps 1652.
Path 32 | total_timesteps 1707.
Path 33 | total_timesteps 1782.
Path 34 | total_timesteps 1845.
Path 35 | total_timesteps 1926.
Path 36 | total_timesteps 1998.
Path 37 | total_timesteps 2047.
Path 38 | total_timesteps 2109.
Path 39 | total_timesteps 2128.
Path 40 | total_timesteps 2190.
Path 41 | total_timesteps 2229.
Path 42 | total_timesteps 2272.
Path 43 | total_timesteps 2307.
Path 44 | total_timesteps 2388.
Path 45 | total_timesteps 2426.
Path 46 | total_timesteps 2475.
Path 47 | total_timesteps 2531.
Path 48 | total_timesteps 2608.
Path 49 | total_timesteps 2659.
Path 50 | total_timesteps 2694.
Path 51 | total_timesteps 2729.
Path 52 | total_timesteps 2777.
Path 53 | total_timesteps 2881.
Path 54 | total_timesteps 2910.
Path 55 | total_timesteps 2947.
Path 56 | total_timesteps 2995.
Path 57 | total_timesteps 3050.
Path 58 | total_timesteps 3100.
Path 59 | total_timesteps 3183.
Path 60 | total_timesteps 3224.
Path 61 | total_timesteps 3261.
Path 62 | total_timesteps 3284.
Path 63 | total_timesteps 3316.
Path 64 | total_timesteps 3345.
Path 65 | total_timesteps 3369.
Path 66 | total_timesteps 3434.
Path 67 | total_timesteps 3472.
Path 68 | total_timesteps 3508.
Path 69 | total_timesteps 3557.
Path 70 | total_timesteps 3630.
Path 71 | total_timesteps 3664.
Path 72 | total_timesteps 3728.
Path 73 | total_timesteps 3802.
Path 74 | total_timesteps 3852.
Path 75 | total_timesteps 3890.
Path 76 | total_timesteps 3941.
Path 77 | total_timesteps 4048.
Path 78 | total_timesteps 4130.
Path 79 | total_timesteps 4166.
Path 80 | total_timesteps 4212.
Path 81 | total_timesteps 4272.
Path 82 | total_timesteps 4301.
Path 83 | total_timesteps 4340.
Path 84 | total_timesteps 4425.
Path 85 | total_timesteps 4500.
Path 86 | total_timesteps 4538.
Path 87 | total_timesteps 4634.
Path 88 | total_timesteps 4690.
Path 89 | total_timesteps 4735.
Path 90 | total_timesteps 4791.
Path 91 | total_timesteps 4836.
Path 92 | total_timesteps 4898.
Path 93 | total_timesteps 4946.
Path 94 | total_timesteps 4984.
Path 95 | total_timesteps 5043.
Path 96 | total_timesteps 5116.
Path 97 | total_timesteps 5167.
Path 98 | total_timesteps 5219.
Path 99 | total_timesteps 5278.
Path 100 | total_timesteps 5336.
Path 101 | total_timesteps 5386.
Path 102 | total_timesteps 5490.
Path 103 | total_timesteps 5587.
Path 104 | total_timesteps 5633.
Path 105 | total_timesteps 5667.
Path 106 | total_timesteps 5710.
Path 107 | total_timesteps 5748.
Path 108 | total_timesteps 5774.
Path 109 | total_timesteps 5846.
Path 110 | total_timesteps 5919.
Path 111 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -13.7    |
| Iteration     | 26       |
| MaximumReturn | 65       |
| MinimumReturn | -60.2    |
| TotalSamples  | 112317   |
----------------------------
itr #27 | 
Fitting dynamics.
Validation loss = 0.006138170603662729
Validation loss = 0.0060306936502456665
Validation loss = 0.005788134876638651
Validation loss = 0.005844138562679291
Validation loss = 0.0056661018170416355
Validation loss = 0.005799619946628809
Validation loss = 0.006001059897243977
Validation loss = 0.005681633483618498
Validation loss = 0.005791973788291216
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 33.
Path 2 | total_timesteps 73.
Path 3 | total_timesteps 109.
Path 4 | total_timesteps 166.
Path 5 | total_timesteps 224.
Path 6 | total_timesteps 253.
Path 7 | total_timesteps 308.
Path 8 | total_timesteps 350.
Path 9 | total_timesteps 379.
Path 10 | total_timesteps 448.
Path 11 | total_timesteps 592.
Path 12 | total_timesteps 655.
Path 13 | total_timesteps 723.
Path 14 | total_timesteps 785.
Path 15 | total_timesteps 836.
Path 16 | total_timesteps 876.
Path 17 | total_timesteps 906.
Path 18 | total_timesteps 964.
Path 19 | total_timesteps 980.
Path 20 | total_timesteps 1015.
Path 21 | total_timesteps 1052.
Path 22 | total_timesteps 1086.
Path 23 | total_timesteps 1122.
Path 24 | total_timesteps 1210.
Path 25 | total_timesteps 1265.
Path 26 | total_timesteps 1344.
Path 27 | total_timesteps 1377.
Path 28 | total_timesteps 1457.
Path 29 | total_timesteps 1502.
Path 30 | total_timesteps 1542.
Path 31 | total_timesteps 1611.
Path 32 | total_timesteps 1674.
Path 33 | total_timesteps 1743.
Path 34 | total_timesteps 1792.
Path 35 | total_timesteps 1821.
Path 36 | total_timesteps 1864.
Path 37 | total_timesteps 1904.
Path 38 | total_timesteps 1957.
Path 39 | total_timesteps 2010.
Path 40 | total_timesteps 2054.
Path 41 | total_timesteps 2097.
Path 42 | total_timesteps 2124.
Path 43 | total_timesteps 2197.
Path 44 | total_timesteps 2227.
Path 45 | total_timesteps 2253.
Path 46 | total_timesteps 2294.
Path 47 | total_timesteps 2367.
Path 48 | total_timesteps 2396.
Path 49 | total_timesteps 2435.
Path 50 | total_timesteps 2492.
Path 51 | total_timesteps 2537.
Path 52 | total_timesteps 2573.
Path 53 | total_timesteps 2615.
Path 54 | total_timesteps 2682.
Path 55 | total_timesteps 2715.
Path 56 | total_timesteps 2782.
Path 57 | total_timesteps 2836.
Path 58 | total_timesteps 2920.
Path 59 | total_timesteps 2963.
Path 60 | total_timesteps 2993.
Path 61 | total_timesteps 3046.
Path 62 | total_timesteps 3116.
Path 63 | total_timesteps 3170.
Path 64 | total_timesteps 3200.
Path 65 | total_timesteps 3278.
Path 66 | total_timesteps 3345.
Path 67 | total_timesteps 3420.
Path 68 | total_timesteps 3493.
Path 69 | total_timesteps 3525.
Path 70 | total_timesteps 3583.
Path 71 | total_timesteps 3632.
Path 72 | total_timesteps 3683.
Path 73 | total_timesteps 3788.
Path 74 | total_timesteps 3874.
Path 75 | total_timesteps 3952.
Path 76 | total_timesteps 3986.
Path 77 | total_timesteps 4050.
Path 78 | total_timesteps 4089.
Path 79 | total_timesteps 4157.
Path 80 | total_timesteps 4180.
Path 81 | total_timesteps 4219.
Path 82 | total_timesteps 4254.
Path 83 | total_timesteps 4309.
Path 84 | total_timesteps 4335.
Path 85 | total_timesteps 4383.
Path 86 | total_timesteps 4431.
Path 87 | total_timesteps 4494.
Path 88 | total_timesteps 4561.
Path 89 | total_timesteps 4625.
Path 90 | total_timesteps 4728.
Path 91 | total_timesteps 4792.
Path 92 | total_timesteps 4841.
Path 93 | total_timesteps 4886.
Path 94 | total_timesteps 4949.
Path 95 | total_timesteps 4996.
Path 96 | total_timesteps 5039.
Path 97 | total_timesteps 5114.
Path 98 | total_timesteps 5160.
Path 99 | total_timesteps 5196.
Path 100 | total_timesteps 5224.
Path 101 | total_timesteps 5291.
Path 102 | total_timesteps 5346.
Path 103 | total_timesteps 5392.
Path 104 | total_timesteps 5472.
Path 105 | total_timesteps 5535.
Path 106 | total_timesteps 5598.
Path 107 | total_timesteps 5682.
Path 108 | total_timesteps 5741.
Path 109 | total_timesteps 5790.
Path 110 | total_timesteps 5848.
Path 111 | total_timesteps 5911.
Path 112 | total_timesteps 5959.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -12.7    |
| Iteration     | 27       |
| MaximumReturn | 31       |
| MinimumReturn | -50.1    |
| TotalSamples  | 116325   |
----------------------------
itr #28 | 
Fitting dynamics.
Validation loss = 0.006092574447393417
Validation loss = 0.005631620530039072
Validation loss = 0.005725765135139227
Validation loss = 0.005805667024105787
Validation loss = 0.005775589030236006
Validation loss = 0.0058677359484136105
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 58.
Path 2 | total_timesteps 124.
Path 3 | total_timesteps 170.
Path 4 | total_timesteps 263.
Path 5 | total_timesteps 317.
Path 6 | total_timesteps 356.
Path 7 | total_timesteps 447.
Path 8 | total_timesteps 506.
Path 9 | total_timesteps 577.
Path 10 | total_timesteps 607.
Path 11 | total_timesteps 649.
Path 12 | total_timesteps 673.
Path 13 | total_timesteps 712.
Path 14 | total_timesteps 766.
Path 15 | total_timesteps 818.
Path 16 | total_timesteps 859.
Path 17 | total_timesteps 904.
Path 18 | total_timesteps 949.
Path 19 | total_timesteps 1009.
Path 20 | total_timesteps 1070.
Path 21 | total_timesteps 1120.
Path 22 | total_timesteps 1142.
Path 23 | total_timesteps 1199.
Path 24 | total_timesteps 1257.
Path 25 | total_timesteps 1307.
Path 26 | total_timesteps 1372.
Path 27 | total_timesteps 1423.
Path 28 | total_timesteps 1468.
Path 29 | total_timesteps 1534.
Path 30 | total_timesteps 1611.
Path 31 | total_timesteps 1666.
Path 32 | total_timesteps 1700.
Path 33 | total_timesteps 1753.
Path 34 | total_timesteps 1801.
Path 35 | total_timesteps 1868.
Path 36 | total_timesteps 1895.
Path 37 | total_timesteps 1919.
Path 38 | total_timesteps 1960.
Path 39 | total_timesteps 2007.
Path 40 | total_timesteps 2064.
Path 41 | total_timesteps 2100.
Path 42 | total_timesteps 2185.
Path 43 | total_timesteps 2240.
Path 44 | total_timesteps 2276.
Path 45 | total_timesteps 2310.
Path 46 | total_timesteps 2373.
Path 47 | total_timesteps 2416.
Path 48 | total_timesteps 2448.
Path 49 | total_timesteps 2502.
Path 50 | total_timesteps 2567.
Path 51 | total_timesteps 2614.
Path 52 | total_timesteps 2652.
Path 53 | total_timesteps 2693.
Path 54 | total_timesteps 2731.
Path 55 | total_timesteps 2767.
Path 56 | total_timesteps 2795.
Path 57 | total_timesteps 2856.
Path 58 | total_timesteps 2899.
Path 59 | total_timesteps 2943.
Path 60 | total_timesteps 2988.
Path 61 | total_timesteps 3044.
Path 62 | total_timesteps 3085.
Path 63 | total_timesteps 3165.
Path 64 | total_timesteps 3221.
Path 65 | total_timesteps 3267.
Path 66 | total_timesteps 3350.
Path 67 | total_timesteps 3434.
Path 68 | total_timesteps 3479.
Path 69 | total_timesteps 3512.
Path 70 | total_timesteps 3545.
Path 71 | total_timesteps 3591.
Path 72 | total_timesteps 3658.
Path 73 | total_timesteps 3688.
Path 74 | total_timesteps 3748.
Path 75 | total_timesteps 3799.
Path 76 | total_timesteps 3852.
Path 77 | total_timesteps 3888.
Path 78 | total_timesteps 3920.
Path 79 | total_timesteps 3959.
Path 80 | total_timesteps 4019.
Path 81 | total_timesteps 4067.
Path 82 | total_timesteps 4129.
Path 83 | total_timesteps 4152.
Path 84 | total_timesteps 4227.
Path 85 | total_timesteps 4292.
Path 86 | total_timesteps 4338.
Path 87 | total_timesteps 4402.
Path 88 | total_timesteps 4462.
Path 89 | total_timesteps 4546.
Path 90 | total_timesteps 4597.
Path 91 | total_timesteps 4652.
Path 92 | total_timesteps 4695.
Path 93 | total_timesteps 4739.
Path 94 | total_timesteps 4786.
Path 95 | total_timesteps 4821.
Path 96 | total_timesteps 4859.
Path 97 | total_timesteps 4910.
Path 98 | total_timesteps 4965.
Path 99 | total_timesteps 4992.
Path 100 | total_timesteps 5053.
Path 101 | total_timesteps 5138.
Path 102 | total_timesteps 5209.
Path 103 | total_timesteps 5251.
Path 104 | total_timesteps 5270.
Path 105 | total_timesteps 5313.
Path 106 | total_timesteps 5357.
Path 107 | total_timesteps 5412.
Path 108 | total_timesteps 5450.
Path 109 | total_timesteps 5515.
Path 110 | total_timesteps 5547.
Path 111 | total_timesteps 5579.
Path 112 | total_timesteps 5618.
Path 113 | total_timesteps 5668.
Path 114 | total_timesteps 5724.
Path 115 | total_timesteps 5777.
Path 116 | total_timesteps 5838.
Path 117 | total_timesteps 5922.
Path 118 | total_timesteps 5953.
Path 119 | total_timesteps 5986.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -12.3    |
| Iteration     | 28       |
| MaximumReturn | 18.6     |
| MinimumReturn | -50.6    |
| TotalSamples  | 120359   |
----------------------------
itr #29 | 
Fitting dynamics.
Validation loss = 0.006084040738642216
Validation loss = 0.005916190799325705
Validation loss = 0.006182216107845306
Validation loss = 0.005755182821303606
Validation loss = 0.0057586850598454475
Validation loss = 0.005755194928497076
Validation loss = 0.005707657430320978
Validation loss = 0.006089359056204557
Validation loss = 0.006125429645180702
Validation loss = 0.00596996396780014
Validation loss = 0.005721067078411579
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 33.
Path 2 | total_timesteps 82.
Path 3 | total_timesteps 138.
Path 4 | total_timesteps 224.
Path 5 | total_timesteps 264.
Path 6 | total_timesteps 329.
Path 7 | total_timesteps 399.
Path 8 | total_timesteps 456.
Path 9 | total_timesteps 524.
Path 10 | total_timesteps 560.
Path 11 | total_timesteps 591.
Path 12 | total_timesteps 655.
Path 13 | total_timesteps 683.
Path 14 | total_timesteps 729.
Path 15 | total_timesteps 768.
Path 16 | total_timesteps 835.
Path 17 | total_timesteps 869.
Path 18 | total_timesteps 900.
Path 19 | total_timesteps 945.
Path 20 | total_timesteps 991.
Path 21 | total_timesteps 1036.
Path 22 | total_timesteps 1087.
Path 23 | total_timesteps 1156.
Path 24 | total_timesteps 1182.
Path 25 | total_timesteps 1237.
Path 26 | total_timesteps 1297.
Path 27 | total_timesteps 1341.
Path 28 | total_timesteps 1372.
Path 29 | total_timesteps 1424.
Path 30 | total_timesteps 1457.
Path 31 | total_timesteps 1501.
Path 32 | total_timesteps 1530.
Path 33 | total_timesteps 1571.
Path 34 | total_timesteps 1603.
Path 35 | total_timesteps 1665.
Path 36 | total_timesteps 1711.
Path 37 | total_timesteps 1749.
Path 38 | total_timesteps 1795.
Path 39 | total_timesteps 1818.
Path 40 | total_timesteps 1854.
Path 41 | total_timesteps 1951.
Path 42 | total_timesteps 2053.
Path 43 | total_timesteps 2102.
Path 44 | total_timesteps 2175.
Path 45 | total_timesteps 2214.
Path 46 | total_timesteps 2261.
Path 47 | total_timesteps 2312.
Path 48 | total_timesteps 2344.
Path 49 | total_timesteps 2395.
Path 50 | total_timesteps 2438.
Path 51 | total_timesteps 2464.
Path 52 | total_timesteps 2509.
Path 53 | total_timesteps 2549.
Path 54 | total_timesteps 2576.
Path 55 | total_timesteps 2630.
Path 56 | total_timesteps 2701.
Path 57 | total_timesteps 2741.
Path 58 | total_timesteps 2846.
Path 59 | total_timesteps 2901.
Path 60 | total_timesteps 2942.
Path 61 | total_timesteps 2998.
Path 62 | total_timesteps 3058.
Path 63 | total_timesteps 3111.
Path 64 | total_timesteps 3149.
Path 65 | total_timesteps 3172.
Path 66 | total_timesteps 3212.
Path 67 | total_timesteps 3272.
Path 68 | total_timesteps 3303.
Path 69 | total_timesteps 3342.
Path 70 | total_timesteps 3411.
Path 71 | total_timesteps 3438.
Path 72 | total_timesteps 3485.
Path 73 | total_timesteps 3525.
Path 74 | total_timesteps 3561.
Path 75 | total_timesteps 3593.
Path 76 | total_timesteps 3647.
Path 77 | total_timesteps 3682.
Path 78 | total_timesteps 3723.
Path 79 | total_timesteps 3772.
Path 80 | total_timesteps 3810.
Path 81 | total_timesteps 3842.
Path 82 | total_timesteps 3877.
Path 83 | total_timesteps 3919.
Path 84 | total_timesteps 3957.
Path 85 | total_timesteps 4004.
Path 86 | total_timesteps 4054.
Path 87 | total_timesteps 4109.
Path 88 | total_timesteps 4152.
Path 89 | total_timesteps 4208.
Path 90 | total_timesteps 4240.
Path 91 | total_timesteps 4303.
Path 92 | total_timesteps 4347.
Path 93 | total_timesteps 4412.
Path 94 | total_timesteps 4444.
Path 95 | total_timesteps 4490.
Path 96 | total_timesteps 4533.
Path 97 | total_timesteps 4581.
Path 98 | total_timesteps 4629.
Path 99 | total_timesteps 4655.
Path 100 | total_timesteps 4695.
Path 101 | total_timesteps 4727.
Path 102 | total_timesteps 4789.
Path 103 | total_timesteps 4850.
Path 104 | total_timesteps 4904.
Path 105 | total_timesteps 4961.
Path 106 | total_timesteps 4994.
Path 107 | total_timesteps 5031.
Path 108 | total_timesteps 5075.
Path 109 | total_timesteps 5091.
Path 110 | total_timesteps 5129.
Path 111 | total_timesteps 5168.
Path 112 | total_timesteps 5207.
Path 113 | total_timesteps 5247.
Path 114 | total_timesteps 5272.
Path 115 | total_timesteps 5296.
Path 116 | total_timesteps 5335.
Path 117 | total_timesteps 5400.
Path 118 | total_timesteps 5449.
Path 119 | total_timesteps 5499.
Path 120 | total_timesteps 5549.
Path 121 | total_timesteps 5580.
Path 122 | total_timesteps 5607.
Path 123 | total_timesteps 5651.
Path 124 | total_timesteps 5683.
Path 125 | total_timesteps 5740.
Path 126 | total_timesteps 5790.
Path 127 | total_timesteps 5843.
Path 128 | total_timesteps 5906.
Path 129 | total_timesteps 5933.
Path 130 | total_timesteps 5983.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -12.7    |
| Iteration     | 29       |
| MaximumReturn | 86.9     |
| MinimumReturn | -44.5    |
| TotalSamples  | 124396   |
----------------------------
itr #30 | 
Fitting dynamics.
Validation loss = 0.005952517502009869
Validation loss = 0.005652356892824173
Validation loss = 0.005863802973181009
Validation loss = 0.005612463690340519
Validation loss = 0.005568406544625759
Validation loss = 0.005614540074020624
Validation loss = 0.005538993049412966
Validation loss = 0.005625992082059383
Validation loss = 0.005464971996843815
Validation loss = 0.005618822295218706
Validation loss = 0.005491738207638264
Validation loss = 0.0054547893814742565
Validation loss = 0.005542928818613291
Validation loss = 0.0060075074434280396
Validation loss = 0.00541952159255743
Validation loss = 0.005660259630531073
Validation loss = 0.005440146662294865
Validation loss = 0.005419347435235977
Validation loss = 0.0059902723878622055
Validation loss = 0.005626472178846598
Validation loss = 0.005595637951046228
Validation loss = 0.00547489570453763
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 53.
Path 2 | total_timesteps 107.
Path 3 | total_timesteps 143.
Path 4 | total_timesteps 180.
Path 5 | total_timesteps 217.
Path 6 | total_timesteps 262.
Path 7 | total_timesteps 323.
Path 8 | total_timesteps 353.
Path 9 | total_timesteps 431.
Path 10 | total_timesteps 468.
Path 11 | total_timesteps 528.
Path 12 | total_timesteps 579.
Path 13 | total_timesteps 612.
Path 14 | total_timesteps 649.
Path 15 | total_timesteps 695.
Path 16 | total_timesteps 716.
Path 17 | total_timesteps 832.
Path 18 | total_timesteps 896.
Path 19 | total_timesteps 945.
Path 20 | total_timesteps 1019.
Path 21 | total_timesteps 1041.
Path 22 | total_timesteps 1164.
Path 23 | total_timesteps 1207.
Path 24 | total_timesteps 1257.
Path 25 | total_timesteps 1302.
Path 26 | total_timesteps 1327.
Path 27 | total_timesteps 1370.
Path 28 | total_timesteps 1424.
Path 29 | total_timesteps 1496.
Path 30 | total_timesteps 1527.
Path 31 | total_timesteps 1567.
Path 32 | total_timesteps 1597.
Path 33 | total_timesteps 1654.
Path 34 | total_timesteps 1685.
Path 35 | total_timesteps 1741.
Path 36 | total_timesteps 1792.
Path 37 | total_timesteps 1829.
Path 38 | total_timesteps 1885.
Path 39 | total_timesteps 1996.
Path 40 | total_timesteps 2060.
Path 41 | total_timesteps 2111.
Path 42 | total_timesteps 2156.
Path 43 | total_timesteps 2186.
Path 44 | total_timesteps 2217.
Path 45 | total_timesteps 2267.
Path 46 | total_timesteps 2326.
Path 47 | total_timesteps 2387.
Path 48 | total_timesteps 2411.
Path 49 | total_timesteps 2462.
Path 50 | total_timesteps 2509.
Path 51 | total_timesteps 2604.
Path 52 | total_timesteps 2639.
Path 53 | total_timesteps 2681.
Path 54 | total_timesteps 2755.
Path 55 | total_timesteps 2785.
Path 56 | total_timesteps 2850.
Path 57 | total_timesteps 2880.
Path 58 | total_timesteps 2918.
Path 59 | total_timesteps 2952.
Path 60 | total_timesteps 2987.
Path 61 | total_timesteps 3023.
Path 62 | total_timesteps 3059.
Path 63 | total_timesteps 3091.
Path 64 | total_timesteps 3140.
Path 65 | total_timesteps 3186.
Path 66 | total_timesteps 3217.
Path 67 | total_timesteps 3245.
Path 68 | total_timesteps 3272.
Path 69 | total_timesteps 3301.
Path 70 | total_timesteps 3349.
Path 71 | total_timesteps 3369.
Path 72 | total_timesteps 3409.
Path 73 | total_timesteps 3448.
Path 74 | total_timesteps 3499.
Path 75 | total_timesteps 3554.
Path 76 | total_timesteps 3590.
Path 77 | total_timesteps 3640.
Path 78 | total_timesteps 3675.
Path 79 | total_timesteps 3713.
Path 80 | total_timesteps 3745.
Path 81 | total_timesteps 3777.
Path 82 | total_timesteps 3840.
Path 83 | total_timesteps 3893.
Path 84 | total_timesteps 3935.
Path 85 | total_timesteps 3978.
Path 86 | total_timesteps 4050.
Path 87 | total_timesteps 4091.
Path 88 | total_timesteps 4146.
Path 89 | total_timesteps 4193.
Path 90 | total_timesteps 4258.
Path 91 | total_timesteps 4328.
Path 92 | total_timesteps 4365.
Path 93 | total_timesteps 4427.
Path 94 | total_timesteps 4472.
Path 95 | total_timesteps 4510.
Path 96 | total_timesteps 4538.
Path 97 | total_timesteps 4584.
Path 98 | total_timesteps 4615.
Path 99 | total_timesteps 4637.
Path 100 | total_timesteps 4708.
Path 101 | total_timesteps 4759.
Path 102 | total_timesteps 4799.
Path 103 | total_timesteps 4831.
Path 104 | total_timesteps 4874.
Path 105 | total_timesteps 4955.
Path 106 | total_timesteps 5022.
Path 107 | total_timesteps 5066.
Path 108 | total_timesteps 5106.
Path 109 | total_timesteps 5153.
Path 110 | total_timesteps 5195.
Path 111 | total_timesteps 5230.
Path 112 | total_timesteps 5265.
Path 113 | total_timesteps 5310.
Path 114 | total_timesteps 5358.
Path 115 | total_timesteps 5421.
Path 116 | total_timesteps 5475.
Path 117 | total_timesteps 5544.
Path 118 | total_timesteps 5583.
Path 119 | total_timesteps 5607.
Path 120 | total_timesteps 5665.
Path 121 | total_timesteps 5704.
Path 122 | total_timesteps 5756.
Path 123 | total_timesteps 5796.
Path 124 | total_timesteps 5849.
Path 125 | total_timesteps 5896.
Path 126 | total_timesteps 5968.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -13.9    |
| Iteration     | 30       |
| MaximumReturn | 59.4     |
| MinimumReturn | -43.5    |
| TotalSamples  | 128415   |
----------------------------
itr #31 | 
Fitting dynamics.
Validation loss = 0.006615688093006611
Validation loss = 0.005444964859634638
Validation loss = 0.00537335779517889
Validation loss = 0.005406755022704601
Validation loss = 0.005439574830234051
Validation loss = 0.005697146989405155
Validation loss = 0.005333651788532734
Validation loss = 0.005282707046717405
Validation loss = 0.005539225414395332
Validation loss = 0.005490867421030998
Validation loss = 0.005328909028321505
Validation loss = 0.005506871268153191
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 45.
Path 2 | total_timesteps 79.
Path 3 | total_timesteps 121.
Path 4 | total_timesteps 170.
Path 5 | total_timesteps 223.
Path 6 | total_timesteps 261.
Path 7 | total_timesteps 291.
Path 8 | total_timesteps 367.
Path 9 | total_timesteps 398.
Path 10 | total_timesteps 431.
Path 11 | total_timesteps 459.
Path 12 | total_timesteps 528.
Path 13 | total_timesteps 580.
Path 14 | total_timesteps 632.
Path 15 | total_timesteps 673.
Path 16 | total_timesteps 763.
Path 17 | total_timesteps 848.
Path 18 | total_timesteps 896.
Path 19 | total_timesteps 924.
Path 20 | total_timesteps 958.
Path 21 | total_timesteps 1017.
Path 22 | total_timesteps 1071.
Path 23 | total_timesteps 1127.
Path 24 | total_timesteps 1176.
Path 25 | total_timesteps 1305.
Path 26 | total_timesteps 1341.
Path 27 | total_timesteps 1421.
Path 28 | total_timesteps 1498.
Path 29 | total_timesteps 1553.
Path 30 | total_timesteps 1603.
Path 31 | total_timesteps 1645.
Path 32 | total_timesteps 1675.
Path 33 | total_timesteps 1715.
Path 34 | total_timesteps 1752.
Path 35 | total_timesteps 1770.
Path 36 | total_timesteps 1841.
Path 37 | total_timesteps 1876.
Path 38 | total_timesteps 1905.
Path 39 | total_timesteps 1975.
Path 40 | total_timesteps 2040.
Path 41 | total_timesteps 2083.
Path 42 | total_timesteps 2143.
Path 43 | total_timesteps 2209.
Path 44 | total_timesteps 2267.
Path 45 | total_timesteps 2292.
Path 46 | total_timesteps 2369.
Path 47 | total_timesteps 2445.
Path 48 | total_timesteps 2476.
Path 49 | total_timesteps 2503.
Path 50 | total_timesteps 2561.
Path 51 | total_timesteps 2605.
Path 52 | total_timesteps 2652.
Path 53 | total_timesteps 2697.
Path 54 | total_timesteps 2736.
Path 55 | total_timesteps 2770.
Path 56 | total_timesteps 2800.
Path 57 | total_timesteps 2861.
Path 58 | total_timesteps 2942.
Path 59 | total_timesteps 2984.
Path 60 | total_timesteps 3027.
Path 61 | total_timesteps 3132.
Path 62 | total_timesteps 3191.
Path 63 | total_timesteps 3243.
Path 64 | total_timesteps 3366.
Path 65 | total_timesteps 3468.
Path 66 | total_timesteps 3494.
Path 67 | total_timesteps 3552.
Path 68 | total_timesteps 3609.
Path 69 | total_timesteps 3657.
Path 70 | total_timesteps 3716.
Path 71 | total_timesteps 3742.
Path 72 | total_timesteps 3768.
Path 73 | total_timesteps 3825.
Path 74 | total_timesteps 3859.
Path 75 | total_timesteps 3902.
Path 76 | total_timesteps 3939.
Path 77 | total_timesteps 4007.
Path 78 | total_timesteps 4054.
Path 79 | total_timesteps 4108.
Path 80 | total_timesteps 4178.
Path 81 | total_timesteps 4225.
Path 82 | total_timesteps 4291.
Path 83 | total_timesteps 4328.
Path 84 | total_timesteps 4374.
Path 85 | total_timesteps 4442.
Path 86 | total_timesteps 4474.
Path 87 | total_timesteps 4525.
Path 88 | total_timesteps 4593.
Path 89 | total_timesteps 4638.
Path 90 | total_timesteps 4724.
Path 91 | total_timesteps 4775.
Path 92 | total_timesteps 4827.
Path 93 | total_timesteps 4917.
Path 94 | total_timesteps 4948.
Path 95 | total_timesteps 5024.
Path 96 | total_timesteps 5055.
Path 97 | total_timesteps 5100.
Path 98 | total_timesteps 5147.
Path 99 | total_timesteps 5212.
Path 100 | total_timesteps 5266.
Path 101 | total_timesteps 5305.
Path 102 | total_timesteps 5349.
Path 103 | total_timesteps 5389.
Path 104 | total_timesteps 5453.
Path 105 | total_timesteps 5530.
Path 106 | total_timesteps 5557.
Path 107 | total_timesteps 5620.
Path 108 | total_timesteps 5673.
Path 109 | total_timesteps 5726.
Path 110 | total_timesteps 5794.
Path 111 | total_timesteps 5869.
Path 112 | total_timesteps 5904.
Path 113 | total_timesteps 5942.
Path 114 | total_timesteps 5977.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.7    |
| Iteration     | 31       |
| MaximumReturn | 118      |
| MinimumReturn | -47.1    |
| TotalSamples  | 132425   |
----------------------------
itr #32 | 
Fitting dynamics.
Validation loss = 0.0056418366730213165
Validation loss = 0.005436582490801811
Validation loss = 0.005274437367916107
Validation loss = 0.005440784618258476
Validation loss = 0.005430480930954218
Validation loss = 0.005529292393475771
Validation loss = 0.00528373708948493
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 49.
Path 2 | total_timesteps 87.
Path 3 | total_timesteps 123.
Path 4 | total_timesteps 184.
Path 5 | total_timesteps 240.
Path 6 | total_timesteps 280.
Path 7 | total_timesteps 326.
Path 8 | total_timesteps 381.
Path 9 | total_timesteps 444.
Path 10 | total_timesteps 480.
Path 11 | total_timesteps 537.
Path 12 | total_timesteps 609.
Path 13 | total_timesteps 691.
Path 14 | total_timesteps 720.
Path 15 | total_timesteps 753.
Path 16 | total_timesteps 787.
Path 17 | total_timesteps 843.
Path 18 | total_timesteps 873.
Path 19 | total_timesteps 905.
Path 20 | total_timesteps 945.
Path 21 | total_timesteps 992.
Path 22 | total_timesteps 1067.
Path 23 | total_timesteps 1145.
Path 24 | total_timesteps 1199.
Path 25 | total_timesteps 1231.
Path 26 | total_timesteps 1303.
Path 27 | total_timesteps 1348.
Path 28 | total_timesteps 1380.
Path 29 | total_timesteps 1437.
Path 30 | total_timesteps 1491.
Path 31 | total_timesteps 1551.
Path 32 | total_timesteps 1597.
Path 33 | total_timesteps 1652.
Path 34 | total_timesteps 1691.
Path 35 | total_timesteps 1731.
Path 36 | total_timesteps 1799.
Path 37 | total_timesteps 1828.
Path 38 | total_timesteps 1864.
Path 39 | total_timesteps 1907.
Path 40 | total_timesteps 1944.
Path 41 | total_timesteps 1989.
Path 42 | total_timesteps 2043.
Path 43 | total_timesteps 2066.
Path 44 | total_timesteps 2134.
Path 45 | total_timesteps 2215.
Path 46 | total_timesteps 2285.
Path 47 | total_timesteps 2332.
Path 48 | total_timesteps 2387.
Path 49 | total_timesteps 2421.
Path 50 | total_timesteps 2455.
Path 51 | total_timesteps 2493.
Path 52 | total_timesteps 2572.
Path 53 | total_timesteps 2633.
Path 54 | total_timesteps 2673.
Path 55 | total_timesteps 2726.
Path 56 | total_timesteps 2767.
Path 57 | total_timesteps 2825.
Path 58 | total_timesteps 2903.
Path 59 | total_timesteps 2951.
Path 60 | total_timesteps 3062.
Path 61 | total_timesteps 3119.
Path 62 | total_timesteps 3157.
Path 63 | total_timesteps 3227.
Path 64 | total_timesteps 3276.
Path 65 | total_timesteps 3311.
Path 66 | total_timesteps 3353.
Path 67 | total_timesteps 3422.
Path 68 | total_timesteps 3458.
Path 69 | total_timesteps 3497.
Path 70 | total_timesteps 3564.
Path 71 | total_timesteps 3595.
Path 72 | total_timesteps 3654.
Path 73 | total_timesteps 3691.
Path 74 | total_timesteps 3710.
Path 75 | total_timesteps 3743.
Path 76 | total_timesteps 3785.
Path 77 | total_timesteps 3821.
Path 78 | total_timesteps 3876.
Path 79 | total_timesteps 3905.
Path 80 | total_timesteps 3966.
Path 81 | total_timesteps 4019.
Path 82 | total_timesteps 4057.
Path 83 | total_timesteps 4083.
Path 84 | total_timesteps 4136.
Path 85 | total_timesteps 4171.
Path 86 | total_timesteps 4210.
Path 87 | total_timesteps 4295.
Path 88 | total_timesteps 4329.
Path 89 | total_timesteps 4365.
Path 90 | total_timesteps 4410.
Path 91 | total_timesteps 4456.
Path 92 | total_timesteps 4497.
Path 93 | total_timesteps 4538.
Path 94 | total_timesteps 4605.
Path 95 | total_timesteps 4641.
Path 96 | total_timesteps 4664.
Path 97 | total_timesteps 4729.
Path 98 | total_timesteps 4788.
Path 99 | total_timesteps 4837.
Path 100 | total_timesteps 4932.
Path 101 | total_timesteps 4970.
Path 102 | total_timesteps 5046.
Path 103 | total_timesteps 5079.
Path 104 | total_timesteps 5127.
Path 105 | total_timesteps 5172.
Path 106 | total_timesteps 5219.
Path 107 | total_timesteps 5290.
Path 108 | total_timesteps 5340.
Path 109 | total_timesteps 5379.
Path 110 | total_timesteps 5420.
Path 111 | total_timesteps 5443.
Path 112 | total_timesteps 5494.
Path 113 | total_timesteps 5553.
Path 114 | total_timesteps 5610.
Path 115 | total_timesteps 5692.
Path 116 | total_timesteps 5769.
Path 117 | total_timesteps 5812.
Path 118 | total_timesteps 5839.
Path 119 | total_timesteps 5950.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.9    |
| Iteration     | 32       |
| MaximumReturn | 55.3     |
| MinimumReturn | -51.6    |
| TotalSamples  | 136435   |
----------------------------
