Logging to experiments/gym_fwalker2d/W/Mon-07-Nov-2022-10-28-41-AM-CST_gym_fwalker2d_trpo_iteration_20_seed2531
Print configuration .....
{'env_name': 'gym_fwalker2d', 'random_seeds': [3214, 2431, 2531, 2231], 'save_variables': False, 'model_save_dir': '/tmp/gym_fwalker2d_models/', 'restore_variables': False, 'start_onpol_iter': 0, 'onpol_iters': 33, 'num_path_random': 6, 'num_path_onpol': 6, 'env_horizon': 1000, 'max_train_data': 200000, 'max_val_data': 100000, 'discard_ratio': 0.0, 'dynamics': {'pre_training': {'mode': 'intrinsic_reward', 'itr': 0, 'policy_itr': 20}, 'model': 'nn', 'ensemble': False, 'ensemble_model_count': 5, 'enable_particle_ensemble': True, 'particles': 5, 'obs_var': 1.0, 'intrinsic_reward_coeff': 1.0, 'ita': 1.0, 'mode': 'random', 'val': True, 'n_layers': 4, 'hidden_size': 1000, 'activation': 'relu', 'batch_size': 1000, 'learning_rate': 0.001, 'reg_coeff': 0.0, 'epochs': 200, 'kfac_params': {'learning_rate': 0.1, 'damping': 0.001, 'momentum': 0.9, 'kl_clip': 0.0001, 'cov_ema_decay': 0.99}}, 'policy': {'network_shape': [64, 64], 'init_logstd': 0.0, 'activation': 'tanh', 'reinitialize_every_itr': False}, 'trpo': {'horizon': 1000, 'gamma': 0.99, 'step_size': 0.01, 'iterations': 20, 'batch_size': 50000, 'gae': 0.95, 'visualization': False, 'visualize_iterations': [0]}, 'algo': 'trpo'}
Generating random rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 27.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 62.
Path 5 | total_timesteps 77.
Path 6 | total_timesteps 93.
Path 7 | total_timesteps 106.
Path 8 | total_timesteps 129.
Path 9 | total_timesteps 142.
Path 10 | total_timesteps 173.
Path 11 | total_timesteps 187.
Path 12 | total_timesteps 199.
Path 13 | total_timesteps 249.
Path 14 | total_timesteps 260.
Path 15 | total_timesteps 285.
Path 16 | total_timesteps 302.
Path 17 | total_timesteps 319.
Path 18 | total_timesteps 353.
Path 19 | total_timesteps 372.
Path 20 | total_timesteps 393.
Path 21 | total_timesteps 405.
Path 22 | total_timesteps 430.
Path 23 | total_timesteps 457.
Path 24 | total_timesteps 481.
Path 25 | total_timesteps 492.
Path 26 | total_timesteps 504.
Path 27 | total_timesteps 538.
Path 28 | total_timesteps 558.
Path 29 | total_timesteps 578.
Path 30 | total_timesteps 599.
Path 31 | total_timesteps 625.
Path 32 | total_timesteps 654.
Path 33 | total_timesteps 667.
Path 34 | total_timesteps 688.
Path 35 | total_timesteps 700.
Path 36 | total_timesteps 715.
Path 37 | total_timesteps 739.
Path 38 | total_timesteps 754.
Path 39 | total_timesteps 767.
Path 40 | total_timesteps 786.
Path 41 | total_timesteps 804.
Path 42 | total_timesteps 818.
Path 43 | total_timesteps 841.
Path 44 | total_timesteps 862.
Path 45 | total_timesteps 885.
Path 46 | total_timesteps 896.
Path 47 | total_timesteps 909.
Path 48 | total_timesteps 924.
Path 49 | total_timesteps 936.
Path 50 | total_timesteps 947.
Path 51 | total_timesteps 969.
Path 52 | total_timesteps 984.
Path 53 | total_timesteps 1009.
Path 54 | total_timesteps 1047.
Path 55 | total_timesteps 1067.
Path 56 | total_timesteps 1090.
Path 57 | total_timesteps 1120.
Path 58 | total_timesteps 1132.
Path 59 | total_timesteps 1149.
Path 60 | total_timesteps 1182.
Path 61 | total_timesteps 1202.
Path 62 | total_timesteps 1224.
Path 63 | total_timesteps 1240.
Path 64 | total_timesteps 1256.
Path 65 | total_timesteps 1285.
Path 66 | total_timesteps 1308.
Path 67 | total_timesteps 1332.
Path 68 | total_timesteps 1352.
Path 69 | total_timesteps 1365.
Path 70 | total_timesteps 1384.
Path 71 | total_timesteps 1400.
Path 72 | total_timesteps 1414.
Path 73 | total_timesteps 1432.
Path 74 | total_timesteps 1449.
Path 75 | total_timesteps 1466.
Path 76 | total_timesteps 1487.
Path 77 | total_timesteps 1510.
Path 78 | total_timesteps 1520.
Path 79 | total_timesteps 1540.
Path 80 | total_timesteps 1568.
Path 81 | total_timesteps 1586.
Path 82 | total_timesteps 1604.
Path 83 | total_timesteps 1621.
Path 84 | total_timesteps 1646.
Path 85 | total_timesteps 1665.
Path 86 | total_timesteps 1684.
Path 87 | total_timesteps 1701.
Path 88 | total_timesteps 1736.
Path 89 | total_timesteps 1748.
Path 90 | total_timesteps 1783.
Path 91 | total_timesteps 1796.
Path 92 | total_timesteps 1817.
Path 93 | total_timesteps 1831.
Path 94 | total_timesteps 1867.
Path 95 | total_timesteps 1888.
Path 96 | total_timesteps 1914.
Path 97 | total_timesteps 1925.
Path 98 | total_timesteps 1937.
Path 99 | total_timesteps 1960.
Path 100 | total_timesteps 1988.
Path 101 | total_timesteps 2004.
Path 102 | total_timesteps 2035.
Path 103 | total_timesteps 2048.
Path 104 | total_timesteps 2068.
Path 105 | total_timesteps 2085.
Path 106 | total_timesteps 2099.
Path 107 | total_timesteps 2124.
Path 108 | total_timesteps 2150.
Path 109 | total_timesteps 2165.
Path 110 | total_timesteps 2176.
Path 111 | total_timesteps 2189.
Path 112 | total_timesteps 2202.
Path 113 | total_timesteps 2222.
Path 114 | total_timesteps 2239.
Path 115 | total_timesteps 2254.
Path 116 | total_timesteps 2269.
Path 117 | total_timesteps 2298.
Path 118 | total_timesteps 2310.
Path 119 | total_timesteps 2344.
Path 120 | total_timesteps 2357.
Path 121 | total_timesteps 2372.
Path 122 | total_timesteps 2393.
Path 123 | total_timesteps 2410.
Path 124 | total_timesteps 2423.
Path 125 | total_timesteps 2444.
Path 126 | total_timesteps 2469.
Path 127 | total_timesteps 2481.
Path 128 | total_timesteps 2493.
Path 129 | total_timesteps 2536.
Path 130 | total_timesteps 2557.
Path 131 | total_timesteps 2576.
Path 132 | total_timesteps 2610.
Path 133 | total_timesteps 2631.
Path 134 | total_timesteps 2651.
Path 135 | total_timesteps 2666.
Path 136 | total_timesteps 2687.
Path 137 | total_timesteps 2699.
Path 138 | total_timesteps 2719.
Path 139 | total_timesteps 2732.
Path 140 | total_timesteps 2747.
Path 141 | total_timesteps 2761.
Path 142 | total_timesteps 2807.
Path 143 | total_timesteps 2837.
Path 144 | total_timesteps 2868.
Path 145 | total_timesteps 2880.
Path 146 | total_timesteps 2894.
Path 147 | total_timesteps 2918.
Path 148 | total_timesteps 2937.
Path 149 | total_timesteps 2950.
Path 150 | total_timesteps 2967.
Path 151 | total_timesteps 2977.
Path 152 | total_timesteps 2993.
Path 153 | total_timesteps 3013.
Path 154 | total_timesteps 3027.
Path 155 | total_timesteps 3038.
Path 156 | total_timesteps 3069.
Path 157 | total_timesteps 3080.
Path 158 | total_timesteps 3093.
Path 159 | total_timesteps 3105.
Path 160 | total_timesteps 3139.
Path 161 | total_timesteps 3159.
Path 162 | total_timesteps 3174.
Path 163 | total_timesteps 3198.
Path 164 | total_timesteps 3226.
Path 165 | total_timesteps 3242.
Path 166 | total_timesteps 3278.
Path 167 | total_timesteps 3294.
Path 168 | total_timesteps 3334.
Path 169 | total_timesteps 3354.
Path 170 | total_timesteps 3377.
Path 171 | total_timesteps 3401.
Path 172 | total_timesteps 3431.
Path 173 | total_timesteps 3455.
Path 174 | total_timesteps 3482.
Path 175 | total_timesteps 3500.
Path 176 | total_timesteps 3519.
Path 177 | total_timesteps 3544.
Path 178 | total_timesteps 3569.
Path 179 | total_timesteps 3600.
Path 180 | total_timesteps 3661.
Path 181 | total_timesteps 3693.
Path 182 | total_timesteps 3715.
Path 183 | total_timesteps 3741.
Path 184 | total_timesteps 3771.
Path 185 | total_timesteps 3781.
Path 186 | total_timesteps 3799.
Path 187 | total_timesteps 3816.
Path 188 | total_timesteps 3867.
Path 189 | total_timesteps 3881.
Path 190 | total_timesteps 3917.
Path 191 | total_timesteps 3933.
Path 192 | total_timesteps 3950.
Path 193 | total_timesteps 3966.
Path 194 | total_timesteps 3980.
Path 195 | total_timesteps 4004.
Path 196 | total_timesteps 4021.
Path 197 | total_timesteps 4058.
Path 198 | total_timesteps 4074.
Path 199 | total_timesteps 4085.
Path 200 | total_timesteps 4107.
Path 201 | total_timesteps 4122.
Path 202 | total_timesteps 4151.
Path 203 | total_timesteps 4164.
Path 204 | total_timesteps 4201.
Path 205 | total_timesteps 4222.
Path 206 | total_timesteps 4237.
Path 207 | total_timesteps 4258.
Path 208 | total_timesteps 4272.
Path 209 | total_timesteps 4308.
Path 210 | total_timesteps 4325.
Path 211 | total_timesteps 4357.
Path 212 | total_timesteps 4374.
Path 213 | total_timesteps 4425.
Path 214 | total_timesteps 4444.
Path 215 | total_timesteps 4479.
Path 216 | total_timesteps 4499.
Path 217 | total_timesteps 4524.
Path 218 | total_timesteps 4545.
Path 219 | total_timesteps 4560.
Path 220 | total_timesteps 4582.
Path 221 | total_timesteps 4612.
Path 222 | total_timesteps 4623.
Path 223 | total_timesteps 4635.
Path 224 | total_timesteps 4684.
Path 225 | total_timesteps 4696.
Path 226 | total_timesteps 4717.
Path 227 | total_timesteps 4740.
Path 228 | total_timesteps 4761.
Path 229 | total_timesteps 4782.
Path 230 | total_timesteps 4808.
Path 231 | total_timesteps 4832.
Path 232 | total_timesteps 4849.
Path 233 | total_timesteps 4871.
Path 234 | total_timesteps 4892.
Path 235 | total_timesteps 4906.
Path 236 | total_timesteps 4923.
Path 237 | total_timesteps 4937.
Path 238 | total_timesteps 4954.
Path 239 | total_timesteps 4983.
Path 240 | total_timesteps 4991.
Path 241 | total_timesteps 5017.
Path 242 | total_timesteps 5034.
Path 243 | total_timesteps 5055.
Path 244 | total_timesteps 5078.
Path 245 | total_timesteps 5097.
Path 246 | total_timesteps 5113.
Path 247 | total_timesteps 5126.
Path 248 | total_timesteps 5166.
Path 249 | total_timesteps 5177.
Path 250 | total_timesteps 5204.
Path 251 | total_timesteps 5233.
Path 252 | total_timesteps 5254.
Path 253 | total_timesteps 5271.
Path 254 | total_timesteps 5287.
Path 255 | total_timesteps 5312.
Path 256 | total_timesteps 5361.
Path 257 | total_timesteps 5386.
Path 258 | total_timesteps 5409.
Path 259 | total_timesteps 5419.
Path 260 | total_timesteps 5452.
Path 261 | total_timesteps 5470.
Path 262 | total_timesteps 5491.
Path 263 | total_timesteps 5546.
Path 264 | total_timesteps 5570.
Path 265 | total_timesteps 5589.
Path 266 | total_timesteps 5606.
Path 267 | total_timesteps 5626.
Path 268 | total_timesteps 5639.
Path 269 | total_timesteps 5659.
Path 270 | total_timesteps 5675.
Path 271 | total_timesteps 5700.
Path 272 | total_timesteps 5712.
Path 273 | total_timesteps 5723.
Path 274 | total_timesteps 5741.
Path 275 | total_timesteps 5752.
Path 276 | total_timesteps 5761.
Path 277 | total_timesteps 5782.
Path 278 | total_timesteps 5798.
Path 279 | total_timesteps 5817.
Path 280 | total_timesteps 5829.
Path 281 | total_timesteps 5843.
Path 282 | total_timesteps 5858.
Path 283 | total_timesteps 5876.
Path 284 | total_timesteps 5889.
Path 285 | total_timesteps 5909.
Path 286 | total_timesteps 5933.
Path 287 | total_timesteps 5950.
Path 288 | total_timesteps 5975.
Done generating random rollouts.
Creating normalization for training data.
Done creating normalization for training data.
Train dynamics model with intrinsic reward only? False
Pre-training enabled. Using only intrinsic reward.
Pre-training dynamics model for 0 iterations...
Done pre-training dynamics model.
Using external reward only.
itr #0 | 
Fitting dynamics.
Validation loss = 0.41490891575813293
Validation loss = 0.13056816160678864
Validation loss = 0.09496882557868958
Validation loss = 0.08060669898986816
Validation loss = 0.07375720888376236
Validation loss = 0.07191038131713867
Validation loss = 0.06396597623825073
Validation loss = 0.06553985178470612
Validation loss = 0.05665464699268341
Validation loss = 0.059851549565792084
Validation loss = 0.05233413353562355
Validation loss = 0.053266048431396484
Validation loss = 0.054167501628398895
Validation loss = 0.054748550057411194
Validation loss = 0.04718507081270218
Validation loss = 0.05301092565059662
Validation loss = 0.04711303859949112
Validation loss = 0.04574478790163994
Validation loss = 0.046250492334365845
Validation loss = 0.046888791024684906
Validation loss = 0.04949513077735901
Validation loss = 0.04469384253025055
Validation loss = 0.04510895907878876
Validation loss = 0.04465799033641815
Validation loss = 0.05352656543254852
Validation loss = 0.04289165884256363
Validation loss = 0.04488825052976608
Validation loss = 0.04771295189857483
Validation loss = 0.04555481672286987
Validation loss = 0.043929267674684525
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 33.
Path 3 | total_timesteps 57.
Path 4 | total_timesteps 73.
Path 5 | total_timesteps 93.
Path 6 | total_timesteps 111.
Path 7 | total_timesteps 135.
Path 8 | total_timesteps 145.
Path 9 | total_timesteps 198.
Path 10 | total_timesteps 225.
Path 11 | total_timesteps 240.
Path 12 | total_timesteps 256.
Path 13 | total_timesteps 278.
Path 14 | total_timesteps 311.
Path 15 | total_timesteps 324.
Path 16 | total_timesteps 387.
Path 17 | total_timesteps 407.
Path 18 | total_timesteps 420.
Path 19 | total_timesteps 440.
Path 20 | total_timesteps 469.
Path 21 | total_timesteps 509.
Path 22 | total_timesteps 535.
Path 23 | total_timesteps 591.
Path 24 | total_timesteps 628.
Path 25 | total_timesteps 659.
Path 26 | total_timesteps 676.
Path 27 | total_timesteps 703.
Path 28 | total_timesteps 732.
Path 29 | total_timesteps 761.
Path 30 | total_timesteps 780.
Path 31 | total_timesteps 799.
Path 32 | total_timesteps 809.
Path 33 | total_timesteps 828.
Path 34 | total_timesteps 838.
Path 35 | total_timesteps 885.
Path 36 | total_timesteps 917.
Path 37 | total_timesteps 936.
Path 38 | total_timesteps 947.
Path 39 | total_timesteps 980.
Path 40 | total_timesteps 997.
Path 41 | total_timesteps 1026.
Path 42 | total_timesteps 1045.
Path 43 | total_timesteps 1056.
Path 44 | total_timesteps 1072.
Path 45 | total_timesteps 1092.
Path 46 | total_timesteps 1122.
Path 47 | total_timesteps 1132.
Path 48 | total_timesteps 1143.
Path 49 | total_timesteps 1162.
Path 50 | total_timesteps 1186.
Path 51 | total_timesteps 1223.
Path 52 | total_timesteps 1236.
Path 53 | total_timesteps 1258.
Path 54 | total_timesteps 1279.
Path 55 | total_timesteps 1291.
Path 56 | total_timesteps 1308.
Path 57 | total_timesteps 1334.
Path 58 | total_timesteps 1367.
Path 59 | total_timesteps 1404.
Path 60 | total_timesteps 1420.
Path 61 | total_timesteps 1438.
Path 62 | total_timesteps 1451.
Path 63 | total_timesteps 1469.
Path 64 | total_timesteps 1508.
Path 65 | total_timesteps 1535.
Path 66 | total_timesteps 1545.
Path 67 | total_timesteps 1570.
Path 68 | total_timesteps 1600.
Path 69 | total_timesteps 1618.
Path 70 | total_timesteps 1653.
Path 71 | total_timesteps 1687.
Path 72 | total_timesteps 1724.
Path 73 | total_timesteps 1749.
Path 74 | total_timesteps 1792.
Path 75 | total_timesteps 1805.
Path 76 | total_timesteps 1837.
Path 77 | total_timesteps 1853.
Path 78 | total_timesteps 1877.
Path 79 | total_timesteps 1907.
Path 80 | total_timesteps 1932.
Path 81 | total_timesteps 1959.
Path 82 | total_timesteps 1980.
Path 83 | total_timesteps 1992.
Path 84 | total_timesteps 2018.
Path 85 | total_timesteps 2051.
Path 86 | total_timesteps 2065.
Path 87 | total_timesteps 2081.
Path 88 | total_timesteps 2102.
Path 89 | total_timesteps 2150.
Path 90 | total_timesteps 2183.
Path 91 | total_timesteps 2196.
Path 92 | total_timesteps 2245.
Path 93 | total_timesteps 2259.
Path 94 | total_timesteps 2277.
Path 95 | total_timesteps 2293.
Path 96 | total_timesteps 2309.
Path 97 | total_timesteps 2343.
Path 98 | total_timesteps 2353.
Path 99 | total_timesteps 2368.
Path 100 | total_timesteps 2380.
Path 101 | total_timesteps 2404.
Path 102 | total_timesteps 2423.
Path 103 | total_timesteps 2441.
Path 104 | total_timesteps 2453.
Path 105 | total_timesteps 2489.
Path 106 | total_timesteps 2509.
Path 107 | total_timesteps 2539.
Path 108 | total_timesteps 2553.
Path 109 | total_timesteps 2568.
Path 110 | total_timesteps 2590.
Path 111 | total_timesteps 2616.
Path 112 | total_timesteps 2653.
Path 113 | total_timesteps 2671.
Path 114 | total_timesteps 2697.
Path 115 | total_timesteps 2716.
Path 116 | total_timesteps 2735.
Path 117 | total_timesteps 2750.
Path 118 | total_timesteps 2789.
Path 119 | total_timesteps 2822.
Path 120 | total_timesteps 2858.
Path 121 | total_timesteps 2866.
Path 122 | total_timesteps 2878.
Path 123 | total_timesteps 2906.
Path 124 | total_timesteps 2927.
Path 125 | total_timesteps 2951.
Path 126 | total_timesteps 2974.
Path 127 | total_timesteps 2988.
Path 128 | total_timesteps 3017.
Path 129 | total_timesteps 3040.
Path 130 | total_timesteps 3056.
Path 131 | total_timesteps 3075.
Path 132 | total_timesteps 3089.
Path 133 | total_timesteps 3102.
Path 134 | total_timesteps 3115.
Path 135 | total_timesteps 3131.
Path 136 | total_timesteps 3146.
Path 137 | total_timesteps 3166.
Path 138 | total_timesteps 3182.
Path 139 | total_timesteps 3201.
Path 140 | total_timesteps 3227.
Path 141 | total_timesteps 3251.
Path 142 | total_timesteps 3270.
Path 143 | total_timesteps 3295.
Path 144 | total_timesteps 3306.
Path 145 | total_timesteps 3330.
Path 146 | total_timesteps 3358.
Path 147 | total_timesteps 3380.
Path 148 | total_timesteps 3424.
Path 149 | total_timesteps 3445.
Path 150 | total_timesteps 3471.
Path 151 | total_timesteps 3490.
Path 152 | total_timesteps 3523.
Path 153 | total_timesteps 3538.
Path 154 | total_timesteps 3572.
Path 155 | total_timesteps 3583.
Path 156 | total_timesteps 3614.
Path 157 | total_timesteps 3649.
Path 158 | total_timesteps 3658.
Path 159 | total_timesteps 3689.
Path 160 | total_timesteps 3713.
Path 161 | total_timesteps 3754.
Path 162 | total_timesteps 3779.
Path 163 | total_timesteps 3802.
Path 164 | total_timesteps 3870.
Path 165 | total_timesteps 3891.
Path 166 | total_timesteps 3932.
Path 167 | total_timesteps 3954.
Path 168 | total_timesteps 3966.
Path 169 | total_timesteps 3994.
Path 170 | total_timesteps 4020.
Path 171 | total_timesteps 4050.
Path 172 | total_timesteps 4079.
Path 173 | total_timesteps 4107.
Path 174 | total_timesteps 4128.
Path 175 | total_timesteps 4154.
Path 176 | total_timesteps 4181.
Path 177 | total_timesteps 4194.
Path 178 | total_timesteps 4209.
Path 179 | total_timesteps 4236.
Path 180 | total_timesteps 4248.
Path 181 | total_timesteps 4270.
Path 182 | total_timesteps 4296.
Path 183 | total_timesteps 4310.
Path 184 | total_timesteps 4335.
Path 185 | total_timesteps 4352.
Path 186 | total_timesteps 4368.
Path 187 | total_timesteps 4390.
Path 188 | total_timesteps 4404.
Path 189 | total_timesteps 4413.
Path 190 | total_timesteps 4451.
Path 191 | total_timesteps 4475.
Path 192 | total_timesteps 4488.
Path 193 | total_timesteps 4502.
Path 194 | total_timesteps 4516.
Path 195 | total_timesteps 4532.
Path 196 | total_timesteps 4545.
Path 197 | total_timesteps 4571.
Path 198 | total_timesteps 4582.
Path 199 | total_timesteps 4599.
Path 200 | total_timesteps 4612.
Path 201 | total_timesteps 4629.
Path 202 | total_timesteps 4650.
Path 203 | total_timesteps 4686.
Path 204 | total_timesteps 4733.
Path 205 | total_timesteps 4756.
Path 206 | total_timesteps 4793.
Path 207 | total_timesteps 4810.
Path 208 | total_timesteps 4826.
Path 209 | total_timesteps 4848.
Path 210 | total_timesteps 4869.
Path 211 | total_timesteps 4881.
Path 212 | total_timesteps 4914.
Path 213 | total_timesteps 4933.
Path 214 | total_timesteps 4944.
Path 215 | total_timesteps 4980.
Path 216 | total_timesteps 4998.
Path 217 | total_timesteps 5009.
Path 218 | total_timesteps 5028.
Path 219 | total_timesteps 5053.
Path 220 | total_timesteps 5097.
Path 221 | total_timesteps 5110.
Path 222 | total_timesteps 5155.
Path 223 | total_timesteps 5215.
Path 224 | total_timesteps 5234.
Path 225 | total_timesteps 5281.
Path 226 | total_timesteps 5304.
Path 227 | total_timesteps 5332.
Path 228 | total_timesteps 5355.
Path 229 | total_timesteps 5373.
Path 230 | total_timesteps 5412.
Path 231 | total_timesteps 5440.
Path 232 | total_timesteps 5450.
Path 233 | total_timesteps 5464.
Path 234 | total_timesteps 5476.
Path 235 | total_timesteps 5487.
Path 236 | total_timesteps 5520.
Path 237 | total_timesteps 5539.
Path 238 | total_timesteps 5557.
Path 239 | total_timesteps 5575.
Path 240 | total_timesteps 5609.
Path 241 | total_timesteps 5635.
Path 242 | total_timesteps 5665.
Path 243 | total_timesteps 5690.
Path 244 | total_timesteps 5721.
Path 245 | total_timesteps 5743.
Path 246 | total_timesteps 5778.
Path 247 | total_timesteps 5786.
Path 248 | total_timesteps 5802.
Path 249 | total_timesteps 5822.
Path 250 | total_timesteps 5832.
Path 251 | total_timesteps 5856.
Path 252 | total_timesteps 5918.
Path 253 | total_timesteps 5931.
Path 254 | total_timesteps 5938.
Path 255 | total_timesteps 5953.
Path 256 | total_timesteps 5969.
Path 257 | total_timesteps 5982.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.55    |
| Iteration     | 0        |
| MaximumReturn | 33       |
| MinimumReturn | -20.9    |
| TotalSamples  | 8018     |
----------------------------
itr #1 | 
Fitting dynamics.
Validation loss = 0.0941946730017662
Validation loss = 0.0654686838388443
Validation loss = 0.05643230676651001
Validation loss = 0.046177007257938385
Validation loss = 0.0416671521961689
Validation loss = 0.04649874195456505
Validation loss = 0.0394238643348217
Validation loss = 0.04139389097690582
Validation loss = 0.03971679136157036
Validation loss = 0.0391727052628994
Validation loss = 0.03791828081011772
Validation loss = 0.036358416080474854
Validation loss = 0.03917752578854561
Validation loss = 0.03515859693288803
Validation loss = 0.0351652167737484
Validation loss = 0.05381327122449875
Validation loss = 0.03344006836414337
Validation loss = 0.03569326922297478
Validation loss = 0.03495005890727043
Validation loss = 0.033049143850803375
Validation loss = 0.03518104553222656
Validation loss = 0.03817271068692207
Validation loss = 0.03561723232269287
Validation loss = 0.0370456725358963
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 49.
Path 3 | total_timesteps 107.
Path 4 | total_timesteps 121.
Path 5 | total_timesteps 175.
Path 6 | total_timesteps 239.
Path 7 | total_timesteps 267.
Path 8 | total_timesteps 323.
Path 9 | total_timesteps 350.
Path 10 | total_timesteps 369.
Path 11 | total_timesteps 398.
Path 12 | total_timesteps 432.
Path 13 | total_timesteps 490.
Path 14 | total_timesteps 524.
Path 15 | total_timesteps 548.
Path 16 | total_timesteps 569.
Path 17 | total_timesteps 604.
Path 18 | total_timesteps 645.
Path 19 | total_timesteps 698.
Path 20 | total_timesteps 775.
Path 21 | total_timesteps 805.
Path 22 | total_timesteps 816.
Path 23 | total_timesteps 853.
Path 24 | total_timesteps 886.
Path 25 | total_timesteps 919.
Path 26 | total_timesteps 948.
Path 27 | total_timesteps 1002.
Path 28 | total_timesteps 1069.
Path 29 | total_timesteps 1099.
Path 30 | total_timesteps 1131.
Path 31 | total_timesteps 1147.
Path 32 | total_timesteps 1157.
Path 33 | total_timesteps 1178.
Path 34 | total_timesteps 1205.
Path 35 | total_timesteps 1241.
Path 36 | total_timesteps 1259.
Path 37 | total_timesteps 1277.
Path 38 | total_timesteps 1311.
Path 39 | total_timesteps 1341.
Path 40 | total_timesteps 1368.
Path 41 | total_timesteps 1384.
Path 42 | total_timesteps 1415.
Path 43 | total_timesteps 1492.
Path 44 | total_timesteps 1512.
Path 45 | total_timesteps 1559.
Path 46 | total_timesteps 1606.
Path 47 | total_timesteps 1640.
Path 48 | total_timesteps 1692.
Path 49 | total_timesteps 1731.
Path 50 | total_timesteps 1748.
Path 51 | total_timesteps 1758.
Path 52 | total_timesteps 1799.
Path 53 | total_timesteps 1817.
Path 54 | total_timesteps 1841.
Path 55 | total_timesteps 1870.
Path 56 | total_timesteps 1905.
Path 57 | total_timesteps 1954.
Path 58 | total_timesteps 2006.
Path 59 | total_timesteps 2028.
Path 60 | total_timesteps 2055.
Path 61 | total_timesteps 2111.
Path 62 | total_timesteps 2123.
Path 63 | total_timesteps 2145.
Path 64 | total_timesteps 2232.
Path 65 | total_timesteps 2254.
Path 66 | total_timesteps 2275.
Path 67 | total_timesteps 2316.
Path 68 | total_timesteps 2355.
Path 69 | total_timesteps 2435.
Path 70 | total_timesteps 2448.
Path 71 | total_timesteps 2491.
Path 72 | total_timesteps 2522.
Path 73 | total_timesteps 2545.
Path 74 | total_timesteps 2604.
Path 75 | total_timesteps 2638.
Path 76 | total_timesteps 2668.
Path 77 | total_timesteps 2687.
Path 78 | total_timesteps 2708.
Path 79 | total_timesteps 2748.
Path 80 | total_timesteps 2782.
Path 81 | total_timesteps 2790.
Path 82 | total_timesteps 2824.
Path 83 | total_timesteps 2879.
Path 84 | total_timesteps 2904.
Path 85 | total_timesteps 2936.
Path 86 | total_timesteps 2971.
Path 87 | total_timesteps 2997.
Path 88 | total_timesteps 3007.
Path 89 | total_timesteps 3042.
Path 90 | total_timesteps 3072.
Path 91 | total_timesteps 3095.
Path 92 | total_timesteps 3114.
Path 93 | total_timesteps 3150.
Path 94 | total_timesteps 3203.
Path 95 | total_timesteps 3224.
Path 96 | total_timesteps 3250.
Path 97 | total_timesteps 3310.
Path 98 | total_timesteps 3330.
Path 99 | total_timesteps 3378.
Path 100 | total_timesteps 3398.
Path 101 | total_timesteps 3426.
Path 102 | total_timesteps 3457.
Path 103 | total_timesteps 3505.
Path 104 | total_timesteps 3564.
Path 105 | total_timesteps 3621.
Path 106 | total_timesteps 3656.
Path 107 | total_timesteps 3680.
Path 108 | total_timesteps 3724.
Path 109 | total_timesteps 3746.
Path 110 | total_timesteps 3784.
Path 111 | total_timesteps 3795.
Path 112 | total_timesteps 3808.
Path 113 | total_timesteps 3840.
Path 114 | total_timesteps 3889.
Path 115 | total_timesteps 3905.
Path 116 | total_timesteps 3934.
Path 117 | total_timesteps 3961.
Path 118 | total_timesteps 4018.
Path 119 | total_timesteps 4030.
Path 120 | total_timesteps 4062.
Path 121 | total_timesteps 4097.
Path 122 | total_timesteps 4150.
Path 123 | total_timesteps 4176.
Path 124 | total_timesteps 4214.
Path 125 | total_timesteps 4243.
Path 126 | total_timesteps 4256.
Path 127 | total_timesteps 4317.
Path 128 | total_timesteps 4345.
Path 129 | total_timesteps 4379.
Path 130 | total_timesteps 4415.
Path 131 | total_timesteps 4451.
Path 132 | total_timesteps 4484.
Path 133 | total_timesteps 4519.
Path 134 | total_timesteps 4556.
Path 135 | total_timesteps 4569.
Path 136 | total_timesteps 4626.
Path 137 | total_timesteps 4646.
Path 138 | total_timesteps 4662.
Path 139 | total_timesteps 4700.
Path 140 | total_timesteps 4753.
Path 141 | total_timesteps 4772.
Path 142 | total_timesteps 4803.
Path 143 | total_timesteps 4819.
Path 144 | total_timesteps 4861.
Path 145 | total_timesteps 4895.
Path 146 | total_timesteps 4911.
Path 147 | total_timesteps 4936.
Path 148 | total_timesteps 4962.
Path 149 | total_timesteps 4989.
Path 150 | total_timesteps 5009.
Path 151 | total_timesteps 5040.
Path 152 | total_timesteps 5067.
Path 153 | total_timesteps 5098.
Path 154 | total_timesteps 5110.
Path 155 | total_timesteps 5133.
Path 156 | total_timesteps 5153.
Path 157 | total_timesteps 5182.
Path 158 | total_timesteps 5222.
Path 159 | total_timesteps 5242.
Path 160 | total_timesteps 5265.
Path 161 | total_timesteps 5322.
Path 162 | total_timesteps 5350.
Path 163 | total_timesteps 5430.
Path 164 | total_timesteps 5461.
Path 165 | total_timesteps 5489.
Path 166 | total_timesteps 5520.
Path 167 | total_timesteps 5550.
Path 168 | total_timesteps 5582.
Path 169 | total_timesteps 5640.
Path 170 | total_timesteps 5700.
Path 171 | total_timesteps 5732.
Path 172 | total_timesteps 5767.
Path 173 | total_timesteps 5792.
Path 174 | total_timesteps 5825.
Path 175 | total_timesteps 5841.
Path 176 | total_timesteps 5890.
Path 177 | total_timesteps 5933.
Path 178 | total_timesteps 5983.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.39    |
| Iteration     | 1        |
| MaximumReturn | 48       |
| MinimumReturn | -33.2    |
| TotalSamples  | 12024    |
----------------------------
itr #2 | 
Fitting dynamics.
Validation loss = 0.049996521323919296
Validation loss = 0.03142551705241203
Validation loss = 0.030929284170269966
Validation loss = 0.032394085079431534
Validation loss = 0.029503004625439644
Validation loss = 0.02828402817249298
Validation loss = 0.02913757972419262
Validation loss = 0.03296366333961487
Validation loss = 0.027595823630690575
Validation loss = 0.028724631294608116
Validation loss = 0.02678190916776657
Validation loss = 0.026680365204811096
Validation loss = 0.026254361495375633
Validation loss = 0.027562886476516724
Validation loss = 0.025968139991164207
Validation loss = 0.025785379111766815
Validation loss = 0.02490568719804287
Validation loss = 0.026648089289665222
Validation loss = 0.02602870762348175
Validation loss = 0.025010952726006508
Validation loss = 0.029898757115006447
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 42.
Path 2 | total_timesteps 56.
Path 3 | total_timesteps 69.
Path 4 | total_timesteps 94.
Path 5 | total_timesteps 128.
Path 6 | total_timesteps 168.
Path 7 | total_timesteps 198.
Path 8 | total_timesteps 226.
Path 9 | total_timesteps 249.
Path 10 | total_timesteps 269.
Path 11 | total_timesteps 281.
Path 12 | total_timesteps 313.
Path 13 | total_timesteps 327.
Path 14 | total_timesteps 354.
Path 15 | total_timesteps 379.
Path 16 | total_timesteps 408.
Path 17 | total_timesteps 445.
Path 18 | total_timesteps 485.
Path 19 | total_timesteps 543.
Path 20 | total_timesteps 563.
Path 21 | total_timesteps 582.
Path 22 | total_timesteps 611.
Path 23 | total_timesteps 640.
Path 24 | total_timesteps 695.
Path 25 | total_timesteps 711.
Path 26 | total_timesteps 726.
Path 27 | total_timesteps 739.
Path 28 | total_timesteps 772.
Path 29 | total_timesteps 789.
Path 30 | total_timesteps 806.
Path 31 | total_timesteps 838.
Path 32 | total_timesteps 848.
Path 33 | total_timesteps 865.
Path 34 | total_timesteps 887.
Path 35 | total_timesteps 909.
Path 36 | total_timesteps 926.
Path 37 | total_timesteps 963.
Path 38 | total_timesteps 992.
Path 39 | total_timesteps 1017.
Path 40 | total_timesteps 1036.
Path 41 | total_timesteps 1083.
Path 42 | total_timesteps 1103.
Path 43 | total_timesteps 1119.
Path 44 | total_timesteps 1128.
Path 45 | total_timesteps 1138.
Path 46 | total_timesteps 1148.
Path 47 | total_timesteps 1189.
Path 48 | total_timesteps 1209.
Path 49 | total_timesteps 1225.
Path 50 | total_timesteps 1234.
Path 51 | total_timesteps 1251.
Path 52 | total_timesteps 1292.
Path 53 | total_timesteps 1316.
Path 54 | total_timesteps 1342.
Path 55 | total_timesteps 1355.
Path 56 | total_timesteps 1391.
Path 57 | total_timesteps 1414.
Path 58 | total_timesteps 1431.
Path 59 | total_timesteps 1449.
Path 60 | total_timesteps 1464.
Path 61 | total_timesteps 1482.
Path 62 | total_timesteps 1513.
Path 63 | total_timesteps 1529.
Path 64 | total_timesteps 1547.
Path 65 | total_timesteps 1560.
Path 66 | total_timesteps 1573.
Path 67 | total_timesteps 1613.
Path 68 | total_timesteps 1639.
Path 69 | total_timesteps 1649.
Path 70 | total_timesteps 1679.
Path 71 | total_timesteps 1690.
Path 72 | total_timesteps 1704.
Path 73 | total_timesteps 1720.
Path 74 | total_timesteps 1740.
Path 75 | total_timesteps 1759.
Path 76 | total_timesteps 1787.
Path 77 | total_timesteps 1842.
Path 78 | total_timesteps 1855.
Path 79 | total_timesteps 1881.
Path 80 | total_timesteps 1909.
Path 81 | total_timesteps 1958.
Path 82 | total_timesteps 1976.
Path 83 | total_timesteps 2005.
Path 84 | total_timesteps 2020.
Path 85 | total_timesteps 2047.
Path 86 | total_timesteps 2089.
Path 87 | total_timesteps 2119.
Path 88 | total_timesteps 2153.
Path 89 | total_timesteps 2162.
Path 90 | total_timesteps 2197.
Path 91 | total_timesteps 2254.
Path 92 | total_timesteps 2272.
Path 93 | total_timesteps 2299.
Path 94 | total_timesteps 2324.
Path 95 | total_timesteps 2364.
Path 96 | total_timesteps 2391.
Path 97 | total_timesteps 2436.
Path 98 | total_timesteps 2467.
Path 99 | total_timesteps 2493.
Path 100 | total_timesteps 2512.
Path 101 | total_timesteps 2524.
Path 102 | total_timesteps 2546.
Path 103 | total_timesteps 2573.
Path 104 | total_timesteps 2596.
Path 105 | total_timesteps 2643.
Path 106 | total_timesteps 2654.
Path 107 | total_timesteps 2693.
Path 108 | total_timesteps 2720.
Path 109 | total_timesteps 2731.
Path 110 | total_timesteps 2750.
Path 111 | total_timesteps 2763.
Path 112 | total_timesteps 2777.
Path 113 | total_timesteps 2816.
Path 114 | total_timesteps 2825.
Path 115 | total_timesteps 2866.
Path 116 | total_timesteps 2879.
Path 117 | total_timesteps 2889.
Path 118 | total_timesteps 2905.
Path 119 | total_timesteps 2940.
Path 120 | total_timesteps 2958.
Path 121 | total_timesteps 2977.
Path 122 | total_timesteps 2989.
Path 123 | total_timesteps 3033.
Path 124 | total_timesteps 3058.
Path 125 | total_timesteps 3093.
Path 126 | total_timesteps 3113.
Path 127 | total_timesteps 3140.
Path 128 | total_timesteps 3182.
Path 129 | total_timesteps 3207.
Path 130 | total_timesteps 3221.
Path 131 | total_timesteps 3235.
Path 132 | total_timesteps 3261.
Path 133 | total_timesteps 3331.
Path 134 | total_timesteps 3348.
Path 135 | total_timesteps 3365.
Path 136 | total_timesteps 3389.
Path 137 | total_timesteps 3426.
Path 138 | total_timesteps 3462.
Path 139 | total_timesteps 3491.
Path 140 | total_timesteps 3506.
Path 141 | total_timesteps 3520.
Path 142 | total_timesteps 3539.
Path 143 | total_timesteps 3557.
Path 144 | total_timesteps 3588.
Path 145 | total_timesteps 3631.
Path 146 | total_timesteps 3683.
Path 147 | total_timesteps 3700.
Path 148 | total_timesteps 3738.
Path 149 | total_timesteps 3764.
Path 150 | total_timesteps 3781.
Path 151 | total_timesteps 3810.
Path 152 | total_timesteps 3831.
Path 153 | total_timesteps 3845.
Path 154 | total_timesteps 3862.
Path 155 | total_timesteps 3895.
Path 156 | total_timesteps 3909.
Path 157 | total_timesteps 3925.
Path 158 | total_timesteps 3943.
Path 159 | total_timesteps 3968.
Path 160 | total_timesteps 4001.
Path 161 | total_timesteps 4028.
Path 162 | total_timesteps 4075.
Path 163 | total_timesteps 4103.
Path 164 | total_timesteps 4157.
Path 165 | total_timesteps 4183.
Path 166 | total_timesteps 4201.
Path 167 | total_timesteps 4215.
Path 168 | total_timesteps 4239.
Path 169 | total_timesteps 4268.
Path 170 | total_timesteps 4299.
Path 171 | total_timesteps 4320.
Path 172 | total_timesteps 4346.
Path 173 | total_timesteps 4395.
Path 174 | total_timesteps 4419.
Path 175 | total_timesteps 4448.
Path 176 | total_timesteps 4463.
Path 177 | total_timesteps 4474.
Path 178 | total_timesteps 4504.
Path 179 | total_timesteps 4524.
Path 180 | total_timesteps 4537.
Path 181 | total_timesteps 4552.
Path 182 | total_timesteps 4577.
Path 183 | total_timesteps 4595.
Path 184 | total_timesteps 4615.
Path 185 | total_timesteps 4641.
Path 186 | total_timesteps 4656.
Path 187 | total_timesteps 4679.
Path 188 | total_timesteps 4695.
Path 189 | total_timesteps 4725.
Path 190 | total_timesteps 4744.
Path 191 | total_timesteps 4771.
Path 192 | total_timesteps 4791.
Path 193 | total_timesteps 4806.
Path 194 | total_timesteps 4825.
Path 195 | total_timesteps 4843.
Path 196 | total_timesteps 4859.
Path 197 | total_timesteps 4876.
Path 198 | total_timesteps 4892.
Path 199 | total_timesteps 4909.
Path 200 | total_timesteps 4938.
Path 201 | total_timesteps 4958.
Path 202 | total_timesteps 4968.
Path 203 | total_timesteps 5000.
Path 204 | total_timesteps 5024.
Path 205 | total_timesteps 5041.
Path 206 | total_timesteps 5054.
Path 207 | total_timesteps 5121.
Path 208 | total_timesteps 5189.
Path 209 | total_timesteps 5212.
Path 210 | total_timesteps 5247.
Path 211 | total_timesteps 5272.
Path 212 | total_timesteps 5316.
Path 213 | total_timesteps 5337.
Path 214 | total_timesteps 5390.
Path 215 | total_timesteps 5402.
Path 216 | total_timesteps 5415.
Path 217 | total_timesteps 5440.
Path 218 | total_timesteps 5452.
Path 219 | total_timesteps 5468.
Path 220 | total_timesteps 5491.
Path 221 | total_timesteps 5507.
Path 222 | total_timesteps 5548.
Path 223 | total_timesteps 5558.
Path 224 | total_timesteps 5568.
Path 225 | total_timesteps 5617.
Path 226 | total_timesteps 5630.
Path 227 | total_timesteps 5643.
Path 228 | total_timesteps 5678.
Path 229 | total_timesteps 5693.
Path 230 | total_timesteps 5724.
Path 231 | total_timesteps 5738.
Path 232 | total_timesteps 5759.
Path 233 | total_timesteps 5786.
Path 234 | total_timesteps 5815.
Path 235 | total_timesteps 5832.
Path 236 | total_timesteps 5850.
Path 237 | total_timesteps 5882.
Path 238 | total_timesteps 5926.
Path 239 | total_timesteps 5956.
Path 240 | total_timesteps 5972.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.24    |
| Iteration     | 2        |
| MaximumReturn | 46.3     |
| MinimumReturn | -23.8    |
| TotalSamples  | 16030    |
----------------------------
itr #3 | 
Fitting dynamics.
Validation loss = 0.031510647386312485
Validation loss = 0.0226018987596035
Validation loss = 0.022817540913820267
Validation loss = 0.022121716290712357
Validation loss = 0.02246442437171936
Validation loss = 0.022555425763130188
Validation loss = 0.022071663290262222
Validation loss = 0.02285817638039589
Validation loss = 0.023955147713422775
Validation loss = 0.021012254059314728
Validation loss = 0.020062562078237534
Validation loss = 0.020090175792574883
Validation loss = 0.022283654659986496
Validation loss = 0.020067483186721802
Validation loss = 0.02629587985575199
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 42.
Path 2 | total_timesteps 67.
Path 3 | total_timesteps 82.
Path 4 | total_timesteps 124.
Path 5 | total_timesteps 182.
Path 6 | total_timesteps 208.
Path 7 | total_timesteps 239.
Path 8 | total_timesteps 270.
Path 9 | total_timesteps 299.
Path 10 | total_timesteps 341.
Path 11 | total_timesteps 349.
Path 12 | total_timesteps 364.
Path 13 | total_timesteps 399.
Path 14 | total_timesteps 425.
Path 15 | total_timesteps 446.
Path 16 | total_timesteps 475.
Path 17 | total_timesteps 502.
Path 18 | total_timesteps 525.
Path 19 | total_timesteps 552.
Path 20 | total_timesteps 590.
Path 21 | total_timesteps 605.
Path 22 | total_timesteps 672.
Path 23 | total_timesteps 707.
Path 24 | total_timesteps 728.
Path 25 | total_timesteps 739.
Path 26 | total_timesteps 780.
Path 27 | total_timesteps 816.
Path 28 | total_timesteps 852.
Path 29 | total_timesteps 884.
Path 30 | total_timesteps 903.
Path 31 | total_timesteps 925.
Path 32 | total_timesteps 951.
Path 33 | total_timesteps 978.
Path 34 | total_timesteps 1021.
Path 35 | total_timesteps 1045.
Path 36 | total_timesteps 1081.
Path 37 | total_timesteps 1122.
Path 38 | total_timesteps 1163.
Path 39 | total_timesteps 1175.
Path 40 | total_timesteps 1211.
Path 41 | total_timesteps 1231.
Path 42 | total_timesteps 1265.
Path 43 | total_timesteps 1307.
Path 44 | total_timesteps 1323.
Path 45 | total_timesteps 1351.
Path 46 | total_timesteps 1385.
Path 47 | total_timesteps 1446.
Path 48 | total_timesteps 1472.
Path 49 | total_timesteps 1501.
Path 50 | total_timesteps 1521.
Path 51 | total_timesteps 1540.
Path 52 | total_timesteps 1558.
Path 53 | total_timesteps 1588.
Path 54 | total_timesteps 1637.
Path 55 | total_timesteps 1654.
Path 56 | total_timesteps 1697.
Path 57 | total_timesteps 1741.
Path 58 | total_timesteps 1766.
Path 59 | total_timesteps 1785.
Path 60 | total_timesteps 1826.
Path 61 | total_timesteps 1850.
Path 62 | total_timesteps 1922.
Path 63 | total_timesteps 1970.
Path 64 | total_timesteps 1998.
Path 65 | total_timesteps 2039.
Path 66 | total_timesteps 2062.
Path 67 | total_timesteps 2102.
Path 68 | total_timesteps 2156.
Path 69 | total_timesteps 2188.
Path 70 | total_timesteps 2206.
Path 71 | total_timesteps 2224.
Path 72 | total_timesteps 2242.
Path 73 | total_timesteps 2252.
Path 74 | total_timesteps 2277.
Path 75 | total_timesteps 2356.
Path 76 | total_timesteps 2393.
Path 77 | total_timesteps 2411.
Path 78 | total_timesteps 2450.
Path 79 | total_timesteps 2472.
Path 80 | total_timesteps 2491.
Path 81 | total_timesteps 2542.
Path 82 | total_timesteps 2589.
Path 83 | total_timesteps 2639.
Path 84 | total_timesteps 2667.
Path 85 | total_timesteps 2679.
Path 86 | total_timesteps 2698.
Path 87 | total_timesteps 2727.
Path 88 | total_timesteps 2744.
Path 89 | total_timesteps 2772.
Path 90 | total_timesteps 2791.
Path 91 | total_timesteps 2855.
Path 92 | total_timesteps 2900.
Path 93 | total_timesteps 2952.
Path 94 | total_timesteps 2996.
Path 95 | total_timesteps 3030.
Path 96 | total_timesteps 3049.
Path 97 | total_timesteps 3068.
Path 98 | total_timesteps 3094.
Path 99 | total_timesteps 3146.
Path 100 | total_timesteps 3169.
Path 101 | total_timesteps 3221.
Path 102 | total_timesteps 3238.
Path 103 | total_timesteps 3266.
Path 104 | total_timesteps 3296.
Path 105 | total_timesteps 3331.
Path 106 | total_timesteps 3361.
Path 107 | total_timesteps 3395.
Path 108 | total_timesteps 3423.
Path 109 | total_timesteps 3437.
Path 110 | total_timesteps 3488.
Path 111 | total_timesteps 3519.
Path 112 | total_timesteps 3543.
Path 113 | total_timesteps 3616.
Path 114 | total_timesteps 3654.
Path 115 | total_timesteps 3678.
Path 116 | total_timesteps 3707.
Path 117 | total_timesteps 3740.
Path 118 | total_timesteps 3770.
Path 119 | total_timesteps 3788.
Path 120 | total_timesteps 3817.
Path 121 | total_timesteps 3829.
Path 122 | total_timesteps 3872.
Path 123 | total_timesteps 3912.
Path 124 | total_timesteps 3936.
Path 125 | total_timesteps 3963.
Path 126 | total_timesteps 4047.
Path 127 | total_timesteps 4073.
Path 128 | total_timesteps 4128.
Path 129 | total_timesteps 4159.
Path 130 | total_timesteps 4188.
Path 131 | total_timesteps 4228.
Path 132 | total_timesteps 4254.
Path 133 | total_timesteps 4315.
Path 134 | total_timesteps 4347.
Path 135 | total_timesteps 4386.
Path 136 | total_timesteps 4418.
Path 137 | total_timesteps 4436.
Path 138 | total_timesteps 4456.
Path 139 | total_timesteps 4504.
Path 140 | total_timesteps 4538.
Path 141 | total_timesteps 4562.
Path 142 | total_timesteps 4625.
Path 143 | total_timesteps 4644.
Path 144 | total_timesteps 4682.
Path 145 | total_timesteps 4698.
Path 146 | total_timesteps 4721.
Path 147 | total_timesteps 4741.
Path 148 | total_timesteps 4768.
Path 149 | total_timesteps 4794.
Path 150 | total_timesteps 4831.
Path 151 | total_timesteps 4853.
Path 152 | total_timesteps 4893.
Path 153 | total_timesteps 4904.
Path 154 | total_timesteps 4954.
Path 155 | total_timesteps 4967.
Path 156 | total_timesteps 5007.
Path 157 | total_timesteps 5025.
Path 158 | total_timesteps 5074.
Path 159 | total_timesteps 5096.
Path 160 | total_timesteps 5145.
Path 161 | total_timesteps 5179.
Path 162 | total_timesteps 5204.
Path 163 | total_timesteps 5239.
Path 164 | total_timesteps 5276.
Path 165 | total_timesteps 5307.
Path 166 | total_timesteps 5371.
Path 167 | total_timesteps 5399.
Path 168 | total_timesteps 5421.
Path 169 | total_timesteps 5475.
Path 170 | total_timesteps 5509.
Path 171 | total_timesteps 5540.
Path 172 | total_timesteps 5557.
Path 173 | total_timesteps 5588.
Path 174 | total_timesteps 5622.
Path 175 | total_timesteps 5665.
Path 176 | total_timesteps 5691.
Path 177 | total_timesteps 5735.
Path 178 | total_timesteps 5746.
Path 179 | total_timesteps 5761.
Path 180 | total_timesteps 5823.
Path 181 | total_timesteps 5846.
Path 182 | total_timesteps 5860.
Path 183 | total_timesteps 5885.
Path 184 | total_timesteps 5921.
Path 185 | total_timesteps 5952.
Path 186 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.75    |
| Iteration     | 3        |
| MaximumReturn | 30.4     |
| MinimumReturn | -22.4    |
| TotalSamples  | 20032    |
----------------------------
itr #4 | 
Fitting dynamics.
Validation loss = 0.023317696526646614
Validation loss = 0.022013762965798378
Validation loss = 0.018759312108159065
Validation loss = 0.019772527739405632
Validation loss = 0.0193764828145504
Validation loss = 0.019345957785844803
Validation loss = 0.0181512963026762
Validation loss = 0.01805097423493862
Validation loss = 0.017751943320035934
Validation loss = 0.01950642839074135
Validation loss = 0.019628409296274185
Validation loss = 0.016983430832624435
Validation loss = 0.018084127455949783
Validation loss = 0.017551502212882042
Validation loss = 0.026921432465314865
Validation loss = 0.0172322578728199
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 28.
Path 2 | total_timesteps 85.
Path 3 | total_timesteps 105.
Path 4 | total_timesteps 131.
Path 5 | total_timesteps 152.
Path 6 | total_timesteps 171.
Path 7 | total_timesteps 210.
Path 8 | total_timesteps 253.
Path 9 | total_timesteps 272.
Path 10 | total_timesteps 286.
Path 11 | total_timesteps 315.
Path 12 | total_timesteps 340.
Path 13 | total_timesteps 364.
Path 14 | total_timesteps 373.
Path 15 | total_timesteps 411.
Path 16 | total_timesteps 437.
Path 17 | total_timesteps 456.
Path 18 | total_timesteps 497.
Path 19 | total_timesteps 528.
Path 20 | total_timesteps 558.
Path 21 | total_timesteps 587.
Path 22 | total_timesteps 615.
Path 23 | total_timesteps 642.
Path 24 | total_timesteps 678.
Path 25 | total_timesteps 697.
Path 26 | total_timesteps 713.
Path 27 | total_timesteps 736.
Path 28 | total_timesteps 754.
Path 29 | total_timesteps 810.
Path 30 | total_timesteps 827.
Path 31 | total_timesteps 851.
Path 32 | total_timesteps 866.
Path 33 | total_timesteps 900.
Path 34 | total_timesteps 923.
Path 35 | total_timesteps 944.
Path 36 | total_timesteps 967.
Path 37 | total_timesteps 996.
Path 38 | total_timesteps 1012.
Path 39 | total_timesteps 1038.
Path 40 | total_timesteps 1059.
Path 41 | total_timesteps 1105.
Path 42 | total_timesteps 1149.
Path 43 | total_timesteps 1164.
Path 44 | total_timesteps 1203.
Path 45 | total_timesteps 1217.
Path 46 | total_timesteps 1243.
Path 47 | total_timesteps 1266.
Path 48 | total_timesteps 1301.
Path 49 | total_timesteps 1330.
Path 50 | total_timesteps 1351.
Path 51 | total_timesteps 1377.
Path 52 | total_timesteps 1393.
Path 53 | total_timesteps 1410.
Path 54 | total_timesteps 1430.
Path 55 | total_timesteps 1445.
Path 56 | total_timesteps 1484.
Path 57 | total_timesteps 1495.
Path 58 | total_timesteps 1507.
Path 59 | total_timesteps 1533.
Path 60 | total_timesteps 1562.
Path 61 | total_timesteps 1581.
Path 62 | total_timesteps 1610.
Path 63 | total_timesteps 1633.
Path 64 | total_timesteps 1660.
Path 65 | total_timesteps 1694.
Path 66 | total_timesteps 1725.
Path 67 | total_timesteps 1763.
Path 68 | total_timesteps 1781.
Path 69 | total_timesteps 1804.
Path 70 | total_timesteps 1831.
Path 71 | total_timesteps 1857.
Path 72 | total_timesteps 1883.
Path 73 | total_timesteps 1919.
Path 74 | total_timesteps 1997.
Path 75 | total_timesteps 2010.
Path 76 | total_timesteps 2033.
Path 77 | total_timesteps 2045.
Path 78 | total_timesteps 2093.
Path 79 | total_timesteps 2120.
Path 80 | total_timesteps 2153.
Path 81 | total_timesteps 2166.
Path 82 | total_timesteps 2192.
Path 83 | total_timesteps 2216.
Path 84 | total_timesteps 2254.
Path 85 | total_timesteps 2304.
Path 86 | total_timesteps 2334.
Path 87 | total_timesteps 2359.
Path 88 | total_timesteps 2377.
Path 89 | total_timesteps 2396.
Path 90 | total_timesteps 2414.
Path 91 | total_timesteps 2425.
Path 92 | total_timesteps 2440.
Path 93 | total_timesteps 2471.
Path 94 | total_timesteps 2499.
Path 95 | total_timesteps 2515.
Path 96 | total_timesteps 2536.
Path 97 | total_timesteps 2554.
Path 98 | total_timesteps 2570.
Path 99 | total_timesteps 2587.
Path 100 | total_timesteps 2619.
Path 101 | total_timesteps 2635.
Path 102 | total_timesteps 2663.
Path 103 | total_timesteps 2684.
Path 104 | total_timesteps 2697.
Path 105 | total_timesteps 2727.
Path 106 | total_timesteps 2746.
Path 107 | total_timesteps 2762.
Path 108 | total_timesteps 2801.
Path 109 | total_timesteps 2815.
Path 110 | total_timesteps 2860.
Path 111 | total_timesteps 2889.
Path 112 | total_timesteps 2914.
Path 113 | total_timesteps 2930.
Path 114 | total_timesteps 2952.
Path 115 | total_timesteps 2968.
Path 116 | total_timesteps 2996.
Path 117 | total_timesteps 3012.
Path 118 | total_timesteps 3032.
Path 119 | total_timesteps 3046.
Path 120 | total_timesteps 3089.
Path 121 | total_timesteps 3107.
Path 122 | total_timesteps 3150.
Path 123 | total_timesteps 3163.
Path 124 | total_timesteps 3187.
Path 125 | total_timesteps 3201.
Path 126 | total_timesteps 3272.
Path 127 | total_timesteps 3316.
Path 128 | total_timesteps 3338.
Path 129 | total_timesteps 3360.
Path 130 | total_timesteps 3402.
Path 131 | total_timesteps 3421.
Path 132 | total_timesteps 3441.
Path 133 | total_timesteps 3453.
Path 134 | total_timesteps 3490.
Path 135 | total_timesteps 3521.
Path 136 | total_timesteps 3548.
Path 137 | total_timesteps 3563.
Path 138 | total_timesteps 3598.
Path 139 | total_timesteps 3612.
Path 140 | total_timesteps 3641.
Path 141 | total_timesteps 3687.
Path 142 | total_timesteps 3714.
Path 143 | total_timesteps 3746.
Path 144 | total_timesteps 3760.
Path 145 | total_timesteps 3774.
Path 146 | total_timesteps 3799.
Path 147 | total_timesteps 3818.
Path 148 | total_timesteps 3841.
Path 149 | total_timesteps 3855.
Path 150 | total_timesteps 3875.
Path 151 | total_timesteps 3908.
Path 152 | total_timesteps 3931.
Path 153 | total_timesteps 3986.
Path 154 | total_timesteps 4001.
Path 155 | total_timesteps 4032.
Path 156 | total_timesteps 4061.
Path 157 | total_timesteps 4079.
Path 158 | total_timesteps 4090.
Path 159 | total_timesteps 4107.
Path 160 | total_timesteps 4139.
Path 161 | total_timesteps 4153.
Path 162 | total_timesteps 4175.
Path 163 | total_timesteps 4202.
Path 164 | total_timesteps 4252.
Path 165 | total_timesteps 4278.
Path 166 | total_timesteps 4293.
Path 167 | total_timesteps 4304.
Path 168 | total_timesteps 4334.
Path 169 | total_timesteps 4388.
Path 170 | total_timesteps 4406.
Path 171 | total_timesteps 4425.
Path 172 | total_timesteps 4466.
Path 173 | total_timesteps 4509.
Path 174 | total_timesteps 4555.
Path 175 | total_timesteps 4571.
Path 176 | total_timesteps 4593.
Path 177 | total_timesteps 4617.
Path 178 | total_timesteps 4638.
Path 179 | total_timesteps 4660.
Path 180 | total_timesteps 4676.
Path 181 | total_timesteps 4694.
Path 182 | total_timesteps 4721.
Path 183 | total_timesteps 4741.
Path 184 | total_timesteps 4774.
Path 185 | total_timesteps 4788.
Path 186 | total_timesteps 4807.
Path 187 | total_timesteps 4844.
Path 188 | total_timesteps 4854.
Path 189 | total_timesteps 4877.
Path 190 | total_timesteps 4901.
Path 191 | total_timesteps 4925.
Path 192 | total_timesteps 4940.
Path 193 | total_timesteps 4969.
Path 194 | total_timesteps 4984.
Path 195 | total_timesteps 5017.
Path 196 | total_timesteps 5046.
Path 197 | total_timesteps 5062.
Path 198 | total_timesteps 5091.
Path 199 | total_timesteps 5114.
Path 200 | total_timesteps 5128.
Path 201 | total_timesteps 5146.
Path 202 | total_timesteps 5168.
Path 203 | total_timesteps 5210.
Path 204 | total_timesteps 5234.
Path 205 | total_timesteps 5267.
Path 206 | total_timesteps 5277.
Path 207 | total_timesteps 5290.
Path 208 | total_timesteps 5321.
Path 209 | total_timesteps 5353.
Path 210 | total_timesteps 5375.
Path 211 | total_timesteps 5395.
Path 212 | total_timesteps 5428.
Path 213 | total_timesteps 5466.
Path 214 | total_timesteps 5493.
Path 215 | total_timesteps 5542.
Path 216 | total_timesteps 5558.
Path 217 | total_timesteps 5581.
Path 218 | total_timesteps 5592.
Path 219 | total_timesteps 5612.
Path 220 | total_timesteps 5648.
Path 221 | total_timesteps 5682.
Path 222 | total_timesteps 5704.
Path 223 | total_timesteps 5731.
Path 224 | total_timesteps 5777.
Path 225 | total_timesteps 5819.
Path 226 | total_timesteps 5838.
Path 227 | total_timesteps 5875.
Path 228 | total_timesteps 5885.
Path 229 | total_timesteps 5907.
Path 230 | total_timesteps 5931.
Path 231 | total_timesteps 5946.
Path 232 | total_timesteps 5985.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.94    |
| Iteration     | 4        |
| MaximumReturn | 56       |
| MinimumReturn | -20.9    |
| TotalSamples  | 24050    |
----------------------------
itr #5 | 
Fitting dynamics.
Validation loss = 0.019011594355106354
Validation loss = 0.016447437927126884
Validation loss = 0.018001999706029892
Validation loss = 0.01818232797086239
Validation loss = 0.016750922426581383
Validation loss = 0.01622563973069191
Validation loss = 0.0164815541356802
Validation loss = 0.01521936897188425
Validation loss = 0.015843087807297707
Validation loss = 0.01529445219784975
Validation loss = 0.0144178606569767
Validation loss = 0.014053336344659328
Validation loss = 0.01618468202650547
Validation loss = 0.015987055376172066
Validation loss = 0.01558340061455965
Validation loss = 0.014439009130001068
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 58.
Path 3 | total_timesteps 78.
Path 4 | total_timesteps 99.
Path 5 | total_timesteps 147.
Path 6 | total_timesteps 215.
Path 7 | total_timesteps 241.
Path 8 | total_timesteps 264.
Path 9 | total_timesteps 288.
Path 10 | total_timesteps 314.
Path 11 | total_timesteps 335.
Path 12 | total_timesteps 361.
Path 13 | total_timesteps 385.
Path 14 | total_timesteps 408.
Path 15 | total_timesteps 461.
Path 16 | total_timesteps 479.
Path 17 | total_timesteps 497.
Path 18 | total_timesteps 526.
Path 19 | total_timesteps 592.
Path 20 | total_timesteps 607.
Path 21 | total_timesteps 632.
Path 22 | total_timesteps 707.
Path 23 | total_timesteps 748.
Path 24 | total_timesteps 777.
Path 25 | total_timesteps 799.
Path 26 | total_timesteps 816.
Path 27 | total_timesteps 830.
Path 28 | total_timesteps 847.
Path 29 | total_timesteps 867.
Path 30 | total_timesteps 896.
Path 31 | total_timesteps 945.
Path 32 | total_timesteps 982.
Path 33 | total_timesteps 1007.
Path 34 | total_timesteps 1035.
Path 35 | total_timesteps 1055.
Path 36 | total_timesteps 1083.
Path 37 | total_timesteps 1099.
Path 38 | total_timesteps 1143.
Path 39 | total_timesteps 1153.
Path 40 | total_timesteps 1204.
Path 41 | total_timesteps 1230.
Path 42 | total_timesteps 1276.
Path 43 | total_timesteps 1299.
Path 44 | total_timesteps 1314.
Path 45 | total_timesteps 1330.
Path 46 | total_timesteps 1353.
Path 47 | total_timesteps 1379.
Path 48 | total_timesteps 1423.
Path 49 | total_timesteps 1437.
Path 50 | total_timesteps 1460.
Path 51 | total_timesteps 1495.
Path 52 | total_timesteps 1535.
Path 53 | total_timesteps 1583.
Path 54 | total_timesteps 1636.
Path 55 | total_timesteps 1675.
Path 56 | total_timesteps 1698.
Path 57 | total_timesteps 1712.
Path 58 | total_timesteps 1731.
Path 59 | total_timesteps 1785.
Path 60 | total_timesteps 1798.
Path 61 | total_timesteps 1814.
Path 62 | total_timesteps 1856.
Path 63 | total_timesteps 1874.
Path 64 | total_timesteps 1903.
Path 65 | total_timesteps 1928.
Path 66 | total_timesteps 1952.
Path 67 | total_timesteps 1981.
Path 68 | total_timesteps 2014.
Path 69 | total_timesteps 2037.
Path 70 | total_timesteps 2077.
Path 71 | total_timesteps 2098.
Path 72 | total_timesteps 2115.
Path 73 | total_timesteps 2156.
Path 74 | total_timesteps 2178.
Path 75 | total_timesteps 2191.
Path 76 | total_timesteps 2210.
Path 77 | total_timesteps 2236.
Path 78 | total_timesteps 2277.
Path 79 | total_timesteps 2303.
Path 80 | total_timesteps 2365.
Path 81 | total_timesteps 2384.
Path 82 | total_timesteps 2402.
Path 83 | total_timesteps 2420.
Path 84 | total_timesteps 2439.
Path 85 | total_timesteps 2465.
Path 86 | total_timesteps 2488.
Path 87 | total_timesteps 2521.
Path 88 | total_timesteps 2546.
Path 89 | total_timesteps 2588.
Path 90 | total_timesteps 2601.
Path 91 | total_timesteps 2635.
Path 92 | total_timesteps 2688.
Path 93 | total_timesteps 2714.
Path 94 | total_timesteps 2764.
Path 95 | total_timesteps 2814.
Path 96 | total_timesteps 2848.
Path 97 | total_timesteps 2863.
Path 98 | total_timesteps 2898.
Path 99 | total_timesteps 2952.
Path 100 | total_timesteps 2993.
Path 101 | total_timesteps 3034.
Path 102 | total_timesteps 3069.
Path 103 | total_timesteps 3083.
Path 104 | total_timesteps 3096.
Path 105 | total_timesteps 3122.
Path 106 | total_timesteps 3151.
Path 107 | total_timesteps 3212.
Path 108 | total_timesteps 3237.
Path 109 | total_timesteps 3254.
Path 110 | total_timesteps 3281.
Path 111 | total_timesteps 3302.
Path 112 | total_timesteps 3322.
Path 113 | total_timesteps 3355.
Path 114 | total_timesteps 3375.
Path 115 | total_timesteps 3400.
Path 116 | total_timesteps 3461.
Path 117 | total_timesteps 3474.
Path 118 | total_timesteps 3506.
Path 119 | total_timesteps 3526.
Path 120 | total_timesteps 3549.
Path 121 | total_timesteps 3564.
Path 122 | total_timesteps 3602.
Path 123 | total_timesteps 3631.
Path 124 | total_timesteps 3655.
Path 125 | total_timesteps 3674.
Path 126 | total_timesteps 3686.
Path 127 | total_timesteps 3725.
Path 128 | total_timesteps 3768.
Path 129 | total_timesteps 3803.
Path 130 | total_timesteps 3828.
Path 131 | total_timesteps 3850.
Path 132 | total_timesteps 3877.
Path 133 | total_timesteps 3896.
Path 134 | total_timesteps 3913.
Path 135 | total_timesteps 3950.
Path 136 | total_timesteps 3977.
Path 137 | total_timesteps 4008.
Path 138 | total_timesteps 4035.
Path 139 | total_timesteps 4067.
Path 140 | total_timesteps 4133.
Path 141 | total_timesteps 4148.
Path 142 | total_timesteps 4184.
Path 143 | total_timesteps 4214.
Path 144 | total_timesteps 4234.
Path 145 | total_timesteps 4259.
Path 146 | total_timesteps 4293.
Path 147 | total_timesteps 4317.
Path 148 | total_timesteps 4356.
Path 149 | total_timesteps 4392.
Path 150 | total_timesteps 4418.
Path 151 | total_timesteps 4430.
Path 152 | total_timesteps 4452.
Path 153 | total_timesteps 4493.
Path 154 | total_timesteps 4511.
Path 155 | total_timesteps 4540.
Path 156 | total_timesteps 4551.
Path 157 | total_timesteps 4596.
Path 158 | total_timesteps 4605.
Path 159 | total_timesteps 4645.
Path 160 | total_timesteps 4677.
Path 161 | total_timesteps 4699.
Path 162 | total_timesteps 4740.
Path 163 | total_timesteps 4797.
Path 164 | total_timesteps 4816.
Path 165 | total_timesteps 4840.
Path 166 | total_timesteps 4860.
Path 167 | total_timesteps 4903.
Path 168 | total_timesteps 4923.
Path 169 | total_timesteps 4937.
Path 170 | total_timesteps 4969.
Path 171 | total_timesteps 5012.
Path 172 | total_timesteps 5091.
Path 173 | total_timesteps 5104.
Path 174 | total_timesteps 5116.
Path 175 | total_timesteps 5141.
Path 176 | total_timesteps 5162.
Path 177 | total_timesteps 5179.
Path 178 | total_timesteps 5189.
Path 179 | total_timesteps 5217.
Path 180 | total_timesteps 5239.
Path 181 | total_timesteps 5249.
Path 182 | total_timesteps 5265.
Path 183 | total_timesteps 5274.
Path 184 | total_timesteps 5299.
Path 185 | total_timesteps 5362.
Path 186 | total_timesteps 5384.
Path 187 | total_timesteps 5404.
Path 188 | total_timesteps 5484.
Path 189 | total_timesteps 5516.
Path 190 | total_timesteps 5543.
Path 191 | total_timesteps 5554.
Path 192 | total_timesteps 5610.
Path 193 | total_timesteps 5652.
Path 194 | total_timesteps 5682.
Path 195 | total_timesteps 5714.
Path 196 | total_timesteps 5731.
Path 197 | total_timesteps 5754.
Path 198 | total_timesteps 5767.
Path 199 | total_timesteps 5798.
Path 200 | total_timesteps 5809.
Path 201 | total_timesteps 5838.
Path 202 | total_timesteps 5890.
Path 203 | total_timesteps 5913.
Path 204 | total_timesteps 5932.
Path 205 | total_timesteps 5966.
Path 206 | total_timesteps 5986.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.46    |
| Iteration     | 5        |
| MaximumReturn | 33.1     |
| MinimumReturn | -37.5    |
| TotalSamples  | 28054    |
----------------------------
itr #6 | 
Fitting dynamics.
Validation loss = 0.018692899495363235
Validation loss = 0.014256969094276428
Validation loss = 0.014197015203535557
Validation loss = 0.015068955719470978
Validation loss = 0.013252109289169312
Validation loss = 0.013738362118601799
Validation loss = 0.013090086169540882
Validation loss = 0.012412021867930889
Validation loss = 0.01323510892689228
Validation loss = 0.014655070379376411
Validation loss = 0.013077018782496452
Validation loss = 0.013357805088162422
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 20.
Path 2 | total_timesteps 59.
Path 3 | total_timesteps 89.
Path 4 | total_timesteps 129.
Path 5 | total_timesteps 157.
Path 6 | total_timesteps 195.
Path 7 | total_timesteps 249.
Path 8 | total_timesteps 265.
Path 9 | total_timesteps 289.
Path 10 | total_timesteps 309.
Path 11 | total_timesteps 328.
Path 12 | total_timesteps 350.
Path 13 | total_timesteps 381.
Path 14 | total_timesteps 407.
Path 15 | total_timesteps 445.
Path 16 | total_timesteps 502.
Path 17 | total_timesteps 523.
Path 18 | total_timesteps 566.
Path 19 | total_timesteps 618.
Path 20 | total_timesteps 640.
Path 21 | total_timesteps 673.
Path 22 | total_timesteps 703.
Path 23 | total_timesteps 723.
Path 24 | total_timesteps 742.
Path 25 | total_timesteps 759.
Path 26 | total_timesteps 791.
Path 27 | total_timesteps 819.
Path 28 | total_timesteps 841.
Path 29 | total_timesteps 877.
Path 30 | total_timesteps 887.
Path 31 | total_timesteps 951.
Path 32 | total_timesteps 981.
Path 33 | total_timesteps 1022.
Path 34 | total_timesteps 1035.
Path 35 | total_timesteps 1064.
Path 36 | total_timesteps 1083.
Path 37 | total_timesteps 1111.
Path 38 | total_timesteps 1130.
Path 39 | total_timesteps 1160.
Path 40 | total_timesteps 1199.
Path 41 | total_timesteps 1223.
Path 42 | total_timesteps 1237.
Path 43 | total_timesteps 1262.
Path 44 | total_timesteps 1281.
Path 45 | total_timesteps 1308.
Path 46 | total_timesteps 1333.
Path 47 | total_timesteps 1374.
Path 48 | total_timesteps 1424.
Path 49 | total_timesteps 1449.
Path 50 | total_timesteps 1477.
Path 51 | total_timesteps 1521.
Path 52 | total_timesteps 1554.
Path 53 | total_timesteps 1589.
Path 54 | total_timesteps 1618.
Path 55 | total_timesteps 1677.
Path 56 | total_timesteps 1721.
Path 57 | total_timesteps 1748.
Path 58 | total_timesteps 1778.
Path 59 | total_timesteps 1808.
Path 60 | total_timesteps 1847.
Path 61 | total_timesteps 1919.
Path 62 | total_timesteps 1946.
Path 63 | total_timesteps 1960.
Path 64 | total_timesteps 2003.
Path 65 | total_timesteps 2021.
Path 66 | total_timesteps 2039.
Path 67 | total_timesteps 2072.
Path 68 | total_timesteps 2087.
Path 69 | total_timesteps 2108.
Path 70 | total_timesteps 2140.
Path 71 | total_timesteps 2161.
Path 72 | total_timesteps 2191.
Path 73 | total_timesteps 2216.
Path 74 | total_timesteps 2238.
Path 75 | total_timesteps 2279.
Path 76 | total_timesteps 2300.
Path 77 | total_timesteps 2349.
Path 78 | total_timesteps 2369.
Path 79 | total_timesteps 2384.
Path 80 | total_timesteps 2408.
Path 81 | total_timesteps 2427.
Path 82 | total_timesteps 2456.
Path 83 | total_timesteps 2514.
Path 84 | total_timesteps 2581.
Path 85 | total_timesteps 2617.
Path 86 | total_timesteps 2638.
Path 87 | total_timesteps 2686.
Path 88 | total_timesteps 2722.
Path 89 | total_timesteps 2749.
Path 90 | total_timesteps 2767.
Path 91 | total_timesteps 2824.
Path 92 | total_timesteps 2849.
Path 93 | total_timesteps 2876.
Path 94 | total_timesteps 2893.
Path 95 | total_timesteps 2918.
Path 96 | total_timesteps 2954.
Path 97 | total_timesteps 2980.
Path 98 | total_timesteps 2990.
Path 99 | total_timesteps 3058.
Path 100 | total_timesteps 3082.
Path 101 | total_timesteps 3106.
Path 102 | total_timesteps 3136.
Path 103 | total_timesteps 3210.
Path 104 | total_timesteps 3228.
Path 105 | total_timesteps 3287.
Path 106 | total_timesteps 3322.
Path 107 | total_timesteps 3346.
Path 108 | total_timesteps 3384.
Path 109 | total_timesteps 3431.
Path 110 | total_timesteps 3475.
Path 111 | total_timesteps 3509.
Path 112 | total_timesteps 3521.
Path 113 | total_timesteps 3542.
Path 114 | total_timesteps 3576.
Path 115 | total_timesteps 3630.
Path 116 | total_timesteps 3691.
Path 117 | total_timesteps 3727.
Path 118 | total_timesteps 3763.
Path 119 | total_timesteps 3786.
Path 120 | total_timesteps 3819.
Path 121 | total_timesteps 3832.
Path 122 | total_timesteps 3857.
Path 123 | total_timesteps 3893.
Path 124 | total_timesteps 3920.
Path 125 | total_timesteps 3949.
Path 126 | total_timesteps 4002.
Path 127 | total_timesteps 4024.
Path 128 | total_timesteps 4061.
Path 129 | total_timesteps 4084.
Path 130 | total_timesteps 4133.
Path 131 | total_timesteps 4148.
Path 132 | total_timesteps 4180.
Path 133 | total_timesteps 4204.
Path 134 | total_timesteps 4253.
Path 135 | total_timesteps 4273.
Path 136 | total_timesteps 4298.
Path 137 | total_timesteps 4316.
Path 138 | total_timesteps 4380.
Path 139 | total_timesteps 4417.
Path 140 | total_timesteps 4442.
Path 141 | total_timesteps 4459.
Path 142 | total_timesteps 4493.
Path 143 | total_timesteps 4509.
Path 144 | total_timesteps 4531.
Path 145 | total_timesteps 4562.
Path 146 | total_timesteps 4606.
Path 147 | total_timesteps 4638.
Path 148 | total_timesteps 4673.
Path 149 | total_timesteps 4711.
Path 150 | total_timesteps 4740.
Path 151 | total_timesteps 4768.
Path 152 | total_timesteps 4791.
Path 153 | total_timesteps 4850.
Path 154 | total_timesteps 4881.
Path 155 | total_timesteps 4922.
Path 156 | total_timesteps 4955.
Path 157 | total_timesteps 4979.
Path 158 | total_timesteps 5044.
Path 159 | total_timesteps 5084.
Path 160 | total_timesteps 5140.
Path 161 | total_timesteps 5218.
Path 162 | total_timesteps 5244.
Path 163 | total_timesteps 5305.
Path 164 | total_timesteps 5329.
Path 165 | total_timesteps 5365.
Path 166 | total_timesteps 5380.
Path 167 | total_timesteps 5409.
Path 168 | total_timesteps 5455.
Path 169 | total_timesteps 5489.
Path 170 | total_timesteps 5512.
Path 171 | total_timesteps 5528.
Path 172 | total_timesteps 5561.
Path 173 | total_timesteps 5586.
Path 174 | total_timesteps 5616.
Path 175 | total_timesteps 5639.
Path 176 | total_timesteps 5691.
Path 177 | total_timesteps 5716.
Path 178 | total_timesteps 5732.
Path 179 | total_timesteps 5760.
Path 180 | total_timesteps 5781.
Path 181 | total_timesteps 5818.
Path 182 | total_timesteps 5855.
Path 183 | total_timesteps 5885.
Path 184 | total_timesteps 5902.
Path 185 | total_timesteps 5933.
Path 186 | total_timesteps 5967.
Path 187 | total_timesteps 5980.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -3.75    |
| Iteration     | 6        |
| MaximumReturn | 46.1     |
| MinimumReturn | -27.4    |
| TotalSamples  | 32064    |
----------------------------
itr #7 | 
Fitting dynamics.
Validation loss = 0.015083832666277885
Validation loss = 0.011626498773694038
Validation loss = 0.013288088142871857
Validation loss = 0.012024165131151676
Validation loss = 0.012327749282121658
Validation loss = 0.011739328503608704
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 40.
Path 3 | total_timesteps 62.
Path 4 | total_timesteps 72.
Path 5 | total_timesteps 84.
Path 6 | total_timesteps 108.
Path 7 | total_timesteps 135.
Path 8 | total_timesteps 158.
Path 9 | total_timesteps 232.
Path 10 | total_timesteps 283.
Path 11 | total_timesteps 318.
Path 12 | total_timesteps 342.
Path 13 | total_timesteps 379.
Path 14 | total_timesteps 399.
Path 15 | total_timesteps 417.
Path 16 | total_timesteps 429.
Path 17 | total_timesteps 461.
Path 18 | total_timesteps 482.
Path 19 | total_timesteps 500.
Path 20 | total_timesteps 516.
Path 21 | total_timesteps 552.
Path 22 | total_timesteps 586.
Path 23 | total_timesteps 606.
Path 24 | total_timesteps 638.
Path 25 | total_timesteps 669.
Path 26 | total_timesteps 696.
Path 27 | total_timesteps 728.
Path 28 | total_timesteps 755.
Path 29 | total_timesteps 810.
Path 30 | total_timesteps 875.
Path 31 | total_timesteps 912.
Path 32 | total_timesteps 934.
Path 33 | total_timesteps 965.
Path 34 | total_timesteps 1001.
Path 35 | total_timesteps 1058.
Path 36 | total_timesteps 1107.
Path 37 | total_timesteps 1175.
Path 38 | total_timesteps 1209.
Path 39 | total_timesteps 1251.
Path 40 | total_timesteps 1267.
Path 41 | total_timesteps 1303.
Path 42 | total_timesteps 1330.
Path 43 | total_timesteps 1365.
Path 44 | total_timesteps 1399.
Path 45 | total_timesteps 1414.
Path 46 | total_timesteps 1443.
Path 47 | total_timesteps 1491.
Path 48 | total_timesteps 1526.
Path 49 | total_timesteps 1538.
Path 50 | total_timesteps 1570.
Path 51 | total_timesteps 1607.
Path 52 | total_timesteps 1659.
Path 53 | total_timesteps 1703.
Path 54 | total_timesteps 1731.
Path 55 | total_timesteps 1773.
Path 56 | total_timesteps 1795.
Path 57 | total_timesteps 1870.
Path 58 | total_timesteps 1893.
Path 59 | total_timesteps 1963.
Path 60 | total_timesteps 1988.
Path 61 | total_timesteps 2030.
Path 62 | total_timesteps 2061.
Path 63 | total_timesteps 2105.
Path 64 | total_timesteps 2135.
Path 65 | total_timesteps 2165.
Path 66 | total_timesteps 2213.
Path 67 | total_timesteps 2230.
Path 68 | total_timesteps 2274.
Path 69 | total_timesteps 2288.
Path 70 | total_timesteps 2327.
Path 71 | total_timesteps 2358.
Path 72 | total_timesteps 2386.
Path 73 | total_timesteps 2424.
Path 74 | total_timesteps 2456.
Path 75 | total_timesteps 2477.
Path 76 | total_timesteps 2512.
Path 77 | total_timesteps 2558.
Path 78 | total_timesteps 2585.
Path 79 | total_timesteps 2614.
Path 80 | total_timesteps 2649.
Path 81 | total_timesteps 2663.
Path 82 | total_timesteps 2686.
Path 83 | total_timesteps 2712.
Path 84 | total_timesteps 2733.
Path 85 | total_timesteps 2777.
Path 86 | total_timesteps 2806.
Path 87 | total_timesteps 2837.
Path 88 | total_timesteps 2867.
Path 89 | total_timesteps 2890.
Path 90 | total_timesteps 2930.
Path 91 | total_timesteps 2948.
Path 92 | total_timesteps 2980.
Path 93 | total_timesteps 3007.
Path 94 | total_timesteps 3041.
Path 95 | total_timesteps 3057.
Path 96 | total_timesteps 3072.
Path 97 | total_timesteps 3129.
Path 98 | total_timesteps 3153.
Path 99 | total_timesteps 3192.
Path 100 | total_timesteps 3226.
Path 101 | total_timesteps 3252.
Path 102 | total_timesteps 3278.
Path 103 | total_timesteps 3301.
Path 104 | total_timesteps 3319.
Path 105 | total_timesteps 3371.
Path 106 | total_timesteps 3423.
Path 107 | total_timesteps 3445.
Path 108 | total_timesteps 3471.
Path 109 | total_timesteps 3493.
Path 110 | total_timesteps 3517.
Path 111 | total_timesteps 3543.
Path 112 | total_timesteps 3557.
Path 113 | total_timesteps 3580.
Path 114 | total_timesteps 3605.
Path 115 | total_timesteps 3632.
Path 116 | total_timesteps 3670.
Path 117 | total_timesteps 3707.
Path 118 | total_timesteps 3726.
Path 119 | total_timesteps 3741.
Path 120 | total_timesteps 3757.
Path 121 | total_timesteps 3772.
Path 122 | total_timesteps 3811.
Path 123 | total_timesteps 3835.
Path 124 | total_timesteps 3870.
Path 125 | total_timesteps 3890.
Path 126 | total_timesteps 3913.
Path 127 | total_timesteps 3928.
Path 128 | total_timesteps 3953.
Path 129 | total_timesteps 3987.
Path 130 | total_timesteps 4052.
Path 131 | total_timesteps 4077.
Path 132 | total_timesteps 4114.
Path 133 | total_timesteps 4135.
Path 134 | total_timesteps 4160.
Path 135 | total_timesteps 4192.
Path 136 | total_timesteps 4213.
Path 137 | total_timesteps 4227.
Path 138 | total_timesteps 4252.
Path 139 | total_timesteps 4267.
Path 140 | total_timesteps 4292.
Path 141 | total_timesteps 4325.
Path 142 | total_timesteps 4378.
Path 143 | total_timesteps 4420.
Path 144 | total_timesteps 4450.
Path 145 | total_timesteps 4458.
Path 146 | total_timesteps 4483.
Path 147 | total_timesteps 4497.
Path 148 | total_timesteps 4524.
Path 149 | total_timesteps 4556.
Path 150 | total_timesteps 4587.
Path 151 | total_timesteps 4599.
Path 152 | total_timesteps 4614.
Path 153 | total_timesteps 4637.
Path 154 | total_timesteps 4660.
Path 155 | total_timesteps 4675.
Path 156 | total_timesteps 4702.
Path 157 | total_timesteps 4752.
Path 158 | total_timesteps 4771.
Path 159 | total_timesteps 4790.
Path 160 | total_timesteps 4832.
Path 161 | total_timesteps 4893.
Path 162 | total_timesteps 4915.
Path 163 | total_timesteps 4939.
Path 164 | total_timesteps 4975.
Path 165 | total_timesteps 5030.
Path 166 | total_timesteps 5052.
Path 167 | total_timesteps 5091.
Path 168 | total_timesteps 5108.
Path 169 | total_timesteps 5146.
Path 170 | total_timesteps 5161.
Path 171 | total_timesteps 5192.
Path 172 | total_timesteps 5311.
Path 173 | total_timesteps 5346.
Path 174 | total_timesteps 5375.
Path 175 | total_timesteps 5404.
Path 176 | total_timesteps 5444.
Path 177 | total_timesteps 5465.
Path 178 | total_timesteps 5491.
Path 179 | total_timesteps 5518.
Path 180 | total_timesteps 5551.
Path 181 | total_timesteps 5615.
Path 182 | total_timesteps 5667.
Path 183 | total_timesteps 5729.
Path 184 | total_timesteps 5807.
Path 185 | total_timesteps 5825.
Path 186 | total_timesteps 5852.
Path 187 | total_timesteps 5880.
Path 188 | total_timesteps 5917.
Path 189 | total_timesteps 5935.
Path 190 | total_timesteps 5965.
Path 191 | total_timesteps 5978.
Path 192 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -4.64    |
| Iteration     | 7        |
| MaximumReturn | 34.3     |
| MinimumReturn | -22.3    |
| TotalSamples  | 36080    |
----------------------------
itr #8 | 
Fitting dynamics.
Validation loss = 0.014901255257427692
Validation loss = 0.013074991293251514
Validation loss = 0.011212862096726894
Validation loss = 0.011741787195205688
Validation loss = 0.011399287730455399
Validation loss = 0.011919893324375153
Validation loss = 0.010607540607452393
Validation loss = 0.010432505048811436
Validation loss = 0.011634143069386482
Validation loss = 0.011270146816968918
Validation loss = 0.01071704551577568
Validation loss = 0.009753118269145489
Validation loss = 0.010270943865180016
Validation loss = 0.011855857446789742
Validation loss = 0.011246023699641228
Validation loss = 0.009441737085580826
Validation loss = 0.009834345430135727
Validation loss = 0.009613472037017345
Validation loss = 0.010849356651306152
Validation loss = 0.009276374243199825
Validation loss = 0.009748239070177078
Validation loss = 0.008899352513253689
Validation loss = 0.009401206858456135
Validation loss = 0.009263506159186363
Validation loss = 0.00945261586457491
Validation loss = 0.008892232552170753
Validation loss = 0.010300428606569767
Validation loss = 0.009824957698583603
Validation loss = 0.00952167995274067
Validation loss = 0.010341765359044075
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 33.
Path 2 | total_timesteps 67.
Path 3 | total_timesteps 112.
Path 4 | total_timesteps 141.
Path 5 | total_timesteps 156.
Path 6 | total_timesteps 177.
Path 7 | total_timesteps 262.
Path 8 | total_timesteps 288.
Path 9 | total_timesteps 312.
Path 10 | total_timesteps 327.
Path 11 | total_timesteps 398.
Path 12 | total_timesteps 445.
Path 13 | total_timesteps 459.
Path 14 | total_timesteps 470.
Path 15 | total_timesteps 513.
Path 16 | total_timesteps 547.
Path 17 | total_timesteps 599.
Path 18 | total_timesteps 624.
Path 19 | total_timesteps 639.
Path 20 | total_timesteps 674.
Path 21 | total_timesteps 690.
Path 22 | total_timesteps 701.
Path 23 | total_timesteps 720.
Path 24 | total_timesteps 742.
Path 25 | total_timesteps 777.
Path 26 | total_timesteps 792.
Path 27 | total_timesteps 825.
Path 28 | total_timesteps 845.
Path 29 | total_timesteps 878.
Path 30 | total_timesteps 904.
Path 31 | total_timesteps 928.
Path 32 | total_timesteps 956.
Path 33 | total_timesteps 1010.
Path 34 | total_timesteps 1035.
Path 35 | total_timesteps 1064.
Path 36 | total_timesteps 1088.
Path 37 | total_timesteps 1100.
Path 38 | total_timesteps 1126.
Path 39 | total_timesteps 1178.
Path 40 | total_timesteps 1190.
Path 41 | total_timesteps 1205.
Path 42 | total_timesteps 1232.
Path 43 | total_timesteps 1253.
Path 44 | total_timesteps 1268.
Path 45 | total_timesteps 1298.
Path 46 | total_timesteps 1327.
Path 47 | total_timesteps 1349.
Path 48 | total_timesteps 1388.
Path 49 | total_timesteps 1402.
Path 50 | total_timesteps 1438.
Path 51 | total_timesteps 1452.
Path 52 | total_timesteps 1462.
Path 53 | total_timesteps 1472.
Path 54 | total_timesteps 1488.
Path 55 | total_timesteps 1511.
Path 56 | total_timesteps 1528.
Path 57 | total_timesteps 1557.
Path 58 | total_timesteps 1587.
Path 59 | total_timesteps 1604.
Path 60 | total_timesteps 1627.
Path 61 | total_timesteps 1638.
Path 62 | total_timesteps 1650.
Path 63 | total_timesteps 1668.
Path 64 | total_timesteps 1697.
Path 65 | total_timesteps 1766.
Path 66 | total_timesteps 1797.
Path 67 | total_timesteps 1819.
Path 68 | total_timesteps 1837.
Path 69 | total_timesteps 1869.
Path 70 | total_timesteps 1887.
Path 71 | total_timesteps 1898.
Path 72 | total_timesteps 1913.
Path 73 | total_timesteps 1939.
Path 74 | total_timesteps 1949.
Path 75 | total_timesteps 1984.
Path 76 | total_timesteps 2015.
Path 77 | total_timesteps 2039.
Path 78 | total_timesteps 2057.
Path 79 | total_timesteps 2072.
Path 80 | total_timesteps 2098.
Path 81 | total_timesteps 2111.
Path 82 | total_timesteps 2130.
Path 83 | total_timesteps 2144.
Path 84 | total_timesteps 2155.
Path 85 | total_timesteps 2181.
Path 86 | total_timesteps 2212.
Path 87 | total_timesteps 2229.
Path 88 | total_timesteps 2244.
Path 89 | total_timesteps 2276.
Path 90 | total_timesteps 2294.
Path 91 | total_timesteps 2333.
Path 92 | total_timesteps 2361.
Path 93 | total_timesteps 2408.
Path 94 | total_timesteps 2417.
Path 95 | total_timesteps 2451.
Path 96 | total_timesteps 2490.
Path 97 | total_timesteps 2507.
Path 98 | total_timesteps 2544.
Path 99 | total_timesteps 2556.
Path 100 | total_timesteps 2568.
Path 101 | total_timesteps 2617.
Path 102 | total_timesteps 2640.
Path 103 | total_timesteps 2676.
Path 104 | total_timesteps 2699.
Path 105 | total_timesteps 2714.
Path 106 | total_timesteps 2729.
Path 107 | total_timesteps 2761.
Path 108 | total_timesteps 2804.
Path 109 | total_timesteps 2864.
Path 110 | total_timesteps 2897.
Path 111 | total_timesteps 2908.
Path 112 | total_timesteps 2936.
Path 113 | total_timesteps 2952.
Path 114 | total_timesteps 2989.
Path 115 | total_timesteps 3044.
Path 116 | total_timesteps 3069.
Path 117 | total_timesteps 3096.
Path 118 | total_timesteps 3122.
Path 119 | total_timesteps 3154.
Path 120 | total_timesteps 3166.
Path 121 | total_timesteps 3182.
Path 122 | total_timesteps 3213.
Path 123 | total_timesteps 3260.
Path 124 | total_timesteps 3292.
Path 125 | total_timesteps 3323.
Path 126 | total_timesteps 3341.
Path 127 | total_timesteps 3380.
Path 128 | total_timesteps 3417.
Path 129 | total_timesteps 3452.
Path 130 | total_timesteps 3470.
Path 131 | total_timesteps 3503.
Path 132 | total_timesteps 3546.
Path 133 | total_timesteps 3567.
Path 134 | total_timesteps 3579.
Path 135 | total_timesteps 3607.
Path 136 | total_timesteps 3635.
Path 137 | total_timesteps 3656.
Path 138 | total_timesteps 3689.
Path 139 | total_timesteps 3711.
Path 140 | total_timesteps 3729.
Path 141 | total_timesteps 3773.
Path 142 | total_timesteps 3793.
Path 143 | total_timesteps 3814.
Path 144 | total_timesteps 3839.
Path 145 | total_timesteps 3860.
Path 146 | total_timesteps 3877.
Path 147 | total_timesteps 3895.
Path 148 | total_timesteps 3914.
Path 149 | total_timesteps 3934.
Path 150 | total_timesteps 3970.
Path 151 | total_timesteps 3997.
Path 152 | total_timesteps 4037.
Path 153 | total_timesteps 4060.
Path 154 | total_timesteps 4077.
Path 155 | total_timesteps 4109.
Path 156 | total_timesteps 4136.
Path 157 | total_timesteps 4200.
Path 158 | total_timesteps 4210.
Path 159 | total_timesteps 4221.
Path 160 | total_timesteps 4280.
Path 161 | total_timesteps 4302.
Path 162 | total_timesteps 4325.
Path 163 | total_timesteps 4360.
Path 164 | total_timesteps 4397.
Path 165 | total_timesteps 4427.
Path 166 | total_timesteps 4462.
Path 167 | total_timesteps 4498.
Path 168 | total_timesteps 4543.
Path 169 | total_timesteps 4555.
Path 170 | total_timesteps 4576.
Path 171 | total_timesteps 4626.
Path 172 | total_timesteps 4640.
Path 173 | total_timesteps 4669.
Path 174 | total_timesteps 4680.
Path 175 | total_timesteps 4710.
Path 176 | total_timesteps 4730.
Path 177 | total_timesteps 4760.
Path 178 | total_timesteps 4797.
Path 179 | total_timesteps 4827.
Path 180 | total_timesteps 4889.
Path 181 | total_timesteps 4904.
Path 182 | total_timesteps 4914.
Path 183 | total_timesteps 4938.
Path 184 | total_timesteps 4969.
Path 185 | total_timesteps 4984.
Path 186 | total_timesteps 4994.
Path 187 | total_timesteps 5008.
Path 188 | total_timesteps 5028.
Path 189 | total_timesteps 5041.
Path 190 | total_timesteps 5054.
Path 191 | total_timesteps 5110.
Path 192 | total_timesteps 5134.
Path 193 | total_timesteps 5153.
Path 194 | total_timesteps 5175.
Path 195 | total_timesteps 5200.
Path 196 | total_timesteps 5218.
Path 197 | total_timesteps 5246.
Path 198 | total_timesteps 5261.
Path 199 | total_timesteps 5273.
Path 200 | total_timesteps 5311.
Path 201 | total_timesteps 5324.
Path 202 | total_timesteps 5335.
Path 203 | total_timesteps 5352.
Path 204 | total_timesteps 5371.
Path 205 | total_timesteps 5412.
Path 206 | total_timesteps 5435.
Path 207 | total_timesteps 5445.
Path 208 | total_timesteps 5480.
Path 209 | total_timesteps 5535.
Path 210 | total_timesteps 5556.
Path 211 | total_timesteps 5576.
Path 212 | total_timesteps 5584.
Path 213 | total_timesteps 5598.
Path 214 | total_timesteps 5628.
Path 215 | total_timesteps 5668.
Path 216 | total_timesteps 5689.
Path 217 | total_timesteps 5700.
Path 218 | total_timesteps 5733.
Path 219 | total_timesteps 5759.
Path 220 | total_timesteps 5789.
Path 221 | total_timesteps 5809.
Path 222 | total_timesteps 5830.
Path 223 | total_timesteps 5854.
Path 224 | total_timesteps 5896.
Path 225 | total_timesteps 5923.
Path 226 | total_timesteps 5948.
Path 227 | total_timesteps 5972.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.61    |
| Iteration     | 8        |
| MaximumReturn | 28.2     |
| MinimumReturn | -19.2    |
| TotalSamples  | 40081    |
----------------------------
itr #9 | 
Fitting dynamics.
Validation loss = 0.010529253631830215
Validation loss = 0.008702284656465054
Validation loss = 0.008455886505544186
Validation loss = 0.008506020531058311
Validation loss = 0.008664804510772228
Validation loss = 0.008350925520062447
Validation loss = 0.00883664283901453
Validation loss = 0.00994817353785038
Validation loss = 0.009198482148349285
Validation loss = 0.00797799602150917
Validation loss = 0.008144577965140343
Validation loss = 0.00946052372455597
Validation loss = 0.008317746222019196
Validation loss = 0.00909093115478754
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 32.
Path 2 | total_timesteps 61.
Path 3 | total_timesteps 84.
Path 4 | total_timesteps 98.
Path 5 | total_timesteps 121.
Path 6 | total_timesteps 142.
Path 7 | total_timesteps 189.
Path 8 | total_timesteps 203.
Path 9 | total_timesteps 229.
Path 10 | total_timesteps 249.
Path 11 | total_timesteps 280.
Path 12 | total_timesteps 300.
Path 13 | total_timesteps 327.
Path 14 | total_timesteps 357.
Path 15 | total_timesteps 383.
Path 16 | total_timesteps 417.
Path 17 | total_timesteps 435.
Path 18 | total_timesteps 461.
Path 19 | total_timesteps 483.
Path 20 | total_timesteps 499.
Path 21 | total_timesteps 537.
Path 22 | total_timesteps 557.
Path 23 | total_timesteps 599.
Path 24 | total_timesteps 628.
Path 25 | total_timesteps 652.
Path 26 | total_timesteps 673.
Path 27 | total_timesteps 716.
Path 28 | total_timesteps 743.
Path 29 | total_timesteps 761.
Path 30 | total_timesteps 773.
Path 31 | total_timesteps 791.
Path 32 | total_timesteps 804.
Path 33 | total_timesteps 845.
Path 34 | total_timesteps 877.
Path 35 | total_timesteps 892.
Path 36 | total_timesteps 908.
Path 37 | total_timesteps 927.
Path 38 | total_timesteps 949.
Path 39 | total_timesteps 971.
Path 40 | total_timesteps 985.
Path 41 | total_timesteps 1001.
Path 42 | total_timesteps 1026.
Path 43 | total_timesteps 1045.
Path 44 | total_timesteps 1078.
Path 45 | total_timesteps 1133.
Path 46 | total_timesteps 1161.
Path 47 | total_timesteps 1176.
Path 48 | total_timesteps 1203.
Path 49 | total_timesteps 1221.
Path 50 | total_timesteps 1245.
Path 51 | total_timesteps 1263.
Path 52 | total_timesteps 1303.
Path 53 | total_timesteps 1331.
Path 54 | total_timesteps 1357.
Path 55 | total_timesteps 1385.
Path 56 | total_timesteps 1417.
Path 57 | total_timesteps 1434.
Path 58 | total_timesteps 1454.
Path 59 | total_timesteps 1465.
Path 60 | total_timesteps 1492.
Path 61 | total_timesteps 1517.
Path 62 | total_timesteps 1553.
Path 63 | total_timesteps 1573.
Path 64 | total_timesteps 1592.
Path 65 | total_timesteps 1614.
Path 66 | total_timesteps 1650.
Path 67 | total_timesteps 1683.
Path 68 | total_timesteps 1714.
Path 69 | total_timesteps 1732.
Path 70 | total_timesteps 1765.
Path 71 | total_timesteps 1807.
Path 72 | total_timesteps 1832.
Path 73 | total_timesteps 1900.
Path 74 | total_timesteps 1949.
Path 75 | total_timesteps 1972.
Path 76 | total_timesteps 2005.
Path 77 | total_timesteps 2018.
Path 78 | total_timesteps 2045.
Path 79 | total_timesteps 2075.
Path 80 | total_timesteps 2100.
Path 81 | total_timesteps 2119.
Path 82 | total_timesteps 2148.
Path 83 | total_timesteps 2165.
Path 84 | total_timesteps 2182.
Path 85 | total_timesteps 2193.
Path 86 | total_timesteps 2225.
Path 87 | total_timesteps 2236.
Path 88 | total_timesteps 2251.
Path 89 | total_timesteps 2303.
Path 90 | total_timesteps 2339.
Path 91 | total_timesteps 2389.
Path 92 | total_timesteps 2411.
Path 93 | total_timesteps 2440.
Path 94 | total_timesteps 2492.
Path 95 | total_timesteps 2516.
Path 96 | total_timesteps 2544.
Path 97 | total_timesteps 2565.
Path 98 | total_timesteps 2584.
Path 99 | total_timesteps 2631.
Path 100 | total_timesteps 2650.
Path 101 | total_timesteps 2692.
Path 102 | total_timesteps 2722.
Path 103 | total_timesteps 2772.
Path 104 | total_timesteps 2815.
Path 105 | total_timesteps 2834.
Path 106 | total_timesteps 2861.
Path 107 | total_timesteps 2897.
Path 108 | total_timesteps 2914.
Path 109 | total_timesteps 2965.
Path 110 | total_timesteps 2994.
Path 111 | total_timesteps 3023.
Path 112 | total_timesteps 3067.
Path 113 | total_timesteps 3109.
Path 114 | total_timesteps 3130.
Path 115 | total_timesteps 3150.
Path 116 | total_timesteps 3185.
Path 117 | total_timesteps 3203.
Path 118 | total_timesteps 3223.
Path 119 | total_timesteps 3248.
Path 120 | total_timesteps 3275.
Path 121 | total_timesteps 3289.
Path 122 | total_timesteps 3305.
Path 123 | total_timesteps 3330.
Path 124 | total_timesteps 3351.
Path 125 | total_timesteps 3375.
Path 126 | total_timesteps 3391.
Path 127 | total_timesteps 3412.
Path 128 | total_timesteps 3428.
Path 129 | total_timesteps 3445.
Path 130 | total_timesteps 3475.
Path 131 | total_timesteps 3485.
Path 132 | total_timesteps 3501.
Path 133 | total_timesteps 3528.
Path 134 | total_timesteps 3545.
Path 135 | total_timesteps 3560.
Path 136 | total_timesteps 3572.
Path 137 | total_timesteps 3591.
Path 138 | total_timesteps 3638.
Path 139 | total_timesteps 3691.
Path 140 | total_timesteps 3712.
Path 141 | total_timesteps 3729.
Path 142 | total_timesteps 3745.
Path 143 | total_timesteps 3769.
Path 144 | total_timesteps 3788.
Path 145 | total_timesteps 3806.
Path 146 | total_timesteps 3820.
Path 147 | total_timesteps 3836.
Path 148 | total_timesteps 3850.
Path 149 | total_timesteps 3871.
Path 150 | total_timesteps 3885.
Path 151 | total_timesteps 3907.
Path 152 | total_timesteps 3947.
Path 153 | total_timesteps 3993.
Path 154 | total_timesteps 4009.
Path 155 | total_timesteps 4040.
Path 156 | total_timesteps 4053.
Path 157 | total_timesteps 4066.
Path 158 | total_timesteps 4096.
Path 159 | total_timesteps 4111.
Path 160 | total_timesteps 4128.
Path 161 | total_timesteps 4150.
Path 162 | total_timesteps 4171.
Path 163 | total_timesteps 4205.
Path 164 | total_timesteps 4254.
Path 165 | total_timesteps 4275.
Path 166 | total_timesteps 4282.
Path 167 | total_timesteps 4306.
Path 168 | total_timesteps 4322.
Path 169 | total_timesteps 4334.
Path 170 | total_timesteps 4362.
Path 171 | total_timesteps 4393.
Path 172 | total_timesteps 4419.
Path 173 | total_timesteps 4432.
Path 174 | total_timesteps 4463.
Path 175 | total_timesteps 4507.
Path 176 | total_timesteps 4550.
Path 177 | total_timesteps 4577.
Path 178 | total_timesteps 4601.
Path 179 | total_timesteps 4626.
Path 180 | total_timesteps 4641.
Path 181 | total_timesteps 4669.
Path 182 | total_timesteps 4691.
Path 183 | total_timesteps 4718.
Path 184 | total_timesteps 4751.
Path 185 | total_timesteps 4763.
Path 186 | total_timesteps 4816.
Path 187 | total_timesteps 4836.
Path 188 | total_timesteps 4866.
Path 189 | total_timesteps 4895.
Path 190 | total_timesteps 4916.
Path 191 | total_timesteps 4934.
Path 192 | total_timesteps 4967.
Path 193 | total_timesteps 4992.
Path 194 | total_timesteps 5002.
Path 195 | total_timesteps 5019.
Path 196 | total_timesteps 5051.
Path 197 | total_timesteps 5076.
Path 198 | total_timesteps 5096.
Path 199 | total_timesteps 5130.
Path 200 | total_timesteps 5156.
Path 201 | total_timesteps 5203.
Path 202 | total_timesteps 5242.
Path 203 | total_timesteps 5275.
Path 204 | total_timesteps 5304.
Path 205 | total_timesteps 5318.
Path 206 | total_timesteps 5342.
Path 207 | total_timesteps 5378.
Path 208 | total_timesteps 5408.
Path 209 | total_timesteps 5448.
Path 210 | total_timesteps 5479.
Path 211 | total_timesteps 5512.
Path 212 | total_timesteps 5556.
Path 213 | total_timesteps 5590.
Path 214 | total_timesteps 5619.
Path 215 | total_timesteps 5644.
Path 216 | total_timesteps 5685.
Path 217 | total_timesteps 5729.
Path 218 | total_timesteps 5758.
Path 219 | total_timesteps 5799.
Path 220 | total_timesteps 5831.
Path 221 | total_timesteps 5867.
Path 222 | total_timesteps 5893.
Path 223 | total_timesteps 5949.
Path 224 | total_timesteps 5961.
Path 225 | total_timesteps 5978.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.11    |
| Iteration     | 9        |
| MaximumReturn | 17.3     |
| MinimumReturn | -19.7    |
| TotalSamples  | 44086    |
----------------------------
itr #10 | 
Fitting dynamics.
Validation loss = 0.009299739263951778
Validation loss = 0.008019394241273403
Validation loss = 0.00936902966350317
Validation loss = 0.007967176847159863
Validation loss = 0.007520623039454222
Validation loss = 0.007633577100932598
Validation loss = 0.008326507173478603
Validation loss = 0.007592084817588329
Validation loss = 0.0075866179540753365
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 27.
Path 2 | total_timesteps 57.
Path 3 | total_timesteps 92.
Path 4 | total_timesteps 126.
Path 5 | total_timesteps 134.
Path 6 | total_timesteps 161.
Path 7 | total_timesteps 182.
Path 8 | total_timesteps 211.
Path 9 | total_timesteps 241.
Path 10 | total_timesteps 265.
Path 11 | total_timesteps 276.
Path 12 | total_timesteps 309.
Path 13 | total_timesteps 322.
Path 14 | total_timesteps 344.
Path 15 | total_timesteps 374.
Path 16 | total_timesteps 390.
Path 17 | total_timesteps 411.
Path 18 | total_timesteps 433.
Path 19 | total_timesteps 444.
Path 20 | total_timesteps 478.
Path 21 | total_timesteps 510.
Path 22 | total_timesteps 542.
Path 23 | total_timesteps 563.
Path 24 | total_timesteps 600.
Path 25 | total_timesteps 632.
Path 26 | total_timesteps 650.
Path 27 | total_timesteps 668.
Path 28 | total_timesteps 689.
Path 29 | total_timesteps 708.
Path 30 | total_timesteps 759.
Path 31 | total_timesteps 804.
Path 32 | total_timesteps 836.
Path 33 | total_timesteps 853.
Path 34 | total_timesteps 867.
Path 35 | total_timesteps 899.
Path 36 | total_timesteps 912.
Path 37 | total_timesteps 943.
Path 38 | total_timesteps 963.
Path 39 | total_timesteps 983.
Path 40 | total_timesteps 1004.
Path 41 | total_timesteps 1026.
Path 42 | total_timesteps 1050.
Path 43 | total_timesteps 1080.
Path 44 | total_timesteps 1102.
Path 45 | total_timesteps 1126.
Path 46 | total_timesteps 1168.
Path 47 | total_timesteps 1186.
Path 48 | total_timesteps 1209.
Path 49 | total_timesteps 1238.
Path 50 | total_timesteps 1284.
Path 51 | total_timesteps 1341.
Path 52 | total_timesteps 1390.
Path 53 | total_timesteps 1407.
Path 54 | total_timesteps 1440.
Path 55 | total_timesteps 1461.
Path 56 | total_timesteps 1481.
Path 57 | total_timesteps 1509.
Path 58 | total_timesteps 1523.
Path 59 | total_timesteps 1545.
Path 60 | total_timesteps 1571.
Path 61 | total_timesteps 1595.
Path 62 | total_timesteps 1621.
Path 63 | total_timesteps 1641.
Path 64 | total_timesteps 1670.
Path 65 | total_timesteps 1681.
Path 66 | total_timesteps 1706.
Path 67 | total_timesteps 1735.
Path 68 | total_timesteps 1794.
Path 69 | total_timesteps 1831.
Path 70 | total_timesteps 1922.
Path 71 | total_timesteps 1957.
Path 72 | total_timesteps 2007.
Path 73 | total_timesteps 2021.
Path 74 | total_timesteps 2036.
Path 75 | total_timesteps 2077.
Path 76 | total_timesteps 2113.
Path 77 | total_timesteps 2127.
Path 78 | total_timesteps 2136.
Path 79 | total_timesteps 2157.
Path 80 | total_timesteps 2194.
Path 81 | total_timesteps 2222.
Path 82 | total_timesteps 2239.
Path 83 | total_timesteps 2268.
Path 84 | total_timesteps 2290.
Path 85 | total_timesteps 2305.
Path 86 | total_timesteps 2344.
Path 87 | total_timesteps 2365.
Path 88 | total_timesteps 2402.
Path 89 | total_timesteps 2451.
Path 90 | total_timesteps 2467.
Path 91 | total_timesteps 2480.
Path 92 | total_timesteps 2498.
Path 93 | total_timesteps 2527.
Path 94 | total_timesteps 2545.
Path 95 | total_timesteps 2573.
Path 96 | total_timesteps 2587.
Path 97 | total_timesteps 2616.
Path 98 | total_timesteps 2629.
Path 99 | total_timesteps 2664.
Path 100 | total_timesteps 2694.
Path 101 | total_timesteps 2706.
Path 102 | total_timesteps 2760.
Path 103 | total_timesteps 2808.
Path 104 | total_timesteps 2816.
Path 105 | total_timesteps 2837.
Path 106 | total_timesteps 2852.
Path 107 | total_timesteps 2881.
Path 108 | total_timesteps 2904.
Path 109 | total_timesteps 2919.
Path 110 | total_timesteps 2958.
Path 111 | total_timesteps 2979.
Path 112 | total_timesteps 3001.
Path 113 | total_timesteps 3020.
Path 114 | total_timesteps 3054.
Path 115 | total_timesteps 3077.
Path 116 | total_timesteps 3089.
Path 117 | total_timesteps 3164.
Path 118 | total_timesteps 3172.
Path 119 | total_timesteps 3201.
Path 120 | total_timesteps 3236.
Path 121 | total_timesteps 3264.
Path 122 | total_timesteps 3281.
Path 123 | total_timesteps 3315.
Path 124 | total_timesteps 3338.
Path 125 | total_timesteps 3363.
Path 126 | total_timesteps 3382.
Path 127 | total_timesteps 3416.
Path 128 | total_timesteps 3452.
Path 129 | total_timesteps 3477.
Path 130 | total_timesteps 3530.
Path 131 | total_timesteps 3550.
Path 132 | total_timesteps 3574.
Path 133 | total_timesteps 3599.
Path 134 | total_timesteps 3648.
Path 135 | total_timesteps 3660.
Path 136 | total_timesteps 3692.
Path 137 | total_timesteps 3736.
Path 138 | total_timesteps 3754.
Path 139 | total_timesteps 3773.
Path 140 | total_timesteps 3793.
Path 141 | total_timesteps 3813.
Path 142 | total_timesteps 3826.
Path 143 | total_timesteps 3867.
Path 144 | total_timesteps 3888.
Path 145 | total_timesteps 3913.
Path 146 | total_timesteps 3937.
Path 147 | total_timesteps 3963.
Path 148 | total_timesteps 3978.
Path 149 | total_timesteps 3998.
Path 150 | total_timesteps 4018.
Path 151 | total_timesteps 4053.
Path 152 | total_timesteps 4085.
Path 153 | total_timesteps 4120.
Path 154 | total_timesteps 4140.
Path 155 | total_timesteps 4167.
Path 156 | total_timesteps 4183.
Path 157 | total_timesteps 4240.
Path 158 | total_timesteps 4270.
Path 159 | total_timesteps 4284.
Path 160 | total_timesteps 4305.
Path 161 | total_timesteps 4339.
Path 162 | total_timesteps 4370.
Path 163 | total_timesteps 4396.
Path 164 | total_timesteps 4421.
Path 165 | total_timesteps 4443.
Path 166 | total_timesteps 4457.
Path 167 | total_timesteps 4500.
Path 168 | total_timesteps 4512.
Path 169 | total_timesteps 4526.
Path 170 | total_timesteps 4553.
Path 171 | total_timesteps 4582.
Path 172 | total_timesteps 4604.
Path 173 | total_timesteps 4639.
Path 174 | total_timesteps 4647.
Path 175 | total_timesteps 4685.
Path 176 | total_timesteps 4718.
Path 177 | total_timesteps 4772.
Path 178 | total_timesteps 4789.
Path 179 | total_timesteps 4821.
Path 180 | total_timesteps 4843.
Path 181 | total_timesteps 4864.
Path 182 | total_timesteps 4877.
Path 183 | total_timesteps 4920.
Path 184 | total_timesteps 4954.
Path 185 | total_timesteps 4970.
Path 186 | total_timesteps 4988.
Path 187 | total_timesteps 5048.
Path 188 | total_timesteps 5078.
Path 189 | total_timesteps 5089.
Path 190 | total_timesteps 5114.
Path 191 | total_timesteps 5142.
Path 192 | total_timesteps 5183.
Path 193 | total_timesteps 5205.
Path 194 | total_timesteps 5224.
Path 195 | total_timesteps 5247.
Path 196 | total_timesteps 5259.
Path 197 | total_timesteps 5272.
Path 198 | total_timesteps 5303.
Path 199 | total_timesteps 5329.
Path 200 | total_timesteps 5362.
Path 201 | total_timesteps 5417.
Path 202 | total_timesteps 5441.
Path 203 | total_timesteps 5479.
Path 204 | total_timesteps 5489.
Path 205 | total_timesteps 5512.
Path 206 | total_timesteps 5545.
Path 207 | total_timesteps 5560.
Path 208 | total_timesteps 5604.
Path 209 | total_timesteps 5638.
Path 210 | total_timesteps 5655.
Path 211 | total_timesteps 5687.
Path 212 | total_timesteps 5713.
Path 213 | total_timesteps 5732.
Path 214 | total_timesteps 5754.
Path 215 | total_timesteps 5777.
Path 216 | total_timesteps 5797.
Path 217 | total_timesteps 5824.
Path 218 | total_timesteps 5849.
Path 219 | total_timesteps 5862.
Path 220 | total_timesteps 5876.
Path 221 | total_timesteps 5899.
Path 222 | total_timesteps 5933.
Path 223 | total_timesteps 5952.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.12    |
| Iteration     | 10       |
| MaximumReturn | 42.1     |
| MinimumReturn | -21.4    |
| TotalSamples  | 48094    |
----------------------------
itr #11 | 
Fitting dynamics.
Validation loss = 0.007701473776251078
Validation loss = 0.007296728435903788
Validation loss = 0.007218783255666494
Validation loss = 0.006935447920113802
Validation loss = 0.007693871855735779
Validation loss = 0.007674673106521368
Validation loss = 0.006798541639000177
Validation loss = 0.007093951106071472
Validation loss = 0.007212217431515455
Validation loss = 0.007676953449845314
Validation loss = 0.006930625531822443
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 35.
Path 3 | total_timesteps 49.
Path 4 | total_timesteps 71.
Path 5 | total_timesteps 84.
Path 6 | total_timesteps 108.
Path 7 | total_timesteps 121.
Path 8 | total_timesteps 152.
Path 9 | total_timesteps 171.
Path 10 | total_timesteps 198.
Path 11 | total_timesteps 222.
Path 12 | total_timesteps 232.
Path 13 | total_timesteps 262.
Path 14 | total_timesteps 288.
Path 15 | total_timesteps 337.
Path 16 | total_timesteps 349.
Path 17 | total_timesteps 400.
Path 18 | total_timesteps 409.
Path 19 | total_timesteps 424.
Path 20 | total_timesteps 456.
Path 21 | total_timesteps 471.
Path 22 | total_timesteps 506.
Path 23 | total_timesteps 520.
Path 24 | total_timesteps 534.
Path 25 | total_timesteps 552.
Path 26 | total_timesteps 569.
Path 27 | total_timesteps 596.
Path 28 | total_timesteps 614.
Path 29 | total_timesteps 628.
Path 30 | total_timesteps 643.
Path 31 | total_timesteps 657.
Path 32 | total_timesteps 670.
Path 33 | total_timesteps 702.
Path 34 | total_timesteps 714.
Path 35 | total_timesteps 745.
Path 36 | total_timesteps 770.
Path 37 | total_timesteps 786.
Path 38 | total_timesteps 804.
Path 39 | total_timesteps 814.
Path 40 | total_timesteps 828.
Path 41 | total_timesteps 840.
Path 42 | total_timesteps 866.
Path 43 | total_timesteps 881.
Path 44 | total_timesteps 906.
Path 45 | total_timesteps 928.
Path 46 | total_timesteps 940.
Path 47 | total_timesteps 948.
Path 48 | total_timesteps 962.
Path 49 | total_timesteps 971.
Path 50 | total_timesteps 979.
Path 51 | total_timesteps 1009.
Path 52 | total_timesteps 1034.
Path 53 | total_timesteps 1046.
Path 54 | total_timesteps 1065.
Path 55 | total_timesteps 1076.
Path 56 | total_timesteps 1131.
Path 57 | total_timesteps 1143.
Path 58 | total_timesteps 1164.
Path 59 | total_timesteps 1198.
Path 60 | total_timesteps 1207.
Path 61 | total_timesteps 1231.
Path 62 | total_timesteps 1247.
Path 63 | total_timesteps 1271.
Path 64 | total_timesteps 1306.
Path 65 | total_timesteps 1315.
Path 66 | total_timesteps 1342.
Path 67 | total_timesteps 1365.
Path 68 | total_timesteps 1399.
Path 69 | total_timesteps 1423.
Path 70 | total_timesteps 1447.
Path 71 | total_timesteps 1470.
Path 72 | total_timesteps 1498.
Path 73 | total_timesteps 1536.
Path 74 | total_timesteps 1550.
Path 75 | total_timesteps 1581.
Path 76 | total_timesteps 1595.
Path 77 | total_timesteps 1623.
Path 78 | total_timesteps 1632.
Path 79 | total_timesteps 1644.
Path 80 | total_timesteps 1686.
Path 81 | total_timesteps 1702.
Path 82 | total_timesteps 1726.
Path 83 | total_timesteps 1755.
Path 84 | total_timesteps 1762.
Path 85 | total_timesteps 1779.
Path 86 | total_timesteps 1802.
Path 87 | total_timesteps 1838.
Path 88 | total_timesteps 1852.
Path 89 | total_timesteps 1869.
Path 90 | total_timesteps 1889.
Path 91 | total_timesteps 1899.
Path 92 | total_timesteps 1968.
Path 93 | total_timesteps 1992.
Path 94 | total_timesteps 2015.
Path 95 | total_timesteps 2039.
Path 96 | total_timesteps 2051.
Path 97 | total_timesteps 2064.
Path 98 | total_timesteps 2081.
Path 99 | total_timesteps 2104.
Path 100 | total_timesteps 2115.
Path 101 | total_timesteps 2127.
Path 102 | total_timesteps 2139.
Path 103 | total_timesteps 2154.
Path 104 | total_timesteps 2168.
Path 105 | total_timesteps 2187.
Path 106 | total_timesteps 2200.
Path 107 | total_timesteps 2215.
Path 108 | total_timesteps 2224.
Path 109 | total_timesteps 2238.
Path 110 | total_timesteps 2262.
Path 111 | total_timesteps 2275.
Path 112 | total_timesteps 2297.
Path 113 | total_timesteps 2320.
Path 114 | total_timesteps 2350.
Path 115 | total_timesteps 2375.
Path 116 | total_timesteps 2396.
Path 117 | total_timesteps 2417.
Path 118 | total_timesteps 2440.
Path 119 | total_timesteps 2455.
Path 120 | total_timesteps 2492.
Path 121 | total_timesteps 2510.
Path 122 | total_timesteps 2528.
Path 123 | total_timesteps 2559.
Path 124 | total_timesteps 2578.
Path 125 | total_timesteps 2607.
Path 126 | total_timesteps 2615.
Path 127 | total_timesteps 2634.
Path 128 | total_timesteps 2678.
Path 129 | total_timesteps 2700.
Path 130 | total_timesteps 2727.
Path 131 | total_timesteps 2737.
Path 132 | total_timesteps 2754.
Path 133 | total_timesteps 2827.
Path 134 | total_timesteps 2835.
Path 135 | total_timesteps 2852.
Path 136 | total_timesteps 2866.
Path 137 | total_timesteps 2877.
Path 138 | total_timesteps 2892.
Path 139 | total_timesteps 2915.
Path 140 | total_timesteps 2927.
Path 141 | total_timesteps 2949.
Path 142 | total_timesteps 2974.
Path 143 | total_timesteps 2991.
Path 144 | total_timesteps 3023.
Path 145 | total_timesteps 3037.
Path 146 | total_timesteps 3058.
Path 147 | total_timesteps 3078.
Path 148 | total_timesteps 3103.
Path 149 | total_timesteps 3118.
Path 150 | total_timesteps 3141.
Path 151 | total_timesteps 3160.
Path 152 | total_timesteps 3183.
Path 153 | total_timesteps 3197.
Path 154 | total_timesteps 3213.
Path 155 | total_timesteps 3237.
Path 156 | total_timesteps 3264.
Path 157 | total_timesteps 3290.
Path 158 | total_timesteps 3303.
Path 159 | total_timesteps 3321.
Path 160 | total_timesteps 3344.
Path 161 | total_timesteps 3363.
Path 162 | total_timesteps 3382.
Path 163 | total_timesteps 3400.
Path 164 | total_timesteps 3412.
Path 165 | total_timesteps 3428.
Path 166 | total_timesteps 3448.
Path 167 | total_timesteps 3473.
Path 168 | total_timesteps 3499.
Path 169 | total_timesteps 3522.
Path 170 | total_timesteps 3546.
Path 171 | total_timesteps 3555.
Path 172 | total_timesteps 3565.
Path 173 | total_timesteps 3582.
Path 174 | total_timesteps 3615.
Path 175 | total_timesteps 3632.
Path 176 | total_timesteps 3650.
Path 177 | total_timesteps 3669.
Path 178 | total_timesteps 3689.
Path 179 | total_timesteps 3707.
Path 180 | total_timesteps 3730.
Path 181 | total_timesteps 3743.
Path 182 | total_timesteps 3755.
Path 183 | total_timesteps 3782.
Path 184 | total_timesteps 3796.
Path 185 | total_timesteps 3825.
Path 186 | total_timesteps 3852.
Path 187 | total_timesteps 3872.
Path 188 | total_timesteps 3895.
Path 189 | total_timesteps 3916.
Path 190 | total_timesteps 3924.
Path 191 | total_timesteps 3941.
Path 192 | total_timesteps 3974.
Path 193 | total_timesteps 4002.
Path 194 | total_timesteps 4013.
Path 195 | total_timesteps 4029.
Path 196 | total_timesteps 4052.
Path 197 | total_timesteps 4064.
Path 198 | total_timesteps 4086.
Path 199 | total_timesteps 4105.
Path 200 | total_timesteps 4125.
Path 201 | total_timesteps 4153.
Path 202 | total_timesteps 4173.
Path 203 | total_timesteps 4188.
Path 204 | total_timesteps 4201.
Path 205 | total_timesteps 4230.
Path 206 | total_timesteps 4245.
Path 207 | total_timesteps 4268.
Path 208 | total_timesteps 4290.
Path 209 | total_timesteps 4301.
Path 210 | total_timesteps 4312.
Path 211 | total_timesteps 4344.
Path 212 | total_timesteps 4376.
Path 213 | total_timesteps 4391.
Path 214 | total_timesteps 4405.
Path 215 | total_timesteps 4416.
Path 216 | total_timesteps 4450.
Path 217 | total_timesteps 4466.
Path 218 | total_timesteps 4486.
Path 219 | total_timesteps 4505.
Path 220 | total_timesteps 4516.
Path 221 | total_timesteps 4549.
Path 222 | total_timesteps 4565.
Path 223 | total_timesteps 4594.
Path 224 | total_timesteps 4609.
Path 225 | total_timesteps 4619.
Path 226 | total_timesteps 4627.
Path 227 | total_timesteps 4652.
Path 228 | total_timesteps 4682.
Path 229 | total_timesteps 4696.
Path 230 | total_timesteps 4721.
Path 231 | total_timesteps 4740.
Path 232 | total_timesteps 4764.
Path 233 | total_timesteps 4797.
Path 234 | total_timesteps 4829.
Path 235 | total_timesteps 4858.
Path 236 | total_timesteps 4906.
Path 237 | total_timesteps 4925.
Path 238 | total_timesteps 4979.
Path 239 | total_timesteps 4996.
Path 240 | total_timesteps 5014.
Path 241 | total_timesteps 5028.
Path 242 | total_timesteps 5051.
Path 243 | total_timesteps 5060.
Path 244 | total_timesteps 5074.
Path 245 | total_timesteps 5084.
Path 246 | total_timesteps 5099.
Path 247 | total_timesteps 5125.
Path 248 | total_timesteps 5150.
Path 249 | total_timesteps 5167.
Path 250 | total_timesteps 5185.
Path 251 | total_timesteps 5200.
Path 252 | total_timesteps 5220.
Path 253 | total_timesteps 5235.
Path 254 | total_timesteps 5254.
Path 255 | total_timesteps 5266.
Path 256 | total_timesteps 5284.
Path 257 | total_timesteps 5297.
Path 258 | total_timesteps 5333.
Path 259 | total_timesteps 5388.
Path 260 | total_timesteps 5397.
Path 261 | total_timesteps 5413.
Path 262 | total_timesteps 5444.
Path 263 | total_timesteps 5454.
Path 264 | total_timesteps 5490.
Path 265 | total_timesteps 5514.
Path 266 | total_timesteps 5542.
Path 267 | total_timesteps 5554.
Path 268 | total_timesteps 5586.
Path 269 | total_timesteps 5607.
Path 270 | total_timesteps 5642.
Path 271 | total_timesteps 5658.
Path 272 | total_timesteps 5667.
Path 273 | total_timesteps 5684.
Path 274 | total_timesteps 5706.
Path 275 | total_timesteps 5725.
Path 276 | total_timesteps 5746.
Path 277 | total_timesteps 5775.
Path 278 | total_timesteps 5786.
Path 279 | total_timesteps 5800.
Path 280 | total_timesteps 5833.
Path 281 | total_timesteps 5847.
Path 282 | total_timesteps 5871.
Path 283 | total_timesteps 5891.
Path 284 | total_timesteps 5911.
Path 285 | total_timesteps 5935.
Path 286 | total_timesteps 5947.
Path 287 | total_timesteps 5965.
Path 288 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.68    |
| Iteration     | 11       |
| MaximumReturn | 25.3     |
| MinimumReturn | -27      |
| TotalSamples  | 52107    |
----------------------------
itr #12 | 
Fitting dynamics.
Validation loss = 0.007384079042822123
Validation loss = 0.006722648628056049
Validation loss = 0.006915082689374685
Validation loss = 0.00807885266840458
Validation loss = 0.006664273329079151
Validation loss = 0.006632419768720865
Validation loss = 0.006337231956422329
Validation loss = 0.006178286857903004
Validation loss = 0.008071287535130978
Validation loss = 0.006047520320862532
Validation loss = 0.006616158410906792
Validation loss = 0.006132142618298531
Validation loss = 0.006260933820158243
Validation loss = 0.006022224668413401
Validation loss = 0.006189287640154362
Validation loss = 0.005912382621318102
Validation loss = 0.006139468867331743
Validation loss = 0.006063177715986967
Validation loss = 0.006536274217069149
Validation loss = 0.0058713979087769985
Validation loss = 0.0058027650229632854
Validation loss = 0.006196133326739073
Validation loss = 0.005685135722160339
Validation loss = 0.005677231587469578
Validation loss = 0.005667275749146938
Validation loss = 0.006160035263746977
Validation loss = 0.005871118046343327
Validation loss = 0.005772287491708994
Validation loss = 0.005475714337080717
Validation loss = 0.006148355081677437
Validation loss = 0.005710438825190067
Validation loss = 0.005442391615360975
Validation loss = 0.00611555902287364
Validation loss = 0.006693520583212376
Validation loss = 0.00558122992515564
Validation loss = 0.0054375166073441505
Validation loss = 0.005705324467271566
Validation loss = 0.005907549988478422
Validation loss = 0.0060483370907604694
Validation loss = 0.005632974207401276
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 47.
Path 4 | total_timesteps 62.
Path 5 | total_timesteps 92.
Path 6 | total_timesteps 100.
Path 7 | total_timesteps 118.
Path 8 | total_timesteps 144.
Path 9 | total_timesteps 159.
Path 10 | total_timesteps 171.
Path 11 | total_timesteps 187.
Path 12 | total_timesteps 205.
Path 13 | total_timesteps 230.
Path 14 | total_timesteps 270.
Path 15 | total_timesteps 282.
Path 16 | total_timesteps 315.
Path 17 | total_timesteps 330.
Path 18 | total_timesteps 363.
Path 19 | total_timesteps 391.
Path 20 | total_timesteps 409.
Path 21 | total_timesteps 421.
Path 22 | total_timesteps 443.
Path 23 | total_timesteps 478.
Path 24 | total_timesteps 503.
Path 25 | total_timesteps 526.
Path 26 | total_timesteps 543.
Path 27 | total_timesteps 595.
Path 28 | total_timesteps 615.
Path 29 | total_timesteps 632.
Path 30 | total_timesteps 667.
Path 31 | total_timesteps 682.
Path 32 | total_timesteps 691.
Path 33 | total_timesteps 707.
Path 34 | total_timesteps 739.
Path 35 | total_timesteps 767.
Path 36 | total_timesteps 780.
Path 37 | total_timesteps 798.
Path 38 | total_timesteps 825.
Path 39 | total_timesteps 851.
Path 40 | total_timesteps 862.
Path 41 | total_timesteps 877.
Path 42 | total_timesteps 900.
Path 43 | total_timesteps 919.
Path 44 | total_timesteps 931.
Path 45 | total_timesteps 956.
Path 46 | total_timesteps 973.
Path 47 | total_timesteps 987.
Path 48 | total_timesteps 1015.
Path 49 | total_timesteps 1042.
Path 50 | total_timesteps 1070.
Path 51 | total_timesteps 1087.
Path 52 | total_timesteps 1111.
Path 53 | total_timesteps 1128.
Path 54 | total_timesteps 1137.
Path 55 | total_timesteps 1150.
Path 56 | total_timesteps 1182.
Path 57 | total_timesteps 1200.
Path 58 | total_timesteps 1223.
Path 59 | total_timesteps 1239.
Path 60 | total_timesteps 1274.
Path 61 | total_timesteps 1285.
Path 62 | total_timesteps 1299.
Path 63 | total_timesteps 1341.
Path 64 | total_timesteps 1352.
Path 65 | total_timesteps 1375.
Path 66 | total_timesteps 1406.
Path 67 | total_timesteps 1414.
Path 68 | total_timesteps 1432.
Path 69 | total_timesteps 1453.
Path 70 | total_timesteps 1469.
Path 71 | total_timesteps 1488.
Path 72 | total_timesteps 1509.
Path 73 | total_timesteps 1533.
Path 74 | total_timesteps 1567.
Path 75 | total_timesteps 1579.
Path 76 | total_timesteps 1629.
Path 77 | total_timesteps 1642.
Path 78 | total_timesteps 1654.
Path 79 | total_timesteps 1661.
Path 80 | total_timesteps 1670.
Path 81 | total_timesteps 1690.
Path 82 | total_timesteps 1717.
Path 83 | total_timesteps 1730.
Path 84 | total_timesteps 1740.
Path 85 | total_timesteps 1754.
Path 86 | total_timesteps 1765.
Path 87 | total_timesteps 1788.
Path 88 | total_timesteps 1801.
Path 89 | total_timesteps 1812.
Path 90 | total_timesteps 1824.
Path 91 | total_timesteps 1834.
Path 92 | total_timesteps 1863.
Path 93 | total_timesteps 1881.
Path 94 | total_timesteps 1915.
Path 95 | total_timesteps 1924.
Path 96 | total_timesteps 1931.
Path 97 | total_timesteps 1965.
Path 98 | total_timesteps 1981.
Path 99 | total_timesteps 2001.
Path 100 | total_timesteps 2033.
Path 101 | total_timesteps 2055.
Path 102 | total_timesteps 2073.
Path 103 | total_timesteps 2092.
Path 104 | total_timesteps 2110.
Path 105 | total_timesteps 2136.
Path 106 | total_timesteps 2155.
Path 107 | total_timesteps 2175.
Path 108 | total_timesteps 2207.
Path 109 | total_timesteps 2231.
Path 110 | total_timesteps 2240.
Path 111 | total_timesteps 2252.
Path 112 | total_timesteps 2267.
Path 113 | total_timesteps 2298.
Path 114 | total_timesteps 2322.
Path 115 | total_timesteps 2350.
Path 116 | total_timesteps 2371.
Path 117 | total_timesteps 2389.
Path 118 | total_timesteps 2401.
Path 119 | total_timesteps 2417.
Path 120 | total_timesteps 2427.
Path 121 | total_timesteps 2442.
Path 122 | total_timesteps 2460.
Path 123 | total_timesteps 2487.
Path 124 | total_timesteps 2513.
Path 125 | total_timesteps 2527.
Path 126 | total_timesteps 2548.
Path 127 | total_timesteps 2566.
Path 128 | total_timesteps 2579.
Path 129 | total_timesteps 2595.
Path 130 | total_timesteps 2642.
Path 131 | total_timesteps 2657.
Path 132 | total_timesteps 2685.
Path 133 | total_timesteps 2704.
Path 134 | total_timesteps 2718.
Path 135 | total_timesteps 2728.
Path 136 | total_timesteps 2742.
Path 137 | total_timesteps 2758.
Path 138 | total_timesteps 2788.
Path 139 | total_timesteps 2802.
Path 140 | total_timesteps 2834.
Path 141 | total_timesteps 2854.
Path 142 | total_timesteps 2862.
Path 143 | total_timesteps 2873.
Path 144 | total_timesteps 2884.
Path 145 | total_timesteps 2897.
Path 146 | total_timesteps 2919.
Path 147 | total_timesteps 2953.
Path 148 | total_timesteps 2977.
Path 149 | total_timesteps 3008.
Path 150 | total_timesteps 3033.
Path 151 | total_timesteps 3046.
Path 152 | total_timesteps 3073.
Path 153 | total_timesteps 3094.
Path 154 | total_timesteps 3105.
Path 155 | total_timesteps 3141.
Path 156 | total_timesteps 3154.
Path 157 | total_timesteps 3180.
Path 158 | total_timesteps 3208.
Path 159 | total_timesteps 3229.
Path 160 | total_timesteps 3245.
Path 161 | total_timesteps 3259.
Path 162 | total_timesteps 3280.
Path 163 | total_timesteps 3325.
Path 164 | total_timesteps 3349.
Path 165 | total_timesteps 3358.
Path 166 | total_timesteps 3376.
Path 167 | total_timesteps 3391.
Path 168 | total_timesteps 3431.
Path 169 | total_timesteps 3455.
Path 170 | total_timesteps 3470.
Path 171 | total_timesteps 3485.
Path 172 | total_timesteps 3492.
Path 173 | total_timesteps 3501.
Path 174 | total_timesteps 3521.
Path 175 | total_timesteps 3541.
Path 176 | total_timesteps 3559.
Path 177 | total_timesteps 3580.
Path 178 | total_timesteps 3609.
Path 179 | total_timesteps 3630.
Path 180 | total_timesteps 3667.
Path 181 | total_timesteps 3685.
Path 182 | total_timesteps 3705.
Path 183 | total_timesteps 3716.
Path 184 | total_timesteps 3735.
Path 185 | total_timesteps 3753.
Path 186 | total_timesteps 3766.
Path 187 | total_timesteps 3777.
Path 188 | total_timesteps 3789.
Path 189 | total_timesteps 3812.
Path 190 | total_timesteps 3844.
Path 191 | total_timesteps 3865.
Path 192 | total_timesteps 3901.
Path 193 | total_timesteps 3925.
Path 194 | total_timesteps 3959.
Path 195 | total_timesteps 3971.
Path 196 | total_timesteps 3983.
Path 197 | total_timesteps 3992.
Path 198 | total_timesteps 4020.
Path 199 | total_timesteps 4050.
Path 200 | total_timesteps 4068.
Path 201 | total_timesteps 4086.
Path 202 | total_timesteps 4113.
Path 203 | total_timesteps 4123.
Path 204 | total_timesteps 4140.
Path 205 | total_timesteps 4174.
Path 206 | total_timesteps 4189.
Path 207 | total_timesteps 4211.
Path 208 | total_timesteps 4238.
Path 209 | total_timesteps 4267.
Path 210 | total_timesteps 4296.
Path 211 | total_timesteps 4321.
Path 212 | total_timesteps 4344.
Path 213 | total_timesteps 4361.
Path 214 | total_timesteps 4405.
Path 215 | total_timesteps 4415.
Path 216 | total_timesteps 4434.
Path 217 | total_timesteps 4454.
Path 218 | total_timesteps 4466.
Path 219 | total_timesteps 4496.
Path 220 | total_timesteps 4530.
Path 221 | total_timesteps 4549.
Path 222 | total_timesteps 4570.
Path 223 | total_timesteps 4604.
Path 224 | total_timesteps 4618.
Path 225 | total_timesteps 4628.
Path 226 | total_timesteps 4642.
Path 227 | total_timesteps 4695.
Path 228 | total_timesteps 4706.
Path 229 | total_timesteps 4736.
Path 230 | total_timesteps 4754.
Path 231 | total_timesteps 4770.
Path 232 | total_timesteps 4789.
Path 233 | total_timesteps 4805.
Path 234 | total_timesteps 4818.
Path 235 | total_timesteps 4826.
Path 236 | total_timesteps 4837.
Path 237 | total_timesteps 4864.
Path 238 | total_timesteps 4884.
Path 239 | total_timesteps 4896.
Path 240 | total_timesteps 4910.
Path 241 | total_timesteps 4921.
Path 242 | total_timesteps 4946.
Path 243 | total_timesteps 4964.
Path 244 | total_timesteps 4989.
Path 245 | total_timesteps 5016.
Path 246 | total_timesteps 5036.
Path 247 | total_timesteps 5048.
Path 248 | total_timesteps 5060.
Path 249 | total_timesteps 5088.
Path 250 | total_timesteps 5102.
Path 251 | total_timesteps 5113.
Path 252 | total_timesteps 5133.
Path 253 | total_timesteps 5145.
Path 254 | total_timesteps 5160.
Path 255 | total_timesteps 5172.
Path 256 | total_timesteps 5188.
Path 257 | total_timesteps 5199.
Path 258 | total_timesteps 5214.
Path 259 | total_timesteps 5235.
Path 260 | total_timesteps 5258.
Path 261 | total_timesteps 5267.
Path 262 | total_timesteps 5285.
Path 263 | total_timesteps 5294.
Path 264 | total_timesteps 5310.
Path 265 | total_timesteps 5334.
Path 266 | total_timesteps 5356.
Path 267 | total_timesteps 5374.
Path 268 | total_timesteps 5410.
Path 269 | total_timesteps 5417.
Path 270 | total_timesteps 5429.
Path 271 | total_timesteps 5462.
Path 272 | total_timesteps 5491.
Path 273 | total_timesteps 5511.
Path 274 | total_timesteps 5521.
Path 275 | total_timesteps 5537.
Path 276 | total_timesteps 5554.
Path 277 | total_timesteps 5565.
Path 278 | total_timesteps 5577.
Path 279 | total_timesteps 5597.
Path 280 | total_timesteps 5614.
Path 281 | total_timesteps 5631.
Path 282 | total_timesteps 5644.
Path 283 | total_timesteps 5676.
Path 284 | total_timesteps 5685.
Path 285 | total_timesteps 5708.
Path 286 | total_timesteps 5740.
Path 287 | total_timesteps 5765.
Path 288 | total_timesteps 5780.
Path 289 | total_timesteps 5790.
Path 290 | total_timesteps 5821.
Path 291 | total_timesteps 5830.
Path 292 | total_timesteps 5841.
Path 293 | total_timesteps 5859.
Path 294 | total_timesteps 5888.
Path 295 | total_timesteps 5895.
Path 296 | total_timesteps 5907.
Path 297 | total_timesteps 5930.
Path 298 | total_timesteps 5945.
Path 299 | total_timesteps 5964.
Path 300 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.13    |
| Iteration     | 12       |
| MaximumReturn | 11.3     |
| MinimumReturn | -19.8    |
| TotalSamples  | 56112    |
----------------------------
itr #13 | 
Fitting dynamics.
Validation loss = 0.005689834710210562
Validation loss = 0.005336552858352661
Validation loss = 0.0051693604327738285
Validation loss = 0.005964138079434633
Validation loss = 0.005006118677556515
Validation loss = 0.005108089651912451
Validation loss = 0.005121198017150164
Validation loss = 0.005079881753772497
Validation loss = 0.005323447287082672
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 58.
Path 3 | total_timesteps 87.
Path 4 | total_timesteps 110.
Path 5 | total_timesteps 137.
Path 6 | total_timesteps 157.
Path 7 | total_timesteps 166.
Path 8 | total_timesteps 179.
Path 9 | total_timesteps 189.
Path 10 | total_timesteps 205.
Path 11 | total_timesteps 222.
Path 12 | total_timesteps 242.
Path 13 | total_timesteps 258.
Path 14 | total_timesteps 284.
Path 15 | total_timesteps 305.
Path 16 | total_timesteps 320.
Path 17 | total_timesteps 348.
Path 18 | total_timesteps 367.
Path 19 | total_timesteps 394.
Path 20 | total_timesteps 411.
Path 21 | total_timesteps 421.
Path 22 | total_timesteps 439.
Path 23 | total_timesteps 464.
Path 24 | total_timesteps 485.
Path 25 | total_timesteps 503.
Path 26 | total_timesteps 525.
Path 27 | total_timesteps 544.
Path 28 | total_timesteps 558.
Path 29 | total_timesteps 575.
Path 30 | total_timesteps 598.
Path 31 | total_timesteps 615.
Path 32 | total_timesteps 630.
Path 33 | total_timesteps 640.
Path 34 | total_timesteps 655.
Path 35 | total_timesteps 675.
Path 36 | total_timesteps 713.
Path 37 | total_timesteps 724.
Path 38 | total_timesteps 739.
Path 39 | total_timesteps 757.
Path 40 | total_timesteps 781.
Path 41 | total_timesteps 792.
Path 42 | total_timesteps 818.
Path 43 | total_timesteps 833.
Path 44 | total_timesteps 847.
Path 45 | total_timesteps 875.
Path 46 | total_timesteps 887.
Path 47 | total_timesteps 904.
Path 48 | total_timesteps 917.
Path 49 | total_timesteps 930.
Path 50 | total_timesteps 940.
Path 51 | total_timesteps 954.
Path 52 | total_timesteps 966.
Path 53 | total_timesteps 984.
Path 54 | total_timesteps 991.
Path 55 | total_timesteps 1012.
Path 56 | total_timesteps 1023.
Path 57 | total_timesteps 1050.
Path 58 | total_timesteps 1077.
Path 59 | total_timesteps 1093.
Path 60 | total_timesteps 1112.
Path 61 | total_timesteps 1147.
Path 62 | total_timesteps 1166.
Path 63 | total_timesteps 1184.
Path 64 | total_timesteps 1198.
Path 65 | total_timesteps 1216.
Path 66 | total_timesteps 1250.
Path 67 | total_timesteps 1265.
Path 68 | total_timesteps 1285.
Path 69 | total_timesteps 1296.
Path 70 | total_timesteps 1311.
Path 71 | total_timesteps 1327.
Path 72 | total_timesteps 1344.
Path 73 | total_timesteps 1372.
Path 74 | total_timesteps 1393.
Path 75 | total_timesteps 1412.
Path 76 | total_timesteps 1425.
Path 77 | total_timesteps 1447.
Path 78 | total_timesteps 1463.
Path 79 | total_timesteps 1474.
Path 80 | total_timesteps 1508.
Path 81 | total_timesteps 1545.
Path 82 | total_timesteps 1582.
Path 83 | total_timesteps 1594.
Path 84 | total_timesteps 1616.
Path 85 | total_timesteps 1634.
Path 86 | total_timesteps 1651.
Path 87 | total_timesteps 1664.
Path 88 | total_timesteps 1681.
Path 89 | total_timesteps 1708.
Path 90 | total_timesteps 1737.
Path 91 | total_timesteps 1761.
Path 92 | total_timesteps 1779.
Path 93 | total_timesteps 1811.
Path 94 | total_timesteps 1820.
Path 95 | total_timesteps 1829.
Path 96 | total_timesteps 1911.
Path 97 | total_timesteps 1921.
Path 98 | total_timesteps 1937.
Path 99 | total_timesteps 1954.
Path 100 | total_timesteps 1963.
Path 101 | total_timesteps 2000.
Path 102 | total_timesteps 2011.
Path 103 | total_timesteps 2019.
Path 104 | total_timesteps 2038.
Path 105 | total_timesteps 2050.
Path 106 | total_timesteps 2063.
Path 107 | total_timesteps 2091.
Path 108 | total_timesteps 2116.
Path 109 | total_timesteps 2132.
Path 110 | total_timesteps 2155.
Path 111 | total_timesteps 2182.
Path 112 | total_timesteps 2199.
Path 113 | total_timesteps 2211.
Path 114 | total_timesteps 2223.
Path 115 | total_timesteps 2249.
Path 116 | total_timesteps 2265.
Path 117 | total_timesteps 2288.
Path 118 | total_timesteps 2303.
Path 119 | total_timesteps 2330.
Path 120 | total_timesteps 2341.
Path 121 | total_timesteps 2353.
Path 122 | total_timesteps 2383.
Path 123 | total_timesteps 2394.
Path 124 | total_timesteps 2403.
Path 125 | total_timesteps 2415.
Path 126 | total_timesteps 2431.
Path 127 | total_timesteps 2452.
Path 128 | total_timesteps 2467.
Path 129 | total_timesteps 2500.
Path 130 | total_timesteps 2518.
Path 131 | total_timesteps 2540.
Path 132 | total_timesteps 2554.
Path 133 | total_timesteps 2576.
Path 134 | total_timesteps 2592.
Path 135 | total_timesteps 2606.
Path 136 | total_timesteps 2625.
Path 137 | total_timesteps 2672.
Path 138 | total_timesteps 2684.
Path 139 | total_timesteps 2698.
Path 140 | total_timesteps 2715.
Path 141 | total_timesteps 2749.
Path 142 | total_timesteps 2763.
Path 143 | total_timesteps 2783.
Path 144 | total_timesteps 2793.
Path 145 | total_timesteps 2804.
Path 146 | total_timesteps 2816.
Path 147 | total_timesteps 2836.
Path 148 | total_timesteps 2860.
Path 149 | total_timesteps 2880.
Path 150 | total_timesteps 2901.
Path 151 | total_timesteps 2909.
Path 152 | total_timesteps 2934.
Path 153 | total_timesteps 2943.
Path 154 | total_timesteps 2956.
Path 155 | total_timesteps 2970.
Path 156 | total_timesteps 2989.
Path 157 | total_timesteps 3003.
Path 158 | total_timesteps 3032.
Path 159 | total_timesteps 3042.
Path 160 | total_timesteps 3063.
Path 161 | total_timesteps 3090.
Path 162 | total_timesteps 3114.
Path 163 | total_timesteps 3128.
Path 164 | total_timesteps 3152.
Path 165 | total_timesteps 3171.
Path 166 | total_timesteps 3181.
Path 167 | total_timesteps 3210.
Path 168 | total_timesteps 3230.
Path 169 | total_timesteps 3239.
Path 170 | total_timesteps 3265.
Path 171 | total_timesteps 3285.
Path 172 | total_timesteps 3296.
Path 173 | total_timesteps 3314.
Path 174 | total_timesteps 3338.
Path 175 | total_timesteps 3361.
Path 176 | total_timesteps 3371.
Path 177 | total_timesteps 3391.
Path 178 | total_timesteps 3410.
Path 179 | total_timesteps 3441.
Path 180 | total_timesteps 3457.
Path 181 | total_timesteps 3499.
Path 182 | total_timesteps 3518.
Path 183 | total_timesteps 3527.
Path 184 | total_timesteps 3535.
Path 185 | total_timesteps 3568.
Path 186 | total_timesteps 3588.
Path 187 | total_timesteps 3607.
Path 188 | total_timesteps 3624.
Path 189 | total_timesteps 3658.
Path 190 | total_timesteps 3704.
Path 191 | total_timesteps 3714.
Path 192 | total_timesteps 3722.
Path 193 | total_timesteps 3760.
Path 194 | total_timesteps 3787.
Path 195 | total_timesteps 3806.
Path 196 | total_timesteps 3822.
Path 197 | total_timesteps 3839.
Path 198 | total_timesteps 3864.
Path 199 | total_timesteps 3881.
Path 200 | total_timesteps 3904.
Path 201 | total_timesteps 3930.
Path 202 | total_timesteps 3946.
Path 203 | total_timesteps 3961.
Path 204 | total_timesteps 3976.
Path 205 | total_timesteps 3995.
Path 206 | total_timesteps 4019.
Path 207 | total_timesteps 4048.
Path 208 | total_timesteps 4063.
Path 209 | total_timesteps 4080.
Path 210 | total_timesteps 4092.
Path 211 | total_timesteps 4129.
Path 212 | total_timesteps 4156.
Path 213 | total_timesteps 4182.
Path 214 | total_timesteps 4205.
Path 215 | total_timesteps 4214.
Path 216 | total_timesteps 4235.
Path 217 | total_timesteps 4252.
Path 218 | total_timesteps 4263.
Path 219 | total_timesteps 4281.
Path 220 | total_timesteps 4295.
Path 221 | total_timesteps 4324.
Path 222 | total_timesteps 4338.
Path 223 | total_timesteps 4348.
Path 224 | total_timesteps 4356.
Path 225 | total_timesteps 4383.
Path 226 | total_timesteps 4410.
Path 227 | total_timesteps 4426.
Path 228 | total_timesteps 4451.
Path 229 | total_timesteps 4475.
Path 230 | total_timesteps 4511.
Path 231 | total_timesteps 4526.
Path 232 | total_timesteps 4544.
Path 233 | total_timesteps 4558.
Path 234 | total_timesteps 4574.
Path 235 | total_timesteps 4588.
Path 236 | total_timesteps 4609.
Path 237 | total_timesteps 4630.
Path 238 | total_timesteps 4654.
Path 239 | total_timesteps 4672.
Path 240 | total_timesteps 4709.
Path 241 | total_timesteps 4716.
Path 242 | total_timesteps 4747.
Path 243 | total_timesteps 4774.
Path 244 | total_timesteps 4807.
Path 245 | total_timesteps 4821.
Path 246 | total_timesteps 4842.
Path 247 | total_timesteps 4856.
Path 248 | total_timesteps 4879.
Path 249 | total_timesteps 4900.
Path 250 | total_timesteps 4907.
Path 251 | total_timesteps 4918.
Path 252 | total_timesteps 4929.
Path 253 | total_timesteps 4947.
Path 254 | total_timesteps 4960.
Path 255 | total_timesteps 4985.
Path 256 | total_timesteps 5000.
Path 257 | total_timesteps 5021.
Path 258 | total_timesteps 5051.
Path 259 | total_timesteps 5072.
Path 260 | total_timesteps 5090.
Path 261 | total_timesteps 5113.
Path 262 | total_timesteps 5128.
Path 263 | total_timesteps 5139.
Path 264 | total_timesteps 5155.
Path 265 | total_timesteps 5179.
Path 266 | total_timesteps 5191.
Path 267 | total_timesteps 5200.
Path 268 | total_timesteps 5216.
Path 269 | total_timesteps 5227.
Path 270 | total_timesteps 5243.
Path 271 | total_timesteps 5273.
Path 272 | total_timesteps 5285.
Path 273 | total_timesteps 5304.
Path 274 | total_timesteps 5314.
Path 275 | total_timesteps 5329.
Path 276 | total_timesteps 5339.
Path 277 | total_timesteps 5371.
Path 278 | total_timesteps 5380.
Path 279 | total_timesteps 5413.
Path 280 | total_timesteps 5430.
Path 281 | total_timesteps 5457.
Path 282 | total_timesteps 5470.
Path 283 | total_timesteps 5491.
Path 284 | total_timesteps 5503.
Path 285 | total_timesteps 5536.
Path 286 | total_timesteps 5602.
Path 287 | total_timesteps 5612.
Path 288 | total_timesteps 5625.
Path 289 | total_timesteps 5641.
Path 290 | total_timesteps 5652.
Path 291 | total_timesteps 5692.
Path 292 | total_timesteps 5709.
Path 293 | total_timesteps 5718.
Path 294 | total_timesteps 5733.
Path 295 | total_timesteps 5747.
Path 296 | total_timesteps 5761.
Path 297 | total_timesteps 5781.
Path 298 | total_timesteps 5791.
Path 299 | total_timesteps 5808.
Path 300 | total_timesteps 5828.
Path 301 | total_timesteps 5842.
Path 302 | total_timesteps 5856.
Path 303 | total_timesteps 5863.
Path 304 | total_timesteps 5880.
Path 305 | total_timesteps 5899.
Path 306 | total_timesteps 5915.
Path 307 | total_timesteps 5929.
Path 308 | total_timesteps 5947.
Path 309 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.53    |
| Iteration     | 13       |
| MaximumReturn | 20.1     |
| MinimumReturn | -17.5    |
| TotalSamples  | 60124    |
----------------------------
itr #14 | 
Fitting dynamics.
Validation loss = 0.0071031819097697735
Validation loss = 0.005279282107949257
Validation loss = 0.005611109547317028
Validation loss = 0.004765226971358061
Validation loss = 0.005223637446761131
Validation loss = 0.005339448805898428
Validation loss = 0.004929011221975088
Validation loss = 0.005023518111556768
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 35.
Path 3 | total_timesteps 51.
Path 4 | total_timesteps 81.
Path 5 | total_timesteps 90.
Path 6 | total_timesteps 102.
Path 7 | total_timesteps 111.
Path 8 | total_timesteps 144.
Path 9 | total_timesteps 157.
Path 10 | total_timesteps 173.
Path 11 | total_timesteps 197.
Path 12 | total_timesteps 222.
Path 13 | total_timesteps 240.
Path 14 | total_timesteps 251.
Path 15 | total_timesteps 270.
Path 16 | total_timesteps 288.
Path 17 | total_timesteps 302.
Path 18 | total_timesteps 319.
Path 19 | total_timesteps 341.
Path 20 | total_timesteps 367.
Path 21 | total_timesteps 391.
Path 22 | total_timesteps 411.
Path 23 | total_timesteps 441.
Path 24 | total_timesteps 455.
Path 25 | total_timesteps 483.
Path 26 | total_timesteps 531.
Path 27 | total_timesteps 543.
Path 28 | total_timesteps 555.
Path 29 | total_timesteps 564.
Path 30 | total_timesteps 587.
Path 31 | total_timesteps 598.
Path 32 | total_timesteps 606.
Path 33 | total_timesteps 626.
Path 34 | total_timesteps 652.
Path 35 | total_timesteps 668.
Path 36 | total_timesteps 697.
Path 37 | total_timesteps 717.
Path 38 | total_timesteps 732.
Path 39 | total_timesteps 747.
Path 40 | total_timesteps 763.
Path 41 | total_timesteps 784.
Path 42 | total_timesteps 799.
Path 43 | total_timesteps 816.
Path 44 | total_timesteps 829.
Path 45 | total_timesteps 854.
Path 46 | total_timesteps 881.
Path 47 | total_timesteps 900.
Path 48 | total_timesteps 947.
Path 49 | total_timesteps 958.
Path 50 | total_timesteps 980.
Path 51 | total_timesteps 994.
Path 52 | total_timesteps 1016.
Path 53 | total_timesteps 1045.
Path 54 | total_timesteps 1063.
Path 55 | total_timesteps 1082.
Path 56 | total_timesteps 1101.
Path 57 | total_timesteps 1133.
Path 58 | total_timesteps 1146.
Path 59 | total_timesteps 1169.
Path 60 | total_timesteps 1194.
Path 61 | total_timesteps 1206.
Path 62 | total_timesteps 1241.
Path 63 | total_timesteps 1257.
Path 64 | total_timesteps 1281.
Path 65 | total_timesteps 1311.
Path 66 | total_timesteps 1331.
Path 67 | total_timesteps 1362.
Path 68 | total_timesteps 1377.
Path 69 | total_timesteps 1409.
Path 70 | total_timesteps 1419.
Path 71 | total_timesteps 1442.
Path 72 | total_timesteps 1482.
Path 73 | total_timesteps 1502.
Path 74 | total_timesteps 1524.
Path 75 | total_timesteps 1542.
Path 76 | total_timesteps 1561.
Path 77 | total_timesteps 1587.
Path 78 | total_timesteps 1620.
Path 79 | total_timesteps 1653.
Path 80 | total_timesteps 1663.
Path 81 | total_timesteps 1684.
Path 82 | total_timesteps 1705.
Path 83 | total_timesteps 1718.
Path 84 | total_timesteps 1738.
Path 85 | total_timesteps 1768.
Path 86 | total_timesteps 1805.
Path 87 | total_timesteps 1816.
Path 88 | total_timesteps 1847.
Path 89 | total_timesteps 1877.
Path 90 | total_timesteps 1905.
Path 91 | total_timesteps 1921.
Path 92 | total_timesteps 1967.
Path 93 | total_timesteps 1991.
Path 94 | total_timesteps 2009.
Path 95 | total_timesteps 2021.
Path 96 | total_timesteps 2037.
Path 97 | total_timesteps 2062.
Path 98 | total_timesteps 2077.
Path 99 | total_timesteps 2090.
Path 100 | total_timesteps 2114.
Path 101 | total_timesteps 2126.
Path 102 | total_timesteps 2146.
Path 103 | total_timesteps 2164.
Path 104 | total_timesteps 2187.
Path 105 | total_timesteps 2203.
Path 106 | total_timesteps 2215.
Path 107 | total_timesteps 2243.
Path 108 | total_timesteps 2263.
Path 109 | total_timesteps 2275.
Path 110 | total_timesteps 2288.
Path 111 | total_timesteps 2305.
Path 112 | total_timesteps 2327.
Path 113 | total_timesteps 2353.
Path 114 | total_timesteps 2382.
Path 115 | total_timesteps 2415.
Path 116 | total_timesteps 2430.
Path 117 | total_timesteps 2440.
Path 118 | total_timesteps 2459.
Path 119 | total_timesteps 2473.
Path 120 | total_timesteps 2507.
Path 121 | total_timesteps 2521.
Path 122 | total_timesteps 2549.
Path 123 | total_timesteps 2566.
Path 124 | total_timesteps 2601.
Path 125 | total_timesteps 2625.
Path 126 | total_timesteps 2633.
Path 127 | total_timesteps 2650.
Path 128 | total_timesteps 2672.
Path 129 | total_timesteps 2689.
Path 130 | total_timesteps 2709.
Path 131 | total_timesteps 2740.
Path 132 | total_timesteps 2757.
Path 133 | total_timesteps 2775.
Path 134 | total_timesteps 2813.
Path 135 | total_timesteps 2829.
Path 136 | total_timesteps 2870.
Path 137 | total_timesteps 2895.
Path 138 | total_timesteps 2922.
Path 139 | total_timesteps 2939.
Path 140 | total_timesteps 2960.
Path 141 | total_timesteps 2971.
Path 142 | total_timesteps 2987.
Path 143 | total_timesteps 3004.
Path 144 | total_timesteps 3014.
Path 145 | total_timesteps 3041.
Path 146 | total_timesteps 3067.
Path 147 | total_timesteps 3083.
Path 148 | total_timesteps 3122.
Path 149 | total_timesteps 3140.
Path 150 | total_timesteps 3153.
Path 151 | total_timesteps 3192.
Path 152 | total_timesteps 3215.
Path 153 | total_timesteps 3255.
Path 154 | total_timesteps 3282.
Path 155 | total_timesteps 3308.
Path 156 | total_timesteps 3339.
Path 157 | total_timesteps 3379.
Path 158 | total_timesteps 3403.
Path 159 | total_timesteps 3440.
Path 160 | total_timesteps 3451.
Path 161 | total_timesteps 3465.
Path 162 | total_timesteps 3485.
Path 163 | total_timesteps 3523.
Path 164 | total_timesteps 3551.
Path 165 | total_timesteps 3576.
Path 166 | total_timesteps 3592.
Path 167 | total_timesteps 3604.
Path 168 | total_timesteps 3615.
Path 169 | total_timesteps 3640.
Path 170 | total_timesteps 3653.
Path 171 | total_timesteps 3667.
Path 172 | total_timesteps 3692.
Path 173 | total_timesteps 3720.
Path 174 | total_timesteps 3748.
Path 175 | total_timesteps 3761.
Path 176 | total_timesteps 3774.
Path 177 | total_timesteps 3799.
Path 178 | total_timesteps 3833.
Path 179 | total_timesteps 3846.
Path 180 | total_timesteps 3866.
Path 181 | total_timesteps 3883.
Path 182 | total_timesteps 3898.
Path 183 | total_timesteps 3922.
Path 184 | total_timesteps 3935.
Path 185 | total_timesteps 3952.
Path 186 | total_timesteps 3961.
Path 187 | total_timesteps 3985.
Path 188 | total_timesteps 4010.
Path 189 | total_timesteps 4035.
Path 190 | total_timesteps 4055.
Path 191 | total_timesteps 4090.
Path 192 | total_timesteps 4123.
Path 193 | total_timesteps 4146.
Path 194 | total_timesteps 4167.
Path 195 | total_timesteps 4183.
Path 196 | total_timesteps 4230.
Path 197 | total_timesteps 4255.
Path 198 | total_timesteps 4276.
Path 199 | total_timesteps 4293.
Path 200 | total_timesteps 4307.
Path 201 | total_timesteps 4328.
Path 202 | total_timesteps 4365.
Path 203 | total_timesteps 4379.
Path 204 | total_timesteps 4392.
Path 205 | total_timesteps 4406.
Path 206 | total_timesteps 4425.
Path 207 | total_timesteps 4437.
Path 208 | total_timesteps 4461.
Path 209 | total_timesteps 4477.
Path 210 | total_timesteps 4501.
Path 211 | total_timesteps 4520.
Path 212 | total_timesteps 4539.
Path 213 | total_timesteps 4550.
Path 214 | total_timesteps 4565.
Path 215 | total_timesteps 4591.
Path 216 | total_timesteps 4600.
Path 217 | total_timesteps 4625.
Path 218 | total_timesteps 4647.
Path 219 | total_timesteps 4659.
Path 220 | total_timesteps 4677.
Path 221 | total_timesteps 4696.
Path 222 | total_timesteps 4704.
Path 223 | total_timesteps 4723.
Path 224 | total_timesteps 4741.
Path 225 | total_timesteps 4771.
Path 226 | total_timesteps 4795.
Path 227 | total_timesteps 4808.
Path 228 | total_timesteps 4819.
Path 229 | total_timesteps 4836.
Path 230 | total_timesteps 4845.
Path 231 | total_timesteps 4865.
Path 232 | total_timesteps 4886.
Path 233 | total_timesteps 4906.
Path 234 | total_timesteps 4947.
Path 235 | total_timesteps 4977.
Path 236 | total_timesteps 5021.
Path 237 | total_timesteps 5041.
Path 238 | total_timesteps 5056.
Path 239 | total_timesteps 5069.
Path 240 | total_timesteps 5083.
Path 241 | total_timesteps 5093.
Path 242 | total_timesteps 5104.
Path 243 | total_timesteps 5125.
Path 244 | total_timesteps 5149.
Path 245 | total_timesteps 5177.
Path 246 | total_timesteps 5200.
Path 247 | total_timesteps 5212.
Path 248 | total_timesteps 5237.
Path 249 | total_timesteps 5247.
Path 250 | total_timesteps 5263.
Path 251 | total_timesteps 5277.
Path 252 | total_timesteps 5301.
Path 253 | total_timesteps 5321.
Path 254 | total_timesteps 5336.
Path 255 | total_timesteps 5377.
Path 256 | total_timesteps 5419.
Path 257 | total_timesteps 5434.
Path 258 | total_timesteps 5446.
Path 259 | total_timesteps 5465.
Path 260 | total_timesteps 5478.
Path 261 | total_timesteps 5491.
Path 262 | total_timesteps 5504.
Path 263 | total_timesteps 5526.
Path 264 | total_timesteps 5557.
Path 265 | total_timesteps 5580.
Path 266 | total_timesteps 5592.
Path 267 | total_timesteps 5624.
Path 268 | total_timesteps 5645.
Path 269 | total_timesteps 5656.
Path 270 | total_timesteps 5667.
Path 271 | total_timesteps 5691.
Path 272 | total_timesteps 5730.
Path 273 | total_timesteps 5754.
Path 274 | total_timesteps 5768.
Path 275 | total_timesteps 5778.
Path 276 | total_timesteps 5798.
Path 277 | total_timesteps 5806.
Path 278 | total_timesteps 5834.
Path 279 | total_timesteps 5846.
Path 280 | total_timesteps 5856.
Path 281 | total_timesteps 5880.
Path 282 | total_timesteps 5890.
Path 283 | total_timesteps 5902.
Path 284 | total_timesteps 5930.
Path 285 | total_timesteps 5952.
Path 286 | total_timesteps 5985.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.14    |
| Iteration     | 14       |
| MaximumReturn | 25.9     |
| MinimumReturn | -19.3    |
| TotalSamples  | 64125    |
----------------------------
itr #15 | 
Fitting dynamics.
Validation loss = 0.005506974179297686
Validation loss = 0.004788890480995178
Validation loss = 0.004763883538544178
Validation loss = 0.004450383596122265
Validation loss = 0.004365459084510803
Validation loss = 0.004700379446148872
Validation loss = 0.004735809750854969
Validation loss = 0.004807820077985525
Validation loss = 0.0042932238429784775
Validation loss = 0.004462616518139839
Validation loss = 0.004373280797153711
Validation loss = 0.0045838188380002975
Validation loss = 0.004225487355142832
Validation loss = 0.004413413815200329
Validation loss = 0.0041452269069850445
Validation loss = 0.004829952027648687
Validation loss = 0.0043482230976223946
Validation loss = 0.004360724240541458
Validation loss = 0.004570582881569862
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 26.
Path 2 | total_timesteps 37.
Path 3 | total_timesteps 62.
Path 4 | total_timesteps 79.
Path 5 | total_timesteps 90.
Path 6 | total_timesteps 111.
Path 7 | total_timesteps 129.
Path 8 | total_timesteps 154.
Path 9 | total_timesteps 171.
Path 10 | total_timesteps 188.
Path 11 | total_timesteps 229.
Path 12 | total_timesteps 258.
Path 13 | total_timesteps 272.
Path 14 | total_timesteps 292.
Path 15 | total_timesteps 325.
Path 16 | total_timesteps 338.
Path 17 | total_timesteps 371.
Path 18 | total_timesteps 393.
Path 19 | total_timesteps 425.
Path 20 | total_timesteps 443.
Path 21 | total_timesteps 451.
Path 22 | total_timesteps 488.
Path 23 | total_timesteps 501.
Path 24 | total_timesteps 514.
Path 25 | total_timesteps 522.
Path 26 | total_timesteps 535.
Path 27 | total_timesteps 553.
Path 28 | total_timesteps 567.
Path 29 | total_timesteps 581.
Path 30 | total_timesteps 613.
Path 31 | total_timesteps 646.
Path 32 | total_timesteps 662.
Path 33 | total_timesteps 673.
Path 34 | total_timesteps 700.
Path 35 | total_timesteps 709.
Path 36 | total_timesteps 719.
Path 37 | total_timesteps 728.
Path 38 | total_timesteps 754.
Path 39 | total_timesteps 770.
Path 40 | total_timesteps 794.
Path 41 | total_timesteps 823.
Path 42 | total_timesteps 841.
Path 43 | total_timesteps 871.
Path 44 | total_timesteps 883.
Path 45 | total_timesteps 897.
Path 46 | total_timesteps 906.
Path 47 | total_timesteps 924.
Path 48 | total_timesteps 931.
Path 49 | total_timesteps 940.
Path 50 | total_timesteps 969.
Path 51 | total_timesteps 984.
Path 52 | total_timesteps 997.
Path 53 | total_timesteps 1033.
Path 54 | total_timesteps 1043.
Path 55 | total_timesteps 1074.
Path 56 | total_timesteps 1084.
Path 57 | total_timesteps 1097.
Path 58 | total_timesteps 1118.
Path 59 | total_timesteps 1137.
Path 60 | total_timesteps 1154.
Path 61 | total_timesteps 1170.
Path 62 | total_timesteps 1202.
Path 63 | total_timesteps 1232.
Path 64 | total_timesteps 1255.
Path 65 | total_timesteps 1276.
Path 66 | total_timesteps 1323.
Path 67 | total_timesteps 1338.
Path 68 | total_timesteps 1366.
Path 69 | total_timesteps 1396.
Path 70 | total_timesteps 1428.
Path 71 | total_timesteps 1441.
Path 72 | total_timesteps 1456.
Path 73 | total_timesteps 1485.
Path 74 | total_timesteps 1496.
Path 75 | total_timesteps 1509.
Path 76 | total_timesteps 1533.
Path 77 | total_timesteps 1556.
Path 78 | total_timesteps 1585.
Path 79 | total_timesteps 1603.
Path 80 | total_timesteps 1616.
Path 81 | total_timesteps 1633.
Path 82 | total_timesteps 1671.
Path 83 | total_timesteps 1704.
Path 84 | total_timesteps 1730.
Path 85 | total_timesteps 1767.
Path 86 | total_timesteps 1793.
Path 87 | total_timesteps 1807.
Path 88 | total_timesteps 1835.
Path 89 | total_timesteps 1856.
Path 90 | total_timesteps 1883.
Path 91 | total_timesteps 1892.
Path 92 | total_timesteps 1909.
Path 93 | total_timesteps 1923.
Path 94 | total_timesteps 1948.
Path 95 | total_timesteps 1968.
Path 96 | total_timesteps 1995.
Path 97 | total_timesteps 2002.
Path 98 | total_timesteps 2013.
Path 99 | total_timesteps 2038.
Path 100 | total_timesteps 2067.
Path 101 | total_timesteps 2080.
Path 102 | total_timesteps 2092.
Path 103 | total_timesteps 2103.
Path 104 | total_timesteps 2114.
Path 105 | total_timesteps 2131.
Path 106 | total_timesteps 2141.
Path 107 | total_timesteps 2150.
Path 108 | total_timesteps 2176.
Path 109 | total_timesteps 2191.
Path 110 | total_timesteps 2202.
Path 111 | total_timesteps 2222.
Path 112 | total_timesteps 2247.
Path 113 | total_timesteps 2309.
Path 114 | total_timesteps 2329.
Path 115 | total_timesteps 2342.
Path 116 | total_timesteps 2356.
Path 117 | total_timesteps 2376.
Path 118 | total_timesteps 2390.
Path 119 | total_timesteps 2405.
Path 120 | total_timesteps 2425.
Path 121 | total_timesteps 2448.
Path 122 | total_timesteps 2470.
Path 123 | total_timesteps 2481.
Path 124 | total_timesteps 2511.
Path 125 | total_timesteps 2540.
Path 126 | total_timesteps 2554.
Path 127 | total_timesteps 2575.
Path 128 | total_timesteps 2587.
Path 129 | total_timesteps 2611.
Path 130 | total_timesteps 2633.
Path 131 | total_timesteps 2651.
Path 132 | total_timesteps 2677.
Path 133 | total_timesteps 2691.
Path 134 | total_timesteps 2713.
Path 135 | total_timesteps 2731.
Path 136 | total_timesteps 2740.
Path 137 | total_timesteps 2759.
Path 138 | total_timesteps 2767.
Path 139 | total_timesteps 2782.
Path 140 | total_timesteps 2809.
Path 141 | total_timesteps 2843.
Path 142 | total_timesteps 2864.
Path 143 | total_timesteps 2894.
Path 144 | total_timesteps 2909.
Path 145 | total_timesteps 2929.
Path 146 | total_timesteps 2946.
Path 147 | total_timesteps 2955.
Path 148 | total_timesteps 2984.
Path 149 | total_timesteps 3010.
Path 150 | total_timesteps 3030.
Path 151 | total_timesteps 3043.
Path 152 | total_timesteps 3073.
Path 153 | total_timesteps 3088.
Path 154 | total_timesteps 3107.
Path 155 | total_timesteps 3127.
Path 156 | total_timesteps 3137.
Path 157 | total_timesteps 3162.
Path 158 | total_timesteps 3180.
Path 159 | total_timesteps 3207.
Path 160 | total_timesteps 3217.
Path 161 | total_timesteps 3237.
Path 162 | total_timesteps 3260.
Path 163 | total_timesteps 3287.
Path 164 | total_timesteps 3306.
Path 165 | total_timesteps 3318.
Path 166 | total_timesteps 3326.
Path 167 | total_timesteps 3357.
Path 168 | total_timesteps 3372.
Path 169 | total_timesteps 3401.
Path 170 | total_timesteps 3422.
Path 171 | total_timesteps 3452.
Path 172 | total_timesteps 3470.
Path 173 | total_timesteps 3482.
Path 174 | total_timesteps 3505.
Path 175 | total_timesteps 3529.
Path 176 | total_timesteps 3539.
Path 177 | total_timesteps 3566.
Path 178 | total_timesteps 3596.
Path 179 | total_timesteps 3630.
Path 180 | total_timesteps 3652.
Path 181 | total_timesteps 3661.
Path 182 | total_timesteps 3673.
Path 183 | total_timesteps 3715.
Path 184 | total_timesteps 3726.
Path 185 | total_timesteps 3757.
Path 186 | total_timesteps 3780.
Path 187 | total_timesteps 3795.
Path 188 | total_timesteps 3805.
Path 189 | total_timesteps 3817.
Path 190 | total_timesteps 3832.
Path 191 | total_timesteps 3855.
Path 192 | total_timesteps 3864.
Path 193 | total_timesteps 3875.
Path 194 | total_timesteps 3900.
Path 195 | total_timesteps 3924.
Path 196 | total_timesteps 3948.
Path 197 | total_timesteps 3971.
Path 198 | total_timesteps 3986.
Path 199 | total_timesteps 4004.
Path 200 | total_timesteps 4023.
Path 201 | total_timesteps 4058.
Path 202 | total_timesteps 4071.
Path 203 | total_timesteps 4088.
Path 204 | total_timesteps 4141.
Path 205 | total_timesteps 4166.
Path 206 | total_timesteps 4179.
Path 207 | total_timesteps 4208.
Path 208 | total_timesteps 4229.
Path 209 | total_timesteps 4256.
Path 210 | total_timesteps 4296.
Path 211 | total_timesteps 4311.
Path 212 | total_timesteps 4322.
Path 213 | total_timesteps 4338.
Path 214 | total_timesteps 4357.
Path 215 | total_timesteps 4386.
Path 216 | total_timesteps 4405.
Path 217 | total_timesteps 4416.
Path 218 | total_timesteps 4436.
Path 219 | total_timesteps 4461.
Path 220 | total_timesteps 4469.
Path 221 | total_timesteps 4486.
Path 222 | total_timesteps 4505.
Path 223 | total_timesteps 4519.
Path 224 | total_timesteps 4552.
Path 225 | total_timesteps 4589.
Path 226 | total_timesteps 4611.
Path 227 | total_timesteps 4626.
Path 228 | total_timesteps 4652.
Path 229 | total_timesteps 4665.
Path 230 | total_timesteps 4692.
Path 231 | total_timesteps 4707.
Path 232 | total_timesteps 4720.
Path 233 | total_timesteps 4730.
Path 234 | total_timesteps 4741.
Path 235 | total_timesteps 4750.
Path 236 | total_timesteps 4772.
Path 237 | total_timesteps 4789.
Path 238 | total_timesteps 4811.
Path 239 | total_timesteps 4839.
Path 240 | total_timesteps 4862.
Path 241 | total_timesteps 4874.
Path 242 | total_timesteps 4901.
Path 243 | total_timesteps 4926.
Path 244 | total_timesteps 4942.
Path 245 | total_timesteps 4964.
Path 246 | total_timesteps 4977.
Path 247 | total_timesteps 4986.
Path 248 | total_timesteps 5004.
Path 249 | total_timesteps 5017.
Path 250 | total_timesteps 5050.
Path 251 | total_timesteps 5070.
Path 252 | total_timesteps 5088.
Path 253 | total_timesteps 5100.
Path 254 | total_timesteps 5112.
Path 255 | total_timesteps 5133.
Path 256 | total_timesteps 5145.
Path 257 | total_timesteps 5159.
Path 258 | total_timesteps 5187.
Path 259 | total_timesteps 5210.
Path 260 | total_timesteps 5247.
Path 261 | total_timesteps 5265.
Path 262 | total_timesteps 5295.
Path 263 | total_timesteps 5304.
Path 264 | total_timesteps 5316.
Path 265 | total_timesteps 5333.
Path 266 | total_timesteps 5362.
Path 267 | total_timesteps 5375.
Path 268 | total_timesteps 5392.
Path 269 | total_timesteps 5411.
Path 270 | total_timesteps 5436.
Path 271 | total_timesteps 5443.
Path 272 | total_timesteps 5459.
Path 273 | total_timesteps 5482.
Path 274 | total_timesteps 5510.
Path 275 | total_timesteps 5529.
Path 276 | total_timesteps 5548.
Path 277 | total_timesteps 5567.
Path 278 | total_timesteps 5584.
Path 279 | total_timesteps 5603.
Path 280 | total_timesteps 5612.
Path 281 | total_timesteps 5637.
Path 282 | total_timesteps 5653.
Path 283 | total_timesteps 5673.
Path 284 | total_timesteps 5710.
Path 285 | total_timesteps 5731.
Path 286 | total_timesteps 5765.
Path 287 | total_timesteps 5775.
Path 288 | total_timesteps 5798.
Path 289 | total_timesteps 5815.
Path 290 | total_timesteps 5843.
Path 291 | total_timesteps 5857.
Path 292 | total_timesteps 5883.
Path 293 | total_timesteps 5916.
Path 294 | total_timesteps 5931.
Path 295 | total_timesteps 5948.
Path 296 | total_timesteps 5963.
Path 297 | total_timesteps 5976.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.49    |
| Iteration     | 15       |
| MaximumReturn | 18.6     |
| MinimumReturn | -18.8    |
| TotalSamples  | 68127    |
----------------------------
itr #16 | 
Fitting dynamics.
Validation loss = 0.004763116594403982
Validation loss = 0.004314207471907139
Validation loss = 0.004414576571434736
Validation loss = 0.003941268660128117
Validation loss = 0.004278162028640509
Validation loss = 0.00418885238468647
Validation loss = 0.0040485733188688755
Validation loss = 0.0042135827243328094
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 22.
Path 3 | total_timesteps 31.
Path 4 | total_timesteps 48.
Path 5 | total_timesteps 58.
Path 6 | total_timesteps 71.
Path 7 | total_timesteps 84.
Path 8 | total_timesteps 95.
Path 9 | total_timesteps 110.
Path 10 | total_timesteps 120.
Path 11 | total_timesteps 132.
Path 12 | total_timesteps 149.
Path 13 | total_timesteps 163.
Path 14 | total_timesteps 184.
Path 15 | total_timesteps 193.
Path 16 | total_timesteps 213.
Path 17 | total_timesteps 232.
Path 18 | total_timesteps 245.
Path 19 | total_timesteps 271.
Path 20 | total_timesteps 280.
Path 21 | total_timesteps 308.
Path 22 | total_timesteps 319.
Path 23 | total_timesteps 337.
Path 24 | total_timesteps 355.
Path 25 | total_timesteps 382.
Path 26 | total_timesteps 390.
Path 27 | total_timesteps 403.
Path 28 | total_timesteps 427.
Path 29 | total_timesteps 440.
Path 30 | total_timesteps 467.
Path 31 | total_timesteps 481.
Path 32 | total_timesteps 489.
Path 33 | total_timesteps 510.
Path 34 | total_timesteps 525.
Path 35 | total_timesteps 566.
Path 36 | total_timesteps 578.
Path 37 | total_timesteps 595.
Path 38 | total_timesteps 619.
Path 39 | total_timesteps 636.
Path 40 | total_timesteps 655.
Path 41 | total_timesteps 668.
Path 42 | total_timesteps 687.
Path 43 | total_timesteps 709.
Path 44 | total_timesteps 732.
Path 45 | total_timesteps 746.
Path 46 | total_timesteps 766.
Path 47 | total_timesteps 777.
Path 48 | total_timesteps 791.
Path 49 | total_timesteps 813.
Path 50 | total_timesteps 822.
Path 51 | total_timesteps 843.
Path 52 | total_timesteps 854.
Path 53 | total_timesteps 884.
Path 54 | total_timesteps 902.
Path 55 | total_timesteps 926.
Path 56 | total_timesteps 950.
Path 57 | total_timesteps 965.
Path 58 | total_timesteps 1028.
Path 59 | total_timesteps 1043.
Path 60 | total_timesteps 1066.
Path 61 | total_timesteps 1096.
Path 62 | total_timesteps 1120.
Path 63 | total_timesteps 1145.
Path 64 | total_timesteps 1155.
Path 65 | total_timesteps 1167.
Path 66 | total_timesteps 1181.
Path 67 | total_timesteps 1194.
Path 68 | total_timesteps 1220.
Path 69 | total_timesteps 1234.
Path 70 | total_timesteps 1262.
Path 71 | total_timesteps 1279.
Path 72 | total_timesteps 1292.
Path 73 | total_timesteps 1301.
Path 74 | total_timesteps 1319.
Path 75 | total_timesteps 1351.
Path 76 | total_timesteps 1363.
Path 77 | total_timesteps 1374.
Path 78 | total_timesteps 1397.
Path 79 | total_timesteps 1407.
Path 80 | total_timesteps 1431.
Path 81 | total_timesteps 1444.
Path 82 | total_timesteps 1466.
Path 83 | total_timesteps 1482.
Path 84 | total_timesteps 1507.
Path 85 | total_timesteps 1519.
Path 86 | total_timesteps 1529.
Path 87 | total_timesteps 1552.
Path 88 | total_timesteps 1571.
Path 89 | total_timesteps 1601.
Path 90 | total_timesteps 1617.
Path 91 | total_timesteps 1630.
Path 92 | total_timesteps 1644.
Path 93 | total_timesteps 1662.
Path 94 | total_timesteps 1673.
Path 95 | total_timesteps 1714.
Path 96 | total_timesteps 1727.
Path 97 | total_timesteps 1744.
Path 98 | total_timesteps 1753.
Path 99 | total_timesteps 1778.
Path 100 | total_timesteps 1805.
Path 101 | total_timesteps 1821.
Path 102 | total_timesteps 1831.
Path 103 | total_timesteps 1850.
Path 104 | total_timesteps 1879.
Path 105 | total_timesteps 1893.
Path 106 | total_timesteps 1912.
Path 107 | total_timesteps 1922.
Path 108 | total_timesteps 1947.
Path 109 | total_timesteps 1964.
Path 110 | total_timesteps 1986.
Path 111 | total_timesteps 2000.
Path 112 | total_timesteps 2010.
Path 113 | total_timesteps 2025.
Path 114 | total_timesteps 2054.
Path 115 | total_timesteps 2083.
Path 116 | total_timesteps 2091.
Path 117 | total_timesteps 2111.
Path 118 | total_timesteps 2130.
Path 119 | total_timesteps 2142.
Path 120 | total_timesteps 2157.
Path 121 | total_timesteps 2169.
Path 122 | total_timesteps 2188.
Path 123 | total_timesteps 2198.
Path 124 | total_timesteps 2222.
Path 125 | total_timesteps 2238.
Path 126 | total_timesteps 2265.
Path 127 | total_timesteps 2276.
Path 128 | total_timesteps 2292.
Path 129 | total_timesteps 2318.
Path 130 | total_timesteps 2344.
Path 131 | total_timesteps 2359.
Path 132 | total_timesteps 2384.
Path 133 | total_timesteps 2397.
Path 134 | total_timesteps 2418.
Path 135 | total_timesteps 2428.
Path 136 | total_timesteps 2438.
Path 137 | total_timesteps 2457.
Path 138 | total_timesteps 2471.
Path 139 | total_timesteps 2483.
Path 140 | total_timesteps 2502.
Path 141 | total_timesteps 2524.
Path 142 | total_timesteps 2545.
Path 143 | total_timesteps 2557.
Path 144 | total_timesteps 2580.
Path 145 | total_timesteps 2602.
Path 146 | total_timesteps 2612.
Path 147 | total_timesteps 2625.
Path 148 | total_timesteps 2654.
Path 149 | total_timesteps 2666.
Path 150 | total_timesteps 2677.
Path 151 | total_timesteps 2696.
Path 152 | total_timesteps 2716.
Path 153 | total_timesteps 2737.
Path 154 | total_timesteps 2755.
Path 155 | total_timesteps 2786.
Path 156 | total_timesteps 2802.
Path 157 | total_timesteps 2817.
Path 158 | total_timesteps 2844.
Path 159 | total_timesteps 2875.
Path 160 | total_timesteps 2907.
Path 161 | total_timesteps 2920.
Path 162 | total_timesteps 2931.
Path 163 | total_timesteps 2959.
Path 164 | total_timesteps 2977.
Path 165 | total_timesteps 2991.
Path 166 | total_timesteps 3011.
Path 167 | total_timesteps 3026.
Path 168 | total_timesteps 3038.
Path 169 | total_timesteps 3052.
Path 170 | total_timesteps 3065.
Path 171 | total_timesteps 3098.
Path 172 | total_timesteps 3115.
Path 173 | total_timesteps 3133.
Path 174 | total_timesteps 3143.
Path 175 | total_timesteps 3163.
Path 176 | total_timesteps 3175.
Path 177 | total_timesteps 3184.
Path 178 | total_timesteps 3210.
Path 179 | total_timesteps 3224.
Path 180 | total_timesteps 3243.
Path 181 | total_timesteps 3259.
Path 182 | total_timesteps 3272.
Path 183 | total_timesteps 3292.
Path 184 | total_timesteps 3306.
Path 185 | total_timesteps 3324.
Path 186 | total_timesteps 3352.
Path 187 | total_timesteps 3361.
Path 188 | total_timesteps 3380.
Path 189 | total_timesteps 3391.
Path 190 | total_timesteps 3408.
Path 191 | total_timesteps 3420.
Path 192 | total_timesteps 3440.
Path 193 | total_timesteps 3469.
Path 194 | total_timesteps 3502.
Path 195 | total_timesteps 3523.
Path 196 | total_timesteps 3531.
Path 197 | total_timesteps 3548.
Path 198 | total_timesteps 3562.
Path 199 | total_timesteps 3590.
Path 200 | total_timesteps 3600.
Path 201 | total_timesteps 3611.
Path 202 | total_timesteps 3651.
Path 203 | total_timesteps 3678.
Path 204 | total_timesteps 3712.
Path 205 | total_timesteps 3727.
Path 206 | total_timesteps 3745.
Path 207 | total_timesteps 3779.
Path 208 | total_timesteps 3789.
Path 209 | total_timesteps 3800.
Path 210 | total_timesteps 3831.
Path 211 | total_timesteps 3846.
Path 212 | total_timesteps 3861.
Path 213 | total_timesteps 3883.
Path 214 | total_timesteps 3899.
Path 215 | total_timesteps 3919.
Path 216 | total_timesteps 3927.
Path 217 | total_timesteps 3947.
Path 218 | total_timesteps 3978.
Path 219 | total_timesteps 4003.
Path 220 | total_timesteps 4012.
Path 221 | total_timesteps 4036.
Path 222 | total_timesteps 4046.
Path 223 | total_timesteps 4071.
Path 224 | total_timesteps 4080.
Path 225 | total_timesteps 4104.
Path 226 | total_timesteps 4117.
Path 227 | total_timesteps 4151.
Path 228 | total_timesteps 4164.
Path 229 | total_timesteps 4182.
Path 230 | total_timesteps 4213.
Path 231 | total_timesteps 4245.
Path 232 | total_timesteps 4276.
Path 233 | total_timesteps 4301.
Path 234 | total_timesteps 4311.
Path 235 | total_timesteps 4341.
Path 236 | total_timesteps 4358.
Path 237 | total_timesteps 4374.
Path 238 | total_timesteps 4388.
Path 239 | total_timesteps 4399.
Path 240 | total_timesteps 4415.
Path 241 | total_timesteps 4427.
Path 242 | total_timesteps 4437.
Path 243 | total_timesteps 4453.
Path 244 | total_timesteps 4467.
Path 245 | total_timesteps 4477.
Path 246 | total_timesteps 4490.
Path 247 | total_timesteps 4511.
Path 248 | total_timesteps 4532.
Path 249 | total_timesteps 4547.
Path 250 | total_timesteps 4558.
Path 251 | total_timesteps 4585.
Path 252 | total_timesteps 4600.
Path 253 | total_timesteps 4622.
Path 254 | total_timesteps 4666.
Path 255 | total_timesteps 4686.
Path 256 | total_timesteps 4705.
Path 257 | total_timesteps 4721.
Path 258 | total_timesteps 4748.
Path 259 | total_timesteps 4760.
Path 260 | total_timesteps 4770.
Path 261 | total_timesteps 4780.
Path 262 | total_timesteps 4793.
Path 263 | total_timesteps 4802.
Path 264 | total_timesteps 4820.
Path 265 | total_timesteps 4847.
Path 266 | total_timesteps 4867.
Path 267 | total_timesteps 4876.
Path 268 | total_timesteps 4889.
Path 269 | total_timesteps 4898.
Path 270 | total_timesteps 4910.
Path 271 | total_timesteps 4931.
Path 272 | total_timesteps 4943.
Path 273 | total_timesteps 4951.
Path 274 | total_timesteps 4974.
Path 275 | total_timesteps 4987.
Path 276 | total_timesteps 5007.
Path 277 | total_timesteps 5020.
Path 278 | total_timesteps 5041.
Path 279 | total_timesteps 5056.
Path 280 | total_timesteps 5068.
Path 281 | total_timesteps 5086.
Path 282 | total_timesteps 5104.
Path 283 | total_timesteps 5113.
Path 284 | total_timesteps 5125.
Path 285 | total_timesteps 5149.
Path 286 | total_timesteps 5162.
Path 287 | total_timesteps 5182.
Path 288 | total_timesteps 5204.
Path 289 | total_timesteps 5213.
Path 290 | total_timesteps 5227.
Path 291 | total_timesteps 5238.
Path 292 | total_timesteps 5248.
Path 293 | total_timesteps 5262.
Path 294 | total_timesteps 5272.
Path 295 | total_timesteps 5280.
Path 296 | total_timesteps 5294.
Path 297 | total_timesteps 5308.
Path 298 | total_timesteps 5315.
Path 299 | total_timesteps 5335.
Path 300 | total_timesteps 5348.
Path 301 | total_timesteps 5374.
Path 302 | total_timesteps 5391.
Path 303 | total_timesteps 5403.
Path 304 | total_timesteps 5429.
Path 305 | total_timesteps 5450.
Path 306 | total_timesteps 5475.
Path 307 | total_timesteps 5496.
Path 308 | total_timesteps 5511.
Path 309 | total_timesteps 5526.
Path 310 | total_timesteps 5552.
Path 311 | total_timesteps 5564.
Path 312 | total_timesteps 5579.
Path 313 | total_timesteps 5590.
Path 314 | total_timesteps 5607.
Path 315 | total_timesteps 5629.
Path 316 | total_timesteps 5656.
Path 317 | total_timesteps 5667.
Path 318 | total_timesteps 5697.
Path 319 | total_timesteps 5712.
Path 320 | total_timesteps 5735.
Path 321 | total_timesteps 5748.
Path 322 | total_timesteps 5769.
Path 323 | total_timesteps 5783.
Path 324 | total_timesteps 5804.
Path 325 | total_timesteps 5822.
Path 326 | total_timesteps 5838.
Path 327 | total_timesteps 5850.
Path 328 | total_timesteps 5864.
Path 329 | total_timesteps 5899.
Path 330 | total_timesteps 5931.
Path 331 | total_timesteps 5962.
Path 332 | total_timesteps 5975.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.74    |
| Iteration     | 16       |
| MaximumReturn | 9.16     |
| MinimumReturn | -18.9    |
| TotalSamples  | 72127    |
----------------------------
itr #17 | 
Fitting dynamics.
Validation loss = 0.004283109679818153
Validation loss = 0.004070731345564127
Validation loss = 0.0040731108747422695
Validation loss = 0.003993649501353502
Validation loss = 0.003640885930508375
Validation loss = 0.0038023111410439014
Validation loss = 0.0041726562194526196
Validation loss = 0.004305379465222359
Validation loss = 0.0037821887526661158
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 67.
Path 4 | total_timesteps 77.
Path 5 | total_timesteps 87.
Path 6 | total_timesteps 103.
Path 7 | total_timesteps 114.
Path 8 | total_timesteps 123.
Path 9 | total_timesteps 133.
Path 10 | total_timesteps 145.
Path 11 | total_timesteps 165.
Path 12 | total_timesteps 192.
Path 13 | total_timesteps 220.
Path 14 | total_timesteps 243.
Path 15 | total_timesteps 254.
Path 16 | total_timesteps 275.
Path 17 | total_timesteps 305.
Path 18 | total_timesteps 326.
Path 19 | total_timesteps 338.
Path 20 | total_timesteps 351.
Path 21 | total_timesteps 364.
Path 22 | total_timesteps 373.
Path 23 | total_timesteps 386.
Path 24 | total_timesteps 400.
Path 25 | total_timesteps 424.
Path 26 | total_timesteps 437.
Path 27 | total_timesteps 468.
Path 28 | total_timesteps 478.
Path 29 | total_timesteps 486.
Path 30 | total_timesteps 496.
Path 31 | total_timesteps 526.
Path 32 | total_timesteps 541.
Path 33 | total_timesteps 558.
Path 34 | total_timesteps 570.
Path 35 | total_timesteps 584.
Path 36 | total_timesteps 608.
Path 37 | total_timesteps 626.
Path 38 | total_timesteps 653.
Path 39 | total_timesteps 659.
Path 40 | total_timesteps 678.
Path 41 | total_timesteps 689.
Path 42 | total_timesteps 703.
Path 43 | total_timesteps 731.
Path 44 | total_timesteps 766.
Path 45 | total_timesteps 783.
Path 46 | total_timesteps 792.
Path 47 | total_timesteps 811.
Path 48 | total_timesteps 826.
Path 49 | total_timesteps 836.
Path 50 | total_timesteps 847.
Path 51 | total_timesteps 875.
Path 52 | total_timesteps 882.
Path 53 | total_timesteps 892.
Path 54 | total_timesteps 911.
Path 55 | total_timesteps 922.
Path 56 | total_timesteps 939.
Path 57 | total_timesteps 966.
Path 58 | total_timesteps 986.
Path 59 | total_timesteps 1015.
Path 60 | total_timesteps 1034.
Path 61 | total_timesteps 1051.
Path 62 | total_timesteps 1065.
Path 63 | total_timesteps 1091.
Path 64 | total_timesteps 1105.
Path 65 | total_timesteps 1117.
Path 66 | total_timesteps 1132.
Path 67 | total_timesteps 1146.
Path 68 | total_timesteps 1156.
Path 69 | total_timesteps 1168.
Path 70 | total_timesteps 1189.
Path 71 | total_timesteps 1203.
Path 72 | total_timesteps 1228.
Path 73 | total_timesteps 1245.
Path 74 | total_timesteps 1278.
Path 75 | total_timesteps 1310.
Path 76 | total_timesteps 1343.
Path 77 | total_timesteps 1354.
Path 78 | total_timesteps 1364.
Path 79 | total_timesteps 1395.
Path 80 | total_timesteps 1405.
Path 81 | total_timesteps 1417.
Path 82 | total_timesteps 1427.
Path 83 | total_timesteps 1449.
Path 84 | total_timesteps 1468.
Path 85 | total_timesteps 1489.
Path 86 | total_timesteps 1500.
Path 87 | total_timesteps 1523.
Path 88 | total_timesteps 1539.
Path 89 | total_timesteps 1566.
Path 90 | total_timesteps 1590.
Path 91 | total_timesteps 1599.
Path 92 | total_timesteps 1624.
Path 93 | total_timesteps 1637.
Path 94 | total_timesteps 1651.
Path 95 | total_timesteps 1668.
Path 96 | total_timesteps 1682.
Path 97 | total_timesteps 1696.
Path 98 | total_timesteps 1710.
Path 99 | total_timesteps 1724.
Path 100 | total_timesteps 1743.
Path 101 | total_timesteps 1761.
Path 102 | total_timesteps 1781.
Path 103 | total_timesteps 1804.
Path 104 | total_timesteps 1826.
Path 105 | total_timesteps 1874.
Path 106 | total_timesteps 1908.
Path 107 | total_timesteps 1925.
Path 108 | total_timesteps 1942.
Path 109 | total_timesteps 1956.
Path 110 | total_timesteps 1965.
Path 111 | total_timesteps 1994.
Path 112 | total_timesteps 2010.
Path 113 | total_timesteps 2017.
Path 114 | total_timesteps 2026.
Path 115 | total_timesteps 2052.
Path 116 | total_timesteps 2079.
Path 117 | total_timesteps 2117.
Path 118 | total_timesteps 2129.
Path 119 | total_timesteps 2146.
Path 120 | total_timesteps 2158.
Path 121 | total_timesteps 2168.
Path 122 | total_timesteps 2185.
Path 123 | total_timesteps 2196.
Path 124 | total_timesteps 2252.
Path 125 | total_timesteps 2274.
Path 126 | total_timesteps 2286.
Path 127 | total_timesteps 2310.
Path 128 | total_timesteps 2328.
Path 129 | total_timesteps 2338.
Path 130 | total_timesteps 2358.
Path 131 | total_timesteps 2388.
Path 132 | total_timesteps 2400.
Path 133 | total_timesteps 2424.
Path 134 | total_timesteps 2437.
Path 135 | total_timesteps 2448.
Path 136 | total_timesteps 2464.
Path 137 | total_timesteps 2481.
Path 138 | total_timesteps 2511.
Path 139 | total_timesteps 2547.
Path 140 | total_timesteps 2556.
Path 141 | total_timesteps 2570.
Path 142 | total_timesteps 2588.
Path 143 | total_timesteps 2600.
Path 144 | total_timesteps 2626.
Path 145 | total_timesteps 2636.
Path 146 | total_timesteps 2662.
Path 147 | total_timesteps 2674.
Path 148 | total_timesteps 2684.
Path 149 | total_timesteps 2701.
Path 150 | total_timesteps 2712.
Path 151 | total_timesteps 2729.
Path 152 | total_timesteps 2744.
Path 153 | total_timesteps 2761.
Path 154 | total_timesteps 2775.
Path 155 | total_timesteps 2790.
Path 156 | total_timesteps 2807.
Path 157 | total_timesteps 2817.
Path 158 | total_timesteps 2834.
Path 159 | total_timesteps 2864.
Path 160 | total_timesteps 2888.
Path 161 | total_timesteps 2907.
Path 162 | total_timesteps 2931.
Path 163 | total_timesteps 2952.
Path 164 | total_timesteps 2961.
Path 165 | total_timesteps 2969.
Path 166 | total_timesteps 2981.
Path 167 | total_timesteps 2999.
Path 168 | total_timesteps 3035.
Path 169 | total_timesteps 3051.
Path 170 | total_timesteps 3070.
Path 171 | total_timesteps 3081.
Path 172 | total_timesteps 3099.
Path 173 | total_timesteps 3131.
Path 174 | total_timesteps 3158.
Path 175 | total_timesteps 3192.
Path 176 | total_timesteps 3214.
Path 177 | total_timesteps 3231.
Path 178 | total_timesteps 3257.
Path 179 | total_timesteps 3275.
Path 180 | total_timesteps 3284.
Path 181 | total_timesteps 3296.
Path 182 | total_timesteps 3312.
Path 183 | total_timesteps 3327.
Path 184 | total_timesteps 3347.
Path 185 | total_timesteps 3375.
Path 186 | total_timesteps 3390.
Path 187 | total_timesteps 3422.
Path 188 | total_timesteps 3462.
Path 189 | total_timesteps 3474.
Path 190 | total_timesteps 3483.
Path 191 | total_timesteps 3496.
Path 192 | total_timesteps 3506.
Path 193 | total_timesteps 3528.
Path 194 | total_timesteps 3547.
Path 195 | total_timesteps 3562.
Path 196 | total_timesteps 3573.
Path 197 | total_timesteps 3586.
Path 198 | total_timesteps 3599.
Path 199 | total_timesteps 3623.
Path 200 | total_timesteps 3637.
Path 201 | total_timesteps 3651.
Path 202 | total_timesteps 3661.
Path 203 | total_timesteps 3669.
Path 204 | total_timesteps 3682.
Path 205 | total_timesteps 3691.
Path 206 | total_timesteps 3717.
Path 207 | total_timesteps 3741.
Path 208 | total_timesteps 3748.
Path 209 | total_timesteps 3765.
Path 210 | total_timesteps 3777.
Path 211 | total_timesteps 3787.
Path 212 | total_timesteps 3807.
Path 213 | total_timesteps 3822.
Path 214 | total_timesteps 3835.
Path 215 | total_timesteps 3845.
Path 216 | total_timesteps 3860.
Path 217 | total_timesteps 3891.
Path 218 | total_timesteps 3903.
Path 219 | total_timesteps 3927.
Path 220 | total_timesteps 3945.
Path 221 | total_timesteps 3955.
Path 222 | total_timesteps 3969.
Path 223 | total_timesteps 3986.
Path 224 | total_timesteps 3993.
Path 225 | total_timesteps 4006.
Path 226 | total_timesteps 4017.
Path 227 | total_timesteps 4032.
Path 228 | total_timesteps 4069.
Path 229 | total_timesteps 4078.
Path 230 | total_timesteps 4097.
Path 231 | total_timesteps 4124.
Path 232 | total_timesteps 4141.
Path 233 | total_timesteps 4156.
Path 234 | total_timesteps 4174.
Path 235 | total_timesteps 4192.
Path 236 | total_timesteps 4210.
Path 237 | total_timesteps 4229.
Path 238 | total_timesteps 4246.
Path 239 | total_timesteps 4253.
Path 240 | total_timesteps 4262.
Path 241 | total_timesteps 4278.
Path 242 | total_timesteps 4295.
Path 243 | total_timesteps 4307.
Path 244 | total_timesteps 4333.
Path 245 | total_timesteps 4350.
Path 246 | total_timesteps 4364.
Path 247 | total_timesteps 4388.
Path 248 | total_timesteps 4401.
Path 249 | total_timesteps 4410.
Path 250 | total_timesteps 4424.
Path 251 | total_timesteps 4442.
Path 252 | total_timesteps 4451.
Path 253 | total_timesteps 4468.
Path 254 | total_timesteps 4480.
Path 255 | total_timesteps 4491.
Path 256 | total_timesteps 4504.
Path 257 | total_timesteps 4529.
Path 258 | total_timesteps 4540.
Path 259 | total_timesteps 4550.
Path 260 | total_timesteps 4565.
Path 261 | total_timesteps 4575.
Path 262 | total_timesteps 4586.
Path 263 | total_timesteps 4607.
Path 264 | total_timesteps 4620.
Path 265 | total_timesteps 4649.
Path 266 | total_timesteps 4660.
Path 267 | total_timesteps 4700.
Path 268 | total_timesteps 4724.
Path 269 | total_timesteps 4739.
Path 270 | total_timesteps 4747.
Path 271 | total_timesteps 4767.
Path 272 | total_timesteps 4786.
Path 273 | total_timesteps 4798.
Path 274 | total_timesteps 4806.
Path 275 | total_timesteps 4833.
Path 276 | total_timesteps 4844.
Path 277 | total_timesteps 4864.
Path 278 | total_timesteps 4893.
Path 279 | total_timesteps 4904.
Path 280 | total_timesteps 4923.
Path 281 | total_timesteps 4934.
Path 282 | total_timesteps 4944.
Path 283 | total_timesteps 4960.
Path 284 | total_timesteps 4971.
Path 285 | total_timesteps 4991.
Path 286 | total_timesteps 5017.
Path 287 | total_timesteps 5027.
Path 288 | total_timesteps 5049.
Path 289 | total_timesteps 5080.
Path 290 | total_timesteps 5092.
Path 291 | total_timesteps 5115.
Path 292 | total_timesteps 5139.
Path 293 | total_timesteps 5165.
Path 294 | total_timesteps 5173.
Path 295 | total_timesteps 5186.
Path 296 | total_timesteps 5194.
Path 297 | total_timesteps 5213.
Path 298 | total_timesteps 5244.
Path 299 | total_timesteps 5259.
Path 300 | total_timesteps 5271.
Path 301 | total_timesteps 5287.
Path 302 | total_timesteps 5301.
Path 303 | total_timesteps 5311.
Path 304 | total_timesteps 5334.
Path 305 | total_timesteps 5343.
Path 306 | total_timesteps 5351.
Path 307 | total_timesteps 5362.
Path 308 | total_timesteps 5370.
Path 309 | total_timesteps 5384.
Path 310 | total_timesteps 5404.
Path 311 | total_timesteps 5415.
Path 312 | total_timesteps 5434.
Path 313 | total_timesteps 5444.
Path 314 | total_timesteps 5467.
Path 315 | total_timesteps 5487.
Path 316 | total_timesteps 5497.
Path 317 | total_timesteps 5521.
Path 318 | total_timesteps 5532.
Path 319 | total_timesteps 5547.
Path 320 | total_timesteps 5567.
Path 321 | total_timesteps 5598.
Path 322 | total_timesteps 5636.
Path 323 | total_timesteps 5670.
Path 324 | total_timesteps 5694.
Path 325 | total_timesteps 5703.
Path 326 | total_timesteps 5718.
Path 327 | total_timesteps 5726.
Path 328 | total_timesteps 5751.
Path 329 | total_timesteps 5767.
Path 330 | total_timesteps 5784.
Path 331 | total_timesteps 5795.
Path 332 | total_timesteps 5812.
Path 333 | total_timesteps 5819.
Path 334 | total_timesteps 5829.
Path 335 | total_timesteps 5848.
Path 336 | total_timesteps 5880.
Path 337 | total_timesteps 5893.
Path 338 | total_timesteps 5903.
Path 339 | total_timesteps 5915.
Path 340 | total_timesteps 5939.
Path 341 | total_timesteps 5956.
Path 342 | total_timesteps 5968.
Path 343 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.21    |
| Iteration     | 17       |
| MaximumReturn | 9.1      |
| MinimumReturn | -19.5    |
| TotalSamples  | 76129    |
----------------------------
itr #18 | 
Fitting dynamics.
Validation loss = 0.004303573630750179
Validation loss = 0.0036779537331312895
Validation loss = 0.004222243558615446
Validation loss = 0.003585808677598834
Validation loss = 0.003899760078638792
Validation loss = 0.003401379333809018
Validation loss = 0.003440138651058078
Validation loss = 0.00393011886626482
Validation loss = 0.003494038013741374
Validation loss = 0.0033485309686511755
Validation loss = 0.0034848314244300127
Validation loss = 0.003677125321701169
Validation loss = 0.0036420696415007114
Validation loss = 0.003472483018413186
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 37.
Path 3 | total_timesteps 48.
Path 4 | total_timesteps 56.
Path 5 | total_timesteps 67.
Path 6 | total_timesteps 109.
Path 7 | total_timesteps 124.
Path 8 | total_timesteps 136.
Path 9 | total_timesteps 169.
Path 10 | total_timesteps 183.
Path 11 | total_timesteps 205.
Path 12 | total_timesteps 218.
Path 13 | total_timesteps 235.
Path 14 | total_timesteps 262.
Path 15 | total_timesteps 278.
Path 16 | total_timesteps 297.
Path 17 | total_timesteps 324.
Path 18 | total_timesteps 335.
Path 19 | total_timesteps 348.
Path 20 | total_timesteps 361.
Path 21 | total_timesteps 371.
Path 22 | total_timesteps 393.
Path 23 | total_timesteps 421.
Path 24 | total_timesteps 435.
Path 25 | total_timesteps 450.
Path 26 | total_timesteps 464.
Path 27 | total_timesteps 506.
Path 28 | total_timesteps 524.
Path 29 | total_timesteps 549.
Path 30 | total_timesteps 582.
Path 31 | total_timesteps 603.
Path 32 | total_timesteps 614.
Path 33 | total_timesteps 629.
Path 34 | total_timesteps 660.
Path 35 | total_timesteps 680.
Path 36 | total_timesteps 693.
Path 37 | total_timesteps 705.
Path 38 | total_timesteps 716.
Path 39 | total_timesteps 732.
Path 40 | total_timesteps 752.
Path 41 | total_timesteps 776.
Path 42 | total_timesteps 811.
Path 43 | total_timesteps 822.
Path 44 | total_timesteps 846.
Path 45 | total_timesteps 860.
Path 46 | total_timesteps 875.
Path 47 | total_timesteps 887.
Path 48 | total_timesteps 903.
Path 49 | total_timesteps 924.
Path 50 | total_timesteps 935.
Path 51 | total_timesteps 948.
Path 52 | total_timesteps 961.
Path 53 | total_timesteps 971.
Path 54 | total_timesteps 992.
Path 55 | total_timesteps 1002.
Path 56 | total_timesteps 1016.
Path 57 | total_timesteps 1036.
Path 58 | total_timesteps 1046.
Path 59 | total_timesteps 1062.
Path 60 | total_timesteps 1071.
Path 61 | total_timesteps 1108.
Path 62 | total_timesteps 1135.
Path 63 | total_timesteps 1147.
Path 64 | total_timesteps 1163.
Path 65 | total_timesteps 1176.
Path 66 | total_timesteps 1190.
Path 67 | total_timesteps 1203.
Path 68 | total_timesteps 1217.
Path 69 | total_timesteps 1227.
Path 70 | total_timesteps 1248.
Path 71 | total_timesteps 1257.
Path 72 | total_timesteps 1269.
Path 73 | total_timesteps 1285.
Path 74 | total_timesteps 1297.
Path 75 | total_timesteps 1313.
Path 76 | total_timesteps 1326.
Path 77 | total_timesteps 1336.
Path 78 | total_timesteps 1367.
Path 79 | total_timesteps 1380.
Path 80 | total_timesteps 1412.
Path 81 | total_timesteps 1432.
Path 82 | total_timesteps 1454.
Path 83 | total_timesteps 1483.
Path 84 | total_timesteps 1503.
Path 85 | total_timesteps 1522.
Path 86 | total_timesteps 1535.
Path 87 | total_timesteps 1553.
Path 88 | total_timesteps 1564.
Path 89 | total_timesteps 1591.
Path 90 | total_timesteps 1605.
Path 91 | total_timesteps 1630.
Path 92 | total_timesteps 1641.
Path 93 | total_timesteps 1657.
Path 94 | total_timesteps 1684.
Path 95 | total_timesteps 1695.
Path 96 | total_timesteps 1715.
Path 97 | total_timesteps 1736.
Path 98 | total_timesteps 1761.
Path 99 | total_timesteps 1774.
Path 100 | total_timesteps 1785.
Path 101 | total_timesteps 1799.
Path 102 | total_timesteps 1814.
Path 103 | total_timesteps 1827.
Path 104 | total_timesteps 1839.
Path 105 | total_timesteps 1862.
Path 106 | total_timesteps 1872.
Path 107 | total_timesteps 1883.
Path 108 | total_timesteps 1910.
Path 109 | total_timesteps 1940.
Path 110 | total_timesteps 1960.
Path 111 | total_timesteps 1967.
Path 112 | total_timesteps 1984.
Path 113 | total_timesteps 2000.
Path 114 | total_timesteps 2024.
Path 115 | total_timesteps 2036.
Path 116 | total_timesteps 2048.
Path 117 | total_timesteps 2061.
Path 118 | total_timesteps 2073.
Path 119 | total_timesteps 2093.
Path 120 | total_timesteps 2116.
Path 121 | total_timesteps 2126.
Path 122 | total_timesteps 2140.
Path 123 | total_timesteps 2161.
Path 124 | total_timesteps 2188.
Path 125 | total_timesteps 2199.
Path 126 | total_timesteps 2224.
Path 127 | total_timesteps 2241.
Path 128 | total_timesteps 2251.
Path 129 | total_timesteps 2271.
Path 130 | total_timesteps 2286.
Path 131 | total_timesteps 2311.
Path 132 | total_timesteps 2322.
Path 133 | total_timesteps 2336.
Path 134 | total_timesteps 2361.
Path 135 | total_timesteps 2384.
Path 136 | total_timesteps 2398.
Path 137 | total_timesteps 2410.
Path 138 | total_timesteps 2434.
Path 139 | total_timesteps 2454.
Path 140 | total_timesteps 2468.
Path 141 | total_timesteps 2491.
Path 142 | total_timesteps 2505.
Path 143 | total_timesteps 2516.
Path 144 | total_timesteps 2550.
Path 145 | total_timesteps 2569.
Path 146 | total_timesteps 2584.
Path 147 | total_timesteps 2599.
Path 148 | total_timesteps 2609.
Path 149 | total_timesteps 2629.
Path 150 | total_timesteps 2650.
Path 151 | total_timesteps 2668.
Path 152 | total_timesteps 2682.
Path 153 | total_timesteps 2700.
Path 154 | total_timesteps 2716.
Path 155 | total_timesteps 2730.
Path 156 | total_timesteps 2744.
Path 157 | total_timesteps 2759.
Path 158 | total_timesteps 2768.
Path 159 | total_timesteps 2778.
Path 160 | total_timesteps 2786.
Path 161 | total_timesteps 2809.
Path 162 | total_timesteps 2845.
Path 163 | total_timesteps 2861.
Path 164 | total_timesteps 2883.
Path 165 | total_timesteps 2896.
Path 166 | total_timesteps 2911.
Path 167 | total_timesteps 2921.
Path 168 | total_timesteps 2947.
Path 169 | total_timesteps 2967.
Path 170 | total_timesteps 2977.
Path 171 | total_timesteps 3001.
Path 172 | total_timesteps 3014.
Path 173 | total_timesteps 3026.
Path 174 | total_timesteps 3040.
Path 175 | total_timesteps 3064.
Path 176 | total_timesteps 3081.
Path 177 | total_timesteps 3090.
Path 178 | total_timesteps 3116.
Path 179 | total_timesteps 3132.
Path 180 | total_timesteps 3159.
Path 181 | total_timesteps 3169.
Path 182 | total_timesteps 3185.
Path 183 | total_timesteps 3201.
Path 184 | total_timesteps 3217.
Path 185 | total_timesteps 3226.
Path 186 | total_timesteps 3251.
Path 187 | total_timesteps 3261.
Path 188 | total_timesteps 3289.
Path 189 | total_timesteps 3301.
Path 190 | total_timesteps 3325.
Path 191 | total_timesteps 3347.
Path 192 | total_timesteps 3367.
Path 193 | total_timesteps 3385.
Path 194 | total_timesteps 3403.
Path 195 | total_timesteps 3416.
Path 196 | total_timesteps 3449.
Path 197 | total_timesteps 3467.
Path 198 | total_timesteps 3479.
Path 199 | total_timesteps 3501.
Path 200 | total_timesteps 3512.
Path 201 | total_timesteps 3523.
Path 202 | total_timesteps 3565.
Path 203 | total_timesteps 3585.
Path 204 | total_timesteps 3596.
Path 205 | total_timesteps 3616.
Path 206 | total_timesteps 3632.
Path 207 | total_timesteps 3644.
Path 208 | total_timesteps 3656.
Path 209 | total_timesteps 3678.
Path 210 | total_timesteps 3694.
Path 211 | total_timesteps 3725.
Path 212 | total_timesteps 3735.
Path 213 | total_timesteps 3744.
Path 214 | total_timesteps 3752.
Path 215 | total_timesteps 3763.
Path 216 | total_timesteps 3772.
Path 217 | total_timesteps 3782.
Path 218 | total_timesteps 3798.
Path 219 | total_timesteps 3810.
Path 220 | total_timesteps 3830.
Path 221 | total_timesteps 3837.
Path 222 | total_timesteps 3848.
Path 223 | total_timesteps 3872.
Path 224 | total_timesteps 3884.
Path 225 | total_timesteps 3900.
Path 226 | total_timesteps 3919.
Path 227 | total_timesteps 3940.
Path 228 | total_timesteps 3958.
Path 229 | total_timesteps 3981.
Path 230 | total_timesteps 3998.
Path 231 | total_timesteps 4028.
Path 232 | total_timesteps 4052.
Path 233 | total_timesteps 4062.
Path 234 | total_timesteps 4087.
Path 235 | total_timesteps 4101.
Path 236 | total_timesteps 4122.
Path 237 | total_timesteps 4140.
Path 238 | total_timesteps 4149.
Path 239 | total_timesteps 4157.
Path 240 | total_timesteps 4167.
Path 241 | total_timesteps 4182.
Path 242 | total_timesteps 4201.
Path 243 | total_timesteps 4209.
Path 244 | total_timesteps 4222.
Path 245 | total_timesteps 4235.
Path 246 | total_timesteps 4251.
Path 247 | total_timesteps 4267.
Path 248 | total_timesteps 4277.
Path 249 | total_timesteps 4285.
Path 250 | total_timesteps 4302.
Path 251 | total_timesteps 4316.
Path 252 | total_timesteps 4325.
Path 253 | total_timesteps 4342.
Path 254 | total_timesteps 4353.
Path 255 | total_timesteps 4365.
Path 256 | total_timesteps 4386.
Path 257 | total_timesteps 4403.
Path 258 | total_timesteps 4410.
Path 259 | total_timesteps 4425.
Path 260 | total_timesteps 4439.
Path 261 | total_timesteps 4452.
Path 262 | total_timesteps 4471.
Path 263 | total_timesteps 4485.
Path 264 | total_timesteps 4497.
Path 265 | total_timesteps 4517.
Path 266 | total_timesteps 4526.
Path 267 | total_timesteps 4539.
Path 268 | total_timesteps 4564.
Path 269 | total_timesteps 4572.
Path 270 | total_timesteps 4604.
Path 271 | total_timesteps 4620.
Path 272 | total_timesteps 4634.
Path 273 | total_timesteps 4644.
Path 274 | total_timesteps 4662.
Path 275 | total_timesteps 4683.
Path 276 | total_timesteps 4716.
Path 277 | total_timesteps 4737.
Path 278 | total_timesteps 4749.
Path 279 | total_timesteps 4765.
Path 280 | total_timesteps 4774.
Path 281 | total_timesteps 4800.
Path 282 | total_timesteps 4819.
Path 283 | total_timesteps 4832.
Path 284 | total_timesteps 4854.
Path 285 | total_timesteps 4871.
Path 286 | total_timesteps 4885.
Path 287 | total_timesteps 4905.
Path 288 | total_timesteps 4915.
Path 289 | total_timesteps 4928.
Path 290 | total_timesteps 4949.
Path 291 | total_timesteps 4963.
Path 292 | total_timesteps 4983.
Path 293 | total_timesteps 4991.
Path 294 | total_timesteps 5006.
Path 295 | total_timesteps 5035.
Path 296 | total_timesteps 5057.
Path 297 | total_timesteps 5071.
Path 298 | total_timesteps 5079.
Path 299 | total_timesteps 5098.
Path 300 | total_timesteps 5113.
Path 301 | total_timesteps 5142.
Path 302 | total_timesteps 5154.
Path 303 | total_timesteps 5167.
Path 304 | total_timesteps 5181.
Path 305 | total_timesteps 5197.
Path 306 | total_timesteps 5210.
Path 307 | total_timesteps 5235.
Path 308 | total_timesteps 5245.
Path 309 | total_timesteps 5259.
Path 310 | total_timesteps 5276.
Path 311 | total_timesteps 5294.
Path 312 | total_timesteps 5312.
Path 313 | total_timesteps 5329.
Path 314 | total_timesteps 5344.
Path 315 | total_timesteps 5358.
Path 316 | total_timesteps 5388.
Path 317 | total_timesteps 5415.
Path 318 | total_timesteps 5443.
Path 319 | total_timesteps 5463.
Path 320 | total_timesteps 5481.
Path 321 | total_timesteps 5512.
Path 322 | total_timesteps 5522.
Path 323 | total_timesteps 5540.
Path 324 | total_timesteps 5558.
Path 325 | total_timesteps 5571.
Path 326 | total_timesteps 5598.
Path 327 | total_timesteps 5618.
Path 328 | total_timesteps 5635.
Path 329 | total_timesteps 5658.
Path 330 | total_timesteps 5668.
Path 331 | total_timesteps 5680.
Path 332 | total_timesteps 5703.
Path 333 | total_timesteps 5734.
Path 334 | total_timesteps 5744.
Path 335 | total_timesteps 5761.
Path 336 | total_timesteps 5771.
Path 337 | total_timesteps 5779.
Path 338 | total_timesteps 5804.
Path 339 | total_timesteps 5820.
Path 340 | total_timesteps 5827.
Path 341 | total_timesteps 5835.
Path 342 | total_timesteps 5845.
Path 343 | total_timesteps 5856.
Path 344 | total_timesteps 5878.
Path 345 | total_timesteps 5897.
Path 346 | total_timesteps 5908.
Path 347 | total_timesteps 5920.
Path 348 | total_timesteps 5929.
Path 349 | total_timesteps 5945.
Path 350 | total_timesteps 5958.
Path 351 | total_timesteps 5976.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.7     |
| Iteration     | 18       |
| MaximumReturn | 3.69     |
| MinimumReturn | -20.6    |
| TotalSamples  | 80132    |
----------------------------
itr #19 | 
Fitting dynamics.
Validation loss = 0.003410330507904291
Validation loss = 0.0034380308352410793
Validation loss = 0.003315715817734599
Validation loss = 0.003533005015924573
Validation loss = 0.003280260832980275
Validation loss = 0.0036529861390590668
Validation loss = 0.0036641438491642475
Validation loss = 0.003528227563947439
Validation loss = 0.0037193489260971546
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 18.
Path 2 | total_timesteps 38.
Path 3 | total_timesteps 46.
Path 4 | total_timesteps 57.
Path 5 | total_timesteps 65.
Path 6 | total_timesteps 81.
Path 7 | total_timesteps 100.
Path 8 | total_timesteps 108.
Path 9 | total_timesteps 126.
Path 10 | total_timesteps 144.
Path 11 | total_timesteps 154.
Path 12 | total_timesteps 169.
Path 13 | total_timesteps 181.
Path 14 | total_timesteps 205.
Path 15 | total_timesteps 218.
Path 16 | total_timesteps 230.
Path 17 | total_timesteps 246.
Path 18 | total_timesteps 288.
Path 19 | total_timesteps 299.
Path 20 | total_timesteps 309.
Path 21 | total_timesteps 340.
Path 22 | total_timesteps 354.
Path 23 | total_timesteps 370.
Path 24 | total_timesteps 396.
Path 25 | total_timesteps 410.
Path 26 | total_timesteps 421.
Path 27 | total_timesteps 436.
Path 28 | total_timesteps 471.
Path 29 | total_timesteps 483.
Path 30 | total_timesteps 508.
Path 31 | total_timesteps 520.
Path 32 | total_timesteps 527.
Path 33 | total_timesteps 541.
Path 34 | total_timesteps 558.
Path 35 | total_timesteps 594.
Path 36 | total_timesteps 606.
Path 37 | total_timesteps 621.
Path 38 | total_timesteps 636.
Path 39 | total_timesteps 678.
Path 40 | total_timesteps 691.
Path 41 | total_timesteps 703.
Path 42 | total_timesteps 732.
Path 43 | total_timesteps 750.
Path 44 | total_timesteps 771.
Path 45 | total_timesteps 785.
Path 46 | total_timesteps 796.
Path 47 | total_timesteps 815.
Path 48 | total_timesteps 834.
Path 49 | total_timesteps 842.
Path 50 | total_timesteps 850.
Path 51 | total_timesteps 871.
Path 52 | total_timesteps 905.
Path 53 | total_timesteps 925.
Path 54 | total_timesteps 940.
Path 55 | total_timesteps 953.
Path 56 | total_timesteps 966.
Path 57 | total_timesteps 991.
Path 58 | total_timesteps 1021.
Path 59 | total_timesteps 1049.
Path 60 | total_timesteps 1060.
Path 61 | total_timesteps 1074.
Path 62 | total_timesteps 1097.
Path 63 | total_timesteps 1109.
Path 64 | total_timesteps 1138.
Path 65 | total_timesteps 1156.
Path 66 | total_timesteps 1164.
Path 67 | total_timesteps 1191.
Path 68 | total_timesteps 1218.
Path 69 | total_timesteps 1239.
Path 70 | total_timesteps 1260.
Path 71 | total_timesteps 1278.
Path 72 | total_timesteps 1307.
Path 73 | total_timesteps 1316.
Path 74 | total_timesteps 1335.
Path 75 | total_timesteps 1362.
Path 76 | total_timesteps 1384.
Path 77 | total_timesteps 1398.
Path 78 | total_timesteps 1423.
Path 79 | total_timesteps 1431.
Path 80 | total_timesteps 1440.
Path 81 | total_timesteps 1459.
Path 82 | total_timesteps 1483.
Path 83 | total_timesteps 1496.
Path 84 | total_timesteps 1509.
Path 85 | total_timesteps 1522.
Path 86 | total_timesteps 1533.
Path 87 | total_timesteps 1558.
Path 88 | total_timesteps 1569.
Path 89 | total_timesteps 1592.
Path 90 | total_timesteps 1619.
Path 91 | total_timesteps 1639.
Path 92 | total_timesteps 1650.
Path 93 | total_timesteps 1686.
Path 94 | total_timesteps 1698.
Path 95 | total_timesteps 1709.
Path 96 | total_timesteps 1729.
Path 97 | total_timesteps 1742.
Path 98 | total_timesteps 1750.
Path 99 | total_timesteps 1765.
Path 100 | total_timesteps 1775.
Path 101 | total_timesteps 1785.
Path 102 | total_timesteps 1794.
Path 103 | total_timesteps 1805.
Path 104 | total_timesteps 1816.
Path 105 | total_timesteps 1833.
Path 106 | total_timesteps 1868.
Path 107 | total_timesteps 1877.
Path 108 | total_timesteps 1897.
Path 109 | total_timesteps 1929.
Path 110 | total_timesteps 1938.
Path 111 | total_timesteps 1960.
Path 112 | total_timesteps 1992.
Path 113 | total_timesteps 2014.
Path 114 | total_timesteps 2025.
Path 115 | total_timesteps 2036.
Path 116 | total_timesteps 2062.
Path 117 | total_timesteps 2091.
Path 118 | total_timesteps 2100.
Path 119 | total_timesteps 2123.
Path 120 | total_timesteps 2138.
Path 121 | total_timesteps 2151.
Path 122 | total_timesteps 2165.
Path 123 | total_timesteps 2177.
Path 124 | total_timesteps 2199.
Path 125 | total_timesteps 2240.
Path 126 | total_timesteps 2257.
Path 127 | total_timesteps 2269.
Path 128 | total_timesteps 2280.
Path 129 | total_timesteps 2299.
Path 130 | total_timesteps 2321.
Path 131 | total_timesteps 2345.
Path 132 | total_timesteps 2364.
Path 133 | total_timesteps 2381.
Path 134 | total_timesteps 2407.
Path 135 | total_timesteps 2422.
Path 136 | total_timesteps 2437.
Path 137 | total_timesteps 2453.
Path 138 | total_timesteps 2472.
Path 139 | total_timesteps 2481.
Path 140 | total_timesteps 2492.
Path 141 | total_timesteps 2510.
Path 142 | total_timesteps 2527.
Path 143 | total_timesteps 2544.
Path 144 | total_timesteps 2558.
Path 145 | total_timesteps 2575.
Path 146 | total_timesteps 2587.
Path 147 | total_timesteps 2605.
Path 148 | total_timesteps 2622.
Path 149 | total_timesteps 2634.
Path 150 | total_timesteps 2646.
Path 151 | total_timesteps 2658.
Path 152 | total_timesteps 2687.
Path 153 | total_timesteps 2695.
Path 154 | total_timesteps 2707.
Path 155 | total_timesteps 2724.
Path 156 | total_timesteps 2747.
Path 157 | total_timesteps 2778.
Path 158 | total_timesteps 2797.
Path 159 | total_timesteps 2811.
Path 160 | total_timesteps 2831.
Path 161 | total_timesteps 2861.
Path 162 | total_timesteps 2871.
Path 163 | total_timesteps 2879.
Path 164 | total_timesteps 2892.
Path 165 | total_timesteps 2899.
Path 166 | total_timesteps 2912.
Path 167 | total_timesteps 2923.
Path 168 | total_timesteps 2957.
Path 169 | total_timesteps 2974.
Path 170 | total_timesteps 2992.
Path 171 | total_timesteps 3014.
Path 172 | total_timesteps 3034.
Path 173 | total_timesteps 3049.
Path 174 | total_timesteps 3067.
Path 175 | total_timesteps 3087.
Path 176 | total_timesteps 3117.
Path 177 | total_timesteps 3124.
Path 178 | total_timesteps 3140.
Path 179 | total_timesteps 3150.
Path 180 | total_timesteps 3160.
Path 181 | total_timesteps 3171.
Path 182 | total_timesteps 3182.
Path 183 | total_timesteps 3203.
Path 184 | total_timesteps 3217.
Path 185 | total_timesteps 3230.
Path 186 | total_timesteps 3249.
Path 187 | total_timesteps 3271.
Path 188 | total_timesteps 3307.
Path 189 | total_timesteps 3337.
Path 190 | total_timesteps 3347.
Path 191 | total_timesteps 3384.
Path 192 | total_timesteps 3422.
Path 193 | total_timesteps 3445.
Path 194 | total_timesteps 3459.
Path 195 | total_timesteps 3481.
Path 196 | total_timesteps 3498.
Path 197 | total_timesteps 3518.
Path 198 | total_timesteps 3537.
Path 199 | total_timesteps 3557.
Path 200 | total_timesteps 3572.
Path 201 | total_timesteps 3597.
Path 202 | total_timesteps 3628.
Path 203 | total_timesteps 3644.
Path 204 | total_timesteps 3656.
Path 205 | total_timesteps 3677.
Path 206 | total_timesteps 3703.
Path 207 | total_timesteps 3714.
Path 208 | total_timesteps 3727.
Path 209 | total_timesteps 3750.
Path 210 | total_timesteps 3763.
Path 211 | total_timesteps 3780.
Path 212 | total_timesteps 3791.
Path 213 | total_timesteps 3811.
Path 214 | total_timesteps 3835.
Path 215 | total_timesteps 3849.
Path 216 | total_timesteps 3864.
Path 217 | total_timesteps 3891.
Path 218 | total_timesteps 3903.
Path 219 | total_timesteps 3949.
Path 220 | total_timesteps 3968.
Path 221 | total_timesteps 3980.
Path 222 | total_timesteps 4013.
Path 223 | total_timesteps 4027.
Path 224 | total_timesteps 4050.
Path 225 | total_timesteps 4065.
Path 226 | total_timesteps 4079.
Path 227 | total_timesteps 4098.
Path 228 | total_timesteps 4116.
Path 229 | total_timesteps 4127.
Path 230 | total_timesteps 4144.
Path 231 | total_timesteps 4164.
Path 232 | total_timesteps 4182.
Path 233 | total_timesteps 4201.
Path 234 | total_timesteps 4212.
Path 235 | total_timesteps 4234.
Path 236 | total_timesteps 4251.
Path 237 | total_timesteps 4265.
Path 238 | total_timesteps 4295.
Path 239 | total_timesteps 4309.
Path 240 | total_timesteps 4326.
Path 241 | total_timesteps 4341.
Path 242 | total_timesteps 4385.
Path 243 | total_timesteps 4399.
Path 244 | total_timesteps 4412.
Path 245 | total_timesteps 4421.
Path 246 | total_timesteps 4432.
Path 247 | total_timesteps 4458.
Path 248 | total_timesteps 4474.
Path 249 | total_timesteps 4490.
Path 250 | total_timesteps 4512.
Path 251 | total_timesteps 4522.
Path 252 | total_timesteps 4554.
Path 253 | total_timesteps 4579.
Path 254 | total_timesteps 4591.
Path 255 | total_timesteps 4615.
Path 256 | total_timesteps 4622.
Path 257 | total_timesteps 4637.
Path 258 | total_timesteps 4660.
Path 259 | total_timesteps 4678.
Path 260 | total_timesteps 4686.
Path 261 | total_timesteps 4707.
Path 262 | total_timesteps 4719.
Path 263 | total_timesteps 4726.
Path 264 | total_timesteps 4735.
Path 265 | total_timesteps 4744.
Path 266 | total_timesteps 4774.
Path 267 | total_timesteps 4804.
Path 268 | total_timesteps 4817.
Path 269 | total_timesteps 4834.
Path 270 | total_timesteps 4846.
Path 271 | total_timesteps 4865.
Path 272 | total_timesteps 4887.
Path 273 | total_timesteps 4898.
Path 274 | total_timesteps 4917.
Path 275 | total_timesteps 4926.
Path 276 | total_timesteps 4942.
Path 277 | total_timesteps 4971.
Path 278 | total_timesteps 4983.
Path 279 | total_timesteps 5008.
Path 280 | total_timesteps 5017.
Path 281 | total_timesteps 5043.
Path 282 | total_timesteps 5069.
Path 283 | total_timesteps 5078.
Path 284 | total_timesteps 5098.
Path 285 | total_timesteps 5107.
Path 286 | total_timesteps 5117.
Path 287 | total_timesteps 5135.
Path 288 | total_timesteps 5157.
Path 289 | total_timesteps 5169.
Path 290 | total_timesteps 5188.
Path 291 | total_timesteps 5200.
Path 292 | total_timesteps 5231.
Path 293 | total_timesteps 5244.
Path 294 | total_timesteps 5251.
Path 295 | total_timesteps 5290.
Path 296 | total_timesteps 5307.
Path 297 | total_timesteps 5322.
Path 298 | total_timesteps 5348.
Path 299 | total_timesteps 5362.
Path 300 | total_timesteps 5376.
Path 301 | total_timesteps 5392.
Path 302 | total_timesteps 5415.
Path 303 | total_timesteps 5439.
Path 304 | total_timesteps 5458.
Path 305 | total_timesteps 5474.
Path 306 | total_timesteps 5486.
Path 307 | total_timesteps 5511.
Path 308 | total_timesteps 5520.
Path 309 | total_timesteps 5532.
Path 310 | total_timesteps 5549.
Path 311 | total_timesteps 5563.
Path 312 | total_timesteps 5573.
Path 313 | total_timesteps 5585.
Path 314 | total_timesteps 5604.
Path 315 | total_timesteps 5616.
Path 316 | total_timesteps 5635.
Path 317 | total_timesteps 5648.
Path 318 | total_timesteps 5672.
Path 319 | total_timesteps 5692.
Path 320 | total_timesteps 5713.
Path 321 | total_timesteps 5726.
Path 322 | total_timesteps 5736.
Path 323 | total_timesteps 5771.
Path 324 | total_timesteps 5785.
Path 325 | total_timesteps 5798.
Path 326 | total_timesteps 5809.
Path 327 | total_timesteps 5821.
Path 328 | total_timesteps 5844.
Path 329 | total_timesteps 5870.
Path 330 | total_timesteps 5888.
Path 331 | total_timesteps 5902.
Path 332 | total_timesteps 5932.
Path 333 | total_timesteps 5946.
Path 334 | total_timesteps 5956.
Path 335 | total_timesteps 5968.
Path 336 | total_timesteps 5983.
Path 337 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.92    |
| Iteration     | 19       |
| MaximumReturn | 12.6     |
| MinimumReturn | -18.2    |
| TotalSamples  | 84137    |
----------------------------
itr #20 | 
Fitting dynamics.
Validation loss = 0.0036840771790593863
Validation loss = 0.0032149425242096186
Validation loss = 0.003344987751916051
Validation loss = 0.003176380880177021
Validation loss = 0.0034780672285705805
Validation loss = 0.003245913190767169
Validation loss = 0.003362593473866582
Validation loss = 0.0032754901330918074
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 44.
Path 3 | total_timesteps 68.
Path 4 | total_timesteps 98.
Path 5 | total_timesteps 115.
Path 6 | total_timesteps 127.
Path 7 | total_timesteps 147.
Path 8 | total_timesteps 162.
Path 9 | total_timesteps 183.
Path 10 | total_timesteps 197.
Path 11 | total_timesteps 205.
Path 12 | total_timesteps 215.
Path 13 | total_timesteps 224.
Path 14 | total_timesteps 236.
Path 15 | total_timesteps 252.
Path 16 | total_timesteps 268.
Path 17 | total_timesteps 290.
Path 18 | total_timesteps 302.
Path 19 | total_timesteps 314.
Path 20 | total_timesteps 332.
Path 21 | total_timesteps 345.
Path 22 | total_timesteps 359.
Path 23 | total_timesteps 380.
Path 24 | total_timesteps 395.
Path 25 | total_timesteps 418.
Path 26 | total_timesteps 437.
Path 27 | total_timesteps 451.
Path 28 | total_timesteps 469.
Path 29 | total_timesteps 486.
Path 30 | total_timesteps 498.
Path 31 | total_timesteps 508.
Path 32 | total_timesteps 527.
Path 33 | total_timesteps 539.
Path 34 | total_timesteps 556.
Path 35 | total_timesteps 576.
Path 36 | total_timesteps 585.
Path 37 | total_timesteps 596.
Path 38 | total_timesteps 615.
Path 39 | total_timesteps 625.
Path 40 | total_timesteps 650.
Path 41 | total_timesteps 663.
Path 42 | total_timesteps 675.
Path 43 | total_timesteps 693.
Path 44 | total_timesteps 722.
Path 45 | total_timesteps 735.
Path 46 | total_timesteps 749.
Path 47 | total_timesteps 770.
Path 48 | total_timesteps 786.
Path 49 | total_timesteps 811.
Path 50 | total_timesteps 826.
Path 51 | total_timesteps 839.
Path 52 | total_timesteps 855.
Path 53 | total_timesteps 880.
Path 54 | total_timesteps 906.
Path 55 | total_timesteps 927.
Path 56 | total_timesteps 938.
Path 57 | total_timesteps 948.
Path 58 | total_timesteps 960.
Path 59 | total_timesteps 970.
Path 60 | total_timesteps 992.
Path 61 | total_timesteps 1020.
Path 62 | total_timesteps 1033.
Path 63 | total_timesteps 1048.
Path 64 | total_timesteps 1059.
Path 65 | total_timesteps 1074.
Path 66 | total_timesteps 1091.
Path 67 | total_timesteps 1124.
Path 68 | total_timesteps 1137.
Path 69 | total_timesteps 1158.
Path 70 | total_timesteps 1180.
Path 71 | total_timesteps 1199.
Path 72 | total_timesteps 1218.
Path 73 | total_timesteps 1248.
Path 74 | total_timesteps 1262.
Path 75 | total_timesteps 1284.
Path 76 | total_timesteps 1298.
Path 77 | total_timesteps 1313.
Path 78 | total_timesteps 1340.
Path 79 | total_timesteps 1353.
Path 80 | total_timesteps 1370.
Path 81 | total_timesteps 1384.
Path 82 | total_timesteps 1399.
Path 83 | total_timesteps 1406.
Path 84 | total_timesteps 1426.
Path 85 | total_timesteps 1436.
Path 86 | total_timesteps 1446.
Path 87 | total_timesteps 1475.
Path 88 | total_timesteps 1489.
Path 89 | total_timesteps 1501.
Path 90 | total_timesteps 1512.
Path 91 | total_timesteps 1530.
Path 92 | total_timesteps 1547.
Path 93 | total_timesteps 1560.
Path 94 | total_timesteps 1573.
Path 95 | total_timesteps 1594.
Path 96 | total_timesteps 1604.
Path 97 | total_timesteps 1624.
Path 98 | total_timesteps 1636.
Path 99 | total_timesteps 1653.
Path 100 | total_timesteps 1685.
Path 101 | total_timesteps 1695.
Path 102 | total_timesteps 1710.
Path 103 | total_timesteps 1725.
Path 104 | total_timesteps 1733.
Path 105 | total_timesteps 1748.
Path 106 | total_timesteps 1760.
Path 107 | total_timesteps 1775.
Path 108 | total_timesteps 1788.
Path 109 | total_timesteps 1798.
Path 110 | total_timesteps 1822.
Path 111 | total_timesteps 1833.
Path 112 | total_timesteps 1846.
Path 113 | total_timesteps 1873.
Path 114 | total_timesteps 1893.
Path 115 | total_timesteps 1911.
Path 116 | total_timesteps 1923.
Path 117 | total_timesteps 1934.
Path 118 | total_timesteps 1955.
Path 119 | total_timesteps 1976.
Path 120 | total_timesteps 1992.
Path 121 | total_timesteps 2013.
Path 122 | total_timesteps 2028.
Path 123 | total_timesteps 2049.
Path 124 | total_timesteps 2067.
Path 125 | total_timesteps 2086.
Path 126 | total_timesteps 2094.
Path 127 | total_timesteps 2126.
Path 128 | total_timesteps 2152.
Path 129 | total_timesteps 2175.
Path 130 | total_timesteps 2191.
Path 131 | total_timesteps 2201.
Path 132 | total_timesteps 2214.
Path 133 | total_timesteps 2223.
Path 134 | total_timesteps 2242.
Path 135 | total_timesteps 2269.
Path 136 | total_timesteps 2310.
Path 137 | total_timesteps 2320.
Path 138 | total_timesteps 2349.
Path 139 | total_timesteps 2364.
Path 140 | total_timesteps 2380.
Path 141 | total_timesteps 2393.
Path 142 | total_timesteps 2413.
Path 143 | total_timesteps 2428.
Path 144 | total_timesteps 2459.
Path 145 | total_timesteps 2476.
Path 146 | total_timesteps 2507.
Path 147 | total_timesteps 2523.
Path 148 | total_timesteps 2539.
Path 149 | total_timesteps 2569.
Path 150 | total_timesteps 2585.
Path 151 | total_timesteps 2597.
Path 152 | total_timesteps 2608.
Path 153 | total_timesteps 2626.
Path 154 | total_timesteps 2650.
Path 155 | total_timesteps 2664.
Path 156 | total_timesteps 2680.
Path 157 | total_timesteps 2706.
Path 158 | total_timesteps 2715.
Path 159 | total_timesteps 2736.
Path 160 | total_timesteps 2751.
Path 161 | total_timesteps 2762.
Path 162 | total_timesteps 2774.
Path 163 | total_timesteps 2784.
Path 164 | total_timesteps 2798.
Path 165 | total_timesteps 2809.
Path 166 | total_timesteps 2830.
Path 167 | total_timesteps 2839.
Path 168 | total_timesteps 2855.
Path 169 | total_timesteps 2865.
Path 170 | total_timesteps 2879.
Path 171 | total_timesteps 2901.
Path 172 | total_timesteps 2926.
Path 173 | total_timesteps 2942.
Path 174 | total_timesteps 2957.
Path 175 | total_timesteps 2971.
Path 176 | total_timesteps 2986.
Path 177 | total_timesteps 3016.
Path 178 | total_timesteps 3030.
Path 179 | total_timesteps 3042.
Path 180 | total_timesteps 3059.
Path 181 | total_timesteps 3081.
Path 182 | total_timesteps 3094.
Path 183 | total_timesteps 3114.
Path 184 | total_timesteps 3135.
Path 185 | total_timesteps 3145.
Path 186 | total_timesteps 3154.
Path 187 | total_timesteps 3164.
Path 188 | total_timesteps 3178.
Path 189 | total_timesteps 3193.
Path 190 | total_timesteps 3213.
Path 191 | total_timesteps 3227.
Path 192 | total_timesteps 3246.
Path 193 | total_timesteps 3257.
Path 194 | total_timesteps 3279.
Path 195 | total_timesteps 3296.
Path 196 | total_timesteps 3304.
Path 197 | total_timesteps 3318.
Path 198 | total_timesteps 3328.
Path 199 | total_timesteps 3351.
Path 200 | total_timesteps 3365.
Path 201 | total_timesteps 3389.
Path 202 | total_timesteps 3416.
Path 203 | total_timesteps 3429.
Path 204 | total_timesteps 3440.
Path 205 | total_timesteps 3460.
Path 206 | total_timesteps 3474.
Path 207 | total_timesteps 3489.
Path 208 | total_timesteps 3501.
Path 209 | total_timesteps 3519.
Path 210 | total_timesteps 3538.
Path 211 | total_timesteps 3549.
Path 212 | total_timesteps 3578.
Path 213 | total_timesteps 3597.
Path 214 | total_timesteps 3605.
Path 215 | total_timesteps 3629.
Path 216 | total_timesteps 3640.
Path 217 | total_timesteps 3653.
Path 218 | total_timesteps 3663.
Path 219 | total_timesteps 3685.
Path 220 | total_timesteps 3700.
Path 221 | total_timesteps 3715.
Path 222 | total_timesteps 3760.
Path 223 | total_timesteps 3772.
Path 224 | total_timesteps 3784.
Path 225 | total_timesteps 3796.
Path 226 | total_timesteps 3808.
Path 227 | total_timesteps 3838.
Path 228 | total_timesteps 3846.
Path 229 | total_timesteps 3854.
Path 230 | total_timesteps 3869.
Path 231 | total_timesteps 3880.
Path 232 | total_timesteps 3907.
Path 233 | total_timesteps 3927.
Path 234 | total_timesteps 3955.
Path 235 | total_timesteps 3974.
Path 236 | total_timesteps 3985.
Path 237 | total_timesteps 4006.
Path 238 | total_timesteps 4018.
Path 239 | total_timesteps 4032.
Path 240 | total_timesteps 4039.
Path 241 | total_timesteps 4065.
Path 242 | total_timesteps 4083.
Path 243 | total_timesteps 4094.
Path 244 | total_timesteps 4121.
Path 245 | total_timesteps 4138.
Path 246 | total_timesteps 4147.
Path 247 | total_timesteps 4158.
Path 248 | total_timesteps 4166.
Path 249 | total_timesteps 4183.
Path 250 | total_timesteps 4206.
Path 251 | total_timesteps 4224.
Path 252 | total_timesteps 4235.
Path 253 | total_timesteps 4257.
Path 254 | total_timesteps 4266.
Path 255 | total_timesteps 4280.
Path 256 | total_timesteps 4298.
Path 257 | total_timesteps 4320.
Path 258 | total_timesteps 4348.
Path 259 | total_timesteps 4356.
Path 260 | total_timesteps 4374.
Path 261 | total_timesteps 4385.
Path 262 | total_timesteps 4408.
Path 263 | total_timesteps 4417.
Path 264 | total_timesteps 4430.
Path 265 | total_timesteps 4443.
Path 266 | total_timesteps 4456.
Path 267 | total_timesteps 4495.
Path 268 | total_timesteps 4504.
Path 269 | total_timesteps 4523.
Path 270 | total_timesteps 4544.
Path 271 | total_timesteps 4562.
Path 272 | total_timesteps 4573.
Path 273 | total_timesteps 4595.
Path 274 | total_timesteps 4618.
Path 275 | total_timesteps 4634.
Path 276 | total_timesteps 4649.
Path 277 | total_timesteps 4661.
Path 278 | total_timesteps 4682.
Path 279 | total_timesteps 4700.
Path 280 | total_timesteps 4716.
Path 281 | total_timesteps 4735.
Path 282 | total_timesteps 4754.
Path 283 | total_timesteps 4785.
Path 284 | total_timesteps 4799.
Path 285 | total_timesteps 4812.
Path 286 | total_timesteps 4825.
Path 287 | total_timesteps 4839.
Path 288 | total_timesteps 4864.
Path 289 | total_timesteps 4878.
Path 290 | total_timesteps 4900.
Path 291 | total_timesteps 4913.
Path 292 | total_timesteps 4924.
Path 293 | total_timesteps 4940.
Path 294 | total_timesteps 4960.
Path 295 | total_timesteps 4980.
Path 296 | total_timesteps 5000.
Path 297 | total_timesteps 5012.
Path 298 | total_timesteps 5028.
Path 299 | total_timesteps 5044.
Path 300 | total_timesteps 5053.
Path 301 | total_timesteps 5067.
Path 302 | total_timesteps 5111.
Path 303 | total_timesteps 5127.
Path 304 | total_timesteps 5145.
Path 305 | total_timesteps 5172.
Path 306 | total_timesteps 5183.
Path 307 | total_timesteps 5195.
Path 308 | total_timesteps 5215.
Path 309 | total_timesteps 5223.
Path 310 | total_timesteps 5234.
Path 311 | total_timesteps 5247.
Path 312 | total_timesteps 5269.
Path 313 | total_timesteps 5289.
Path 314 | total_timesteps 5317.
Path 315 | total_timesteps 5328.
Path 316 | total_timesteps 5351.
Path 317 | total_timesteps 5379.
Path 318 | total_timesteps 5402.
Path 319 | total_timesteps 5415.
Path 320 | total_timesteps 5424.
Path 321 | total_timesteps 5440.
Path 322 | total_timesteps 5451.
Path 323 | total_timesteps 5470.
Path 324 | total_timesteps 5493.
Path 325 | total_timesteps 5506.
Path 326 | total_timesteps 5514.
Path 327 | total_timesteps 5532.
Path 328 | total_timesteps 5551.
Path 329 | total_timesteps 5564.
Path 330 | total_timesteps 5585.
Path 331 | total_timesteps 5601.
Path 332 | total_timesteps 5627.
Path 333 | total_timesteps 5645.
Path 334 | total_timesteps 5671.
Path 335 | total_timesteps 5690.
Path 336 | total_timesteps 5707.
Path 337 | total_timesteps 5719.
Path 338 | total_timesteps 5742.
Path 339 | total_timesteps 5765.
Path 340 | total_timesteps 5783.
Path 341 | total_timesteps 5802.
Path 342 | total_timesteps 5813.
Path 343 | total_timesteps 5848.
Path 344 | total_timesteps 5882.
Path 345 | total_timesteps 5904.
Path 346 | total_timesteps 5916.
Path 347 | total_timesteps 5938.
Path 348 | total_timesteps 5954.
Path 349 | total_timesteps 5971.
Path 350 | total_timesteps 5979.
Path 351 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.42    |
| Iteration     | 20       |
| MaximumReturn | 7.73     |
| MinimumReturn | -22.6    |
| TotalSamples  | 88139    |
----------------------------
itr #21 | 
Fitting dynamics.
Validation loss = 0.0032805888913571835
Validation loss = 0.0032056039199233055
Validation loss = 0.0032275221310555935
Validation loss = 0.0029578222893178463
Validation loss = 0.003108959412202239
Validation loss = 0.0035855581518262625
Validation loss = 0.0030398929957300425
Validation loss = 0.0033240022603422403
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 18.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 36.
Path 4 | total_timesteps 46.
Path 5 | total_timesteps 60.
Path 6 | total_timesteps 77.
Path 7 | total_timesteps 108.
Path 8 | total_timesteps 130.
Path 9 | total_timesteps 141.
Path 10 | total_timesteps 164.
Path 11 | total_timesteps 175.
Path 12 | total_timesteps 188.
Path 13 | total_timesteps 207.
Path 14 | total_timesteps 218.
Path 15 | total_timesteps 235.
Path 16 | total_timesteps 253.
Path 17 | total_timesteps 273.
Path 18 | total_timesteps 298.
Path 19 | total_timesteps 324.
Path 20 | total_timesteps 343.
Path 21 | total_timesteps 356.
Path 22 | total_timesteps 367.
Path 23 | total_timesteps 382.
Path 24 | total_timesteps 404.
Path 25 | total_timesteps 420.
Path 26 | total_timesteps 434.
Path 27 | total_timesteps 443.
Path 28 | total_timesteps 477.
Path 29 | total_timesteps 498.
Path 30 | total_timesteps 520.
Path 31 | total_timesteps 528.
Path 32 | total_timesteps 548.
Path 33 | total_timesteps 567.
Path 34 | total_timesteps 585.
Path 35 | total_timesteps 604.
Path 36 | total_timesteps 621.
Path 37 | total_timesteps 635.
Path 38 | total_timesteps 650.
Path 39 | total_timesteps 662.
Path 40 | total_timesteps 691.
Path 41 | total_timesteps 712.
Path 42 | total_timesteps 734.
Path 43 | total_timesteps 754.
Path 44 | total_timesteps 780.
Path 45 | total_timesteps 801.
Path 46 | total_timesteps 813.
Path 47 | total_timesteps 830.
Path 48 | total_timesteps 848.
Path 49 | total_timesteps 872.
Path 50 | total_timesteps 883.
Path 51 | total_timesteps 898.
Path 52 | total_timesteps 912.
Path 53 | total_timesteps 922.
Path 54 | total_timesteps 932.
Path 55 | total_timesteps 945.
Path 56 | total_timesteps 954.
Path 57 | total_timesteps 971.
Path 58 | total_timesteps 980.
Path 59 | total_timesteps 990.
Path 60 | total_timesteps 999.
Path 61 | total_timesteps 1010.
Path 62 | total_timesteps 1036.
Path 63 | total_timesteps 1050.
Path 64 | total_timesteps 1070.
Path 65 | total_timesteps 1083.
Path 66 | total_timesteps 1092.
Path 67 | total_timesteps 1115.
Path 68 | total_timesteps 1127.
Path 69 | total_timesteps 1147.
Path 70 | total_timesteps 1168.
Path 71 | total_timesteps 1179.
Path 72 | total_timesteps 1202.
Path 73 | total_timesteps 1214.
Path 74 | total_timesteps 1242.
Path 75 | total_timesteps 1253.
Path 76 | total_timesteps 1284.
Path 77 | total_timesteps 1300.
Path 78 | total_timesteps 1311.
Path 79 | total_timesteps 1328.
Path 80 | total_timesteps 1342.
Path 81 | total_timesteps 1363.
Path 82 | total_timesteps 1373.
Path 83 | total_timesteps 1388.
Path 84 | total_timesteps 1403.
Path 85 | total_timesteps 1418.
Path 86 | total_timesteps 1425.
Path 87 | total_timesteps 1450.
Path 88 | total_timesteps 1462.
Path 89 | total_timesteps 1471.
Path 90 | total_timesteps 1490.
Path 91 | total_timesteps 1510.
Path 92 | total_timesteps 1520.
Path 93 | total_timesteps 1533.
Path 94 | total_timesteps 1562.
Path 95 | total_timesteps 1578.
Path 96 | total_timesteps 1591.
Path 97 | total_timesteps 1604.
Path 98 | total_timesteps 1623.
Path 99 | total_timesteps 1642.
Path 100 | total_timesteps 1652.
Path 101 | total_timesteps 1670.
Path 102 | total_timesteps 1691.
Path 103 | total_timesteps 1714.
Path 104 | total_timesteps 1725.
Path 105 | total_timesteps 1748.
Path 106 | total_timesteps 1758.
Path 107 | total_timesteps 1767.
Path 108 | total_timesteps 1781.
Path 109 | total_timesteps 1799.
Path 110 | total_timesteps 1809.
Path 111 | total_timesteps 1823.
Path 112 | total_timesteps 1841.
Path 113 | total_timesteps 1851.
Path 114 | total_timesteps 1865.
Path 115 | total_timesteps 1890.
Path 116 | total_timesteps 1900.
Path 117 | total_timesteps 1933.
Path 118 | total_timesteps 1968.
Path 119 | total_timesteps 1978.
Path 120 | total_timesteps 1987.
Path 121 | total_timesteps 2001.
Path 122 | total_timesteps 2017.
Path 123 | total_timesteps 2027.
Path 124 | total_timesteps 2046.
Path 125 | total_timesteps 2071.
Path 126 | total_timesteps 2090.
Path 127 | total_timesteps 2102.
Path 128 | total_timesteps 2122.
Path 129 | total_timesteps 2135.
Path 130 | total_timesteps 2163.
Path 131 | total_timesteps 2198.
Path 132 | total_timesteps 2226.
Path 133 | total_timesteps 2243.
Path 134 | total_timesteps 2259.
Path 135 | total_timesteps 2269.
Path 136 | total_timesteps 2285.
Path 137 | total_timesteps 2295.
Path 138 | total_timesteps 2309.
Path 139 | total_timesteps 2328.
Path 140 | total_timesteps 2338.
Path 141 | total_timesteps 2353.
Path 142 | total_timesteps 2364.
Path 143 | total_timesteps 2380.
Path 144 | total_timesteps 2398.
Path 145 | total_timesteps 2412.
Path 146 | total_timesteps 2423.
Path 147 | total_timesteps 2449.
Path 148 | total_timesteps 2471.
Path 149 | total_timesteps 2487.
Path 150 | total_timesteps 2499.
Path 151 | total_timesteps 2523.
Path 152 | total_timesteps 2533.
Path 153 | total_timesteps 2545.
Path 154 | total_timesteps 2565.
Path 155 | total_timesteps 2583.
Path 156 | total_timesteps 2610.
Path 157 | total_timesteps 2618.
Path 158 | total_timesteps 2630.
Path 159 | total_timesteps 2654.
Path 160 | total_timesteps 2665.
Path 161 | total_timesteps 2686.
Path 162 | total_timesteps 2719.
Path 163 | total_timesteps 2740.
Path 164 | total_timesteps 2752.
Path 165 | total_timesteps 2762.
Path 166 | total_timesteps 2788.
Path 167 | total_timesteps 2798.
Path 168 | total_timesteps 2809.
Path 169 | total_timesteps 2820.
Path 170 | total_timesteps 2833.
Path 171 | total_timesteps 2854.
Path 172 | total_timesteps 2867.
Path 173 | total_timesteps 2880.
Path 174 | total_timesteps 2890.
Path 175 | total_timesteps 2905.
Path 176 | total_timesteps 2924.
Path 177 | total_timesteps 2937.
Path 178 | total_timesteps 2945.
Path 179 | total_timesteps 2968.
Path 180 | total_timesteps 3018.
Path 181 | total_timesteps 3038.
Path 182 | total_timesteps 3047.
Path 183 | total_timesteps 3066.
Path 184 | total_timesteps 3099.
Path 185 | total_timesteps 3138.
Path 186 | total_timesteps 3155.
Path 187 | total_timesteps 3170.
Path 188 | total_timesteps 3186.
Path 189 | total_timesteps 3198.
Path 190 | total_timesteps 3211.
Path 191 | total_timesteps 3222.
Path 192 | total_timesteps 3241.
Path 193 | total_timesteps 3257.
Path 194 | total_timesteps 3273.
Path 195 | total_timesteps 3305.
Path 196 | total_timesteps 3314.
Path 197 | total_timesteps 3346.
Path 198 | total_timesteps 3359.
Path 199 | total_timesteps 3374.
Path 200 | total_timesteps 3387.
Path 201 | total_timesteps 3394.
Path 202 | total_timesteps 3409.
Path 203 | total_timesteps 3428.
Path 204 | total_timesteps 3444.
Path 205 | total_timesteps 3463.
Path 206 | total_timesteps 3488.
Path 207 | total_timesteps 3508.
Path 208 | total_timesteps 3542.
Path 209 | total_timesteps 3554.
Path 210 | total_timesteps 3564.
Path 211 | total_timesteps 3592.
Path 212 | total_timesteps 3600.
Path 213 | total_timesteps 3609.
Path 214 | total_timesteps 3625.
Path 215 | total_timesteps 3643.
Path 216 | total_timesteps 3655.
Path 217 | total_timesteps 3670.
Path 218 | total_timesteps 3683.
Path 219 | total_timesteps 3698.
Path 220 | total_timesteps 3725.
Path 221 | total_timesteps 3753.
Path 222 | total_timesteps 3773.
Path 223 | total_timesteps 3787.
Path 224 | total_timesteps 3814.
Path 225 | total_timesteps 3834.
Path 226 | total_timesteps 3847.
Path 227 | total_timesteps 3869.
Path 228 | total_timesteps 3882.
Path 229 | total_timesteps 3893.
Path 230 | total_timesteps 3909.
Path 231 | total_timesteps 3918.
Path 232 | total_timesteps 3934.
Path 233 | total_timesteps 3949.
Path 234 | total_timesteps 3968.
Path 235 | total_timesteps 3989.
Path 236 | total_timesteps 4007.
Path 237 | total_timesteps 4028.
Path 238 | total_timesteps 4038.
Path 239 | total_timesteps 4056.
Path 240 | total_timesteps 4082.
Path 241 | total_timesteps 4108.
Path 242 | total_timesteps 4124.
Path 243 | total_timesteps 4146.
Path 244 | total_timesteps 4159.
Path 245 | total_timesteps 4187.
Path 246 | total_timesteps 4204.
Path 247 | total_timesteps 4218.
Path 248 | total_timesteps 4229.
Path 249 | total_timesteps 4244.
Path 250 | total_timesteps 4281.
Path 251 | total_timesteps 4314.
Path 252 | total_timesteps 4333.
Path 253 | total_timesteps 4355.
Path 254 | total_timesteps 4371.
Path 255 | total_timesteps 4389.
Path 256 | total_timesteps 4412.
Path 257 | total_timesteps 4426.
Path 258 | total_timesteps 4437.
Path 259 | total_timesteps 4453.
Path 260 | total_timesteps 4469.
Path 261 | total_timesteps 4503.
Path 262 | total_timesteps 4518.
Path 263 | total_timesteps 4530.
Path 264 | total_timesteps 4539.
Path 265 | total_timesteps 4552.
Path 266 | total_timesteps 4563.
Path 267 | total_timesteps 4578.
Path 268 | total_timesteps 4610.
Path 269 | total_timesteps 4627.
Path 270 | total_timesteps 4645.
Path 271 | total_timesteps 4655.
Path 272 | total_timesteps 4677.
Path 273 | total_timesteps 4703.
Path 274 | total_timesteps 4721.
Path 275 | total_timesteps 4753.
Path 276 | total_timesteps 4770.
Path 277 | total_timesteps 4779.
Path 278 | total_timesteps 4803.
Path 279 | total_timesteps 4821.
Path 280 | total_timesteps 4836.
Path 281 | total_timesteps 4848.
Path 282 | total_timesteps 4873.
Path 283 | total_timesteps 4896.
Path 284 | total_timesteps 4906.
Path 285 | total_timesteps 4933.
Path 286 | total_timesteps 4956.
Path 287 | total_timesteps 4970.
Path 288 | total_timesteps 5004.
Path 289 | total_timesteps 5034.
Path 290 | total_timesteps 5045.
Path 291 | total_timesteps 5057.
Path 292 | total_timesteps 5075.
Path 293 | total_timesteps 5089.
Path 294 | total_timesteps 5099.
Path 295 | total_timesteps 5107.
Path 296 | total_timesteps 5115.
Path 297 | total_timesteps 5130.
Path 298 | total_timesteps 5154.
Path 299 | total_timesteps 5175.
Path 300 | total_timesteps 5192.
Path 301 | total_timesteps 5201.
Path 302 | total_timesteps 5218.
Path 303 | total_timesteps 5237.
Path 304 | total_timesteps 5251.
Path 305 | total_timesteps 5264.
Path 306 | total_timesteps 5294.
Path 307 | total_timesteps 5304.
Path 308 | total_timesteps 5326.
Path 309 | total_timesteps 5363.
Path 310 | total_timesteps 5375.
Path 311 | total_timesteps 5386.
Path 312 | total_timesteps 5400.
Path 313 | total_timesteps 5417.
Path 314 | total_timesteps 5440.
Path 315 | total_timesteps 5457.
Path 316 | total_timesteps 5490.
Path 317 | total_timesteps 5513.
Path 318 | total_timesteps 5527.
Path 319 | total_timesteps 5535.
Path 320 | total_timesteps 5563.
Path 321 | total_timesteps 5575.
Path 322 | total_timesteps 5589.
Path 323 | total_timesteps 5599.
Path 324 | total_timesteps 5613.
Path 325 | total_timesteps 5624.
Path 326 | total_timesteps 5641.
Path 327 | total_timesteps 5653.
Path 328 | total_timesteps 5670.
Path 329 | total_timesteps 5681.
Path 330 | total_timesteps 5691.
Path 331 | total_timesteps 5729.
Path 332 | total_timesteps 5747.
Path 333 | total_timesteps 5767.
Path 334 | total_timesteps 5791.
Path 335 | total_timesteps 5805.
Path 336 | total_timesteps 5827.
Path 337 | total_timesteps 5842.
Path 338 | total_timesteps 5867.
Path 339 | total_timesteps 5880.
Path 340 | total_timesteps 5892.
Path 341 | total_timesteps 5909.
Path 342 | total_timesteps 5936.
Path 343 | total_timesteps 5951.
Path 344 | total_timesteps 5974.
Path 345 | total_timesteps 5986.
Path 346 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.55    |
| Iteration     | 21       |
| MaximumReturn | 7.48     |
| MinimumReturn | -19.8    |
| TotalSamples  | 92144    |
----------------------------
itr #22 | 
Fitting dynamics.
Validation loss = 0.0032247628550976515
Validation loss = 0.0031078846659511328
Validation loss = 0.0030106795020401478
Validation loss = 0.0035056869965046644
Validation loss = 0.003015283728018403
Validation loss = 0.0031702190171927214
Validation loss = 0.0031022552866488695
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 42.
Path 4 | total_timesteps 60.
Path 5 | total_timesteps 75.
Path 6 | total_timesteps 92.
Path 7 | total_timesteps 104.
Path 8 | total_timesteps 114.
Path 9 | total_timesteps 129.
Path 10 | total_timesteps 148.
Path 11 | total_timesteps 178.
Path 12 | total_timesteps 202.
Path 13 | total_timesteps 237.
Path 14 | total_timesteps 248.
Path 15 | total_timesteps 265.
Path 16 | total_timesteps 295.
Path 17 | total_timesteps 308.
Path 18 | total_timesteps 325.
Path 19 | total_timesteps 337.
Path 20 | total_timesteps 356.
Path 21 | total_timesteps 369.
Path 22 | total_timesteps 378.
Path 23 | total_timesteps 390.
Path 24 | total_timesteps 405.
Path 25 | total_timesteps 419.
Path 26 | total_timesteps 444.
Path 27 | total_timesteps 455.
Path 28 | total_timesteps 473.
Path 29 | total_timesteps 482.
Path 30 | total_timesteps 494.
Path 31 | total_timesteps 507.
Path 32 | total_timesteps 518.
Path 33 | total_timesteps 539.
Path 34 | total_timesteps 561.
Path 35 | total_timesteps 571.
Path 36 | total_timesteps 582.
Path 37 | total_timesteps 599.
Path 38 | total_timesteps 620.
Path 39 | total_timesteps 635.
Path 40 | total_timesteps 652.
Path 41 | total_timesteps 670.
Path 42 | total_timesteps 689.
Path 43 | total_timesteps 705.
Path 44 | total_timesteps 719.
Path 45 | total_timesteps 734.
Path 46 | total_timesteps 742.
Path 47 | total_timesteps 770.
Path 48 | total_timesteps 794.
Path 49 | total_timesteps 812.
Path 50 | total_timesteps 834.
Path 51 | total_timesteps 851.
Path 52 | total_timesteps 870.
Path 53 | total_timesteps 881.
Path 54 | total_timesteps 888.
Path 55 | total_timesteps 917.
Path 56 | total_timesteps 938.
Path 57 | total_timesteps 964.
Path 58 | total_timesteps 990.
Path 59 | total_timesteps 1003.
Path 60 | total_timesteps 1010.
Path 61 | total_timesteps 1023.
Path 62 | total_timesteps 1049.
Path 63 | total_timesteps 1071.
Path 64 | total_timesteps 1084.
Path 65 | total_timesteps 1096.
Path 66 | total_timesteps 1108.
Path 67 | total_timesteps 1126.
Path 68 | total_timesteps 1148.
Path 69 | total_timesteps 1164.
Path 70 | total_timesteps 1172.
Path 71 | total_timesteps 1193.
Path 72 | total_timesteps 1208.
Path 73 | total_timesteps 1220.
Path 74 | total_timesteps 1234.
Path 75 | total_timesteps 1249.
Path 76 | total_timesteps 1274.
Path 77 | total_timesteps 1288.
Path 78 | total_timesteps 1309.
Path 79 | total_timesteps 1339.
Path 80 | total_timesteps 1357.
Path 81 | total_timesteps 1388.
Path 82 | total_timesteps 1405.
Path 83 | total_timesteps 1431.
Path 84 | total_timesteps 1442.
Path 85 | total_timesteps 1456.
Path 86 | total_timesteps 1473.
Path 87 | total_timesteps 1481.
Path 88 | total_timesteps 1491.
Path 89 | total_timesteps 1515.
Path 90 | total_timesteps 1538.
Path 91 | total_timesteps 1548.
Path 92 | total_timesteps 1562.
Path 93 | total_timesteps 1576.
Path 94 | total_timesteps 1593.
Path 95 | total_timesteps 1619.
Path 96 | total_timesteps 1633.
Path 97 | total_timesteps 1648.
Path 98 | total_timesteps 1690.
Path 99 | total_timesteps 1710.
Path 100 | total_timesteps 1724.
Path 101 | total_timesteps 1739.
Path 102 | total_timesteps 1753.
Path 103 | total_timesteps 1777.
Path 104 | total_timesteps 1790.
Path 105 | total_timesteps 1799.
Path 106 | total_timesteps 1812.
Path 107 | total_timesteps 1824.
Path 108 | total_timesteps 1842.
Path 109 | total_timesteps 1861.
Path 110 | total_timesteps 1879.
Path 111 | total_timesteps 1889.
Path 112 | total_timesteps 1923.
Path 113 | total_timesteps 1950.
Path 114 | total_timesteps 1965.
Path 115 | total_timesteps 1983.
Path 116 | total_timesteps 1998.
Path 117 | total_timesteps 2009.
Path 118 | total_timesteps 2043.
Path 119 | total_timesteps 2058.
Path 120 | total_timesteps 2083.
Path 121 | total_timesteps 2096.
Path 122 | total_timesteps 2108.
Path 123 | total_timesteps 2121.
Path 124 | total_timesteps 2135.
Path 125 | total_timesteps 2148.
Path 126 | total_timesteps 2168.
Path 127 | total_timesteps 2182.
Path 128 | total_timesteps 2194.
Path 129 | total_timesteps 2214.
Path 130 | total_timesteps 2224.
Path 131 | total_timesteps 2247.
Path 132 | total_timesteps 2260.
Path 133 | total_timesteps 2280.
Path 134 | total_timesteps 2294.
Path 135 | total_timesteps 2307.
Path 136 | total_timesteps 2322.
Path 137 | total_timesteps 2330.
Path 138 | total_timesteps 2341.
Path 139 | total_timesteps 2356.
Path 140 | total_timesteps 2387.
Path 141 | total_timesteps 2409.
Path 142 | total_timesteps 2417.
Path 143 | total_timesteps 2427.
Path 144 | total_timesteps 2440.
Path 145 | total_timesteps 2469.
Path 146 | total_timesteps 2477.
Path 147 | total_timesteps 2492.
Path 148 | total_timesteps 2511.
Path 149 | total_timesteps 2521.
Path 150 | total_timesteps 2533.
Path 151 | total_timesteps 2551.
Path 152 | total_timesteps 2564.
Path 153 | total_timesteps 2575.
Path 154 | total_timesteps 2605.
Path 155 | total_timesteps 2626.
Path 156 | total_timesteps 2642.
Path 157 | total_timesteps 2668.
Path 158 | total_timesteps 2698.
Path 159 | total_timesteps 2726.
Path 160 | total_timesteps 2741.
Path 161 | total_timesteps 2753.
Path 162 | total_timesteps 2765.
Path 163 | total_timesteps 2780.
Path 164 | total_timesteps 2792.
Path 165 | total_timesteps 2805.
Path 166 | total_timesteps 2826.
Path 167 | total_timesteps 2855.
Path 168 | total_timesteps 2878.
Path 169 | total_timesteps 2885.
Path 170 | total_timesteps 2894.
Path 171 | total_timesteps 2908.
Path 172 | total_timesteps 2939.
Path 173 | total_timesteps 2951.
Path 174 | total_timesteps 2969.
Path 175 | total_timesteps 2983.
Path 176 | total_timesteps 3001.
Path 177 | total_timesteps 3025.
Path 178 | total_timesteps 3036.
Path 179 | total_timesteps 3052.
Path 180 | total_timesteps 3058.
Path 181 | total_timesteps 3073.
Path 182 | total_timesteps 3083.
Path 183 | total_timesteps 3102.
Path 184 | total_timesteps 3110.
Path 185 | total_timesteps 3127.
Path 186 | total_timesteps 3134.
Path 187 | total_timesteps 3144.
Path 188 | total_timesteps 3172.
Path 189 | total_timesteps 3185.
Path 190 | total_timesteps 3193.
Path 191 | total_timesteps 3223.
Path 192 | total_timesteps 3242.
Path 193 | total_timesteps 3252.
Path 194 | total_timesteps 3267.
Path 195 | total_timesteps 3287.
Path 196 | total_timesteps 3297.
Path 197 | total_timesteps 3312.
Path 198 | total_timesteps 3328.
Path 199 | total_timesteps 3340.
Path 200 | total_timesteps 3358.
Path 201 | total_timesteps 3371.
Path 202 | total_timesteps 3389.
Path 203 | total_timesteps 3398.
Path 204 | total_timesteps 3417.
Path 205 | total_timesteps 3438.
Path 206 | total_timesteps 3445.
Path 207 | total_timesteps 3461.
Path 208 | total_timesteps 3479.
Path 209 | total_timesteps 3492.
Path 210 | total_timesteps 3507.
Path 211 | total_timesteps 3534.
Path 212 | total_timesteps 3568.
Path 213 | total_timesteps 3580.
Path 214 | total_timesteps 3592.
Path 215 | total_timesteps 3605.
Path 216 | total_timesteps 3631.
Path 217 | total_timesteps 3646.
Path 218 | total_timesteps 3657.
Path 219 | total_timesteps 3671.
Path 220 | total_timesteps 3682.
Path 221 | total_timesteps 3712.
Path 222 | total_timesteps 3733.
Path 223 | total_timesteps 3746.
Path 224 | total_timesteps 3765.
Path 225 | total_timesteps 3783.
Path 226 | total_timesteps 3816.
Path 227 | total_timesteps 3834.
Path 228 | total_timesteps 3846.
Path 229 | total_timesteps 3860.
Path 230 | total_timesteps 3884.
Path 231 | total_timesteps 3896.
Path 232 | total_timesteps 3911.
Path 233 | total_timesteps 3923.
Path 234 | total_timesteps 3931.
Path 235 | total_timesteps 3944.
Path 236 | total_timesteps 3956.
Path 237 | total_timesteps 3973.
Path 238 | total_timesteps 3984.
Path 239 | total_timesteps 3996.
Path 240 | total_timesteps 4018.
Path 241 | total_timesteps 4027.
Path 242 | total_timesteps 4039.
Path 243 | total_timesteps 4062.
Path 244 | total_timesteps 4076.
Path 245 | total_timesteps 4096.
Path 246 | total_timesteps 4118.
Path 247 | total_timesteps 4135.
Path 248 | total_timesteps 4151.
Path 249 | total_timesteps 4164.
Path 250 | total_timesteps 4180.
Path 251 | total_timesteps 4198.
Path 252 | total_timesteps 4216.
Path 253 | total_timesteps 4234.
Path 254 | total_timesteps 4249.
Path 255 | total_timesteps 4288.
Path 256 | total_timesteps 4297.
Path 257 | total_timesteps 4317.
Path 258 | total_timesteps 4343.
Path 259 | total_timesteps 4377.
Path 260 | total_timesteps 4395.
Path 261 | total_timesteps 4409.
Path 262 | total_timesteps 4432.
Path 263 | total_timesteps 4441.
Path 264 | total_timesteps 4468.
Path 265 | total_timesteps 4493.
Path 266 | total_timesteps 4502.
Path 267 | total_timesteps 4521.
Path 268 | total_timesteps 4539.
Path 269 | total_timesteps 4570.
Path 270 | total_timesteps 4592.
Path 271 | total_timesteps 4607.
Path 272 | total_timesteps 4618.
Path 273 | total_timesteps 4639.
Path 274 | total_timesteps 4655.
Path 275 | total_timesteps 4674.
Path 276 | total_timesteps 4687.
Path 277 | total_timesteps 4705.
Path 278 | total_timesteps 4721.
Path 279 | total_timesteps 4728.
Path 280 | total_timesteps 4745.
Path 281 | total_timesteps 4760.
Path 282 | total_timesteps 4776.
Path 283 | total_timesteps 4800.
Path 284 | total_timesteps 4810.
Path 285 | total_timesteps 4833.
Path 286 | total_timesteps 4847.
Path 287 | total_timesteps 4858.
Path 288 | total_timesteps 4876.
Path 289 | total_timesteps 4891.
Path 290 | total_timesteps 4923.
Path 291 | total_timesteps 4930.
Path 292 | total_timesteps 4945.
Path 293 | total_timesteps 4953.
Path 294 | total_timesteps 4965.
Path 295 | total_timesteps 4974.
Path 296 | total_timesteps 4988.
Path 297 | total_timesteps 5000.
Path 298 | total_timesteps 5013.
Path 299 | total_timesteps 5030.
Path 300 | total_timesteps 5039.
Path 301 | total_timesteps 5051.
Path 302 | total_timesteps 5066.
Path 303 | total_timesteps 5090.
Path 304 | total_timesteps 5099.
Path 305 | total_timesteps 5125.
Path 306 | total_timesteps 5159.
Path 307 | total_timesteps 5181.
Path 308 | total_timesteps 5188.
Path 309 | total_timesteps 5208.
Path 310 | total_timesteps 5223.
Path 311 | total_timesteps 5264.
Path 312 | total_timesteps 5293.
Path 313 | total_timesteps 5319.
Path 314 | total_timesteps 5328.
Path 315 | total_timesteps 5368.
Path 316 | total_timesteps 5393.
Path 317 | total_timesteps 5409.
Path 318 | total_timesteps 5421.
Path 319 | total_timesteps 5433.
Path 320 | total_timesteps 5451.
Path 321 | total_timesteps 5459.
Path 322 | total_timesteps 5476.
Path 323 | total_timesteps 5491.
Path 324 | total_timesteps 5501.
Path 325 | total_timesteps 5513.
Path 326 | total_timesteps 5528.
Path 327 | total_timesteps 5544.
Path 328 | total_timesteps 5558.
Path 329 | total_timesteps 5588.
Path 330 | total_timesteps 5602.
Path 331 | total_timesteps 5610.
Path 332 | total_timesteps 5629.
Path 333 | total_timesteps 5640.
Path 334 | total_timesteps 5659.
Path 335 | total_timesteps 5697.
Path 336 | total_timesteps 5710.
Path 337 | total_timesteps 5727.
Path 338 | total_timesteps 5748.
Path 339 | total_timesteps 5763.
Path 340 | total_timesteps 5779.
Path 341 | total_timesteps 5800.
Path 342 | total_timesteps 5824.
Path 343 | total_timesteps 5841.
Path 344 | total_timesteps 5870.
Path 345 | total_timesteps 5893.
Path 346 | total_timesteps 5915.
Path 347 | total_timesteps 5935.
Path 348 | total_timesteps 5951.
Path 349 | total_timesteps 5965.
Path 350 | total_timesteps 5982.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.59    |
| Iteration     | 22       |
| MaximumReturn | 10       |
| MinimumReturn | -18.8    |
| TotalSamples  | 96144    |
----------------------------
itr #23 | 
Fitting dynamics.
Validation loss = 0.0032346805091947317
Validation loss = 0.002724358579143882
Validation loss = 0.0030397213995456696
Validation loss = 0.0028826501220464706
Validation loss = 0.002764863893389702
Validation loss = 0.002804998541250825
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 51.
Path 4 | total_timesteps 68.
Path 5 | total_timesteps 83.
Path 6 | total_timesteps 105.
Path 7 | total_timesteps 117.
Path 8 | total_timesteps 130.
Path 9 | total_timesteps 146.
Path 10 | total_timesteps 168.
Path 11 | total_timesteps 183.
Path 12 | total_timesteps 196.
Path 13 | total_timesteps 249.
Path 14 | total_timesteps 273.
Path 15 | total_timesteps 288.
Path 16 | total_timesteps 299.
Path 17 | total_timesteps 326.
Path 18 | total_timesteps 343.
Path 19 | total_timesteps 366.
Path 20 | total_timesteps 387.
Path 21 | total_timesteps 416.
Path 22 | total_timesteps 424.
Path 23 | total_timesteps 443.
Path 24 | total_timesteps 457.
Path 25 | total_timesteps 475.
Path 26 | total_timesteps 495.
Path 27 | total_timesteps 531.
Path 28 | total_timesteps 550.
Path 29 | total_timesteps 566.
Path 30 | total_timesteps 575.
Path 31 | total_timesteps 594.
Path 32 | total_timesteps 607.
Path 33 | total_timesteps 619.
Path 34 | total_timesteps 635.
Path 35 | total_timesteps 643.
Path 36 | total_timesteps 675.
Path 37 | total_timesteps 690.
Path 38 | total_timesteps 723.
Path 39 | total_timesteps 739.
Path 40 | total_timesteps 760.
Path 41 | total_timesteps 769.
Path 42 | total_timesteps 781.
Path 43 | total_timesteps 791.
Path 44 | total_timesteps 819.
Path 45 | total_timesteps 851.
Path 46 | total_timesteps 863.
Path 47 | total_timesteps 878.
Path 48 | total_timesteps 888.
Path 49 | total_timesteps 904.
Path 50 | total_timesteps 934.
Path 51 | total_timesteps 946.
Path 52 | total_timesteps 956.
Path 53 | total_timesteps 973.
Path 54 | total_timesteps 994.
Path 55 | total_timesteps 1018.
Path 56 | total_timesteps 1046.
Path 57 | total_timesteps 1065.
Path 58 | total_timesteps 1078.
Path 59 | total_timesteps 1093.
Path 60 | total_timesteps 1112.
Path 61 | total_timesteps 1120.
Path 62 | total_timesteps 1148.
Path 63 | total_timesteps 1164.
Path 64 | total_timesteps 1181.
Path 65 | total_timesteps 1194.
Path 66 | total_timesteps 1204.
Path 67 | total_timesteps 1233.
Path 68 | total_timesteps 1242.
Path 69 | total_timesteps 1251.
Path 70 | total_timesteps 1268.
Path 71 | total_timesteps 1281.
Path 72 | total_timesteps 1296.
Path 73 | total_timesteps 1312.
Path 74 | total_timesteps 1331.
Path 75 | total_timesteps 1347.
Path 76 | total_timesteps 1372.
Path 77 | total_timesteps 1384.
Path 78 | total_timesteps 1403.
Path 79 | total_timesteps 1413.
Path 80 | total_timesteps 1447.
Path 81 | total_timesteps 1468.
Path 82 | total_timesteps 1480.
Path 83 | total_timesteps 1490.
Path 84 | total_timesteps 1498.
Path 85 | total_timesteps 1524.
Path 86 | total_timesteps 1547.
Path 87 | total_timesteps 1573.
Path 88 | total_timesteps 1602.
Path 89 | total_timesteps 1618.
Path 90 | total_timesteps 1643.
Path 91 | total_timesteps 1667.
Path 92 | total_timesteps 1683.
Path 93 | total_timesteps 1696.
Path 94 | total_timesteps 1731.
Path 95 | total_timesteps 1743.
Path 96 | total_timesteps 1772.
Path 97 | total_timesteps 1789.
Path 98 | total_timesteps 1816.
Path 99 | total_timesteps 1829.
Path 100 | total_timesteps 1849.
Path 101 | total_timesteps 1866.
Path 102 | total_timesteps 1883.
Path 103 | total_timesteps 1898.
Path 104 | total_timesteps 1914.
Path 105 | total_timesteps 1922.
Path 106 | total_timesteps 1939.
Path 107 | total_timesteps 1952.
Path 108 | total_timesteps 1968.
Path 109 | total_timesteps 1986.
Path 110 | total_timesteps 2025.
Path 111 | total_timesteps 2039.
Path 112 | total_timesteps 2050.
Path 113 | total_timesteps 2077.
Path 114 | total_timesteps 2093.
Path 115 | total_timesteps 2108.
Path 116 | total_timesteps 2131.
Path 117 | total_timesteps 2153.
Path 118 | total_timesteps 2176.
Path 119 | total_timesteps 2200.
Path 120 | total_timesteps 2227.
Path 121 | total_timesteps 2240.
Path 122 | total_timesteps 2259.
Path 123 | total_timesteps 2276.
Path 124 | total_timesteps 2306.
Path 125 | total_timesteps 2318.
Path 126 | total_timesteps 2335.
Path 127 | total_timesteps 2350.
Path 128 | total_timesteps 2368.
Path 129 | total_timesteps 2377.
Path 130 | total_timesteps 2399.
Path 131 | total_timesteps 2416.
Path 132 | total_timesteps 2423.
Path 133 | total_timesteps 2431.
Path 134 | total_timesteps 2442.
Path 135 | total_timesteps 2451.
Path 136 | total_timesteps 2461.
Path 137 | total_timesteps 2475.
Path 138 | total_timesteps 2498.
Path 139 | total_timesteps 2515.
Path 140 | total_timesteps 2534.
Path 141 | total_timesteps 2560.
Path 142 | total_timesteps 2580.
Path 143 | total_timesteps 2590.
Path 144 | total_timesteps 2610.
Path 145 | total_timesteps 2620.
Path 146 | total_timesteps 2630.
Path 147 | total_timesteps 2655.
Path 148 | total_timesteps 2681.
Path 149 | total_timesteps 2694.
Path 150 | total_timesteps 2710.
Path 151 | total_timesteps 2722.
Path 152 | total_timesteps 2745.
Path 153 | total_timesteps 2768.
Path 154 | total_timesteps 2783.
Path 155 | total_timesteps 2803.
Path 156 | total_timesteps 2818.
Path 157 | total_timesteps 2830.
Path 158 | total_timesteps 2847.
Path 159 | total_timesteps 2873.
Path 160 | total_timesteps 2890.
Path 161 | total_timesteps 2903.
Path 162 | total_timesteps 2914.
Path 163 | total_timesteps 2935.
Path 164 | total_timesteps 2965.
Path 165 | total_timesteps 2989.
Path 166 | total_timesteps 3025.
Path 167 | total_timesteps 3033.
Path 168 | total_timesteps 3046.
Path 169 | total_timesteps 3061.
Path 170 | total_timesteps 3078.
Path 171 | total_timesteps 3089.
Path 172 | total_timesteps 3104.
Path 173 | total_timesteps 3116.
Path 174 | total_timesteps 3127.
Path 175 | total_timesteps 3138.
Path 176 | total_timesteps 3147.
Path 177 | total_timesteps 3173.
Path 178 | total_timesteps 3206.
Path 179 | total_timesteps 3221.
Path 180 | total_timesteps 3245.
Path 181 | total_timesteps 3266.
Path 182 | total_timesteps 3295.
Path 183 | total_timesteps 3309.
Path 184 | total_timesteps 3336.
Path 185 | total_timesteps 3344.
Path 186 | total_timesteps 3356.
Path 187 | total_timesteps 3365.
Path 188 | total_timesteps 3375.
Path 189 | total_timesteps 3389.
Path 190 | total_timesteps 3401.
Path 191 | total_timesteps 3420.
Path 192 | total_timesteps 3428.
Path 193 | total_timesteps 3439.
Path 194 | total_timesteps 3466.
Path 195 | total_timesteps 3488.
Path 196 | total_timesteps 3522.
Path 197 | total_timesteps 3543.
Path 198 | total_timesteps 3566.
Path 199 | total_timesteps 3580.
Path 200 | total_timesteps 3600.
Path 201 | total_timesteps 3618.
Path 202 | total_timesteps 3628.
Path 203 | total_timesteps 3637.
Path 204 | total_timesteps 3650.
Path 205 | total_timesteps 3661.
Path 206 | total_timesteps 3673.
Path 207 | total_timesteps 3692.
Path 208 | total_timesteps 3704.
Path 209 | total_timesteps 3731.
Path 210 | total_timesteps 3753.
Path 211 | total_timesteps 3767.
Path 212 | total_timesteps 3809.
Path 213 | total_timesteps 3824.
Path 214 | total_timesteps 3842.
Path 215 | total_timesteps 3853.
Path 216 | total_timesteps 3882.
Path 217 | total_timesteps 3905.
Path 218 | total_timesteps 3923.
Path 219 | total_timesteps 3933.
Path 220 | total_timesteps 3953.
Path 221 | total_timesteps 3968.
Path 222 | total_timesteps 3983.
Path 223 | total_timesteps 4005.
Path 224 | total_timesteps 4022.
Path 225 | total_timesteps 4030.
Path 226 | total_timesteps 4045.
Path 227 | total_timesteps 4067.
Path 228 | total_timesteps 4086.
Path 229 | total_timesteps 4115.
Path 230 | total_timesteps 4140.
Path 231 | total_timesteps 4164.
Path 232 | total_timesteps 4191.
Path 233 | total_timesteps 4205.
Path 234 | total_timesteps 4217.
Path 235 | total_timesteps 4226.
Path 236 | total_timesteps 4251.
Path 237 | total_timesteps 4277.
Path 238 | total_timesteps 4315.
Path 239 | total_timesteps 4324.
Path 240 | total_timesteps 4335.
Path 241 | total_timesteps 4351.
Path 242 | total_timesteps 4360.
Path 243 | total_timesteps 4389.
Path 244 | total_timesteps 4397.
Path 245 | total_timesteps 4429.
Path 246 | total_timesteps 4444.
Path 247 | total_timesteps 4459.
Path 248 | total_timesteps 4478.
Path 249 | total_timesteps 4488.
Path 250 | total_timesteps 4520.
Path 251 | total_timesteps 4536.
Path 252 | total_timesteps 4556.
Path 253 | total_timesteps 4573.
Path 254 | total_timesteps 4588.
Path 255 | total_timesteps 4616.
Path 256 | total_timesteps 4634.
Path 257 | total_timesteps 4647.
Path 258 | total_timesteps 4664.
Path 259 | total_timesteps 4688.
Path 260 | total_timesteps 4702.
Path 261 | total_timesteps 4722.
Path 262 | total_timesteps 4736.
Path 263 | total_timesteps 4747.
Path 264 | total_timesteps 4760.
Path 265 | total_timesteps 4773.
Path 266 | total_timesteps 4785.
Path 267 | total_timesteps 4810.
Path 268 | total_timesteps 4822.
Path 269 | total_timesteps 4832.
Path 270 | total_timesteps 4848.
Path 271 | total_timesteps 4861.
Path 272 | total_timesteps 4883.
Path 273 | total_timesteps 4908.
Path 274 | total_timesteps 4923.
Path 275 | total_timesteps 4940.
Path 276 | total_timesteps 4958.
Path 277 | total_timesteps 4983.
Path 278 | total_timesteps 5009.
Path 279 | total_timesteps 5022.
Path 280 | total_timesteps 5034.
Path 281 | total_timesteps 5044.
Path 282 | total_timesteps 5076.
Path 283 | total_timesteps 5090.
Path 284 | total_timesteps 5104.
Path 285 | total_timesteps 5118.
Path 286 | total_timesteps 5131.
Path 287 | total_timesteps 5152.
Path 288 | total_timesteps 5191.
Path 289 | total_timesteps 5209.
Path 290 | total_timesteps 5223.
Path 291 | total_timesteps 5237.
Path 292 | total_timesteps 5260.
Path 293 | total_timesteps 5287.
Path 294 | total_timesteps 5303.
Path 295 | total_timesteps 5319.
Path 296 | total_timesteps 5334.
Path 297 | total_timesteps 5350.
Path 298 | total_timesteps 5361.
Path 299 | total_timesteps 5374.
Path 300 | total_timesteps 5391.
Path 301 | total_timesteps 5401.
Path 302 | total_timesteps 5409.
Path 303 | total_timesteps 5437.
Path 304 | total_timesteps 5455.
Path 305 | total_timesteps 5477.
Path 306 | total_timesteps 5495.
Path 307 | total_timesteps 5509.
Path 308 | total_timesteps 5526.
Path 309 | total_timesteps 5553.
Path 310 | total_timesteps 5571.
Path 311 | total_timesteps 5601.
Path 312 | total_timesteps 5621.
Path 313 | total_timesteps 5640.
Path 314 | total_timesteps 5646.
Path 315 | total_timesteps 5659.
Path 316 | total_timesteps 5687.
Path 317 | total_timesteps 5699.
Path 318 | total_timesteps 5726.
Path 319 | total_timesteps 5739.
Path 320 | total_timesteps 5767.
Path 321 | total_timesteps 5784.
Path 322 | total_timesteps 5802.
Path 323 | total_timesteps 5818.
Path 324 | total_timesteps 5832.
Path 325 | total_timesteps 5848.
Path 326 | total_timesteps 5866.
Path 327 | total_timesteps 5885.
Path 328 | total_timesteps 5894.
Path 329 | total_timesteps 5905.
Path 330 | total_timesteps 5923.
Path 331 | total_timesteps 5947.
Path 332 | total_timesteps 5961.
Path 333 | total_timesteps 5976.
Path 334 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.56    |
| Iteration     | 23       |
| MaximumReturn | 8.19     |
| MinimumReturn | -20.3    |
| TotalSamples  | 100145   |
----------------------------
itr #24 | 
Fitting dynamics.
Validation loss = 0.0027839667163789272
Validation loss = 0.00282284221611917
Validation loss = 0.0029361650813370943
Validation loss = 0.0028093743603676558
Validation loss = 0.0029844106175005436
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 23.
Path 2 | total_timesteps 45.
Path 3 | total_timesteps 72.
Path 4 | total_timesteps 90.
Path 5 | total_timesteps 103.
Path 6 | total_timesteps 118.
Path 7 | total_timesteps 132.
Path 8 | total_timesteps 147.
Path 9 | total_timesteps 158.
Path 10 | total_timesteps 183.
Path 11 | total_timesteps 194.
Path 12 | total_timesteps 210.
Path 13 | total_timesteps 221.
Path 14 | total_timesteps 241.
Path 15 | total_timesteps 260.
Path 16 | total_timesteps 284.
Path 17 | total_timesteps 296.
Path 18 | total_timesteps 310.
Path 19 | total_timesteps 334.
Path 20 | total_timesteps 347.
Path 21 | total_timesteps 366.
Path 22 | total_timesteps 374.
Path 23 | total_timesteps 385.
Path 24 | total_timesteps 394.
Path 25 | total_timesteps 407.
Path 26 | total_timesteps 421.
Path 27 | total_timesteps 442.
Path 28 | total_timesteps 450.
Path 29 | total_timesteps 464.
Path 30 | total_timesteps 495.
Path 31 | total_timesteps 513.
Path 32 | total_timesteps 525.
Path 33 | total_timesteps 541.
Path 34 | total_timesteps 556.
Path 35 | total_timesteps 572.
Path 36 | total_timesteps 592.
Path 37 | total_timesteps 605.
Path 38 | total_timesteps 630.
Path 39 | total_timesteps 648.
Path 40 | total_timesteps 661.
Path 41 | total_timesteps 681.
Path 42 | total_timesteps 696.
Path 43 | total_timesteps 708.
Path 44 | total_timesteps 727.
Path 45 | total_timesteps 745.
Path 46 | total_timesteps 757.
Path 47 | total_timesteps 771.
Path 48 | total_timesteps 781.
Path 49 | total_timesteps 798.
Path 50 | total_timesteps 810.
Path 51 | total_timesteps 834.
Path 52 | total_timesteps 853.
Path 53 | total_timesteps 871.
Path 54 | total_timesteps 887.
Path 55 | total_timesteps 901.
Path 56 | total_timesteps 916.
Path 57 | total_timesteps 931.
Path 58 | total_timesteps 942.
Path 59 | total_timesteps 954.
Path 60 | total_timesteps 966.
Path 61 | total_timesteps 975.
Path 62 | total_timesteps 1010.
Path 63 | total_timesteps 1025.
Path 64 | total_timesteps 1060.
Path 65 | total_timesteps 1074.
Path 66 | total_timesteps 1099.
Path 67 | total_timesteps 1107.
Path 68 | total_timesteps 1120.
Path 69 | total_timesteps 1135.
Path 70 | total_timesteps 1155.
Path 71 | total_timesteps 1164.
Path 72 | total_timesteps 1179.
Path 73 | total_timesteps 1196.
Path 74 | total_timesteps 1209.
Path 75 | total_timesteps 1224.
Path 76 | total_timesteps 1233.
Path 77 | total_timesteps 1242.
Path 78 | total_timesteps 1268.
Path 79 | total_timesteps 1289.
Path 80 | total_timesteps 1307.
Path 81 | total_timesteps 1361.
Path 82 | total_timesteps 1390.
Path 83 | total_timesteps 1408.
Path 84 | total_timesteps 1446.
Path 85 | total_timesteps 1466.
Path 86 | total_timesteps 1502.
Path 87 | total_timesteps 1513.
Path 88 | total_timesteps 1526.
Path 89 | total_timesteps 1536.
Path 90 | total_timesteps 1555.
Path 91 | total_timesteps 1591.
Path 92 | total_timesteps 1604.
Path 93 | total_timesteps 1616.
Path 94 | total_timesteps 1636.
Path 95 | total_timesteps 1648.
Path 96 | total_timesteps 1667.
Path 97 | total_timesteps 1697.
Path 98 | total_timesteps 1716.
Path 99 | total_timesteps 1749.
Path 100 | total_timesteps 1764.
Path 101 | total_timesteps 1787.
Path 102 | total_timesteps 1799.
Path 103 | total_timesteps 1825.
Path 104 | total_timesteps 1842.
Path 105 | total_timesteps 1866.
Path 106 | total_timesteps 1884.
Path 107 | total_timesteps 1902.
Path 108 | total_timesteps 1917.
Path 109 | total_timesteps 1949.
Path 110 | total_timesteps 1960.
Path 111 | total_timesteps 1973.
Path 112 | total_timesteps 1987.
Path 113 | total_timesteps 2009.
Path 114 | total_timesteps 2027.
Path 115 | total_timesteps 2038.
Path 116 | total_timesteps 2061.
Path 117 | total_timesteps 2069.
Path 118 | total_timesteps 2086.
Path 119 | total_timesteps 2106.
Path 120 | total_timesteps 2117.
Path 121 | total_timesteps 2131.
Path 122 | total_timesteps 2148.
Path 123 | total_timesteps 2173.
Path 124 | total_timesteps 2191.
Path 125 | total_timesteps 2211.
Path 126 | total_timesteps 2221.
Path 127 | total_timesteps 2236.
Path 128 | total_timesteps 2247.
Path 129 | total_timesteps 2264.
Path 130 | total_timesteps 2277.
Path 131 | total_timesteps 2290.
Path 132 | total_timesteps 2308.
Path 133 | total_timesteps 2317.
Path 134 | total_timesteps 2334.
Path 135 | total_timesteps 2346.
Path 136 | total_timesteps 2365.
Path 137 | total_timesteps 2379.
Path 138 | total_timesteps 2398.
Path 139 | total_timesteps 2424.
Path 140 | total_timesteps 2439.
Path 141 | total_timesteps 2447.
Path 142 | total_timesteps 2466.
Path 143 | total_timesteps 2484.
Path 144 | total_timesteps 2493.
Path 145 | total_timesteps 2517.
Path 146 | total_timesteps 2533.
Path 147 | total_timesteps 2566.
Path 148 | total_timesteps 2575.
Path 149 | total_timesteps 2599.
Path 150 | total_timesteps 2612.
Path 151 | total_timesteps 2622.
Path 152 | total_timesteps 2641.
Path 153 | total_timesteps 2648.
Path 154 | total_timesteps 2662.
Path 155 | total_timesteps 2671.
Path 156 | total_timesteps 2694.
Path 157 | total_timesteps 2711.
Path 158 | total_timesteps 2721.
Path 159 | total_timesteps 2751.
Path 160 | total_timesteps 2770.
Path 161 | total_timesteps 2795.
Path 162 | total_timesteps 2808.
Path 163 | total_timesteps 2828.
Path 164 | total_timesteps 2838.
Path 165 | total_timesteps 2858.
Path 166 | total_timesteps 2876.
Path 167 | total_timesteps 2898.
Path 168 | total_timesteps 2906.
Path 169 | total_timesteps 2929.
Path 170 | total_timesteps 2952.
Path 171 | total_timesteps 2967.
Path 172 | total_timesteps 2998.
Path 173 | total_timesteps 3014.
Path 174 | total_timesteps 3037.
Path 175 | total_timesteps 3051.
Path 176 | total_timesteps 3078.
Path 177 | total_timesteps 3093.
Path 178 | total_timesteps 3112.
Path 179 | total_timesteps 3124.
Path 180 | total_timesteps 3139.
Path 181 | total_timesteps 3153.
Path 182 | total_timesteps 3169.
Path 183 | total_timesteps 3182.
Path 184 | total_timesteps 3202.
Path 185 | total_timesteps 3218.
Path 186 | total_timesteps 3232.
Path 187 | total_timesteps 3254.
Path 188 | total_timesteps 3267.
Path 189 | total_timesteps 3285.
Path 190 | total_timesteps 3305.
Path 191 | total_timesteps 3323.
Path 192 | total_timesteps 3333.
Path 193 | total_timesteps 3349.
Path 194 | total_timesteps 3361.
Path 195 | total_timesteps 3369.
Path 196 | total_timesteps 3388.
Path 197 | total_timesteps 3401.
Path 198 | total_timesteps 3420.
Path 199 | total_timesteps 3432.
Path 200 | total_timesteps 3443.
Path 201 | total_timesteps 3454.
Path 202 | total_timesteps 3476.
Path 203 | total_timesteps 3504.
Path 204 | total_timesteps 3517.
Path 205 | total_timesteps 3539.
Path 206 | total_timesteps 3548.
Path 207 | total_timesteps 3575.
Path 208 | total_timesteps 3583.
Path 209 | total_timesteps 3607.
Path 210 | total_timesteps 3619.
Path 211 | total_timesteps 3649.
Path 212 | total_timesteps 3664.
Path 213 | total_timesteps 3676.
Path 214 | total_timesteps 3685.
Path 215 | total_timesteps 3708.
Path 216 | total_timesteps 3731.
Path 217 | total_timesteps 3764.
Path 218 | total_timesteps 3777.
Path 219 | total_timesteps 3791.
Path 220 | total_timesteps 3818.
Path 221 | total_timesteps 3831.
Path 222 | total_timesteps 3856.
Path 223 | total_timesteps 3868.
Path 224 | total_timesteps 3884.
Path 225 | total_timesteps 3897.
Path 226 | total_timesteps 3906.
Path 227 | total_timesteps 3918.
Path 228 | total_timesteps 3938.
Path 229 | total_timesteps 3949.
Path 230 | total_timesteps 3971.
Path 231 | total_timesteps 3980.
Path 232 | total_timesteps 4004.
Path 233 | total_timesteps 4015.
Path 234 | total_timesteps 4023.
Path 235 | total_timesteps 4045.
Path 236 | total_timesteps 4053.
Path 237 | total_timesteps 4079.
Path 238 | total_timesteps 4089.
Path 239 | total_timesteps 4105.
Path 240 | total_timesteps 4143.
Path 241 | total_timesteps 4163.
Path 242 | total_timesteps 4185.
Path 243 | total_timesteps 4204.
Path 244 | total_timesteps 4213.
Path 245 | total_timesteps 4225.
Path 246 | total_timesteps 4238.
Path 247 | total_timesteps 4247.
Path 248 | total_timesteps 4266.
Path 249 | total_timesteps 4276.
Path 250 | total_timesteps 4286.
Path 251 | total_timesteps 4310.
Path 252 | total_timesteps 4330.
Path 253 | total_timesteps 4356.
Path 254 | total_timesteps 4368.
Path 255 | total_timesteps 4384.
Path 256 | total_timesteps 4402.
Path 257 | total_timesteps 4415.
Path 258 | total_timesteps 4425.
Path 259 | total_timesteps 4434.
Path 260 | total_timesteps 4449.
Path 261 | total_timesteps 4457.
Path 262 | total_timesteps 4480.
Path 263 | total_timesteps 4500.
Path 264 | total_timesteps 4511.
Path 265 | total_timesteps 4535.
Path 266 | total_timesteps 4555.
Path 267 | total_timesteps 4567.
Path 268 | total_timesteps 4577.
Path 269 | total_timesteps 4584.
Path 270 | total_timesteps 4602.
Path 271 | total_timesteps 4612.
Path 272 | total_timesteps 4621.
Path 273 | total_timesteps 4634.
Path 274 | total_timesteps 4664.
Path 275 | total_timesteps 4676.
Path 276 | total_timesteps 4690.
Path 277 | total_timesteps 4700.
Path 278 | total_timesteps 4727.
Path 279 | total_timesteps 4740.
Path 280 | total_timesteps 4760.
Path 281 | total_timesteps 4779.
Path 282 | total_timesteps 4790.
Path 283 | total_timesteps 4818.
Path 284 | total_timesteps 4838.
Path 285 | total_timesteps 4852.
Path 286 | total_timesteps 4861.
Path 287 | total_timesteps 4883.
Path 288 | total_timesteps 4895.
Path 289 | total_timesteps 4915.
Path 290 | total_timesteps 4933.
Path 291 | total_timesteps 4968.
Path 292 | total_timesteps 4987.
Path 293 | total_timesteps 5006.
Path 294 | total_timesteps 5023.
Path 295 | total_timesteps 5037.
Path 296 | total_timesteps 5056.
Path 297 | total_timesteps 5065.
Path 298 | total_timesteps 5074.
Path 299 | total_timesteps 5095.
Path 300 | total_timesteps 5110.
Path 301 | total_timesteps 5130.
Path 302 | total_timesteps 5143.
Path 303 | total_timesteps 5164.
Path 304 | total_timesteps 5180.
Path 305 | total_timesteps 5207.
Path 306 | total_timesteps 5230.
Path 307 | total_timesteps 5248.
Path 308 | total_timesteps 5258.
Path 309 | total_timesteps 5267.
Path 310 | total_timesteps 5285.
Path 311 | total_timesteps 5301.
Path 312 | total_timesteps 5322.
Path 313 | total_timesteps 5332.
Path 314 | total_timesteps 5358.
Path 315 | total_timesteps 5384.
Path 316 | total_timesteps 5407.
Path 317 | total_timesteps 5437.
Path 318 | total_timesteps 5449.
Path 319 | total_timesteps 5470.
Path 320 | total_timesteps 5487.
Path 321 | total_timesteps 5498.
Path 322 | total_timesteps 5510.
Path 323 | total_timesteps 5531.
Path 324 | total_timesteps 5553.
Path 325 | total_timesteps 5563.
Path 326 | total_timesteps 5605.
Path 327 | total_timesteps 5620.
Path 328 | total_timesteps 5635.
Path 329 | total_timesteps 5654.
Path 330 | total_timesteps 5669.
Path 331 | total_timesteps 5685.
Path 332 | total_timesteps 5698.
Path 333 | total_timesteps 5721.
Path 334 | total_timesteps 5734.
Path 335 | total_timesteps 5750.
Path 336 | total_timesteps 5760.
Path 337 | total_timesteps 5769.
Path 338 | total_timesteps 5790.
Path 339 | total_timesteps 5810.
Path 340 | total_timesteps 5825.
Path 341 | total_timesteps 5846.
Path 342 | total_timesteps 5865.
Path 343 | total_timesteps 5880.
Path 344 | total_timesteps 5892.
Path 345 | total_timesteps 5901.
Path 346 | total_timesteps 5916.
Path 347 | total_timesteps 5941.
Path 348 | total_timesteps 5968.
Path 349 | total_timesteps 5985.
Path 350 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.6     |
| Iteration     | 24       |
| MaximumReturn | 6.15     |
| MinimumReturn | -20.9    |
| TotalSamples  | 104149   |
----------------------------
itr #25 | 
Fitting dynamics.
Validation loss = 0.003015204332768917
Validation loss = 0.0027640622574836016
Validation loss = 0.002723164390772581
Validation loss = 0.0027335691265761852
Validation loss = 0.0026698848232626915
Validation loss = 0.0029566658195108175
Validation loss = 0.0030411568004637957
Validation loss = 0.0028282059356570244
Validation loss = 0.0031978413462638855
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 20.
Path 2 | total_timesteps 30.
Path 3 | total_timesteps 46.
Path 4 | total_timesteps 71.
Path 5 | total_timesteps 84.
Path 6 | total_timesteps 102.
Path 7 | total_timesteps 117.
Path 8 | total_timesteps 125.
Path 9 | total_timesteps 135.
Path 10 | total_timesteps 154.
Path 11 | total_timesteps 188.
Path 12 | total_timesteps 208.
Path 13 | total_timesteps 232.
Path 14 | total_timesteps 250.
Path 15 | total_timesteps 269.
Path 16 | total_timesteps 283.
Path 17 | total_timesteps 296.
Path 18 | total_timesteps 314.
Path 19 | total_timesteps 331.
Path 20 | total_timesteps 357.
Path 21 | total_timesteps 371.
Path 22 | total_timesteps 390.
Path 23 | total_timesteps 401.
Path 24 | total_timesteps 428.
Path 25 | total_timesteps 449.
Path 26 | total_timesteps 458.
Path 27 | total_timesteps 476.
Path 28 | total_timesteps 498.
Path 29 | total_timesteps 514.
Path 30 | total_timesteps 527.
Path 31 | total_timesteps 564.
Path 32 | total_timesteps 583.
Path 33 | total_timesteps 602.
Path 34 | total_timesteps 624.
Path 35 | total_timesteps 635.
Path 36 | total_timesteps 667.
Path 37 | total_timesteps 674.
Path 38 | total_timesteps 694.
Path 39 | total_timesteps 708.
Path 40 | total_timesteps 731.
Path 41 | total_timesteps 768.
Path 42 | total_timesteps 776.
Path 43 | total_timesteps 793.
Path 44 | total_timesteps 806.
Path 45 | total_timesteps 826.
Path 46 | total_timesteps 838.
Path 47 | total_timesteps 848.
Path 48 | total_timesteps 863.
Path 49 | total_timesteps 880.
Path 50 | total_timesteps 893.
Path 51 | total_timesteps 901.
Path 52 | total_timesteps 912.
Path 53 | total_timesteps 930.
Path 54 | total_timesteps 943.
Path 55 | total_timesteps 973.
Path 56 | total_timesteps 987.
Path 57 | total_timesteps 997.
Path 58 | total_timesteps 1007.
Path 59 | total_timesteps 1019.
Path 60 | total_timesteps 1031.
Path 61 | total_timesteps 1052.
Path 62 | total_timesteps 1076.
Path 63 | total_timesteps 1087.
Path 64 | total_timesteps 1108.
Path 65 | total_timesteps 1117.
Path 66 | total_timesteps 1142.
Path 67 | total_timesteps 1165.
Path 68 | total_timesteps 1190.
Path 69 | total_timesteps 1203.
Path 70 | total_timesteps 1220.
Path 71 | total_timesteps 1233.
Path 72 | total_timesteps 1255.
Path 73 | total_timesteps 1263.
Path 74 | total_timesteps 1276.
Path 75 | total_timesteps 1294.
Path 76 | total_timesteps 1311.
Path 77 | total_timesteps 1339.
Path 78 | total_timesteps 1355.
Path 79 | total_timesteps 1370.
Path 80 | total_timesteps 1382.
Path 81 | total_timesteps 1392.
Path 82 | total_timesteps 1450.
Path 83 | total_timesteps 1479.
Path 84 | total_timesteps 1499.
Path 85 | total_timesteps 1512.
Path 86 | total_timesteps 1540.
Path 87 | total_timesteps 1577.
Path 88 | total_timesteps 1614.
Path 89 | total_timesteps 1627.
Path 90 | total_timesteps 1650.
Path 91 | total_timesteps 1665.
Path 92 | total_timesteps 1675.
Path 93 | total_timesteps 1690.
Path 94 | total_timesteps 1711.
Path 95 | total_timesteps 1725.
Path 96 | total_timesteps 1734.
Path 97 | total_timesteps 1746.
Path 98 | total_timesteps 1773.
Path 99 | total_timesteps 1791.
Path 100 | total_timesteps 1803.
Path 101 | total_timesteps 1832.
Path 102 | total_timesteps 1851.
Path 103 | total_timesteps 1868.
Path 104 | total_timesteps 1893.
Path 105 | total_timesteps 1901.
Path 106 | total_timesteps 1916.
Path 107 | total_timesteps 1929.
Path 108 | total_timesteps 1969.
Path 109 | total_timesteps 1978.
Path 110 | total_timesteps 1996.
Path 111 | total_timesteps 2008.
Path 112 | total_timesteps 2020.
Path 113 | total_timesteps 2035.
Path 114 | total_timesteps 2059.
Path 115 | total_timesteps 2071.
Path 116 | total_timesteps 2118.
Path 117 | total_timesteps 2134.
Path 118 | total_timesteps 2158.
Path 119 | total_timesteps 2189.
Path 120 | total_timesteps 2200.
Path 121 | total_timesteps 2209.
Path 122 | total_timesteps 2227.
Path 123 | total_timesteps 2257.
Path 124 | total_timesteps 2273.
Path 125 | total_timesteps 2281.
Path 126 | total_timesteps 2306.
Path 127 | total_timesteps 2325.
Path 128 | total_timesteps 2335.
Path 129 | total_timesteps 2359.
Path 130 | total_timesteps 2380.
Path 131 | total_timesteps 2392.
Path 132 | total_timesteps 2407.
Path 133 | total_timesteps 2420.
Path 134 | total_timesteps 2441.
Path 135 | total_timesteps 2454.
Path 136 | total_timesteps 2465.
Path 137 | total_timesteps 2480.
Path 138 | total_timesteps 2492.
Path 139 | total_timesteps 2501.
Path 140 | total_timesteps 2515.
Path 141 | total_timesteps 2533.
Path 142 | total_timesteps 2566.
Path 143 | total_timesteps 2580.
Path 144 | total_timesteps 2605.
Path 145 | total_timesteps 2638.
Path 146 | total_timesteps 2653.
Path 147 | total_timesteps 2671.
Path 148 | total_timesteps 2680.
Path 149 | total_timesteps 2694.
Path 150 | total_timesteps 2714.
Path 151 | total_timesteps 2734.
Path 152 | total_timesteps 2762.
Path 153 | total_timesteps 2774.
Path 154 | total_timesteps 2787.
Path 155 | total_timesteps 2809.
Path 156 | total_timesteps 2820.
Path 157 | total_timesteps 2831.
Path 158 | total_timesteps 2846.
Path 159 | total_timesteps 2869.
Path 160 | total_timesteps 2884.
Path 161 | total_timesteps 2894.
Path 162 | total_timesteps 2911.
Path 163 | total_timesteps 2921.
Path 164 | total_timesteps 2936.
Path 165 | total_timesteps 2982.
Path 166 | total_timesteps 2999.
Path 167 | total_timesteps 3011.
Path 168 | total_timesteps 3020.
Path 169 | total_timesteps 3045.
Path 170 | total_timesteps 3074.
Path 171 | total_timesteps 3088.
Path 172 | total_timesteps 3106.
Path 173 | total_timesteps 3119.
Path 174 | total_timesteps 3128.
Path 175 | total_timesteps 3142.
Path 176 | total_timesteps 3152.
Path 177 | total_timesteps 3170.
Path 178 | total_timesteps 3184.
Path 179 | total_timesteps 3191.
Path 180 | total_timesteps 3206.
Path 181 | total_timesteps 3236.
Path 182 | total_timesteps 3262.
Path 183 | total_timesteps 3277.
Path 184 | total_timesteps 3316.
Path 185 | total_timesteps 3324.
Path 186 | total_timesteps 3342.
Path 187 | total_timesteps 3364.
Path 188 | total_timesteps 3377.
Path 189 | total_timesteps 3391.
Path 190 | total_timesteps 3414.
Path 191 | total_timesteps 3435.
Path 192 | total_timesteps 3453.
Path 193 | total_timesteps 3463.
Path 194 | total_timesteps 3485.
Path 195 | total_timesteps 3497.
Path 196 | total_timesteps 3518.
Path 197 | total_timesteps 3529.
Path 198 | total_timesteps 3539.
Path 199 | total_timesteps 3550.
Path 200 | total_timesteps 3570.
Path 201 | total_timesteps 3581.
Path 202 | total_timesteps 3616.
Path 203 | total_timesteps 3629.
Path 204 | total_timesteps 3647.
Path 205 | total_timesteps 3665.
Path 206 | total_timesteps 3686.
Path 207 | total_timesteps 3695.
Path 208 | total_timesteps 3709.
Path 209 | total_timesteps 3728.
Path 210 | total_timesteps 3744.
Path 211 | total_timesteps 3763.
Path 212 | total_timesteps 3775.
Path 213 | total_timesteps 3785.
Path 214 | total_timesteps 3804.
Path 215 | total_timesteps 3827.
Path 216 | total_timesteps 3847.
Path 217 | total_timesteps 3856.
Path 218 | total_timesteps 3874.
Path 219 | total_timesteps 3894.
Path 220 | total_timesteps 3922.
Path 221 | total_timesteps 3931.
Path 222 | total_timesteps 3956.
Path 223 | total_timesteps 3980.
Path 224 | total_timesteps 4005.
Path 225 | total_timesteps 4016.
Path 226 | total_timesteps 4042.
Path 227 | total_timesteps 4070.
Path 228 | total_timesteps 4078.
Path 229 | total_timesteps 4094.
Path 230 | total_timesteps 4105.
Path 231 | total_timesteps 4138.
Path 232 | total_timesteps 4151.
Path 233 | total_timesteps 4173.
Path 234 | total_timesteps 4194.
Path 235 | total_timesteps 4207.
Path 236 | total_timesteps 4221.
Path 237 | total_timesteps 4235.
Path 238 | total_timesteps 4248.
Path 239 | total_timesteps 4275.
Path 240 | total_timesteps 4289.
Path 241 | total_timesteps 4299.
Path 242 | total_timesteps 4319.
Path 243 | total_timesteps 4335.
Path 244 | total_timesteps 4357.
Path 245 | total_timesteps 4370.
Path 246 | total_timesteps 4389.
Path 247 | total_timesteps 4404.
Path 248 | total_timesteps 4432.
Path 249 | total_timesteps 4447.
Path 250 | total_timesteps 4465.
Path 251 | total_timesteps 4477.
Path 252 | total_timesteps 4492.
Path 253 | total_timesteps 4515.
Path 254 | total_timesteps 4544.
Path 255 | total_timesteps 4556.
Path 256 | total_timesteps 4576.
Path 257 | total_timesteps 4589.
Path 258 | total_timesteps 4597.
Path 259 | total_timesteps 4605.
Path 260 | total_timesteps 4618.
Path 261 | total_timesteps 4626.
Path 262 | total_timesteps 4649.
Path 263 | total_timesteps 4672.
Path 264 | total_timesteps 4682.
Path 265 | total_timesteps 4717.
Path 266 | total_timesteps 4756.
Path 267 | total_timesteps 4767.
Path 268 | total_timesteps 4781.
Path 269 | total_timesteps 4796.
Path 270 | total_timesteps 4822.
Path 271 | total_timesteps 4836.
Path 272 | total_timesteps 4854.
Path 273 | total_timesteps 4864.
Path 274 | total_timesteps 4877.
Path 275 | total_timesteps 4893.
Path 276 | total_timesteps 4904.
Path 277 | total_timesteps 4923.
Path 278 | total_timesteps 4942.
Path 279 | total_timesteps 4953.
Path 280 | total_timesteps 4968.
Path 281 | total_timesteps 4978.
Path 282 | total_timesteps 4993.
Path 283 | total_timesteps 5013.
Path 284 | total_timesteps 5029.
Path 285 | total_timesteps 5050.
Path 286 | total_timesteps 5062.
Path 287 | total_timesteps 5072.
Path 288 | total_timesteps 5097.
Path 289 | total_timesteps 5131.
Path 290 | total_timesteps 5146.
Path 291 | total_timesteps 5157.
Path 292 | total_timesteps 5168.
Path 293 | total_timesteps 5201.
Path 294 | total_timesteps 5215.
Path 295 | total_timesteps 5230.
Path 296 | total_timesteps 5243.
Path 297 | total_timesteps 5284.
Path 298 | total_timesteps 5297.
Path 299 | total_timesteps 5309.
Path 300 | total_timesteps 5333.
Path 301 | total_timesteps 5348.
Path 302 | total_timesteps 5399.
Path 303 | total_timesteps 5474.
Path 304 | total_timesteps 5485.
Path 305 | total_timesteps 5507.
Path 306 | total_timesteps 5522.
Path 307 | total_timesteps 5535.
Path 308 | total_timesteps 5556.
Path 309 | total_timesteps 5572.
Path 310 | total_timesteps 5587.
Path 311 | total_timesteps 5600.
Path 312 | total_timesteps 5618.
Path 313 | total_timesteps 5631.
Path 314 | total_timesteps 5651.
Path 315 | total_timesteps 5659.
Path 316 | total_timesteps 5686.
Path 317 | total_timesteps 5703.
Path 318 | total_timesteps 5716.
Path 319 | total_timesteps 5749.
Path 320 | total_timesteps 5760.
Path 321 | total_timesteps 5777.
Path 322 | total_timesteps 5791.
Path 323 | total_timesteps 5805.
Path 324 | total_timesteps 5820.
Path 325 | total_timesteps 5836.
Path 326 | total_timesteps 5859.
Path 327 | total_timesteps 5877.
Path 328 | total_timesteps 5894.
Path 329 | total_timesteps 5906.
Path 330 | total_timesteps 5954.
Path 331 | total_timesteps 5971.
Path 332 | total_timesteps 5985.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.62    |
| Iteration     | 25       |
| MaximumReturn | 18.5     |
| MinimumReturn | -19.7    |
| TotalSamples  | 108149   |
----------------------------
itr #26 | 
Fitting dynamics.
Validation loss = 0.0025948083493858576
Validation loss = 0.002796008251607418
Validation loss = 0.002756432630121708
Validation loss = 0.0025215009227395058
Validation loss = 0.0028646583668887615
Validation loss = 0.0026012195739895105
Validation loss = 0.0024587721563875675
Validation loss = 0.0024581854231655598
Validation loss = 0.002530824625864625
Validation loss = 0.0026568693574517965
Validation loss = 0.002520662732422352
Validation loss = 0.002880045445635915
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 40.
Path 4 | total_timesteps 48.
Path 5 | total_timesteps 56.
Path 6 | total_timesteps 82.
Path 7 | total_timesteps 96.
Path 8 | total_timesteps 112.
Path 9 | total_timesteps 121.
Path 10 | total_timesteps 135.
Path 11 | total_timesteps 148.
Path 12 | total_timesteps 166.
Path 13 | total_timesteps 186.
Path 14 | total_timesteps 196.
Path 15 | total_timesteps 223.
Path 16 | total_timesteps 237.
Path 17 | total_timesteps 260.
Path 18 | total_timesteps 268.
Path 19 | total_timesteps 277.
Path 20 | total_timesteps 287.
Path 21 | total_timesteps 300.
Path 22 | total_timesteps 316.
Path 23 | total_timesteps 329.
Path 24 | total_timesteps 343.
Path 25 | total_timesteps 357.
Path 26 | total_timesteps 368.
Path 27 | total_timesteps 385.
Path 28 | total_timesteps 403.
Path 29 | total_timesteps 424.
Path 30 | total_timesteps 438.
Path 31 | total_timesteps 451.
Path 32 | total_timesteps 463.
Path 33 | total_timesteps 470.
Path 34 | total_timesteps 486.
Path 35 | total_timesteps 497.
Path 36 | total_timesteps 509.
Path 37 | total_timesteps 518.
Path 38 | total_timesteps 529.
Path 39 | total_timesteps 535.
Path 40 | total_timesteps 545.
Path 41 | total_timesteps 561.
Path 42 | total_timesteps 573.
Path 43 | total_timesteps 591.
Path 44 | total_timesteps 603.
Path 45 | total_timesteps 611.
Path 46 | total_timesteps 628.
Path 47 | total_timesteps 643.
Path 48 | total_timesteps 652.
Path 49 | total_timesteps 665.
Path 50 | total_timesteps 677.
Path 51 | total_timesteps 692.
Path 52 | total_timesteps 703.
Path 53 | total_timesteps 724.
Path 54 | total_timesteps 746.
Path 55 | total_timesteps 758.
Path 56 | total_timesteps 790.
Path 57 | total_timesteps 814.
Path 58 | total_timesteps 839.
Path 59 | total_timesteps 856.
Path 60 | total_timesteps 887.
Path 61 | total_timesteps 904.
Path 62 | total_timesteps 911.
Path 63 | total_timesteps 923.
Path 64 | total_timesteps 938.
Path 65 | total_timesteps 948.
Path 66 | total_timesteps 960.
Path 67 | total_timesteps 970.
Path 68 | total_timesteps 985.
Path 69 | total_timesteps 1001.
Path 70 | total_timesteps 1015.
Path 71 | total_timesteps 1029.
Path 72 | total_timesteps 1038.
Path 73 | total_timesteps 1048.
Path 74 | total_timesteps 1060.
Path 75 | total_timesteps 1070.
Path 76 | total_timesteps 1080.
Path 77 | total_timesteps 1092.
Path 78 | total_timesteps 1104.
Path 79 | total_timesteps 1132.
Path 80 | total_timesteps 1146.
Path 81 | total_timesteps 1162.
Path 82 | total_timesteps 1175.
Path 83 | total_timesteps 1186.
Path 84 | total_timesteps 1200.
Path 85 | total_timesteps 1209.
Path 86 | total_timesteps 1230.
Path 87 | total_timesteps 1246.
Path 88 | total_timesteps 1261.
Path 89 | total_timesteps 1280.
Path 90 | total_timesteps 1304.
Path 91 | total_timesteps 1330.
Path 92 | total_timesteps 1343.
Path 93 | total_timesteps 1355.
Path 94 | total_timesteps 1371.
Path 95 | total_timesteps 1384.
Path 96 | total_timesteps 1392.
Path 97 | total_timesteps 1408.
Path 98 | total_timesteps 1429.
Path 99 | total_timesteps 1441.
Path 100 | total_timesteps 1455.
Path 101 | total_timesteps 1467.
Path 102 | total_timesteps 1484.
Path 103 | total_timesteps 1492.
Path 104 | total_timesteps 1514.
Path 105 | total_timesteps 1535.
Path 106 | total_timesteps 1554.
Path 107 | total_timesteps 1571.
Path 108 | total_timesteps 1594.
Path 109 | total_timesteps 1615.
Path 110 | total_timesteps 1648.
Path 111 | total_timesteps 1659.
Path 112 | total_timesteps 1667.
Path 113 | total_timesteps 1683.
Path 114 | total_timesteps 1692.
Path 115 | total_timesteps 1705.
Path 116 | total_timesteps 1724.
Path 117 | total_timesteps 1740.
Path 118 | total_timesteps 1762.
Path 119 | total_timesteps 1777.
Path 120 | total_timesteps 1805.
Path 121 | total_timesteps 1818.
Path 122 | total_timesteps 1835.
Path 123 | total_timesteps 1849.
Path 124 | total_timesteps 1860.
Path 125 | total_timesteps 1874.
Path 126 | total_timesteps 1889.
Path 127 | total_timesteps 1947.
Path 128 | total_timesteps 1965.
Path 129 | total_timesteps 1979.
Path 130 | total_timesteps 1988.
Path 131 | total_timesteps 1996.
Path 132 | total_timesteps 2018.
Path 133 | total_timesteps 2025.
Path 134 | total_timesteps 2046.
Path 135 | total_timesteps 2056.
Path 136 | total_timesteps 2067.
Path 137 | total_timesteps 2082.
Path 138 | total_timesteps 2101.
Path 139 | total_timesteps 2119.
Path 140 | total_timesteps 2131.
Path 141 | total_timesteps 2149.
Path 142 | total_timesteps 2160.
Path 143 | total_timesteps 2180.
Path 144 | total_timesteps 2211.
Path 145 | total_timesteps 2221.
Path 146 | total_timesteps 2229.
Path 147 | total_timesteps 2241.
Path 148 | total_timesteps 2253.
Path 149 | total_timesteps 2282.
Path 150 | total_timesteps 2303.
Path 151 | total_timesteps 2325.
Path 152 | total_timesteps 2344.
Path 153 | total_timesteps 2357.
Path 154 | total_timesteps 2369.
Path 155 | total_timesteps 2383.
Path 156 | total_timesteps 2393.
Path 157 | total_timesteps 2400.
Path 158 | total_timesteps 2420.
Path 159 | total_timesteps 2428.
Path 160 | total_timesteps 2440.
Path 161 | total_timesteps 2454.
Path 162 | total_timesteps 2467.
Path 163 | total_timesteps 2485.
Path 164 | total_timesteps 2497.
Path 165 | total_timesteps 2514.
Path 166 | total_timesteps 2526.
Path 167 | total_timesteps 2533.
Path 168 | total_timesteps 2554.
Path 169 | total_timesteps 2565.
Path 170 | total_timesteps 2576.
Path 171 | total_timesteps 2601.
Path 172 | total_timesteps 2620.
Path 173 | total_timesteps 2627.
Path 174 | total_timesteps 2647.
Path 175 | total_timesteps 2665.
Path 176 | total_timesteps 2675.
Path 177 | total_timesteps 2687.
Path 178 | total_timesteps 2706.
Path 179 | total_timesteps 2729.
Path 180 | total_timesteps 2736.
Path 181 | total_timesteps 2751.
Path 182 | total_timesteps 2774.
Path 183 | total_timesteps 2789.
Path 184 | total_timesteps 2803.
Path 185 | total_timesteps 2826.
Path 186 | total_timesteps 2838.
Path 187 | total_timesteps 2853.
Path 188 | total_timesteps 2865.
Path 189 | total_timesteps 2896.
Path 190 | total_timesteps 2908.
Path 191 | total_timesteps 2934.
Path 192 | total_timesteps 2947.
Path 193 | total_timesteps 2961.
Path 194 | total_timesteps 2975.
Path 195 | total_timesteps 3001.
Path 196 | total_timesteps 3012.
Path 197 | total_timesteps 3024.
Path 198 | total_timesteps 3043.
Path 199 | total_timesteps 3059.
Path 200 | total_timesteps 3079.
Path 201 | total_timesteps 3091.
Path 202 | total_timesteps 3110.
Path 203 | total_timesteps 3133.
Path 204 | total_timesteps 3147.
Path 205 | total_timesteps 3168.
Path 206 | total_timesteps 3193.
Path 207 | total_timesteps 3205.
Path 208 | total_timesteps 3218.
Path 209 | total_timesteps 3226.
Path 210 | total_timesteps 3235.
Path 211 | total_timesteps 3257.
Path 212 | total_timesteps 3271.
Path 213 | total_timesteps 3291.
Path 214 | total_timesteps 3315.
Path 215 | total_timesteps 3324.
Path 216 | total_timesteps 3333.
Path 217 | total_timesteps 3373.
Path 218 | total_timesteps 3399.
Path 219 | total_timesteps 3413.
Path 220 | total_timesteps 3427.
Path 221 | total_timesteps 3450.
Path 222 | total_timesteps 3459.
Path 223 | total_timesteps 3470.
Path 224 | total_timesteps 3486.
Path 225 | total_timesteps 3512.
Path 226 | total_timesteps 3524.
Path 227 | total_timesteps 3540.
Path 228 | total_timesteps 3553.
Path 229 | total_timesteps 3565.
Path 230 | total_timesteps 3580.
Path 231 | total_timesteps 3595.
Path 232 | total_timesteps 3610.
Path 233 | total_timesteps 3623.
Path 234 | total_timesteps 3633.
Path 235 | total_timesteps 3643.
Path 236 | total_timesteps 3654.
Path 237 | total_timesteps 3670.
Path 238 | total_timesteps 3685.
Path 239 | total_timesteps 3701.
Path 240 | total_timesteps 3713.
Path 241 | total_timesteps 3731.
Path 242 | total_timesteps 3754.
Path 243 | total_timesteps 3773.
Path 244 | total_timesteps 3790.
Path 245 | total_timesteps 3805.
Path 246 | total_timesteps 3819.
Path 247 | total_timesteps 3838.
Path 248 | total_timesteps 3851.
Path 249 | total_timesteps 3865.
Path 250 | total_timesteps 3878.
Path 251 | total_timesteps 3891.
Path 252 | total_timesteps 3901.
Path 253 | total_timesteps 3926.
Path 254 | total_timesteps 3938.
Path 255 | total_timesteps 3952.
Path 256 | total_timesteps 3966.
Path 257 | total_timesteps 3975.
Path 258 | total_timesteps 3994.
Path 259 | total_timesteps 4011.
Path 260 | total_timesteps 4023.
Path 261 | total_timesteps 4035.
Path 262 | total_timesteps 4043.
Path 263 | total_timesteps 4059.
Path 264 | total_timesteps 4070.
Path 265 | total_timesteps 4093.
Path 266 | total_timesteps 4102.
Path 267 | total_timesteps 4114.
Path 268 | total_timesteps 4135.
Path 269 | total_timesteps 4144.
Path 270 | total_timesteps 4157.
Path 271 | total_timesteps 4173.
Path 272 | total_timesteps 4200.
Path 273 | total_timesteps 4224.
Path 274 | total_timesteps 4264.
Path 275 | total_timesteps 4273.
Path 276 | total_timesteps 4286.
Path 277 | total_timesteps 4297.
Path 278 | total_timesteps 4312.
Path 279 | total_timesteps 4330.
Path 280 | total_timesteps 4351.
Path 281 | total_timesteps 4364.
Path 282 | total_timesteps 4382.
Path 283 | total_timesteps 4397.
Path 284 | total_timesteps 4422.
Path 285 | total_timesteps 4445.
Path 286 | total_timesteps 4457.
Path 287 | total_timesteps 4471.
Path 288 | total_timesteps 4480.
Path 289 | total_timesteps 4516.
Path 290 | total_timesteps 4525.
Path 291 | total_timesteps 4536.
Path 292 | total_timesteps 4554.
Path 293 | total_timesteps 4572.
Path 294 | total_timesteps 4594.
Path 295 | total_timesteps 4614.
Path 296 | total_timesteps 4624.
Path 297 | total_timesteps 4633.
Path 298 | total_timesteps 4643.
Path 299 | total_timesteps 4660.
Path 300 | total_timesteps 4670.
Path 301 | total_timesteps 4685.
Path 302 | total_timesteps 4701.
Path 303 | total_timesteps 4715.
Path 304 | total_timesteps 4724.
Path 305 | total_timesteps 4747.
Path 306 | total_timesteps 4759.
Path 307 | total_timesteps 4777.
Path 308 | total_timesteps 4790.
Path 309 | total_timesteps 4799.
Path 310 | total_timesteps 4826.
Path 311 | total_timesteps 4841.
Path 312 | total_timesteps 4855.
Path 313 | total_timesteps 4869.
Path 314 | total_timesteps 4895.
Path 315 | total_timesteps 4934.
Path 316 | total_timesteps 4942.
Path 317 | total_timesteps 4958.
Path 318 | total_timesteps 4969.
Path 319 | total_timesteps 4983.
Path 320 | total_timesteps 5003.
Path 321 | total_timesteps 5015.
Path 322 | total_timesteps 5031.
Path 323 | total_timesteps 5043.
Path 324 | total_timesteps 5056.
Path 325 | total_timesteps 5068.
Path 326 | total_timesteps 5076.
Path 327 | total_timesteps 5124.
Path 328 | total_timesteps 5137.
Path 329 | total_timesteps 5150.
Path 330 | total_timesteps 5166.
Path 331 | total_timesteps 5176.
Path 332 | total_timesteps 5191.
Path 333 | total_timesteps 5209.
Path 334 | total_timesteps 5222.
Path 335 | total_timesteps 5230.
Path 336 | total_timesteps 5243.
Path 337 | total_timesteps 5253.
Path 338 | total_timesteps 5264.
Path 339 | total_timesteps 5283.
Path 340 | total_timesteps 5293.
Path 341 | total_timesteps 5318.
Path 342 | total_timesteps 5335.
Path 343 | total_timesteps 5345.
Path 344 | total_timesteps 5369.
Path 345 | total_timesteps 5384.
Path 346 | total_timesteps 5398.
Path 347 | total_timesteps 5412.
Path 348 | total_timesteps 5424.
Path 349 | total_timesteps 5449.
Path 350 | total_timesteps 5456.
Path 351 | total_timesteps 5471.
Path 352 | total_timesteps 5483.
Path 353 | total_timesteps 5492.
Path 354 | total_timesteps 5510.
Path 355 | total_timesteps 5529.
Path 356 | total_timesteps 5547.
Path 357 | total_timesteps 5573.
Path 358 | total_timesteps 5596.
Path 359 | total_timesteps 5605.
Path 360 | total_timesteps 5617.
Path 361 | total_timesteps 5630.
Path 362 | total_timesteps 5648.
Path 363 | total_timesteps 5666.
Path 364 | total_timesteps 5688.
Path 365 | total_timesteps 5713.
Path 366 | total_timesteps 5728.
Path 367 | total_timesteps 5741.
Path 368 | total_timesteps 5755.
Path 369 | total_timesteps 5777.
Path 370 | total_timesteps 5790.
Path 371 | total_timesteps 5806.
Path 372 | total_timesteps 5816.
Path 373 | total_timesteps 5845.
Path 374 | total_timesteps 5858.
Path 375 | total_timesteps 5869.
Path 376 | total_timesteps 5881.
Path 377 | total_timesteps 5905.
Path 378 | total_timesteps 5931.
Path 379 | total_timesteps 5951.
Path 380 | total_timesteps 5965.
Path 381 | total_timesteps 5987.
Path 382 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.15    |
| Iteration     | 26       |
| MaximumReturn | 8.21     |
| MinimumReturn | -20.1    |
| TotalSamples  | 112155   |
----------------------------
itr #27 | 
Fitting dynamics.
Validation loss = 0.0024103911127895117
Validation loss = 0.002303076209500432
Validation loss = 0.002545755123719573
Validation loss = 0.0025281929410994053
Validation loss = 0.0024888867046684027
Validation loss = 0.002488300669938326
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 52.
Path 4 | total_timesteps 64.
Path 5 | total_timesteps 77.
Path 6 | total_timesteps 85.
Path 7 | total_timesteps 95.
Path 8 | total_timesteps 112.
Path 9 | total_timesteps 128.
Path 10 | total_timesteps 155.
Path 11 | total_timesteps 161.
Path 12 | total_timesteps 188.
Path 13 | total_timesteps 196.
Path 14 | total_timesteps 204.
Path 15 | total_timesteps 218.
Path 16 | total_timesteps 228.
Path 17 | total_timesteps 254.
Path 18 | total_timesteps 262.
Path 19 | total_timesteps 278.
Path 20 | total_timesteps 295.
Path 21 | total_timesteps 303.
Path 22 | total_timesteps 318.
Path 23 | total_timesteps 348.
Path 24 | total_timesteps 359.
Path 25 | total_timesteps 372.
Path 26 | total_timesteps 389.
Path 27 | total_timesteps 403.
Path 28 | total_timesteps 418.
Path 29 | total_timesteps 434.
Path 30 | total_timesteps 448.
Path 31 | total_timesteps 462.
Path 32 | total_timesteps 472.
Path 33 | total_timesteps 491.
Path 34 | total_timesteps 504.
Path 35 | total_timesteps 517.
Path 36 | total_timesteps 532.
Path 37 | total_timesteps 546.
Path 38 | total_timesteps 563.
Path 39 | total_timesteps 572.
Path 40 | total_timesteps 580.
Path 41 | total_timesteps 598.
Path 42 | total_timesteps 612.
Path 43 | total_timesteps 625.
Path 44 | total_timesteps 646.
Path 45 | total_timesteps 658.
Path 46 | total_timesteps 667.
Path 47 | total_timesteps 689.
Path 48 | total_timesteps 703.
Path 49 | total_timesteps 713.
Path 50 | total_timesteps 733.
Path 51 | total_timesteps 752.
Path 52 | total_timesteps 770.
Path 53 | total_timesteps 780.
Path 54 | total_timesteps 789.
Path 55 | total_timesteps 800.
Path 56 | total_timesteps 812.
Path 57 | total_timesteps 826.
Path 58 | total_timesteps 841.
Path 59 | total_timesteps 851.
Path 60 | total_timesteps 878.
Path 61 | total_timesteps 885.
Path 62 | total_timesteps 893.
Path 63 | total_timesteps 909.
Path 64 | total_timesteps 924.
Path 65 | total_timesteps 940.
Path 66 | total_timesteps 953.
Path 67 | total_timesteps 976.
Path 68 | total_timesteps 993.
Path 69 | total_timesteps 1004.
Path 70 | total_timesteps 1024.
Path 71 | total_timesteps 1031.
Path 72 | total_timesteps 1045.
Path 73 | total_timesteps 1052.
Path 74 | total_timesteps 1064.
Path 75 | total_timesteps 1086.
Path 76 | total_timesteps 1094.
Path 77 | total_timesteps 1113.
Path 78 | total_timesteps 1137.
Path 79 | total_timesteps 1146.
Path 80 | total_timesteps 1163.
Path 81 | total_timesteps 1186.
Path 82 | total_timesteps 1207.
Path 83 | total_timesteps 1216.
Path 84 | total_timesteps 1225.
Path 85 | total_timesteps 1234.
Path 86 | total_timesteps 1248.
Path 87 | total_timesteps 1271.
Path 88 | total_timesteps 1280.
Path 89 | total_timesteps 1301.
Path 90 | total_timesteps 1318.
Path 91 | total_timesteps 1335.
Path 92 | total_timesteps 1345.
Path 93 | total_timesteps 1353.
Path 94 | total_timesteps 1366.
Path 95 | total_timesteps 1381.
Path 96 | total_timesteps 1397.
Path 97 | total_timesteps 1408.
Path 98 | total_timesteps 1421.
Path 99 | total_timesteps 1431.
Path 100 | total_timesteps 1459.
Path 101 | total_timesteps 1468.
Path 102 | total_timesteps 1480.
Path 103 | total_timesteps 1490.
Path 104 | total_timesteps 1501.
Path 105 | total_timesteps 1521.
Path 106 | total_timesteps 1528.
Path 107 | total_timesteps 1540.
Path 108 | total_timesteps 1555.
Path 109 | total_timesteps 1571.
Path 110 | total_timesteps 1588.
Path 111 | total_timesteps 1605.
Path 112 | total_timesteps 1615.
Path 113 | total_timesteps 1623.
Path 114 | total_timesteps 1641.
Path 115 | total_timesteps 1664.
Path 116 | total_timesteps 1672.
Path 117 | total_timesteps 1681.
Path 118 | total_timesteps 1700.
Path 119 | total_timesteps 1712.
Path 120 | total_timesteps 1733.
Path 121 | total_timesteps 1743.
Path 122 | total_timesteps 1760.
Path 123 | total_timesteps 1769.
Path 124 | total_timesteps 1782.
Path 125 | total_timesteps 1806.
Path 126 | total_timesteps 1820.
Path 127 | total_timesteps 1845.
Path 128 | total_timesteps 1859.
Path 129 | total_timesteps 1869.
Path 130 | total_timesteps 1884.
Path 131 | total_timesteps 1906.
Path 132 | total_timesteps 1922.
Path 133 | total_timesteps 1933.
Path 134 | total_timesteps 1941.
Path 135 | total_timesteps 1950.
Path 136 | total_timesteps 1969.
Path 137 | total_timesteps 1978.
Path 138 | total_timesteps 1993.
Path 139 | total_timesteps 2002.
Path 140 | total_timesteps 2013.
Path 141 | total_timesteps 2033.
Path 142 | total_timesteps 2041.
Path 143 | total_timesteps 2050.
Path 144 | total_timesteps 2060.
Path 145 | total_timesteps 2103.
Path 146 | total_timesteps 2112.
Path 147 | total_timesteps 2141.
Path 148 | total_timesteps 2152.
Path 149 | total_timesteps 2167.
Path 150 | total_timesteps 2184.
Path 151 | total_timesteps 2198.
Path 152 | total_timesteps 2210.
Path 153 | total_timesteps 2222.
Path 154 | total_timesteps 2233.
Path 155 | total_timesteps 2250.
Path 156 | total_timesteps 2265.
Path 157 | total_timesteps 2301.
Path 158 | total_timesteps 2313.
Path 159 | total_timesteps 2323.
Path 160 | total_timesteps 2345.
Path 161 | total_timesteps 2357.
Path 162 | total_timesteps 2370.
Path 163 | total_timesteps 2386.
Path 164 | total_timesteps 2406.
Path 165 | total_timesteps 2426.
Path 166 | total_timesteps 2447.
Path 167 | total_timesteps 2455.
Path 168 | total_timesteps 2462.
Path 169 | total_timesteps 2473.
Path 170 | total_timesteps 2488.
Path 171 | total_timesteps 2503.
Path 172 | total_timesteps 2527.
Path 173 | total_timesteps 2536.
Path 174 | total_timesteps 2552.
Path 175 | total_timesteps 2571.
Path 176 | total_timesteps 2599.
Path 177 | total_timesteps 2611.
Path 178 | total_timesteps 2630.
Path 179 | total_timesteps 2653.
Path 180 | total_timesteps 2675.
Path 181 | total_timesteps 2687.
Path 182 | total_timesteps 2699.
Path 183 | total_timesteps 2709.
Path 184 | total_timesteps 2726.
Path 185 | total_timesteps 2737.
Path 186 | total_timesteps 2750.
Path 187 | total_timesteps 2760.
Path 188 | total_timesteps 2787.
Path 189 | total_timesteps 2808.
Path 190 | total_timesteps 2820.
Path 191 | total_timesteps 2832.
Path 192 | total_timesteps 2844.
Path 193 | total_timesteps 2861.
Path 194 | total_timesteps 2874.
Path 195 | total_timesteps 2890.
Path 196 | total_timesteps 2905.
Path 197 | total_timesteps 2932.
Path 198 | total_timesteps 2940.
Path 199 | total_timesteps 2952.
Path 200 | total_timesteps 2964.
Path 201 | total_timesteps 2976.
Path 202 | total_timesteps 2990.
Path 203 | total_timesteps 3013.
Path 204 | total_timesteps 3024.
Path 205 | total_timesteps 3039.
Path 206 | total_timesteps 3051.
Path 207 | total_timesteps 3063.
Path 208 | total_timesteps 3086.
Path 209 | total_timesteps 3100.
Path 210 | total_timesteps 3114.
Path 211 | total_timesteps 3129.
Path 212 | total_timesteps 3151.
Path 213 | total_timesteps 3163.
Path 214 | total_timesteps 3177.
Path 215 | total_timesteps 3185.
Path 216 | total_timesteps 3200.
Path 217 | total_timesteps 3217.
Path 218 | total_timesteps 3231.
Path 219 | total_timesteps 3248.
Path 220 | total_timesteps 3257.
Path 221 | total_timesteps 3265.
Path 222 | total_timesteps 3292.
Path 223 | total_timesteps 3308.
Path 224 | total_timesteps 3321.
Path 225 | total_timesteps 3337.
Path 226 | total_timesteps 3351.
Path 227 | total_timesteps 3368.
Path 228 | total_timesteps 3384.
Path 229 | total_timesteps 3396.
Path 230 | total_timesteps 3419.
Path 231 | total_timesteps 3430.
Path 232 | total_timesteps 3440.
Path 233 | total_timesteps 3451.
Path 234 | total_timesteps 3468.
Path 235 | total_timesteps 3477.
Path 236 | total_timesteps 3502.
Path 237 | total_timesteps 3522.
Path 238 | total_timesteps 3538.
Path 239 | total_timesteps 3548.
Path 240 | total_timesteps 3561.
Path 241 | total_timesteps 3574.
Path 242 | total_timesteps 3582.
Path 243 | total_timesteps 3597.
Path 244 | total_timesteps 3610.
Path 245 | total_timesteps 3630.
Path 246 | total_timesteps 3645.
Path 247 | total_timesteps 3670.
Path 248 | total_timesteps 3692.
Path 249 | total_timesteps 3703.
Path 250 | total_timesteps 3724.
Path 251 | total_timesteps 3762.
Path 252 | total_timesteps 3773.
Path 253 | total_timesteps 3791.
Path 254 | total_timesteps 3811.
Path 255 | total_timesteps 3820.
Path 256 | total_timesteps 3832.
Path 257 | total_timesteps 3840.
Path 258 | total_timesteps 3861.
Path 259 | total_timesteps 3880.
Path 260 | total_timesteps 3903.
Path 261 | total_timesteps 3913.
Path 262 | total_timesteps 3926.
Path 263 | total_timesteps 3941.
Path 264 | total_timesteps 3964.
Path 265 | total_timesteps 3979.
Path 266 | total_timesteps 3988.
Path 267 | total_timesteps 4028.
Path 268 | total_timesteps 4040.
Path 269 | total_timesteps 4052.
Path 270 | total_timesteps 4069.
Path 271 | total_timesteps 4099.
Path 272 | total_timesteps 4109.
Path 273 | total_timesteps 4121.
Path 274 | total_timesteps 4141.
Path 275 | total_timesteps 4169.
Path 276 | total_timesteps 4179.
Path 277 | total_timesteps 4191.
Path 278 | total_timesteps 4212.
Path 279 | total_timesteps 4236.
Path 280 | total_timesteps 4250.
Path 281 | total_timesteps 4264.
Path 282 | total_timesteps 4279.
Path 283 | total_timesteps 4288.
Path 284 | total_timesteps 4309.
Path 285 | total_timesteps 4321.
Path 286 | total_timesteps 4334.
Path 287 | total_timesteps 4348.
Path 288 | total_timesteps 4366.
Path 289 | total_timesteps 4386.
Path 290 | total_timesteps 4403.
Path 291 | total_timesteps 4422.
Path 292 | total_timesteps 4443.
Path 293 | total_timesteps 4453.
Path 294 | total_timesteps 4463.
Path 295 | total_timesteps 4475.
Path 296 | total_timesteps 4489.
Path 297 | total_timesteps 4520.
Path 298 | total_timesteps 4531.
Path 299 | total_timesteps 4542.
Path 300 | total_timesteps 4552.
Path 301 | total_timesteps 4569.
Path 302 | total_timesteps 4578.
Path 303 | total_timesteps 4587.
Path 304 | total_timesteps 4603.
Path 305 | total_timesteps 4620.
Path 306 | total_timesteps 4629.
Path 307 | total_timesteps 4651.
Path 308 | total_timesteps 4663.
Path 309 | total_timesteps 4681.
Path 310 | total_timesteps 4701.
Path 311 | total_timesteps 4735.
Path 312 | total_timesteps 4746.
Path 313 | total_timesteps 4776.
Path 314 | total_timesteps 4793.
Path 315 | total_timesteps 4804.
Path 316 | total_timesteps 4817.
Path 317 | total_timesteps 4832.
Path 318 | total_timesteps 4848.
Path 319 | total_timesteps 4864.
Path 320 | total_timesteps 4873.
Path 321 | total_timesteps 4895.
Path 322 | total_timesteps 4911.
Path 323 | total_timesteps 4924.
Path 324 | total_timesteps 4944.
Path 325 | total_timesteps 4962.
Path 326 | total_timesteps 4969.
Path 327 | total_timesteps 4978.
Path 328 | total_timesteps 4993.
Path 329 | total_timesteps 5006.
Path 330 | total_timesteps 5026.
Path 331 | total_timesteps 5041.
Path 332 | total_timesteps 5066.
Path 333 | total_timesteps 5083.
Path 334 | total_timesteps 5093.
Path 335 | total_timesteps 5116.
Path 336 | total_timesteps 5128.
Path 337 | total_timesteps 5148.
Path 338 | total_timesteps 5168.
Path 339 | total_timesteps 5178.
Path 340 | total_timesteps 5203.
Path 341 | total_timesteps 5233.
Path 342 | total_timesteps 5244.
Path 343 | total_timesteps 5274.
Path 344 | total_timesteps 5286.
Path 345 | total_timesteps 5303.
Path 346 | total_timesteps 5330.
Path 347 | total_timesteps 5354.
Path 348 | total_timesteps 5368.
Path 349 | total_timesteps 5380.
Path 350 | total_timesteps 5392.
Path 351 | total_timesteps 5419.
Path 352 | total_timesteps 5432.
Path 353 | total_timesteps 5445.
Path 354 | total_timesteps 5455.
Path 355 | total_timesteps 5465.
Path 356 | total_timesteps 5479.
Path 357 | total_timesteps 5517.
Path 358 | total_timesteps 5536.
Path 359 | total_timesteps 5554.
Path 360 | total_timesteps 5567.
Path 361 | total_timesteps 5576.
Path 362 | total_timesteps 5595.
Path 363 | total_timesteps 5608.
Path 364 | total_timesteps 5620.
Path 365 | total_timesteps 5635.
Path 366 | total_timesteps 5652.
Path 367 | total_timesteps 5667.
Path 368 | total_timesteps 5682.
Path 369 | total_timesteps 5696.
Path 370 | total_timesteps 5721.
Path 371 | total_timesteps 5732.
Path 372 | total_timesteps 5750.
Path 373 | total_timesteps 5763.
Path 374 | total_timesteps 5780.
Path 375 | total_timesteps 5801.
Path 376 | total_timesteps 5815.
Path 377 | total_timesteps 5844.
Path 378 | total_timesteps 5862.
Path 379 | total_timesteps 5874.
Path 380 | total_timesteps 5887.
Path 381 | total_timesteps 5908.
Path 382 | total_timesteps 5919.
Path 383 | total_timesteps 5930.
Path 384 | total_timesteps 5942.
Path 385 | total_timesteps 5980.
Path 386 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.3     |
| Iteration     | 27       |
| MaximumReturn | 10.6     |
| MinimumReturn | -20.3    |
| TotalSamples  | 116158   |
----------------------------
itr #28 | 
Fitting dynamics.
Validation loss = 0.0025315729435533285
Validation loss = 0.002906669396907091
Validation loss = 0.002539794659242034
Validation loss = 0.0023374096490442753
Validation loss = 0.0024606389924883842
Validation loss = 0.002488198922947049
Validation loss = 0.002573699690401554
Validation loss = 0.002204492222517729
Validation loss = 0.002300861058756709
Validation loss = 0.002282161498442292
Validation loss = 0.0025201565586030483
Validation loss = 0.002669938374310732
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 46.
Path 4 | total_timesteps 59.
Path 5 | total_timesteps 74.
Path 6 | total_timesteps 83.
Path 7 | total_timesteps 91.
Path 8 | total_timesteps 109.
Path 9 | total_timesteps 139.
Path 10 | total_timesteps 155.
Path 11 | total_timesteps 167.
Path 12 | total_timesteps 199.
Path 13 | total_timesteps 210.
Path 14 | total_timesteps 220.
Path 15 | total_timesteps 228.
Path 16 | total_timesteps 238.
Path 17 | total_timesteps 248.
Path 18 | total_timesteps 257.
Path 19 | total_timesteps 271.
Path 20 | total_timesteps 293.
Path 21 | total_timesteps 309.
Path 22 | total_timesteps 320.
Path 23 | total_timesteps 333.
Path 24 | total_timesteps 342.
Path 25 | total_timesteps 352.
Path 26 | total_timesteps 364.
Path 27 | total_timesteps 376.
Path 28 | total_timesteps 399.
Path 29 | total_timesteps 409.
Path 30 | total_timesteps 418.
Path 31 | total_timesteps 427.
Path 32 | total_timesteps 451.
Path 33 | total_timesteps 465.
Path 34 | total_timesteps 474.
Path 35 | total_timesteps 489.
Path 36 | total_timesteps 496.
Path 37 | total_timesteps 506.
Path 38 | total_timesteps 514.
Path 39 | total_timesteps 521.
Path 40 | total_timesteps 532.
Path 41 | total_timesteps 546.
Path 42 | total_timesteps 555.
Path 43 | total_timesteps 563.
Path 44 | total_timesteps 572.
Path 45 | total_timesteps 582.
Path 46 | total_timesteps 597.
Path 47 | total_timesteps 611.
Path 48 | total_timesteps 622.
Path 49 | total_timesteps 637.
Path 50 | total_timesteps 649.
Path 51 | total_timesteps 658.
Path 52 | total_timesteps 668.
Path 53 | total_timesteps 680.
Path 54 | total_timesteps 690.
Path 55 | total_timesteps 707.
Path 56 | total_timesteps 718.
Path 57 | total_timesteps 727.
Path 58 | total_timesteps 735.
Path 59 | total_timesteps 747.
Path 60 | total_timesteps 757.
Path 61 | total_timesteps 775.
Path 62 | total_timesteps 788.
Path 63 | total_timesteps 800.
Path 64 | total_timesteps 813.
Path 65 | total_timesteps 836.
Path 66 | total_timesteps 849.
Path 67 | total_timesteps 862.
Path 68 | total_timesteps 876.
Path 69 | total_timesteps 890.
Path 70 | total_timesteps 901.
Path 71 | total_timesteps 913.
Path 72 | total_timesteps 929.
Path 73 | total_timesteps 949.
Path 74 | total_timesteps 959.
Path 75 | total_timesteps 969.
Path 76 | total_timesteps 980.
Path 77 | total_timesteps 1009.
Path 78 | total_timesteps 1020.
Path 79 | total_timesteps 1045.
Path 80 | total_timesteps 1061.
Path 81 | total_timesteps 1082.
Path 82 | total_timesteps 1094.
Path 83 | total_timesteps 1108.
Path 84 | total_timesteps 1117.
Path 85 | total_timesteps 1131.
Path 86 | total_timesteps 1154.
Path 87 | total_timesteps 1175.
Path 88 | total_timesteps 1198.
Path 89 | total_timesteps 1209.
Path 90 | total_timesteps 1221.
Path 91 | total_timesteps 1230.
Path 92 | total_timesteps 1238.
Path 93 | total_timesteps 1257.
Path 94 | total_timesteps 1271.
Path 95 | total_timesteps 1280.
Path 96 | total_timesteps 1292.
Path 97 | total_timesteps 1303.
Path 98 | total_timesteps 1317.
Path 99 | total_timesteps 1330.
Path 100 | total_timesteps 1343.
Path 101 | total_timesteps 1353.
Path 102 | total_timesteps 1364.
Path 103 | total_timesteps 1377.
Path 104 | total_timesteps 1389.
Path 105 | total_timesteps 1400.
Path 106 | total_timesteps 1411.
Path 107 | total_timesteps 1422.
Path 108 | total_timesteps 1436.
Path 109 | total_timesteps 1451.
Path 110 | total_timesteps 1460.
Path 111 | total_timesteps 1487.
Path 112 | total_timesteps 1499.
Path 113 | total_timesteps 1510.
Path 114 | total_timesteps 1520.
Path 115 | total_timesteps 1540.
Path 116 | total_timesteps 1561.
Path 117 | total_timesteps 1575.
Path 118 | total_timesteps 1610.
Path 119 | total_timesteps 1620.
Path 120 | total_timesteps 1631.
Path 121 | total_timesteps 1643.
Path 122 | total_timesteps 1657.
Path 123 | total_timesteps 1681.
Path 124 | total_timesteps 1692.
Path 125 | total_timesteps 1702.
Path 126 | total_timesteps 1716.
Path 127 | total_timesteps 1726.
Path 128 | total_timesteps 1750.
Path 129 | total_timesteps 1761.
Path 130 | total_timesteps 1785.
Path 131 | total_timesteps 1796.
Path 132 | total_timesteps 1805.
Path 133 | total_timesteps 1824.
Path 134 | total_timesteps 1836.
Path 135 | total_timesteps 1848.
Path 136 | total_timesteps 1862.
Path 137 | total_timesteps 1873.
Path 138 | total_timesteps 1881.
Path 139 | total_timesteps 1895.
Path 140 | total_timesteps 1913.
Path 141 | total_timesteps 1923.
Path 142 | total_timesteps 1938.
Path 143 | total_timesteps 1951.
Path 144 | total_timesteps 1960.
Path 145 | total_timesteps 1974.
Path 146 | total_timesteps 1986.
Path 147 | total_timesteps 1996.
Path 148 | total_timesteps 2013.
Path 149 | total_timesteps 2023.
Path 150 | total_timesteps 2035.
Path 151 | total_timesteps 2057.
Path 152 | total_timesteps 2065.
Path 153 | total_timesteps 2075.
Path 154 | total_timesteps 2086.
Path 155 | total_timesteps 2098.
Path 156 | total_timesteps 2117.
Path 157 | total_timesteps 2126.
Path 158 | total_timesteps 2139.
Path 159 | total_timesteps 2160.
Path 160 | total_timesteps 2178.
Path 161 | total_timesteps 2187.
Path 162 | total_timesteps 2198.
Path 163 | total_timesteps 2206.
Path 164 | total_timesteps 2217.
Path 165 | total_timesteps 2228.
Path 166 | total_timesteps 2237.
Path 167 | total_timesteps 2247.
Path 168 | total_timesteps 2259.
Path 169 | total_timesteps 2272.
Path 170 | total_timesteps 2283.
Path 171 | total_timesteps 2298.
Path 172 | total_timesteps 2312.
Path 173 | total_timesteps 2331.
Path 174 | total_timesteps 2342.
Path 175 | total_timesteps 2348.
Path 176 | total_timesteps 2355.
Path 177 | total_timesteps 2370.
Path 178 | total_timesteps 2381.
Path 179 | total_timesteps 2400.
Path 180 | total_timesteps 2414.
Path 181 | total_timesteps 2426.
Path 182 | total_timesteps 2441.
Path 183 | total_timesteps 2457.
Path 184 | total_timesteps 2475.
Path 185 | total_timesteps 2484.
Path 186 | total_timesteps 2498.
Path 187 | total_timesteps 2506.
Path 188 | total_timesteps 2516.
Path 189 | total_timesteps 2531.
Path 190 | total_timesteps 2541.
Path 191 | total_timesteps 2555.
Path 192 | total_timesteps 2566.
Path 193 | total_timesteps 2574.
Path 194 | total_timesteps 2584.
Path 195 | total_timesteps 2594.
Path 196 | total_timesteps 2612.
Path 197 | total_timesteps 2622.
Path 198 | total_timesteps 2637.
Path 199 | total_timesteps 2646.
Path 200 | total_timesteps 2656.
Path 201 | total_timesteps 2678.
Path 202 | total_timesteps 2686.
Path 203 | total_timesteps 2701.
Path 204 | total_timesteps 2713.
Path 205 | total_timesteps 2722.
Path 206 | total_timesteps 2741.
Path 207 | total_timesteps 2772.
Path 208 | total_timesteps 2784.
Path 209 | total_timesteps 2796.
Path 210 | total_timesteps 2820.
Path 211 | total_timesteps 2837.
Path 212 | total_timesteps 2849.
Path 213 | total_timesteps 2862.
Path 214 | total_timesteps 2877.
Path 215 | total_timesteps 2890.
Path 216 | total_timesteps 2901.
Path 217 | total_timesteps 2914.
Path 218 | total_timesteps 2921.
Path 219 | total_timesteps 2945.
Path 220 | total_timesteps 2961.
Path 221 | total_timesteps 2976.
Path 222 | total_timesteps 2984.
Path 223 | total_timesteps 2994.
Path 224 | total_timesteps 3024.
Path 225 | total_timesteps 3040.
Path 226 | total_timesteps 3051.
Path 227 | total_timesteps 3063.
Path 228 | total_timesteps 3076.
Path 229 | total_timesteps 3084.
Path 230 | total_timesteps 3113.
Path 231 | total_timesteps 3126.
Path 232 | total_timesteps 3141.
Path 233 | total_timesteps 3149.
Path 234 | total_timesteps 3160.
Path 235 | total_timesteps 3168.
Path 236 | total_timesteps 3180.
Path 237 | total_timesteps 3191.
Path 238 | total_timesteps 3206.
Path 239 | total_timesteps 3215.
Path 240 | total_timesteps 3225.
Path 241 | total_timesteps 3239.
Path 242 | total_timesteps 3251.
Path 243 | total_timesteps 3262.
Path 244 | total_timesteps 3276.
Path 245 | total_timesteps 3285.
Path 246 | total_timesteps 3297.
Path 247 | total_timesteps 3305.
Path 248 | total_timesteps 3312.
Path 249 | total_timesteps 3327.
Path 250 | total_timesteps 3354.
Path 251 | total_timesteps 3368.
Path 252 | total_timesteps 3388.
Path 253 | total_timesteps 3399.
Path 254 | total_timesteps 3414.
Path 255 | total_timesteps 3422.
Path 256 | total_timesteps 3432.
Path 257 | total_timesteps 3446.
Path 258 | total_timesteps 3460.
Path 259 | total_timesteps 3469.
Path 260 | total_timesteps 3476.
Path 261 | total_timesteps 3485.
Path 262 | total_timesteps 3492.
Path 263 | total_timesteps 3499.
Path 264 | total_timesteps 3506.
Path 265 | total_timesteps 3515.
Path 266 | total_timesteps 3528.
Path 267 | total_timesteps 3554.
Path 268 | total_timesteps 3563.
Path 269 | total_timesteps 3580.
Path 270 | total_timesteps 3593.
Path 271 | total_timesteps 3608.
Path 272 | total_timesteps 3632.
Path 273 | total_timesteps 3642.
Path 274 | total_timesteps 3653.
Path 275 | total_timesteps 3666.
Path 276 | total_timesteps 3678.
Path 277 | total_timesteps 3694.
Path 278 | total_timesteps 3708.
Path 279 | total_timesteps 3720.
Path 280 | total_timesteps 3734.
Path 281 | total_timesteps 3760.
Path 282 | total_timesteps 3768.
Path 283 | total_timesteps 3778.
Path 284 | total_timesteps 3789.
Path 285 | total_timesteps 3801.
Path 286 | total_timesteps 3814.
Path 287 | total_timesteps 3829.
Path 288 | total_timesteps 3844.
Path 289 | total_timesteps 3855.
Path 290 | total_timesteps 3867.
Path 291 | total_timesteps 3904.
Path 292 | total_timesteps 3918.
Path 293 | total_timesteps 3943.
Path 294 | total_timesteps 3954.
Path 295 | total_timesteps 3965.
Path 296 | total_timesteps 3976.
Path 297 | total_timesteps 3986.
Path 298 | total_timesteps 3995.
Path 299 | total_timesteps 4017.
Path 300 | total_timesteps 4028.
Path 301 | total_timesteps 4040.
Path 302 | total_timesteps 4049.
Path 303 | total_timesteps 4059.
Path 304 | total_timesteps 4072.
Path 305 | total_timesteps 4080.
Path 306 | total_timesteps 4090.
Path 307 | total_timesteps 4100.
Path 308 | total_timesteps 4110.
Path 309 | total_timesteps 4120.
Path 310 | total_timesteps 4127.
Path 311 | total_timesteps 4139.
Path 312 | total_timesteps 4149.
Path 313 | total_timesteps 4164.
Path 314 | total_timesteps 4172.
Path 315 | total_timesteps 4183.
Path 316 | total_timesteps 4197.
Path 317 | total_timesteps 4206.
Path 318 | total_timesteps 4227.
Path 319 | total_timesteps 4249.
Path 320 | total_timesteps 4263.
Path 321 | total_timesteps 4272.
Path 322 | total_timesteps 4288.
Path 323 | total_timesteps 4298.
Path 324 | total_timesteps 4309.
Path 325 | total_timesteps 4320.
Path 326 | total_timesteps 4334.
Path 327 | total_timesteps 4348.
Path 328 | total_timesteps 4358.
Path 329 | total_timesteps 4364.
Path 330 | total_timesteps 4375.
Path 331 | total_timesteps 4386.
Path 332 | total_timesteps 4401.
Path 333 | total_timesteps 4426.
Path 334 | total_timesteps 4441.
Path 335 | total_timesteps 4449.
Path 336 | total_timesteps 4463.
Path 337 | total_timesteps 4477.
Path 338 | total_timesteps 4487.
Path 339 | total_timesteps 4500.
Path 340 | total_timesteps 4511.
Path 341 | total_timesteps 4520.
Path 342 | total_timesteps 4532.
Path 343 | total_timesteps 4555.
Path 344 | total_timesteps 4579.
Path 345 | total_timesteps 4605.
Path 346 | total_timesteps 4613.
Path 347 | total_timesteps 4629.
Path 348 | total_timesteps 4642.
Path 349 | total_timesteps 4660.
Path 350 | total_timesteps 4673.
Path 351 | total_timesteps 4681.
Path 352 | total_timesteps 4708.
Path 353 | total_timesteps 4723.
Path 354 | total_timesteps 4736.
Path 355 | total_timesteps 4758.
Path 356 | total_timesteps 4770.
Path 357 | total_timesteps 4778.
Path 358 | total_timesteps 4801.
Path 359 | total_timesteps 4809.
Path 360 | total_timesteps 4820.
Path 361 | total_timesteps 4837.
Path 362 | total_timesteps 4844.
Path 363 | total_timesteps 4854.
Path 364 | total_timesteps 4861.
Path 365 | total_timesteps 4873.
Path 366 | total_timesteps 4884.
Path 367 | total_timesteps 4894.
Path 368 | total_timesteps 4903.
Path 369 | total_timesteps 4915.
Path 370 | total_timesteps 4929.
Path 371 | total_timesteps 4940.
Path 372 | total_timesteps 4953.
Path 373 | total_timesteps 4991.
Path 374 | total_timesteps 5003.
Path 375 | total_timesteps 5015.
Path 376 | total_timesteps 5036.
Path 377 | total_timesteps 5062.
Path 378 | total_timesteps 5071.
Path 379 | total_timesteps 5087.
Path 380 | total_timesteps 5097.
Path 381 | total_timesteps 5109.
Path 382 | total_timesteps 5126.
Path 383 | total_timesteps 5146.
Path 384 | total_timesteps 5162.
Path 385 | total_timesteps 5179.
Path 386 | total_timesteps 5188.
Path 387 | total_timesteps 5198.
Path 388 | total_timesteps 5220.
Path 389 | total_timesteps 5236.
Path 390 | total_timesteps 5251.
Path 391 | total_timesteps 5258.
Path 392 | total_timesteps 5273.
Path 393 | total_timesteps 5281.
Path 394 | total_timesteps 5293.
Path 395 | total_timesteps 5305.
Path 396 | total_timesteps 5313.
Path 397 | total_timesteps 5323.
Path 398 | total_timesteps 5333.
Path 399 | total_timesteps 5344.
Path 400 | total_timesteps 5354.
Path 401 | total_timesteps 5365.
Path 402 | total_timesteps 5373.
Path 403 | total_timesteps 5384.
Path 404 | total_timesteps 5396.
Path 405 | total_timesteps 5406.
Path 406 | total_timesteps 5418.
Path 407 | total_timesteps 5432.
Path 408 | total_timesteps 5445.
Path 409 | total_timesteps 5454.
Path 410 | total_timesteps 5466.
Path 411 | total_timesteps 5475.
Path 412 | total_timesteps 5489.
Path 413 | total_timesteps 5502.
Path 414 | total_timesteps 5515.
Path 415 | total_timesteps 5526.
Path 416 | total_timesteps 5539.
Path 417 | total_timesteps 5552.
Path 418 | total_timesteps 5566.
Path 419 | total_timesteps 5576.
Path 420 | total_timesteps 5591.
Path 421 | total_timesteps 5614.
Path 422 | total_timesteps 5627.
Path 423 | total_timesteps 5644.
Path 424 | total_timesteps 5654.
Path 425 | total_timesteps 5664.
Path 426 | total_timesteps 5678.
Path 427 | total_timesteps 5694.
Path 428 | total_timesteps 5705.
Path 429 | total_timesteps 5719.
Path 430 | total_timesteps 5734.
Path 431 | total_timesteps 5746.
Path 432 | total_timesteps 5760.
Path 433 | total_timesteps 5769.
Path 434 | total_timesteps 5779.
Path 435 | total_timesteps 5789.
Path 436 | total_timesteps 5806.
Path 437 | total_timesteps 5817.
Path 438 | total_timesteps 5832.
Path 439 | total_timesteps 5851.
Path 440 | total_timesteps 5860.
Path 441 | total_timesteps 5873.
Path 442 | total_timesteps 5885.
Path 443 | total_timesteps 5897.
Path 444 | total_timesteps 5908.
Path 445 | total_timesteps 5916.
Path 446 | total_timesteps 5925.
Path 447 | total_timesteps 5938.
Path 448 | total_timesteps 5963.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.45    |
| Iteration     | 28       |
| MaximumReturn | 6.29     |
| MinimumReturn | -19.6    |
| TotalSamples  | 120162   |
----------------------------
itr #29 | 
Fitting dynamics.
Validation loss = 0.0022924127988517284
Validation loss = 0.002338013844564557
Validation loss = 0.0023211389780044556
Validation loss = 0.002587497467175126
Validation loss = 0.002583854366093874
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 45.
Path 4 | total_timesteps 60.
Path 5 | total_timesteps 71.
Path 6 | total_timesteps 84.
Path 7 | total_timesteps 95.
Path 8 | total_timesteps 107.
Path 9 | total_timesteps 118.
Path 10 | total_timesteps 134.
Path 11 | total_timesteps 145.
Path 12 | total_timesteps 162.
Path 13 | total_timesteps 177.
Path 14 | total_timesteps 185.
Path 15 | total_timesteps 194.
Path 16 | total_timesteps 219.
Path 17 | total_timesteps 226.
Path 18 | total_timesteps 238.
Path 19 | total_timesteps 264.
Path 20 | total_timesteps 271.
Path 21 | total_timesteps 287.
Path 22 | total_timesteps 298.
Path 23 | total_timesteps 315.
Path 24 | total_timesteps 330.
Path 25 | total_timesteps 352.
Path 26 | total_timesteps 364.
Path 27 | total_timesteps 373.
Path 28 | total_timesteps 387.
Path 29 | total_timesteps 397.
Path 30 | total_timesteps 405.
Path 31 | total_timesteps 416.
Path 32 | total_timesteps 428.
Path 33 | total_timesteps 438.
Path 34 | total_timesteps 457.
Path 35 | total_timesteps 467.
Path 36 | total_timesteps 476.
Path 37 | total_timesteps 487.
Path 38 | total_timesteps 506.
Path 39 | total_timesteps 517.
Path 40 | total_timesteps 526.
Path 41 | total_timesteps 535.
Path 42 | total_timesteps 544.
Path 43 | total_timesteps 553.
Path 44 | total_timesteps 567.
Path 45 | total_timesteps 580.
Path 46 | total_timesteps 594.
Path 47 | total_timesteps 603.
Path 48 | total_timesteps 617.
Path 49 | total_timesteps 624.
Path 50 | total_timesteps 648.
Path 51 | total_timesteps 655.
Path 52 | total_timesteps 666.
Path 53 | total_timesteps 680.
Path 54 | total_timesteps 691.
Path 55 | total_timesteps 705.
Path 56 | total_timesteps 716.
Path 57 | total_timesteps 725.
Path 58 | total_timesteps 742.
Path 59 | total_timesteps 755.
Path 60 | total_timesteps 765.
Path 61 | total_timesteps 787.
Path 62 | total_timesteps 801.
Path 63 | total_timesteps 837.
Path 64 | total_timesteps 849.
Path 65 | total_timesteps 865.
Path 66 | total_timesteps 881.
Path 67 | total_timesteps 892.
Path 68 | total_timesteps 907.
Path 69 | total_timesteps 916.
Path 70 | total_timesteps 929.
Path 71 | total_timesteps 939.
Path 72 | total_timesteps 956.
Path 73 | total_timesteps 970.
Path 74 | total_timesteps 983.
Path 75 | total_timesteps 999.
Path 76 | total_timesteps 1013.
Path 77 | total_timesteps 1022.
Path 78 | total_timesteps 1035.
Path 79 | total_timesteps 1048.
Path 80 | total_timesteps 1059.
Path 81 | total_timesteps 1072.
Path 82 | total_timesteps 1083.
Path 83 | total_timesteps 1100.
Path 84 | total_timesteps 1109.
Path 85 | total_timesteps 1128.
Path 86 | total_timesteps 1140.
Path 87 | total_timesteps 1153.
Path 88 | total_timesteps 1164.
Path 89 | total_timesteps 1176.
Path 90 | total_timesteps 1185.
Path 91 | total_timesteps 1197.
Path 92 | total_timesteps 1207.
Path 93 | total_timesteps 1217.
Path 94 | total_timesteps 1224.
Path 95 | total_timesteps 1232.
Path 96 | total_timesteps 1240.
Path 97 | total_timesteps 1249.
Path 98 | total_timesteps 1260.
Path 99 | total_timesteps 1273.
Path 100 | total_timesteps 1280.
Path 101 | total_timesteps 1293.
Path 102 | total_timesteps 1307.
Path 103 | total_timesteps 1322.
Path 104 | total_timesteps 1332.
Path 105 | total_timesteps 1341.
Path 106 | total_timesteps 1347.
Path 107 | total_timesteps 1366.
Path 108 | total_timesteps 1378.
Path 109 | total_timesteps 1400.
Path 110 | total_timesteps 1410.
Path 111 | total_timesteps 1421.
Path 112 | total_timesteps 1433.
Path 113 | total_timesteps 1449.
Path 114 | total_timesteps 1460.
Path 115 | total_timesteps 1479.
Path 116 | total_timesteps 1487.
Path 117 | total_timesteps 1499.
Path 118 | total_timesteps 1507.
Path 119 | total_timesteps 1516.
Path 120 | total_timesteps 1528.
Path 121 | total_timesteps 1547.
Path 122 | total_timesteps 1558.
Path 123 | total_timesteps 1567.
Path 124 | total_timesteps 1577.
Path 125 | total_timesteps 1585.
Path 126 | total_timesteps 1593.
Path 127 | total_timesteps 1609.
Path 128 | total_timesteps 1619.
Path 129 | total_timesteps 1632.
Path 130 | total_timesteps 1643.
Path 131 | total_timesteps 1659.
Path 132 | total_timesteps 1669.
Path 133 | total_timesteps 1682.
Path 134 | total_timesteps 1693.
Path 135 | total_timesteps 1704.
Path 136 | total_timesteps 1721.
Path 137 | total_timesteps 1738.
Path 138 | total_timesteps 1750.
Path 139 | total_timesteps 1765.
Path 140 | total_timesteps 1773.
Path 141 | total_timesteps 1782.
Path 142 | total_timesteps 1791.
Path 143 | total_timesteps 1805.
Path 144 | total_timesteps 1824.
Path 145 | total_timesteps 1833.
Path 146 | total_timesteps 1855.
Path 147 | total_timesteps 1872.
Path 148 | total_timesteps 1882.
Path 149 | total_timesteps 1894.
Path 150 | total_timesteps 1904.
Path 151 | total_timesteps 1912.
Path 152 | total_timesteps 1921.
Path 153 | total_timesteps 1933.
Path 154 | total_timesteps 1941.
Path 155 | total_timesteps 1954.
Path 156 | total_timesteps 1973.
Path 157 | total_timesteps 1985.
Path 158 | total_timesteps 1998.
Path 159 | total_timesteps 2008.
Path 160 | total_timesteps 2019.
Path 161 | total_timesteps 2027.
Path 162 | total_timesteps 2042.
Path 163 | total_timesteps 2054.
Path 164 | total_timesteps 2064.
Path 165 | total_timesteps 2088.
Path 166 | total_timesteps 2095.
Path 167 | total_timesteps 2104.
Path 168 | total_timesteps 2112.
Path 169 | total_timesteps 2132.
Path 170 | total_timesteps 2145.
Path 171 | total_timesteps 2158.
Path 172 | total_timesteps 2167.
Path 173 | total_timesteps 2178.
Path 174 | total_timesteps 2191.
Path 175 | total_timesteps 2211.
Path 176 | total_timesteps 2219.
Path 177 | total_timesteps 2227.
Path 178 | total_timesteps 2246.
Path 179 | total_timesteps 2258.
Path 180 | total_timesteps 2270.
Path 181 | total_timesteps 2280.
Path 182 | total_timesteps 2293.
Path 183 | total_timesteps 2312.
Path 184 | total_timesteps 2327.
Path 185 | total_timesteps 2342.
Path 186 | total_timesteps 2355.
Path 187 | total_timesteps 2366.
Path 188 | total_timesteps 2375.
Path 189 | total_timesteps 2383.
Path 190 | total_timesteps 2397.
Path 191 | total_timesteps 2408.
Path 192 | total_timesteps 2423.
Path 193 | total_timesteps 2432.
Path 194 | total_timesteps 2443.
Path 195 | total_timesteps 2458.
Path 196 | total_timesteps 2470.
Path 197 | total_timesteps 2480.
Path 198 | total_timesteps 2501.
Path 199 | total_timesteps 2516.
Path 200 | total_timesteps 2525.
Path 201 | total_timesteps 2532.
Path 202 | total_timesteps 2553.
Path 203 | total_timesteps 2566.
Path 204 | total_timesteps 2579.
Path 205 | total_timesteps 2595.
Path 206 | total_timesteps 2605.
Path 207 | total_timesteps 2619.
Path 208 | total_timesteps 2627.
Path 209 | total_timesteps 2643.
Path 210 | total_timesteps 2652.
Path 211 | total_timesteps 2665.
Path 212 | total_timesteps 2690.
Path 213 | total_timesteps 2705.
Path 214 | total_timesteps 2714.
Path 215 | total_timesteps 2727.
Path 216 | total_timesteps 2740.
Path 217 | total_timesteps 2754.
Path 218 | total_timesteps 2764.
Path 219 | total_timesteps 2793.
Path 220 | total_timesteps 2805.
Path 221 | total_timesteps 2830.
Path 222 | total_timesteps 2840.
Path 223 | total_timesteps 2859.
Path 224 | total_timesteps 2873.
Path 225 | total_timesteps 2886.
Path 226 | total_timesteps 2897.
Path 227 | total_timesteps 2905.
Path 228 | total_timesteps 2917.
Path 229 | total_timesteps 2932.
Path 230 | total_timesteps 2942.
Path 231 | total_timesteps 2953.
Path 232 | total_timesteps 2964.
Path 233 | total_timesteps 2973.
Path 234 | total_timesteps 2993.
Path 235 | total_timesteps 3003.
Path 236 | total_timesteps 3021.
Path 237 | total_timesteps 3033.
Path 238 | total_timesteps 3058.
Path 239 | total_timesteps 3078.
Path 240 | total_timesteps 3094.
Path 241 | total_timesteps 3112.
Path 242 | total_timesteps 3129.
Path 243 | total_timesteps 3155.
Path 244 | total_timesteps 3165.
Path 245 | total_timesteps 3182.
Path 246 | total_timesteps 3210.
Path 247 | total_timesteps 3225.
Path 248 | total_timesteps 3236.
Path 249 | total_timesteps 3244.
Path 250 | total_timesteps 3267.
Path 251 | total_timesteps 3276.
Path 252 | total_timesteps 3284.
Path 253 | total_timesteps 3301.
Path 254 | total_timesteps 3308.
Path 255 | total_timesteps 3321.
Path 256 | total_timesteps 3330.
Path 257 | total_timesteps 3343.
Path 258 | total_timesteps 3354.
Path 259 | total_timesteps 3365.
Path 260 | total_timesteps 3389.
Path 261 | total_timesteps 3399.
Path 262 | total_timesteps 3409.
Path 263 | total_timesteps 3420.
Path 264 | total_timesteps 3432.
Path 265 | total_timesteps 3442.
Path 266 | total_timesteps 3450.
Path 267 | total_timesteps 3461.
Path 268 | total_timesteps 3471.
Path 269 | total_timesteps 3495.
Path 270 | total_timesteps 3513.
Path 271 | total_timesteps 3522.
Path 272 | total_timesteps 3536.
Path 273 | total_timesteps 3562.
Path 274 | total_timesteps 3582.
Path 275 | total_timesteps 3601.
Path 276 | total_timesteps 3620.
Path 277 | total_timesteps 3631.
Path 278 | total_timesteps 3642.
Path 279 | total_timesteps 3650.
Path 280 | total_timesteps 3658.
Path 281 | total_timesteps 3668.
Path 282 | total_timesteps 3681.
Path 283 | total_timesteps 3692.
Path 284 | total_timesteps 3705.
Path 285 | total_timesteps 3730.
Path 286 | total_timesteps 3744.
Path 287 | total_timesteps 3762.
Path 288 | total_timesteps 3769.
Path 289 | total_timesteps 3794.
Path 290 | total_timesteps 3802.
Path 291 | total_timesteps 3825.
Path 292 | total_timesteps 3834.
Path 293 | total_timesteps 3847.
Path 294 | total_timesteps 3856.
Path 295 | total_timesteps 3870.
Path 296 | total_timesteps 3900.
Path 297 | total_timesteps 3918.
Path 298 | total_timesteps 3933.
Path 299 | total_timesteps 3940.
Path 300 | total_timesteps 3954.
Path 301 | total_timesteps 3962.
Path 302 | total_timesteps 3971.
Path 303 | total_timesteps 3986.
Path 304 | total_timesteps 4016.
Path 305 | total_timesteps 4034.
Path 306 | total_timesteps 4046.
Path 307 | total_timesteps 4055.
Path 308 | total_timesteps 4085.
Path 309 | total_timesteps 4106.
Path 310 | total_timesteps 4115.
Path 311 | total_timesteps 4127.
Path 312 | total_timesteps 4142.
Path 313 | total_timesteps 4153.
Path 314 | total_timesteps 4171.
Path 315 | total_timesteps 4189.
Path 316 | total_timesteps 4198.
Path 317 | total_timesteps 4206.
Path 318 | total_timesteps 4225.
Path 319 | total_timesteps 4242.
Path 320 | total_timesteps 4255.
Path 321 | total_timesteps 4267.
Path 322 | total_timesteps 4277.
Path 323 | total_timesteps 4301.
Path 324 | total_timesteps 4311.
Path 325 | total_timesteps 4328.
Path 326 | total_timesteps 4344.
Path 327 | total_timesteps 4357.
Path 328 | total_timesteps 4369.
Path 329 | total_timesteps 4396.
Path 330 | total_timesteps 4405.
Path 331 | total_timesteps 4422.
Path 332 | total_timesteps 4431.
Path 333 | total_timesteps 4439.
Path 334 | total_timesteps 4456.
Path 335 | total_timesteps 4465.
Path 336 | total_timesteps 4482.
Path 337 | total_timesteps 4512.
Path 338 | total_timesteps 4538.
Path 339 | total_timesteps 4557.
Path 340 | total_timesteps 4573.
Path 341 | total_timesteps 4583.
Path 342 | total_timesteps 4608.
Path 343 | total_timesteps 4622.
Path 344 | total_timesteps 4638.
Path 345 | total_timesteps 4651.
Path 346 | total_timesteps 4661.
Path 347 | total_timesteps 4682.
Path 348 | total_timesteps 4699.
Path 349 | total_timesteps 4708.
Path 350 | total_timesteps 4731.
Path 351 | total_timesteps 4743.
Path 352 | total_timesteps 4770.
Path 353 | total_timesteps 4778.
Path 354 | total_timesteps 4785.
Path 355 | total_timesteps 4800.
Path 356 | total_timesteps 4814.
Path 357 | total_timesteps 4823.
Path 358 | total_timesteps 4832.
Path 359 | total_timesteps 4844.
Path 360 | total_timesteps 4851.
Path 361 | total_timesteps 4865.
Path 362 | total_timesteps 4877.
Path 363 | total_timesteps 4886.
Path 364 | total_timesteps 4896.
Path 365 | total_timesteps 4905.
Path 366 | total_timesteps 4919.
Path 367 | total_timesteps 4952.
Path 368 | total_timesteps 4971.
Path 369 | total_timesteps 5000.
Path 370 | total_timesteps 5012.
Path 371 | total_timesteps 5021.
Path 372 | total_timesteps 5033.
Path 373 | total_timesteps 5042.
Path 374 | total_timesteps 5059.
Path 375 | total_timesteps 5071.
Path 376 | total_timesteps 5085.
Path 377 | total_timesteps 5103.
Path 378 | total_timesteps 5109.
Path 379 | total_timesteps 5117.
Path 380 | total_timesteps 5131.
Path 381 | total_timesteps 5140.
Path 382 | total_timesteps 5148.
Path 383 | total_timesteps 5159.
Path 384 | total_timesteps 5180.
Path 385 | total_timesteps 5190.
Path 386 | total_timesteps 5211.
Path 387 | total_timesteps 5223.
Path 388 | total_timesteps 5233.
Path 389 | total_timesteps 5241.
Path 390 | total_timesteps 5248.
Path 391 | total_timesteps 5260.
Path 392 | total_timesteps 5268.
Path 393 | total_timesteps 5279.
Path 394 | total_timesteps 5290.
Path 395 | total_timesteps 5305.
Path 396 | total_timesteps 5314.
Path 397 | total_timesteps 5322.
Path 398 | total_timesteps 5338.
Path 399 | total_timesteps 5347.
Path 400 | total_timesteps 5359.
Path 401 | total_timesteps 5368.
Path 402 | total_timesteps 5407.
Path 403 | total_timesteps 5418.
Path 404 | total_timesteps 5449.
Path 405 | total_timesteps 5459.
Path 406 | total_timesteps 5475.
Path 407 | total_timesteps 5483.
Path 408 | total_timesteps 5490.
Path 409 | total_timesteps 5499.
Path 410 | total_timesteps 5510.
Path 411 | total_timesteps 5537.
Path 412 | total_timesteps 5557.
Path 413 | total_timesteps 5574.
Path 414 | total_timesteps 5589.
Path 415 | total_timesteps 5602.
Path 416 | total_timesteps 5616.
Path 417 | total_timesteps 5623.
Path 418 | total_timesteps 5641.
Path 419 | total_timesteps 5664.
Path 420 | total_timesteps 5674.
Path 421 | total_timesteps 5686.
Path 422 | total_timesteps 5699.
Path 423 | total_timesteps 5708.
Path 424 | total_timesteps 5717.
Path 425 | total_timesteps 5727.
Path 426 | total_timesteps 5739.
Path 427 | total_timesteps 5756.
Path 428 | total_timesteps 5765.
Path 429 | total_timesteps 5775.
Path 430 | total_timesteps 5785.
Path 431 | total_timesteps 5814.
Path 432 | total_timesteps 5835.
Path 433 | total_timesteps 5847.
Path 434 | total_timesteps 5853.
Path 435 | total_timesteps 5869.
Path 436 | total_timesteps 5882.
Path 437 | total_timesteps 5907.
Path 438 | total_timesteps 5922.
Path 439 | total_timesteps 5932.
Path 440 | total_timesteps 5944.
Path 441 | total_timesteps 5953.
Path 442 | total_timesteps 5963.
Path 443 | total_timesteps 5976.
Path 444 | total_timesteps 5986.
Path 445 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.76    |
| Iteration     | 29       |
| MaximumReturn | 2.11     |
| MinimumReturn | -19.8    |
| TotalSamples  | 124167   |
----------------------------
itr #30 | 
Fitting dynamics.
Validation loss = 0.0025011475663632154
Validation loss = 0.002296878956258297
Validation loss = 0.002404862781986594
Validation loss = 0.0021950257942080498
Validation loss = 0.0022375949192792177
Validation loss = 0.0022117355838418007
Validation loss = 0.002348997863009572
Validation loss = 0.002360860351473093
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 44.
Path 5 | total_timesteps 55.
Path 6 | total_timesteps 80.
Path 7 | total_timesteps 90.
Path 8 | total_timesteps 102.
Path 9 | total_timesteps 112.
Path 10 | total_timesteps 124.
Path 11 | total_timesteps 135.
Path 12 | total_timesteps 145.
Path 13 | total_timesteps 158.
Path 14 | total_timesteps 185.
Path 15 | total_timesteps 200.
Path 16 | total_timesteps 207.
Path 17 | total_timesteps 218.
Path 18 | total_timesteps 233.
Path 19 | total_timesteps 246.
Path 20 | total_timesteps 289.
Path 21 | total_timesteps 302.
Path 22 | total_timesteps 313.
Path 23 | total_timesteps 331.
Path 24 | total_timesteps 372.
Path 25 | total_timesteps 388.
Path 26 | total_timesteps 396.
Path 27 | total_timesteps 410.
Path 28 | total_timesteps 421.
Path 29 | total_timesteps 430.
Path 30 | total_timesteps 443.
Path 31 | total_timesteps 468.
Path 32 | total_timesteps 481.
Path 33 | total_timesteps 499.
Path 34 | total_timesteps 509.
Path 35 | total_timesteps 525.
Path 36 | total_timesteps 533.
Path 37 | total_timesteps 546.
Path 38 | total_timesteps 558.
Path 39 | total_timesteps 573.
Path 40 | total_timesteps 585.
Path 41 | total_timesteps 598.
Path 42 | total_timesteps 636.
Path 43 | total_timesteps 648.
Path 44 | total_timesteps 669.
Path 45 | total_timesteps 676.
Path 46 | total_timesteps 690.
Path 47 | total_timesteps 701.
Path 48 | total_timesteps 720.
Path 49 | total_timesteps 734.
Path 50 | total_timesteps 747.
Path 51 | total_timesteps 756.
Path 52 | total_timesteps 775.
Path 53 | total_timesteps 786.
Path 54 | total_timesteps 793.
Path 55 | total_timesteps 801.
Path 56 | total_timesteps 811.
Path 57 | total_timesteps 824.
Path 58 | total_timesteps 832.
Path 59 | total_timesteps 843.
Path 60 | total_timesteps 853.
Path 61 | total_timesteps 869.
Path 62 | total_timesteps 885.
Path 63 | total_timesteps 896.
Path 64 | total_timesteps 903.
Path 65 | total_timesteps 914.
Path 66 | total_timesteps 931.
Path 67 | total_timesteps 948.
Path 68 | total_timesteps 958.
Path 69 | total_timesteps 966.
Path 70 | total_timesteps 985.
Path 71 | total_timesteps 994.
Path 72 | total_timesteps 1010.
Path 73 | total_timesteps 1019.
Path 74 | total_timesteps 1033.
Path 75 | total_timesteps 1048.
Path 76 | total_timesteps 1062.
Path 77 | total_timesteps 1075.
Path 78 | total_timesteps 1092.
Path 79 | total_timesteps 1107.
Path 80 | total_timesteps 1119.
Path 81 | total_timesteps 1129.
Path 82 | total_timesteps 1144.
Path 83 | total_timesteps 1153.
Path 84 | total_timesteps 1164.
Path 85 | total_timesteps 1175.
Path 86 | total_timesteps 1184.
Path 87 | total_timesteps 1210.
Path 88 | total_timesteps 1219.
Path 89 | total_timesteps 1228.
Path 90 | total_timesteps 1250.
Path 91 | total_timesteps 1262.
Path 92 | total_timesteps 1274.
Path 93 | total_timesteps 1288.
Path 94 | total_timesteps 1311.
Path 95 | total_timesteps 1322.
Path 96 | total_timesteps 1338.
Path 97 | total_timesteps 1348.
Path 98 | total_timesteps 1360.
Path 99 | total_timesteps 1371.
Path 100 | total_timesteps 1380.
Path 101 | total_timesteps 1394.
Path 102 | total_timesteps 1404.
Path 103 | total_timesteps 1414.
Path 104 | total_timesteps 1432.
Path 105 | total_timesteps 1442.
Path 106 | total_timesteps 1459.
Path 107 | total_timesteps 1475.
Path 108 | total_timesteps 1486.
Path 109 | total_timesteps 1497.
Path 110 | total_timesteps 1505.
Path 111 | total_timesteps 1519.
Path 112 | total_timesteps 1529.
Path 113 | total_timesteps 1560.
Path 114 | total_timesteps 1569.
Path 115 | total_timesteps 1584.
Path 116 | total_timesteps 1594.
Path 117 | total_timesteps 1608.
Path 118 | total_timesteps 1617.
Path 119 | total_timesteps 1634.
Path 120 | total_timesteps 1642.
Path 121 | total_timesteps 1654.
Path 122 | total_timesteps 1664.
Path 123 | total_timesteps 1681.
Path 124 | total_timesteps 1691.
Path 125 | total_timesteps 1701.
Path 126 | total_timesteps 1715.
Path 127 | total_timesteps 1733.
Path 128 | total_timesteps 1746.
Path 129 | total_timesteps 1764.
Path 130 | total_timesteps 1781.
Path 131 | total_timesteps 1798.
Path 132 | total_timesteps 1815.
Path 133 | total_timesteps 1825.
Path 134 | total_timesteps 1856.
Path 135 | total_timesteps 1882.
Path 136 | total_timesteps 1898.
Path 137 | total_timesteps 1912.
Path 138 | total_timesteps 1919.
Path 139 | total_timesteps 1933.
Path 140 | total_timesteps 1942.
Path 141 | total_timesteps 1953.
Path 142 | total_timesteps 1964.
Path 143 | total_timesteps 1986.
Path 144 | total_timesteps 1996.
Path 145 | total_timesteps 2009.
Path 146 | total_timesteps 2020.
Path 147 | total_timesteps 2032.
Path 148 | total_timesteps 2044.
Path 149 | total_timesteps 2058.
Path 150 | total_timesteps 2073.
Path 151 | total_timesteps 2086.
Path 152 | total_timesteps 2094.
Path 153 | total_timesteps 2104.
Path 154 | total_timesteps 2119.
Path 155 | total_timesteps 2133.
Path 156 | total_timesteps 2144.
Path 157 | total_timesteps 2155.
Path 158 | total_timesteps 2167.
Path 159 | total_timesteps 2187.
Path 160 | total_timesteps 2195.
Path 161 | total_timesteps 2207.
Path 162 | total_timesteps 2221.
Path 163 | total_timesteps 2241.
Path 164 | total_timesteps 2251.
Path 165 | total_timesteps 2270.
Path 166 | total_timesteps 2288.
Path 167 | total_timesteps 2299.
Path 168 | total_timesteps 2307.
Path 169 | total_timesteps 2314.
Path 170 | total_timesteps 2334.
Path 171 | total_timesteps 2347.
Path 172 | total_timesteps 2357.
Path 173 | total_timesteps 2365.
Path 174 | total_timesteps 2371.
Path 175 | total_timesteps 2385.
Path 176 | total_timesteps 2398.
Path 177 | total_timesteps 2406.
Path 178 | total_timesteps 2415.
Path 179 | total_timesteps 2423.
Path 180 | total_timesteps 2433.
Path 181 | total_timesteps 2448.
Path 182 | total_timesteps 2459.
Path 183 | total_timesteps 2468.
Path 184 | total_timesteps 2484.
Path 185 | total_timesteps 2503.
Path 186 | total_timesteps 2514.
Path 187 | total_timesteps 2523.
Path 188 | total_timesteps 2535.
Path 189 | total_timesteps 2546.
Path 190 | total_timesteps 2561.
Path 191 | total_timesteps 2571.
Path 192 | total_timesteps 2590.
Path 193 | total_timesteps 2608.
Path 194 | total_timesteps 2619.
Path 195 | total_timesteps 2632.
Path 196 | total_timesteps 2639.
Path 197 | total_timesteps 2650.
Path 198 | total_timesteps 2663.
Path 199 | total_timesteps 2672.
Path 200 | total_timesteps 2692.
Path 201 | total_timesteps 2711.
Path 202 | total_timesteps 2723.
Path 203 | total_timesteps 2732.
Path 204 | total_timesteps 2741.
Path 205 | total_timesteps 2752.
Path 206 | total_timesteps 2764.
Path 207 | total_timesteps 2779.
Path 208 | total_timesteps 2793.
Path 209 | total_timesteps 2802.
Path 210 | total_timesteps 2814.
Path 211 | total_timesteps 2828.
Path 212 | total_timesteps 2839.
Path 213 | total_timesteps 2857.
Path 214 | total_timesteps 2866.
Path 215 | total_timesteps 2884.
Path 216 | total_timesteps 2895.
Path 217 | total_timesteps 2907.
Path 218 | total_timesteps 2916.
Path 219 | total_timesteps 2923.
Path 220 | total_timesteps 2932.
Path 221 | total_timesteps 2945.
Path 222 | total_timesteps 2957.
Path 223 | total_timesteps 2968.
Path 224 | total_timesteps 2988.
Path 225 | total_timesteps 2997.
Path 226 | total_timesteps 3023.
Path 227 | total_timesteps 3036.
Path 228 | total_timesteps 3048.
Path 229 | total_timesteps 3065.
Path 230 | total_timesteps 3079.
Path 231 | total_timesteps 3088.
Path 232 | total_timesteps 3097.
Path 233 | total_timesteps 3111.
Path 234 | total_timesteps 3131.
Path 235 | total_timesteps 3141.
Path 236 | total_timesteps 3156.
Path 237 | total_timesteps 3175.
Path 238 | total_timesteps 3181.
Path 239 | total_timesteps 3191.
Path 240 | total_timesteps 3198.
Path 241 | total_timesteps 3211.
Path 242 | total_timesteps 3218.
Path 243 | total_timesteps 3231.
Path 244 | total_timesteps 3243.
Path 245 | total_timesteps 3254.
Path 246 | total_timesteps 3266.
Path 247 | total_timesteps 3278.
Path 248 | total_timesteps 3303.
Path 249 | total_timesteps 3317.
Path 250 | total_timesteps 3325.
Path 251 | total_timesteps 3342.
Path 252 | total_timesteps 3357.
Path 253 | total_timesteps 3379.
Path 254 | total_timesteps 3395.
Path 255 | total_timesteps 3406.
Path 256 | total_timesteps 3420.
Path 257 | total_timesteps 3435.
Path 258 | total_timesteps 3446.
Path 259 | total_timesteps 3458.
Path 260 | total_timesteps 3467.
Path 261 | total_timesteps 3478.
Path 262 | total_timesteps 3491.
Path 263 | total_timesteps 3501.
Path 264 | total_timesteps 3514.
Path 265 | total_timesteps 3527.
Path 266 | total_timesteps 3542.
Path 267 | total_timesteps 3552.
Path 268 | total_timesteps 3561.
Path 269 | total_timesteps 3582.
Path 270 | total_timesteps 3599.
Path 271 | total_timesteps 3609.
Path 272 | total_timesteps 3619.
Path 273 | total_timesteps 3630.
Path 274 | total_timesteps 3640.
Path 275 | total_timesteps 3652.
Path 276 | total_timesteps 3665.
Path 277 | total_timesteps 3672.
Path 278 | total_timesteps 3691.
Path 279 | total_timesteps 3706.
Path 280 | total_timesteps 3715.
Path 281 | total_timesteps 3723.
Path 282 | total_timesteps 3732.
Path 283 | total_timesteps 3744.
Path 284 | total_timesteps 3755.
Path 285 | total_timesteps 3772.
Path 286 | total_timesteps 3779.
Path 287 | total_timesteps 3791.
Path 288 | total_timesteps 3807.
Path 289 | total_timesteps 3814.
Path 290 | total_timesteps 3826.
Path 291 | total_timesteps 3837.
Path 292 | total_timesteps 3849.
Path 293 | total_timesteps 3866.
Path 294 | total_timesteps 3876.
Path 295 | total_timesteps 3885.
Path 296 | total_timesteps 3900.
Path 297 | total_timesteps 3910.
Path 298 | total_timesteps 3922.
Path 299 | total_timesteps 3931.
Path 300 | total_timesteps 3943.
Path 301 | total_timesteps 3963.
Path 302 | total_timesteps 3974.
Path 303 | total_timesteps 3983.
Path 304 | total_timesteps 3997.
Path 305 | total_timesteps 4013.
Path 306 | total_timesteps 4024.
Path 307 | total_timesteps 4032.
Path 308 | total_timesteps 4043.
Path 309 | total_timesteps 4060.
Path 310 | total_timesteps 4067.
Path 311 | total_timesteps 4079.
Path 312 | total_timesteps 4089.
Path 313 | total_timesteps 4098.
Path 314 | total_timesteps 4125.
Path 315 | total_timesteps 4136.
Path 316 | total_timesteps 4147.
Path 317 | total_timesteps 4162.
Path 318 | total_timesteps 4173.
Path 319 | total_timesteps 4183.
Path 320 | total_timesteps 4195.
Path 321 | total_timesteps 4204.
Path 322 | total_timesteps 4212.
Path 323 | total_timesteps 4221.
Path 324 | total_timesteps 4235.
Path 325 | total_timesteps 4247.
Path 326 | total_timesteps 4261.
Path 327 | total_timesteps 4274.
Path 328 | total_timesteps 4288.
Path 329 | total_timesteps 4304.
Path 330 | total_timesteps 4312.
Path 331 | total_timesteps 4324.
Path 332 | total_timesteps 4351.
Path 333 | total_timesteps 4364.
Path 334 | total_timesteps 4379.
Path 335 | total_timesteps 4392.
Path 336 | total_timesteps 4401.
Path 337 | total_timesteps 4431.
Path 338 | total_timesteps 4447.
Path 339 | total_timesteps 4456.
Path 340 | total_timesteps 4463.
Path 341 | total_timesteps 4487.
Path 342 | total_timesteps 4507.
Path 343 | total_timesteps 4517.
Path 344 | total_timesteps 4531.
Path 345 | total_timesteps 4551.
Path 346 | total_timesteps 4570.
Path 347 | total_timesteps 4583.
Path 348 | total_timesteps 4595.
Path 349 | total_timesteps 4614.
Path 350 | total_timesteps 4633.
Path 351 | total_timesteps 4640.
Path 352 | total_timesteps 4650.
Path 353 | total_timesteps 4659.
Path 354 | total_timesteps 4670.
Path 355 | total_timesteps 4681.
Path 356 | total_timesteps 4707.
Path 357 | total_timesteps 4716.
Path 358 | total_timesteps 4734.
Path 359 | total_timesteps 4750.
Path 360 | total_timesteps 4763.
Path 361 | total_timesteps 4773.
Path 362 | total_timesteps 4793.
Path 363 | total_timesteps 4801.
Path 364 | total_timesteps 4811.
Path 365 | total_timesteps 4821.
Path 366 | total_timesteps 4828.
Path 367 | total_timesteps 4846.
Path 368 | total_timesteps 4853.
Path 369 | total_timesteps 4862.
Path 370 | total_timesteps 4871.
Path 371 | total_timesteps 4893.
Path 372 | total_timesteps 4907.
Path 373 | total_timesteps 4917.
Path 374 | total_timesteps 4933.
Path 375 | total_timesteps 4943.
Path 376 | total_timesteps 4955.
Path 377 | total_timesteps 4966.
Path 378 | total_timesteps 4977.
Path 379 | total_timesteps 4988.
Path 380 | total_timesteps 4998.
Path 381 | total_timesteps 5011.
Path 382 | total_timesteps 5023.
Path 383 | total_timesteps 5032.
Path 384 | total_timesteps 5041.
Path 385 | total_timesteps 5048.
Path 386 | total_timesteps 5060.
Path 387 | total_timesteps 5072.
Path 388 | total_timesteps 5092.
Path 389 | total_timesteps 5111.
Path 390 | total_timesteps 5125.
Path 391 | total_timesteps 5135.
Path 392 | total_timesteps 5149.
Path 393 | total_timesteps 5160.
Path 394 | total_timesteps 5168.
Path 395 | total_timesteps 5177.
Path 396 | total_timesteps 5184.
Path 397 | total_timesteps 5197.
Path 398 | total_timesteps 5211.
Path 399 | total_timesteps 5231.
Path 400 | total_timesteps 5239.
Path 401 | total_timesteps 5248.
Path 402 | total_timesteps 5257.
Path 403 | total_timesteps 5266.
Path 404 | total_timesteps 5273.
Path 405 | total_timesteps 5285.
Path 406 | total_timesteps 5308.
Path 407 | total_timesteps 5317.
Path 408 | total_timesteps 5326.
Path 409 | total_timesteps 5341.
Path 410 | total_timesteps 5350.
Path 411 | total_timesteps 5370.
Path 412 | total_timesteps 5392.
Path 413 | total_timesteps 5403.
Path 414 | total_timesteps 5412.
Path 415 | total_timesteps 5424.
Path 416 | total_timesteps 5444.
Path 417 | total_timesteps 5459.
Path 418 | total_timesteps 5468.
Path 419 | total_timesteps 5476.
Path 420 | total_timesteps 5488.
Path 421 | total_timesteps 5506.
Path 422 | total_timesteps 5513.
Path 423 | total_timesteps 5521.
Path 424 | total_timesteps 5533.
Path 425 | total_timesteps 5564.
Path 426 | total_timesteps 5579.
Path 427 | total_timesteps 5600.
Path 428 | total_timesteps 5613.
Path 429 | total_timesteps 5630.
Path 430 | total_timesteps 5649.
Path 431 | total_timesteps 5667.
Path 432 | total_timesteps 5677.
Path 433 | total_timesteps 5685.
Path 434 | total_timesteps 5703.
Path 435 | total_timesteps 5713.
Path 436 | total_timesteps 5720.
Path 437 | total_timesteps 5729.
Path 438 | total_timesteps 5737.
Path 439 | total_timesteps 5747.
Path 440 | total_timesteps 5757.
Path 441 | total_timesteps 5771.
Path 442 | total_timesteps 5780.
Path 443 | total_timesteps 5793.
Path 444 | total_timesteps 5803.
Path 445 | total_timesteps 5812.
Path 446 | total_timesteps 5820.
Path 447 | total_timesteps 5847.
Path 448 | total_timesteps 5866.
Path 449 | total_timesteps 5881.
Path 450 | total_timesteps 5891.
Path 451 | total_timesteps 5912.
Path 452 | total_timesteps 5924.
Path 453 | total_timesteps 5941.
Path 454 | total_timesteps 5956.
Path 455 | total_timesteps 5966.
Path 456 | total_timesteps 5983.
Path 457 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.69    |
| Iteration     | 30       |
| MaximumReturn | 1.76     |
| MinimumReturn | -18.9    |
| TotalSamples  | 128167   |
----------------------------
itr #31 | 
Fitting dynamics.
Validation loss = 0.0021678509656339884
Validation loss = 0.0021215365268290043
Validation loss = 0.002357556950300932
Validation loss = 0.0022632647305727005
Validation loss = 0.0022179216612130404
Validation loss = 0.002260028850287199
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 46.
Path 4 | total_timesteps 58.
Path 5 | total_timesteps 66.
Path 6 | total_timesteps 82.
Path 7 | total_timesteps 90.
Path 8 | total_timesteps 101.
Path 9 | total_timesteps 112.
Path 10 | total_timesteps 122.
Path 11 | total_timesteps 137.
Path 12 | total_timesteps 144.
Path 13 | total_timesteps 157.
Path 14 | total_timesteps 174.
Path 15 | total_timesteps 184.
Path 16 | total_timesteps 197.
Path 17 | total_timesteps 207.
Path 18 | total_timesteps 222.
Path 19 | total_timesteps 233.
Path 20 | total_timesteps 246.
Path 21 | total_timesteps 254.
Path 22 | total_timesteps 262.
Path 23 | total_timesteps 289.
Path 24 | total_timesteps 320.
Path 25 | total_timesteps 333.
Path 26 | total_timesteps 347.
Path 27 | total_timesteps 359.
Path 28 | total_timesteps 377.
Path 29 | total_timesteps 389.
Path 30 | total_timesteps 400.
Path 31 | total_timesteps 413.
Path 32 | total_timesteps 424.
Path 33 | total_timesteps 445.
Path 34 | total_timesteps 456.
Path 35 | total_timesteps 468.
Path 36 | total_timesteps 480.
Path 37 | total_timesteps 510.
Path 38 | total_timesteps 522.
Path 39 | total_timesteps 533.
Path 40 | total_timesteps 569.
Path 41 | total_timesteps 578.
Path 42 | total_timesteps 595.
Path 43 | total_timesteps 608.
Path 44 | total_timesteps 615.
Path 45 | total_timesteps 625.
Path 46 | total_timesteps 636.
Path 47 | total_timesteps 648.
Path 48 | total_timesteps 671.
Path 49 | total_timesteps 681.
Path 50 | total_timesteps 692.
Path 51 | total_timesteps 711.
Path 52 | total_timesteps 717.
Path 53 | total_timesteps 734.
Path 54 | total_timesteps 750.
Path 55 | total_timesteps 763.
Path 56 | total_timesteps 790.
Path 57 | total_timesteps 805.
Path 58 | total_timesteps 819.
Path 59 | total_timesteps 827.
Path 60 | total_timesteps 837.
Path 61 | total_timesteps 848.
Path 62 | total_timesteps 857.
Path 63 | total_timesteps 867.
Path 64 | total_timesteps 876.
Path 65 | total_timesteps 896.
Path 66 | total_timesteps 903.
Path 67 | total_timesteps 910.
Path 68 | total_timesteps 919.
Path 69 | total_timesteps 932.
Path 70 | total_timesteps 951.
Path 71 | total_timesteps 960.
Path 72 | total_timesteps 975.
Path 73 | total_timesteps 986.
Path 74 | total_timesteps 997.
Path 75 | total_timesteps 1014.
Path 76 | total_timesteps 1025.
Path 77 | total_timesteps 1033.
Path 78 | total_timesteps 1046.
Path 79 | total_timesteps 1058.
Path 80 | total_timesteps 1067.
Path 81 | total_timesteps 1094.
Path 82 | total_timesteps 1107.
Path 83 | total_timesteps 1131.
Path 84 | total_timesteps 1144.
Path 85 | total_timesteps 1155.
Path 86 | total_timesteps 1169.
Path 87 | total_timesteps 1190.
Path 88 | total_timesteps 1206.
Path 89 | total_timesteps 1221.
Path 90 | total_timesteps 1243.
Path 91 | total_timesteps 1266.
Path 92 | total_timesteps 1281.
Path 93 | total_timesteps 1288.
Path 94 | total_timesteps 1295.
Path 95 | total_timesteps 1306.
Path 96 | total_timesteps 1325.
Path 97 | total_timesteps 1338.
Path 98 | total_timesteps 1353.
Path 99 | total_timesteps 1364.
Path 100 | total_timesteps 1382.
Path 101 | total_timesteps 1397.
Path 102 | total_timesteps 1412.
Path 103 | total_timesteps 1425.
Path 104 | total_timesteps 1434.
Path 105 | total_timesteps 1452.
Path 106 | total_timesteps 1459.
Path 107 | total_timesteps 1470.
Path 108 | total_timesteps 1490.
Path 109 | total_timesteps 1500.
Path 110 | total_timesteps 1512.
Path 111 | total_timesteps 1531.
Path 112 | total_timesteps 1549.
Path 113 | total_timesteps 1563.
Path 114 | total_timesteps 1572.
Path 115 | total_timesteps 1595.
Path 116 | total_timesteps 1604.
Path 117 | total_timesteps 1613.
Path 118 | total_timesteps 1621.
Path 119 | total_timesteps 1634.
Path 120 | total_timesteps 1649.
Path 121 | total_timesteps 1662.
Path 122 | total_timesteps 1675.
Path 123 | total_timesteps 1689.
Path 124 | total_timesteps 1708.
Path 125 | total_timesteps 1722.
Path 126 | total_timesteps 1731.
Path 127 | total_timesteps 1746.
Path 128 | total_timesteps 1765.
Path 129 | total_timesteps 1773.
Path 130 | total_timesteps 1782.
Path 131 | total_timesteps 1800.
Path 132 | total_timesteps 1809.
Path 133 | total_timesteps 1825.
Path 134 | total_timesteps 1836.
Path 135 | total_timesteps 1855.
Path 136 | total_timesteps 1874.
Path 137 | total_timesteps 1889.
Path 138 | total_timesteps 1900.
Path 139 | total_timesteps 1914.
Path 140 | total_timesteps 1929.
Path 141 | total_timesteps 1940.
Path 142 | total_timesteps 1956.
Path 143 | total_timesteps 1975.
Path 144 | total_timesteps 1985.
Path 145 | total_timesteps 1995.
Path 146 | total_timesteps 2017.
Path 147 | total_timesteps 2038.
Path 148 | total_timesteps 2054.
Path 149 | total_timesteps 2072.
Path 150 | total_timesteps 2081.
Path 151 | total_timesteps 2089.
Path 152 | total_timesteps 2099.
Path 153 | total_timesteps 2106.
Path 154 | total_timesteps 2121.
Path 155 | total_timesteps 2136.
Path 156 | total_timesteps 2147.
Path 157 | total_timesteps 2157.
Path 158 | total_timesteps 2167.
Path 159 | total_timesteps 2177.
Path 160 | total_timesteps 2185.
Path 161 | total_timesteps 2205.
Path 162 | total_timesteps 2217.
Path 163 | total_timesteps 2229.
Path 164 | total_timesteps 2240.
Path 165 | total_timesteps 2257.
Path 166 | total_timesteps 2275.
Path 167 | total_timesteps 2290.
Path 168 | total_timesteps 2298.
Path 169 | total_timesteps 2309.
Path 170 | total_timesteps 2325.
Path 171 | total_timesteps 2335.
Path 172 | total_timesteps 2355.
Path 173 | total_timesteps 2377.
Path 174 | total_timesteps 2387.
Path 175 | total_timesteps 2398.
Path 176 | total_timesteps 2410.
Path 177 | total_timesteps 2419.
Path 178 | total_timesteps 2431.
Path 179 | total_timesteps 2447.
Path 180 | total_timesteps 2464.
Path 181 | total_timesteps 2486.
Path 182 | total_timesteps 2497.
Path 183 | total_timesteps 2513.
Path 184 | total_timesteps 2522.
Path 185 | total_timesteps 2528.
Path 186 | total_timesteps 2536.
Path 187 | total_timesteps 2548.
Path 188 | total_timesteps 2561.
Path 189 | total_timesteps 2571.
Path 190 | total_timesteps 2579.
Path 191 | total_timesteps 2596.
Path 192 | total_timesteps 2605.
Path 193 | total_timesteps 2615.
Path 194 | total_timesteps 2623.
Path 195 | total_timesteps 2632.
Path 196 | total_timesteps 2644.
Path 197 | total_timesteps 2665.
Path 198 | total_timesteps 2679.
Path 199 | total_timesteps 2687.
Path 200 | total_timesteps 2697.
Path 201 | total_timesteps 2712.
Path 202 | total_timesteps 2725.
Path 203 | total_timesteps 2740.
Path 204 | total_timesteps 2752.
Path 205 | total_timesteps 2762.
Path 206 | total_timesteps 2772.
Path 207 | total_timesteps 2782.
Path 208 | total_timesteps 2796.
Path 209 | total_timesteps 2817.
Path 210 | total_timesteps 2845.
Path 211 | total_timesteps 2854.
Path 212 | total_timesteps 2862.
Path 213 | total_timesteps 2879.
Path 214 | total_timesteps 2893.
Path 215 | total_timesteps 2906.
Path 216 | total_timesteps 2917.
Path 217 | total_timesteps 2929.
Path 218 | total_timesteps 2940.
Path 219 | total_timesteps 2964.
Path 220 | total_timesteps 2975.
Path 221 | total_timesteps 2986.
Path 222 | total_timesteps 2996.
Path 223 | total_timesteps 3006.
Path 224 | total_timesteps 3018.
Path 225 | total_timesteps 3026.
Path 226 | total_timesteps 3039.
Path 227 | total_timesteps 3048.
Path 228 | total_timesteps 3063.
Path 229 | total_timesteps 3076.
Path 230 | total_timesteps 3091.
Path 231 | total_timesteps 3100.
Path 232 | total_timesteps 3128.
Path 233 | total_timesteps 3136.
Path 234 | total_timesteps 3145.
Path 235 | total_timesteps 3155.
Path 236 | total_timesteps 3167.
Path 237 | total_timesteps 3182.
Path 238 | total_timesteps 3191.
Path 239 | total_timesteps 3197.
Path 240 | total_timesteps 3210.
Path 241 | total_timesteps 3219.
Path 242 | total_timesteps 3239.
Path 243 | total_timesteps 3251.
Path 244 | total_timesteps 3261.
Path 245 | total_timesteps 3270.
Path 246 | total_timesteps 3279.
Path 247 | total_timesteps 3291.
Path 248 | total_timesteps 3300.
Path 249 | total_timesteps 3307.
Path 250 | total_timesteps 3317.
Path 251 | total_timesteps 3326.
Path 252 | total_timesteps 3341.
Path 253 | total_timesteps 3352.
Path 254 | total_timesteps 3375.
Path 255 | total_timesteps 3391.
Path 256 | total_timesteps 3403.
Path 257 | total_timesteps 3412.
Path 258 | total_timesteps 3420.
Path 259 | total_timesteps 3432.
Path 260 | total_timesteps 3450.
Path 261 | total_timesteps 3462.
Path 262 | total_timesteps 3473.
Path 263 | total_timesteps 3485.
Path 264 | total_timesteps 3497.
Path 265 | total_timesteps 3512.
Path 266 | total_timesteps 3530.
Path 267 | total_timesteps 3547.
Path 268 | total_timesteps 3559.
Path 269 | total_timesteps 3576.
Path 270 | total_timesteps 3587.
Path 271 | total_timesteps 3595.
Path 272 | total_timesteps 3614.
Path 273 | total_timesteps 3631.
Path 274 | total_timesteps 3641.
Path 275 | total_timesteps 3670.
Path 276 | total_timesteps 3691.
Path 277 | total_timesteps 3705.
Path 278 | total_timesteps 3713.
Path 279 | total_timesteps 3734.
Path 280 | total_timesteps 3744.
Path 281 | total_timesteps 3756.
Path 282 | total_timesteps 3764.
Path 283 | total_timesteps 3775.
Path 284 | total_timesteps 3808.
Path 285 | total_timesteps 3816.
Path 286 | total_timesteps 3825.
Path 287 | total_timesteps 3844.
Path 288 | total_timesteps 3869.
Path 289 | total_timesteps 3890.
Path 290 | total_timesteps 3903.
Path 291 | total_timesteps 3913.
Path 292 | total_timesteps 3927.
Path 293 | total_timesteps 3936.
Path 294 | total_timesteps 3943.
Path 295 | total_timesteps 3955.
Path 296 | total_timesteps 3978.
Path 297 | total_timesteps 3992.
Path 298 | total_timesteps 4001.
Path 299 | total_timesteps 4015.
Path 300 | total_timesteps 4026.
Path 301 | total_timesteps 4040.
Path 302 | total_timesteps 4051.
Path 303 | total_timesteps 4062.
Path 304 | total_timesteps 4070.
Path 305 | total_timesteps 4078.
Path 306 | total_timesteps 4089.
Path 307 | total_timesteps 4097.
Path 308 | total_timesteps 4120.
Path 309 | total_timesteps 4132.
Path 310 | total_timesteps 4143.
Path 311 | total_timesteps 4159.
Path 312 | total_timesteps 4187.
Path 313 | total_timesteps 4203.
Path 314 | total_timesteps 4212.
Path 315 | total_timesteps 4228.
Path 316 | total_timesteps 4238.
Path 317 | total_timesteps 4255.
Path 318 | total_timesteps 4265.
Path 319 | total_timesteps 4275.
Path 320 | total_timesteps 4285.
Path 321 | total_timesteps 4303.
Path 322 | total_timesteps 4317.
Path 323 | total_timesteps 4329.
Path 324 | total_timesteps 4341.
Path 325 | total_timesteps 4349.
Path 326 | total_timesteps 4362.
Path 327 | total_timesteps 4370.
Path 328 | total_timesteps 4388.
Path 329 | total_timesteps 4398.
Path 330 | total_timesteps 4407.
Path 331 | total_timesteps 4438.
Path 332 | total_timesteps 4450.
Path 333 | total_timesteps 4461.
Path 334 | total_timesteps 4471.
Path 335 | total_timesteps 4481.
Path 336 | total_timesteps 4488.
Path 337 | total_timesteps 4504.
Path 338 | total_timesteps 4513.
Path 339 | total_timesteps 4533.
Path 340 | total_timesteps 4546.
Path 341 | total_timesteps 4559.
Path 342 | total_timesteps 4568.
Path 343 | total_timesteps 4580.
Path 344 | total_timesteps 4593.
Path 345 | total_timesteps 4604.
Path 346 | total_timesteps 4615.
Path 347 | total_timesteps 4626.
Path 348 | total_timesteps 4656.
Path 349 | total_timesteps 4677.
Path 350 | total_timesteps 4688.
Path 351 | total_timesteps 4706.
Path 352 | total_timesteps 4716.
Path 353 | total_timesteps 4724.
Path 354 | total_timesteps 4734.
Path 355 | total_timesteps 4745.
Path 356 | total_timesteps 4756.
Path 357 | total_timesteps 4766.
Path 358 | total_timesteps 4775.
Path 359 | total_timesteps 4784.
Path 360 | total_timesteps 4795.
Path 361 | total_timesteps 4806.
Path 362 | total_timesteps 4817.
Path 363 | total_timesteps 4826.
Path 364 | total_timesteps 4835.
Path 365 | total_timesteps 4847.
Path 366 | total_timesteps 4860.
Path 367 | total_timesteps 4870.
Path 368 | total_timesteps 4878.
Path 369 | total_timesteps 4885.
Path 370 | total_timesteps 4903.
Path 371 | total_timesteps 4917.
Path 372 | total_timesteps 4932.
Path 373 | total_timesteps 4954.
Path 374 | total_timesteps 4965.
Path 375 | total_timesteps 4974.
Path 376 | total_timesteps 4985.
Path 377 | total_timesteps 4996.
Path 378 | total_timesteps 5014.
Path 379 | total_timesteps 5031.
Path 380 | total_timesteps 5041.
Path 381 | total_timesteps 5053.
Path 382 | total_timesteps 5071.
Path 383 | total_timesteps 5092.
Path 384 | total_timesteps 5104.
Path 385 | total_timesteps 5115.
Path 386 | total_timesteps 5123.
Path 387 | total_timesteps 5140.
Path 388 | total_timesteps 5151.
Path 389 | total_timesteps 5161.
Path 390 | total_timesteps 5171.
Path 391 | total_timesteps 5182.
Path 392 | total_timesteps 5200.
Path 393 | total_timesteps 5236.
Path 394 | total_timesteps 5244.
Path 395 | total_timesteps 5267.
Path 396 | total_timesteps 5287.
Path 397 | total_timesteps 5302.
Path 398 | total_timesteps 5311.
Path 399 | total_timesteps 5319.
Path 400 | total_timesteps 5329.
Path 401 | total_timesteps 5343.
Path 402 | total_timesteps 5367.
Path 403 | total_timesteps 5375.
Path 404 | total_timesteps 5384.
Path 405 | total_timesteps 5391.
Path 406 | total_timesteps 5401.
Path 407 | total_timesteps 5413.
Path 408 | total_timesteps 5421.
Path 409 | total_timesteps 5430.
Path 410 | total_timesteps 5442.
Path 411 | total_timesteps 5453.
Path 412 | total_timesteps 5465.
Path 413 | total_timesteps 5473.
Path 414 | total_timesteps 5484.
Path 415 | total_timesteps 5516.
Path 416 | total_timesteps 5535.
Path 417 | total_timesteps 5545.
Path 418 | total_timesteps 5556.
Path 419 | total_timesteps 5567.
Path 420 | total_timesteps 5578.
Path 421 | total_timesteps 5612.
Path 422 | total_timesteps 5635.
Path 423 | total_timesteps 5647.
Path 424 | total_timesteps 5656.
Path 425 | total_timesteps 5670.
Path 426 | total_timesteps 5682.
Path 427 | total_timesteps 5693.
Path 428 | total_timesteps 5721.
Path 429 | total_timesteps 5738.
Path 430 | total_timesteps 5751.
Path 431 | total_timesteps 5763.
Path 432 | total_timesteps 5797.
Path 433 | total_timesteps 5807.
Path 434 | total_timesteps 5817.
Path 435 | total_timesteps 5829.
Path 436 | total_timesteps 5838.
Path 437 | total_timesteps 5850.
Path 438 | total_timesteps 5881.
Path 439 | total_timesteps 5893.
Path 440 | total_timesteps 5903.
Path 441 | total_timesteps 5911.
Path 442 | total_timesteps 5922.
Path 443 | total_timesteps 5937.
Path 444 | total_timesteps 5950.
Path 445 | total_timesteps 5958.
Path 446 | total_timesteps 5969.
Path 447 | total_timesteps 5982.
Path 448 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.57    |
| Iteration     | 31       |
| MaximumReturn | 1.25     |
| MinimumReturn | -18.3    |
| TotalSamples  | 132169   |
----------------------------
itr #32 | 
Fitting dynamics.
Validation loss = 0.002195008797571063
Validation loss = 0.002411802066490054
Validation loss = 0.002115839160978794
Validation loss = 0.0023672098759561777
Validation loss = 0.0021562932524830103
Validation loss = 0.0021670700516551733
Validation loss = 0.0022904041688889265
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 31.
Path 4 | total_timesteps 45.
Path 5 | total_timesteps 62.
Path 6 | total_timesteps 73.
Path 7 | total_timesteps 88.
Path 8 | total_timesteps 101.
Path 9 | total_timesteps 110.
Path 10 | total_timesteps 119.
Path 11 | total_timesteps 125.
Path 12 | total_timesteps 138.
Path 13 | total_timesteps 147.
Path 14 | total_timesteps 165.
Path 15 | total_timesteps 173.
Path 16 | total_timesteps 187.
Path 17 | total_timesteps 205.
Path 18 | total_timesteps 216.
Path 19 | total_timesteps 222.
Path 20 | total_timesteps 246.
Path 21 | total_timesteps 255.
Path 22 | total_timesteps 266.
Path 23 | total_timesteps 277.
Path 24 | total_timesteps 290.
Path 25 | total_timesteps 304.
Path 26 | total_timesteps 314.
Path 27 | total_timesteps 322.
Path 28 | total_timesteps 335.
Path 29 | total_timesteps 358.
Path 30 | total_timesteps 372.
Path 31 | total_timesteps 398.
Path 32 | total_timesteps 410.
Path 33 | total_timesteps 416.
Path 34 | total_timesteps 434.
Path 35 | total_timesteps 444.
Path 36 | total_timesteps 460.
Path 37 | total_timesteps 480.
Path 38 | total_timesteps 494.
Path 39 | total_timesteps 506.
Path 40 | total_timesteps 517.
Path 41 | total_timesteps 537.
Path 42 | total_timesteps 552.
Path 43 | total_timesteps 572.
Path 44 | total_timesteps 582.
Path 45 | total_timesteps 592.
Path 46 | total_timesteps 610.
Path 47 | total_timesteps 621.
Path 48 | total_timesteps 636.
Path 49 | total_timesteps 645.
Path 50 | total_timesteps 655.
Path 51 | total_timesteps 670.
Path 52 | total_timesteps 686.
Path 53 | total_timesteps 703.
Path 54 | total_timesteps 710.
Path 55 | total_timesteps 726.
Path 56 | total_timesteps 739.
Path 57 | total_timesteps 752.
Path 58 | total_timesteps 761.
Path 59 | total_timesteps 770.
Path 60 | total_timesteps 782.
Path 61 | total_timesteps 796.
Path 62 | total_timesteps 808.
Path 63 | total_timesteps 822.
Path 64 | total_timesteps 837.
Path 65 | total_timesteps 844.
Path 66 | total_timesteps 854.
Path 67 | total_timesteps 866.
Path 68 | total_timesteps 880.
Path 69 | total_timesteps 889.
Path 70 | total_timesteps 897.
Path 71 | total_timesteps 908.
Path 72 | total_timesteps 923.
Path 73 | total_timesteps 934.
Path 74 | total_timesteps 953.
Path 75 | total_timesteps 972.
Path 76 | total_timesteps 985.
Path 77 | total_timesteps 1003.
Path 78 | total_timesteps 1013.
Path 79 | total_timesteps 1024.
Path 80 | total_timesteps 1039.
Path 81 | total_timesteps 1048.
Path 82 | total_timesteps 1057.
Path 83 | total_timesteps 1070.
Path 84 | total_timesteps 1080.
Path 85 | total_timesteps 1100.
Path 86 | total_timesteps 1108.
Path 87 | total_timesteps 1118.
Path 88 | total_timesteps 1129.
Path 89 | total_timesteps 1143.
Path 90 | total_timesteps 1156.
Path 91 | total_timesteps 1167.
Path 92 | total_timesteps 1176.
Path 93 | total_timesteps 1191.
Path 94 | total_timesteps 1203.
Path 95 | total_timesteps 1214.
Path 96 | total_timesteps 1222.
Path 97 | total_timesteps 1230.
Path 98 | total_timesteps 1263.
Path 99 | total_timesteps 1271.
Path 100 | total_timesteps 1283.
Path 101 | total_timesteps 1292.
Path 102 | total_timesteps 1316.
Path 103 | total_timesteps 1327.
Path 104 | total_timesteps 1338.
Path 105 | total_timesteps 1352.
Path 106 | total_timesteps 1363.
Path 107 | total_timesteps 1377.
Path 108 | total_timesteps 1390.
Path 109 | total_timesteps 1402.
Path 110 | total_timesteps 1419.
Path 111 | total_timesteps 1431.
Path 112 | total_timesteps 1442.
Path 113 | total_timesteps 1457.
Path 114 | total_timesteps 1469.
Path 115 | total_timesteps 1479.
Path 116 | total_timesteps 1493.
Path 117 | total_timesteps 1509.
Path 118 | total_timesteps 1522.
Path 119 | total_timesteps 1531.
Path 120 | total_timesteps 1543.
Path 121 | total_timesteps 1553.
Path 122 | total_timesteps 1560.
Path 123 | total_timesteps 1568.
Path 124 | total_timesteps 1593.
Path 125 | total_timesteps 1603.
Path 126 | total_timesteps 1614.
Path 127 | total_timesteps 1627.
Path 128 | total_timesteps 1648.
Path 129 | total_timesteps 1659.
Path 130 | total_timesteps 1668.
Path 131 | total_timesteps 1678.
Path 132 | total_timesteps 1692.
Path 133 | total_timesteps 1702.
Path 134 | total_timesteps 1712.
Path 135 | total_timesteps 1725.
Path 136 | total_timesteps 1734.
Path 137 | total_timesteps 1759.
Path 138 | total_timesteps 1790.
Path 139 | total_timesteps 1805.
Path 140 | total_timesteps 1813.
Path 141 | total_timesteps 1827.
Path 142 | total_timesteps 1844.
Path 143 | total_timesteps 1863.
Path 144 | total_timesteps 1875.
Path 145 | total_timesteps 1883.
Path 146 | total_timesteps 1894.
Path 147 | total_timesteps 1905.
Path 148 | total_timesteps 1915.
Path 149 | total_timesteps 1926.
Path 150 | total_timesteps 1949.
Path 151 | total_timesteps 1959.
Path 152 | total_timesteps 1967.
Path 153 | total_timesteps 1979.
Path 154 | total_timesteps 2004.
Path 155 | total_timesteps 2026.
Path 156 | total_timesteps 2039.
Path 157 | total_timesteps 2048.
Path 158 | total_timesteps 2059.
Path 159 | total_timesteps 2068.
Path 160 | total_timesteps 2078.
Path 161 | total_timesteps 2088.
Path 162 | total_timesteps 2096.
Path 163 | total_timesteps 2108.
Path 164 | total_timesteps 2120.
Path 165 | total_timesteps 2128.
Path 166 | total_timesteps 2140.
Path 167 | total_timesteps 2157.
Path 168 | total_timesteps 2168.
Path 169 | total_timesteps 2183.
Path 170 | total_timesteps 2215.
Path 171 | total_timesteps 2227.
Path 172 | total_timesteps 2240.
Path 173 | total_timesteps 2253.
Path 174 | total_timesteps 2267.
Path 175 | total_timesteps 2293.
Path 176 | total_timesteps 2313.
Path 177 | total_timesteps 2324.
Path 178 | total_timesteps 2344.
Path 179 | total_timesteps 2352.
Path 180 | total_timesteps 2362.
Path 181 | total_timesteps 2373.
Path 182 | total_timesteps 2382.
Path 183 | total_timesteps 2393.
Path 184 | total_timesteps 2404.
Path 185 | total_timesteps 2418.
Path 186 | total_timesteps 2427.
Path 187 | total_timesteps 2436.
Path 188 | total_timesteps 2448.
Path 189 | total_timesteps 2465.
Path 190 | total_timesteps 2476.
Path 191 | total_timesteps 2492.
Path 192 | total_timesteps 2504.
Path 193 | total_timesteps 2518.
Path 194 | total_timesteps 2525.
Path 195 | total_timesteps 2534.
Path 196 | total_timesteps 2549.
Path 197 | total_timesteps 2562.
Path 198 | total_timesteps 2575.
Path 199 | total_timesteps 2587.
Path 200 | total_timesteps 2599.
Path 201 | total_timesteps 2615.
Path 202 | total_timesteps 2635.
Path 203 | total_timesteps 2644.
Path 204 | total_timesteps 2651.
Path 205 | total_timesteps 2660.
Path 206 | total_timesteps 2676.
Path 207 | total_timesteps 2686.
Path 208 | total_timesteps 2695.
Path 209 | total_timesteps 2705.
Path 210 | total_timesteps 2716.
Path 211 | total_timesteps 2739.
Path 212 | total_timesteps 2751.
Path 213 | total_timesteps 2760.
Path 214 | total_timesteps 2770.
Path 215 | total_timesteps 2777.
Path 216 | total_timesteps 2787.
Path 217 | total_timesteps 2811.
Path 218 | total_timesteps 2817.
Path 219 | total_timesteps 2826.
Path 220 | total_timesteps 2843.
Path 221 | total_timesteps 2858.
Path 222 | total_timesteps 2868.
Path 223 | total_timesteps 2879.
Path 224 | total_timesteps 2887.
Path 225 | total_timesteps 2897.
Path 226 | total_timesteps 2906.
Path 227 | total_timesteps 2915.
Path 228 | total_timesteps 2926.
Path 229 | total_timesteps 2937.
Path 230 | total_timesteps 2956.
Path 231 | total_timesteps 2968.
Path 232 | total_timesteps 2983.
Path 233 | total_timesteps 2997.
Path 234 | total_timesteps 3007.
Path 235 | total_timesteps 3020.
Path 236 | total_timesteps 3031.
Path 237 | total_timesteps 3042.
Path 238 | total_timesteps 3052.
Path 239 | total_timesteps 3071.
Path 240 | total_timesteps 3089.
Path 241 | total_timesteps 3097.
Path 242 | total_timesteps 3105.
Path 243 | total_timesteps 3116.
Path 244 | total_timesteps 3129.
Path 245 | total_timesteps 3141.
Path 246 | total_timesteps 3149.
Path 247 | total_timesteps 3158.
Path 248 | total_timesteps 3165.
Path 249 | total_timesteps 3177.
Path 250 | total_timesteps 3194.
Path 251 | total_timesteps 3212.
Path 252 | total_timesteps 3223.
Path 253 | total_timesteps 3232.
Path 254 | total_timesteps 3246.
Path 255 | total_timesteps 3253.
Path 256 | total_timesteps 3260.
Path 257 | total_timesteps 3272.
Path 258 | total_timesteps 3280.
Path 259 | total_timesteps 3290.
Path 260 | total_timesteps 3297.
Path 261 | total_timesteps 3304.
Path 262 | total_timesteps 3319.
Path 263 | total_timesteps 3333.
Path 264 | total_timesteps 3342.
Path 265 | total_timesteps 3359.
Path 266 | total_timesteps 3368.
Path 267 | total_timesteps 3378.
Path 268 | total_timesteps 3388.
Path 269 | total_timesteps 3399.
Path 270 | total_timesteps 3422.
Path 271 | total_timesteps 3443.
Path 272 | total_timesteps 3454.
Path 273 | total_timesteps 3465.
Path 274 | total_timesteps 3481.
Path 275 | total_timesteps 3499.
Path 276 | total_timesteps 3508.
Path 277 | total_timesteps 3523.
Path 278 | total_timesteps 3535.
Path 279 | total_timesteps 3557.
Path 280 | total_timesteps 3579.
Path 281 | total_timesteps 3586.
Path 282 | total_timesteps 3603.
Path 283 | total_timesteps 3617.
Path 284 | total_timesteps 3628.
Path 285 | total_timesteps 3639.
Path 286 | total_timesteps 3648.
Path 287 | total_timesteps 3658.
Path 288 | total_timesteps 3666.
Path 289 | total_timesteps 3676.
Path 290 | total_timesteps 3683.
Path 291 | total_timesteps 3705.
Path 292 | total_timesteps 3718.
Path 293 | total_timesteps 3730.
Path 294 | total_timesteps 3741.
Path 295 | total_timesteps 3753.
Path 296 | total_timesteps 3762.
Path 297 | total_timesteps 3774.
Path 298 | total_timesteps 3788.
Path 299 | total_timesteps 3797.
Path 300 | total_timesteps 3808.
Path 301 | total_timesteps 3832.
Path 302 | total_timesteps 3839.
Path 303 | total_timesteps 3846.
Path 304 | total_timesteps 3860.
Path 305 | total_timesteps 3888.
Path 306 | total_timesteps 3899.
Path 307 | total_timesteps 3911.
Path 308 | total_timesteps 3923.
Path 309 | total_timesteps 3932.
Path 310 | total_timesteps 3955.
Path 311 | total_timesteps 3967.
Path 312 | total_timesteps 3978.
Path 313 | total_timesteps 3995.
Path 314 | total_timesteps 4008.
Path 315 | total_timesteps 4018.
Path 316 | total_timesteps 4027.
Path 317 | total_timesteps 4037.
Path 318 | total_timesteps 4047.
Path 319 | total_timesteps 4065.
Path 320 | total_timesteps 4081.
Path 321 | total_timesteps 4088.
Path 322 | total_timesteps 4097.
Path 323 | total_timesteps 4117.
Path 324 | total_timesteps 4133.
Path 325 | total_timesteps 4149.
Path 326 | total_timesteps 4170.
Path 327 | total_timesteps 4181.
Path 328 | total_timesteps 4190.
Path 329 | total_timesteps 4200.
Path 330 | total_timesteps 4208.
Path 331 | total_timesteps 4219.
Path 332 | total_timesteps 4231.
Path 333 | total_timesteps 4243.
Path 334 | total_timesteps 4255.
Path 335 | total_timesteps 4272.
Path 336 | total_timesteps 4284.
Path 337 | total_timesteps 4291.
Path 338 | total_timesteps 4302.
Path 339 | total_timesteps 4311.
Path 340 | total_timesteps 4325.
Path 341 | total_timesteps 4338.
Path 342 | total_timesteps 4352.
Path 343 | total_timesteps 4361.
Path 344 | total_timesteps 4376.
Path 345 | total_timesteps 4386.
Path 346 | total_timesteps 4397.
Path 347 | total_timesteps 4406.
Path 348 | total_timesteps 4418.
Path 349 | total_timesteps 4429.
Path 350 | total_timesteps 4445.
Path 351 | total_timesteps 4455.
Path 352 | total_timesteps 4464.
Path 353 | total_timesteps 4477.
Path 354 | total_timesteps 4487.
Path 355 | total_timesteps 4498.
Path 356 | total_timesteps 4504.
Path 357 | total_timesteps 4516.
Path 358 | total_timesteps 4531.
Path 359 | total_timesteps 4544.
Path 360 | total_timesteps 4558.
Path 361 | total_timesteps 4579.
Path 362 | total_timesteps 4593.
Path 363 | total_timesteps 4601.
Path 364 | total_timesteps 4625.
Path 365 | total_timesteps 4635.
Path 366 | total_timesteps 4643.
Path 367 | total_timesteps 4654.
Path 368 | total_timesteps 4666.
Path 369 | total_timesteps 4679.
Path 370 | total_timesteps 4688.
Path 371 | total_timesteps 4706.
Path 372 | total_timesteps 4724.
Path 373 | total_timesteps 4733.
Path 374 | total_timesteps 4747.
Path 375 | total_timesteps 4756.
Path 376 | total_timesteps 4777.
Path 377 | total_timesteps 4787.
Path 378 | total_timesteps 4798.
Path 379 | total_timesteps 4814.
Path 380 | total_timesteps 4824.
Path 381 | total_timesteps 4832.
Path 382 | total_timesteps 4854.
Path 383 | total_timesteps 4863.
Path 384 | total_timesteps 4887.
Path 385 | total_timesteps 4912.
Path 386 | total_timesteps 4922.
Path 387 | total_timesteps 4940.
Path 388 | total_timesteps 4954.
Path 389 | total_timesteps 4966.
Path 390 | total_timesteps 4984.
Path 391 | total_timesteps 5005.
Path 392 | total_timesteps 5017.
Path 393 | total_timesteps 5037.
Path 394 | total_timesteps 5051.
Path 395 | total_timesteps 5059.
Path 396 | total_timesteps 5069.
Path 397 | total_timesteps 5080.
Path 398 | total_timesteps 5105.
Path 399 | total_timesteps 5115.
Path 400 | total_timesteps 5124.
Path 401 | total_timesteps 5136.
Path 402 | total_timesteps 5149.
Path 403 | total_timesteps 5159.
Path 404 | total_timesteps 5169.
Path 405 | total_timesteps 5178.
Path 406 | total_timesteps 5202.
Path 407 | total_timesteps 5211.
Path 408 | total_timesteps 5224.
Path 409 | total_timesteps 5243.
Path 410 | total_timesteps 5254.
Path 411 | total_timesteps 5263.
Path 412 | total_timesteps 5275.
Path 413 | total_timesteps 5284.
Path 414 | total_timesteps 5290.
Path 415 | total_timesteps 5297.
Path 416 | total_timesteps 5306.
Path 417 | total_timesteps 5314.
Path 418 | total_timesteps 5336.
Path 419 | total_timesteps 5345.
Path 420 | total_timesteps 5360.
Path 421 | total_timesteps 5377.
Path 422 | total_timesteps 5389.
Path 423 | total_timesteps 5397.
Path 424 | total_timesteps 5421.
Path 425 | total_timesteps 5428.
Path 426 | total_timesteps 5444.
Path 427 | total_timesteps 5459.
Path 428 | total_timesteps 5478.
Path 429 | total_timesteps 5517.
Path 430 | total_timesteps 5526.
Path 431 | total_timesteps 5537.
Path 432 | total_timesteps 5557.
Path 433 | total_timesteps 5568.
Path 434 | total_timesteps 5586.
Path 435 | total_timesteps 5598.
Path 436 | total_timesteps 5608.
Path 437 | total_timesteps 5629.
Path 438 | total_timesteps 5636.
Path 439 | total_timesteps 5649.
Path 440 | total_timesteps 5660.
Path 441 | total_timesteps 5677.
Path 442 | total_timesteps 5685.
Path 443 | total_timesteps 5695.
Path 444 | total_timesteps 5706.
Path 445 | total_timesteps 5716.
Path 446 | total_timesteps 5730.
Path 447 | total_timesteps 5740.
Path 448 | total_timesteps 5752.
Path 449 | total_timesteps 5763.
Path 450 | total_timesteps 5783.
Path 451 | total_timesteps 5794.
Path 452 | total_timesteps 5811.
Path 453 | total_timesteps 5818.
Path 454 | total_timesteps 5833.
Path 455 | total_timesteps 5843.
Path 456 | total_timesteps 5852.
Path 457 | total_timesteps 5859.
Path 458 | total_timesteps 5871.
Path 459 | total_timesteps 5883.
Path 460 | total_timesteps 5893.
Path 461 | total_timesteps 5904.
Path 462 | total_timesteps 5913.
Path 463 | total_timesteps 5932.
Path 464 | total_timesteps 5952.
Path 465 | total_timesteps 5981.
Path 466 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.37    |
| Iteration     | 32       |
| MaximumReturn | 8.16     |
| MinimumReturn | -18.4    |
| TotalSamples  | 136177   |
----------------------------
