Logging to experiments/gym_fwalker2d/Wa01/Mon-07-Nov-2022-10-29-40-AM-CST_gym_fwalker2d_trpo_iteration_20_seed2431
Print configuration .....
{'env_name': 'gym_fwalker2d', 'random_seeds': [3214, 2431, 2531, 2231], 'save_variables': False, 'model_save_dir': '/tmp/gym_fwalker2d_models/', 'restore_variables': False, 'start_onpol_iter': 0, 'onpol_iters': 33, 'num_path_random': 6, 'num_path_onpol': 6, 'env_horizon': 1000, 'max_train_data': 200000, 'max_val_data': 100000, 'discard_ratio': 0.0, 'dynamics': {'pre_training': {'mode': 'intrinsic_reward', 'itr': 0, 'policy_itr': 20}, 'model': 'nn', 'ensemble': False, 'ensemble_model_count': 5, 'enable_particle_ensemble': True, 'particles': 5, 'obs_var': 1.0, 'intrinsic_reward_coeff': 1.0, 'ita': 1.0, 'mode': 'random', 'val': True, 'n_layers': 4, 'hidden_size': 1000, 'activation': 'relu', 'batch_size': 1000, 'learning_rate': 0.001, 'reg_coeff': 0.0, 'epochs': 200, 'kfac_params': {'learning_rate': 0.1, 'damping': 0.001, 'momentum': 0.9, 'kl_clip': 0.0001, 'cov_ema_decay': 0.99}}, 'policy': {'network_shape': [64, 64], 'init_logstd': 0.0, 'activation': 'tanh', 'reinitialize_every_itr': False}, 'trpo': {'horizon': 1000, 'gamma': 0.99, 'step_size': 0.01, 'iterations': 20, 'batch_size': 50000, 'gae': 0.95, 'visualization': False, 'visualize_iterations': [0]}, 'algo': 'trpo'}
Generating random rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 46.
Path 4 | total_timesteps 78.
Path 5 | total_timesteps 96.
Path 6 | total_timesteps 143.
Path 7 | total_timesteps 154.
Path 8 | total_timesteps 179.
Path 9 | total_timesteps 198.
Path 10 | total_timesteps 209.
Path 11 | total_timesteps 226.
Path 12 | total_timesteps 242.
Path 13 | total_timesteps 252.
Path 14 | total_timesteps 265.
Path 15 | total_timesteps 275.
Path 16 | total_timesteps 291.
Path 17 | total_timesteps 313.
Path 18 | total_timesteps 327.
Path 19 | total_timesteps 335.
Path 20 | total_timesteps 353.
Path 21 | total_timesteps 371.
Path 22 | total_timesteps 387.
Path 23 | total_timesteps 418.
Path 24 | total_timesteps 430.
Path 25 | total_timesteps 460.
Path 26 | total_timesteps 478.
Path 27 | total_timesteps 505.
Path 28 | total_timesteps 521.
Path 29 | total_timesteps 551.
Path 30 | total_timesteps 575.
Path 31 | total_timesteps 590.
Path 32 | total_timesteps 602.
Path 33 | total_timesteps 610.
Path 34 | total_timesteps 648.
Path 35 | total_timesteps 672.
Path 36 | total_timesteps 705.
Path 37 | total_timesteps 725.
Path 38 | total_timesteps 757.
Path 39 | total_timesteps 783.
Path 40 | total_timesteps 809.
Path 41 | total_timesteps 823.
Path 42 | total_timesteps 833.
Path 43 | total_timesteps 842.
Path 44 | total_timesteps 867.
Path 45 | total_timesteps 892.
Path 46 | total_timesteps 915.
Path 47 | total_timesteps 932.
Path 48 | total_timesteps 953.
Path 49 | total_timesteps 971.
Path 50 | total_timesteps 989.
Path 51 | total_timesteps 1001.
Path 52 | total_timesteps 1010.
Path 53 | total_timesteps 1022.
Path 54 | total_timesteps 1043.
Path 55 | total_timesteps 1058.
Path 56 | total_timesteps 1079.
Path 57 | total_timesteps 1095.
Path 58 | total_timesteps 1106.
Path 59 | total_timesteps 1124.
Path 60 | total_timesteps 1158.
Path 61 | total_timesteps 1167.
Path 62 | total_timesteps 1188.
Path 63 | total_timesteps 1198.
Path 64 | total_timesteps 1213.
Path 65 | total_timesteps 1227.
Path 66 | total_timesteps 1247.
Path 67 | total_timesteps 1258.
Path 68 | total_timesteps 1276.
Path 69 | total_timesteps 1291.
Path 70 | total_timesteps 1302.
Path 71 | total_timesteps 1326.
Path 72 | total_timesteps 1347.
Path 73 | total_timesteps 1364.
Path 74 | total_timesteps 1379.
Path 75 | total_timesteps 1397.
Path 76 | total_timesteps 1405.
Path 77 | total_timesteps 1432.
Path 78 | total_timesteps 1448.
Path 79 | total_timesteps 1458.
Path 80 | total_timesteps 1481.
Path 81 | total_timesteps 1492.
Path 82 | total_timesteps 1505.
Path 83 | total_timesteps 1555.
Path 84 | total_timesteps 1569.
Path 85 | total_timesteps 1582.
Path 86 | total_timesteps 1593.
Path 87 | total_timesteps 1618.
Path 88 | total_timesteps 1633.
Path 89 | total_timesteps 1669.
Path 90 | total_timesteps 1689.
Path 91 | total_timesteps 1708.
Path 92 | total_timesteps 1727.
Path 93 | total_timesteps 1739.
Path 94 | total_timesteps 1755.
Path 95 | total_timesteps 1765.
Path 96 | total_timesteps 1791.
Path 97 | total_timesteps 1818.
Path 98 | total_timesteps 1829.
Path 99 | total_timesteps 1844.
Path 100 | total_timesteps 1868.
Path 101 | total_timesteps 1879.
Path 102 | total_timesteps 1908.
Path 103 | total_timesteps 1931.
Path 104 | total_timesteps 1982.
Path 105 | total_timesteps 1999.
Path 106 | total_timesteps 2035.
Path 107 | total_timesteps 2049.
Path 108 | total_timesteps 2061.
Path 109 | total_timesteps 2080.
Path 110 | total_timesteps 2099.
Path 111 | total_timesteps 2116.
Path 112 | total_timesteps 2149.
Path 113 | total_timesteps 2161.
Path 114 | total_timesteps 2171.
Path 115 | total_timesteps 2184.
Path 116 | total_timesteps 2201.
Path 117 | total_timesteps 2219.
Path 118 | total_timesteps 2250.
Path 119 | total_timesteps 2285.
Path 120 | total_timesteps 2310.
Path 121 | total_timesteps 2325.
Path 122 | total_timesteps 2348.
Path 123 | total_timesteps 2362.
Path 124 | total_timesteps 2384.
Path 125 | total_timesteps 2403.
Path 126 | total_timesteps 2420.
Path 127 | total_timesteps 2434.
Path 128 | total_timesteps 2448.
Path 129 | total_timesteps 2469.
Path 130 | total_timesteps 2530.
Path 131 | total_timesteps 2557.
Path 132 | total_timesteps 2584.
Path 133 | total_timesteps 2598.
Path 134 | total_timesteps 2610.
Path 135 | total_timesteps 2623.
Path 136 | total_timesteps 2644.
Path 137 | total_timesteps 2657.
Path 138 | total_timesteps 2672.
Path 139 | total_timesteps 2687.
Path 140 | total_timesteps 2703.
Path 141 | total_timesteps 2714.
Path 142 | total_timesteps 2728.
Path 143 | total_timesteps 2743.
Path 144 | total_timesteps 2759.
Path 145 | total_timesteps 2773.
Path 146 | total_timesteps 2788.
Path 147 | total_timesteps 2805.
Path 148 | total_timesteps 2833.
Path 149 | total_timesteps 2863.
Path 150 | total_timesteps 2892.
Path 151 | total_timesteps 2905.
Path 152 | total_timesteps 2935.
Path 153 | total_timesteps 2946.
Path 154 | total_timesteps 2965.
Path 155 | total_timesteps 2993.
Path 156 | total_timesteps 3004.
Path 157 | total_timesteps 3018.
Path 158 | total_timesteps 3039.
Path 159 | total_timesteps 3071.
Path 160 | total_timesteps 3101.
Path 161 | total_timesteps 3115.
Path 162 | total_timesteps 3133.
Path 163 | total_timesteps 3145.
Path 164 | total_timesteps 3162.
Path 165 | total_timesteps 3196.
Path 166 | total_timesteps 3212.
Path 167 | total_timesteps 3232.
Path 168 | total_timesteps 3248.
Path 169 | total_timesteps 3266.
Path 170 | total_timesteps 3286.
Path 171 | total_timesteps 3315.
Path 172 | total_timesteps 3338.
Path 173 | total_timesteps 3368.
Path 174 | total_timesteps 3393.
Path 175 | total_timesteps 3405.
Path 176 | total_timesteps 3424.
Path 177 | total_timesteps 3457.
Path 178 | total_timesteps 3468.
Path 179 | total_timesteps 3482.
Path 180 | total_timesteps 3515.
Path 181 | total_timesteps 3541.
Path 182 | total_timesteps 3563.
Path 183 | total_timesteps 3583.
Path 184 | total_timesteps 3592.
Path 185 | total_timesteps 3633.
Path 186 | total_timesteps 3647.
Path 187 | total_timesteps 3667.
Path 188 | total_timesteps 3700.
Path 189 | total_timesteps 3713.
Path 190 | total_timesteps 3726.
Path 191 | total_timesteps 3743.
Path 192 | total_timesteps 3756.
Path 193 | total_timesteps 3789.
Path 194 | total_timesteps 3811.
Path 195 | total_timesteps 3820.
Path 196 | total_timesteps 3841.
Path 197 | total_timesteps 3859.
Path 198 | total_timesteps 3872.
Path 199 | total_timesteps 3911.
Path 200 | total_timesteps 3923.
Path 201 | total_timesteps 3934.
Path 202 | total_timesteps 3946.
Path 203 | total_timesteps 3995.
Path 204 | total_timesteps 4024.
Path 205 | total_timesteps 4046.
Path 206 | total_timesteps 4058.
Path 207 | total_timesteps 4080.
Path 208 | total_timesteps 4091.
Path 209 | total_timesteps 4106.
Path 210 | total_timesteps 4132.
Path 211 | total_timesteps 4152.
Path 212 | total_timesteps 4205.
Path 213 | total_timesteps 4219.
Path 214 | total_timesteps 4235.
Path 215 | total_timesteps 4249.
Path 216 | total_timesteps 4261.
Path 217 | total_timesteps 4289.
Path 218 | total_timesteps 4334.
Path 219 | total_timesteps 4351.
Path 220 | total_timesteps 4377.
Path 221 | total_timesteps 4400.
Path 222 | total_timesteps 4429.
Path 223 | total_timesteps 4443.
Path 224 | total_timesteps 4458.
Path 225 | total_timesteps 4474.
Path 226 | total_timesteps 4507.
Path 227 | total_timesteps 4516.
Path 228 | total_timesteps 4531.
Path 229 | total_timesteps 4557.
Path 230 | total_timesteps 4577.
Path 231 | total_timesteps 4594.
Path 232 | total_timesteps 4614.
Path 233 | total_timesteps 4630.
Path 234 | total_timesteps 4645.
Path 235 | total_timesteps 4676.
Path 236 | total_timesteps 4692.
Path 237 | total_timesteps 4707.
Path 238 | total_timesteps 4717.
Path 239 | total_timesteps 4733.
Path 240 | total_timesteps 4747.
Path 241 | total_timesteps 4758.
Path 242 | total_timesteps 4775.
Path 243 | total_timesteps 4790.
Path 244 | total_timesteps 4814.
Path 245 | total_timesteps 4837.
Path 246 | total_timesteps 4860.
Path 247 | total_timesteps 4885.
Path 248 | total_timesteps 4910.
Path 249 | total_timesteps 4930.
Path 250 | total_timesteps 4944.
Path 251 | total_timesteps 4991.
Path 252 | total_timesteps 5019.
Path 253 | total_timesteps 5063.
Path 254 | total_timesteps 5075.
Path 255 | total_timesteps 5098.
Path 256 | total_timesteps 5118.
Path 257 | total_timesteps 5147.
Path 258 | total_timesteps 5174.
Path 259 | total_timesteps 5191.
Path 260 | total_timesteps 5203.
Path 261 | total_timesteps 5222.
Path 262 | total_timesteps 5249.
Path 263 | total_timesteps 5269.
Path 264 | total_timesteps 5298.
Path 265 | total_timesteps 5320.
Path 266 | total_timesteps 5336.
Path 267 | total_timesteps 5351.
Path 268 | total_timesteps 5370.
Path 269 | total_timesteps 5384.
Path 270 | total_timesteps 5423.
Path 271 | total_timesteps 5447.
Path 272 | total_timesteps 5462.
Path 273 | total_timesteps 5474.
Path 274 | total_timesteps 5487.
Path 275 | total_timesteps 5516.
Path 276 | total_timesteps 5532.
Path 277 | total_timesteps 5550.
Path 278 | total_timesteps 5569.
Path 279 | total_timesteps 5585.
Path 280 | total_timesteps 5612.
Path 281 | total_timesteps 5624.
Path 282 | total_timesteps 5655.
Path 283 | total_timesteps 5673.
Path 284 | total_timesteps 5696.
Path 285 | total_timesteps 5718.
Path 286 | total_timesteps 5733.
Path 287 | total_timesteps 5753.
Path 288 | total_timesteps 5769.
Path 289 | total_timesteps 5789.
Path 290 | total_timesteps 5806.
Path 291 | total_timesteps 5837.
Path 292 | total_timesteps 5853.
Path 293 | total_timesteps 5863.
Path 294 | total_timesteps 5894.
Path 295 | total_timesteps 5903.
Path 296 | total_timesteps 5918.
Path 297 | total_timesteps 5935.
Path 298 | total_timesteps 5970.
Path 299 | total_timesteps 5992.
Done generating random rollouts.
Creating normalization for training data.
Done creating normalization for training data.
Train dynamics model with intrinsic reward only? False
Pre-training enabled. Using only intrinsic reward.
Pre-training dynamics model for 0 iterations...
Done pre-training dynamics model.
Using external reward only.
itr #0 | 
Fitting dynamics.
Validation loss = 0.4158517122268677
Validation loss = 0.14192509651184082
Validation loss = 0.10765025019645691
Validation loss = 0.0935194119811058
Validation loss = 0.08362209796905518
Validation loss = 0.07827040553092957
Validation loss = 0.07416126877069473
Validation loss = 0.07499925792217255
Validation loss = 0.06609215587377548
Validation loss = 0.06231323629617691
Validation loss = 0.061581023037433624
Validation loss = 0.05697290599346161
Validation loss = 0.11253983527421951
Validation loss = 0.058346085250377655
Validation loss = 0.053093429654836655
Validation loss = 0.05283523350954056
Validation loss = 0.06012337654829025
Validation loss = 0.05199766904115677
Validation loss = 0.057655591517686844
Validation loss = 0.05185350775718689
Validation loss = 0.05357328802347183
Validation loss = 0.07080231606960297
Validation loss = 0.053374722599983215
Validation loss = 0.049133289605379105
Validation loss = 0.05246449634432793
Validation loss = 0.049794457852840424
Validation loss = 0.049509502947330475
Validation loss = 0.05183771252632141
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 28.
Path 2 | total_timesteps 47.
Path 3 | total_timesteps 62.
Path 4 | total_timesteps 80.
Path 5 | total_timesteps 92.
Path 6 | total_timesteps 104.
Path 7 | total_timesteps 124.
Path 8 | total_timesteps 148.
Path 9 | total_timesteps 168.
Path 10 | total_timesteps 180.
Path 11 | total_timesteps 204.
Path 12 | total_timesteps 243.
Path 13 | total_timesteps 266.
Path 14 | total_timesteps 281.
Path 15 | total_timesteps 293.
Path 16 | total_timesteps 309.
Path 17 | total_timesteps 323.
Path 18 | total_timesteps 351.
Path 19 | total_timesteps 366.
Path 20 | total_timesteps 378.
Path 21 | total_timesteps 395.
Path 22 | total_timesteps 410.
Path 23 | total_timesteps 432.
Path 24 | total_timesteps 440.
Path 25 | total_timesteps 460.
Path 26 | total_timesteps 489.
Path 27 | total_timesteps 504.
Path 28 | total_timesteps 548.
Path 29 | total_timesteps 573.
Path 30 | total_timesteps 589.
Path 31 | total_timesteps 609.
Path 32 | total_timesteps 626.
Path 33 | total_timesteps 648.
Path 34 | total_timesteps 666.
Path 35 | total_timesteps 679.
Path 36 | total_timesteps 687.
Path 37 | total_timesteps 700.
Path 38 | total_timesteps 715.
Path 39 | total_timesteps 729.
Path 40 | total_timesteps 763.
Path 41 | total_timesteps 792.
Path 42 | total_timesteps 813.
Path 43 | total_timesteps 831.
Path 44 | total_timesteps 860.
Path 45 | total_timesteps 887.
Path 46 | total_timesteps 900.
Path 47 | total_timesteps 916.
Path 48 | total_timesteps 927.
Path 49 | total_timesteps 949.
Path 50 | total_timesteps 964.
Path 51 | total_timesteps 972.
Path 52 | total_timesteps 1008.
Path 53 | total_timesteps 1021.
Path 54 | total_timesteps 1034.
Path 55 | total_timesteps 1053.
Path 56 | total_timesteps 1065.
Path 57 | total_timesteps 1091.
Path 58 | total_timesteps 1107.
Path 59 | total_timesteps 1120.
Path 60 | total_timesteps 1137.
Path 61 | total_timesteps 1159.
Path 62 | total_timesteps 1172.
Path 63 | total_timesteps 1246.
Path 64 | total_timesteps 1266.
Path 65 | total_timesteps 1279.
Path 66 | total_timesteps 1294.
Path 67 | total_timesteps 1315.
Path 68 | total_timesteps 1335.
Path 69 | total_timesteps 1363.
Path 70 | total_timesteps 1376.
Path 71 | total_timesteps 1406.
Path 72 | total_timesteps 1416.
Path 73 | total_timesteps 1431.
Path 74 | total_timesteps 1441.
Path 75 | total_timesteps 1473.
Path 76 | total_timesteps 1494.
Path 77 | total_timesteps 1524.
Path 78 | total_timesteps 1535.
Path 79 | total_timesteps 1556.
Path 80 | total_timesteps 1582.
Path 81 | total_timesteps 1596.
Path 82 | total_timesteps 1608.
Path 83 | total_timesteps 1618.
Path 84 | total_timesteps 1629.
Path 85 | total_timesteps 1647.
Path 86 | total_timesteps 1692.
Path 87 | total_timesteps 1717.
Path 88 | total_timesteps 1777.
Path 89 | total_timesteps 1793.
Path 90 | total_timesteps 1803.
Path 91 | total_timesteps 1834.
Path 92 | total_timesteps 1849.
Path 93 | total_timesteps 1869.
Path 94 | total_timesteps 1886.
Path 95 | total_timesteps 1915.
Path 96 | total_timesteps 1926.
Path 97 | total_timesteps 1938.
Path 98 | total_timesteps 1954.
Path 99 | total_timesteps 1976.
Path 100 | total_timesteps 1996.
Path 101 | total_timesteps 2039.
Path 102 | total_timesteps 2054.
Path 103 | total_timesteps 2064.
Path 104 | total_timesteps 2081.
Path 105 | total_timesteps 2096.
Path 106 | total_timesteps 2113.
Path 107 | total_timesteps 2139.
Path 108 | total_timesteps 2155.
Path 109 | total_timesteps 2169.
Path 110 | total_timesteps 2196.
Path 111 | total_timesteps 2214.
Path 112 | total_timesteps 2246.
Path 113 | total_timesteps 2263.
Path 114 | total_timesteps 2279.
Path 115 | total_timesteps 2302.
Path 116 | total_timesteps 2358.
Path 117 | total_timesteps 2371.
Path 118 | total_timesteps 2384.
Path 119 | total_timesteps 2399.
Path 120 | total_timesteps 2425.
Path 121 | total_timesteps 2440.
Path 122 | total_timesteps 2452.
Path 123 | total_timesteps 2462.
Path 124 | total_timesteps 2478.
Path 125 | total_timesteps 2503.
Path 126 | total_timesteps 2527.
Path 127 | total_timesteps 2543.
Path 128 | total_timesteps 2553.
Path 129 | total_timesteps 2565.
Path 130 | total_timesteps 2584.
Path 131 | total_timesteps 2607.
Path 132 | total_timesteps 2638.
Path 133 | total_timesteps 2655.
Path 134 | total_timesteps 2673.
Path 135 | total_timesteps 2694.
Path 136 | total_timesteps 2716.
Path 137 | total_timesteps 2731.
Path 138 | total_timesteps 2753.
Path 139 | total_timesteps 2784.
Path 140 | total_timesteps 2800.
Path 141 | total_timesteps 2810.
Path 142 | total_timesteps 2834.
Path 143 | total_timesteps 2847.
Path 144 | total_timesteps 2863.
Path 145 | total_timesteps 2882.
Path 146 | total_timesteps 2893.
Path 147 | total_timesteps 2912.
Path 148 | total_timesteps 2928.
Path 149 | total_timesteps 2942.
Path 150 | total_timesteps 2953.
Path 151 | total_timesteps 2973.
Path 152 | total_timesteps 2988.
Path 153 | total_timesteps 3002.
Path 154 | total_timesteps 3034.
Path 155 | total_timesteps 3043.
Path 156 | total_timesteps 3057.
Path 157 | total_timesteps 3066.
Path 158 | total_timesteps 3082.
Path 159 | total_timesteps 3099.
Path 160 | total_timesteps 3128.
Path 161 | total_timesteps 3142.
Path 162 | total_timesteps 3151.
Path 163 | total_timesteps 3173.
Path 164 | total_timesteps 3201.
Path 165 | total_timesteps 3229.
Path 166 | total_timesteps 3261.
Path 167 | total_timesteps 3293.
Path 168 | total_timesteps 3314.
Path 169 | total_timesteps 3333.
Path 170 | total_timesteps 3345.
Path 171 | total_timesteps 3366.
Path 172 | total_timesteps 3381.
Path 173 | total_timesteps 3396.
Path 174 | total_timesteps 3412.
Path 175 | total_timesteps 3431.
Path 176 | total_timesteps 3445.
Path 177 | total_timesteps 3465.
Path 178 | total_timesteps 3485.
Path 179 | total_timesteps 3516.
Path 180 | total_timesteps 3536.
Path 181 | total_timesteps 3550.
Path 182 | total_timesteps 3568.
Path 183 | total_timesteps 3587.
Path 184 | total_timesteps 3617.
Path 185 | total_timesteps 3628.
Path 186 | total_timesteps 3636.
Path 187 | total_timesteps 3705.
Path 188 | total_timesteps 3728.
Path 189 | total_timesteps 3747.
Path 190 | total_timesteps 3762.
Path 191 | total_timesteps 3778.
Path 192 | total_timesteps 3800.
Path 193 | total_timesteps 3814.
Path 194 | total_timesteps 3836.
Path 195 | total_timesteps 3857.
Path 196 | total_timesteps 3886.
Path 197 | total_timesteps 3901.
Path 198 | total_timesteps 3913.
Path 199 | total_timesteps 3935.
Path 200 | total_timesteps 3950.
Path 201 | total_timesteps 3963.
Path 202 | total_timesteps 3982.
Path 203 | total_timesteps 3992.
Path 204 | total_timesteps 4013.
Path 205 | total_timesteps 4024.
Path 206 | total_timesteps 4042.
Path 207 | total_timesteps 4053.
Path 208 | total_timesteps 4065.
Path 209 | total_timesteps 4082.
Path 210 | total_timesteps 4096.
Path 211 | total_timesteps 4117.
Path 212 | total_timesteps 4140.
Path 213 | total_timesteps 4149.
Path 214 | total_timesteps 4159.
Path 215 | total_timesteps 4185.
Path 216 | total_timesteps 4208.
Path 217 | total_timesteps 4227.
Path 218 | total_timesteps 4271.
Path 219 | total_timesteps 4317.
Path 220 | total_timesteps 4329.
Path 221 | total_timesteps 4340.
Path 222 | total_timesteps 4349.
Path 223 | total_timesteps 4367.
Path 224 | total_timesteps 4385.
Path 225 | total_timesteps 4395.
Path 226 | total_timesteps 4419.
Path 227 | total_timesteps 4441.
Path 228 | total_timesteps 4452.
Path 229 | total_timesteps 4466.
Path 230 | total_timesteps 4484.
Path 231 | total_timesteps 4508.
Path 232 | total_timesteps 4517.
Path 233 | total_timesteps 4538.
Path 234 | total_timesteps 4561.
Path 235 | total_timesteps 4574.
Path 236 | total_timesteps 4593.
Path 237 | total_timesteps 4619.
Path 238 | total_timesteps 4636.
Path 239 | total_timesteps 4665.
Path 240 | total_timesteps 4685.
Path 241 | total_timesteps 4695.
Path 242 | total_timesteps 4713.
Path 243 | total_timesteps 4736.
Path 244 | total_timesteps 4750.
Path 245 | total_timesteps 4768.
Path 246 | total_timesteps 4780.
Path 247 | total_timesteps 4800.
Path 248 | total_timesteps 4811.
Path 249 | total_timesteps 4831.
Path 250 | total_timesteps 4847.
Path 251 | total_timesteps 4857.
Path 252 | total_timesteps 4875.
Path 253 | total_timesteps 4887.
Path 254 | total_timesteps 4900.
Path 255 | total_timesteps 4921.
Path 256 | total_timesteps 4936.
Path 257 | total_timesteps 4957.
Path 258 | total_timesteps 4980.
Path 259 | total_timesteps 4993.
Path 260 | total_timesteps 5015.
Path 261 | total_timesteps 5028.
Path 262 | total_timesteps 5047.
Path 263 | total_timesteps 5067.
Path 264 | total_timesteps 5094.
Path 265 | total_timesteps 5114.
Path 266 | total_timesteps 5129.
Path 267 | total_timesteps 5145.
Path 268 | total_timesteps 5157.
Path 269 | total_timesteps 5174.
Path 270 | total_timesteps 5201.
Path 271 | total_timesteps 5220.
Path 272 | total_timesteps 5240.
Path 273 | total_timesteps 5263.
Path 274 | total_timesteps 5284.
Path 275 | total_timesteps 5304.
Path 276 | total_timesteps 5336.
Path 277 | total_timesteps 5369.
Path 278 | total_timesteps 5389.
Path 279 | total_timesteps 5407.
Path 280 | total_timesteps 5467.
Path 281 | total_timesteps 5504.
Path 282 | total_timesteps 5526.
Path 283 | total_timesteps 5554.
Path 284 | total_timesteps 5572.
Path 285 | total_timesteps 5580.
Path 286 | total_timesteps 5594.
Path 287 | total_timesteps 5609.
Path 288 | total_timesteps 5629.
Path 289 | total_timesteps 5644.
Path 290 | total_timesteps 5656.
Path 291 | total_timesteps 5684.
Path 292 | total_timesteps 5696.
Path 293 | total_timesteps 5711.
Path 294 | total_timesteps 5725.
Path 295 | total_timesteps 5747.
Path 296 | total_timesteps 5773.
Path 297 | total_timesteps 5788.
Path 298 | total_timesteps 5810.
Path 299 | total_timesteps 5826.
Path 300 | total_timesteps 5841.
Path 301 | total_timesteps 5854.
Path 302 | total_timesteps 5874.
Path 303 | total_timesteps 5890.
Path 304 | total_timesteps 5900.
Path 305 | total_timesteps 5918.
Path 306 | total_timesteps 5943.
Path 307 | total_timesteps 5977.
Path 308 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.17    |
| Iteration     | 0        |
| MaximumReturn | 29.5     |
| MinimumReturn | -27.2    |
| TotalSamples  | 8018     |
----------------------------
itr #1 | 
Fitting dynamics.
Validation loss = 0.08979242295026779
Validation loss = 0.06608938425779343
Validation loss = 0.05592447146773338
Validation loss = 0.046839021146297455
Validation loss = 0.04645882174372673
Validation loss = 0.04513467848300934
Validation loss = 0.04443805292248726
Validation loss = 0.041527971625328064
Validation loss = 0.0397094301879406
Validation loss = 0.04148627817630768
Validation loss = 0.04335983842611313
Validation loss = 0.04070892184972763
Validation loss = 0.04321153461933136
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 77.
Path 3 | total_timesteps 101.
Path 4 | total_timesteps 120.
Path 5 | total_timesteps 140.
Path 6 | total_timesteps 166.
Path 7 | total_timesteps 189.
Path 8 | total_timesteps 223.
Path 9 | total_timesteps 251.
Path 10 | total_timesteps 270.
Path 11 | total_timesteps 301.
Path 12 | total_timesteps 342.
Path 13 | total_timesteps 374.
Path 14 | total_timesteps 391.
Path 15 | total_timesteps 409.
Path 16 | total_timesteps 433.
Path 17 | total_timesteps 467.
Path 18 | total_timesteps 501.
Path 19 | total_timesteps 527.
Path 20 | total_timesteps 552.
Path 21 | total_timesteps 566.
Path 22 | total_timesteps 590.
Path 23 | total_timesteps 604.
Path 24 | total_timesteps 630.
Path 25 | total_timesteps 647.
Path 26 | total_timesteps 704.
Path 27 | total_timesteps 723.
Path 28 | total_timesteps 735.
Path 29 | total_timesteps 766.
Path 30 | total_timesteps 778.
Path 31 | total_timesteps 803.
Path 32 | total_timesteps 819.
Path 33 | total_timesteps 856.
Path 34 | total_timesteps 883.
Path 35 | total_timesteps 915.
Path 36 | total_timesteps 944.
Path 37 | total_timesteps 966.
Path 38 | total_timesteps 986.
Path 39 | total_timesteps 1010.
Path 40 | total_timesteps 1038.
Path 41 | total_timesteps 1064.
Path 42 | total_timesteps 1082.
Path 43 | total_timesteps 1111.
Path 44 | total_timesteps 1149.
Path 45 | total_timesteps 1164.
Path 46 | total_timesteps 1189.
Path 47 | total_timesteps 1228.
Path 48 | total_timesteps 1257.
Path 49 | total_timesteps 1277.
Path 50 | total_timesteps 1307.
Path 51 | total_timesteps 1326.
Path 52 | total_timesteps 1343.
Path 53 | total_timesteps 1369.
Path 54 | total_timesteps 1402.
Path 55 | total_timesteps 1419.
Path 56 | total_timesteps 1436.
Path 57 | total_timesteps 1468.
Path 58 | total_timesteps 1485.
Path 59 | total_timesteps 1509.
Path 60 | total_timesteps 1526.
Path 61 | total_timesteps 1536.
Path 62 | total_timesteps 1549.
Path 63 | total_timesteps 1584.
Path 64 | total_timesteps 1624.
Path 65 | total_timesteps 1652.
Path 66 | total_timesteps 1707.
Path 67 | total_timesteps 1734.
Path 68 | total_timesteps 1746.
Path 69 | total_timesteps 1770.
Path 70 | total_timesteps 1785.
Path 71 | total_timesteps 1801.
Path 72 | total_timesteps 1819.
Path 73 | total_timesteps 1830.
Path 74 | total_timesteps 1845.
Path 75 | total_timesteps 1897.
Path 76 | total_timesteps 1916.
Path 77 | total_timesteps 1942.
Path 78 | total_timesteps 1960.
Path 79 | total_timesteps 2004.
Path 80 | total_timesteps 2019.
Path 81 | total_timesteps 2058.
Path 82 | total_timesteps 2074.
Path 83 | total_timesteps 2111.
Path 84 | total_timesteps 2125.
Path 85 | total_timesteps 2146.
Path 86 | total_timesteps 2158.
Path 87 | total_timesteps 2177.
Path 88 | total_timesteps 2193.
Path 89 | total_timesteps 2211.
Path 90 | total_timesteps 2230.
Path 91 | total_timesteps 2246.
Path 92 | total_timesteps 2258.
Path 93 | total_timesteps 2286.
Path 94 | total_timesteps 2306.
Path 95 | total_timesteps 2331.
Path 96 | total_timesteps 2367.
Path 97 | total_timesteps 2389.
Path 98 | total_timesteps 2401.
Path 99 | total_timesteps 2419.
Path 100 | total_timesteps 2431.
Path 101 | total_timesteps 2457.
Path 102 | total_timesteps 2476.
Path 103 | total_timesteps 2494.
Path 104 | total_timesteps 2510.
Path 105 | total_timesteps 2526.
Path 106 | total_timesteps 2546.
Path 107 | total_timesteps 2560.
Path 108 | total_timesteps 2576.
Path 109 | total_timesteps 2597.
Path 110 | total_timesteps 2611.
Path 111 | total_timesteps 2626.
Path 112 | total_timesteps 2635.
Path 113 | total_timesteps 2657.
Path 114 | total_timesteps 2678.
Path 115 | total_timesteps 2704.
Path 116 | total_timesteps 2727.
Path 117 | total_timesteps 2754.
Path 118 | total_timesteps 2774.
Path 119 | total_timesteps 2808.
Path 120 | total_timesteps 2841.
Path 121 | total_timesteps 2867.
Path 122 | total_timesteps 2879.
Path 123 | total_timesteps 2901.
Path 124 | total_timesteps 2926.
Path 125 | total_timesteps 2953.
Path 126 | total_timesteps 2986.
Path 127 | total_timesteps 3006.
Path 128 | total_timesteps 3018.
Path 129 | total_timesteps 3039.
Path 130 | total_timesteps 3060.
Path 131 | total_timesteps 3095.
Path 132 | total_timesteps 3121.
Path 133 | total_timesteps 3139.
Path 134 | total_timesteps 3158.
Path 135 | total_timesteps 3186.
Path 136 | total_timesteps 3208.
Path 137 | total_timesteps 3237.
Path 138 | total_timesteps 3255.
Path 139 | total_timesteps 3279.
Path 140 | total_timesteps 3300.
Path 141 | total_timesteps 3314.
Path 142 | total_timesteps 3338.
Path 143 | total_timesteps 3362.
Path 144 | total_timesteps 3373.
Path 145 | total_timesteps 3399.
Path 146 | total_timesteps 3411.
Path 147 | total_timesteps 3436.
Path 148 | total_timesteps 3456.
Path 149 | total_timesteps 3478.
Path 150 | total_timesteps 3501.
Path 151 | total_timesteps 3520.
Path 152 | total_timesteps 3532.
Path 153 | total_timesteps 3551.
Path 154 | total_timesteps 3564.
Path 155 | total_timesteps 3586.
Path 156 | total_timesteps 3596.
Path 157 | total_timesteps 3622.
Path 158 | total_timesteps 3656.
Path 159 | total_timesteps 3678.
Path 160 | total_timesteps 3703.
Path 161 | total_timesteps 3724.
Path 162 | total_timesteps 3742.
Path 163 | total_timesteps 3761.
Path 164 | total_timesteps 3783.
Path 165 | total_timesteps 3793.
Path 166 | total_timesteps 3811.
Path 167 | total_timesteps 3824.
Path 168 | total_timesteps 3841.
Path 169 | total_timesteps 3902.
Path 170 | total_timesteps 3945.
Path 171 | total_timesteps 3989.
Path 172 | total_timesteps 4012.
Path 173 | total_timesteps 4033.
Path 174 | total_timesteps 4054.
Path 175 | total_timesteps 4086.
Path 176 | total_timesteps 4097.
Path 177 | total_timesteps 4110.
Path 178 | total_timesteps 4138.
Path 179 | total_timesteps 4165.
Path 180 | total_timesteps 4186.
Path 181 | total_timesteps 4212.
Path 182 | total_timesteps 4233.
Path 183 | total_timesteps 4259.
Path 184 | total_timesteps 4283.
Path 185 | total_timesteps 4294.
Path 186 | total_timesteps 4331.
Path 187 | total_timesteps 4351.
Path 188 | total_timesteps 4379.
Path 189 | total_timesteps 4397.
Path 190 | total_timesteps 4437.
Path 191 | total_timesteps 4454.
Path 192 | total_timesteps 4469.
Path 193 | total_timesteps 4494.
Path 194 | total_timesteps 4526.
Path 195 | total_timesteps 4547.
Path 196 | total_timesteps 4571.
Path 197 | total_timesteps 4588.
Path 198 | total_timesteps 4602.
Path 199 | total_timesteps 4614.
Path 200 | total_timesteps 4640.
Path 201 | total_timesteps 4652.
Path 202 | total_timesteps 4672.
Path 203 | total_timesteps 4705.
Path 204 | total_timesteps 4727.
Path 205 | total_timesteps 4752.
Path 206 | total_timesteps 4778.
Path 207 | total_timesteps 4805.
Path 208 | total_timesteps 4827.
Path 209 | total_timesteps 4853.
Path 210 | total_timesteps 4873.
Path 211 | total_timesteps 4892.
Path 212 | total_timesteps 4904.
Path 213 | total_timesteps 4924.
Path 214 | total_timesteps 4942.
Path 215 | total_timesteps 4971.
Path 216 | total_timesteps 4981.
Path 217 | total_timesteps 5002.
Path 218 | total_timesteps 5018.
Path 219 | total_timesteps 5038.
Path 220 | total_timesteps 5055.
Path 221 | total_timesteps 5070.
Path 222 | total_timesteps 5095.
Path 223 | total_timesteps 5104.
Path 224 | total_timesteps 5122.
Path 225 | total_timesteps 5132.
Path 226 | total_timesteps 5151.
Path 227 | total_timesteps 5182.
Path 228 | total_timesteps 5199.
Path 229 | total_timesteps 5234.
Path 230 | total_timesteps 5250.
Path 231 | total_timesteps 5272.
Path 232 | total_timesteps 5292.
Path 233 | total_timesteps 5303.
Path 234 | total_timesteps 5335.
Path 235 | total_timesteps 5351.
Path 236 | total_timesteps 5375.
Path 237 | total_timesteps 5396.
Path 238 | total_timesteps 5416.
Path 239 | total_timesteps 5448.
Path 240 | total_timesteps 5465.
Path 241 | total_timesteps 5483.
Path 242 | total_timesteps 5510.
Path 243 | total_timesteps 5526.
Path 244 | total_timesteps 5542.
Path 245 | total_timesteps 5555.
Path 246 | total_timesteps 5570.
Path 247 | total_timesteps 5606.
Path 248 | total_timesteps 5616.
Path 249 | total_timesteps 5636.
Path 250 | total_timesteps 5669.
Path 251 | total_timesteps 5691.
Path 252 | total_timesteps 5700.
Path 253 | total_timesteps 5727.
Path 254 | total_timesteps 5738.
Path 255 | total_timesteps 5757.
Path 256 | total_timesteps 5778.
Path 257 | total_timesteps 5804.
Path 258 | total_timesteps 5822.
Path 259 | total_timesteps 5861.
Path 260 | total_timesteps 5871.
Path 261 | total_timesteps 5893.
Path 262 | total_timesteps 5913.
Path 263 | total_timesteps 5938.
Path 264 | total_timesteps 5969.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.9     |
| Iteration     | 1        |
| MaximumReturn | 9.31     |
| MinimumReturn | -31      |
| TotalSamples  | 12018    |
----------------------------
itr #2 | 
Fitting dynamics.
Validation loss = 0.05831335857510567
Validation loss = 0.03788686916232109
Validation loss = 0.039112940430641174
Validation loss = 0.034231994301080704
Validation loss = 0.03369830176234245
Validation loss = 0.03541960194706917
Validation loss = 0.033930033445358276
Validation loss = 0.03591983765363693
Validation loss = 0.03345554694533348
Validation loss = 0.031837381422519684
Validation loss = 0.03231674060225487
Validation loss = 0.03262815251946449
Validation loss = 0.032016728073358536
Validation loss = 0.029920628294348717
Validation loss = 0.03437122330069542
Validation loss = 0.03106044791638851
Validation loss = 0.033455025404691696
Validation loss = 0.039800580590963364
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 28.
Path 2 | total_timesteps 44.
Path 3 | total_timesteps 70.
Path 4 | total_timesteps 100.
Path 5 | total_timesteps 112.
Path 6 | total_timesteps 122.
Path 7 | total_timesteps 135.
Path 8 | total_timesteps 152.
Path 9 | total_timesteps 165.
Path 10 | total_timesteps 190.
Path 11 | total_timesteps 201.
Path 12 | total_timesteps 224.
Path 13 | total_timesteps 240.
Path 14 | total_timesteps 262.
Path 15 | total_timesteps 283.
Path 16 | total_timesteps 307.
Path 17 | total_timesteps 329.
Path 18 | total_timesteps 392.
Path 19 | total_timesteps 415.
Path 20 | total_timesteps 442.
Path 21 | total_timesteps 462.
Path 22 | total_timesteps 477.
Path 23 | total_timesteps 504.
Path 24 | total_timesteps 526.
Path 25 | total_timesteps 561.
Path 26 | total_timesteps 581.
Path 27 | total_timesteps 597.
Path 28 | total_timesteps 614.
Path 29 | total_timesteps 648.
Path 30 | total_timesteps 663.
Path 31 | total_timesteps 691.
Path 32 | total_timesteps 721.
Path 33 | total_timesteps 732.
Path 34 | total_timesteps 754.
Path 35 | total_timesteps 771.
Path 36 | total_timesteps 784.
Path 37 | total_timesteps 798.
Path 38 | total_timesteps 814.
Path 39 | total_timesteps 827.
Path 40 | total_timesteps 851.
Path 41 | total_timesteps 863.
Path 42 | total_timesteps 895.
Path 43 | total_timesteps 905.
Path 44 | total_timesteps 926.
Path 45 | total_timesteps 938.
Path 46 | total_timesteps 966.
Path 47 | total_timesteps 1001.
Path 48 | total_timesteps 1023.
Path 49 | total_timesteps 1042.
Path 50 | total_timesteps 1058.
Path 51 | total_timesteps 1070.
Path 52 | total_timesteps 1102.
Path 53 | total_timesteps 1125.
Path 54 | total_timesteps 1141.
Path 55 | total_timesteps 1156.
Path 56 | total_timesteps 1180.
Path 57 | total_timesteps 1193.
Path 58 | total_timesteps 1211.
Path 59 | total_timesteps 1233.
Path 60 | total_timesteps 1263.
Path 61 | total_timesteps 1293.
Path 62 | total_timesteps 1317.
Path 63 | total_timesteps 1342.
Path 64 | total_timesteps 1366.
Path 65 | total_timesteps 1385.
Path 66 | total_timesteps 1397.
Path 67 | total_timesteps 1425.
Path 68 | total_timesteps 1445.
Path 69 | total_timesteps 1458.
Path 70 | total_timesteps 1487.
Path 71 | total_timesteps 1519.
Path 72 | total_timesteps 1535.
Path 73 | total_timesteps 1551.
Path 74 | total_timesteps 1571.
Path 75 | total_timesteps 1583.
Path 76 | total_timesteps 1600.
Path 77 | total_timesteps 1622.
Path 78 | total_timesteps 1641.
Path 79 | total_timesteps 1697.
Path 80 | total_timesteps 1724.
Path 81 | total_timesteps 1757.
Path 82 | total_timesteps 1770.
Path 83 | total_timesteps 1790.
Path 84 | total_timesteps 1834.
Path 85 | total_timesteps 1860.
Path 86 | total_timesteps 1867.
Path 87 | total_timesteps 1894.
Path 88 | total_timesteps 1913.
Path 89 | total_timesteps 1933.
Path 90 | total_timesteps 1954.
Path 91 | total_timesteps 1994.
Path 92 | total_timesteps 2019.
Path 93 | total_timesteps 2035.
Path 94 | total_timesteps 2060.
Path 95 | total_timesteps 2082.
Path 96 | total_timesteps 2095.
Path 97 | total_timesteps 2117.
Path 98 | total_timesteps 2129.
Path 99 | total_timesteps 2150.
Path 100 | total_timesteps 2173.
Path 101 | total_timesteps 2196.
Path 102 | total_timesteps 2215.
Path 103 | total_timesteps 2235.
Path 104 | total_timesteps 2252.
Path 105 | total_timesteps 2275.
Path 106 | total_timesteps 2294.
Path 107 | total_timesteps 2308.
Path 108 | total_timesteps 2327.
Path 109 | total_timesteps 2342.
Path 110 | total_timesteps 2365.
Path 111 | total_timesteps 2393.
Path 112 | total_timesteps 2429.
Path 113 | total_timesteps 2480.
Path 114 | total_timesteps 2494.
Path 115 | total_timesteps 2536.
Path 116 | total_timesteps 2558.
Path 117 | total_timesteps 2585.
Path 118 | total_timesteps 2611.
Path 119 | total_timesteps 2632.
Path 120 | total_timesteps 2654.
Path 121 | total_timesteps 2697.
Path 122 | total_timesteps 2720.
Path 123 | total_timesteps 2737.
Path 124 | total_timesteps 2766.
Path 125 | total_timesteps 2809.
Path 126 | total_timesteps 2835.
Path 127 | total_timesteps 2854.
Path 128 | total_timesteps 2885.
Path 129 | total_timesteps 2911.
Path 130 | total_timesteps 2928.
Path 131 | total_timesteps 2950.
Path 132 | total_timesteps 2982.
Path 133 | total_timesteps 2998.
Path 134 | total_timesteps 3022.
Path 135 | total_timesteps 3037.
Path 136 | total_timesteps 3054.
Path 137 | total_timesteps 3071.
Path 138 | total_timesteps 3089.
Path 139 | total_timesteps 3103.
Path 140 | total_timesteps 3120.
Path 141 | total_timesteps 3136.
Path 142 | total_timesteps 3148.
Path 143 | total_timesteps 3164.
Path 144 | total_timesteps 3174.
Path 145 | total_timesteps 3192.
Path 146 | total_timesteps 3206.
Path 147 | total_timesteps 3228.
Path 148 | total_timesteps 3240.
Path 149 | total_timesteps 3267.
Path 150 | total_timesteps 3291.
Path 151 | total_timesteps 3322.
Path 152 | total_timesteps 3334.
Path 153 | total_timesteps 3347.
Path 154 | total_timesteps 3381.
Path 155 | total_timesteps 3415.
Path 156 | total_timesteps 3444.
Path 157 | total_timesteps 3456.
Path 158 | total_timesteps 3477.
Path 159 | total_timesteps 3498.
Path 160 | total_timesteps 3527.
Path 161 | total_timesteps 3561.
Path 162 | total_timesteps 3570.
Path 163 | total_timesteps 3609.
Path 164 | total_timesteps 3633.
Path 165 | total_timesteps 3661.
Path 166 | total_timesteps 3682.
Path 167 | total_timesteps 3706.
Path 168 | total_timesteps 3748.
Path 169 | total_timesteps 3776.
Path 170 | total_timesteps 3808.
Path 171 | total_timesteps 3829.
Path 172 | total_timesteps 3837.
Path 173 | total_timesteps 3855.
Path 174 | total_timesteps 3878.
Path 175 | total_timesteps 3896.
Path 176 | total_timesteps 3917.
Path 177 | total_timesteps 3939.
Path 178 | total_timesteps 3948.
Path 179 | total_timesteps 3965.
Path 180 | total_timesteps 3983.
Path 181 | total_timesteps 4003.
Path 182 | total_timesteps 4025.
Path 183 | total_timesteps 4046.
Path 184 | total_timesteps 4063.
Path 185 | total_timesteps 4076.
Path 186 | total_timesteps 4099.
Path 187 | total_timesteps 4126.
Path 188 | total_timesteps 4148.
Path 189 | total_timesteps 4160.
Path 190 | total_timesteps 4170.
Path 191 | total_timesteps 4187.
Path 192 | total_timesteps 4207.
Path 193 | total_timesteps 4227.
Path 194 | total_timesteps 4257.
Path 195 | total_timesteps 4280.
Path 196 | total_timesteps 4291.
Path 197 | total_timesteps 4303.
Path 198 | total_timesteps 4313.
Path 199 | total_timesteps 4341.
Path 200 | total_timesteps 4358.
Path 201 | total_timesteps 4378.
Path 202 | total_timesteps 4396.
Path 203 | total_timesteps 4411.
Path 204 | total_timesteps 4431.
Path 205 | total_timesteps 4471.
Path 206 | total_timesteps 4492.
Path 207 | total_timesteps 4504.
Path 208 | total_timesteps 4524.
Path 209 | total_timesteps 4544.
Path 210 | total_timesteps 4563.
Path 211 | total_timesteps 4587.
Path 212 | total_timesteps 4604.
Path 213 | total_timesteps 4616.
Path 214 | total_timesteps 4635.
Path 215 | total_timesteps 4676.
Path 216 | total_timesteps 4705.
Path 217 | total_timesteps 4733.
Path 218 | total_timesteps 4752.
Path 219 | total_timesteps 4763.
Path 220 | total_timesteps 4771.
Path 221 | total_timesteps 4794.
Path 222 | total_timesteps 4809.
Path 223 | total_timesteps 4820.
Path 224 | total_timesteps 4841.
Path 225 | total_timesteps 4902.
Path 226 | total_timesteps 4927.
Path 227 | total_timesteps 4950.
Path 228 | total_timesteps 4977.
Path 229 | total_timesteps 5002.
Path 230 | total_timesteps 5026.
Path 231 | total_timesteps 5039.
Path 232 | total_timesteps 5070.
Path 233 | total_timesteps 5087.
Path 234 | total_timesteps 5114.
Path 235 | total_timesteps 5134.
Path 236 | total_timesteps 5149.
Path 237 | total_timesteps 5163.
Path 238 | total_timesteps 5180.
Path 239 | total_timesteps 5195.
Path 240 | total_timesteps 5217.
Path 241 | total_timesteps 5235.
Path 242 | total_timesteps 5254.
Path 243 | total_timesteps 5283.
Path 244 | total_timesteps 5305.
Path 245 | total_timesteps 5318.
Path 246 | total_timesteps 5328.
Path 247 | total_timesteps 5346.
Path 248 | total_timesteps 5359.
Path 249 | total_timesteps 5384.
Path 250 | total_timesteps 5397.
Path 251 | total_timesteps 5418.
Path 252 | total_timesteps 5437.
Path 253 | total_timesteps 5459.
Path 254 | total_timesteps 5468.
Path 255 | total_timesteps 5493.
Path 256 | total_timesteps 5533.
Path 257 | total_timesteps 5546.
Path 258 | total_timesteps 5600.
Path 259 | total_timesteps 5614.
Path 260 | total_timesteps 5633.
Path 261 | total_timesteps 5647.
Path 262 | total_timesteps 5679.
Path 263 | total_timesteps 5697.
Path 264 | total_timesteps 5717.
Path 265 | total_timesteps 5736.
Path 266 | total_timesteps 5758.
Path 267 | total_timesteps 5806.
Path 268 | total_timesteps 5844.
Path 269 | total_timesteps 5858.
Path 270 | total_timesteps 5879.
Path 271 | total_timesteps 5890.
Path 272 | total_timesteps 5915.
Path 273 | total_timesteps 5935.
Path 274 | total_timesteps 5961.
Path 275 | total_timesteps 5983.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.41    |
| Iteration     | 2        |
| MaximumReturn | 11.2     |
| MinimumReturn | -22.9    |
| TotalSamples  | 16021    |
----------------------------
itr #3 | 
Fitting dynamics.
Validation loss = 0.035467080771923065
Validation loss = 0.030846428126096725
Validation loss = 0.027250654995441437
Validation loss = 0.02857566997408867
Validation loss = 0.02684599719941616
Validation loss = 0.02567773312330246
Validation loss = 0.02466360107064247
Validation loss = 0.02628643438220024
Validation loss = 0.025084614753723145
Validation loss = 0.02579141966998577
Validation loss = 0.025643330067396164
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 21.
Path 2 | total_timesteps 59.
Path 3 | total_timesteps 77.
Path 4 | total_timesteps 101.
Path 5 | total_timesteps 120.
Path 6 | total_timesteps 132.
Path 7 | total_timesteps 144.
Path 8 | total_timesteps 161.
Path 9 | total_timesteps 183.
Path 10 | total_timesteps 215.
Path 11 | total_timesteps 239.
Path 12 | total_timesteps 270.
Path 13 | total_timesteps 299.
Path 14 | total_timesteps 329.
Path 15 | total_timesteps 348.
Path 16 | total_timesteps 375.
Path 17 | total_timesteps 391.
Path 18 | total_timesteps 425.
Path 19 | total_timesteps 450.
Path 20 | total_timesteps 478.
Path 21 | total_timesteps 497.
Path 22 | total_timesteps 525.
Path 23 | total_timesteps 548.
Path 24 | total_timesteps 564.
Path 25 | total_timesteps 595.
Path 26 | total_timesteps 612.
Path 27 | total_timesteps 633.
Path 28 | total_timesteps 646.
Path 29 | total_timesteps 659.
Path 30 | total_timesteps 686.
Path 31 | total_timesteps 712.
Path 32 | total_timesteps 729.
Path 33 | total_timesteps 765.
Path 34 | total_timesteps 783.
Path 35 | total_timesteps 807.
Path 36 | total_timesteps 827.
Path 37 | total_timesteps 842.
Path 38 | total_timesteps 873.
Path 39 | total_timesteps 891.
Path 40 | total_timesteps 939.
Path 41 | total_timesteps 997.
Path 42 | total_timesteps 1021.
Path 43 | total_timesteps 1033.
Path 44 | total_timesteps 1049.
Path 45 | total_timesteps 1071.
Path 46 | total_timesteps 1095.
Path 47 | total_timesteps 1116.
Path 48 | total_timesteps 1138.
Path 49 | total_timesteps 1168.
Path 50 | total_timesteps 1184.
Path 51 | total_timesteps 1202.
Path 52 | total_timesteps 1236.
Path 53 | total_timesteps 1259.
Path 54 | total_timesteps 1274.
Path 55 | total_timesteps 1286.
Path 56 | total_timesteps 1293.
Path 57 | total_timesteps 1315.
Path 58 | total_timesteps 1358.
Path 59 | total_timesteps 1381.
Path 60 | total_timesteps 1417.
Path 61 | total_timesteps 1447.
Path 62 | total_timesteps 1466.
Path 63 | total_timesteps 1480.
Path 64 | total_timesteps 1492.
Path 65 | total_timesteps 1510.
Path 66 | total_timesteps 1524.
Path 67 | total_timesteps 1552.
Path 68 | total_timesteps 1569.
Path 69 | total_timesteps 1580.
Path 70 | total_timesteps 1613.
Path 71 | total_timesteps 1632.
Path 72 | total_timesteps 1647.
Path 73 | total_timesteps 1662.
Path 74 | total_timesteps 1696.
Path 75 | total_timesteps 1716.
Path 76 | total_timesteps 1731.
Path 77 | total_timesteps 1752.
Path 78 | total_timesteps 1788.
Path 79 | total_timesteps 1814.
Path 80 | total_timesteps 1824.
Path 81 | total_timesteps 1840.
Path 82 | total_timesteps 1866.
Path 83 | total_timesteps 1884.
Path 84 | total_timesteps 1902.
Path 85 | total_timesteps 1931.
Path 86 | total_timesteps 1945.
Path 87 | total_timesteps 1966.
Path 88 | total_timesteps 1977.
Path 89 | total_timesteps 1993.
Path 90 | total_timesteps 2010.
Path 91 | total_timesteps 2023.
Path 92 | total_timesteps 2046.
Path 93 | total_timesteps 2060.
Path 94 | total_timesteps 2078.
Path 95 | total_timesteps 2097.
Path 96 | total_timesteps 2117.
Path 97 | total_timesteps 2147.
Path 98 | total_timesteps 2168.
Path 99 | total_timesteps 2191.
Path 100 | total_timesteps 2222.
Path 101 | total_timesteps 2243.
Path 102 | total_timesteps 2266.
Path 103 | total_timesteps 2284.
Path 104 | total_timesteps 2308.
Path 105 | total_timesteps 2327.
Path 106 | total_timesteps 2347.
Path 107 | total_timesteps 2370.
Path 108 | total_timesteps 2406.
Path 109 | total_timesteps 2415.
Path 110 | total_timesteps 2434.
Path 111 | total_timesteps 2458.
Path 112 | total_timesteps 2482.
Path 113 | total_timesteps 2511.
Path 114 | total_timesteps 2518.
Path 115 | total_timesteps 2531.
Path 116 | total_timesteps 2550.
Path 117 | total_timesteps 2565.
Path 118 | total_timesteps 2580.
Path 119 | total_timesteps 2589.
Path 120 | total_timesteps 2647.
Path 121 | total_timesteps 2683.
Path 122 | total_timesteps 2706.
Path 123 | total_timesteps 2737.
Path 124 | total_timesteps 2779.
Path 125 | total_timesteps 2788.
Path 126 | total_timesteps 2817.
Path 127 | total_timesteps 2837.
Path 128 | total_timesteps 2862.
Path 129 | total_timesteps 2883.
Path 130 | total_timesteps 2903.
Path 131 | total_timesteps 2931.
Path 132 | total_timesteps 2953.
Path 133 | total_timesteps 2962.
Path 134 | total_timesteps 2984.
Path 135 | total_timesteps 3007.
Path 136 | total_timesteps 3033.
Path 137 | total_timesteps 3060.
Path 138 | total_timesteps 3077.
Path 139 | total_timesteps 3111.
Path 140 | total_timesteps 3156.
Path 141 | total_timesteps 3175.
Path 142 | total_timesteps 3200.
Path 143 | total_timesteps 3234.
Path 144 | total_timesteps 3251.
Path 145 | total_timesteps 3268.
Path 146 | total_timesteps 3285.
Path 147 | total_timesteps 3302.
Path 148 | total_timesteps 3315.
Path 149 | total_timesteps 3329.
Path 150 | total_timesteps 3355.
Path 151 | total_timesteps 3364.
Path 152 | total_timesteps 3386.
Path 153 | total_timesteps 3417.
Path 154 | total_timesteps 3447.
Path 155 | total_timesteps 3467.
Path 156 | total_timesteps 3500.
Path 157 | total_timesteps 3547.
Path 158 | total_timesteps 3560.
Path 159 | total_timesteps 3578.
Path 160 | total_timesteps 3600.
Path 161 | total_timesteps 3612.
Path 162 | total_timesteps 3634.
Path 163 | total_timesteps 3649.
Path 164 | total_timesteps 3680.
Path 165 | total_timesteps 3708.
Path 166 | total_timesteps 3720.
Path 167 | total_timesteps 3751.
Path 168 | total_timesteps 3781.
Path 169 | total_timesteps 3802.
Path 170 | total_timesteps 3834.
Path 171 | total_timesteps 3855.
Path 172 | total_timesteps 3878.
Path 173 | total_timesteps 3909.
Path 174 | total_timesteps 3944.
Path 175 | total_timesteps 3959.
Path 176 | total_timesteps 3972.
Path 177 | total_timesteps 4000.
Path 178 | total_timesteps 4038.
Path 179 | total_timesteps 4049.
Path 180 | total_timesteps 4083.
Path 181 | total_timesteps 4109.
Path 182 | total_timesteps 4148.
Path 183 | total_timesteps 4181.
Path 184 | total_timesteps 4204.
Path 185 | total_timesteps 4247.
Path 186 | total_timesteps 4274.
Path 187 | total_timesteps 4315.
Path 188 | total_timesteps 4333.
Path 189 | total_timesteps 4353.
Path 190 | total_timesteps 4362.
Path 191 | total_timesteps 4397.
Path 192 | total_timesteps 4431.
Path 193 | total_timesteps 4447.
Path 194 | total_timesteps 4472.
Path 195 | total_timesteps 4497.
Path 196 | total_timesteps 4524.
Path 197 | total_timesteps 4552.
Path 198 | total_timesteps 4580.
Path 199 | total_timesteps 4607.
Path 200 | total_timesteps 4625.
Path 201 | total_timesteps 4635.
Path 202 | total_timesteps 4672.
Path 203 | total_timesteps 4705.
Path 204 | total_timesteps 4715.
Path 205 | total_timesteps 4741.
Path 206 | total_timesteps 4757.
Path 207 | total_timesteps 4783.
Path 208 | total_timesteps 4806.
Path 209 | total_timesteps 4841.
Path 210 | total_timesteps 4859.
Path 211 | total_timesteps 4876.
Path 212 | total_timesteps 4895.
Path 213 | total_timesteps 4920.
Path 214 | total_timesteps 4938.
Path 215 | total_timesteps 4952.
Path 216 | total_timesteps 4998.
Path 217 | total_timesteps 5057.
Path 218 | total_timesteps 5081.
Path 219 | total_timesteps 5127.
Path 220 | total_timesteps 5167.
Path 221 | total_timesteps 5180.
Path 222 | total_timesteps 5195.
Path 223 | total_timesteps 5216.
Path 224 | total_timesteps 5277.
Path 225 | total_timesteps 5312.
Path 226 | total_timesteps 5329.
Path 227 | total_timesteps 5361.
Path 228 | total_timesteps 5381.
Path 229 | total_timesteps 5410.
Path 230 | total_timesteps 5430.
Path 231 | total_timesteps 5442.
Path 232 | total_timesteps 5473.
Path 233 | total_timesteps 5505.
Path 234 | total_timesteps 5520.
Path 235 | total_timesteps 5549.
Path 236 | total_timesteps 5567.
Path 237 | total_timesteps 5594.
Path 238 | total_timesteps 5618.
Path 239 | total_timesteps 5636.
Path 240 | total_timesteps 5649.
Path 241 | total_timesteps 5676.
Path 242 | total_timesteps 5702.
Path 243 | total_timesteps 5718.
Path 244 | total_timesteps 5756.
Path 245 | total_timesteps 5769.
Path 246 | total_timesteps 5779.
Path 247 | total_timesteps 5801.
Path 248 | total_timesteps 5819.
Path 249 | total_timesteps 5840.
Path 250 | total_timesteps 5865.
Path 251 | total_timesteps 5896.
Path 252 | total_timesteps 5925.
Path 253 | total_timesteps 5952.
Path 254 | total_timesteps 5971.
Path 255 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.84    |
| Iteration     | 3        |
| MaximumReturn | 29.8     |
| MinimumReturn | -22.8    |
| TotalSamples  | 20043    |
----------------------------
itr #4 | 
Fitting dynamics.
Validation loss = 0.02811761200428009
Validation loss = 0.022930575534701347
Validation loss = 0.023200685158371925
Validation loss = 0.024780794978141785
Validation loss = 0.02322467789053917
Validation loss = 0.021937308833003044
Validation loss = 0.023688841611146927
Validation loss = 0.02289535664021969
Validation loss = 0.02194693312048912
Validation loss = 0.024007264524698257
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 39.
Path 3 | total_timesteps 59.
Path 4 | total_timesteps 83.
Path 5 | total_timesteps 101.
Path 6 | total_timesteps 127.
Path 7 | total_timesteps 140.
Path 8 | total_timesteps 158.
Path 9 | total_timesteps 183.
Path 10 | total_timesteps 198.
Path 11 | total_timesteps 217.
Path 12 | total_timesteps 235.
Path 13 | total_timesteps 261.
Path 14 | total_timesteps 301.
Path 15 | total_timesteps 315.
Path 16 | total_timesteps 334.
Path 17 | total_timesteps 347.
Path 18 | total_timesteps 365.
Path 19 | total_timesteps 388.
Path 20 | total_timesteps 405.
Path 21 | total_timesteps 435.
Path 22 | total_timesteps 454.
Path 23 | total_timesteps 469.
Path 24 | total_timesteps 502.
Path 25 | total_timesteps 523.
Path 26 | total_timesteps 546.
Path 27 | total_timesteps 562.
Path 28 | total_timesteps 587.
Path 29 | total_timesteps 612.
Path 30 | total_timesteps 641.
Path 31 | total_timesteps 693.
Path 32 | total_timesteps 712.
Path 33 | total_timesteps 737.
Path 34 | total_timesteps 750.
Path 35 | total_timesteps 767.
Path 36 | total_timesteps 798.
Path 37 | total_timesteps 831.
Path 38 | total_timesteps 842.
Path 39 | total_timesteps 874.
Path 40 | total_timesteps 891.
Path 41 | total_timesteps 913.
Path 42 | total_timesteps 937.
Path 43 | total_timesteps 950.
Path 44 | total_timesteps 985.
Path 45 | total_timesteps 1002.
Path 46 | total_timesteps 1017.
Path 47 | total_timesteps 1038.
Path 48 | total_timesteps 1059.
Path 49 | total_timesteps 1074.
Path 50 | total_timesteps 1094.
Path 51 | total_timesteps 1121.
Path 52 | total_timesteps 1134.
Path 53 | total_timesteps 1164.
Path 54 | total_timesteps 1193.
Path 55 | total_timesteps 1206.
Path 56 | total_timesteps 1233.
Path 57 | total_timesteps 1261.
Path 58 | total_timesteps 1285.
Path 59 | total_timesteps 1302.
Path 60 | total_timesteps 1324.
Path 61 | total_timesteps 1340.
Path 62 | total_timesteps 1359.
Path 63 | total_timesteps 1385.
Path 64 | total_timesteps 1413.
Path 65 | total_timesteps 1444.
Path 66 | total_timesteps 1482.
Path 67 | total_timesteps 1494.
Path 68 | total_timesteps 1521.
Path 69 | total_timesteps 1544.
Path 70 | total_timesteps 1556.
Path 71 | total_timesteps 1578.
Path 72 | total_timesteps 1604.
Path 73 | total_timesteps 1615.
Path 74 | total_timesteps 1648.
Path 75 | total_timesteps 1668.
Path 76 | total_timesteps 1685.
Path 77 | total_timesteps 1698.
Path 78 | total_timesteps 1730.
Path 79 | total_timesteps 1747.
Path 80 | total_timesteps 1775.
Path 81 | total_timesteps 1789.
Path 82 | total_timesteps 1805.
Path 83 | total_timesteps 1825.
Path 84 | total_timesteps 1837.
Path 85 | total_timesteps 1860.
Path 86 | total_timesteps 1899.
Path 87 | total_timesteps 1941.
Path 88 | total_timesteps 1966.
Path 89 | total_timesteps 1986.
Path 90 | total_timesteps 2000.
Path 91 | total_timesteps 2029.
Path 92 | total_timesteps 2069.
Path 93 | total_timesteps 2087.
Path 94 | total_timesteps 2100.
Path 95 | total_timesteps 2132.
Path 96 | total_timesteps 2151.
Path 97 | total_timesteps 2178.
Path 98 | total_timesteps 2199.
Path 99 | total_timesteps 2228.
Path 100 | total_timesteps 2258.
Path 101 | total_timesteps 2270.
Path 102 | total_timesteps 2282.
Path 103 | total_timesteps 2302.
Path 104 | total_timesteps 2325.
Path 105 | total_timesteps 2339.
Path 106 | total_timesteps 2370.
Path 107 | total_timesteps 2389.
Path 108 | total_timesteps 2414.
Path 109 | total_timesteps 2435.
Path 110 | total_timesteps 2447.
Path 111 | total_timesteps 2459.
Path 112 | total_timesteps 2480.
Path 113 | total_timesteps 2491.
Path 114 | total_timesteps 2535.
Path 115 | total_timesteps 2549.
Path 116 | total_timesteps 2567.
Path 117 | total_timesteps 2580.
Path 118 | total_timesteps 2609.
Path 119 | total_timesteps 2649.
Path 120 | total_timesteps 2676.
Path 121 | total_timesteps 2686.
Path 122 | total_timesteps 2702.
Path 123 | total_timesteps 2730.
Path 124 | total_timesteps 2748.
Path 125 | total_timesteps 2757.
Path 126 | total_timesteps 2797.
Path 127 | total_timesteps 2815.
Path 128 | total_timesteps 2839.
Path 129 | total_timesteps 2869.
Path 130 | total_timesteps 2886.
Path 131 | total_timesteps 2898.
Path 132 | total_timesteps 2914.
Path 133 | total_timesteps 2939.
Path 134 | total_timesteps 2955.
Path 135 | total_timesteps 2973.
Path 136 | total_timesteps 3014.
Path 137 | total_timesteps 3024.
Path 138 | total_timesteps 3042.
Path 139 | total_timesteps 3062.
Path 140 | total_timesteps 3081.
Path 141 | total_timesteps 3113.
Path 142 | total_timesteps 3137.
Path 143 | total_timesteps 3153.
Path 144 | total_timesteps 3182.
Path 145 | total_timesteps 3192.
Path 146 | total_timesteps 3218.
Path 147 | total_timesteps 3273.
Path 148 | total_timesteps 3289.
Path 149 | total_timesteps 3321.
Path 150 | total_timesteps 3330.
Path 151 | total_timesteps 3342.
Path 152 | total_timesteps 3360.
Path 153 | total_timesteps 3370.
Path 154 | total_timesteps 3396.
Path 155 | total_timesteps 3419.
Path 156 | total_timesteps 3443.
Path 157 | total_timesteps 3481.
Path 158 | total_timesteps 3501.
Path 159 | total_timesteps 3525.
Path 160 | total_timesteps 3555.
Path 161 | total_timesteps 3569.
Path 162 | total_timesteps 3586.
Path 163 | total_timesteps 3595.
Path 164 | total_timesteps 3619.
Path 165 | total_timesteps 3645.
Path 166 | total_timesteps 3660.
Path 167 | total_timesteps 3683.
Path 168 | total_timesteps 3702.
Path 169 | total_timesteps 3719.
Path 170 | total_timesteps 3745.
Path 171 | total_timesteps 3762.
Path 172 | total_timesteps 3781.
Path 173 | total_timesteps 3809.
Path 174 | total_timesteps 3832.
Path 175 | total_timesteps 3850.
Path 176 | total_timesteps 3878.
Path 177 | total_timesteps 3889.
Path 178 | total_timesteps 3917.
Path 179 | total_timesteps 3939.
Path 180 | total_timesteps 3970.
Path 181 | total_timesteps 3986.
Path 182 | total_timesteps 4021.
Path 183 | total_timesteps 4037.
Path 184 | total_timesteps 4054.
Path 185 | total_timesteps 4068.
Path 186 | total_timesteps 4089.
Path 187 | total_timesteps 4115.
Path 188 | total_timesteps 4137.
Path 189 | total_timesteps 4159.
Path 190 | total_timesteps 4191.
Path 191 | total_timesteps 4208.
Path 192 | total_timesteps 4235.
Path 193 | total_timesteps 4262.
Path 194 | total_timesteps 4283.
Path 195 | total_timesteps 4296.
Path 196 | total_timesteps 4323.
Path 197 | total_timesteps 4342.
Path 198 | total_timesteps 4367.
Path 199 | total_timesteps 4382.
Path 200 | total_timesteps 4404.
Path 201 | total_timesteps 4413.
Path 202 | total_timesteps 4439.
Path 203 | total_timesteps 4464.
Path 204 | total_timesteps 4484.
Path 205 | total_timesteps 4506.
Path 206 | total_timesteps 4535.
Path 207 | total_timesteps 4558.
Path 208 | total_timesteps 4591.
Path 209 | total_timesteps 4604.
Path 210 | total_timesteps 4634.
Path 211 | total_timesteps 4653.
Path 212 | total_timesteps 4675.
Path 213 | total_timesteps 4697.
Path 214 | total_timesteps 4732.
Path 215 | total_timesteps 4751.
Path 216 | total_timesteps 4785.
Path 217 | total_timesteps 4799.
Path 218 | total_timesteps 4816.
Path 219 | total_timesteps 4851.
Path 220 | total_timesteps 4870.
Path 221 | total_timesteps 4890.
Path 222 | total_timesteps 4905.
Path 223 | total_timesteps 4922.
Path 224 | total_timesteps 4942.
Path 225 | total_timesteps 4970.
Path 226 | total_timesteps 4986.
Path 227 | total_timesteps 5004.
Path 228 | total_timesteps 5026.
Path 229 | total_timesteps 5045.
Path 230 | total_timesteps 5057.
Path 231 | total_timesteps 5103.
Path 232 | total_timesteps 5136.
Path 233 | total_timesteps 5159.
Path 234 | total_timesteps 5183.
Path 235 | total_timesteps 5210.
Path 236 | total_timesteps 5229.
Path 237 | total_timesteps 5243.
Path 238 | total_timesteps 5266.
Path 239 | total_timesteps 5287.
Path 240 | total_timesteps 5325.
Path 241 | total_timesteps 5345.
Path 242 | total_timesteps 5374.
Path 243 | total_timesteps 5389.
Path 244 | total_timesteps 5410.
Path 245 | total_timesteps 5441.
Path 246 | total_timesteps 5469.
Path 247 | total_timesteps 5497.
Path 248 | total_timesteps 5531.
Path 249 | total_timesteps 5548.
Path 250 | total_timesteps 5575.
Path 251 | total_timesteps 5590.
Path 252 | total_timesteps 5611.
Path 253 | total_timesteps 5632.
Path 254 | total_timesteps 5647.
Path 255 | total_timesteps 5679.
Path 256 | total_timesteps 5699.
Path 257 | total_timesteps 5720.
Path 258 | total_timesteps 5749.
Path 259 | total_timesteps 5778.
Path 260 | total_timesteps 5798.
Path 261 | total_timesteps 5813.
Path 262 | total_timesteps 5836.
Path 263 | total_timesteps 5883.
Path 264 | total_timesteps 5920.
Path 265 | total_timesteps 5948.
Path 266 | total_timesteps 5967.
Path 267 | total_timesteps 5984.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.11    |
| Iteration     | 4        |
| MaximumReturn | 6.9      |
| MinimumReturn | -30      |
| TotalSamples  | 24051    |
----------------------------
itr #5 | 
Fitting dynamics.
Validation loss = 0.023115335032343864
Validation loss = 0.027484891936182976
Validation loss = 0.020138414576649666
Validation loss = 0.022069774568080902
Validation loss = 0.020149758085608482
Validation loss = 0.020290786400437355
Validation loss = 0.019250066950917244
Validation loss = 0.02116488665342331
Validation loss = 0.018586693331599236
Validation loss = 0.020436851307749748
Validation loss = 0.021845495328307152
Validation loss = 0.020277241244912148
Validation loss = 0.01993940956890583
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 51.
Path 3 | total_timesteps 68.
Path 4 | total_timesteps 87.
Path 5 | total_timesteps 109.
Path 6 | total_timesteps 129.
Path 7 | total_timesteps 144.
Path 8 | total_timesteps 157.
Path 9 | total_timesteps 171.
Path 10 | total_timesteps 193.
Path 11 | total_timesteps 204.
Path 12 | total_timesteps 214.
Path 13 | total_timesteps 230.
Path 14 | total_timesteps 246.
Path 15 | total_timesteps 264.
Path 16 | total_timesteps 292.
Path 17 | total_timesteps 306.
Path 18 | total_timesteps 319.
Path 19 | total_timesteps 332.
Path 20 | total_timesteps 354.
Path 21 | total_timesteps 364.
Path 22 | total_timesteps 383.
Path 23 | total_timesteps 398.
Path 24 | total_timesteps 409.
Path 25 | total_timesteps 423.
Path 26 | total_timesteps 446.
Path 27 | total_timesteps 473.
Path 28 | total_timesteps 489.
Path 29 | total_timesteps 534.
Path 30 | total_timesteps 553.
Path 31 | total_timesteps 564.
Path 32 | total_timesteps 582.
Path 33 | total_timesteps 600.
Path 34 | total_timesteps 616.
Path 35 | total_timesteps 645.
Path 36 | total_timesteps 661.
Path 37 | total_timesteps 679.
Path 38 | total_timesteps 691.
Path 39 | total_timesteps 710.
Path 40 | total_timesteps 736.
Path 41 | total_timesteps 751.
Path 42 | total_timesteps 771.
Path 43 | total_timesteps 789.
Path 44 | total_timesteps 805.
Path 45 | total_timesteps 819.
Path 46 | total_timesteps 858.
Path 47 | total_timesteps 871.
Path 48 | total_timesteps 887.
Path 49 | total_timesteps 896.
Path 50 | total_timesteps 913.
Path 51 | total_timesteps 951.
Path 52 | total_timesteps 969.
Path 53 | total_timesteps 985.
Path 54 | total_timesteps 1008.
Path 55 | total_timesteps 1020.
Path 56 | total_timesteps 1057.
Path 57 | total_timesteps 1072.
Path 58 | total_timesteps 1083.
Path 59 | total_timesteps 1101.
Path 60 | total_timesteps 1136.
Path 61 | total_timesteps 1175.
Path 62 | total_timesteps 1198.
Path 63 | total_timesteps 1219.
Path 64 | total_timesteps 1233.
Path 65 | total_timesteps 1251.
Path 66 | total_timesteps 1272.
Path 67 | total_timesteps 1283.
Path 68 | total_timesteps 1315.
Path 69 | total_timesteps 1327.
Path 70 | total_timesteps 1343.
Path 71 | total_timesteps 1357.
Path 72 | total_timesteps 1382.
Path 73 | total_timesteps 1396.
Path 74 | total_timesteps 1443.
Path 75 | total_timesteps 1467.
Path 76 | total_timesteps 1490.
Path 77 | total_timesteps 1509.
Path 78 | total_timesteps 1523.
Path 79 | total_timesteps 1545.
Path 80 | total_timesteps 1554.
Path 81 | total_timesteps 1588.
Path 82 | total_timesteps 1611.
Path 83 | total_timesteps 1622.
Path 84 | total_timesteps 1655.
Path 85 | total_timesteps 1668.
Path 86 | total_timesteps 1683.
Path 87 | total_timesteps 1730.
Path 88 | total_timesteps 1751.
Path 89 | total_timesteps 1762.
Path 90 | total_timesteps 1798.
Path 91 | total_timesteps 1815.
Path 92 | total_timesteps 1838.
Path 93 | total_timesteps 1874.
Path 94 | total_timesteps 1887.
Path 95 | total_timesteps 1915.
Path 96 | total_timesteps 1925.
Path 97 | total_timesteps 1942.
Path 98 | total_timesteps 1979.
Path 99 | total_timesteps 1995.
Path 100 | total_timesteps 2023.
Path 101 | total_timesteps 2037.
Path 102 | total_timesteps 2056.
Path 103 | total_timesteps 2079.
Path 104 | total_timesteps 2105.
Path 105 | total_timesteps 2129.
Path 106 | total_timesteps 2147.
Path 107 | total_timesteps 2170.
Path 108 | total_timesteps 2202.
Path 109 | total_timesteps 2225.
Path 110 | total_timesteps 2249.
Path 111 | total_timesteps 2282.
Path 112 | total_timesteps 2294.
Path 113 | total_timesteps 2309.
Path 114 | total_timesteps 2332.
Path 115 | total_timesteps 2352.
Path 116 | total_timesteps 2369.
Path 117 | total_timesteps 2379.
Path 118 | total_timesteps 2399.
Path 119 | total_timesteps 2425.
Path 120 | total_timesteps 2442.
Path 121 | total_timesteps 2457.
Path 122 | total_timesteps 2477.
Path 123 | total_timesteps 2499.
Path 124 | total_timesteps 2507.
Path 125 | total_timesteps 2525.
Path 126 | total_timesteps 2544.
Path 127 | total_timesteps 2566.
Path 128 | total_timesteps 2575.
Path 129 | total_timesteps 2590.
Path 130 | total_timesteps 2604.
Path 131 | total_timesteps 2622.
Path 132 | total_timesteps 2641.
Path 133 | total_timesteps 2658.
Path 134 | total_timesteps 2681.
Path 135 | total_timesteps 2698.
Path 136 | total_timesteps 2713.
Path 137 | total_timesteps 2722.
Path 138 | total_timesteps 2742.
Path 139 | total_timesteps 2811.
Path 140 | total_timesteps 2834.
Path 141 | total_timesteps 2851.
Path 142 | total_timesteps 2861.
Path 143 | total_timesteps 2869.
Path 144 | total_timesteps 2903.
Path 145 | total_timesteps 2933.
Path 146 | total_timesteps 2956.
Path 147 | total_timesteps 2981.
Path 148 | total_timesteps 3018.
Path 149 | total_timesteps 3036.
Path 150 | total_timesteps 3062.
Path 151 | total_timesteps 3092.
Path 152 | total_timesteps 3105.
Path 153 | total_timesteps 3127.
Path 154 | total_timesteps 3158.
Path 155 | total_timesteps 3170.
Path 156 | total_timesteps 3195.
Path 157 | total_timesteps 3207.
Path 158 | total_timesteps 3231.
Path 159 | total_timesteps 3261.
Path 160 | total_timesteps 3275.
Path 161 | total_timesteps 3295.
Path 162 | total_timesteps 3319.
Path 163 | total_timesteps 3332.
Path 164 | total_timesteps 3354.
Path 165 | total_timesteps 3385.
Path 166 | total_timesteps 3401.
Path 167 | total_timesteps 3415.
Path 168 | total_timesteps 3429.
Path 169 | total_timesteps 3444.
Path 170 | total_timesteps 3468.
Path 171 | total_timesteps 3482.
Path 172 | total_timesteps 3515.
Path 173 | total_timesteps 3538.
Path 174 | total_timesteps 3560.
Path 175 | total_timesteps 3617.
Path 176 | total_timesteps 3637.
Path 177 | total_timesteps 3648.
Path 178 | total_timesteps 3667.
Path 179 | total_timesteps 3693.
Path 180 | total_timesteps 3709.
Path 181 | total_timesteps 3731.
Path 182 | total_timesteps 3745.
Path 183 | total_timesteps 3768.
Path 184 | total_timesteps 3781.
Path 185 | total_timesteps 3799.
Path 186 | total_timesteps 3814.
Path 187 | total_timesteps 3829.
Path 188 | total_timesteps 3854.
Path 189 | total_timesteps 3866.
Path 190 | total_timesteps 3882.
Path 191 | total_timesteps 3891.
Path 192 | total_timesteps 3910.
Path 193 | total_timesteps 3926.
Path 194 | total_timesteps 3944.
Path 195 | total_timesteps 3954.
Path 196 | total_timesteps 3969.
Path 197 | total_timesteps 4002.
Path 198 | total_timesteps 4013.
Path 199 | total_timesteps 4025.
Path 200 | total_timesteps 4050.
Path 201 | total_timesteps 4090.
Path 202 | total_timesteps 4115.
Path 203 | total_timesteps 4131.
Path 204 | total_timesteps 4155.
Path 205 | total_timesteps 4179.
Path 206 | total_timesteps 4197.
Path 207 | total_timesteps 4214.
Path 208 | total_timesteps 4229.
Path 209 | total_timesteps 4258.
Path 210 | total_timesteps 4274.
Path 211 | total_timesteps 4302.
Path 212 | total_timesteps 4321.
Path 213 | total_timesteps 4340.
Path 214 | total_timesteps 4357.
Path 215 | total_timesteps 4375.
Path 216 | total_timesteps 4388.
Path 217 | total_timesteps 4409.
Path 218 | total_timesteps 4427.
Path 219 | total_timesteps 4452.
Path 220 | total_timesteps 4471.
Path 221 | total_timesteps 4493.
Path 222 | total_timesteps 4502.
Path 223 | total_timesteps 4513.
Path 224 | total_timesteps 4538.
Path 225 | total_timesteps 4557.
Path 226 | total_timesteps 4575.
Path 227 | total_timesteps 4609.
Path 228 | total_timesteps 4625.
Path 229 | total_timesteps 4639.
Path 230 | total_timesteps 4652.
Path 231 | total_timesteps 4665.
Path 232 | total_timesteps 4701.
Path 233 | total_timesteps 4730.
Path 234 | total_timesteps 4755.
Path 235 | total_timesteps 4773.
Path 236 | total_timesteps 4798.
Path 237 | total_timesteps 4811.
Path 238 | total_timesteps 4825.
Path 239 | total_timesteps 4851.
Path 240 | total_timesteps 4879.
Path 241 | total_timesteps 4890.
Path 242 | total_timesteps 4905.
Path 243 | total_timesteps 4917.
Path 244 | total_timesteps 4928.
Path 245 | total_timesteps 4958.
Path 246 | total_timesteps 4976.
Path 247 | total_timesteps 5003.
Path 248 | total_timesteps 5031.
Path 249 | total_timesteps 5045.
Path 250 | total_timesteps 5072.
Path 251 | total_timesteps 5081.
Path 252 | total_timesteps 5105.
Path 253 | total_timesteps 5130.
Path 254 | total_timesteps 5149.
Path 255 | total_timesteps 5163.
Path 256 | total_timesteps 5172.
Path 257 | total_timesteps 5188.
Path 258 | total_timesteps 5203.
Path 259 | total_timesteps 5215.
Path 260 | total_timesteps 5241.
Path 261 | total_timesteps 5260.
Path 262 | total_timesteps 5287.
Path 263 | total_timesteps 5316.
Path 264 | total_timesteps 5329.
Path 265 | total_timesteps 5348.
Path 266 | total_timesteps 5369.
Path 267 | total_timesteps 5389.
Path 268 | total_timesteps 5398.
Path 269 | total_timesteps 5411.
Path 270 | total_timesteps 5422.
Path 271 | total_timesteps 5436.
Path 272 | total_timesteps 5449.
Path 273 | total_timesteps 5470.
Path 274 | total_timesteps 5495.
Path 275 | total_timesteps 5520.
Path 276 | total_timesteps 5535.
Path 277 | total_timesteps 5553.
Path 278 | total_timesteps 5571.
Path 279 | total_timesteps 5586.
Path 280 | total_timesteps 5601.
Path 281 | total_timesteps 5620.
Path 282 | total_timesteps 5636.
Path 283 | total_timesteps 5659.
Path 284 | total_timesteps 5669.
Path 285 | total_timesteps 5681.
Path 286 | total_timesteps 5694.
Path 287 | total_timesteps 5710.
Path 288 | total_timesteps 5729.
Path 289 | total_timesteps 5740.
Path 290 | total_timesteps 5753.
Path 291 | total_timesteps 5768.
Path 292 | total_timesteps 5786.
Path 293 | total_timesteps 5812.
Path 294 | total_timesteps 5832.
Path 295 | total_timesteps 5859.
Path 296 | total_timesteps 5881.
Path 297 | total_timesteps 5893.
Path 298 | total_timesteps 5907.
Path 299 | total_timesteps 5918.
Path 300 | total_timesteps 5953.
Path 301 | total_timesteps 5966.
Path 302 | total_timesteps 5980.
Path 303 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.41    |
| Iteration     | 5        |
| MaximumReturn | 12.4     |
| MinimumReturn | -24.4    |
| TotalSamples  | 28059    |
----------------------------
itr #6 | 
Fitting dynamics.
Validation loss = 0.02123129554092884
Validation loss = 0.01820729672908783
Validation loss = 0.018436864018440247
Validation loss = 0.019377276301383972
Validation loss = 0.018004005774855614
Validation loss = 0.019777266308665276
Validation loss = 0.01980496570467949
Validation loss = 0.018428297713398933
Validation loss = 0.016902325674891472
Validation loss = 0.01823161169886589
Validation loss = 0.019689280539751053
Validation loss = 0.017908956855535507
Validation loss = 0.01812264882028103
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 29.
Path 3 | total_timesteps 52.
Path 4 | total_timesteps 75.
Path 5 | total_timesteps 103.
Path 6 | total_timesteps 131.
Path 7 | total_timesteps 140.
Path 8 | total_timesteps 158.
Path 9 | total_timesteps 178.
Path 10 | total_timesteps 193.
Path 11 | total_timesteps 206.
Path 12 | total_timesteps 221.
Path 13 | total_timesteps 237.
Path 14 | total_timesteps 255.
Path 15 | total_timesteps 275.
Path 16 | total_timesteps 291.
Path 17 | total_timesteps 306.
Path 18 | total_timesteps 332.
Path 19 | total_timesteps 350.
Path 20 | total_timesteps 369.
Path 21 | total_timesteps 391.
Path 22 | total_timesteps 406.
Path 23 | total_timesteps 417.
Path 24 | total_timesteps 442.
Path 25 | total_timesteps 462.
Path 26 | total_timesteps 486.
Path 27 | total_timesteps 501.
Path 28 | total_timesteps 512.
Path 29 | total_timesteps 522.
Path 30 | total_timesteps 536.
Path 31 | total_timesteps 568.
Path 32 | total_timesteps 582.
Path 33 | total_timesteps 592.
Path 34 | total_timesteps 635.
Path 35 | total_timesteps 644.
Path 36 | total_timesteps 661.
Path 37 | total_timesteps 671.
Path 38 | total_timesteps 687.
Path 39 | total_timesteps 717.
Path 40 | total_timesteps 726.
Path 41 | total_timesteps 749.
Path 42 | total_timesteps 759.
Path 43 | total_timesteps 782.
Path 44 | total_timesteps 796.
Path 45 | total_timesteps 812.
Path 46 | total_timesteps 830.
Path 47 | total_timesteps 845.
Path 48 | total_timesteps 864.
Path 49 | total_timesteps 887.
Path 50 | total_timesteps 906.
Path 51 | total_timesteps 923.
Path 52 | total_timesteps 933.
Path 53 | total_timesteps 952.
Path 54 | total_timesteps 966.
Path 55 | total_timesteps 981.
Path 56 | total_timesteps 999.
Path 57 | total_timesteps 1009.
Path 58 | total_timesteps 1028.
Path 59 | total_timesteps 1052.
Path 60 | total_timesteps 1069.
Path 61 | total_timesteps 1092.
Path 62 | total_timesteps 1142.
Path 63 | total_timesteps 1153.
Path 64 | total_timesteps 1179.
Path 65 | total_timesteps 1199.
Path 66 | total_timesteps 1222.
Path 67 | total_timesteps 1239.
Path 68 | total_timesteps 1247.
Path 69 | total_timesteps 1268.
Path 70 | total_timesteps 1291.
Path 71 | total_timesteps 1316.
Path 72 | total_timesteps 1329.
Path 73 | total_timesteps 1354.
Path 74 | total_timesteps 1369.
Path 75 | total_timesteps 1380.
Path 76 | total_timesteps 1401.
Path 77 | total_timesteps 1423.
Path 78 | total_timesteps 1446.
Path 79 | total_timesteps 1466.
Path 80 | total_timesteps 1477.
Path 81 | total_timesteps 1489.
Path 82 | total_timesteps 1507.
Path 83 | total_timesteps 1524.
Path 84 | total_timesteps 1544.
Path 85 | total_timesteps 1571.
Path 86 | total_timesteps 1589.
Path 87 | total_timesteps 1602.
Path 88 | total_timesteps 1615.
Path 89 | total_timesteps 1631.
Path 90 | total_timesteps 1648.
Path 91 | total_timesteps 1676.
Path 92 | total_timesteps 1694.
Path 93 | total_timesteps 1718.
Path 94 | total_timesteps 1749.
Path 95 | total_timesteps 1772.
Path 96 | total_timesteps 1793.
Path 97 | total_timesteps 1804.
Path 98 | total_timesteps 1816.
Path 99 | total_timesteps 1848.
Path 100 | total_timesteps 1861.
Path 101 | total_timesteps 1888.
Path 102 | total_timesteps 1911.
Path 103 | total_timesteps 1935.
Path 104 | total_timesteps 1966.
Path 105 | total_timesteps 1977.
Path 106 | total_timesteps 1989.
Path 107 | total_timesteps 2005.
Path 108 | total_timesteps 2034.
Path 109 | total_timesteps 2048.
Path 110 | total_timesteps 2072.
Path 111 | total_timesteps 2086.
Path 112 | total_timesteps 2107.
Path 113 | total_timesteps 2123.
Path 114 | total_timesteps 2134.
Path 115 | total_timesteps 2162.
Path 116 | total_timesteps 2179.
Path 117 | total_timesteps 2192.
Path 118 | total_timesteps 2207.
Path 119 | total_timesteps 2221.
Path 120 | total_timesteps 2237.
Path 121 | total_timesteps 2253.
Path 122 | total_timesteps 2268.
Path 123 | total_timesteps 2288.
Path 124 | total_timesteps 2304.
Path 125 | total_timesteps 2318.
Path 126 | total_timesteps 2327.
Path 127 | total_timesteps 2342.
Path 128 | total_timesteps 2360.
Path 129 | total_timesteps 2390.
Path 130 | total_timesteps 2401.
Path 131 | total_timesteps 2418.
Path 132 | total_timesteps 2440.
Path 133 | total_timesteps 2464.
Path 134 | total_timesteps 2489.
Path 135 | total_timesteps 2517.
Path 136 | total_timesteps 2542.
Path 137 | total_timesteps 2552.
Path 138 | total_timesteps 2575.
Path 139 | total_timesteps 2598.
Path 140 | total_timesteps 2623.
Path 141 | total_timesteps 2643.
Path 142 | total_timesteps 2666.
Path 143 | total_timesteps 2694.
Path 144 | total_timesteps 2711.
Path 145 | total_timesteps 2727.
Path 146 | total_timesteps 2737.
Path 147 | total_timesteps 2758.
Path 148 | total_timesteps 2769.
Path 149 | total_timesteps 2804.
Path 150 | total_timesteps 2825.
Path 151 | total_timesteps 2842.
Path 152 | total_timesteps 2858.
Path 153 | total_timesteps 2880.
Path 154 | total_timesteps 2894.
Path 155 | total_timesteps 2907.
Path 156 | total_timesteps 2920.
Path 157 | total_timesteps 2935.
Path 158 | total_timesteps 2958.
Path 159 | total_timesteps 2969.
Path 160 | total_timesteps 3006.
Path 161 | total_timesteps 3025.
Path 162 | total_timesteps 3037.
Path 163 | total_timesteps 3069.
Path 164 | total_timesteps 3091.
Path 165 | total_timesteps 3108.
Path 166 | total_timesteps 3123.
Path 167 | total_timesteps 3148.
Path 168 | total_timesteps 3158.
Path 169 | total_timesteps 3174.
Path 170 | total_timesteps 3191.
Path 171 | total_timesteps 3210.
Path 172 | total_timesteps 3220.
Path 173 | total_timesteps 3231.
Path 174 | total_timesteps 3247.
Path 175 | total_timesteps 3262.
Path 176 | total_timesteps 3281.
Path 177 | total_timesteps 3291.
Path 178 | total_timesteps 3302.
Path 179 | total_timesteps 3311.
Path 180 | total_timesteps 3328.
Path 181 | total_timesteps 3365.
Path 182 | total_timesteps 3375.
Path 183 | total_timesteps 3395.
Path 184 | total_timesteps 3405.
Path 185 | total_timesteps 3422.
Path 186 | total_timesteps 3437.
Path 187 | total_timesteps 3460.
Path 188 | total_timesteps 3475.
Path 189 | total_timesteps 3485.
Path 190 | total_timesteps 3508.
Path 191 | total_timesteps 3524.
Path 192 | total_timesteps 3543.
Path 193 | total_timesteps 3555.
Path 194 | total_timesteps 3576.
Path 195 | total_timesteps 3594.
Path 196 | total_timesteps 3616.
Path 197 | total_timesteps 3628.
Path 198 | total_timesteps 3654.
Path 199 | total_timesteps 3669.
Path 200 | total_timesteps 3682.
Path 201 | total_timesteps 3694.
Path 202 | total_timesteps 3721.
Path 203 | total_timesteps 3735.
Path 204 | total_timesteps 3765.
Path 205 | total_timesteps 3786.
Path 206 | total_timesteps 3797.
Path 207 | total_timesteps 3812.
Path 208 | total_timesteps 3827.
Path 209 | total_timesteps 3842.
Path 210 | total_timesteps 3864.
Path 211 | total_timesteps 3886.
Path 212 | total_timesteps 3898.
Path 213 | total_timesteps 3917.
Path 214 | total_timesteps 3944.
Path 215 | total_timesteps 3958.
Path 216 | total_timesteps 3982.
Path 217 | total_timesteps 4021.
Path 218 | total_timesteps 4041.
Path 219 | total_timesteps 4059.
Path 220 | total_timesteps 4079.
Path 221 | total_timesteps 4085.
Path 222 | total_timesteps 4099.
Path 223 | total_timesteps 4116.
Path 224 | total_timesteps 4131.
Path 225 | total_timesteps 4151.
Path 226 | total_timesteps 4166.
Path 227 | total_timesteps 4184.
Path 228 | total_timesteps 4199.
Path 229 | total_timesteps 4211.
Path 230 | total_timesteps 4226.
Path 231 | total_timesteps 4247.
Path 232 | total_timesteps 4263.
Path 233 | total_timesteps 4280.
Path 234 | total_timesteps 4298.
Path 235 | total_timesteps 4306.
Path 236 | total_timesteps 4318.
Path 237 | total_timesteps 4332.
Path 238 | total_timesteps 4347.
Path 239 | total_timesteps 4364.
Path 240 | total_timesteps 4390.
Path 241 | total_timesteps 4410.
Path 242 | total_timesteps 4425.
Path 243 | total_timesteps 4437.
Path 244 | total_timesteps 4462.
Path 245 | total_timesteps 4477.
Path 246 | total_timesteps 4493.
Path 247 | total_timesteps 4516.
Path 248 | total_timesteps 4536.
Path 249 | total_timesteps 4550.
Path 250 | total_timesteps 4565.
Path 251 | total_timesteps 4579.
Path 252 | total_timesteps 4602.
Path 253 | total_timesteps 4613.
Path 254 | total_timesteps 4637.
Path 255 | total_timesteps 4656.
Path 256 | total_timesteps 4676.
Path 257 | total_timesteps 4688.
Path 258 | total_timesteps 4706.
Path 259 | total_timesteps 4719.
Path 260 | total_timesteps 4732.
Path 261 | total_timesteps 4766.
Path 262 | total_timesteps 4774.
Path 263 | total_timesteps 4791.
Path 264 | total_timesteps 4804.
Path 265 | total_timesteps 4817.
Path 266 | total_timesteps 4839.
Path 267 | total_timesteps 4850.
Path 268 | total_timesteps 4867.
Path 269 | total_timesteps 4883.
Path 270 | total_timesteps 4894.
Path 271 | total_timesteps 4911.
Path 272 | total_timesteps 4927.
Path 273 | total_timesteps 4938.
Path 274 | total_timesteps 4951.
Path 275 | total_timesteps 4962.
Path 276 | total_timesteps 4971.
Path 277 | total_timesteps 4983.
Path 278 | total_timesteps 4995.
Path 279 | total_timesteps 5021.
Path 280 | total_timesteps 5037.
Path 281 | total_timesteps 5058.
Path 282 | total_timesteps 5073.
Path 283 | total_timesteps 5088.
Path 284 | total_timesteps 5115.
Path 285 | total_timesteps 5130.
Path 286 | total_timesteps 5147.
Path 287 | total_timesteps 5156.
Path 288 | total_timesteps 5169.
Path 289 | total_timesteps 5190.
Path 290 | total_timesteps 5199.
Path 291 | total_timesteps 5211.
Path 292 | total_timesteps 5224.
Path 293 | total_timesteps 5240.
Path 294 | total_timesteps 5266.
Path 295 | total_timesteps 5280.
Path 296 | total_timesteps 5293.
Path 297 | total_timesteps 5306.
Path 298 | total_timesteps 5323.
Path 299 | total_timesteps 5335.
Path 300 | total_timesteps 5361.
Path 301 | total_timesteps 5372.
Path 302 | total_timesteps 5383.
Path 303 | total_timesteps 5394.
Path 304 | total_timesteps 5407.
Path 305 | total_timesteps 5437.
Path 306 | total_timesteps 5459.
Path 307 | total_timesteps 5472.
Path 308 | total_timesteps 5485.
Path 309 | total_timesteps 5507.
Path 310 | total_timesteps 5526.
Path 311 | total_timesteps 5537.
Path 312 | total_timesteps 5563.
Path 313 | total_timesteps 5582.
Path 314 | total_timesteps 5611.
Path 315 | total_timesteps 5624.
Path 316 | total_timesteps 5638.
Path 317 | total_timesteps 5654.
Path 318 | total_timesteps 5672.
Path 319 | total_timesteps 5684.
Path 320 | total_timesteps 5701.
Path 321 | total_timesteps 5716.
Path 322 | total_timesteps 5727.
Path 323 | total_timesteps 5743.
Path 324 | total_timesteps 5761.
Path 325 | total_timesteps 5775.
Path 326 | total_timesteps 5792.
Path 327 | total_timesteps 5831.
Path 328 | total_timesteps 5849.
Path 329 | total_timesteps 5858.
Path 330 | total_timesteps 5881.
Path 331 | total_timesteps 5899.
Path 332 | total_timesteps 5913.
Path 333 | total_timesteps 5929.
Path 334 | total_timesteps 5940.
Path 335 | total_timesteps 5973.
Path 336 | total_timesteps 5986.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.63    |
| Iteration     | 6        |
| MaximumReturn | 9.1      |
| MinimumReturn | -21.3    |
| TotalSamples  | 32061    |
----------------------------
itr #7 | 
Fitting dynamics.
Validation loss = 0.018190471455454826
Validation loss = 0.01782240718603134
Validation loss = 0.017339853569865227
Validation loss = 0.016643155366182327
Validation loss = 0.016563257202506065
Validation loss = 0.017315059900283813
Validation loss = 0.01578289084136486
Validation loss = 0.016281185671687126
Validation loss = 0.01611173152923584
Validation loss = 0.016480570659041405
Validation loss = 0.015799060463905334
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 20.
Path 2 | total_timesteps 35.
Path 3 | total_timesteps 46.
Path 4 | total_timesteps 64.
Path 5 | total_timesteps 77.
Path 6 | total_timesteps 90.
Path 7 | total_timesteps 109.
Path 8 | total_timesteps 139.
Path 9 | total_timesteps 157.
Path 10 | total_timesteps 210.
Path 11 | total_timesteps 230.
Path 12 | total_timesteps 263.
Path 13 | total_timesteps 294.
Path 14 | total_timesteps 308.
Path 15 | total_timesteps 317.
Path 16 | total_timesteps 329.
Path 17 | total_timesteps 341.
Path 18 | total_timesteps 352.
Path 19 | total_timesteps 360.
Path 20 | total_timesteps 374.
Path 21 | total_timesteps 385.
Path 22 | total_timesteps 411.
Path 23 | total_timesteps 423.
Path 24 | total_timesteps 437.
Path 25 | total_timesteps 450.
Path 26 | total_timesteps 476.
Path 27 | total_timesteps 487.
Path 28 | total_timesteps 507.
Path 29 | total_timesteps 531.
Path 30 | total_timesteps 544.
Path 31 | total_timesteps 572.
Path 32 | total_timesteps 598.
Path 33 | total_timesteps 617.
Path 34 | total_timesteps 631.
Path 35 | total_timesteps 654.
Path 36 | total_timesteps 661.
Path 37 | total_timesteps 675.
Path 38 | total_timesteps 688.
Path 39 | total_timesteps 705.
Path 40 | total_timesteps 726.
Path 41 | total_timesteps 737.
Path 42 | total_timesteps 762.
Path 43 | total_timesteps 774.
Path 44 | total_timesteps 789.
Path 45 | total_timesteps 811.
Path 46 | total_timesteps 838.
Path 47 | total_timesteps 856.
Path 48 | total_timesteps 875.
Path 49 | total_timesteps 898.
Path 50 | total_timesteps 913.
Path 51 | total_timesteps 937.
Path 52 | total_timesteps 961.
Path 53 | total_timesteps 983.
Path 54 | total_timesteps 992.
Path 55 | total_timesteps 1011.
Path 56 | total_timesteps 1042.
Path 57 | total_timesteps 1063.
Path 58 | total_timesteps 1076.
Path 59 | total_timesteps 1108.
Path 60 | total_timesteps 1127.
Path 61 | total_timesteps 1141.
Path 62 | total_timesteps 1167.
Path 63 | total_timesteps 1186.
Path 64 | total_timesteps 1207.
Path 65 | total_timesteps 1219.
Path 66 | total_timesteps 1232.
Path 67 | total_timesteps 1251.
Path 68 | total_timesteps 1268.
Path 69 | total_timesteps 1287.
Path 70 | total_timesteps 1303.
Path 71 | total_timesteps 1327.
Path 72 | total_timesteps 1355.
Path 73 | total_timesteps 1368.
Path 74 | total_timesteps 1379.
Path 75 | total_timesteps 1393.
Path 76 | total_timesteps 1412.
Path 77 | total_timesteps 1424.
Path 78 | total_timesteps 1449.
Path 79 | total_timesteps 1474.
Path 80 | total_timesteps 1503.
Path 81 | total_timesteps 1513.
Path 82 | total_timesteps 1533.
Path 83 | total_timesteps 1575.
Path 84 | total_timesteps 1585.
Path 85 | total_timesteps 1602.
Path 86 | total_timesteps 1611.
Path 87 | total_timesteps 1632.
Path 88 | total_timesteps 1651.
Path 89 | total_timesteps 1664.
Path 90 | total_timesteps 1682.
Path 91 | total_timesteps 1699.
Path 92 | total_timesteps 1713.
Path 93 | total_timesteps 1725.
Path 94 | total_timesteps 1734.
Path 95 | total_timesteps 1742.
Path 96 | total_timesteps 1773.
Path 97 | total_timesteps 1783.
Path 98 | total_timesteps 1794.
Path 99 | total_timesteps 1821.
Path 100 | total_timesteps 1836.
Path 101 | total_timesteps 1860.
Path 102 | total_timesteps 1877.
Path 103 | total_timesteps 1899.
Path 104 | total_timesteps 1912.
Path 105 | total_timesteps 1924.
Path 106 | total_timesteps 1938.
Path 107 | total_timesteps 1955.
Path 108 | total_timesteps 1973.
Path 109 | total_timesteps 1989.
Path 110 | total_timesteps 2006.
Path 111 | total_timesteps 2029.
Path 112 | total_timesteps 2065.
Path 113 | total_timesteps 2096.
Path 114 | total_timesteps 2119.
Path 115 | total_timesteps 2132.
Path 116 | total_timesteps 2144.
Path 117 | total_timesteps 2162.
Path 118 | total_timesteps 2189.
Path 119 | total_timesteps 2201.
Path 120 | total_timesteps 2221.
Path 121 | total_timesteps 2236.
Path 122 | total_timesteps 2279.
Path 123 | total_timesteps 2294.
Path 124 | total_timesteps 2313.
Path 125 | total_timesteps 2336.
Path 126 | total_timesteps 2355.
Path 127 | total_timesteps 2371.
Path 128 | total_timesteps 2382.
Path 129 | total_timesteps 2403.
Path 130 | total_timesteps 2422.
Path 131 | total_timesteps 2435.
Path 132 | total_timesteps 2452.
Path 133 | total_timesteps 2467.
Path 134 | total_timesteps 2486.
Path 135 | total_timesteps 2497.
Path 136 | total_timesteps 2509.
Path 137 | total_timesteps 2521.
Path 138 | total_timesteps 2538.
Path 139 | total_timesteps 2568.
Path 140 | total_timesteps 2585.
Path 141 | total_timesteps 2602.
Path 142 | total_timesteps 2613.
Path 143 | total_timesteps 2628.
Path 144 | total_timesteps 2646.
Path 145 | total_timesteps 2658.
Path 146 | total_timesteps 2674.
Path 147 | total_timesteps 2689.
Path 148 | total_timesteps 2704.
Path 149 | total_timesteps 2735.
Path 150 | total_timesteps 2756.
Path 151 | total_timesteps 2776.
Path 152 | total_timesteps 2798.
Path 153 | total_timesteps 2819.
Path 154 | total_timesteps 2832.
Path 155 | total_timesteps 2855.
Path 156 | total_timesteps 2872.
Path 157 | total_timesteps 2887.
Path 158 | total_timesteps 2901.
Path 159 | total_timesteps 2919.
Path 160 | total_timesteps 2930.
Path 161 | total_timesteps 2951.
Path 162 | total_timesteps 2959.
Path 163 | total_timesteps 2970.
Path 164 | total_timesteps 2988.
Path 165 | total_timesteps 3005.
Path 166 | total_timesteps 3018.
Path 167 | total_timesteps 3029.
Path 168 | total_timesteps 3045.
Path 169 | total_timesteps 3060.
Path 170 | total_timesteps 3072.
Path 171 | total_timesteps 3097.
Path 172 | total_timesteps 3109.
Path 173 | total_timesteps 3127.
Path 174 | total_timesteps 3139.
Path 175 | total_timesteps 3150.
Path 176 | total_timesteps 3166.
Path 177 | total_timesteps 3193.
Path 178 | total_timesteps 3215.
Path 179 | total_timesteps 3226.
Path 180 | total_timesteps 3245.
Path 181 | total_timesteps 3260.
Path 182 | total_timesteps 3276.
Path 183 | total_timesteps 3295.
Path 184 | total_timesteps 3323.
Path 185 | total_timesteps 3336.
Path 186 | total_timesteps 3351.
Path 187 | total_timesteps 3367.
Path 188 | total_timesteps 3386.
Path 189 | total_timesteps 3403.
Path 190 | total_timesteps 3415.
Path 191 | total_timesteps 3426.
Path 192 | total_timesteps 3440.
Path 193 | total_timesteps 3456.
Path 194 | total_timesteps 3466.
Path 195 | total_timesteps 3484.
Path 196 | total_timesteps 3499.
Path 197 | total_timesteps 3531.
Path 198 | total_timesteps 3545.
Path 199 | total_timesteps 3563.
Path 200 | total_timesteps 3584.
Path 201 | total_timesteps 3613.
Path 202 | total_timesteps 3631.
Path 203 | total_timesteps 3648.
Path 204 | total_timesteps 3684.
Path 205 | total_timesteps 3707.
Path 206 | total_timesteps 3720.
Path 207 | total_timesteps 3733.
Path 208 | total_timesteps 3751.
Path 209 | total_timesteps 3771.
Path 210 | total_timesteps 3780.
Path 211 | total_timesteps 3793.
Path 212 | total_timesteps 3809.
Path 213 | total_timesteps 3820.
Path 214 | total_timesteps 3833.
Path 215 | total_timesteps 3846.
Path 216 | total_timesteps 3873.
Path 217 | total_timesteps 3888.
Path 218 | total_timesteps 3912.
Path 219 | total_timesteps 3935.
Path 220 | total_timesteps 3950.
Path 221 | total_timesteps 3970.
Path 222 | total_timesteps 3984.
Path 223 | total_timesteps 3996.
Path 224 | total_timesteps 4010.
Path 225 | total_timesteps 4027.
Path 226 | total_timesteps 4042.
Path 227 | total_timesteps 4049.
Path 228 | total_timesteps 4065.
Path 229 | total_timesteps 4079.
Path 230 | total_timesteps 4092.
Path 231 | total_timesteps 4107.
Path 232 | total_timesteps 4122.
Path 233 | total_timesteps 4132.
Path 234 | total_timesteps 4155.
Path 235 | total_timesteps 4172.
Path 236 | total_timesteps 4188.
Path 237 | total_timesteps 4210.
Path 238 | total_timesteps 4223.
Path 239 | total_timesteps 4233.
Path 240 | total_timesteps 4260.
Path 241 | total_timesteps 4273.
Path 242 | total_timesteps 4293.
Path 243 | total_timesteps 4308.
Path 244 | total_timesteps 4323.
Path 245 | total_timesteps 4341.
Path 246 | total_timesteps 4359.
Path 247 | total_timesteps 4376.
Path 248 | total_timesteps 4393.
Path 249 | total_timesteps 4404.
Path 250 | total_timesteps 4418.
Path 251 | total_timesteps 4432.
Path 252 | total_timesteps 4452.
Path 253 | total_timesteps 4475.
Path 254 | total_timesteps 4489.
Path 255 | total_timesteps 4500.
Path 256 | total_timesteps 4521.
Path 257 | total_timesteps 4540.
Path 258 | total_timesteps 4553.
Path 259 | total_timesteps 4582.
Path 260 | total_timesteps 4596.
Path 261 | total_timesteps 4609.
Path 262 | total_timesteps 4630.
Path 263 | total_timesteps 4655.
Path 264 | total_timesteps 4666.
Path 265 | total_timesteps 4677.
Path 266 | total_timesteps 4706.
Path 267 | total_timesteps 4719.
Path 268 | total_timesteps 4730.
Path 269 | total_timesteps 4741.
Path 270 | total_timesteps 4761.
Path 271 | total_timesteps 4774.
Path 272 | total_timesteps 4783.
Path 273 | total_timesteps 4799.
Path 274 | total_timesteps 4826.
Path 275 | total_timesteps 4850.
Path 276 | total_timesteps 4884.
Path 277 | total_timesteps 4909.
Path 278 | total_timesteps 4926.
Path 279 | total_timesteps 4940.
Path 280 | total_timesteps 4952.
Path 281 | total_timesteps 4961.
Path 282 | total_timesteps 4969.
Path 283 | total_timesteps 4987.
Path 284 | total_timesteps 5007.
Path 285 | total_timesteps 5022.
Path 286 | total_timesteps 5041.
Path 287 | total_timesteps 5069.
Path 288 | total_timesteps 5086.
Path 289 | total_timesteps 5111.
Path 290 | total_timesteps 5129.
Path 291 | total_timesteps 5142.
Path 292 | total_timesteps 5163.
Path 293 | total_timesteps 5173.
Path 294 | total_timesteps 5195.
Path 295 | total_timesteps 5216.
Path 296 | total_timesteps 5235.
Path 297 | total_timesteps 5270.
Path 298 | total_timesteps 5293.
Path 299 | total_timesteps 5316.
Path 300 | total_timesteps 5337.
Path 301 | total_timesteps 5356.
Path 302 | total_timesteps 5367.
Path 303 | total_timesteps 5388.
Path 304 | total_timesteps 5399.
Path 305 | total_timesteps 5455.
Path 306 | total_timesteps 5467.
Path 307 | total_timesteps 5491.
Path 308 | total_timesteps 5512.
Path 309 | total_timesteps 5533.
Path 310 | total_timesteps 5567.
Path 311 | total_timesteps 5584.
Path 312 | total_timesteps 5602.
Path 313 | total_timesteps 5630.
Path 314 | total_timesteps 5655.
Path 315 | total_timesteps 5673.
Path 316 | total_timesteps 5682.
Path 317 | total_timesteps 5701.
Path 318 | total_timesteps 5718.
Path 319 | total_timesteps 5735.
Path 320 | total_timesteps 5752.
Path 321 | total_timesteps 5776.
Path 322 | total_timesteps 5795.
Path 323 | total_timesteps 5821.
Path 324 | total_timesteps 5833.
Path 325 | total_timesteps 5856.
Path 326 | total_timesteps 5863.
Path 327 | total_timesteps 5878.
Path 328 | total_timesteps 5885.
Path 329 | total_timesteps 5900.
Path 330 | total_timesteps 5912.
Path 331 | total_timesteps 5924.
Path 332 | total_timesteps 5943.
Path 333 | total_timesteps 5968.
Path 334 | total_timesteps 5985.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.62    |
| Iteration     | 7        |
| MaximumReturn | 6.42     |
| MinimumReturn | -22.4    |
| TotalSamples  | 36065    |
----------------------------
itr #8 | 
Fitting dynamics.
Validation loss = 0.01692582480609417
Validation loss = 0.01620662771165371
Validation loss = 0.014350427314639091
Validation loss = 0.016099922358989716
Validation loss = 0.015056582167744637
Validation loss = 0.01647486351430416
Validation loss = 0.015327740460634232
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 47.
Path 4 | total_timesteps 70.
Path 5 | total_timesteps 87.
Path 6 | total_timesteps 96.
Path 7 | total_timesteps 118.
Path 8 | total_timesteps 131.
Path 9 | total_timesteps 147.
Path 10 | total_timesteps 154.
Path 11 | total_timesteps 174.
Path 12 | total_timesteps 188.
Path 13 | total_timesteps 210.
Path 14 | total_timesteps 230.
Path 15 | total_timesteps 248.
Path 16 | total_timesteps 272.
Path 17 | total_timesteps 290.
Path 18 | total_timesteps 303.
Path 19 | total_timesteps 316.
Path 20 | total_timesteps 339.
Path 21 | total_timesteps 352.
Path 22 | total_timesteps 367.
Path 23 | total_timesteps 382.
Path 24 | total_timesteps 396.
Path 25 | total_timesteps 413.
Path 26 | total_timesteps 439.
Path 27 | total_timesteps 454.
Path 28 | total_timesteps 472.
Path 29 | total_timesteps 490.
Path 30 | total_timesteps 505.
Path 31 | total_timesteps 517.
Path 32 | total_timesteps 528.
Path 33 | total_timesteps 547.
Path 34 | total_timesteps 565.
Path 35 | total_timesteps 587.
Path 36 | total_timesteps 602.
Path 37 | total_timesteps 611.
Path 38 | total_timesteps 628.
Path 39 | total_timesteps 641.
Path 40 | total_timesteps 657.
Path 41 | total_timesteps 677.
Path 42 | total_timesteps 713.
Path 43 | total_timesteps 725.
Path 44 | total_timesteps 752.
Path 45 | total_timesteps 769.
Path 46 | total_timesteps 782.
Path 47 | total_timesteps 803.
Path 48 | total_timesteps 822.
Path 49 | total_timesteps 842.
Path 50 | total_timesteps 855.
Path 51 | total_timesteps 878.
Path 52 | total_timesteps 901.
Path 53 | total_timesteps 919.
Path 54 | total_timesteps 937.
Path 55 | total_timesteps 948.
Path 56 | total_timesteps 970.
Path 57 | total_timesteps 1002.
Path 58 | total_timesteps 1047.
Path 59 | total_timesteps 1065.
Path 60 | total_timesteps 1079.
Path 61 | total_timesteps 1092.
Path 62 | total_timesteps 1107.
Path 63 | total_timesteps 1126.
Path 64 | total_timesteps 1140.
Path 65 | total_timesteps 1157.
Path 66 | total_timesteps 1168.
Path 67 | total_timesteps 1183.
Path 68 | total_timesteps 1193.
Path 69 | total_timesteps 1227.
Path 70 | total_timesteps 1241.
Path 71 | total_timesteps 1256.
Path 72 | total_timesteps 1281.
Path 73 | total_timesteps 1295.
Path 74 | total_timesteps 1304.
Path 75 | total_timesteps 1313.
Path 76 | total_timesteps 1321.
Path 77 | total_timesteps 1340.
Path 78 | total_timesteps 1353.
Path 79 | total_timesteps 1367.
Path 80 | total_timesteps 1387.
Path 81 | total_timesteps 1404.
Path 82 | total_timesteps 1419.
Path 83 | total_timesteps 1433.
Path 84 | total_timesteps 1482.
Path 85 | total_timesteps 1494.
Path 86 | total_timesteps 1506.
Path 87 | total_timesteps 1526.
Path 88 | total_timesteps 1541.
Path 89 | total_timesteps 1554.
Path 90 | total_timesteps 1571.
Path 91 | total_timesteps 1594.
Path 92 | total_timesteps 1614.
Path 93 | total_timesteps 1633.
Path 94 | total_timesteps 1642.
Path 95 | total_timesteps 1657.
Path 96 | total_timesteps 1669.
Path 97 | total_timesteps 1679.
Path 98 | total_timesteps 1691.
Path 99 | total_timesteps 1728.
Path 100 | total_timesteps 1767.
Path 101 | total_timesteps 1787.
Path 102 | total_timesteps 1796.
Path 103 | total_timesteps 1810.
Path 104 | total_timesteps 1825.
Path 105 | total_timesteps 1836.
Path 106 | total_timesteps 1849.
Path 107 | total_timesteps 1859.
Path 108 | total_timesteps 1874.
Path 109 | total_timesteps 1900.
Path 110 | total_timesteps 1927.
Path 111 | total_timesteps 1940.
Path 112 | total_timesteps 1965.
Path 113 | total_timesteps 1986.
Path 114 | total_timesteps 1996.
Path 115 | total_timesteps 2011.
Path 116 | total_timesteps 2025.
Path 117 | total_timesteps 2046.
Path 118 | total_timesteps 2056.
Path 119 | total_timesteps 2091.
Path 120 | total_timesteps 2116.
Path 121 | total_timesteps 2131.
Path 122 | total_timesteps 2146.
Path 123 | total_timesteps 2157.
Path 124 | total_timesteps 2180.
Path 125 | total_timesteps 2194.
Path 126 | total_timesteps 2208.
Path 127 | total_timesteps 2222.
Path 128 | total_timesteps 2235.
Path 129 | total_timesteps 2248.
Path 130 | total_timesteps 2261.
Path 131 | total_timesteps 2272.
Path 132 | total_timesteps 2288.
Path 133 | total_timesteps 2305.
Path 134 | total_timesteps 2353.
Path 135 | total_timesteps 2372.
Path 136 | total_timesteps 2390.
Path 137 | total_timesteps 2400.
Path 138 | total_timesteps 2414.
Path 139 | total_timesteps 2432.
Path 140 | total_timesteps 2456.
Path 141 | total_timesteps 2480.
Path 142 | total_timesteps 2493.
Path 143 | total_timesteps 2504.
Path 144 | total_timesteps 2524.
Path 145 | total_timesteps 2539.
Path 146 | total_timesteps 2557.
Path 147 | total_timesteps 2577.
Path 148 | total_timesteps 2595.
Path 149 | total_timesteps 2617.
Path 150 | total_timesteps 2631.
Path 151 | total_timesteps 2652.
Path 152 | total_timesteps 2667.
Path 153 | total_timesteps 2681.
Path 154 | total_timesteps 2720.
Path 155 | total_timesteps 2738.
Path 156 | total_timesteps 2758.
Path 157 | total_timesteps 2773.
Path 158 | total_timesteps 2783.
Path 159 | total_timesteps 2805.
Path 160 | total_timesteps 2819.
Path 161 | total_timesteps 2834.
Path 162 | total_timesteps 2853.
Path 163 | total_timesteps 2863.
Path 164 | total_timesteps 2881.
Path 165 | total_timesteps 2908.
Path 166 | total_timesteps 2937.
Path 167 | total_timesteps 2947.
Path 168 | total_timesteps 2960.
Path 169 | total_timesteps 2971.
Path 170 | total_timesteps 2980.
Path 171 | total_timesteps 3006.
Path 172 | total_timesteps 3030.
Path 173 | total_timesteps 3038.
Path 174 | total_timesteps 3059.
Path 175 | total_timesteps 3084.
Path 176 | total_timesteps 3107.
Path 177 | total_timesteps 3118.
Path 178 | total_timesteps 3132.
Path 179 | total_timesteps 3155.
Path 180 | total_timesteps 3180.
Path 181 | total_timesteps 3192.
Path 182 | total_timesteps 3207.
Path 183 | total_timesteps 3218.
Path 184 | total_timesteps 3237.
Path 185 | total_timesteps 3250.
Path 186 | total_timesteps 3271.
Path 187 | total_timesteps 3315.
Path 188 | total_timesteps 3323.
Path 189 | total_timesteps 3339.
Path 190 | total_timesteps 3354.
Path 191 | total_timesteps 3370.
Path 192 | total_timesteps 3383.
Path 193 | total_timesteps 3394.
Path 194 | total_timesteps 3410.
Path 195 | total_timesteps 3437.
Path 196 | total_timesteps 3447.
Path 197 | total_timesteps 3460.
Path 198 | total_timesteps 3472.
Path 199 | total_timesteps 3489.
Path 200 | total_timesteps 3510.
Path 201 | total_timesteps 3521.
Path 202 | total_timesteps 3534.
Path 203 | total_timesteps 3549.
Path 204 | total_timesteps 3565.
Path 205 | total_timesteps 3574.
Path 206 | total_timesteps 3599.
Path 207 | total_timesteps 3627.
Path 208 | total_timesteps 3645.
Path 209 | total_timesteps 3680.
Path 210 | total_timesteps 3700.
Path 211 | total_timesteps 3717.
Path 212 | total_timesteps 3733.
Path 213 | total_timesteps 3751.
Path 214 | total_timesteps 3763.
Path 215 | total_timesteps 3773.
Path 216 | total_timesteps 3797.
Path 217 | total_timesteps 3808.
Path 218 | total_timesteps 3822.
Path 219 | total_timesteps 3841.
Path 220 | total_timesteps 3864.
Path 221 | total_timesteps 3877.
Path 222 | total_timesteps 3897.
Path 223 | total_timesteps 3913.
Path 224 | total_timesteps 3923.
Path 225 | total_timesteps 3947.
Path 226 | total_timesteps 3967.
Path 227 | total_timesteps 3985.
Path 228 | total_timesteps 3998.
Path 229 | total_timesteps 4015.
Path 230 | total_timesteps 4042.
Path 231 | total_timesteps 4059.
Path 232 | total_timesteps 4073.
Path 233 | total_timesteps 4083.
Path 234 | total_timesteps 4104.
Path 235 | total_timesteps 4119.
Path 236 | total_timesteps 4130.
Path 237 | total_timesteps 4143.
Path 238 | total_timesteps 4158.
Path 239 | total_timesteps 4172.
Path 240 | total_timesteps 4184.
Path 241 | total_timesteps 4199.
Path 242 | total_timesteps 4214.
Path 243 | total_timesteps 4229.
Path 244 | total_timesteps 4251.
Path 245 | total_timesteps 4264.
Path 246 | total_timesteps 4279.
Path 247 | total_timesteps 4323.
Path 248 | total_timesteps 4337.
Path 249 | total_timesteps 4350.
Path 250 | total_timesteps 4363.
Path 251 | total_timesteps 4379.
Path 252 | total_timesteps 4399.
Path 253 | total_timesteps 4410.
Path 254 | total_timesteps 4425.
Path 255 | total_timesteps 4436.
Path 256 | total_timesteps 4450.
Path 257 | total_timesteps 4470.
Path 258 | total_timesteps 4487.
Path 259 | total_timesteps 4513.
Path 260 | total_timesteps 4531.
Path 261 | total_timesteps 4544.
Path 262 | total_timesteps 4556.
Path 263 | total_timesteps 4567.
Path 264 | total_timesteps 4592.
Path 265 | total_timesteps 4610.
Path 266 | total_timesteps 4633.
Path 267 | total_timesteps 4650.
Path 268 | total_timesteps 4665.
Path 269 | total_timesteps 4680.
Path 270 | total_timesteps 4702.
Path 271 | total_timesteps 4711.
Path 272 | total_timesteps 4725.
Path 273 | total_timesteps 4762.
Path 274 | total_timesteps 4773.
Path 275 | total_timesteps 4792.
Path 276 | total_timesteps 4816.
Path 277 | total_timesteps 4848.
Path 278 | total_timesteps 4867.
Path 279 | total_timesteps 4881.
Path 280 | total_timesteps 4892.
Path 281 | total_timesteps 4909.
Path 282 | total_timesteps 4931.
Path 283 | total_timesteps 4944.
Path 284 | total_timesteps 4954.
Path 285 | total_timesteps 4973.
Path 286 | total_timesteps 4986.
Path 287 | total_timesteps 5010.
Path 288 | total_timesteps 5019.
Path 289 | total_timesteps 5035.
Path 290 | total_timesteps 5049.
Path 291 | total_timesteps 5059.
Path 292 | total_timesteps 5073.
Path 293 | total_timesteps 5091.
Path 294 | total_timesteps 5107.
Path 295 | total_timesteps 5124.
Path 296 | total_timesteps 5148.
Path 297 | total_timesteps 5162.
Path 298 | total_timesteps 5186.
Path 299 | total_timesteps 5200.
Path 300 | total_timesteps 5211.
Path 301 | total_timesteps 5223.
Path 302 | total_timesteps 5245.
Path 303 | total_timesteps 5268.
Path 304 | total_timesteps 5292.
Path 305 | total_timesteps 5309.
Path 306 | total_timesteps 5320.
Path 307 | total_timesteps 5339.
Path 308 | total_timesteps 5360.
Path 309 | total_timesteps 5381.
Path 310 | total_timesteps 5396.
Path 311 | total_timesteps 5409.
Path 312 | total_timesteps 5427.
Path 313 | total_timesteps 5443.
Path 314 | total_timesteps 5468.
Path 315 | total_timesteps 5491.
Path 316 | total_timesteps 5514.
Path 317 | total_timesteps 5551.
Path 318 | total_timesteps 5574.
Path 319 | total_timesteps 5592.
Path 320 | total_timesteps 5605.
Path 321 | total_timesteps 5636.
Path 322 | total_timesteps 5651.
Path 323 | total_timesteps 5661.
Path 324 | total_timesteps 5671.
Path 325 | total_timesteps 5684.
Path 326 | total_timesteps 5701.
Path 327 | total_timesteps 5719.
Path 328 | total_timesteps 5735.
Path 329 | total_timesteps 5748.
Path 330 | total_timesteps 5777.
Path 331 | total_timesteps 5798.
Path 332 | total_timesteps 5813.
Path 333 | total_timesteps 5830.
Path 334 | total_timesteps 5840.
Path 335 | total_timesteps 5852.
Path 336 | total_timesteps 5862.
Path 337 | total_timesteps 5876.
Path 338 | total_timesteps 5895.
Path 339 | total_timesteps 5906.
Path 340 | total_timesteps 5928.
Path 341 | total_timesteps 5939.
Path 342 | total_timesteps 5975.
Path 343 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.35    |
| Iteration     | 8        |
| MaximumReturn | 7.88     |
| MinimumReturn | -27.8    |
| TotalSamples  | 40079    |
----------------------------
itr #9 | 
Fitting dynamics.
Validation loss = 0.015246326103806496
Validation loss = 0.013630407862365246
Validation loss = 0.014112783595919609
Validation loss = 0.01688113436102867
Validation loss = 0.01316586323082447
Validation loss = 0.013617211952805519
Validation loss = 0.013745422475039959
Validation loss = 0.013855245895683765
Validation loss = 0.013019388541579247
Validation loss = 0.013796387240290642
Validation loss = 0.016240760684013367
Validation loss = 0.014482079073786736
Validation loss = 0.013035234995186329
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 21.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 49.
Path 4 | total_timesteps 65.
Path 5 | total_timesteps 80.
Path 6 | total_timesteps 104.
Path 7 | total_timesteps 120.
Path 8 | total_timesteps 147.
Path 9 | total_timesteps 166.
Path 10 | total_timesteps 186.
Path 11 | total_timesteps 210.
Path 12 | total_timesteps 227.
Path 13 | total_timesteps 252.
Path 14 | total_timesteps 268.
Path 15 | total_timesteps 291.
Path 16 | total_timesteps 303.
Path 17 | total_timesteps 321.
Path 18 | total_timesteps 346.
Path 19 | total_timesteps 361.
Path 20 | total_timesteps 377.
Path 21 | total_timesteps 408.
Path 22 | total_timesteps 421.
Path 23 | total_timesteps 445.
Path 24 | total_timesteps 466.
Path 25 | total_timesteps 481.
Path 26 | total_timesteps 497.
Path 27 | total_timesteps 514.
Path 28 | total_timesteps 536.
Path 29 | total_timesteps 554.
Path 30 | total_timesteps 570.
Path 31 | total_timesteps 585.
Path 32 | total_timesteps 594.
Path 33 | total_timesteps 624.
Path 34 | total_timesteps 635.
Path 35 | total_timesteps 659.
Path 36 | total_timesteps 672.
Path 37 | total_timesteps 688.
Path 38 | total_timesteps 700.
Path 39 | total_timesteps 719.
Path 40 | total_timesteps 745.
Path 41 | total_timesteps 758.
Path 42 | total_timesteps 777.
Path 43 | total_timesteps 790.
Path 44 | total_timesteps 803.
Path 45 | total_timesteps 827.
Path 46 | total_timesteps 870.
Path 47 | total_timesteps 899.
Path 48 | total_timesteps 911.
Path 49 | total_timesteps 924.
Path 50 | total_timesteps 936.
Path 51 | total_timesteps 962.
Path 52 | total_timesteps 974.
Path 53 | total_timesteps 991.
Path 54 | total_timesteps 1003.
Path 55 | total_timesteps 1026.
Path 56 | total_timesteps 1056.
Path 57 | total_timesteps 1081.
Path 58 | total_timesteps 1094.
Path 59 | total_timesteps 1106.
Path 60 | total_timesteps 1119.
Path 61 | total_timesteps 1133.
Path 62 | total_timesteps 1154.
Path 63 | total_timesteps 1176.
Path 64 | total_timesteps 1195.
Path 65 | total_timesteps 1213.
Path 66 | total_timesteps 1243.
Path 67 | total_timesteps 1267.
Path 68 | total_timesteps 1311.
Path 69 | total_timesteps 1322.
Path 70 | total_timesteps 1334.
Path 71 | total_timesteps 1354.
Path 72 | total_timesteps 1369.
Path 73 | total_timesteps 1402.
Path 74 | total_timesteps 1418.
Path 75 | total_timesteps 1434.
Path 76 | total_timesteps 1448.
Path 77 | total_timesteps 1470.
Path 78 | total_timesteps 1479.
Path 79 | total_timesteps 1494.
Path 80 | total_timesteps 1509.
Path 81 | total_timesteps 1540.
Path 82 | total_timesteps 1557.
Path 83 | total_timesteps 1573.
Path 84 | total_timesteps 1595.
Path 85 | total_timesteps 1610.
Path 86 | total_timesteps 1629.
Path 87 | total_timesteps 1653.
Path 88 | total_timesteps 1669.
Path 89 | total_timesteps 1686.
Path 90 | total_timesteps 1704.
Path 91 | total_timesteps 1724.
Path 92 | total_timesteps 1739.
Path 93 | total_timesteps 1750.
Path 94 | total_timesteps 1785.
Path 95 | total_timesteps 1802.
Path 96 | total_timesteps 1812.
Path 97 | total_timesteps 1830.
Path 98 | total_timesteps 1841.
Path 99 | total_timesteps 1857.
Path 100 | total_timesteps 1866.
Path 101 | total_timesteps 1895.
Path 102 | total_timesteps 1906.
Path 103 | total_timesteps 1915.
Path 104 | total_timesteps 1936.
Path 105 | total_timesteps 1949.
Path 106 | total_timesteps 1977.
Path 107 | total_timesteps 2004.
Path 108 | total_timesteps 2020.
Path 109 | total_timesteps 2040.
Path 110 | total_timesteps 2071.
Path 111 | total_timesteps 2087.
Path 112 | total_timesteps 2109.
Path 113 | total_timesteps 2125.
Path 114 | total_timesteps 2143.
Path 115 | total_timesteps 2192.
Path 116 | total_timesteps 2211.
Path 117 | total_timesteps 2222.
Path 118 | total_timesteps 2235.
Path 119 | total_timesteps 2268.
Path 120 | total_timesteps 2285.
Path 121 | total_timesteps 2309.
Path 122 | total_timesteps 2326.
Path 123 | total_timesteps 2350.
Path 124 | total_timesteps 2362.
Path 125 | total_timesteps 2375.
Path 126 | total_timesteps 2391.
Path 127 | total_timesteps 2409.
Path 128 | total_timesteps 2422.
Path 129 | total_timesteps 2439.
Path 130 | total_timesteps 2451.
Path 131 | total_timesteps 2467.
Path 132 | total_timesteps 2478.
Path 133 | total_timesteps 2504.
Path 134 | total_timesteps 2517.
Path 135 | total_timesteps 2542.
Path 136 | total_timesteps 2558.
Path 137 | total_timesteps 2575.
Path 138 | total_timesteps 2585.
Path 139 | total_timesteps 2604.
Path 140 | total_timesteps 2635.
Path 141 | total_timesteps 2643.
Path 142 | total_timesteps 2673.
Path 143 | total_timesteps 2696.
Path 144 | total_timesteps 2708.
Path 145 | total_timesteps 2723.
Path 146 | total_timesteps 2740.
Path 147 | total_timesteps 2751.
Path 148 | total_timesteps 2769.
Path 149 | total_timesteps 2780.
Path 150 | total_timesteps 2800.
Path 151 | total_timesteps 2820.
Path 152 | total_timesteps 2838.
Path 153 | total_timesteps 2858.
Path 154 | total_timesteps 2887.
Path 155 | total_timesteps 2901.
Path 156 | total_timesteps 2915.
Path 157 | total_timesteps 2927.
Path 158 | total_timesteps 2940.
Path 159 | total_timesteps 2969.
Path 160 | total_timesteps 3003.
Path 161 | total_timesteps 3017.
Path 162 | total_timesteps 3025.
Path 163 | total_timesteps 3043.
Path 164 | total_timesteps 3057.
Path 165 | total_timesteps 3085.
Path 166 | total_timesteps 3098.
Path 167 | total_timesteps 3113.
Path 168 | total_timesteps 3128.
Path 169 | total_timesteps 3144.
Path 170 | total_timesteps 3175.
Path 171 | total_timesteps 3202.
Path 172 | total_timesteps 3223.
Path 173 | total_timesteps 3257.
Path 174 | total_timesteps 3274.
Path 175 | total_timesteps 3305.
Path 176 | total_timesteps 3333.
Path 177 | total_timesteps 3345.
Path 178 | total_timesteps 3359.
Path 179 | total_timesteps 3373.
Path 180 | total_timesteps 3396.
Path 181 | total_timesteps 3436.
Path 182 | total_timesteps 3450.
Path 183 | total_timesteps 3474.
Path 184 | total_timesteps 3492.
Path 185 | total_timesteps 3515.
Path 186 | total_timesteps 3530.
Path 187 | total_timesteps 3539.
Path 188 | total_timesteps 3556.
Path 189 | total_timesteps 3565.
Path 190 | total_timesteps 3577.
Path 191 | total_timesteps 3591.
Path 192 | total_timesteps 3618.
Path 193 | total_timesteps 3636.
Path 194 | total_timesteps 3646.
Path 195 | total_timesteps 3666.
Path 196 | total_timesteps 3681.
Path 197 | total_timesteps 3705.
Path 198 | total_timesteps 3722.
Path 199 | total_timesteps 3743.
Path 200 | total_timesteps 3755.
Path 201 | total_timesteps 3773.
Path 202 | total_timesteps 3802.
Path 203 | total_timesteps 3823.
Path 204 | total_timesteps 3836.
Path 205 | total_timesteps 3854.
Path 206 | total_timesteps 3880.
Path 207 | total_timesteps 3893.
Path 208 | total_timesteps 3906.
Path 209 | total_timesteps 3920.
Path 210 | total_timesteps 3933.
Path 211 | total_timesteps 3949.
Path 212 | total_timesteps 3967.
Path 213 | total_timesteps 3985.
Path 214 | total_timesteps 4016.
Path 215 | total_timesteps 4032.
Path 216 | total_timesteps 4050.
Path 217 | total_timesteps 4062.
Path 218 | total_timesteps 4088.
Path 219 | total_timesteps 4099.
Path 220 | total_timesteps 4124.
Path 221 | total_timesteps 4152.
Path 222 | total_timesteps 4170.
Path 223 | total_timesteps 4191.
Path 224 | total_timesteps 4207.
Path 225 | total_timesteps 4226.
Path 226 | total_timesteps 4246.
Path 227 | total_timesteps 4261.
Path 228 | total_timesteps 4277.
Path 229 | total_timesteps 4295.
Path 230 | total_timesteps 4313.
Path 231 | total_timesteps 4328.
Path 232 | total_timesteps 4343.
Path 233 | total_timesteps 4355.
Path 234 | total_timesteps 4371.
Path 235 | total_timesteps 4384.
Path 236 | total_timesteps 4408.
Path 237 | total_timesteps 4421.
Path 238 | total_timesteps 4435.
Path 239 | total_timesteps 4454.
Path 240 | total_timesteps 4469.
Path 241 | total_timesteps 4491.
Path 242 | total_timesteps 4525.
Path 243 | total_timesteps 4546.
Path 244 | total_timesteps 4566.
Path 245 | total_timesteps 4585.
Path 246 | total_timesteps 4612.
Path 247 | total_timesteps 4627.
Path 248 | total_timesteps 4639.
Path 249 | total_timesteps 4651.
Path 250 | total_timesteps 4663.
Path 251 | total_timesteps 4676.
Path 252 | total_timesteps 4686.
Path 253 | total_timesteps 4708.
Path 254 | total_timesteps 4726.
Path 255 | total_timesteps 4741.
Path 256 | total_timesteps 4755.
Path 257 | total_timesteps 4784.
Path 258 | total_timesteps 4804.
Path 259 | total_timesteps 4830.
Path 260 | total_timesteps 4840.
Path 261 | total_timesteps 4859.
Path 262 | total_timesteps 4884.
Path 263 | total_timesteps 4895.
Path 264 | total_timesteps 4906.
Path 265 | total_timesteps 4921.
Path 266 | total_timesteps 4940.
Path 267 | total_timesteps 4960.
Path 268 | total_timesteps 4978.
Path 269 | total_timesteps 5006.
Path 270 | total_timesteps 5034.
Path 271 | total_timesteps 5051.
Path 272 | total_timesteps 5081.
Path 273 | total_timesteps 5093.
Path 274 | total_timesteps 5111.
Path 275 | total_timesteps 5125.
Path 276 | total_timesteps 5149.
Path 277 | total_timesteps 5164.
Path 278 | total_timesteps 5174.
Path 279 | total_timesteps 5190.
Path 280 | total_timesteps 5198.
Path 281 | total_timesteps 5221.
Path 282 | total_timesteps 5233.
Path 283 | total_timesteps 5251.
Path 284 | total_timesteps 5269.
Path 285 | total_timesteps 5284.
Path 286 | total_timesteps 5301.
Path 287 | total_timesteps 5313.
Path 288 | total_timesteps 5331.
Path 289 | total_timesteps 5347.
Path 290 | total_timesteps 5359.
Path 291 | total_timesteps 5383.
Path 292 | total_timesteps 5396.
Path 293 | total_timesteps 5408.
Path 294 | total_timesteps 5423.
Path 295 | total_timesteps 5445.
Path 296 | total_timesteps 5456.
Path 297 | total_timesteps 5470.
Path 298 | total_timesteps 5487.
Path 299 | total_timesteps 5505.
Path 300 | total_timesteps 5515.
Path 301 | total_timesteps 5530.
Path 302 | total_timesteps 5544.
Path 303 | total_timesteps 5562.
Path 304 | total_timesteps 5583.
Path 305 | total_timesteps 5599.
Path 306 | total_timesteps 5614.
Path 307 | total_timesteps 5624.
Path 308 | total_timesteps 5645.
Path 309 | total_timesteps 5660.
Path 310 | total_timesteps 5675.
Path 311 | total_timesteps 5693.
Path 312 | total_timesteps 5708.
Path 313 | total_timesteps 5731.
Path 314 | total_timesteps 5750.
Path 315 | total_timesteps 5768.
Path 316 | total_timesteps 5785.
Path 317 | total_timesteps 5793.
Path 318 | total_timesteps 5801.
Path 319 | total_timesteps 5811.
Path 320 | total_timesteps 5829.
Path 321 | total_timesteps 5845.
Path 322 | total_timesteps 5859.
Path 323 | total_timesteps 5892.
Path 324 | total_timesteps 5913.
Path 325 | total_timesteps 5923.
Path 326 | total_timesteps 5945.
Path 327 | total_timesteps 5956.
Path 328 | total_timesteps 5968.
Path 329 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.39    |
| Iteration     | 9        |
| MaximumReturn | 4.86     |
| MinimumReturn | -22.3    |
| TotalSamples  | 44096    |
----------------------------
itr #10 | 
Fitting dynamics.
Validation loss = 0.013696172274649143
Validation loss = 0.012278396636247635
Validation loss = 0.01247809175401926
Validation loss = 0.01279662549495697
Validation loss = 0.013114208355545998
Validation loss = 0.013669290579855442
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 25.
Path 3 | total_timesteps 35.
Path 4 | total_timesteps 59.
Path 5 | total_timesteps 82.
Path 6 | total_timesteps 93.
Path 7 | total_timesteps 109.
Path 8 | total_timesteps 123.
Path 9 | total_timesteps 136.
Path 10 | total_timesteps 145.
Path 11 | total_timesteps 156.
Path 12 | total_timesteps 166.
Path 13 | total_timesteps 192.
Path 14 | total_timesteps 208.
Path 15 | total_timesteps 229.
Path 16 | total_timesteps 238.
Path 17 | total_timesteps 252.
Path 18 | total_timesteps 277.
Path 19 | total_timesteps 292.
Path 20 | total_timesteps 306.
Path 21 | total_timesteps 316.
Path 22 | total_timesteps 328.
Path 23 | total_timesteps 341.
Path 24 | total_timesteps 355.
Path 25 | total_timesteps 363.
Path 26 | total_timesteps 401.
Path 27 | total_timesteps 410.
Path 28 | total_timesteps 426.
Path 29 | total_timesteps 441.
Path 30 | total_timesteps 453.
Path 31 | total_timesteps 471.
Path 32 | total_timesteps 479.
Path 33 | total_timesteps 499.
Path 34 | total_timesteps 525.
Path 35 | total_timesteps 542.
Path 36 | total_timesteps 556.
Path 37 | total_timesteps 566.
Path 38 | total_timesteps 576.
Path 39 | total_timesteps 591.
Path 40 | total_timesteps 605.
Path 41 | total_timesteps 623.
Path 42 | total_timesteps 642.
Path 43 | total_timesteps 664.
Path 44 | total_timesteps 675.
Path 45 | total_timesteps 691.
Path 46 | total_timesteps 708.
Path 47 | total_timesteps 722.
Path 48 | total_timesteps 730.
Path 49 | total_timesteps 741.
Path 50 | total_timesteps 749.
Path 51 | total_timesteps 772.
Path 52 | total_timesteps 789.
Path 53 | total_timesteps 813.
Path 54 | total_timesteps 823.
Path 55 | total_timesteps 837.
Path 56 | total_timesteps 859.
Path 57 | total_timesteps 867.
Path 58 | total_timesteps 884.
Path 59 | total_timesteps 900.
Path 60 | total_timesteps 914.
Path 61 | total_timesteps 941.
Path 62 | total_timesteps 954.
Path 63 | total_timesteps 965.
Path 64 | total_timesteps 993.
Path 65 | total_timesteps 1019.
Path 66 | total_timesteps 1032.
Path 67 | total_timesteps 1044.
Path 68 | total_timesteps 1089.
Path 69 | total_timesteps 1107.
Path 70 | total_timesteps 1125.
Path 71 | total_timesteps 1141.
Path 72 | total_timesteps 1164.
Path 73 | total_timesteps 1180.
Path 74 | total_timesteps 1193.
Path 75 | total_timesteps 1208.
Path 76 | total_timesteps 1232.
Path 77 | total_timesteps 1247.
Path 78 | total_timesteps 1268.
Path 79 | total_timesteps 1288.
Path 80 | total_timesteps 1309.
Path 81 | total_timesteps 1326.
Path 82 | total_timesteps 1348.
Path 83 | total_timesteps 1363.
Path 84 | total_timesteps 1380.
Path 85 | total_timesteps 1390.
Path 86 | total_timesteps 1410.
Path 87 | total_timesteps 1430.
Path 88 | total_timesteps 1443.
Path 89 | total_timesteps 1454.
Path 90 | total_timesteps 1467.
Path 91 | total_timesteps 1487.
Path 92 | total_timesteps 1497.
Path 93 | total_timesteps 1507.
Path 94 | total_timesteps 1524.
Path 95 | total_timesteps 1538.
Path 96 | total_timesteps 1548.
Path 97 | total_timesteps 1563.
Path 98 | total_timesteps 1587.
Path 99 | total_timesteps 1600.
Path 100 | total_timesteps 1614.
Path 101 | total_timesteps 1639.
Path 102 | total_timesteps 1649.
Path 103 | total_timesteps 1662.
Path 104 | total_timesteps 1680.
Path 105 | total_timesteps 1695.
Path 106 | total_timesteps 1713.
Path 107 | total_timesteps 1729.
Path 108 | total_timesteps 1738.
Path 109 | total_timesteps 1747.
Path 110 | total_timesteps 1762.
Path 111 | total_timesteps 1771.
Path 112 | total_timesteps 1797.
Path 113 | total_timesteps 1809.
Path 114 | total_timesteps 1825.
Path 115 | total_timesteps 1834.
Path 116 | total_timesteps 1853.
Path 117 | total_timesteps 1874.
Path 118 | total_timesteps 1886.
Path 119 | total_timesteps 1906.
Path 120 | total_timesteps 1930.
Path 121 | total_timesteps 1950.
Path 122 | total_timesteps 1963.
Path 123 | total_timesteps 1988.
Path 124 | total_timesteps 2000.
Path 125 | total_timesteps 2015.
Path 126 | total_timesteps 2038.
Path 127 | total_timesteps 2054.
Path 128 | total_timesteps 2063.
Path 129 | total_timesteps 2077.
Path 130 | total_timesteps 2092.
Path 131 | total_timesteps 2111.
Path 132 | total_timesteps 2128.
Path 133 | total_timesteps 2141.
Path 134 | total_timesteps 2160.
Path 135 | total_timesteps 2171.
Path 136 | total_timesteps 2181.
Path 137 | total_timesteps 2194.
Path 138 | total_timesteps 2206.
Path 139 | total_timesteps 2222.
Path 140 | total_timesteps 2234.
Path 141 | total_timesteps 2249.
Path 142 | total_timesteps 2279.
Path 143 | total_timesteps 2290.
Path 144 | total_timesteps 2304.
Path 145 | total_timesteps 2317.
Path 146 | total_timesteps 2359.
Path 147 | total_timesteps 2384.
Path 148 | total_timesteps 2400.
Path 149 | total_timesteps 2421.
Path 150 | total_timesteps 2453.
Path 151 | total_timesteps 2461.
Path 152 | total_timesteps 2486.
Path 153 | total_timesteps 2499.
Path 154 | total_timesteps 2510.
Path 155 | total_timesteps 2522.
Path 156 | total_timesteps 2543.
Path 157 | total_timesteps 2565.
Path 158 | total_timesteps 2581.
Path 159 | total_timesteps 2598.
Path 160 | total_timesteps 2611.
Path 161 | total_timesteps 2623.
Path 162 | total_timesteps 2635.
Path 163 | total_timesteps 2643.
Path 164 | total_timesteps 2662.
Path 165 | total_timesteps 2671.
Path 166 | total_timesteps 2698.
Path 167 | total_timesteps 2715.
Path 168 | total_timesteps 2726.
Path 169 | total_timesteps 2743.
Path 170 | total_timesteps 2760.
Path 171 | total_timesteps 2772.
Path 172 | total_timesteps 2784.
Path 173 | total_timesteps 2797.
Path 174 | total_timesteps 2817.
Path 175 | total_timesteps 2831.
Path 176 | total_timesteps 2857.
Path 177 | total_timesteps 2875.
Path 178 | total_timesteps 2900.
Path 179 | total_timesteps 2914.
Path 180 | total_timesteps 2925.
Path 181 | total_timesteps 2938.
Path 182 | total_timesteps 2952.
Path 183 | total_timesteps 2963.
Path 184 | total_timesteps 2979.
Path 185 | total_timesteps 2989.
Path 186 | total_timesteps 3003.
Path 187 | total_timesteps 3025.
Path 188 | total_timesteps 3060.
Path 189 | total_timesteps 3075.
Path 190 | total_timesteps 3089.
Path 191 | total_timesteps 3102.
Path 192 | total_timesteps 3112.
Path 193 | total_timesteps 3122.
Path 194 | total_timesteps 3135.
Path 195 | total_timesteps 3145.
Path 196 | total_timesteps 3158.
Path 197 | total_timesteps 3168.
Path 198 | total_timesteps 3187.
Path 199 | total_timesteps 3198.
Path 200 | total_timesteps 3208.
Path 201 | total_timesteps 3231.
Path 202 | total_timesteps 3243.
Path 203 | total_timesteps 3262.
Path 204 | total_timesteps 3278.
Path 205 | total_timesteps 3290.
Path 206 | total_timesteps 3299.
Path 207 | total_timesteps 3317.
Path 208 | total_timesteps 3340.
Path 209 | total_timesteps 3353.
Path 210 | total_timesteps 3364.
Path 211 | total_timesteps 3384.
Path 212 | total_timesteps 3399.
Path 213 | total_timesteps 3415.
Path 214 | total_timesteps 3430.
Path 215 | total_timesteps 3458.
Path 216 | total_timesteps 3469.
Path 217 | total_timesteps 3486.
Path 218 | total_timesteps 3495.
Path 219 | total_timesteps 3503.
Path 220 | total_timesteps 3523.
Path 221 | total_timesteps 3540.
Path 222 | total_timesteps 3563.
Path 223 | total_timesteps 3574.
Path 224 | total_timesteps 3587.
Path 225 | total_timesteps 3597.
Path 226 | total_timesteps 3620.
Path 227 | total_timesteps 3635.
Path 228 | total_timesteps 3659.
Path 229 | total_timesteps 3680.
Path 230 | total_timesteps 3701.
Path 231 | total_timesteps 3717.
Path 232 | total_timesteps 3730.
Path 233 | total_timesteps 3745.
Path 234 | total_timesteps 3783.
Path 235 | total_timesteps 3798.
Path 236 | total_timesteps 3805.
Path 237 | total_timesteps 3818.
Path 238 | total_timesteps 3834.
Path 239 | total_timesteps 3855.
Path 240 | total_timesteps 3875.
Path 241 | total_timesteps 3884.
Path 242 | total_timesteps 3902.
Path 243 | total_timesteps 3918.
Path 244 | total_timesteps 3937.
Path 245 | total_timesteps 3952.
Path 246 | total_timesteps 3983.
Path 247 | total_timesteps 3994.
Path 248 | total_timesteps 4008.
Path 249 | total_timesteps 4028.
Path 250 | total_timesteps 4039.
Path 251 | total_timesteps 4054.
Path 252 | total_timesteps 4065.
Path 253 | total_timesteps 4073.
Path 254 | total_timesteps 4094.
Path 255 | total_timesteps 4103.
Path 256 | total_timesteps 4114.
Path 257 | total_timesteps 4127.
Path 258 | total_timesteps 4150.
Path 259 | total_timesteps 4165.
Path 260 | total_timesteps 4191.
Path 261 | total_timesteps 4206.
Path 262 | total_timesteps 4215.
Path 263 | total_timesteps 4225.
Path 264 | total_timesteps 4242.
Path 265 | total_timesteps 4258.
Path 266 | total_timesteps 4271.
Path 267 | total_timesteps 4284.
Path 268 | total_timesteps 4313.
Path 269 | total_timesteps 4324.
Path 270 | total_timesteps 4341.
Path 271 | total_timesteps 4350.
Path 272 | total_timesteps 4372.
Path 273 | total_timesteps 4386.
Path 274 | total_timesteps 4397.
Path 275 | total_timesteps 4408.
Path 276 | total_timesteps 4422.
Path 277 | total_timesteps 4433.
Path 278 | total_timesteps 4469.
Path 279 | total_timesteps 4482.
Path 280 | total_timesteps 4496.
Path 281 | total_timesteps 4517.
Path 282 | total_timesteps 4539.
Path 283 | total_timesteps 4559.
Path 284 | total_timesteps 4580.
Path 285 | total_timesteps 4595.
Path 286 | total_timesteps 4604.
Path 287 | total_timesteps 4636.
Path 288 | total_timesteps 4646.
Path 289 | total_timesteps 4657.
Path 290 | total_timesteps 4671.
Path 291 | total_timesteps 4681.
Path 292 | total_timesteps 4694.
Path 293 | total_timesteps 4709.
Path 294 | total_timesteps 4717.
Path 295 | total_timesteps 4727.
Path 296 | total_timesteps 4738.
Path 297 | total_timesteps 4759.
Path 298 | total_timesteps 4779.
Path 299 | total_timesteps 4805.
Path 300 | total_timesteps 4825.
Path 301 | total_timesteps 4840.
Path 302 | total_timesteps 4853.
Path 303 | total_timesteps 4873.
Path 304 | total_timesteps 4888.
Path 305 | total_timesteps 4905.
Path 306 | total_timesteps 4922.
Path 307 | total_timesteps 4935.
Path 308 | total_timesteps 4947.
Path 309 | total_timesteps 4960.
Path 310 | total_timesteps 4975.
Path 311 | total_timesteps 4991.
Path 312 | total_timesteps 5006.
Path 313 | total_timesteps 5021.
Path 314 | total_timesteps 5034.
Path 315 | total_timesteps 5043.
Path 316 | total_timesteps 5073.
Path 317 | total_timesteps 5086.
Path 318 | total_timesteps 5099.
Path 319 | total_timesteps 5111.
Path 320 | total_timesteps 5141.
Path 321 | total_timesteps 5173.
Path 322 | total_timesteps 5183.
Path 323 | total_timesteps 5204.
Path 324 | total_timesteps 5218.
Path 325 | total_timesteps 5251.
Path 326 | total_timesteps 5269.
Path 327 | total_timesteps 5289.
Path 328 | total_timesteps 5305.
Path 329 | total_timesteps 5315.
Path 330 | total_timesteps 5342.
Path 331 | total_timesteps 5360.
Path 332 | total_timesteps 5369.
Path 333 | total_timesteps 5379.
Path 334 | total_timesteps 5397.
Path 335 | total_timesteps 5407.
Path 336 | total_timesteps 5421.
Path 337 | total_timesteps 5434.
Path 338 | total_timesteps 5442.
Path 339 | total_timesteps 5451.
Path 340 | total_timesteps 5465.
Path 341 | total_timesteps 5486.
Path 342 | total_timesteps 5509.
Path 343 | total_timesteps 5520.
Path 344 | total_timesteps 5541.
Path 345 | total_timesteps 5562.
Path 346 | total_timesteps 5576.
Path 347 | total_timesteps 5587.
Path 348 | total_timesteps 5600.
Path 349 | total_timesteps 5613.
Path 350 | total_timesteps 5626.
Path 351 | total_timesteps 5635.
Path 352 | total_timesteps 5645.
Path 353 | total_timesteps 5660.
Path 354 | total_timesteps 5675.
Path 355 | total_timesteps 5686.
Path 356 | total_timesteps 5701.
Path 357 | total_timesteps 5714.
Path 358 | total_timesteps 5733.
Path 359 | total_timesteps 5752.
Path 360 | total_timesteps 5768.
Path 361 | total_timesteps 5779.
Path 362 | total_timesteps 5802.
Path 363 | total_timesteps 5822.
Path 364 | total_timesteps 5837.
Path 365 | total_timesteps 5854.
Path 366 | total_timesteps 5873.
Path 367 | total_timesteps 5887.
Path 368 | total_timesteps 5900.
Path 369 | total_timesteps 5917.
Path 370 | total_timesteps 5928.
Path 371 | total_timesteps 5938.
Path 372 | total_timesteps 5959.
Path 373 | total_timesteps 5977.
Path 374 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.16    |
| Iteration     | 10       |
| MaximumReturn | 5.3      |
| MinimumReturn | -21.3    |
| TotalSamples  | 48108    |
----------------------------
itr #11 | 
Fitting dynamics.
Validation loss = 0.01314624398946762
Validation loss = 0.012055049650371075
Validation loss = 0.012053119949996471
Validation loss = 0.011450618505477905
Validation loss = 0.014038736931979656
Validation loss = 0.011681794188916683
Validation loss = 0.011859425343573093
Validation loss = 0.012126725167036057
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 30.
Path 3 | total_timesteps 49.
Path 4 | total_timesteps 58.
Path 5 | total_timesteps 68.
Path 6 | total_timesteps 90.
Path 7 | total_timesteps 99.
Path 8 | total_timesteps 124.
Path 9 | total_timesteps 136.
Path 10 | total_timesteps 148.
Path 11 | total_timesteps 159.
Path 12 | total_timesteps 175.
Path 13 | total_timesteps 192.
Path 14 | total_timesteps 210.
Path 15 | total_timesteps 224.
Path 16 | total_timesteps 248.
Path 17 | total_timesteps 273.
Path 18 | total_timesteps 292.
Path 19 | total_timesteps 300.
Path 20 | total_timesteps 339.
Path 21 | total_timesteps 349.
Path 22 | total_timesteps 371.
Path 23 | total_timesteps 392.
Path 24 | total_timesteps 404.
Path 25 | total_timesteps 412.
Path 26 | total_timesteps 432.
Path 27 | total_timesteps 445.
Path 28 | total_timesteps 458.
Path 29 | total_timesteps 468.
Path 30 | total_timesteps 482.
Path 31 | total_timesteps 500.
Path 32 | total_timesteps 510.
Path 33 | total_timesteps 522.
Path 34 | total_timesteps 537.
Path 35 | total_timesteps 567.
Path 36 | total_timesteps 577.
Path 37 | total_timesteps 604.
Path 38 | total_timesteps 623.
Path 39 | total_timesteps 639.
Path 40 | total_timesteps 654.
Path 41 | total_timesteps 674.
Path 42 | total_timesteps 685.
Path 43 | total_timesteps 715.
Path 44 | total_timesteps 731.
Path 45 | total_timesteps 741.
Path 46 | total_timesteps 752.
Path 47 | total_timesteps 759.
Path 48 | total_timesteps 769.
Path 49 | total_timesteps 784.
Path 50 | total_timesteps 808.
Path 51 | total_timesteps 826.
Path 52 | total_timesteps 837.
Path 53 | total_timesteps 857.
Path 54 | total_timesteps 870.
Path 55 | total_timesteps 887.
Path 56 | total_timesteps 917.
Path 57 | total_timesteps 934.
Path 58 | total_timesteps 950.
Path 59 | total_timesteps 970.
Path 60 | total_timesteps 981.
Path 61 | total_timesteps 996.
Path 62 | total_timesteps 1010.
Path 63 | total_timesteps 1024.
Path 64 | total_timesteps 1035.
Path 65 | total_timesteps 1056.
Path 66 | total_timesteps 1071.
Path 67 | total_timesteps 1093.
Path 68 | total_timesteps 1106.
Path 69 | total_timesteps 1121.
Path 70 | total_timesteps 1136.
Path 71 | total_timesteps 1155.
Path 72 | total_timesteps 1173.
Path 73 | total_timesteps 1189.
Path 74 | total_timesteps 1205.
Path 75 | total_timesteps 1220.
Path 76 | total_timesteps 1238.
Path 77 | total_timesteps 1253.
Path 78 | total_timesteps 1269.
Path 79 | total_timesteps 1303.
Path 80 | total_timesteps 1315.
Path 81 | total_timesteps 1338.
Path 82 | total_timesteps 1352.
Path 83 | total_timesteps 1375.
Path 84 | total_timesteps 1390.
Path 85 | total_timesteps 1411.
Path 86 | total_timesteps 1424.
Path 87 | total_timesteps 1444.
Path 88 | total_timesteps 1456.
Path 89 | total_timesteps 1482.
Path 90 | total_timesteps 1503.
Path 91 | total_timesteps 1521.
Path 92 | total_timesteps 1536.
Path 93 | total_timesteps 1552.
Path 94 | total_timesteps 1564.
Path 95 | total_timesteps 1580.
Path 96 | total_timesteps 1591.
Path 97 | total_timesteps 1600.
Path 98 | total_timesteps 1610.
Path 99 | total_timesteps 1633.
Path 100 | total_timesteps 1648.
Path 101 | total_timesteps 1662.
Path 102 | total_timesteps 1681.
Path 103 | total_timesteps 1699.
Path 104 | total_timesteps 1717.
Path 105 | total_timesteps 1733.
Path 106 | total_timesteps 1751.
Path 107 | total_timesteps 1771.
Path 108 | total_timesteps 1784.
Path 109 | total_timesteps 1795.
Path 110 | total_timesteps 1810.
Path 111 | total_timesteps 1835.
Path 112 | total_timesteps 1851.
Path 113 | total_timesteps 1862.
Path 114 | total_timesteps 1875.
Path 115 | total_timesteps 1890.
Path 116 | total_timesteps 1905.
Path 117 | total_timesteps 1941.
Path 118 | total_timesteps 1956.
Path 119 | total_timesteps 1974.
Path 120 | total_timesteps 1997.
Path 121 | total_timesteps 2024.
Path 122 | total_timesteps 2034.
Path 123 | total_timesteps 2045.
Path 124 | total_timesteps 2066.
Path 125 | total_timesteps 2086.
Path 126 | total_timesteps 2097.
Path 127 | total_timesteps 2112.
Path 128 | total_timesteps 2127.
Path 129 | total_timesteps 2148.
Path 130 | total_timesteps 2160.
Path 131 | total_timesteps 2183.
Path 132 | total_timesteps 2212.
Path 133 | total_timesteps 2224.
Path 134 | total_timesteps 2240.
Path 135 | total_timesteps 2264.
Path 136 | total_timesteps 2285.
Path 137 | total_timesteps 2295.
Path 138 | total_timesteps 2330.
Path 139 | total_timesteps 2341.
Path 140 | total_timesteps 2353.
Path 141 | total_timesteps 2365.
Path 142 | total_timesteps 2374.
Path 143 | total_timesteps 2394.
Path 144 | total_timesteps 2409.
Path 145 | total_timesteps 2425.
Path 146 | total_timesteps 2435.
Path 147 | total_timesteps 2456.
Path 148 | total_timesteps 2466.
Path 149 | total_timesteps 2483.
Path 150 | total_timesteps 2506.
Path 151 | total_timesteps 2521.
Path 152 | total_timesteps 2533.
Path 153 | total_timesteps 2545.
Path 154 | total_timesteps 2567.
Path 155 | total_timesteps 2610.
Path 156 | total_timesteps 2624.
Path 157 | total_timesteps 2634.
Path 158 | total_timesteps 2649.
Path 159 | total_timesteps 2660.
Path 160 | total_timesteps 2676.
Path 161 | total_timesteps 2692.
Path 162 | total_timesteps 2704.
Path 163 | total_timesteps 2715.
Path 164 | total_timesteps 2731.
Path 165 | total_timesteps 2745.
Path 166 | total_timesteps 2770.
Path 167 | total_timesteps 2784.
Path 168 | total_timesteps 2797.
Path 169 | total_timesteps 2805.
Path 170 | total_timesteps 2816.
Path 171 | total_timesteps 2830.
Path 172 | total_timesteps 2846.
Path 173 | total_timesteps 2857.
Path 174 | total_timesteps 2873.
Path 175 | total_timesteps 2889.
Path 176 | total_timesteps 2899.
Path 177 | total_timesteps 2912.
Path 178 | total_timesteps 2931.
Path 179 | total_timesteps 2948.
Path 180 | total_timesteps 2956.
Path 181 | total_timesteps 2969.
Path 182 | total_timesteps 2989.
Path 183 | total_timesteps 2997.
Path 184 | total_timesteps 3011.
Path 185 | total_timesteps 3027.
Path 186 | total_timesteps 3049.
Path 187 | total_timesteps 3066.
Path 188 | total_timesteps 3084.
Path 189 | total_timesteps 3100.
Path 190 | total_timesteps 3142.
Path 191 | total_timesteps 3157.
Path 192 | total_timesteps 3174.
Path 193 | total_timesteps 3198.
Path 194 | total_timesteps 3213.
Path 195 | total_timesteps 3224.
Path 196 | total_timesteps 3252.
Path 197 | total_timesteps 3268.
Path 198 | total_timesteps 3281.
Path 199 | total_timesteps 3304.
Path 200 | total_timesteps 3317.
Path 201 | total_timesteps 3329.
Path 202 | total_timesteps 3351.
Path 203 | total_timesteps 3384.
Path 204 | total_timesteps 3394.
Path 205 | total_timesteps 3402.
Path 206 | total_timesteps 3421.
Path 207 | total_timesteps 3446.
Path 208 | total_timesteps 3457.
Path 209 | total_timesteps 3475.
Path 210 | total_timesteps 3491.
Path 211 | total_timesteps 3502.
Path 212 | total_timesteps 3539.
Path 213 | total_timesteps 3560.
Path 214 | total_timesteps 3585.
Path 215 | total_timesteps 3601.
Path 216 | total_timesteps 3617.
Path 217 | total_timesteps 3638.
Path 218 | total_timesteps 3656.
Path 219 | total_timesteps 3677.
Path 220 | total_timesteps 3689.
Path 221 | total_timesteps 3710.
Path 222 | total_timesteps 3744.
Path 223 | total_timesteps 3754.
Path 224 | total_timesteps 3770.
Path 225 | total_timesteps 3780.
Path 226 | total_timesteps 3793.
Path 227 | total_timesteps 3805.
Path 228 | total_timesteps 3817.
Path 229 | total_timesteps 3831.
Path 230 | total_timesteps 3844.
Path 231 | total_timesteps 3862.
Path 232 | total_timesteps 3884.
Path 233 | total_timesteps 3897.
Path 234 | total_timesteps 3924.
Path 235 | total_timesteps 3943.
Path 236 | total_timesteps 3954.
Path 237 | total_timesteps 3965.
Path 238 | total_timesteps 3982.
Path 239 | total_timesteps 3999.
Path 240 | total_timesteps 4013.
Path 241 | total_timesteps 4024.
Path 242 | total_timesteps 4035.
Path 243 | total_timesteps 4049.
Path 244 | total_timesteps 4061.
Path 245 | total_timesteps 4079.
Path 246 | total_timesteps 4087.
Path 247 | total_timesteps 4106.
Path 248 | total_timesteps 4125.
Path 249 | total_timesteps 4141.
Path 250 | total_timesteps 4159.
Path 251 | total_timesteps 4171.
Path 252 | total_timesteps 4183.
Path 253 | total_timesteps 4199.
Path 254 | total_timesteps 4227.
Path 255 | total_timesteps 4237.
Path 256 | total_timesteps 4254.
Path 257 | total_timesteps 4263.
Path 258 | total_timesteps 4273.
Path 259 | total_timesteps 4288.
Path 260 | total_timesteps 4310.
Path 261 | total_timesteps 4318.
Path 262 | total_timesteps 4327.
Path 263 | total_timesteps 4363.
Path 264 | total_timesteps 4378.
Path 265 | total_timesteps 4393.
Path 266 | total_timesteps 4404.
Path 267 | total_timesteps 4424.
Path 268 | total_timesteps 4434.
Path 269 | total_timesteps 4452.
Path 270 | total_timesteps 4468.
Path 271 | total_timesteps 4507.
Path 272 | total_timesteps 4528.
Path 273 | total_timesteps 4540.
Path 274 | total_timesteps 4556.
Path 275 | total_timesteps 4568.
Path 276 | total_timesteps 4624.
Path 277 | total_timesteps 4636.
Path 278 | total_timesteps 4646.
Path 279 | total_timesteps 4669.
Path 280 | total_timesteps 4691.
Path 281 | total_timesteps 4706.
Path 282 | total_timesteps 4722.
Path 283 | total_timesteps 4745.
Path 284 | total_timesteps 4760.
Path 285 | total_timesteps 4770.
Path 286 | total_timesteps 4804.
Path 287 | total_timesteps 4815.
Path 288 | total_timesteps 4824.
Path 289 | total_timesteps 4842.
Path 290 | total_timesteps 4857.
Path 291 | total_timesteps 4876.
Path 292 | total_timesteps 4891.
Path 293 | total_timesteps 4924.
Path 294 | total_timesteps 4939.
Path 295 | total_timesteps 4967.
Path 296 | total_timesteps 4984.
Path 297 | total_timesteps 4991.
Path 298 | total_timesteps 5001.
Path 299 | total_timesteps 5011.
Path 300 | total_timesteps 5025.
Path 301 | total_timesteps 5037.
Path 302 | total_timesteps 5046.
Path 303 | total_timesteps 5057.
Path 304 | total_timesteps 5073.
Path 305 | total_timesteps 5102.
Path 306 | total_timesteps 5125.
Path 307 | total_timesteps 5137.
Path 308 | total_timesteps 5159.
Path 309 | total_timesteps 5173.
Path 310 | total_timesteps 5183.
Path 311 | total_timesteps 5194.
Path 312 | total_timesteps 5217.
Path 313 | total_timesteps 5229.
Path 314 | total_timesteps 5248.
Path 315 | total_timesteps 5262.
Path 316 | total_timesteps 5286.
Path 317 | total_timesteps 5303.
Path 318 | total_timesteps 5314.
Path 319 | total_timesteps 5321.
Path 320 | total_timesteps 5330.
Path 321 | total_timesteps 5343.
Path 322 | total_timesteps 5363.
Path 323 | total_timesteps 5385.
Path 324 | total_timesteps 5397.
Path 325 | total_timesteps 5409.
Path 326 | total_timesteps 5417.
Path 327 | total_timesteps 5429.
Path 328 | total_timesteps 5460.
Path 329 | total_timesteps 5477.
Path 330 | total_timesteps 5491.
Path 331 | total_timesteps 5507.
Path 332 | total_timesteps 5533.
Path 333 | total_timesteps 5546.
Path 334 | total_timesteps 5565.
Path 335 | total_timesteps 5579.
Path 336 | total_timesteps 5596.
Path 337 | total_timesteps 5610.
Path 338 | total_timesteps 5631.
Path 339 | total_timesteps 5647.
Path 340 | total_timesteps 5659.
Path 341 | total_timesteps 5677.
Path 342 | total_timesteps 5701.
Path 343 | total_timesteps 5711.
Path 344 | total_timesteps 5724.
Path 345 | total_timesteps 5751.
Path 346 | total_timesteps 5779.
Path 347 | total_timesteps 5796.
Path 348 | total_timesteps 5812.
Path 349 | total_timesteps 5826.
Path 350 | total_timesteps 5840.
Path 351 | total_timesteps 5851.
Path 352 | total_timesteps 5861.
Path 353 | total_timesteps 5884.
Path 354 | total_timesteps 5900.
Path 355 | total_timesteps 5930.
Path 356 | total_timesteps 5947.
Path 357 | total_timesteps 5959.
Path 358 | total_timesteps 5968.
Path 359 | total_timesteps 5976.
Path 360 | total_timesteps 5987.
Path 361 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.85    |
| Iteration     | 11       |
| MaximumReturn | 7.04     |
| MinimumReturn | -23.6    |
| TotalSamples  | 52112    |
----------------------------
itr #12 | 
Fitting dynamics.
Validation loss = 0.01192893274128437
Validation loss = 0.01100915763527155
Validation loss = 0.011671929620206356
Validation loss = 0.013175089843571186
Validation loss = 0.011871250346302986
Validation loss = 0.011258166283369064
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 41.
Path 4 | total_timesteps 56.
Path 5 | total_timesteps 74.
Path 6 | total_timesteps 99.
Path 7 | total_timesteps 112.
Path 8 | total_timesteps 129.
Path 9 | total_timesteps 143.
Path 10 | total_timesteps 158.
Path 11 | total_timesteps 174.
Path 12 | total_timesteps 189.
Path 13 | total_timesteps 199.
Path 14 | total_timesteps 229.
Path 15 | total_timesteps 239.
Path 16 | total_timesteps 251.
Path 17 | total_timesteps 260.
Path 18 | total_timesteps 275.
Path 19 | total_timesteps 296.
Path 20 | total_timesteps 328.
Path 21 | total_timesteps 341.
Path 22 | total_timesteps 351.
Path 23 | total_timesteps 361.
Path 24 | total_timesteps 377.
Path 25 | total_timesteps 389.
Path 26 | total_timesteps 415.
Path 27 | total_timesteps 441.
Path 28 | total_timesteps 470.
Path 29 | total_timesteps 482.
Path 30 | total_timesteps 499.
Path 31 | total_timesteps 515.
Path 32 | total_timesteps 531.
Path 33 | total_timesteps 550.
Path 34 | total_timesteps 557.
Path 35 | total_timesteps 569.
Path 36 | total_timesteps 580.
Path 37 | total_timesteps 597.
Path 38 | total_timesteps 611.
Path 39 | total_timesteps 641.
Path 40 | total_timesteps 650.
Path 41 | total_timesteps 663.
Path 42 | total_timesteps 678.
Path 43 | total_timesteps 699.
Path 44 | total_timesteps 713.
Path 45 | total_timesteps 725.
Path 46 | total_timesteps 736.
Path 47 | total_timesteps 755.
Path 48 | total_timesteps 770.
Path 49 | total_timesteps 788.
Path 50 | total_timesteps 800.
Path 51 | total_timesteps 822.
Path 52 | total_timesteps 832.
Path 53 | total_timesteps 853.
Path 54 | total_timesteps 869.
Path 55 | total_timesteps 888.
Path 56 | total_timesteps 906.
Path 57 | total_timesteps 924.
Path 58 | total_timesteps 937.
Path 59 | total_timesteps 947.
Path 60 | total_timesteps 961.
Path 61 | total_timesteps 970.
Path 62 | total_timesteps 979.
Path 63 | total_timesteps 1003.
Path 64 | total_timesteps 1017.
Path 65 | total_timesteps 1038.
Path 66 | total_timesteps 1055.
Path 67 | total_timesteps 1072.
Path 68 | total_timesteps 1084.
Path 69 | total_timesteps 1094.
Path 70 | total_timesteps 1112.
Path 71 | total_timesteps 1132.
Path 72 | total_timesteps 1147.
Path 73 | total_timesteps 1177.
Path 74 | total_timesteps 1194.
Path 75 | total_timesteps 1217.
Path 76 | total_timesteps 1234.
Path 77 | total_timesteps 1251.
Path 78 | total_timesteps 1264.
Path 79 | total_timesteps 1287.
Path 80 | total_timesteps 1299.
Path 81 | total_timesteps 1317.
Path 82 | total_timesteps 1331.
Path 83 | total_timesteps 1342.
Path 84 | total_timesteps 1357.
Path 85 | total_timesteps 1369.
Path 86 | total_timesteps 1390.
Path 87 | total_timesteps 1402.
Path 88 | total_timesteps 1414.
Path 89 | total_timesteps 1427.
Path 90 | total_timesteps 1438.
Path 91 | total_timesteps 1448.
Path 92 | total_timesteps 1464.
Path 93 | total_timesteps 1481.
Path 94 | total_timesteps 1506.
Path 95 | total_timesteps 1514.
Path 96 | total_timesteps 1526.
Path 97 | total_timesteps 1541.
Path 98 | total_timesteps 1555.
Path 99 | total_timesteps 1571.
Path 100 | total_timesteps 1579.
Path 101 | total_timesteps 1598.
Path 102 | total_timesteps 1617.
Path 103 | total_timesteps 1630.
Path 104 | total_timesteps 1643.
Path 105 | total_timesteps 1656.
Path 106 | total_timesteps 1667.
Path 107 | total_timesteps 1679.
Path 108 | total_timesteps 1690.
Path 109 | total_timesteps 1701.
Path 110 | total_timesteps 1721.
Path 111 | total_timesteps 1745.
Path 112 | total_timesteps 1755.
Path 113 | total_timesteps 1776.
Path 114 | total_timesteps 1788.
Path 115 | total_timesteps 1810.
Path 116 | total_timesteps 1822.
Path 117 | total_timesteps 1840.
Path 118 | total_timesteps 1862.
Path 119 | total_timesteps 1883.
Path 120 | total_timesteps 1899.
Path 121 | total_timesteps 1911.
Path 122 | total_timesteps 1927.
Path 123 | total_timesteps 1941.
Path 124 | total_timesteps 1961.
Path 125 | total_timesteps 1976.
Path 126 | total_timesteps 1994.
Path 127 | total_timesteps 2019.
Path 128 | total_timesteps 2030.
Path 129 | total_timesteps 2039.
Path 130 | total_timesteps 2049.
Path 131 | total_timesteps 2066.
Path 132 | total_timesteps 2093.
Path 133 | total_timesteps 2106.
Path 134 | total_timesteps 2133.
Path 135 | total_timesteps 2152.
Path 136 | total_timesteps 2172.
Path 137 | total_timesteps 2191.
Path 138 | total_timesteps 2206.
Path 139 | total_timesteps 2227.
Path 140 | total_timesteps 2235.
Path 141 | total_timesteps 2271.
Path 142 | total_timesteps 2300.
Path 143 | total_timesteps 2314.
Path 144 | total_timesteps 2325.
Path 145 | total_timesteps 2353.
Path 146 | total_timesteps 2363.
Path 147 | total_timesteps 2377.
Path 148 | total_timesteps 2388.
Path 149 | total_timesteps 2400.
Path 150 | total_timesteps 2412.
Path 151 | total_timesteps 2431.
Path 152 | total_timesteps 2453.
Path 153 | total_timesteps 2461.
Path 154 | total_timesteps 2491.
Path 155 | total_timesteps 2510.
Path 156 | total_timesteps 2521.
Path 157 | total_timesteps 2536.
Path 158 | total_timesteps 2552.
Path 159 | total_timesteps 2561.
Path 160 | total_timesteps 2588.
Path 161 | total_timesteps 2609.
Path 162 | total_timesteps 2623.
Path 163 | total_timesteps 2635.
Path 164 | total_timesteps 2650.
Path 165 | total_timesteps 2668.
Path 166 | total_timesteps 2679.
Path 167 | total_timesteps 2690.
Path 168 | total_timesteps 2702.
Path 169 | total_timesteps 2717.
Path 170 | total_timesteps 2727.
Path 171 | total_timesteps 2756.
Path 172 | total_timesteps 2767.
Path 173 | total_timesteps 2775.
Path 174 | total_timesteps 2803.
Path 175 | total_timesteps 2816.
Path 176 | total_timesteps 2836.
Path 177 | total_timesteps 2852.
Path 178 | total_timesteps 2868.
Path 179 | total_timesteps 2881.
Path 180 | total_timesteps 2889.
Path 181 | total_timesteps 2899.
Path 182 | total_timesteps 2922.
Path 183 | total_timesteps 2931.
Path 184 | total_timesteps 2942.
Path 185 | total_timesteps 2981.
Path 186 | total_timesteps 2995.
Path 187 | total_timesteps 3008.
Path 188 | total_timesteps 3030.
Path 189 | total_timesteps 3049.
Path 190 | total_timesteps 3060.
Path 191 | total_timesteps 3068.
Path 192 | total_timesteps 3080.
Path 193 | total_timesteps 3092.
Path 194 | total_timesteps 3103.
Path 195 | total_timesteps 3126.
Path 196 | total_timesteps 3139.
Path 197 | total_timesteps 3158.
Path 198 | total_timesteps 3168.
Path 199 | total_timesteps 3178.
Path 200 | total_timesteps 3190.
Path 201 | total_timesteps 3203.
Path 202 | total_timesteps 3218.
Path 203 | total_timesteps 3225.
Path 204 | total_timesteps 3243.
Path 205 | total_timesteps 3254.
Path 206 | total_timesteps 3297.
Path 207 | total_timesteps 3322.
Path 208 | total_timesteps 3334.
Path 209 | total_timesteps 3355.
Path 210 | total_timesteps 3375.
Path 211 | total_timesteps 3388.
Path 212 | total_timesteps 3404.
Path 213 | total_timesteps 3424.
Path 214 | total_timesteps 3485.
Path 215 | total_timesteps 3498.
Path 216 | total_timesteps 3518.
Path 217 | total_timesteps 3539.
Path 218 | total_timesteps 3551.
Path 219 | total_timesteps 3577.
Path 220 | total_timesteps 3586.
Path 221 | total_timesteps 3633.
Path 222 | total_timesteps 3648.
Path 223 | total_timesteps 3661.
Path 224 | total_timesteps 3673.
Path 225 | total_timesteps 3697.
Path 226 | total_timesteps 3714.
Path 227 | total_timesteps 3739.
Path 228 | total_timesteps 3754.
Path 229 | total_timesteps 3769.
Path 230 | total_timesteps 3786.
Path 231 | total_timesteps 3811.
Path 232 | total_timesteps 3833.
Path 233 | total_timesteps 3843.
Path 234 | total_timesteps 3859.
Path 235 | total_timesteps 3885.
Path 236 | total_timesteps 3899.
Path 237 | total_timesteps 3915.
Path 238 | total_timesteps 3948.
Path 239 | total_timesteps 3958.
Path 240 | total_timesteps 3982.
Path 241 | total_timesteps 4011.
Path 242 | total_timesteps 4021.
Path 243 | total_timesteps 4034.
Path 244 | total_timesteps 4063.
Path 245 | total_timesteps 4079.
Path 246 | total_timesteps 4094.
Path 247 | total_timesteps 4102.
Path 248 | total_timesteps 4113.
Path 249 | total_timesteps 4133.
Path 250 | total_timesteps 4148.
Path 251 | total_timesteps 4167.
Path 252 | total_timesteps 4181.
Path 253 | total_timesteps 4191.
Path 254 | total_timesteps 4213.
Path 255 | total_timesteps 4224.
Path 256 | total_timesteps 4244.
Path 257 | total_timesteps 4258.
Path 258 | total_timesteps 4271.
Path 259 | total_timesteps 4289.
Path 260 | total_timesteps 4305.
Path 261 | total_timesteps 4315.
Path 262 | total_timesteps 4338.
Path 263 | total_timesteps 4356.
Path 264 | total_timesteps 4378.
Path 265 | total_timesteps 4399.
Path 266 | total_timesteps 4409.
Path 267 | total_timesteps 4421.
Path 268 | total_timesteps 4441.
Path 269 | total_timesteps 4464.
Path 270 | total_timesteps 4473.
Path 271 | total_timesteps 4495.
Path 272 | total_timesteps 4516.
Path 273 | total_timesteps 4530.
Path 274 | total_timesteps 4540.
Path 275 | total_timesteps 4565.
Path 276 | total_timesteps 4575.
Path 277 | total_timesteps 4591.
Path 278 | total_timesteps 4600.
Path 279 | total_timesteps 4617.
Path 280 | total_timesteps 4632.
Path 281 | total_timesteps 4650.
Path 282 | total_timesteps 4661.
Path 283 | total_timesteps 4684.
Path 284 | total_timesteps 4704.
Path 285 | total_timesteps 4715.
Path 286 | total_timesteps 4736.
Path 287 | total_timesteps 4752.
Path 288 | total_timesteps 4772.
Path 289 | total_timesteps 4787.
Path 290 | total_timesteps 4804.
Path 291 | total_timesteps 4820.
Path 292 | total_timesteps 4839.
Path 293 | total_timesteps 4851.
Path 294 | total_timesteps 4875.
Path 295 | total_timesteps 4893.
Path 296 | total_timesteps 4909.
Path 297 | total_timesteps 4927.
Path 298 | total_timesteps 4941.
Path 299 | total_timesteps 4954.
Path 300 | total_timesteps 4967.
Path 301 | total_timesteps 4977.
Path 302 | total_timesteps 4994.
Path 303 | total_timesteps 5013.
Path 304 | total_timesteps 5026.
Path 305 | total_timesteps 5039.
Path 306 | total_timesteps 5071.
Path 307 | total_timesteps 5087.
Path 308 | total_timesteps 5099.
Path 309 | total_timesteps 5124.
Path 310 | total_timesteps 5145.
Path 311 | total_timesteps 5168.
Path 312 | total_timesteps 5177.
Path 313 | total_timesteps 5195.
Path 314 | total_timesteps 5210.
Path 315 | total_timesteps 5230.
Path 316 | total_timesteps 5261.
Path 317 | total_timesteps 5276.
Path 318 | total_timesteps 5310.
Path 319 | total_timesteps 5325.
Path 320 | total_timesteps 5337.
Path 321 | total_timesteps 5348.
Path 322 | total_timesteps 5366.
Path 323 | total_timesteps 5379.
Path 324 | total_timesteps 5387.
Path 325 | total_timesteps 5395.
Path 326 | total_timesteps 5406.
Path 327 | total_timesteps 5415.
Path 328 | total_timesteps 5449.
Path 329 | total_timesteps 5466.
Path 330 | total_timesteps 5495.
Path 331 | total_timesteps 5513.
Path 332 | total_timesteps 5522.
Path 333 | total_timesteps 5535.
Path 334 | total_timesteps 5545.
Path 335 | total_timesteps 5558.
Path 336 | total_timesteps 5573.
Path 337 | total_timesteps 5592.
Path 338 | total_timesteps 5610.
Path 339 | total_timesteps 5635.
Path 340 | total_timesteps 5648.
Path 341 | total_timesteps 5664.
Path 342 | total_timesteps 5677.
Path 343 | total_timesteps 5694.
Path 344 | total_timesteps 5712.
Path 345 | total_timesteps 5725.
Path 346 | total_timesteps 5750.
Path 347 | total_timesteps 5767.
Path 348 | total_timesteps 5788.
Path 349 | total_timesteps 5796.
Path 350 | total_timesteps 5821.
Path 351 | total_timesteps 5832.
Path 352 | total_timesteps 5849.
Path 353 | total_timesteps 5863.
Path 354 | total_timesteps 5879.
Path 355 | total_timesteps 5890.
Path 356 | total_timesteps 5902.
Path 357 | total_timesteps 5920.
Path 358 | total_timesteps 5936.
Path 359 | total_timesteps 5950.
Path 360 | total_timesteps 5964.
Path 361 | total_timesteps 5973.
Path 362 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.44    |
| Iteration     | 12       |
| MaximumReturn | 8.18     |
| MinimumReturn | -18.7    |
| TotalSamples  | 56118    |
----------------------------
itr #13 | 
Fitting dynamics.
Validation loss = 0.010878263972699642
Validation loss = 0.010350662283599377
Validation loss = 0.01105632446706295
Validation loss = 0.010752280242741108
Validation loss = 0.010616636835038662
Validation loss = 0.010854276828467846
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 37.
Path 4 | total_timesteps 50.
Path 5 | total_timesteps 62.
Path 6 | total_timesteps 96.
Path 7 | total_timesteps 110.
Path 8 | total_timesteps 121.
Path 9 | total_timesteps 137.
Path 10 | total_timesteps 152.
Path 11 | total_timesteps 173.
Path 12 | total_timesteps 187.
Path 13 | total_timesteps 199.
Path 14 | total_timesteps 215.
Path 15 | total_timesteps 230.
Path 16 | total_timesteps 239.
Path 17 | total_timesteps 252.
Path 18 | total_timesteps 265.
Path 19 | total_timesteps 287.
Path 20 | total_timesteps 302.
Path 21 | total_timesteps 314.
Path 22 | total_timesteps 328.
Path 23 | total_timesteps 354.
Path 24 | total_timesteps 385.
Path 25 | total_timesteps 403.
Path 26 | total_timesteps 415.
Path 27 | total_timesteps 428.
Path 28 | total_timesteps 441.
Path 29 | total_timesteps 451.
Path 30 | total_timesteps 471.
Path 31 | total_timesteps 487.
Path 32 | total_timesteps 515.
Path 33 | total_timesteps 546.
Path 34 | total_timesteps 563.
Path 35 | total_timesteps 585.
Path 36 | total_timesteps 598.
Path 37 | total_timesteps 609.
Path 38 | total_timesteps 626.
Path 39 | total_timesteps 640.
Path 40 | total_timesteps 655.
Path 41 | total_timesteps 671.
Path 42 | total_timesteps 678.
Path 43 | total_timesteps 696.
Path 44 | total_timesteps 719.
Path 45 | total_timesteps 733.
Path 46 | total_timesteps 749.
Path 47 | total_timesteps 774.
Path 48 | total_timesteps 789.
Path 49 | total_timesteps 805.
Path 50 | total_timesteps 824.
Path 51 | total_timesteps 840.
Path 52 | total_timesteps 852.
Path 53 | total_timesteps 871.
Path 54 | total_timesteps 887.
Path 55 | total_timesteps 900.
Path 56 | total_timesteps 924.
Path 57 | total_timesteps 945.
Path 58 | total_timesteps 971.
Path 59 | total_timesteps 985.
Path 60 | total_timesteps 998.
Path 61 | total_timesteps 1008.
Path 62 | total_timesteps 1025.
Path 63 | total_timesteps 1041.
Path 64 | total_timesteps 1064.
Path 65 | total_timesteps 1083.
Path 66 | total_timesteps 1115.
Path 67 | total_timesteps 1128.
Path 68 | total_timesteps 1146.
Path 69 | total_timesteps 1164.
Path 70 | total_timesteps 1203.
Path 71 | total_timesteps 1215.
Path 72 | total_timesteps 1233.
Path 73 | total_timesteps 1244.
Path 74 | total_timesteps 1258.
Path 75 | total_timesteps 1269.
Path 76 | total_timesteps 1281.
Path 77 | total_timesteps 1294.
Path 78 | total_timesteps 1301.
Path 79 | total_timesteps 1323.
Path 80 | total_timesteps 1340.
Path 81 | total_timesteps 1357.
Path 82 | total_timesteps 1395.
Path 83 | total_timesteps 1413.
Path 84 | total_timesteps 1440.
Path 85 | total_timesteps 1456.
Path 86 | total_timesteps 1483.
Path 87 | total_timesteps 1499.
Path 88 | total_timesteps 1517.
Path 89 | total_timesteps 1532.
Path 90 | total_timesteps 1557.
Path 91 | total_timesteps 1574.
Path 92 | total_timesteps 1587.
Path 93 | total_timesteps 1606.
Path 94 | total_timesteps 1621.
Path 95 | total_timesteps 1632.
Path 96 | total_timesteps 1641.
Path 97 | total_timesteps 1661.
Path 98 | total_timesteps 1676.
Path 99 | total_timesteps 1693.
Path 100 | total_timesteps 1713.
Path 101 | total_timesteps 1723.
Path 102 | total_timesteps 1741.
Path 103 | total_timesteps 1756.
Path 104 | total_timesteps 1771.
Path 105 | total_timesteps 1796.
Path 106 | total_timesteps 1820.
Path 107 | total_timesteps 1834.
Path 108 | total_timesteps 1849.
Path 109 | total_timesteps 1866.
Path 110 | total_timesteps 1883.
Path 111 | total_timesteps 1898.
Path 112 | total_timesteps 1911.
Path 113 | total_timesteps 1920.
Path 114 | total_timesteps 1937.
Path 115 | total_timesteps 1957.
Path 116 | total_timesteps 1971.
Path 117 | total_timesteps 1985.
Path 118 | total_timesteps 2006.
Path 119 | total_timesteps 2019.
Path 120 | total_timesteps 2038.
Path 121 | total_timesteps 2059.
Path 122 | total_timesteps 2068.
Path 123 | total_timesteps 2088.
Path 124 | total_timesteps 2111.
Path 125 | total_timesteps 2124.
Path 126 | total_timesteps 2138.
Path 127 | total_timesteps 2149.
Path 128 | total_timesteps 2166.
Path 129 | total_timesteps 2180.
Path 130 | total_timesteps 2197.
Path 131 | total_timesteps 2211.
Path 132 | total_timesteps 2236.
Path 133 | total_timesteps 2256.
Path 134 | total_timesteps 2275.
Path 135 | total_timesteps 2291.
Path 136 | total_timesteps 2301.
Path 137 | total_timesteps 2318.
Path 138 | total_timesteps 2334.
Path 139 | total_timesteps 2357.
Path 140 | total_timesteps 2371.
Path 141 | total_timesteps 2392.
Path 142 | total_timesteps 2411.
Path 143 | total_timesteps 2430.
Path 144 | total_timesteps 2465.
Path 145 | total_timesteps 2477.
Path 146 | total_timesteps 2496.
Path 147 | total_timesteps 2509.
Path 148 | total_timesteps 2522.
Path 149 | total_timesteps 2555.
Path 150 | total_timesteps 2568.
Path 151 | total_timesteps 2585.
Path 152 | total_timesteps 2601.
Path 153 | total_timesteps 2618.
Path 154 | total_timesteps 2627.
Path 155 | total_timesteps 2638.
Path 156 | total_timesteps 2665.
Path 157 | total_timesteps 2696.
Path 158 | total_timesteps 2714.
Path 159 | total_timesteps 2725.
Path 160 | total_timesteps 2741.
Path 161 | total_timesteps 2755.
Path 162 | total_timesteps 2773.
Path 163 | total_timesteps 2784.
Path 164 | total_timesteps 2799.
Path 165 | total_timesteps 2812.
Path 166 | total_timesteps 2823.
Path 167 | total_timesteps 2841.
Path 168 | total_timesteps 2864.
Path 169 | total_timesteps 2886.
Path 170 | total_timesteps 2900.
Path 171 | total_timesteps 2929.
Path 172 | total_timesteps 2946.
Path 173 | total_timesteps 2971.
Path 174 | total_timesteps 3000.
Path 175 | total_timesteps 3012.
Path 176 | total_timesteps 3021.
Path 177 | total_timesteps 3043.
Path 178 | total_timesteps 3057.
Path 179 | total_timesteps 3079.
Path 180 | total_timesteps 3103.
Path 181 | total_timesteps 3115.
Path 182 | total_timesteps 3136.
Path 183 | total_timesteps 3146.
Path 184 | total_timesteps 3158.
Path 185 | total_timesteps 3167.
Path 186 | total_timesteps 3179.
Path 187 | total_timesteps 3197.
Path 188 | total_timesteps 3215.
Path 189 | total_timesteps 3235.
Path 190 | total_timesteps 3251.
Path 191 | total_timesteps 3279.
Path 192 | total_timesteps 3293.
Path 193 | total_timesteps 3306.
Path 194 | total_timesteps 3322.
Path 195 | total_timesteps 3341.
Path 196 | total_timesteps 3353.
Path 197 | total_timesteps 3374.
Path 198 | total_timesteps 3403.
Path 199 | total_timesteps 3425.
Path 200 | total_timesteps 3442.
Path 201 | total_timesteps 3454.
Path 202 | total_timesteps 3479.
Path 203 | total_timesteps 3499.
Path 204 | total_timesteps 3509.
Path 205 | total_timesteps 3523.
Path 206 | total_timesteps 3541.
Path 207 | total_timesteps 3563.
Path 208 | total_timesteps 3578.
Path 209 | total_timesteps 3601.
Path 210 | total_timesteps 3616.
Path 211 | total_timesteps 3625.
Path 212 | total_timesteps 3641.
Path 213 | total_timesteps 3658.
Path 214 | total_timesteps 3676.
Path 215 | total_timesteps 3686.
Path 216 | total_timesteps 3715.
Path 217 | total_timesteps 3727.
Path 218 | total_timesteps 3746.
Path 219 | total_timesteps 3754.
Path 220 | total_timesteps 3768.
Path 221 | total_timesteps 3776.
Path 222 | total_timesteps 3789.
Path 223 | total_timesteps 3802.
Path 224 | total_timesteps 3821.
Path 225 | total_timesteps 3848.
Path 226 | total_timesteps 3863.
Path 227 | total_timesteps 3883.
Path 228 | total_timesteps 3892.
Path 229 | total_timesteps 3909.
Path 230 | total_timesteps 3920.
Path 231 | total_timesteps 3943.
Path 232 | total_timesteps 3959.
Path 233 | total_timesteps 3967.
Path 234 | total_timesteps 3975.
Path 235 | total_timesteps 3989.
Path 236 | total_timesteps 4018.
Path 237 | total_timesteps 4031.
Path 238 | total_timesteps 4044.
Path 239 | total_timesteps 4081.
Path 240 | total_timesteps 4095.
Path 241 | total_timesteps 4110.
Path 242 | total_timesteps 4136.
Path 243 | total_timesteps 4161.
Path 244 | total_timesteps 4176.
Path 245 | total_timesteps 4184.
Path 246 | total_timesteps 4204.
Path 247 | total_timesteps 4220.
Path 248 | total_timesteps 4235.
Path 249 | total_timesteps 4249.
Path 250 | total_timesteps 4262.
Path 251 | total_timesteps 4276.
Path 252 | total_timesteps 4294.
Path 253 | total_timesteps 4308.
Path 254 | total_timesteps 4318.
Path 255 | total_timesteps 4334.
Path 256 | total_timesteps 4346.
Path 257 | total_timesteps 4368.
Path 258 | total_timesteps 4382.
Path 259 | total_timesteps 4393.
Path 260 | total_timesteps 4410.
Path 261 | total_timesteps 4425.
Path 262 | total_timesteps 4437.
Path 263 | total_timesteps 4455.
Path 264 | total_timesteps 4470.
Path 265 | total_timesteps 4480.
Path 266 | total_timesteps 4494.
Path 267 | total_timesteps 4516.
Path 268 | total_timesteps 4546.
Path 269 | total_timesteps 4556.
Path 270 | total_timesteps 4569.
Path 271 | total_timesteps 4581.
Path 272 | total_timesteps 4610.
Path 273 | total_timesteps 4628.
Path 274 | total_timesteps 4650.
Path 275 | total_timesteps 4661.
Path 276 | total_timesteps 4677.
Path 277 | total_timesteps 4690.
Path 278 | total_timesteps 4704.
Path 279 | total_timesteps 4724.
Path 280 | total_timesteps 4738.
Path 281 | total_timesteps 4747.
Path 282 | total_timesteps 4757.
Path 283 | total_timesteps 4769.
Path 284 | total_timesteps 4788.
Path 285 | total_timesteps 4798.
Path 286 | total_timesteps 4809.
Path 287 | total_timesteps 4820.
Path 288 | total_timesteps 4838.
Path 289 | total_timesteps 4852.
Path 290 | total_timesteps 4873.
Path 291 | total_timesteps 4886.
Path 292 | total_timesteps 4899.
Path 293 | total_timesteps 4918.
Path 294 | total_timesteps 4933.
Path 295 | total_timesteps 4946.
Path 296 | total_timesteps 4962.
Path 297 | total_timesteps 4977.
Path 298 | total_timesteps 5013.
Path 299 | total_timesteps 5032.
Path 300 | total_timesteps 5044.
Path 301 | total_timesteps 5054.
Path 302 | total_timesteps 5072.
Path 303 | total_timesteps 5094.
Path 304 | total_timesteps 5141.
Path 305 | total_timesteps 5163.
Path 306 | total_timesteps 5178.
Path 307 | total_timesteps 5198.
Path 308 | total_timesteps 5213.
Path 309 | total_timesteps 5226.
Path 310 | total_timesteps 5244.
Path 311 | total_timesteps 5266.
Path 312 | total_timesteps 5276.
Path 313 | total_timesteps 5287.
Path 314 | total_timesteps 5299.
Path 315 | total_timesteps 5312.
Path 316 | total_timesteps 5336.
Path 317 | total_timesteps 5344.
Path 318 | total_timesteps 5359.
Path 319 | total_timesteps 5368.
Path 320 | total_timesteps 5386.
Path 321 | total_timesteps 5396.
Path 322 | total_timesteps 5411.
Path 323 | total_timesteps 5426.
Path 324 | total_timesteps 5438.
Path 325 | total_timesteps 5448.
Path 326 | total_timesteps 5464.
Path 327 | total_timesteps 5477.
Path 328 | total_timesteps 5490.
Path 329 | total_timesteps 5528.
Path 330 | total_timesteps 5540.
Path 331 | total_timesteps 5552.
Path 332 | total_timesteps 5571.
Path 333 | total_timesteps 5587.
Path 334 | total_timesteps 5597.
Path 335 | total_timesteps 5622.
Path 336 | total_timesteps 5633.
Path 337 | total_timesteps 5660.
Path 338 | total_timesteps 5672.
Path 339 | total_timesteps 5682.
Path 340 | total_timesteps 5694.
Path 341 | total_timesteps 5705.
Path 342 | total_timesteps 5716.
Path 343 | total_timesteps 5731.
Path 344 | total_timesteps 5744.
Path 345 | total_timesteps 5761.
Path 346 | total_timesteps 5778.
Path 347 | total_timesteps 5795.
Path 348 | total_timesteps 5812.
Path 349 | total_timesteps 5826.
Path 350 | total_timesteps 5851.
Path 351 | total_timesteps 5862.
Path 352 | total_timesteps 5876.
Path 353 | total_timesteps 5921.
Path 354 | total_timesteps 5948.
Path 355 | total_timesteps 5977.
Path 356 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.32    |
| Iteration     | 13       |
| MaximumReturn | 12.3     |
| MinimumReturn | -21      |
| TotalSamples  | 60121    |
----------------------------
itr #14 | 
Fitting dynamics.
Validation loss = 0.010969346389174461
Validation loss = 0.009906698018312454
Validation loss = 0.01067641843110323
Validation loss = 0.010194028727710247
Validation loss = 0.009862086735665798
Validation loss = 0.009674200788140297
Validation loss = 0.009554628282785416
Validation loss = 0.009285238571465015
Validation loss = 0.010546078905463219
Validation loss = 0.009798869490623474
Validation loss = 0.009266775101423264
Validation loss = 0.00951398815959692
Validation loss = 0.00920499674975872
Validation loss = 0.009205818176269531
Validation loss = 0.009470177814364433
Validation loss = 0.009657138958573341
Validation loss = 0.009606057778000832
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 49.
Path 4 | total_timesteps 68.
Path 5 | total_timesteps 87.
Path 6 | total_timesteps 98.
Path 7 | total_timesteps 108.
Path 8 | total_timesteps 131.
Path 9 | total_timesteps 145.
Path 10 | total_timesteps 167.
Path 11 | total_timesteps 179.
Path 12 | total_timesteps 202.
Path 13 | total_timesteps 217.
Path 14 | total_timesteps 253.
Path 15 | total_timesteps 264.
Path 16 | total_timesteps 277.
Path 17 | total_timesteps 295.
Path 18 | total_timesteps 332.
Path 19 | total_timesteps 350.
Path 20 | total_timesteps 360.
Path 21 | total_timesteps 378.
Path 22 | total_timesteps 391.
Path 23 | total_timesteps 407.
Path 24 | total_timesteps 414.
Path 25 | total_timesteps 428.
Path 26 | total_timesteps 444.
Path 27 | total_timesteps 458.
Path 28 | total_timesteps 479.
Path 29 | total_timesteps 497.
Path 30 | total_timesteps 526.
Path 31 | total_timesteps 538.
Path 32 | total_timesteps 557.
Path 33 | total_timesteps 577.
Path 34 | total_timesteps 611.
Path 35 | total_timesteps 627.
Path 36 | total_timesteps 638.
Path 37 | total_timesteps 650.
Path 38 | total_timesteps 663.
Path 39 | total_timesteps 672.
Path 40 | total_timesteps 745.
Path 41 | total_timesteps 770.
Path 42 | total_timesteps 787.
Path 43 | total_timesteps 803.
Path 44 | total_timesteps 809.
Path 45 | total_timesteps 833.
Path 46 | total_timesteps 849.
Path 47 | total_timesteps 864.
Path 48 | total_timesteps 881.
Path 49 | total_timesteps 897.
Path 50 | total_timesteps 908.
Path 51 | total_timesteps 919.
Path 52 | total_timesteps 930.
Path 53 | total_timesteps 946.
Path 54 | total_timesteps 962.
Path 55 | total_timesteps 973.
Path 56 | total_timesteps 992.
Path 57 | total_timesteps 1002.
Path 58 | total_timesteps 1018.
Path 59 | total_timesteps 1041.
Path 60 | total_timesteps 1055.
Path 61 | total_timesteps 1076.
Path 62 | total_timesteps 1110.
Path 63 | total_timesteps 1124.
Path 64 | total_timesteps 1138.
Path 65 | total_timesteps 1149.
Path 66 | total_timesteps 1169.
Path 67 | total_timesteps 1192.
Path 68 | total_timesteps 1213.
Path 69 | total_timesteps 1250.
Path 70 | total_timesteps 1276.
Path 71 | total_timesteps 1286.
Path 72 | total_timesteps 1304.
Path 73 | total_timesteps 1322.
Path 74 | total_timesteps 1346.
Path 75 | total_timesteps 1364.
Path 76 | total_timesteps 1392.
Path 77 | total_timesteps 1418.
Path 78 | total_timesteps 1439.
Path 79 | total_timesteps 1461.
Path 80 | total_timesteps 1473.
Path 81 | total_timesteps 1487.
Path 82 | total_timesteps 1505.
Path 83 | total_timesteps 1522.
Path 84 | total_timesteps 1548.
Path 85 | total_timesteps 1559.
Path 86 | total_timesteps 1571.
Path 87 | total_timesteps 1583.
Path 88 | total_timesteps 1594.
Path 89 | total_timesteps 1611.
Path 90 | total_timesteps 1628.
Path 91 | total_timesteps 1642.
Path 92 | total_timesteps 1660.
Path 93 | total_timesteps 1673.
Path 94 | total_timesteps 1683.
Path 95 | total_timesteps 1705.
Path 96 | total_timesteps 1721.
Path 97 | total_timesteps 1731.
Path 98 | total_timesteps 1746.
Path 99 | total_timesteps 1764.
Path 100 | total_timesteps 1798.
Path 101 | total_timesteps 1815.
Path 102 | total_timesteps 1831.
Path 103 | total_timesteps 1850.
Path 104 | total_timesteps 1860.
Path 105 | total_timesteps 1869.
Path 106 | total_timesteps 1885.
Path 107 | total_timesteps 1908.
Path 108 | total_timesteps 1925.
Path 109 | total_timesteps 1940.
Path 110 | total_timesteps 1950.
Path 111 | total_timesteps 1980.
Path 112 | total_timesteps 1993.
Path 113 | total_timesteps 2010.
Path 114 | total_timesteps 2018.
Path 115 | total_timesteps 2031.
Path 116 | total_timesteps 2048.
Path 117 | total_timesteps 2065.
Path 118 | total_timesteps 2094.
Path 119 | total_timesteps 2107.
Path 120 | total_timesteps 2126.
Path 121 | total_timesteps 2138.
Path 122 | total_timesteps 2155.
Path 123 | total_timesteps 2170.
Path 124 | total_timesteps 2186.
Path 125 | total_timesteps 2196.
Path 126 | total_timesteps 2227.
Path 127 | total_timesteps 2239.
Path 128 | total_timesteps 2251.
Path 129 | total_timesteps 2285.
Path 130 | total_timesteps 2296.
Path 131 | total_timesteps 2328.
Path 132 | total_timesteps 2342.
Path 133 | total_timesteps 2358.
Path 134 | total_timesteps 2378.
Path 135 | total_timesteps 2398.
Path 136 | total_timesteps 2407.
Path 137 | total_timesteps 2428.
Path 138 | total_timesteps 2441.
Path 139 | total_timesteps 2457.
Path 140 | total_timesteps 2475.
Path 141 | total_timesteps 2488.
Path 142 | total_timesteps 2501.
Path 143 | total_timesteps 2512.
Path 144 | total_timesteps 2521.
Path 145 | total_timesteps 2534.
Path 146 | total_timesteps 2547.
Path 147 | total_timesteps 2563.
Path 148 | total_timesteps 2588.
Path 149 | total_timesteps 2601.
Path 150 | total_timesteps 2612.
Path 151 | total_timesteps 2632.
Path 152 | total_timesteps 2657.
Path 153 | total_timesteps 2681.
Path 154 | total_timesteps 2698.
Path 155 | total_timesteps 2713.
Path 156 | total_timesteps 2724.
Path 157 | total_timesteps 2752.
Path 158 | total_timesteps 2762.
Path 159 | total_timesteps 2814.
Path 160 | total_timesteps 2827.
Path 161 | total_timesteps 2856.
Path 162 | total_timesteps 2875.
Path 163 | total_timesteps 2884.
Path 164 | total_timesteps 2901.
Path 165 | total_timesteps 2912.
Path 166 | total_timesteps 2928.
Path 167 | total_timesteps 2955.
Path 168 | total_timesteps 2971.
Path 169 | total_timesteps 2991.
Path 170 | total_timesteps 3001.
Path 171 | total_timesteps 3024.
Path 172 | total_timesteps 3035.
Path 173 | total_timesteps 3051.
Path 174 | total_timesteps 3064.
Path 175 | total_timesteps 3083.
Path 176 | total_timesteps 3099.
Path 177 | total_timesteps 3113.
Path 178 | total_timesteps 3143.
Path 179 | total_timesteps 3154.
Path 180 | total_timesteps 3172.
Path 181 | total_timesteps 3189.
Path 182 | total_timesteps 3202.
Path 183 | total_timesteps 3217.
Path 184 | total_timesteps 3226.
Path 185 | total_timesteps 3234.
Path 186 | total_timesteps 3253.
Path 187 | total_timesteps 3287.
Path 188 | total_timesteps 3300.
Path 189 | total_timesteps 3315.
Path 190 | total_timesteps 3331.
Path 191 | total_timesteps 3344.
Path 192 | total_timesteps 3359.
Path 193 | total_timesteps 3368.
Path 194 | total_timesteps 3376.
Path 195 | total_timesteps 3389.
Path 196 | total_timesteps 3409.
Path 197 | total_timesteps 3421.
Path 198 | total_timesteps 3437.
Path 199 | total_timesteps 3464.
Path 200 | total_timesteps 3479.
Path 201 | total_timesteps 3493.
Path 202 | total_timesteps 3505.
Path 203 | total_timesteps 3519.
Path 204 | total_timesteps 3532.
Path 205 | total_timesteps 3549.
Path 206 | total_timesteps 3561.
Path 207 | total_timesteps 3586.
Path 208 | total_timesteps 3604.
Path 209 | total_timesteps 3631.
Path 210 | total_timesteps 3645.
Path 211 | total_timesteps 3664.
Path 212 | total_timesteps 3679.
Path 213 | total_timesteps 3695.
Path 214 | total_timesteps 3706.
Path 215 | total_timesteps 3717.
Path 216 | total_timesteps 3727.
Path 217 | total_timesteps 3737.
Path 218 | total_timesteps 3765.
Path 219 | total_timesteps 3781.
Path 220 | total_timesteps 3792.
Path 221 | total_timesteps 3807.
Path 222 | total_timesteps 3832.
Path 223 | total_timesteps 3845.
Path 224 | total_timesteps 3869.
Path 225 | total_timesteps 3889.
Path 226 | total_timesteps 3901.
Path 227 | total_timesteps 3913.
Path 228 | total_timesteps 3936.
Path 229 | total_timesteps 3955.
Path 230 | total_timesteps 3971.
Path 231 | total_timesteps 3988.
Path 232 | total_timesteps 4003.
Path 233 | total_timesteps 4026.
Path 234 | total_timesteps 4042.
Path 235 | total_timesteps 4057.
Path 236 | total_timesteps 4071.
Path 237 | total_timesteps 4101.
Path 238 | total_timesteps 4114.
Path 239 | total_timesteps 4136.
Path 240 | total_timesteps 4174.
Path 241 | total_timesteps 4194.
Path 242 | total_timesteps 4232.
Path 243 | total_timesteps 4247.
Path 244 | total_timesteps 4262.
Path 245 | total_timesteps 4284.
Path 246 | total_timesteps 4294.
Path 247 | total_timesteps 4313.
Path 248 | total_timesteps 4335.
Path 249 | total_timesteps 4350.
Path 250 | total_timesteps 4367.
Path 251 | total_timesteps 4379.
Path 252 | total_timesteps 4404.
Path 253 | total_timesteps 4421.
Path 254 | total_timesteps 4434.
Path 255 | total_timesteps 4448.
Path 256 | total_timesteps 4463.
Path 257 | total_timesteps 4477.
Path 258 | total_timesteps 4488.
Path 259 | total_timesteps 4499.
Path 260 | total_timesteps 4517.
Path 261 | total_timesteps 4539.
Path 262 | total_timesteps 4549.
Path 263 | total_timesteps 4566.
Path 264 | total_timesteps 4581.
Path 265 | total_timesteps 4591.
Path 266 | total_timesteps 4606.
Path 267 | total_timesteps 4619.
Path 268 | total_timesteps 4637.
Path 269 | total_timesteps 4656.
Path 270 | total_timesteps 4669.
Path 271 | total_timesteps 4680.
Path 272 | total_timesteps 4696.
Path 273 | total_timesteps 4716.
Path 274 | total_timesteps 4726.
Path 275 | total_timesteps 4746.
Path 276 | total_timesteps 4759.
Path 277 | total_timesteps 4770.
Path 278 | total_timesteps 4781.
Path 279 | total_timesteps 4799.
Path 280 | total_timesteps 4811.
Path 281 | total_timesteps 4826.
Path 282 | total_timesteps 4838.
Path 283 | total_timesteps 4849.
Path 284 | total_timesteps 4865.
Path 285 | total_timesteps 4881.
Path 286 | total_timesteps 4890.
Path 287 | total_timesteps 4897.
Path 288 | total_timesteps 4911.
Path 289 | total_timesteps 4923.
Path 290 | total_timesteps 4933.
Path 291 | total_timesteps 4946.
Path 292 | total_timesteps 4957.
Path 293 | total_timesteps 4986.
Path 294 | total_timesteps 5007.
Path 295 | total_timesteps 5025.
Path 296 | total_timesteps 5043.
Path 297 | total_timesteps 5060.
Path 298 | total_timesteps 5083.
Path 299 | total_timesteps 5099.
Path 300 | total_timesteps 5116.
Path 301 | total_timesteps 5135.
Path 302 | total_timesteps 5153.
Path 303 | total_timesteps 5161.
Path 304 | total_timesteps 5178.
Path 305 | total_timesteps 5189.
Path 306 | total_timesteps 5208.
Path 307 | total_timesteps 5218.
Path 308 | total_timesteps 5264.
Path 309 | total_timesteps 5276.
Path 310 | total_timesteps 5313.
Path 311 | total_timesteps 5334.
Path 312 | total_timesteps 5344.
Path 313 | total_timesteps 5357.
Path 314 | total_timesteps 5369.
Path 315 | total_timesteps 5388.
Path 316 | total_timesteps 5404.
Path 317 | total_timesteps 5413.
Path 318 | total_timesteps 5428.
Path 319 | total_timesteps 5444.
Path 320 | total_timesteps 5469.
Path 321 | total_timesteps 5488.
Path 322 | total_timesteps 5521.
Path 323 | total_timesteps 5555.
Path 324 | total_timesteps 5580.
Path 325 | total_timesteps 5592.
Path 326 | total_timesteps 5604.
Path 327 | total_timesteps 5622.
Path 328 | total_timesteps 5640.
Path 329 | total_timesteps 5663.
Path 330 | total_timesteps 5675.
Path 331 | total_timesteps 5692.
Path 332 | total_timesteps 5706.
Path 333 | total_timesteps 5717.
Path 334 | total_timesteps 5739.
Path 335 | total_timesteps 5753.
Path 336 | total_timesteps 5774.
Path 337 | total_timesteps 5804.
Path 338 | total_timesteps 5823.
Path 339 | total_timesteps 5852.
Path 340 | total_timesteps 5863.
Path 341 | total_timesteps 5871.
Path 342 | total_timesteps 5891.
Path 343 | total_timesteps 5906.
Path 344 | total_timesteps 5915.
Path 345 | total_timesteps 5930.
Path 346 | total_timesteps 5969.
Path 347 | total_timesteps 5984.
Path 348 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.99    |
| Iteration     | 14       |
| MaximumReturn | 14.4     |
| MinimumReturn | -18.8    |
| TotalSamples  | 64131    |
----------------------------
itr #15 | 
Fitting dynamics.
Validation loss = 0.008905380964279175
Validation loss = 0.009789271280169487
Validation loss = 0.008884962648153305
Validation loss = 0.008598393760621548
Validation loss = 0.008614052087068558
Validation loss = 0.00928354263305664
Validation loss = 0.009103301912546158
Validation loss = 0.008803129196166992
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 48.
Path 4 | total_timesteps 64.
Path 5 | total_timesteps 84.
Path 6 | total_timesteps 102.
Path 7 | total_timesteps 110.
Path 8 | total_timesteps 129.
Path 9 | total_timesteps 151.
Path 10 | total_timesteps 163.
Path 11 | total_timesteps 180.
Path 12 | total_timesteps 193.
Path 13 | total_timesteps 217.
Path 14 | total_timesteps 231.
Path 15 | total_timesteps 249.
Path 16 | total_timesteps 261.
Path 17 | total_timesteps 270.
Path 18 | total_timesteps 279.
Path 19 | total_timesteps 300.
Path 20 | total_timesteps 320.
Path 21 | total_timesteps 347.
Path 22 | total_timesteps 376.
Path 23 | total_timesteps 389.
Path 24 | total_timesteps 396.
Path 25 | total_timesteps 406.
Path 26 | total_timesteps 419.
Path 27 | total_timesteps 436.
Path 28 | total_timesteps 445.
Path 29 | total_timesteps 454.
Path 30 | total_timesteps 465.
Path 31 | total_timesteps 488.
Path 32 | total_timesteps 515.
Path 33 | total_timesteps 531.
Path 34 | total_timesteps 550.
Path 35 | total_timesteps 563.
Path 36 | total_timesteps 584.
Path 37 | total_timesteps 607.
Path 38 | total_timesteps 619.
Path 39 | total_timesteps 634.
Path 40 | total_timesteps 647.
Path 41 | total_timesteps 665.
Path 42 | total_timesteps 677.
Path 43 | total_timesteps 695.
Path 44 | total_timesteps 705.
Path 45 | total_timesteps 717.
Path 46 | total_timesteps 732.
Path 47 | total_timesteps 747.
Path 48 | total_timesteps 758.
Path 49 | total_timesteps 766.
Path 50 | total_timesteps 783.
Path 51 | total_timesteps 805.
Path 52 | total_timesteps 820.
Path 53 | total_timesteps 836.
Path 54 | total_timesteps 861.
Path 55 | total_timesteps 883.
Path 56 | total_timesteps 891.
Path 57 | total_timesteps 902.
Path 58 | total_timesteps 917.
Path 59 | total_timesteps 930.
Path 60 | total_timesteps 949.
Path 61 | total_timesteps 963.
Path 62 | total_timesteps 979.
Path 63 | total_timesteps 992.
Path 64 | total_timesteps 1015.
Path 65 | total_timesteps 1025.
Path 66 | total_timesteps 1037.
Path 67 | total_timesteps 1060.
Path 68 | total_timesteps 1076.
Path 69 | total_timesteps 1087.
Path 70 | total_timesteps 1097.
Path 71 | total_timesteps 1115.
Path 72 | total_timesteps 1128.
Path 73 | total_timesteps 1143.
Path 74 | total_timesteps 1166.
Path 75 | total_timesteps 1178.
Path 76 | total_timesteps 1195.
Path 77 | total_timesteps 1208.
Path 78 | total_timesteps 1224.
Path 79 | total_timesteps 1232.
Path 80 | total_timesteps 1247.
Path 81 | total_timesteps 1260.
Path 82 | total_timesteps 1288.
Path 83 | total_timesteps 1304.
Path 84 | total_timesteps 1320.
Path 85 | total_timesteps 1330.
Path 86 | total_timesteps 1341.
Path 87 | total_timesteps 1358.
Path 88 | total_timesteps 1375.
Path 89 | total_timesteps 1395.
Path 90 | total_timesteps 1409.
Path 91 | total_timesteps 1418.
Path 92 | total_timesteps 1442.
Path 93 | total_timesteps 1453.
Path 94 | total_timesteps 1472.
Path 95 | total_timesteps 1490.
Path 96 | total_timesteps 1499.
Path 97 | total_timesteps 1521.
Path 98 | total_timesteps 1540.
Path 99 | total_timesteps 1552.
Path 100 | total_timesteps 1569.
Path 101 | total_timesteps 1580.
Path 102 | total_timesteps 1594.
Path 103 | total_timesteps 1613.
Path 104 | total_timesteps 1623.
Path 105 | total_timesteps 1638.
Path 106 | total_timesteps 1656.
Path 107 | total_timesteps 1676.
Path 108 | total_timesteps 1692.
Path 109 | total_timesteps 1705.
Path 110 | total_timesteps 1721.
Path 111 | total_timesteps 1740.
Path 112 | total_timesteps 1761.
Path 113 | total_timesteps 1775.
Path 114 | total_timesteps 1797.
Path 115 | total_timesteps 1813.
Path 116 | total_timesteps 1831.
Path 117 | total_timesteps 1851.
Path 118 | total_timesteps 1869.
Path 119 | total_timesteps 1883.
Path 120 | total_timesteps 1903.
Path 121 | total_timesteps 1921.
Path 122 | total_timesteps 1937.
Path 123 | total_timesteps 1951.
Path 124 | total_timesteps 1970.
Path 125 | total_timesteps 1990.
Path 126 | total_timesteps 2013.
Path 127 | total_timesteps 2041.
Path 128 | total_timesteps 2048.
Path 129 | total_timesteps 2066.
Path 130 | total_timesteps 2088.
Path 131 | total_timesteps 2102.
Path 132 | total_timesteps 2119.
Path 133 | total_timesteps 2133.
Path 134 | total_timesteps 2150.
Path 135 | total_timesteps 2166.
Path 136 | total_timesteps 2186.
Path 137 | total_timesteps 2198.
Path 138 | total_timesteps 2217.
Path 139 | total_timesteps 2231.
Path 140 | total_timesteps 2242.
Path 141 | total_timesteps 2259.
Path 142 | total_timesteps 2274.
Path 143 | total_timesteps 2292.
Path 144 | total_timesteps 2302.
Path 145 | total_timesteps 2315.
Path 146 | total_timesteps 2324.
Path 147 | total_timesteps 2343.
Path 148 | total_timesteps 2354.
Path 149 | total_timesteps 2369.
Path 150 | total_timesteps 2389.
Path 151 | total_timesteps 2401.
Path 152 | total_timesteps 2412.
Path 153 | total_timesteps 2422.
Path 154 | total_timesteps 2434.
Path 155 | total_timesteps 2450.
Path 156 | total_timesteps 2468.
Path 157 | total_timesteps 2477.
Path 158 | total_timesteps 2495.
Path 159 | total_timesteps 2507.
Path 160 | total_timesteps 2528.
Path 161 | total_timesteps 2542.
Path 162 | total_timesteps 2556.
Path 163 | total_timesteps 2574.
Path 164 | total_timesteps 2584.
Path 165 | total_timesteps 2596.
Path 166 | total_timesteps 2607.
Path 167 | total_timesteps 2619.
Path 168 | total_timesteps 2646.
Path 169 | total_timesteps 2664.
Path 170 | total_timesteps 2679.
Path 171 | total_timesteps 2693.
Path 172 | total_timesteps 2712.
Path 173 | total_timesteps 2729.
Path 174 | total_timesteps 2740.
Path 175 | total_timesteps 2753.
Path 176 | total_timesteps 2769.
Path 177 | total_timesteps 2784.
Path 178 | total_timesteps 2800.
Path 179 | total_timesteps 2813.
Path 180 | total_timesteps 2842.
Path 181 | total_timesteps 2859.
Path 182 | total_timesteps 2874.
Path 183 | total_timesteps 2889.
Path 184 | total_timesteps 2907.
Path 185 | total_timesteps 2926.
Path 186 | total_timesteps 2937.
Path 187 | total_timesteps 2961.
Path 188 | total_timesteps 2972.
Path 189 | total_timesteps 2981.
Path 190 | total_timesteps 2988.
Path 191 | total_timesteps 2995.
Path 192 | total_timesteps 3015.
Path 193 | total_timesteps 3037.
Path 194 | total_timesteps 3064.
Path 195 | total_timesteps 3079.
Path 196 | total_timesteps 3097.
Path 197 | total_timesteps 3111.
Path 198 | total_timesteps 3128.
Path 199 | total_timesteps 3147.
Path 200 | total_timesteps 3163.
Path 201 | total_timesteps 3176.
Path 202 | total_timesteps 3190.
Path 203 | total_timesteps 3199.
Path 204 | total_timesteps 3211.
Path 205 | total_timesteps 3228.
Path 206 | total_timesteps 3243.
Path 207 | total_timesteps 3256.
Path 208 | total_timesteps 3278.
Path 209 | total_timesteps 3295.
Path 210 | total_timesteps 3309.
Path 211 | total_timesteps 3326.
Path 212 | total_timesteps 3342.
Path 213 | total_timesteps 3353.
Path 214 | total_timesteps 3377.
Path 215 | total_timesteps 3392.
Path 216 | total_timesteps 3411.
Path 217 | total_timesteps 3420.
Path 218 | total_timesteps 3433.
Path 219 | total_timesteps 3450.
Path 220 | total_timesteps 3469.
Path 221 | total_timesteps 3486.
Path 222 | total_timesteps 3510.
Path 223 | total_timesteps 3526.
Path 224 | total_timesteps 3538.
Path 225 | total_timesteps 3548.
Path 226 | total_timesteps 3561.
Path 227 | total_timesteps 3582.
Path 228 | total_timesteps 3592.
Path 229 | total_timesteps 3613.
Path 230 | total_timesteps 3623.
Path 231 | total_timesteps 3647.
Path 232 | total_timesteps 3668.
Path 233 | total_timesteps 3688.
Path 234 | total_timesteps 3697.
Path 235 | total_timesteps 3722.
Path 236 | total_timesteps 3735.
Path 237 | total_timesteps 3743.
Path 238 | total_timesteps 3752.
Path 239 | total_timesteps 3764.
Path 240 | total_timesteps 3782.
Path 241 | total_timesteps 3800.
Path 242 | total_timesteps 3824.
Path 243 | total_timesteps 3838.
Path 244 | total_timesteps 3848.
Path 245 | total_timesteps 3869.
Path 246 | total_timesteps 3889.
Path 247 | total_timesteps 3910.
Path 248 | total_timesteps 3924.
Path 249 | total_timesteps 3936.
Path 250 | total_timesteps 3950.
Path 251 | total_timesteps 3981.
Path 252 | total_timesteps 3991.
Path 253 | total_timesteps 4005.
Path 254 | total_timesteps 4014.
Path 255 | total_timesteps 4052.
Path 256 | total_timesteps 4070.
Path 257 | total_timesteps 4080.
Path 258 | total_timesteps 4097.
Path 259 | total_timesteps 4108.
Path 260 | total_timesteps 4123.
Path 261 | total_timesteps 4139.
Path 262 | total_timesteps 4148.
Path 263 | total_timesteps 4162.
Path 264 | total_timesteps 4177.
Path 265 | total_timesteps 4188.
Path 266 | total_timesteps 4200.
Path 267 | total_timesteps 4218.
Path 268 | total_timesteps 4232.
Path 269 | total_timesteps 4243.
Path 270 | total_timesteps 4257.
Path 271 | total_timesteps 4269.
Path 272 | total_timesteps 4284.
Path 273 | total_timesteps 4301.
Path 274 | total_timesteps 4313.
Path 275 | total_timesteps 4328.
Path 276 | total_timesteps 4340.
Path 277 | total_timesteps 4355.
Path 278 | total_timesteps 4365.
Path 279 | total_timesteps 4378.
Path 280 | total_timesteps 4394.
Path 281 | total_timesteps 4410.
Path 282 | total_timesteps 4434.
Path 283 | total_timesteps 4444.
Path 284 | total_timesteps 4472.
Path 285 | total_timesteps 4483.
Path 286 | total_timesteps 4499.
Path 287 | total_timesteps 4511.
Path 288 | total_timesteps 4526.
Path 289 | total_timesteps 4537.
Path 290 | total_timesteps 4558.
Path 291 | total_timesteps 4576.
Path 292 | total_timesteps 4596.
Path 293 | total_timesteps 4609.
Path 294 | total_timesteps 4627.
Path 295 | total_timesteps 4642.
Path 296 | total_timesteps 4656.
Path 297 | total_timesteps 4670.
Path 298 | total_timesteps 4678.
Path 299 | total_timesteps 4690.
Path 300 | total_timesteps 4714.
Path 301 | total_timesteps 4728.
Path 302 | total_timesteps 4743.
Path 303 | total_timesteps 4758.
Path 304 | total_timesteps 4782.
Path 305 | total_timesteps 4794.
Path 306 | total_timesteps 4810.
Path 307 | total_timesteps 4821.
Path 308 | total_timesteps 4842.
Path 309 | total_timesteps 4859.
Path 310 | total_timesteps 4874.
Path 311 | total_timesteps 4890.
Path 312 | total_timesteps 4921.
Path 313 | total_timesteps 4942.
Path 314 | total_timesteps 4962.
Path 315 | total_timesteps 4979.
Path 316 | total_timesteps 5001.
Path 317 | total_timesteps 5014.
Path 318 | total_timesteps 5022.
Path 319 | total_timesteps 5036.
Path 320 | total_timesteps 5054.
Path 321 | total_timesteps 5069.
Path 322 | total_timesteps 5089.
Path 323 | total_timesteps 5107.
Path 324 | total_timesteps 5118.
Path 325 | total_timesteps 5132.
Path 326 | total_timesteps 5153.
Path 327 | total_timesteps 5167.
Path 328 | total_timesteps 5184.
Path 329 | total_timesteps 5196.
Path 330 | total_timesteps 5213.
Path 331 | total_timesteps 5228.
Path 332 | total_timesteps 5249.
Path 333 | total_timesteps 5263.
Path 334 | total_timesteps 5280.
Path 335 | total_timesteps 5301.
Path 336 | total_timesteps 5313.
Path 337 | total_timesteps 5333.
Path 338 | total_timesteps 5343.
Path 339 | total_timesteps 5355.
Path 340 | total_timesteps 5370.
Path 341 | total_timesteps 5392.
Path 342 | total_timesteps 5414.
Path 343 | total_timesteps 5431.
Path 344 | total_timesteps 5441.
Path 345 | total_timesteps 5454.
Path 346 | total_timesteps 5468.
Path 347 | total_timesteps 5493.
Path 348 | total_timesteps 5508.
Path 349 | total_timesteps 5522.
Path 350 | total_timesteps 5537.
Path 351 | total_timesteps 5560.
Path 352 | total_timesteps 5573.
Path 353 | total_timesteps 5591.
Path 354 | total_timesteps 5602.
Path 355 | total_timesteps 5615.
Path 356 | total_timesteps 5628.
Path 357 | total_timesteps 5647.
Path 358 | total_timesteps 5659.
Path 359 | total_timesteps 5674.
Path 360 | total_timesteps 5689.
Path 361 | total_timesteps 5701.
Path 362 | total_timesteps 5711.
Path 363 | total_timesteps 5723.
Path 364 | total_timesteps 5739.
Path 365 | total_timesteps 5750.
Path 366 | total_timesteps 5762.
Path 367 | total_timesteps 5775.
Path 368 | total_timesteps 5783.
Path 369 | total_timesteps 5791.
Path 370 | total_timesteps 5805.
Path 371 | total_timesteps 5825.
Path 372 | total_timesteps 5843.
Path 373 | total_timesteps 5864.
Path 374 | total_timesteps 5879.
Path 375 | total_timesteps 5892.
Path 376 | total_timesteps 5902.
Path 377 | total_timesteps 5914.
Path 378 | total_timesteps 5936.
Path 379 | total_timesteps 5953.
Path 380 | total_timesteps 5975.
Path 381 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11      |
| Iteration     | 15       |
| MaximumReturn | 2.05     |
| MinimumReturn | -22.9    |
| TotalSamples  | 68132    |
----------------------------
itr #16 | 
Fitting dynamics.
Validation loss = 0.008755506947636604
Validation loss = 0.00820080190896988
Validation loss = 0.00836253073066473
Validation loss = 0.008728832937777042
Validation loss = 0.009398805908858776
Validation loss = 0.008618771098554134
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 39.
Path 4 | total_timesteps 50.
Path 5 | total_timesteps 66.
Path 6 | total_timesteps 75.
Path 7 | total_timesteps 99.
Path 8 | total_timesteps 124.
Path 9 | total_timesteps 140.
Path 10 | total_timesteps 149.
Path 11 | total_timesteps 165.
Path 12 | total_timesteps 176.
Path 13 | total_timesteps 198.
Path 14 | total_timesteps 219.
Path 15 | total_timesteps 229.
Path 16 | total_timesteps 247.
Path 17 | total_timesteps 274.
Path 18 | total_timesteps 291.
Path 19 | total_timesteps 304.
Path 20 | total_timesteps 334.
Path 21 | total_timesteps 348.
Path 22 | total_timesteps 369.
Path 23 | total_timesteps 394.
Path 24 | total_timesteps 413.
Path 25 | total_timesteps 431.
Path 26 | total_timesteps 447.
Path 27 | total_timesteps 458.
Path 28 | total_timesteps 474.
Path 29 | total_timesteps 505.
Path 30 | total_timesteps 518.
Path 31 | total_timesteps 530.
Path 32 | total_timesteps 540.
Path 33 | total_timesteps 551.
Path 34 | total_timesteps 563.
Path 35 | total_timesteps 579.
Path 36 | total_timesteps 604.
Path 37 | total_timesteps 625.
Path 38 | total_timesteps 641.
Path 39 | total_timesteps 658.
Path 40 | total_timesteps 674.
Path 41 | total_timesteps 683.
Path 42 | total_timesteps 694.
Path 43 | total_timesteps 706.
Path 44 | total_timesteps 727.
Path 45 | total_timesteps 751.
Path 46 | total_timesteps 760.
Path 47 | total_timesteps 782.
Path 48 | total_timesteps 792.
Path 49 | total_timesteps 805.
Path 50 | total_timesteps 819.
Path 51 | total_timesteps 842.
Path 52 | total_timesteps 867.
Path 53 | total_timesteps 876.
Path 54 | total_timesteps 891.
Path 55 | total_timesteps 901.
Path 56 | total_timesteps 913.
Path 57 | total_timesteps 925.
Path 58 | total_timesteps 935.
Path 59 | total_timesteps 946.
Path 60 | total_timesteps 965.
Path 61 | total_timesteps 977.
Path 62 | total_timesteps 995.
Path 63 | total_timesteps 1013.
Path 64 | total_timesteps 1033.
Path 65 | total_timesteps 1048.
Path 66 | total_timesteps 1063.
Path 67 | total_timesteps 1084.
Path 68 | total_timesteps 1105.
Path 69 | total_timesteps 1124.
Path 70 | total_timesteps 1137.
Path 71 | total_timesteps 1152.
Path 72 | total_timesteps 1173.
Path 73 | total_timesteps 1190.
Path 74 | total_timesteps 1206.
Path 75 | total_timesteps 1222.
Path 76 | total_timesteps 1235.
Path 77 | total_timesteps 1257.
Path 78 | total_timesteps 1281.
Path 79 | total_timesteps 1295.
Path 80 | total_timesteps 1317.
Path 81 | total_timesteps 1334.
Path 82 | total_timesteps 1345.
Path 83 | total_timesteps 1361.
Path 84 | total_timesteps 1376.
Path 85 | total_timesteps 1391.
Path 86 | total_timesteps 1410.
Path 87 | total_timesteps 1430.
Path 88 | total_timesteps 1450.
Path 89 | total_timesteps 1466.
Path 90 | total_timesteps 1474.
Path 91 | total_timesteps 1492.
Path 92 | total_timesteps 1510.
Path 93 | total_timesteps 1520.
Path 94 | total_timesteps 1536.
Path 95 | total_timesteps 1548.
Path 96 | total_timesteps 1564.
Path 97 | total_timesteps 1585.
Path 98 | total_timesteps 1610.
Path 99 | total_timesteps 1636.
Path 100 | total_timesteps 1657.
Path 101 | total_timesteps 1674.
Path 102 | total_timesteps 1685.
Path 103 | total_timesteps 1699.
Path 104 | total_timesteps 1718.
Path 105 | total_timesteps 1733.
Path 106 | total_timesteps 1747.
Path 107 | total_timesteps 1767.
Path 108 | total_timesteps 1784.
Path 109 | total_timesteps 1805.
Path 110 | total_timesteps 1818.
Path 111 | total_timesteps 1831.
Path 112 | total_timesteps 1848.
Path 113 | total_timesteps 1858.
Path 114 | total_timesteps 1877.
Path 115 | total_timesteps 1888.
Path 116 | total_timesteps 1903.
Path 117 | total_timesteps 1914.
Path 118 | total_timesteps 1928.
Path 119 | total_timesteps 1944.
Path 120 | total_timesteps 1960.
Path 121 | total_timesteps 1977.
Path 122 | total_timesteps 1988.
Path 123 | total_timesteps 2011.
Path 124 | total_timesteps 2036.
Path 125 | total_timesteps 2053.
Path 126 | total_timesteps 2062.
Path 127 | total_timesteps 2072.
Path 128 | total_timesteps 2088.
Path 129 | total_timesteps 2100.
Path 130 | total_timesteps 2111.
Path 131 | total_timesteps 2129.
Path 132 | total_timesteps 2151.
Path 133 | total_timesteps 2167.
Path 134 | total_timesteps 2177.
Path 135 | total_timesteps 2201.
Path 136 | total_timesteps 2215.
Path 137 | total_timesteps 2237.
Path 138 | total_timesteps 2257.
Path 139 | total_timesteps 2274.
Path 140 | total_timesteps 2299.
Path 141 | total_timesteps 2313.
Path 142 | total_timesteps 2326.
Path 143 | total_timesteps 2346.
Path 144 | total_timesteps 2362.
Path 145 | total_timesteps 2372.
Path 146 | total_timesteps 2393.
Path 147 | total_timesteps 2415.
Path 148 | total_timesteps 2430.
Path 149 | total_timesteps 2452.
Path 150 | total_timesteps 2471.
Path 151 | total_timesteps 2483.
Path 152 | total_timesteps 2498.
Path 153 | total_timesteps 2517.
Path 154 | total_timesteps 2532.
Path 155 | total_timesteps 2548.
Path 156 | total_timesteps 2558.
Path 157 | total_timesteps 2569.
Path 158 | total_timesteps 2579.
Path 159 | total_timesteps 2587.
Path 160 | total_timesteps 2600.
Path 161 | total_timesteps 2611.
Path 162 | total_timesteps 2629.
Path 163 | total_timesteps 2643.
Path 164 | total_timesteps 2656.
Path 165 | total_timesteps 2684.
Path 166 | total_timesteps 2703.
Path 167 | total_timesteps 2721.
Path 168 | total_timesteps 2744.
Path 169 | total_timesteps 2775.
Path 170 | total_timesteps 2787.
Path 171 | total_timesteps 2800.
Path 172 | total_timesteps 2811.
Path 173 | total_timesteps 2824.
Path 174 | total_timesteps 2840.
Path 175 | total_timesteps 2852.
Path 176 | total_timesteps 2860.
Path 177 | total_timesteps 2886.
Path 178 | total_timesteps 2898.
Path 179 | total_timesteps 2918.
Path 180 | total_timesteps 2941.
Path 181 | total_timesteps 2953.
Path 182 | total_timesteps 2964.
Path 183 | total_timesteps 2977.
Path 184 | total_timesteps 2992.
Path 185 | total_timesteps 3002.
Path 186 | total_timesteps 3015.
Path 187 | total_timesteps 3028.
Path 188 | total_timesteps 3041.
Path 189 | total_timesteps 3052.
Path 190 | total_timesteps 3068.
Path 191 | total_timesteps 3081.
Path 192 | total_timesteps 3092.
Path 193 | total_timesteps 3103.
Path 194 | total_timesteps 3125.
Path 195 | total_timesteps 3140.
Path 196 | total_timesteps 3157.
Path 197 | total_timesteps 3173.
Path 198 | total_timesteps 3185.
Path 199 | total_timesteps 3195.
Path 200 | total_timesteps 3205.
Path 201 | total_timesteps 3229.
Path 202 | total_timesteps 3248.
Path 203 | total_timesteps 3261.
Path 204 | total_timesteps 3292.
Path 205 | total_timesteps 3334.
Path 206 | total_timesteps 3357.
Path 207 | total_timesteps 3376.
Path 208 | total_timesteps 3397.
Path 209 | total_timesteps 3416.
Path 210 | total_timesteps 3434.
Path 211 | total_timesteps 3450.
Path 212 | total_timesteps 3469.
Path 213 | total_timesteps 3490.
Path 214 | total_timesteps 3500.
Path 215 | total_timesteps 3514.
Path 216 | total_timesteps 3530.
Path 217 | total_timesteps 3543.
Path 218 | total_timesteps 3566.
Path 219 | total_timesteps 3575.
Path 220 | total_timesteps 3595.
Path 221 | total_timesteps 3618.
Path 222 | total_timesteps 3629.
Path 223 | total_timesteps 3643.
Path 224 | total_timesteps 3652.
Path 225 | total_timesteps 3668.
Path 226 | total_timesteps 3685.
Path 227 | total_timesteps 3706.
Path 228 | total_timesteps 3723.
Path 229 | total_timesteps 3745.
Path 230 | total_timesteps 3753.
Path 231 | total_timesteps 3770.
Path 232 | total_timesteps 3788.
Path 233 | total_timesteps 3802.
Path 234 | total_timesteps 3821.
Path 235 | total_timesteps 3836.
Path 236 | total_timesteps 3851.
Path 237 | total_timesteps 3867.
Path 238 | total_timesteps 3883.
Path 239 | total_timesteps 3893.
Path 240 | total_timesteps 3909.
Path 241 | total_timesteps 3935.
Path 242 | total_timesteps 3946.
Path 243 | total_timesteps 3956.
Path 244 | total_timesteps 3970.
Path 245 | total_timesteps 3988.
Path 246 | total_timesteps 4009.
Path 247 | total_timesteps 4026.
Path 248 | total_timesteps 4036.
Path 249 | total_timesteps 4057.
Path 250 | total_timesteps 4072.
Path 251 | total_timesteps 4084.
Path 252 | total_timesteps 4098.
Path 253 | total_timesteps 4106.
Path 254 | total_timesteps 4119.
Path 255 | total_timesteps 4136.
Path 256 | total_timesteps 4153.
Path 257 | total_timesteps 4176.
Path 258 | total_timesteps 4189.
Path 259 | total_timesteps 4202.
Path 260 | total_timesteps 4224.
Path 261 | total_timesteps 4242.
Path 262 | total_timesteps 4254.
Path 263 | total_timesteps 4272.
Path 264 | total_timesteps 4282.
Path 265 | total_timesteps 4297.
Path 266 | total_timesteps 4323.
Path 267 | total_timesteps 4344.
Path 268 | total_timesteps 4369.
Path 269 | total_timesteps 4387.
Path 270 | total_timesteps 4398.
Path 271 | total_timesteps 4413.
Path 272 | total_timesteps 4421.
Path 273 | total_timesteps 4441.
Path 274 | total_timesteps 4459.
Path 275 | total_timesteps 4476.
Path 276 | total_timesteps 4504.
Path 277 | total_timesteps 4524.
Path 278 | total_timesteps 4550.
Path 279 | total_timesteps 4565.
Path 280 | total_timesteps 4572.
Path 281 | total_timesteps 4588.
Path 282 | total_timesteps 4601.
Path 283 | total_timesteps 4616.
Path 284 | total_timesteps 4630.
Path 285 | total_timesteps 4647.
Path 286 | total_timesteps 4667.
Path 287 | total_timesteps 4685.
Path 288 | total_timesteps 4697.
Path 289 | total_timesteps 4712.
Path 290 | total_timesteps 4727.
Path 291 | total_timesteps 4746.
Path 292 | total_timesteps 4762.
Path 293 | total_timesteps 4772.
Path 294 | total_timesteps 4796.
Path 295 | total_timesteps 4816.
Path 296 | total_timesteps 4835.
Path 297 | total_timesteps 4845.
Path 298 | total_timesteps 4855.
Path 299 | total_timesteps 4869.
Path 300 | total_timesteps 4889.
Path 301 | total_timesteps 4911.
Path 302 | total_timesteps 4930.
Path 303 | total_timesteps 4946.
Path 304 | total_timesteps 4971.
Path 305 | total_timesteps 4994.
Path 306 | total_timesteps 5004.
Path 307 | total_timesteps 5018.
Path 308 | total_timesteps 5038.
Path 309 | total_timesteps 5061.
Path 310 | total_timesteps 5080.
Path 311 | total_timesteps 5094.
Path 312 | total_timesteps 5109.
Path 313 | total_timesteps 5125.
Path 314 | total_timesteps 5145.
Path 315 | total_timesteps 5167.
Path 316 | total_timesteps 5190.
Path 317 | total_timesteps 5204.
Path 318 | total_timesteps 5216.
Path 319 | total_timesteps 5228.
Path 320 | total_timesteps 5239.
Path 321 | total_timesteps 5265.
Path 322 | total_timesteps 5283.
Path 323 | total_timesteps 5300.
Path 324 | total_timesteps 5316.
Path 325 | total_timesteps 5324.
Path 326 | total_timesteps 5342.
Path 327 | total_timesteps 5360.
Path 328 | total_timesteps 5369.
Path 329 | total_timesteps 5383.
Path 330 | total_timesteps 5418.
Path 331 | total_timesteps 5428.
Path 332 | total_timesteps 5438.
Path 333 | total_timesteps 5449.
Path 334 | total_timesteps 5470.
Path 335 | total_timesteps 5495.
Path 336 | total_timesteps 5508.
Path 337 | total_timesteps 5528.
Path 338 | total_timesteps 5544.
Path 339 | total_timesteps 5571.
Path 340 | total_timesteps 5582.
Path 341 | total_timesteps 5592.
Path 342 | total_timesteps 5604.
Path 343 | total_timesteps 5614.
Path 344 | total_timesteps 5623.
Path 345 | total_timesteps 5635.
Path 346 | total_timesteps 5651.
Path 347 | total_timesteps 5667.
Path 348 | total_timesteps 5682.
Path 349 | total_timesteps 5689.
Path 350 | total_timesteps 5699.
Path 351 | total_timesteps 5722.
Path 352 | total_timesteps 5745.
Path 353 | total_timesteps 5765.
Path 354 | total_timesteps 5783.
Path 355 | total_timesteps 5795.
Path 356 | total_timesteps 5808.
Path 357 | total_timesteps 5830.
Path 358 | total_timesteps 5851.
Path 359 | total_timesteps 5866.
Path 360 | total_timesteps 5885.
Path 361 | total_timesteps 5905.
Path 362 | total_timesteps 5917.
Path 363 | total_timesteps 5933.
Path 364 | total_timesteps 5944.
Path 365 | total_timesteps 5957.
Path 366 | total_timesteps 5970.
Path 367 | total_timesteps 5983.
Path 368 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.5    |
| Iteration     | 16       |
| MaximumReturn | 1.13     |
| MinimumReturn | -20.8    |
| TotalSamples  | 72142    |
----------------------------
itr #17 | 
Fitting dynamics.
Validation loss = 0.009421388618648052
Validation loss = 0.008409635163843632
Validation loss = 0.008149621076881886
Validation loss = 0.009478673338890076
Validation loss = 0.008035026490688324
Validation loss = 0.007874932140111923
Validation loss = 0.008083198219537735
Validation loss = 0.008319908753037453
Validation loss = 0.008134787902235985
Validation loss = 0.007787412963807583
Validation loss = 0.008318815380334854
Validation loss = 0.008305389434099197
Validation loss = 0.00772741949185729
Validation loss = 0.00898403488099575
Validation loss = 0.008109058253467083
Validation loss = 0.00863311905413866
Validation loss = 0.008218586444854736
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 39.
Path 3 | total_timesteps 64.
Path 4 | total_timesteps 87.
Path 5 | total_timesteps 100.
Path 6 | total_timesteps 122.
Path 7 | total_timesteps 134.
Path 8 | total_timesteps 149.
Path 9 | total_timesteps 166.
Path 10 | total_timesteps 180.
Path 11 | total_timesteps 199.
Path 12 | total_timesteps 210.
Path 13 | total_timesteps 228.
Path 14 | total_timesteps 250.
Path 15 | total_timesteps 266.
Path 16 | total_timesteps 285.
Path 17 | total_timesteps 308.
Path 18 | total_timesteps 325.
Path 19 | total_timesteps 342.
Path 20 | total_timesteps 357.
Path 21 | total_timesteps 378.
Path 22 | total_timesteps 395.
Path 23 | total_timesteps 417.
Path 24 | total_timesteps 435.
Path 25 | total_timesteps 453.
Path 26 | total_timesteps 482.
Path 27 | total_timesteps 508.
Path 28 | total_timesteps 530.
Path 29 | total_timesteps 547.
Path 30 | total_timesteps 565.
Path 31 | total_timesteps 604.
Path 32 | total_timesteps 625.
Path 33 | total_timesteps 639.
Path 34 | total_timesteps 666.
Path 35 | total_timesteps 687.
Path 36 | total_timesteps 699.
Path 37 | total_timesteps 723.
Path 38 | total_timesteps 752.
Path 39 | total_timesteps 765.
Path 40 | total_timesteps 779.
Path 41 | total_timesteps 791.
Path 42 | total_timesteps 807.
Path 43 | total_timesteps 828.
Path 44 | total_timesteps 842.
Path 45 | total_timesteps 851.
Path 46 | total_timesteps 876.
Path 47 | total_timesteps 902.
Path 48 | total_timesteps 919.
Path 49 | total_timesteps 944.
Path 50 | total_timesteps 963.
Path 51 | total_timesteps 988.
Path 52 | total_timesteps 998.
Path 53 | total_timesteps 1016.
Path 54 | total_timesteps 1037.
Path 55 | total_timesteps 1062.
Path 56 | total_timesteps 1081.
Path 57 | total_timesteps 1096.
Path 58 | total_timesteps 1118.
Path 59 | total_timesteps 1145.
Path 60 | total_timesteps 1163.
Path 61 | total_timesteps 1184.
Path 62 | total_timesteps 1203.
Path 63 | total_timesteps 1221.
Path 64 | total_timesteps 1230.
Path 65 | total_timesteps 1246.
Path 66 | total_timesteps 1264.
Path 67 | total_timesteps 1286.
Path 68 | total_timesteps 1305.
Path 69 | total_timesteps 1316.
Path 70 | total_timesteps 1334.
Path 71 | total_timesteps 1350.
Path 72 | total_timesteps 1372.
Path 73 | total_timesteps 1389.
Path 74 | total_timesteps 1405.
Path 75 | total_timesteps 1418.
Path 76 | total_timesteps 1436.
Path 77 | total_timesteps 1451.
Path 78 | total_timesteps 1472.
Path 79 | total_timesteps 1508.
Path 80 | total_timesteps 1522.
Path 81 | total_timesteps 1541.
Path 82 | total_timesteps 1589.
Path 83 | total_timesteps 1599.
Path 84 | total_timesteps 1613.
Path 85 | total_timesteps 1633.
Path 86 | total_timesteps 1650.
Path 87 | total_timesteps 1670.
Path 88 | total_timesteps 1690.
Path 89 | total_timesteps 1707.
Path 90 | total_timesteps 1724.
Path 91 | total_timesteps 1745.
Path 92 | total_timesteps 1771.
Path 93 | total_timesteps 1785.
Path 94 | total_timesteps 1808.
Path 95 | total_timesteps 1833.
Path 96 | total_timesteps 1852.
Path 97 | total_timesteps 1871.
Path 98 | total_timesteps 1890.
Path 99 | total_timesteps 1937.
Path 100 | total_timesteps 1955.
Path 101 | total_timesteps 1974.
Path 102 | total_timesteps 1986.
Path 103 | total_timesteps 2001.
Path 104 | total_timesteps 2025.
Path 105 | total_timesteps 2041.
Path 106 | total_timesteps 2064.
Path 107 | total_timesteps 2083.
Path 108 | total_timesteps 2106.
Path 109 | total_timesteps 2129.
Path 110 | total_timesteps 2148.
Path 111 | total_timesteps 2166.
Path 112 | total_timesteps 2185.
Path 113 | total_timesteps 2209.
Path 114 | total_timesteps 2224.
Path 115 | total_timesteps 2239.
Path 116 | total_timesteps 2261.
Path 117 | total_timesteps 2282.
Path 118 | total_timesteps 2299.
Path 119 | total_timesteps 2319.
Path 120 | total_timesteps 2342.
Path 121 | total_timesteps 2360.
Path 122 | total_timesteps 2380.
Path 123 | total_timesteps 2393.
Path 124 | total_timesteps 2410.
Path 125 | total_timesteps 2424.
Path 126 | total_timesteps 2449.
Path 127 | total_timesteps 2460.
Path 128 | total_timesteps 2487.
Path 129 | total_timesteps 2500.
Path 130 | total_timesteps 2516.
Path 131 | total_timesteps 2534.
Path 132 | total_timesteps 2555.
Path 133 | total_timesteps 2577.
Path 134 | total_timesteps 2601.
Path 135 | total_timesteps 2613.
Path 136 | total_timesteps 2632.
Path 137 | total_timesteps 2658.
Path 138 | total_timesteps 2692.
Path 139 | total_timesteps 2707.
Path 140 | total_timesteps 2723.
Path 141 | total_timesteps 2743.
Path 142 | total_timesteps 2765.
Path 143 | total_timesteps 2790.
Path 144 | total_timesteps 2807.
Path 145 | total_timesteps 2832.
Path 146 | total_timesteps 2849.
Path 147 | total_timesteps 2865.
Path 148 | total_timesteps 2879.
Path 149 | total_timesteps 2890.
Path 150 | total_timesteps 2910.
Path 151 | total_timesteps 2926.
Path 152 | total_timesteps 2942.
Path 153 | total_timesteps 2969.
Path 154 | total_timesteps 2987.
Path 155 | total_timesteps 2999.
Path 156 | total_timesteps 3021.
Path 157 | total_timesteps 3039.
Path 158 | total_timesteps 3051.
Path 159 | total_timesteps 3069.
Path 160 | total_timesteps 3077.
Path 161 | total_timesteps 3092.
Path 162 | total_timesteps 3107.
Path 163 | total_timesteps 3125.
Path 164 | total_timesteps 3147.
Path 165 | total_timesteps 3180.
Path 166 | total_timesteps 3199.
Path 167 | total_timesteps 3219.
Path 168 | total_timesteps 3238.
Path 169 | total_timesteps 3258.
Path 170 | total_timesteps 3279.
Path 171 | total_timesteps 3302.
Path 172 | total_timesteps 3324.
Path 173 | total_timesteps 3348.
Path 174 | total_timesteps 3365.
Path 175 | total_timesteps 3379.
Path 176 | total_timesteps 3401.
Path 177 | total_timesteps 3428.
Path 178 | total_timesteps 3449.
Path 179 | total_timesteps 3462.
Path 180 | total_timesteps 3474.
Path 181 | total_timesteps 3494.
Path 182 | total_timesteps 3519.
Path 183 | total_timesteps 3549.
Path 184 | total_timesteps 3564.
Path 185 | total_timesteps 3589.
Path 186 | total_timesteps 3603.
Path 187 | total_timesteps 3620.
Path 188 | total_timesteps 3636.
Path 189 | total_timesteps 3654.
Path 190 | total_timesteps 3674.
Path 191 | total_timesteps 3691.
Path 192 | total_timesteps 3709.
Path 193 | total_timesteps 3726.
Path 194 | total_timesteps 3747.
Path 195 | total_timesteps 3764.
Path 196 | total_timesteps 3787.
Path 197 | total_timesteps 3808.
Path 198 | total_timesteps 3823.
Path 199 | total_timesteps 3848.
Path 200 | total_timesteps 3866.
Path 201 | total_timesteps 3879.
Path 202 | total_timesteps 3897.
Path 203 | total_timesteps 3919.
Path 204 | total_timesteps 3938.
Path 205 | total_timesteps 3956.
Path 206 | total_timesteps 3974.
Path 207 | total_timesteps 3990.
Path 208 | total_timesteps 4009.
Path 209 | total_timesteps 4029.
Path 210 | total_timesteps 4049.
Path 211 | total_timesteps 4066.
Path 212 | total_timesteps 4083.
Path 213 | total_timesteps 4097.
Path 214 | total_timesteps 4113.
Path 215 | total_timesteps 4125.
Path 216 | total_timesteps 4146.
Path 217 | total_timesteps 4167.
Path 218 | total_timesteps 4199.
Path 219 | total_timesteps 4211.
Path 220 | total_timesteps 4228.
Path 221 | total_timesteps 4249.
Path 222 | total_timesteps 4264.
Path 223 | total_timesteps 4282.
Path 224 | total_timesteps 4301.
Path 225 | total_timesteps 4319.
Path 226 | total_timesteps 4339.
Path 227 | total_timesteps 4357.
Path 228 | total_timesteps 4375.
Path 229 | total_timesteps 4398.
Path 230 | total_timesteps 4413.
Path 231 | total_timesteps 4433.
Path 232 | total_timesteps 4454.
Path 233 | total_timesteps 4467.
Path 234 | total_timesteps 4487.
Path 235 | total_timesteps 4507.
Path 236 | total_timesteps 4526.
Path 237 | total_timesteps 4546.
Path 238 | total_timesteps 4560.
Path 239 | total_timesteps 4574.
Path 240 | total_timesteps 4583.
Path 241 | total_timesteps 4613.
Path 242 | total_timesteps 4637.
Path 243 | total_timesteps 4655.
Path 244 | total_timesteps 4673.
Path 245 | total_timesteps 4687.
Path 246 | total_timesteps 4701.
Path 247 | total_timesteps 4717.
Path 248 | total_timesteps 4732.
Path 249 | total_timesteps 4759.
Path 250 | total_timesteps 4776.
Path 251 | total_timesteps 4795.
Path 252 | total_timesteps 4816.
Path 253 | total_timesteps 4842.
Path 254 | total_timesteps 4872.
Path 255 | total_timesteps 4902.
Path 256 | total_timesteps 4923.
Path 257 | total_timesteps 4937.
Path 258 | total_timesteps 4957.
Path 259 | total_timesteps 4972.
Path 260 | total_timesteps 4995.
Path 261 | total_timesteps 5005.
Path 262 | total_timesteps 5019.
Path 263 | total_timesteps 5035.
Path 264 | total_timesteps 5052.
Path 265 | total_timesteps 5075.
Path 266 | total_timesteps 5090.
Path 267 | total_timesteps 5105.
Path 268 | total_timesteps 5127.
Path 269 | total_timesteps 5151.
Path 270 | total_timesteps 5170.
Path 271 | total_timesteps 5191.
Path 272 | total_timesteps 5212.
Path 273 | total_timesteps 5236.
Path 274 | total_timesteps 5258.
Path 275 | total_timesteps 5277.
Path 276 | total_timesteps 5314.
Path 277 | total_timesteps 5328.
Path 278 | total_timesteps 5343.
Path 279 | total_timesteps 5363.
Path 280 | total_timesteps 5383.
Path 281 | total_timesteps 5400.
Path 282 | total_timesteps 5418.
Path 283 | total_timesteps 5436.
Path 284 | total_timesteps 5455.
Path 285 | total_timesteps 5481.
Path 286 | total_timesteps 5497.
Path 287 | total_timesteps 5509.
Path 288 | total_timesteps 5526.
Path 289 | total_timesteps 5543.
Path 290 | total_timesteps 5561.
Path 291 | total_timesteps 5578.
Path 292 | total_timesteps 5595.
Path 293 | total_timesteps 5616.
Path 294 | total_timesteps 5638.
Path 295 | total_timesteps 5660.
Path 296 | total_timesteps 5682.
Path 297 | total_timesteps 5700.
Path 298 | total_timesteps 5719.
Path 299 | total_timesteps 5727.
Path 300 | total_timesteps 5748.
Path 301 | total_timesteps 5772.
Path 302 | total_timesteps 5793.
Path 303 | total_timesteps 5814.
Path 304 | total_timesteps 5834.
Path 305 | total_timesteps 5850.
Path 306 | total_timesteps 5867.
Path 307 | total_timesteps 5892.
Path 308 | total_timesteps 5903.
Path 309 | total_timesteps 5922.
Path 310 | total_timesteps 5948.
Path 311 | total_timesteps 5962.
Path 312 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -13.8    |
| Iteration     | 17       |
| MaximumReturn | 2.58     |
| MinimumReturn | -20.6    |
| TotalSamples  | 76144    |
----------------------------
itr #18 | 
Fitting dynamics.
Validation loss = 0.008490113541483879
Validation loss = 0.008135891519486904
Validation loss = 0.008286402560770512
Validation loss = 0.007693630177527666
Validation loss = 0.008232410997152328
Validation loss = 0.008344636298716068
Validation loss = 0.007551682647317648
Validation loss = 0.0077095055021345615
Validation loss = 0.008171601220965385
Validation loss = 0.0075688897632062435
Validation loss = 0.008105590008199215
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 54.
Path 4 | total_timesteps 66.
Path 5 | total_timesteps 89.
Path 6 | total_timesteps 110.
Path 7 | total_timesteps 123.
Path 8 | total_timesteps 155.
Path 9 | total_timesteps 171.
Path 10 | total_timesteps 194.
Path 11 | total_timesteps 215.
Path 12 | total_timesteps 231.
Path 13 | total_timesteps 267.
Path 14 | total_timesteps 286.
Path 15 | total_timesteps 312.
Path 16 | total_timesteps 331.
Path 17 | total_timesteps 343.
Path 18 | total_timesteps 358.
Path 19 | total_timesteps 373.
Path 20 | total_timesteps 392.
Path 21 | total_timesteps 420.
Path 22 | total_timesteps 443.
Path 23 | total_timesteps 467.
Path 24 | total_timesteps 483.
Path 25 | total_timesteps 496.
Path 26 | total_timesteps 516.
Path 27 | total_timesteps 534.
Path 28 | total_timesteps 558.
Path 29 | total_timesteps 574.
Path 30 | total_timesteps 584.
Path 31 | total_timesteps 606.
Path 32 | total_timesteps 636.
Path 33 | total_timesteps 661.
Path 34 | total_timesteps 686.
Path 35 | total_timesteps 707.
Path 36 | total_timesteps 718.
Path 37 | total_timesteps 740.
Path 38 | total_timesteps 773.
Path 39 | total_timesteps 789.
Path 40 | total_timesteps 806.
Path 41 | total_timesteps 827.
Path 42 | total_timesteps 860.
Path 43 | total_timesteps 884.
Path 44 | total_timesteps 904.
Path 45 | total_timesteps 933.
Path 46 | total_timesteps 949.
Path 47 | total_timesteps 968.
Path 48 | total_timesteps 988.
Path 49 | total_timesteps 997.
Path 50 | total_timesteps 1018.
Path 51 | total_timesteps 1042.
Path 52 | total_timesteps 1062.
Path 53 | total_timesteps 1074.
Path 54 | total_timesteps 1097.
Path 55 | total_timesteps 1111.
Path 56 | total_timesteps 1132.
Path 57 | total_timesteps 1152.
Path 58 | total_timesteps 1171.
Path 59 | total_timesteps 1184.
Path 60 | total_timesteps 1201.
Path 61 | total_timesteps 1215.
Path 62 | total_timesteps 1236.
Path 63 | total_timesteps 1272.
Path 64 | total_timesteps 1300.
Path 65 | total_timesteps 1321.
Path 66 | total_timesteps 1346.
Path 67 | total_timesteps 1371.
Path 68 | total_timesteps 1390.
Path 69 | total_timesteps 1409.
Path 70 | total_timesteps 1431.
Path 71 | total_timesteps 1443.
Path 72 | total_timesteps 1465.
Path 73 | total_timesteps 1491.
Path 74 | total_timesteps 1510.
Path 75 | total_timesteps 1535.
Path 76 | total_timesteps 1552.
Path 77 | total_timesteps 1571.
Path 78 | total_timesteps 1591.
Path 79 | total_timesteps 1611.
Path 80 | total_timesteps 1632.
Path 81 | total_timesteps 1653.
Path 82 | total_timesteps 1712.
Path 83 | total_timesteps 1740.
Path 84 | total_timesteps 1757.
Path 85 | total_timesteps 1771.
Path 86 | total_timesteps 1789.
Path 87 | total_timesteps 1806.
Path 88 | total_timesteps 1824.
Path 89 | total_timesteps 1844.
Path 90 | total_timesteps 1863.
Path 91 | total_timesteps 1889.
Path 92 | total_timesteps 1905.
Path 93 | total_timesteps 1925.
Path 94 | total_timesteps 1942.
Path 95 | total_timesteps 1962.
Path 96 | total_timesteps 1980.
Path 97 | total_timesteps 2001.
Path 98 | total_timesteps 2015.
Path 99 | total_timesteps 2029.
Path 100 | total_timesteps 2044.
Path 101 | total_timesteps 2064.
Path 102 | total_timesteps 2083.
Path 103 | total_timesteps 2106.
Path 104 | total_timesteps 2122.
Path 105 | total_timesteps 2147.
Path 106 | total_timesteps 2161.
Path 107 | total_timesteps 2187.
Path 108 | total_timesteps 2212.
Path 109 | total_timesteps 2231.
Path 110 | total_timesteps 2258.
Path 111 | total_timesteps 2275.
Path 112 | total_timesteps 2289.
Path 113 | total_timesteps 2308.
Path 114 | total_timesteps 2325.
Path 115 | total_timesteps 2357.
Path 116 | total_timesteps 2380.
Path 117 | total_timesteps 2392.
Path 118 | total_timesteps 2411.
Path 119 | total_timesteps 2439.
Path 120 | total_timesteps 2456.
Path 121 | total_timesteps 2481.
Path 122 | total_timesteps 2500.
Path 123 | total_timesteps 2518.
Path 124 | total_timesteps 2535.
Path 125 | total_timesteps 2554.
Path 126 | total_timesteps 2574.
Path 127 | total_timesteps 2594.
Path 128 | total_timesteps 2606.
Path 129 | total_timesteps 2625.
Path 130 | total_timesteps 2642.
Path 131 | total_timesteps 2663.
Path 132 | total_timesteps 2674.
Path 133 | total_timesteps 2699.
Path 134 | total_timesteps 2714.
Path 135 | total_timesteps 2733.
Path 136 | total_timesteps 2750.
Path 137 | total_timesteps 2772.
Path 138 | total_timesteps 2786.
Path 139 | total_timesteps 2805.
Path 140 | total_timesteps 2826.
Path 141 | total_timesteps 2846.
Path 142 | total_timesteps 2865.
Path 143 | total_timesteps 2884.
Path 144 | total_timesteps 2907.
Path 145 | total_timesteps 2925.
Path 146 | total_timesteps 2948.
Path 147 | total_timesteps 2960.
Path 148 | total_timesteps 2987.
Path 149 | total_timesteps 3006.
Path 150 | total_timesteps 3021.
Path 151 | total_timesteps 3052.
Path 152 | total_timesteps 3073.
Path 153 | total_timesteps 3088.
Path 154 | total_timesteps 3109.
Path 155 | total_timesteps 3129.
Path 156 | total_timesteps 3151.
Path 157 | total_timesteps 3164.
Path 158 | total_timesteps 3181.
Path 159 | total_timesteps 3204.
Path 160 | total_timesteps 3218.
Path 161 | total_timesteps 3237.
Path 162 | total_timesteps 3253.
Path 163 | total_timesteps 3275.
Path 164 | total_timesteps 3292.
Path 165 | total_timesteps 3315.
Path 166 | total_timesteps 3337.
Path 167 | total_timesteps 3356.
Path 168 | total_timesteps 3378.
Path 169 | total_timesteps 3396.
Path 170 | total_timesteps 3417.
Path 171 | total_timesteps 3431.
Path 172 | total_timesteps 3448.
Path 173 | total_timesteps 3467.
Path 174 | total_timesteps 3484.
Path 175 | total_timesteps 3502.
Path 176 | total_timesteps 3521.
Path 177 | total_timesteps 3545.
Path 178 | total_timesteps 3569.
Path 179 | total_timesteps 3588.
Path 180 | total_timesteps 3599.
Path 181 | total_timesteps 3620.
Path 182 | total_timesteps 3631.
Path 183 | total_timesteps 3651.
Path 184 | total_timesteps 3665.
Path 185 | total_timesteps 3686.
Path 186 | total_timesteps 3711.
Path 187 | total_timesteps 3728.
Path 188 | total_timesteps 3747.
Path 189 | total_timesteps 3765.
Path 190 | total_timesteps 3782.
Path 191 | total_timesteps 3803.
Path 192 | total_timesteps 3811.
Path 193 | total_timesteps 3828.
Path 194 | total_timesteps 3842.
Path 195 | total_timesteps 3862.
Path 196 | total_timesteps 3882.
Path 197 | total_timesteps 3897.
Path 198 | total_timesteps 3913.
Path 199 | total_timesteps 3930.
Path 200 | total_timesteps 3946.
Path 201 | total_timesteps 3975.
Path 202 | total_timesteps 3998.
Path 203 | total_timesteps 4015.
Path 204 | total_timesteps 4038.
Path 205 | total_timesteps 4055.
Path 206 | total_timesteps 4072.
Path 207 | total_timesteps 4088.
Path 208 | total_timesteps 4111.
Path 209 | total_timesteps 4131.
Path 210 | total_timesteps 4149.
Path 211 | total_timesteps 4165.
Path 212 | total_timesteps 4184.
Path 213 | total_timesteps 4198.
Path 214 | total_timesteps 4217.
Path 215 | total_timesteps 4240.
Path 216 | total_timesteps 4262.
Path 217 | total_timesteps 4290.
Path 218 | total_timesteps 4320.
Path 219 | total_timesteps 4344.
Path 220 | total_timesteps 4364.
Path 221 | total_timesteps 4381.
Path 222 | total_timesteps 4409.
Path 223 | total_timesteps 4425.
Path 224 | total_timesteps 4442.
Path 225 | total_timesteps 4456.
Path 226 | total_timesteps 4468.
Path 227 | total_timesteps 4479.
Path 228 | total_timesteps 4500.
Path 229 | total_timesteps 4517.
Path 230 | total_timesteps 4540.
Path 231 | total_timesteps 4553.
Path 232 | total_timesteps 4571.
Path 233 | total_timesteps 4591.
Path 234 | total_timesteps 4625.
Path 235 | total_timesteps 4638.
Path 236 | total_timesteps 4653.
Path 237 | total_timesteps 4665.
Path 238 | total_timesteps 4686.
Path 239 | total_timesteps 4707.
Path 240 | total_timesteps 4728.
Path 241 | total_timesteps 4746.
Path 242 | total_timesteps 4770.
Path 243 | total_timesteps 4789.
Path 244 | total_timesteps 4807.
Path 245 | total_timesteps 4824.
Path 246 | total_timesteps 4849.
Path 247 | total_timesteps 4872.
Path 248 | total_timesteps 4918.
Path 249 | total_timesteps 4934.
Path 250 | total_timesteps 4954.
Path 251 | total_timesteps 4981.
Path 252 | total_timesteps 5006.
Path 253 | total_timesteps 5025.
Path 254 | total_timesteps 5043.
Path 255 | total_timesteps 5055.
Path 256 | total_timesteps 5075.
Path 257 | total_timesteps 5085.
Path 258 | total_timesteps 5103.
Path 259 | total_timesteps 5127.
Path 260 | total_timesteps 5150.
Path 261 | total_timesteps 5169.
Path 262 | total_timesteps 5190.
Path 263 | total_timesteps 5209.
Path 264 | total_timesteps 5232.
Path 265 | total_timesteps 5250.
Path 266 | total_timesteps 5262.
Path 267 | total_timesteps 5279.
Path 268 | total_timesteps 5297.
Path 269 | total_timesteps 5312.
Path 270 | total_timesteps 5337.
Path 271 | total_timesteps 5355.
Path 272 | total_timesteps 5383.
Path 273 | total_timesteps 5399.
Path 274 | total_timesteps 5421.
Path 275 | total_timesteps 5442.
Path 276 | total_timesteps 5454.
Path 277 | total_timesteps 5480.
Path 278 | total_timesteps 5508.
Path 279 | total_timesteps 5556.
Path 280 | total_timesteps 5577.
Path 281 | total_timesteps 5592.
Path 282 | total_timesteps 5610.
Path 283 | total_timesteps 5627.
Path 284 | total_timesteps 5648.
Path 285 | total_timesteps 5668.
Path 286 | total_timesteps 5680.
Path 287 | total_timesteps 5696.
Path 288 | total_timesteps 5718.
Path 289 | total_timesteps 5757.
Path 290 | total_timesteps 5775.
Path 291 | total_timesteps 5793.
Path 292 | total_timesteps 5813.
Path 293 | total_timesteps 5834.
Path 294 | total_timesteps 5853.
Path 295 | total_timesteps 5881.
Path 296 | total_timesteps 5900.
Path 297 | total_timesteps 5911.
Path 298 | total_timesteps 5926.
Path 299 | total_timesteps 5942.
Path 300 | total_timesteps 5966.
Path 301 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -14.1    |
| Iteration     | 18       |
| MaximumReturn | 1.86     |
| MinimumReturn | -22      |
| TotalSamples  | 80152    |
----------------------------
itr #19 | 
Fitting dynamics.
Validation loss = 0.00830494612455368
Validation loss = 0.007737568113952875
Validation loss = 0.007627496030181646
Validation loss = 0.00716695049777627
Validation loss = 0.007867136970162392
Validation loss = 0.007674968335777521
Validation loss = 0.007900439202785492
Validation loss = 0.007500494364649057
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 20.
Path 2 | total_timesteps 51.
Path 3 | total_timesteps 75.
Path 4 | total_timesteps 91.
Path 5 | total_timesteps 113.
Path 6 | total_timesteps 136.
Path 7 | total_timesteps 163.
Path 8 | total_timesteps 180.
Path 9 | total_timesteps 208.
Path 10 | total_timesteps 220.
Path 11 | total_timesteps 251.
Path 12 | total_timesteps 269.
Path 13 | total_timesteps 283.
Path 14 | total_timesteps 309.
Path 15 | total_timesteps 326.
Path 16 | total_timesteps 338.
Path 17 | total_timesteps 359.
Path 18 | total_timesteps 379.
Path 19 | total_timesteps 415.
Path 20 | total_timesteps 435.
Path 21 | total_timesteps 455.
Path 22 | total_timesteps 465.
Path 23 | total_timesteps 483.
Path 24 | total_timesteps 504.
Path 25 | total_timesteps 536.
Path 26 | total_timesteps 562.
Path 27 | total_timesteps 580.
Path 28 | total_timesteps 591.
Path 29 | total_timesteps 612.
Path 30 | total_timesteps 639.
Path 31 | total_timesteps 661.
Path 32 | total_timesteps 675.
Path 33 | total_timesteps 690.
Path 34 | total_timesteps 713.
Path 35 | total_timesteps 733.
Path 36 | total_timesteps 748.
Path 37 | total_timesteps 775.
Path 38 | total_timesteps 795.
Path 39 | total_timesteps 824.
Path 40 | total_timesteps 855.
Path 41 | total_timesteps 882.
Path 42 | total_timesteps 904.
Path 43 | total_timesteps 938.
Path 44 | total_timesteps 961.
Path 45 | total_timesteps 982.
Path 46 | total_timesteps 1000.
Path 47 | total_timesteps 1017.
Path 48 | total_timesteps 1033.
Path 49 | total_timesteps 1056.
Path 50 | total_timesteps 1086.
Path 51 | total_timesteps 1112.
Path 52 | total_timesteps 1132.
Path 53 | total_timesteps 1148.
Path 54 | total_timesteps 1169.
Path 55 | total_timesteps 1188.
Path 56 | total_timesteps 1212.
Path 57 | total_timesteps 1235.
Path 58 | total_timesteps 1253.
Path 59 | total_timesteps 1275.
Path 60 | total_timesteps 1294.
Path 61 | total_timesteps 1325.
Path 62 | total_timesteps 1350.
Path 63 | total_timesteps 1371.
Path 64 | total_timesteps 1381.
Path 65 | total_timesteps 1403.
Path 66 | total_timesteps 1429.
Path 67 | total_timesteps 1451.
Path 68 | total_timesteps 1476.
Path 69 | total_timesteps 1496.
Path 70 | total_timesteps 1515.
Path 71 | total_timesteps 1546.
Path 72 | total_timesteps 1565.
Path 73 | total_timesteps 1606.
Path 74 | total_timesteps 1626.
Path 75 | total_timesteps 1640.
Path 76 | total_timesteps 1661.
Path 77 | total_timesteps 1692.
Path 78 | total_timesteps 1711.
Path 79 | total_timesteps 1737.
Path 80 | total_timesteps 1759.
Path 81 | total_timesteps 1779.
Path 82 | total_timesteps 1803.
Path 83 | total_timesteps 1826.
Path 84 | total_timesteps 1850.
Path 85 | total_timesteps 1882.
Path 86 | total_timesteps 1900.
Path 87 | total_timesteps 1922.
Path 88 | total_timesteps 1939.
Path 89 | total_timesteps 1964.
Path 90 | total_timesteps 1981.
Path 91 | total_timesteps 2008.
Path 92 | total_timesteps 2029.
Path 93 | total_timesteps 2054.
Path 94 | total_timesteps 2075.
Path 95 | total_timesteps 2092.
Path 96 | total_timesteps 2115.
Path 97 | total_timesteps 2133.
Path 98 | total_timesteps 2150.
Path 99 | total_timesteps 2166.
Path 100 | total_timesteps 2175.
Path 101 | total_timesteps 2200.
Path 102 | total_timesteps 2226.
Path 103 | total_timesteps 2247.
Path 104 | total_timesteps 2272.
Path 105 | total_timesteps 2293.
Path 106 | total_timesteps 2317.
Path 107 | total_timesteps 2337.
Path 108 | total_timesteps 2364.
Path 109 | total_timesteps 2386.
Path 110 | total_timesteps 2402.
Path 111 | total_timesteps 2420.
Path 112 | total_timesteps 2452.
Path 113 | total_timesteps 2467.
Path 114 | total_timesteps 2486.
Path 115 | total_timesteps 2508.
Path 116 | total_timesteps 2530.
Path 117 | total_timesteps 2553.
Path 118 | total_timesteps 2569.
Path 119 | total_timesteps 2594.
Path 120 | total_timesteps 2614.
Path 121 | total_timesteps 2630.
Path 122 | total_timesteps 2654.
Path 123 | total_timesteps 2677.
Path 124 | total_timesteps 2699.
Path 125 | total_timesteps 2718.
Path 126 | total_timesteps 2749.
Path 127 | total_timesteps 2757.
Path 128 | total_timesteps 2782.
Path 129 | total_timesteps 2806.
Path 130 | total_timesteps 2820.
Path 131 | total_timesteps 2828.
Path 132 | total_timesteps 2846.
Path 133 | total_timesteps 2876.
Path 134 | total_timesteps 2902.
Path 135 | total_timesteps 2915.
Path 136 | total_timesteps 2936.
Path 137 | total_timesteps 2958.
Path 138 | total_timesteps 2980.
Path 139 | total_timesteps 2996.
Path 140 | total_timesteps 3020.
Path 141 | total_timesteps 3044.
Path 142 | total_timesteps 3065.
Path 143 | total_timesteps 3080.
Path 144 | total_timesteps 3106.
Path 145 | total_timesteps 3116.
Path 146 | total_timesteps 3152.
Path 147 | total_timesteps 3165.
Path 148 | total_timesteps 3176.
Path 149 | total_timesteps 3198.
Path 150 | total_timesteps 3214.
Path 151 | total_timesteps 3237.
Path 152 | total_timesteps 3264.
Path 153 | total_timesteps 3284.
Path 154 | total_timesteps 3302.
Path 155 | total_timesteps 3319.
Path 156 | total_timesteps 3336.
Path 157 | total_timesteps 3361.
Path 158 | total_timesteps 3388.
Path 159 | total_timesteps 3415.
Path 160 | total_timesteps 3436.
Path 161 | total_timesteps 3483.
Path 162 | total_timesteps 3498.
Path 163 | total_timesteps 3514.
Path 164 | total_timesteps 3534.
Path 165 | total_timesteps 3556.
Path 166 | total_timesteps 3569.
Path 167 | total_timesteps 3578.
Path 168 | total_timesteps 3595.
Path 169 | total_timesteps 3610.
Path 170 | total_timesteps 3621.
Path 171 | total_timesteps 3645.
Path 172 | total_timesteps 3662.
Path 173 | total_timesteps 3691.
Path 174 | total_timesteps 3712.
Path 175 | total_timesteps 3737.
Path 176 | total_timesteps 3746.
Path 177 | total_timesteps 3762.
Path 178 | total_timesteps 3779.
Path 179 | total_timesteps 3809.
Path 180 | total_timesteps 3830.
Path 181 | total_timesteps 3854.
Path 182 | total_timesteps 3878.
Path 183 | total_timesteps 3898.
Path 184 | total_timesteps 3925.
Path 185 | total_timesteps 3955.
Path 186 | total_timesteps 3975.
Path 187 | total_timesteps 3993.
Path 188 | total_timesteps 4008.
Path 189 | total_timesteps 4027.
Path 190 | total_timesteps 4056.
Path 191 | total_timesteps 4074.
Path 192 | total_timesteps 4090.
Path 193 | total_timesteps 4109.
Path 194 | total_timesteps 4127.
Path 195 | total_timesteps 4148.
Path 196 | total_timesteps 4166.
Path 197 | total_timesteps 4182.
Path 198 | total_timesteps 4200.
Path 199 | total_timesteps 4215.
Path 200 | total_timesteps 4235.
Path 201 | total_timesteps 4257.
Path 202 | total_timesteps 4276.
Path 203 | total_timesteps 4299.
Path 204 | total_timesteps 4318.
Path 205 | total_timesteps 4337.
Path 206 | total_timesteps 4359.
Path 207 | total_timesteps 4372.
Path 208 | total_timesteps 4404.
Path 209 | total_timesteps 4420.
Path 210 | total_timesteps 4438.
Path 211 | total_timesteps 4459.
Path 212 | total_timesteps 4474.
Path 213 | total_timesteps 4492.
Path 214 | total_timesteps 4510.
Path 215 | total_timesteps 4533.
Path 216 | total_timesteps 4548.
Path 217 | total_timesteps 4571.
Path 218 | total_timesteps 4589.
Path 219 | total_timesteps 4610.
Path 220 | total_timesteps 4632.
Path 221 | total_timesteps 4652.
Path 222 | total_timesteps 4683.
Path 223 | total_timesteps 4703.
Path 224 | total_timesteps 4723.
Path 225 | total_timesteps 4744.
Path 226 | total_timesteps 4765.
Path 227 | total_timesteps 4783.
Path 228 | total_timesteps 4805.
Path 229 | total_timesteps 4830.
Path 230 | total_timesteps 4850.
Path 231 | total_timesteps 4866.
Path 232 | total_timesteps 4902.
Path 233 | total_timesteps 4916.
Path 234 | total_timesteps 4952.
Path 235 | total_timesteps 4959.
Path 236 | total_timesteps 4985.
Path 237 | total_timesteps 5008.
Path 238 | total_timesteps 5028.
Path 239 | total_timesteps 5049.
Path 240 | total_timesteps 5072.
Path 241 | total_timesteps 5092.
Path 242 | total_timesteps 5111.
Path 243 | total_timesteps 5136.
Path 244 | total_timesteps 5172.
Path 245 | total_timesteps 5194.
Path 246 | total_timesteps 5215.
Path 247 | total_timesteps 5247.
Path 248 | total_timesteps 5261.
Path 249 | total_timesteps 5286.
Path 250 | total_timesteps 5310.
Path 251 | total_timesteps 5331.
Path 252 | total_timesteps 5352.
Path 253 | total_timesteps 5369.
Path 254 | total_timesteps 5393.
Path 255 | total_timesteps 5422.
Path 256 | total_timesteps 5442.
Path 257 | total_timesteps 5458.
Path 258 | total_timesteps 5490.
Path 259 | total_timesteps 5508.
Path 260 | total_timesteps 5536.
Path 261 | total_timesteps 5553.
Path 262 | total_timesteps 5569.
Path 263 | total_timesteps 5590.
Path 264 | total_timesteps 5611.
Path 265 | total_timesteps 5627.
Path 266 | total_timesteps 5650.
Path 267 | total_timesteps 5673.
Path 268 | total_timesteps 5686.
Path 269 | total_timesteps 5705.
Path 270 | total_timesteps 5718.
Path 271 | total_timesteps 5740.
Path 272 | total_timesteps 5764.
Path 273 | total_timesteps 5790.
Path 274 | total_timesteps 5829.
Path 275 | total_timesteps 5863.
Path 276 | total_timesteps 5882.
Path 277 | total_timesteps 5907.
Path 278 | total_timesteps 5931.
Path 279 | total_timesteps 5946.
Path 280 | total_timesteps 5964.
Path 281 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -13.5    |
| Iteration     | 19       |
| MaximumReturn | 12       |
| MinimumReturn | -21.5    |
| TotalSamples  | 84158    |
----------------------------
itr #20 | 
Fitting dynamics.
Validation loss = 0.007741138339042664
Validation loss = 0.0077107688412070274
Validation loss = 0.00737000023946166
Validation loss = 0.007331821136176586
Validation loss = 0.007354766130447388
Validation loss = 0.007196991704404354
Validation loss = 0.00758947292342782
Validation loss = 0.007087555713951588
Validation loss = 0.007229423150420189
Validation loss = 0.007035015616565943
Validation loss = 0.007658486720174551
Validation loss = 0.006983189843595028
Validation loss = 0.00710823480039835
Validation loss = 0.007015458773821592
Validation loss = 0.00697306077927351
Validation loss = 0.006979186087846756
Validation loss = 0.006850284989923239
Validation loss = 0.007100729737430811
Validation loss = 0.006941173225641251
Validation loss = 0.006918130908161402
Validation loss = 0.006942781154066324
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 23.
Path 2 | total_timesteps 34.
Path 3 | total_timesteps 56.
Path 4 | total_timesteps 70.
Path 5 | total_timesteps 93.
Path 6 | total_timesteps 109.
Path 7 | total_timesteps 142.
Path 8 | total_timesteps 154.
Path 9 | total_timesteps 171.
Path 10 | total_timesteps 193.
Path 11 | total_timesteps 212.
Path 12 | total_timesteps 226.
Path 13 | total_timesteps 249.
Path 14 | total_timesteps 277.
Path 15 | total_timesteps 296.
Path 16 | total_timesteps 312.
Path 17 | total_timesteps 331.
Path 18 | total_timesteps 359.
Path 19 | total_timesteps 374.
Path 20 | total_timesteps 409.
Path 21 | total_timesteps 432.
Path 22 | total_timesteps 455.
Path 23 | total_timesteps 468.
Path 24 | total_timesteps 492.
Path 25 | total_timesteps 506.
Path 26 | total_timesteps 560.
Path 27 | total_timesteps 576.
Path 28 | total_timesteps 601.
Path 29 | total_timesteps 619.
Path 30 | total_timesteps 636.
Path 31 | total_timesteps 651.
Path 32 | total_timesteps 663.
Path 33 | total_timesteps 695.
Path 34 | total_timesteps 706.
Path 35 | total_timesteps 725.
Path 36 | total_timesteps 745.
Path 37 | total_timesteps 759.
Path 38 | total_timesteps 776.
Path 39 | total_timesteps 790.
Path 40 | total_timesteps 806.
Path 41 | total_timesteps 829.
Path 42 | total_timesteps 849.
Path 43 | total_timesteps 861.
Path 44 | total_timesteps 880.
Path 45 | total_timesteps 898.
Path 46 | total_timesteps 916.
Path 47 | total_timesteps 935.
Path 48 | total_timesteps 960.
Path 49 | total_timesteps 1029.
Path 50 | total_timesteps 1052.
Path 51 | total_timesteps 1076.
Path 52 | total_timesteps 1096.
Path 53 | total_timesteps 1113.
Path 54 | total_timesteps 1131.
Path 55 | total_timesteps 1153.
Path 56 | total_timesteps 1172.
Path 57 | total_timesteps 1197.
Path 58 | total_timesteps 1217.
Path 59 | total_timesteps 1236.
Path 60 | total_timesteps 1259.
Path 61 | total_timesteps 1276.
Path 62 | total_timesteps 1297.
Path 63 | total_timesteps 1311.
Path 64 | total_timesteps 1332.
Path 65 | total_timesteps 1355.
Path 66 | total_timesteps 1376.
Path 67 | total_timesteps 1397.
Path 68 | total_timesteps 1420.
Path 69 | total_timesteps 1430.
Path 70 | total_timesteps 1448.
Path 71 | total_timesteps 1467.
Path 72 | total_timesteps 1489.
Path 73 | total_timesteps 1507.
Path 74 | total_timesteps 1530.
Path 75 | total_timesteps 1549.
Path 76 | total_timesteps 1576.
Path 77 | total_timesteps 1608.
Path 78 | total_timesteps 1631.
Path 79 | total_timesteps 1650.
Path 80 | total_timesteps 1668.
Path 81 | total_timesteps 1690.
Path 82 | total_timesteps 1713.
Path 83 | total_timesteps 1752.
Path 84 | total_timesteps 1771.
Path 85 | total_timesteps 1790.
Path 86 | total_timesteps 1811.
Path 87 | total_timesteps 1832.
Path 88 | total_timesteps 1850.
Path 89 | total_timesteps 1868.
Path 90 | total_timesteps 1896.
Path 91 | total_timesteps 1915.
Path 92 | total_timesteps 1934.
Path 93 | total_timesteps 1953.
Path 94 | total_timesteps 1969.
Path 95 | total_timesteps 1994.
Path 96 | total_timesteps 2010.
Path 97 | total_timesteps 2031.
Path 98 | total_timesteps 2060.
Path 99 | total_timesteps 2080.
Path 100 | total_timesteps 2093.
Path 101 | total_timesteps 2112.
Path 102 | total_timesteps 2138.
Path 103 | total_timesteps 2160.
Path 104 | total_timesteps 2178.
Path 105 | total_timesteps 2197.
Path 106 | total_timesteps 2214.
Path 107 | total_timesteps 2231.
Path 108 | total_timesteps 2253.
Path 109 | total_timesteps 2278.
Path 110 | total_timesteps 2300.
Path 111 | total_timesteps 2318.
Path 112 | total_timesteps 2328.
Path 113 | total_timesteps 2350.
Path 114 | total_timesteps 2392.
Path 115 | total_timesteps 2419.
Path 116 | total_timesteps 2438.
Path 117 | total_timesteps 2448.
Path 118 | total_timesteps 2462.
Path 119 | total_timesteps 2481.
Path 120 | total_timesteps 2506.
Path 121 | total_timesteps 2526.
Path 122 | total_timesteps 2541.
Path 123 | total_timesteps 2557.
Path 124 | total_timesteps 2574.
Path 125 | total_timesteps 2598.
Path 126 | total_timesteps 2614.
Path 127 | total_timesteps 2637.
Path 128 | total_timesteps 2655.
Path 129 | total_timesteps 2670.
Path 130 | total_timesteps 2687.
Path 131 | total_timesteps 2706.
Path 132 | total_timesteps 2730.
Path 133 | total_timesteps 2757.
Path 134 | total_timesteps 2774.
Path 135 | total_timesteps 2793.
Path 136 | total_timesteps 2850.
Path 137 | total_timesteps 2862.
Path 138 | total_timesteps 2880.
Path 139 | total_timesteps 2898.
Path 140 | total_timesteps 2921.
Path 141 | total_timesteps 2947.
Path 142 | total_timesteps 2965.
Path 143 | total_timesteps 2990.
Path 144 | total_timesteps 3019.
Path 145 | total_timesteps 3036.
Path 146 | total_timesteps 3062.
Path 147 | total_timesteps 3078.
Path 148 | total_timesteps 3100.
Path 149 | total_timesteps 3113.
Path 150 | total_timesteps 3129.
Path 151 | total_timesteps 3143.
Path 152 | total_timesteps 3162.
Path 153 | total_timesteps 3182.
Path 154 | total_timesteps 3206.
Path 155 | total_timesteps 3235.
Path 156 | total_timesteps 3259.
Path 157 | total_timesteps 3274.
Path 158 | total_timesteps 3298.
Path 159 | total_timesteps 3315.
Path 160 | total_timesteps 3337.
Path 161 | total_timesteps 3356.
Path 162 | total_timesteps 3372.
Path 163 | total_timesteps 3389.
Path 164 | total_timesteps 3412.
Path 165 | total_timesteps 3432.
Path 166 | total_timesteps 3453.
Path 167 | total_timesteps 3472.
Path 168 | total_timesteps 3487.
Path 169 | total_timesteps 3505.
Path 170 | total_timesteps 3537.
Path 171 | total_timesteps 3554.
Path 172 | total_timesteps 3576.
Path 173 | total_timesteps 3611.
Path 174 | total_timesteps 3630.
Path 175 | total_timesteps 3649.
Path 176 | total_timesteps 3666.
Path 177 | total_timesteps 3696.
Path 178 | total_timesteps 3717.
Path 179 | total_timesteps 3737.
Path 180 | total_timesteps 3756.
Path 181 | total_timesteps 3777.
Path 182 | total_timesteps 3790.
Path 183 | total_timesteps 3814.
Path 184 | total_timesteps 3835.
Path 185 | total_timesteps 3860.
Path 186 | total_timesteps 3880.
Path 187 | total_timesteps 3903.
Path 188 | total_timesteps 3922.
Path 189 | total_timesteps 3943.
Path 190 | total_timesteps 3964.
Path 191 | total_timesteps 3989.
Path 192 | total_timesteps 4012.
Path 193 | total_timesteps 4030.
Path 194 | total_timesteps 4056.
Path 195 | total_timesteps 4078.
Path 196 | total_timesteps 4096.
Path 197 | total_timesteps 4116.
Path 198 | total_timesteps 4136.
Path 199 | total_timesteps 4154.
Path 200 | total_timesteps 4177.
Path 201 | total_timesteps 4193.
Path 202 | total_timesteps 4211.
Path 203 | total_timesteps 4223.
Path 204 | total_timesteps 4246.
Path 205 | total_timesteps 4261.
Path 206 | total_timesteps 4283.
Path 207 | total_timesteps 4309.
Path 208 | total_timesteps 4326.
Path 209 | total_timesteps 4350.
Path 210 | total_timesteps 4368.
Path 211 | total_timesteps 4386.
Path 212 | total_timesteps 4413.
Path 213 | total_timesteps 4433.
Path 214 | total_timesteps 4454.
Path 215 | total_timesteps 4467.
Path 216 | total_timesteps 4490.
Path 217 | total_timesteps 4510.
Path 218 | total_timesteps 4528.
Path 219 | total_timesteps 4553.
Path 220 | total_timesteps 4588.
Path 221 | total_timesteps 4606.
Path 222 | total_timesteps 4615.
Path 223 | total_timesteps 4631.
Path 224 | total_timesteps 4659.
Path 225 | total_timesteps 4699.
Path 226 | total_timesteps 4713.
Path 227 | total_timesteps 4734.
Path 228 | total_timesteps 4753.
Path 229 | total_timesteps 4770.
Path 230 | total_timesteps 4794.
Path 231 | total_timesteps 4815.
Path 232 | total_timesteps 4844.
Path 233 | total_timesteps 4857.
Path 234 | total_timesteps 4880.
Path 235 | total_timesteps 4917.
Path 236 | total_timesteps 4958.
Path 237 | total_timesteps 4985.
Path 238 | total_timesteps 5011.
Path 239 | total_timesteps 5035.
Path 240 | total_timesteps 5060.
Path 241 | total_timesteps 5085.
Path 242 | total_timesteps 5105.
Path 243 | total_timesteps 5123.
Path 244 | total_timesteps 5148.
Path 245 | total_timesteps 5176.
Path 246 | total_timesteps 5193.
Path 247 | total_timesteps 5218.
Path 248 | total_timesteps 5235.
Path 249 | total_timesteps 5254.
Path 250 | total_timesteps 5275.
Path 251 | total_timesteps 5289.
Path 252 | total_timesteps 5310.
Path 253 | total_timesteps 5334.
Path 254 | total_timesteps 5350.
Path 255 | total_timesteps 5367.
Path 256 | total_timesteps 5383.
Path 257 | total_timesteps 5400.
Path 258 | total_timesteps 5418.
Path 259 | total_timesteps 5436.
Path 260 | total_timesteps 5452.
Path 261 | total_timesteps 5473.
Path 262 | total_timesteps 5499.
Path 263 | total_timesteps 5520.
Path 264 | total_timesteps 5537.
Path 265 | total_timesteps 5556.
Path 266 | total_timesteps 5573.
Path 267 | total_timesteps 5594.
Path 268 | total_timesteps 5625.
Path 269 | total_timesteps 5645.
Path 270 | total_timesteps 5664.
Path 271 | total_timesteps 5682.
Path 272 | total_timesteps 5700.
Path 273 | total_timesteps 5719.
Path 274 | total_timesteps 5742.
Path 275 | total_timesteps 5760.
Path 276 | total_timesteps 5782.
Path 277 | total_timesteps 5804.
Path 278 | total_timesteps 5821.
Path 279 | total_timesteps 5843.
Path 280 | total_timesteps 5877.
Path 281 | total_timesteps 5893.
Path 282 | total_timesteps 5921.
Path 283 | total_timesteps 5953.
Path 284 | total_timesteps 5970.
Path 285 | total_timesteps 5984.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -14.4    |
| Iteration     | 20       |
| MaximumReturn | 16.1     |
| MinimumReturn | -20.9    |
| TotalSamples  | 88160    |
----------------------------
itr #21 | 
Fitting dynamics.
Validation loss = 0.007465105038136244
Validation loss = 0.006936992518603802
Validation loss = 0.006926086265593767
Validation loss = 0.006715858355164528
Validation loss = 0.006885959301143885
Validation loss = 0.00703079579398036
Validation loss = 0.0067558083683252335
Validation loss = 0.0071267662569880486
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 22.
Path 2 | total_timesteps 44.
Path 3 | total_timesteps 58.
Path 4 | total_timesteps 80.
Path 5 | total_timesteps 101.
Path 6 | total_timesteps 123.
Path 7 | total_timesteps 145.
Path 8 | total_timesteps 166.
Path 9 | total_timesteps 180.
Path 10 | total_timesteps 196.
Path 11 | total_timesteps 214.
Path 12 | total_timesteps 231.
Path 13 | total_timesteps 250.
Path 14 | total_timesteps 267.
Path 15 | total_timesteps 291.
Path 16 | total_timesteps 311.
Path 17 | total_timesteps 330.
Path 18 | total_timesteps 353.
Path 19 | total_timesteps 369.
Path 20 | total_timesteps 388.
Path 21 | total_timesteps 404.
Path 22 | total_timesteps 421.
Path 23 | total_timesteps 439.
Path 24 | total_timesteps 460.
Path 25 | total_timesteps 507.
Path 26 | total_timesteps 535.
Path 27 | total_timesteps 555.
Path 28 | total_timesteps 580.
Path 29 | total_timesteps 603.
Path 30 | total_timesteps 625.
Path 31 | total_timesteps 642.
Path 32 | total_timesteps 662.
Path 33 | total_timesteps 705.
Path 34 | total_timesteps 720.
Path 35 | total_timesteps 762.
Path 36 | total_timesteps 781.
Path 37 | total_timesteps 796.
Path 38 | total_timesteps 814.
Path 39 | total_timesteps 841.
Path 40 | total_timesteps 861.
Path 41 | total_timesteps 880.
Path 42 | total_timesteps 898.
Path 43 | total_timesteps 913.
Path 44 | total_timesteps 930.
Path 45 | total_timesteps 961.
Path 46 | total_timesteps 991.
Path 47 | total_timesteps 1012.
Path 48 | total_timesteps 1050.
Path 49 | total_timesteps 1072.
Path 50 | total_timesteps 1093.
Path 51 | total_timesteps 1111.
Path 52 | total_timesteps 1137.
Path 53 | total_timesteps 1157.
Path 54 | total_timesteps 1181.
Path 55 | total_timesteps 1195.
Path 56 | total_timesteps 1208.
Path 57 | total_timesteps 1227.
Path 58 | total_timesteps 1250.
Path 59 | total_timesteps 1267.
Path 60 | total_timesteps 1282.
Path 61 | total_timesteps 1303.
Path 62 | total_timesteps 1317.
Path 63 | total_timesteps 1353.
Path 64 | total_timesteps 1377.
Path 65 | total_timesteps 1400.
Path 66 | total_timesteps 1429.
Path 67 | total_timesteps 1450.
Path 68 | total_timesteps 1470.
Path 69 | total_timesteps 1497.
Path 70 | total_timesteps 1520.
Path 71 | total_timesteps 1536.
Path 72 | total_timesteps 1554.
Path 73 | total_timesteps 1574.
Path 74 | total_timesteps 1600.
Path 75 | total_timesteps 1623.
Path 76 | total_timesteps 1644.
Path 77 | total_timesteps 1655.
Path 78 | total_timesteps 1681.
Path 79 | total_timesteps 1698.
Path 80 | total_timesteps 1720.
Path 81 | total_timesteps 1746.
Path 82 | total_timesteps 1772.
Path 83 | total_timesteps 1793.
Path 84 | total_timesteps 1817.
Path 85 | total_timesteps 1832.
Path 86 | total_timesteps 1851.
Path 87 | total_timesteps 1870.
Path 88 | total_timesteps 1889.
Path 89 | total_timesteps 1917.
Path 90 | total_timesteps 1956.
Path 91 | total_timesteps 1978.
Path 92 | total_timesteps 1997.
Path 93 | total_timesteps 2018.
Path 94 | total_timesteps 2052.
Path 95 | total_timesteps 2069.
Path 96 | total_timesteps 2088.
Path 97 | total_timesteps 2120.
Path 98 | total_timesteps 2149.
Path 99 | total_timesteps 2168.
Path 100 | total_timesteps 2185.
Path 101 | total_timesteps 2204.
Path 102 | total_timesteps 2221.
Path 103 | total_timesteps 2243.
Path 104 | total_timesteps 2261.
Path 105 | total_timesteps 2285.
Path 106 | total_timesteps 2295.
Path 107 | total_timesteps 2311.
Path 108 | total_timesteps 2333.
Path 109 | total_timesteps 2349.
Path 110 | total_timesteps 2370.
Path 111 | total_timesteps 2389.
Path 112 | total_timesteps 2417.
Path 113 | total_timesteps 2434.
Path 114 | total_timesteps 2455.
Path 115 | total_timesteps 2472.
Path 116 | total_timesteps 2486.
Path 117 | total_timesteps 2501.
Path 118 | total_timesteps 2519.
Path 119 | total_timesteps 2536.
Path 120 | total_timesteps 2559.
Path 121 | total_timesteps 2576.
Path 122 | total_timesteps 2595.
Path 123 | total_timesteps 2623.
Path 124 | total_timesteps 2644.
Path 125 | total_timesteps 2660.
Path 126 | total_timesteps 2674.
Path 127 | total_timesteps 2694.
Path 128 | total_timesteps 2715.
Path 129 | total_timesteps 2732.
Path 130 | total_timesteps 2757.
Path 131 | total_timesteps 2775.
Path 132 | total_timesteps 2803.
Path 133 | total_timesteps 2829.
Path 134 | total_timesteps 2850.
Path 135 | total_timesteps 2871.
Path 136 | total_timesteps 2892.
Path 137 | total_timesteps 2917.
Path 138 | total_timesteps 2947.
Path 139 | total_timesteps 2971.
Path 140 | total_timesteps 2994.
Path 141 | total_timesteps 3021.
Path 142 | total_timesteps 3043.
Path 143 | total_timesteps 3061.
Path 144 | total_timesteps 3085.
Path 145 | total_timesteps 3103.
Path 146 | total_timesteps 3124.
Path 147 | total_timesteps 3141.
Path 148 | total_timesteps 3158.
Path 149 | total_timesteps 3177.
Path 150 | total_timesteps 3196.
Path 151 | total_timesteps 3217.
Path 152 | total_timesteps 3235.
Path 153 | total_timesteps 3255.
Path 154 | total_timesteps 3271.
Path 155 | total_timesteps 3293.
Path 156 | total_timesteps 3314.
Path 157 | total_timesteps 3331.
Path 158 | total_timesteps 3348.
Path 159 | total_timesteps 3366.
Path 160 | total_timesteps 3388.
Path 161 | total_timesteps 3408.
Path 162 | total_timesteps 3435.
Path 163 | total_timesteps 3458.
Path 164 | total_timesteps 3482.
Path 165 | total_timesteps 3507.
Path 166 | total_timesteps 3518.
Path 167 | total_timesteps 3546.
Path 168 | total_timesteps 3570.
Path 169 | total_timesteps 3584.
Path 170 | total_timesteps 3603.
Path 171 | total_timesteps 3623.
Path 172 | total_timesteps 3642.
Path 173 | total_timesteps 3664.
Path 174 | total_timesteps 3682.
Path 175 | total_timesteps 3707.
Path 176 | total_timesteps 3732.
Path 177 | total_timesteps 3750.
Path 178 | total_timesteps 3777.
Path 179 | total_timesteps 3806.
Path 180 | total_timesteps 3829.
Path 181 | total_timesteps 3846.
Path 182 | total_timesteps 3868.
Path 183 | total_timesteps 3891.
Path 184 | total_timesteps 3906.
Path 185 | total_timesteps 3927.
Path 186 | total_timesteps 3940.
Path 187 | total_timesteps 3962.
Path 188 | total_timesteps 3978.
Path 189 | total_timesteps 3997.
Path 190 | total_timesteps 4022.
Path 191 | total_timesteps 4039.
Path 192 | total_timesteps 4063.
Path 193 | total_timesteps 4079.
Path 194 | total_timesteps 4104.
Path 195 | total_timesteps 4125.
Path 196 | total_timesteps 4148.
Path 197 | total_timesteps 4179.
Path 198 | total_timesteps 4201.
Path 199 | total_timesteps 4221.
Path 200 | total_timesteps 4245.
Path 201 | total_timesteps 4263.
Path 202 | total_timesteps 4286.
Path 203 | total_timesteps 4304.
Path 204 | total_timesteps 4317.
Path 205 | total_timesteps 4334.
Path 206 | total_timesteps 4351.
Path 207 | total_timesteps 4371.
Path 208 | total_timesteps 4393.
Path 209 | total_timesteps 4407.
Path 210 | total_timesteps 4423.
Path 211 | total_timesteps 4442.
Path 212 | total_timesteps 4484.
Path 213 | total_timesteps 4506.
Path 214 | total_timesteps 4529.
Path 215 | total_timesteps 4550.
Path 216 | total_timesteps 4578.
Path 217 | total_timesteps 4603.
Path 218 | total_timesteps 4615.
Path 219 | total_timesteps 4637.
Path 220 | total_timesteps 4676.
Path 221 | total_timesteps 4688.
Path 222 | total_timesteps 4713.
Path 223 | total_timesteps 4722.
Path 224 | total_timesteps 4742.
Path 225 | total_timesteps 4761.
Path 226 | total_timesteps 4793.
Path 227 | total_timesteps 4815.
Path 228 | total_timesteps 4831.
Path 229 | total_timesteps 4853.
Path 230 | total_timesteps 4876.
Path 231 | total_timesteps 4893.
Path 232 | total_timesteps 4934.
Path 233 | total_timesteps 4951.
Path 234 | total_timesteps 4975.
Path 235 | total_timesteps 4996.
Path 236 | total_timesteps 5016.
Path 237 | total_timesteps 5044.
Path 238 | total_timesteps 5059.
Path 239 | total_timesteps 5074.
Path 240 | total_timesteps 5092.
Path 241 | total_timesteps 5117.
Path 242 | total_timesteps 5136.
Path 243 | total_timesteps 5165.
Path 244 | total_timesteps 5185.
Path 245 | total_timesteps 5205.
Path 246 | total_timesteps 5228.
Path 247 | total_timesteps 5239.
Path 248 | total_timesteps 5265.
Path 249 | total_timesteps 5285.
Path 250 | total_timesteps 5306.
Path 251 | total_timesteps 5351.
Path 252 | total_timesteps 5370.
Path 253 | total_timesteps 5385.
Path 254 | total_timesteps 5407.
Path 255 | total_timesteps 5425.
Path 256 | total_timesteps 5444.
Path 257 | total_timesteps 5461.
Path 258 | total_timesteps 5477.
Path 259 | total_timesteps 5534.
Path 260 | total_timesteps 5555.
Path 261 | total_timesteps 5587.
Path 262 | total_timesteps 5608.
Path 263 | total_timesteps 5632.
Path 264 | total_timesteps 5669.
Path 265 | total_timesteps 5690.
Path 266 | total_timesteps 5711.
Path 267 | total_timesteps 5733.
Path 268 | total_timesteps 5756.
Path 269 | total_timesteps 5775.
Path 270 | total_timesteps 5793.
Path 271 | total_timesteps 5813.
Path 272 | total_timesteps 5838.
Path 273 | total_timesteps 5866.
Path 274 | total_timesteps 5886.
Path 275 | total_timesteps 5907.
Path 276 | total_timesteps 5924.
Path 277 | total_timesteps 5938.
Path 278 | total_timesteps 5964.
Path 279 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -14.2    |
| Iteration     | 21       |
| MaximumReturn | 2.26     |
| MinimumReturn | -22.8    |
| TotalSamples  | 92164    |
----------------------------
itr #22 | 
Fitting dynamics.
Validation loss = 0.0072555034421384335
Validation loss = 0.0065977140329778194
Validation loss = 0.006799386348575354
Validation loss = 0.0066923885606229305
Validation loss = 0.006817324552685022
Validation loss = 0.00681332778185606
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 18.
Path 2 | total_timesteps 42.
Path 3 | total_timesteps 56.
Path 4 | total_timesteps 81.
Path 5 | total_timesteps 98.
Path 6 | total_timesteps 118.
Path 7 | total_timesteps 145.
Path 8 | total_timesteps 164.
Path 9 | total_timesteps 179.
Path 10 | total_timesteps 198.
Path 11 | total_timesteps 215.
Path 12 | total_timesteps 232.
Path 13 | total_timesteps 251.
Path 14 | total_timesteps 263.
Path 15 | total_timesteps 276.
Path 16 | total_timesteps 307.
Path 17 | total_timesteps 322.
Path 18 | total_timesteps 346.
Path 19 | total_timesteps 365.
Path 20 | total_timesteps 393.
Path 21 | total_timesteps 412.
Path 22 | total_timesteps 436.
Path 23 | total_timesteps 455.
Path 24 | total_timesteps 471.
Path 25 | total_timesteps 486.
Path 26 | total_timesteps 510.
Path 27 | total_timesteps 531.
Path 28 | total_timesteps 545.
Path 29 | total_timesteps 568.
Path 30 | total_timesteps 587.
Path 31 | total_timesteps 608.
Path 32 | total_timesteps 622.
Path 33 | total_timesteps 639.
Path 34 | total_timesteps 671.
Path 35 | total_timesteps 688.
Path 36 | total_timesteps 700.
Path 37 | total_timesteps 717.
Path 38 | total_timesteps 730.
Path 39 | total_timesteps 749.
Path 40 | total_timesteps 765.
Path 41 | total_timesteps 782.
Path 42 | total_timesteps 798.
Path 43 | total_timesteps 812.
Path 44 | total_timesteps 829.
Path 45 | total_timesteps 850.
Path 46 | total_timesteps 869.
Path 47 | total_timesteps 884.
Path 48 | total_timesteps 901.
Path 49 | total_timesteps 915.
Path 50 | total_timesteps 929.
Path 51 | total_timesteps 947.
Path 52 | total_timesteps 965.
Path 53 | total_timesteps 983.
Path 54 | total_timesteps 996.
Path 55 | total_timesteps 1014.
Path 56 | total_timesteps 1034.
Path 57 | total_timesteps 1054.
Path 58 | total_timesteps 1071.
Path 59 | total_timesteps 1087.
Path 60 | total_timesteps 1102.
Path 61 | total_timesteps 1113.
Path 62 | total_timesteps 1130.
Path 63 | total_timesteps 1160.
Path 64 | total_timesteps 1172.
Path 65 | total_timesteps 1187.
Path 66 | total_timesteps 1213.
Path 67 | total_timesteps 1224.
Path 68 | total_timesteps 1243.
Path 69 | total_timesteps 1258.
Path 70 | total_timesteps 1276.
Path 71 | total_timesteps 1303.
Path 72 | total_timesteps 1319.
Path 73 | total_timesteps 1339.
Path 74 | total_timesteps 1363.
Path 75 | total_timesteps 1382.
Path 76 | total_timesteps 1400.
Path 77 | total_timesteps 1420.
Path 78 | total_timesteps 1434.
Path 79 | total_timesteps 1459.
Path 80 | total_timesteps 1475.
Path 81 | total_timesteps 1492.
Path 82 | total_timesteps 1506.
Path 83 | total_timesteps 1527.
Path 84 | total_timesteps 1544.
Path 85 | total_timesteps 1559.
Path 86 | total_timesteps 1596.
Path 87 | total_timesteps 1614.
Path 88 | total_timesteps 1625.
Path 89 | total_timesteps 1644.
Path 90 | total_timesteps 1663.
Path 91 | total_timesteps 1678.
Path 92 | total_timesteps 1696.
Path 93 | total_timesteps 1720.
Path 94 | total_timesteps 1741.
Path 95 | total_timesteps 1759.
Path 96 | total_timesteps 1778.
Path 97 | total_timesteps 1790.
Path 98 | total_timesteps 1797.
Path 99 | total_timesteps 1813.
Path 100 | total_timesteps 1829.
Path 101 | total_timesteps 1845.
Path 102 | total_timesteps 1862.
Path 103 | total_timesteps 1877.
Path 104 | total_timesteps 1917.
Path 105 | total_timesteps 1934.
Path 106 | total_timesteps 1951.
Path 107 | total_timesteps 1967.
Path 108 | total_timesteps 1990.
Path 109 | total_timesteps 2005.
Path 110 | total_timesteps 2028.
Path 111 | total_timesteps 2051.
Path 112 | total_timesteps 2069.
Path 113 | total_timesteps 2092.
Path 114 | total_timesteps 2115.
Path 115 | total_timesteps 2138.
Path 116 | total_timesteps 2152.
Path 117 | total_timesteps 2174.
Path 118 | total_timesteps 2200.
Path 119 | total_timesteps 2231.
Path 120 | total_timesteps 2271.
Path 121 | total_timesteps 2293.
Path 122 | total_timesteps 2338.
Path 123 | total_timesteps 2356.
Path 124 | total_timesteps 2374.
Path 125 | total_timesteps 2402.
Path 126 | total_timesteps 2420.
Path 127 | total_timesteps 2439.
Path 128 | total_timesteps 2449.
Path 129 | total_timesteps 2468.
Path 130 | total_timesteps 2486.
Path 131 | total_timesteps 2510.
Path 132 | total_timesteps 2533.
Path 133 | total_timesteps 2571.
Path 134 | total_timesteps 2591.
Path 135 | total_timesteps 2611.
Path 136 | total_timesteps 2628.
Path 137 | total_timesteps 2647.
Path 138 | total_timesteps 2669.
Path 139 | total_timesteps 2690.
Path 140 | total_timesteps 2705.
Path 141 | total_timesteps 2726.
Path 142 | total_timesteps 2749.
Path 143 | total_timesteps 2775.
Path 144 | total_timesteps 2790.
Path 145 | total_timesteps 2823.
Path 146 | total_timesteps 2861.
Path 147 | total_timesteps 2877.
Path 148 | total_timesteps 2897.
Path 149 | total_timesteps 2913.
Path 150 | total_timesteps 2932.
Path 151 | total_timesteps 2963.
Path 152 | total_timesteps 2979.
Path 153 | total_timesteps 2994.
Path 154 | total_timesteps 3010.
Path 155 | total_timesteps 3029.
Path 156 | total_timesteps 3064.
Path 157 | total_timesteps 3085.
Path 158 | total_timesteps 3107.
Path 159 | total_timesteps 3129.
Path 160 | total_timesteps 3147.
Path 161 | total_timesteps 3164.
Path 162 | total_timesteps 3181.
Path 163 | total_timesteps 3198.
Path 164 | total_timesteps 3212.
Path 165 | total_timesteps 3231.
Path 166 | total_timesteps 3249.
Path 167 | total_timesteps 3262.
Path 168 | total_timesteps 3281.
Path 169 | total_timesteps 3298.
Path 170 | total_timesteps 3319.
Path 171 | total_timesteps 3332.
Path 172 | total_timesteps 3355.
Path 173 | total_timesteps 3376.
Path 174 | total_timesteps 3394.
Path 175 | total_timesteps 3415.
Path 176 | total_timesteps 3433.
Path 177 | total_timesteps 3455.
Path 178 | total_timesteps 3475.
Path 179 | total_timesteps 3499.
Path 180 | total_timesteps 3521.
Path 181 | total_timesteps 3544.
Path 182 | total_timesteps 3560.
Path 183 | total_timesteps 3576.
Path 184 | total_timesteps 3590.
Path 185 | total_timesteps 3602.
Path 186 | total_timesteps 3624.
Path 187 | total_timesteps 3665.
Path 188 | total_timesteps 3688.
Path 189 | total_timesteps 3718.
Path 190 | total_timesteps 3731.
Path 191 | total_timesteps 3749.
Path 192 | total_timesteps 3764.
Path 193 | total_timesteps 3786.
Path 194 | total_timesteps 3795.
Path 195 | total_timesteps 3809.
Path 196 | total_timesteps 3833.
Path 197 | total_timesteps 3840.
Path 198 | total_timesteps 3859.
Path 199 | total_timesteps 3886.
Path 200 | total_timesteps 3898.
Path 201 | total_timesteps 3914.
Path 202 | total_timesteps 3923.
Path 203 | total_timesteps 3937.
Path 204 | total_timesteps 3953.
Path 205 | total_timesteps 3971.
Path 206 | total_timesteps 3980.
Path 207 | total_timesteps 3998.
Path 208 | total_timesteps 4020.
Path 209 | total_timesteps 4031.
Path 210 | total_timesteps 4048.
Path 211 | total_timesteps 4072.
Path 212 | total_timesteps 4094.
Path 213 | total_timesteps 4115.
Path 214 | total_timesteps 4142.
Path 215 | total_timesteps 4159.
Path 216 | total_timesteps 4180.
Path 217 | total_timesteps 4194.
Path 218 | total_timesteps 4220.
Path 219 | total_timesteps 4239.
Path 220 | total_timesteps 4256.
Path 221 | total_timesteps 4273.
Path 222 | total_timesteps 4290.
Path 223 | total_timesteps 4308.
Path 224 | total_timesteps 4327.
Path 225 | total_timesteps 4346.
Path 226 | total_timesteps 4357.
Path 227 | total_timesteps 4371.
Path 228 | total_timesteps 4394.
Path 229 | total_timesteps 4414.
Path 230 | total_timesteps 4435.
Path 231 | total_timesteps 4451.
Path 232 | total_timesteps 4469.
Path 233 | total_timesteps 4513.
Path 234 | total_timesteps 4531.
Path 235 | total_timesteps 4544.
Path 236 | total_timesteps 4558.
Path 237 | total_timesteps 4586.
Path 238 | total_timesteps 4604.
Path 239 | total_timesteps 4620.
Path 240 | total_timesteps 4640.
Path 241 | total_timesteps 4659.
Path 242 | total_timesteps 4673.
Path 243 | total_timesteps 4690.
Path 244 | total_timesteps 4708.
Path 245 | total_timesteps 4745.
Path 246 | total_timesteps 4766.
Path 247 | total_timesteps 4782.
Path 248 | total_timesteps 4798.
Path 249 | total_timesteps 4812.
Path 250 | total_timesteps 4834.
Path 251 | total_timesteps 4854.
Path 252 | total_timesteps 4872.
Path 253 | total_timesteps 4885.
Path 254 | total_timesteps 4904.
Path 255 | total_timesteps 4920.
Path 256 | total_timesteps 4948.
Path 257 | total_timesteps 4970.
Path 258 | total_timesteps 5001.
Path 259 | total_timesteps 5038.
Path 260 | total_timesteps 5054.
Path 261 | total_timesteps 5068.
Path 262 | total_timesteps 5086.
Path 263 | total_timesteps 5105.
Path 264 | total_timesteps 5121.
Path 265 | total_timesteps 5138.
Path 266 | total_timesteps 5154.
Path 267 | total_timesteps 5173.
Path 268 | total_timesteps 5205.
Path 269 | total_timesteps 5224.
Path 270 | total_timesteps 5244.
Path 271 | total_timesteps 5255.
Path 272 | total_timesteps 5278.
Path 273 | total_timesteps 5298.
Path 274 | total_timesteps 5324.
Path 275 | total_timesteps 5340.
Path 276 | total_timesteps 5358.
Path 277 | total_timesteps 5376.
Path 278 | total_timesteps 5398.
Path 279 | total_timesteps 5417.
Path 280 | total_timesteps 5436.
Path 281 | total_timesteps 5459.
Path 282 | total_timesteps 5480.
Path 283 | total_timesteps 5503.
Path 284 | total_timesteps 5525.
Path 285 | total_timesteps 5545.
Path 286 | total_timesteps 5564.
Path 287 | total_timesteps 5583.
Path 288 | total_timesteps 5600.
Path 289 | total_timesteps 5620.
Path 290 | total_timesteps 5629.
Path 291 | total_timesteps 5643.
Path 292 | total_timesteps 5661.
Path 293 | total_timesteps 5686.
Path 294 | total_timesteps 5707.
Path 295 | total_timesteps 5727.
Path 296 | total_timesteps 5745.
Path 297 | total_timesteps 5763.
Path 298 | total_timesteps 5775.
Path 299 | total_timesteps 5792.
Path 300 | total_timesteps 5822.
Path 301 | total_timesteps 5845.
Path 302 | total_timesteps 5862.
Path 303 | total_timesteps 5879.
Path 304 | total_timesteps 5899.
Path 305 | total_timesteps 5918.
Path 306 | total_timesteps 5939.
Path 307 | total_timesteps 5949.
Path 308 | total_timesteps 5967.
Path 309 | total_timesteps 5983.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -14.3    |
| Iteration     | 22       |
| MaximumReturn | 14.9     |
| MinimumReturn | -22      |
| TotalSamples  | 96164    |
----------------------------
itr #23 | 
Fitting dynamics.
Validation loss = 0.00698776775971055
Validation loss = 0.00680851424112916
Validation loss = 0.00661858543753624
Validation loss = 0.006608044262975454
Validation loss = 0.00662971893325448
Validation loss = 0.006397288758307695
Validation loss = 0.006545720621943474
Validation loss = 0.006342811044305563
Validation loss = 0.006625266280025244
Validation loss = 0.006837282795459032
Validation loss = 0.006461307406425476
Validation loss = 0.006961135659366846
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 45.
Path 4 | total_timesteps 67.
Path 5 | total_timesteps 83.
Path 6 | total_timesteps 96.
Path 7 | total_timesteps 112.
Path 8 | total_timesteps 138.
Path 9 | total_timesteps 159.
Path 10 | total_timesteps 171.
Path 11 | total_timesteps 189.
Path 12 | total_timesteps 202.
Path 13 | total_timesteps 213.
Path 14 | total_timesteps 229.
Path 15 | total_timesteps 250.
Path 16 | total_timesteps 279.
Path 17 | total_timesteps 287.
Path 18 | total_timesteps 301.
Path 19 | total_timesteps 319.
Path 20 | total_timesteps 340.
Path 21 | total_timesteps 358.
Path 22 | total_timesteps 376.
Path 23 | total_timesteps 394.
Path 24 | total_timesteps 414.
Path 25 | total_timesteps 437.
Path 26 | total_timesteps 456.
Path 27 | total_timesteps 469.
Path 28 | total_timesteps 487.
Path 29 | total_timesteps 498.
Path 30 | total_timesteps 521.
Path 31 | total_timesteps 534.
Path 32 | total_timesteps 552.
Path 33 | total_timesteps 580.
Path 34 | total_timesteps 600.
Path 35 | total_timesteps 618.
Path 36 | total_timesteps 635.
Path 37 | total_timesteps 662.
Path 38 | total_timesteps 680.
Path 39 | total_timesteps 698.
Path 40 | total_timesteps 709.
Path 41 | total_timesteps 719.
Path 42 | total_timesteps 739.
Path 43 | total_timesteps 757.
Path 44 | total_timesteps 775.
Path 45 | total_timesteps 796.
Path 46 | total_timesteps 816.
Path 47 | total_timesteps 834.
Path 48 | total_timesteps 850.
Path 49 | total_timesteps 873.
Path 50 | total_timesteps 896.
Path 51 | total_timesteps 917.
Path 52 | total_timesteps 927.
Path 53 | total_timesteps 946.
Path 54 | total_timesteps 970.
Path 55 | total_timesteps 987.
Path 56 | total_timesteps 1002.
Path 57 | total_timesteps 1020.
Path 58 | total_timesteps 1037.
Path 59 | total_timesteps 1053.
Path 60 | total_timesteps 1065.
Path 61 | total_timesteps 1083.
Path 62 | total_timesteps 1098.
Path 63 | total_timesteps 1114.
Path 64 | total_timesteps 1140.
Path 65 | total_timesteps 1161.
Path 66 | total_timesteps 1175.
Path 67 | total_timesteps 1190.
Path 68 | total_timesteps 1211.
Path 69 | total_timesteps 1230.
Path 70 | total_timesteps 1251.
Path 71 | total_timesteps 1278.
Path 72 | total_timesteps 1291.
Path 73 | total_timesteps 1304.
Path 74 | total_timesteps 1314.
Path 75 | total_timesteps 1334.
Path 76 | total_timesteps 1355.
Path 77 | total_timesteps 1371.
Path 78 | total_timesteps 1388.
Path 79 | total_timesteps 1406.
Path 80 | total_timesteps 1421.
Path 81 | total_timesteps 1439.
Path 82 | total_timesteps 1456.
Path 83 | total_timesteps 1476.
Path 84 | total_timesteps 1484.
Path 85 | total_timesteps 1500.
Path 86 | total_timesteps 1510.
Path 87 | total_timesteps 1526.
Path 88 | total_timesteps 1540.
Path 89 | total_timesteps 1556.
Path 90 | total_timesteps 1564.
Path 91 | total_timesteps 1582.
Path 92 | total_timesteps 1601.
Path 93 | total_timesteps 1615.
Path 94 | total_timesteps 1636.
Path 95 | total_timesteps 1661.
Path 96 | total_timesteps 1671.
Path 97 | total_timesteps 1689.
Path 98 | total_timesteps 1708.
Path 99 | total_timesteps 1730.
Path 100 | total_timesteps 1749.
Path 101 | total_timesteps 1770.
Path 102 | total_timesteps 1787.
Path 103 | total_timesteps 1810.
Path 104 | total_timesteps 1830.
Path 105 | total_timesteps 1851.
Path 106 | total_timesteps 1867.
Path 107 | total_timesteps 1883.
Path 108 | total_timesteps 1896.
Path 109 | total_timesteps 1915.
Path 110 | total_timesteps 1932.
Path 111 | total_timesteps 1944.
Path 112 | total_timesteps 1959.
Path 113 | total_timesteps 1976.
Path 114 | total_timesteps 1997.
Path 115 | total_timesteps 2005.
Path 116 | total_timesteps 2019.
Path 117 | total_timesteps 2033.
Path 118 | total_timesteps 2061.
Path 119 | total_timesteps 2083.
Path 120 | total_timesteps 2095.
Path 121 | total_timesteps 2116.
Path 122 | total_timesteps 2141.
Path 123 | total_timesteps 2158.
Path 124 | total_timesteps 2183.
Path 125 | total_timesteps 2201.
Path 126 | total_timesteps 2214.
Path 127 | total_timesteps 2230.
Path 128 | total_timesteps 2249.
Path 129 | total_timesteps 2281.
Path 130 | total_timesteps 2302.
Path 131 | total_timesteps 2319.
Path 132 | total_timesteps 2331.
Path 133 | total_timesteps 2349.
Path 134 | total_timesteps 2365.
Path 135 | total_timesteps 2382.
Path 136 | total_timesteps 2401.
Path 137 | total_timesteps 2411.
Path 138 | total_timesteps 2425.
Path 139 | total_timesteps 2441.
Path 140 | total_timesteps 2455.
Path 141 | total_timesteps 2464.
Path 142 | total_timesteps 2480.
Path 143 | total_timesteps 2497.
Path 144 | total_timesteps 2514.
Path 145 | total_timesteps 2535.
Path 146 | total_timesteps 2552.
Path 147 | total_timesteps 2561.
Path 148 | total_timesteps 2578.
Path 149 | total_timesteps 2589.
Path 150 | total_timesteps 2603.
Path 151 | total_timesteps 2620.
Path 152 | total_timesteps 2635.
Path 153 | total_timesteps 2646.
Path 154 | total_timesteps 2665.
Path 155 | total_timesteps 2689.
Path 156 | total_timesteps 2702.
Path 157 | total_timesteps 2725.
Path 158 | total_timesteps 2753.
Path 159 | total_timesteps 2775.
Path 160 | total_timesteps 2797.
Path 161 | total_timesteps 2813.
Path 162 | total_timesteps 2833.
Path 163 | total_timesteps 2850.
Path 164 | total_timesteps 2866.
Path 165 | total_timesteps 2885.
Path 166 | total_timesteps 2902.
Path 167 | total_timesteps 2918.
Path 168 | total_timesteps 2938.
Path 169 | total_timesteps 2951.
Path 170 | total_timesteps 2969.
Path 171 | total_timesteps 2988.
Path 172 | total_timesteps 3008.
Path 173 | total_timesteps 3028.
Path 174 | total_timesteps 3053.
Path 175 | total_timesteps 3069.
Path 176 | total_timesteps 3096.
Path 177 | total_timesteps 3111.
Path 178 | total_timesteps 3125.
Path 179 | total_timesteps 3139.
Path 180 | total_timesteps 3150.
Path 181 | total_timesteps 3173.
Path 182 | total_timesteps 3189.
Path 183 | total_timesteps 3199.
Path 184 | total_timesteps 3223.
Path 185 | total_timesteps 3243.
Path 186 | total_timesteps 3258.
Path 187 | total_timesteps 3275.
Path 188 | total_timesteps 3292.
Path 189 | total_timesteps 3309.
Path 190 | total_timesteps 3334.
Path 191 | total_timesteps 3348.
Path 192 | total_timesteps 3372.
Path 193 | total_timesteps 3389.
Path 194 | total_timesteps 3408.
Path 195 | total_timesteps 3426.
Path 196 | total_timesteps 3438.
Path 197 | total_timesteps 3451.
Path 198 | total_timesteps 3463.
Path 199 | total_timesteps 3479.
Path 200 | total_timesteps 3497.
Path 201 | total_timesteps 3520.
Path 202 | total_timesteps 3533.
Path 203 | total_timesteps 3547.
Path 204 | total_timesteps 3566.
Path 205 | total_timesteps 3579.
Path 206 | total_timesteps 3602.
Path 207 | total_timesteps 3624.
Path 208 | total_timesteps 3649.
Path 209 | total_timesteps 3669.
Path 210 | total_timesteps 3690.
Path 211 | total_timesteps 3701.
Path 212 | total_timesteps 3717.
Path 213 | total_timesteps 3733.
Path 214 | total_timesteps 3754.
Path 215 | total_timesteps 3767.
Path 216 | total_timesteps 3782.
Path 217 | total_timesteps 3790.
Path 218 | total_timesteps 3809.
Path 219 | total_timesteps 3826.
Path 220 | total_timesteps 3846.
Path 221 | total_timesteps 3862.
Path 222 | total_timesteps 3886.
Path 223 | total_timesteps 3909.
Path 224 | total_timesteps 3927.
Path 225 | total_timesteps 3947.
Path 226 | total_timesteps 3973.
Path 227 | total_timesteps 3989.
Path 228 | total_timesteps 4001.
Path 229 | total_timesteps 4020.
Path 230 | total_timesteps 4038.
Path 231 | total_timesteps 4053.
Path 232 | total_timesteps 4066.
Path 233 | total_timesteps 4081.
Path 234 | total_timesteps 4098.
Path 235 | total_timesteps 4115.
Path 236 | total_timesteps 4132.
Path 237 | total_timesteps 4152.
Path 238 | total_timesteps 4171.
Path 239 | total_timesteps 4194.
Path 240 | total_timesteps 4214.
Path 241 | total_timesteps 4224.
Path 242 | total_timesteps 4245.
Path 243 | total_timesteps 4267.
Path 244 | total_timesteps 4282.
Path 245 | total_timesteps 4295.
Path 246 | total_timesteps 4310.
Path 247 | total_timesteps 4328.
Path 248 | total_timesteps 4348.
Path 249 | total_timesteps 4367.
Path 250 | total_timesteps 4387.
Path 251 | total_timesteps 4407.
Path 252 | total_timesteps 4425.
Path 253 | total_timesteps 4444.
Path 254 | total_timesteps 4466.
Path 255 | total_timesteps 4483.
Path 256 | total_timesteps 4501.
Path 257 | total_timesteps 4522.
Path 258 | total_timesteps 4542.
Path 259 | total_timesteps 4554.
Path 260 | total_timesteps 4569.
Path 261 | total_timesteps 4589.
Path 262 | total_timesteps 4603.
Path 263 | total_timesteps 4625.
Path 264 | total_timesteps 4640.
Path 265 | total_timesteps 4654.
Path 266 | total_timesteps 4682.
Path 267 | total_timesteps 4703.
Path 268 | total_timesteps 4717.
Path 269 | total_timesteps 4738.
Path 270 | total_timesteps 4758.
Path 271 | total_timesteps 4774.
Path 272 | total_timesteps 4790.
Path 273 | total_timesteps 4811.
Path 274 | total_timesteps 4829.
Path 275 | total_timesteps 4849.
Path 276 | total_timesteps 4865.
Path 277 | total_timesteps 4884.
Path 278 | total_timesteps 4898.
Path 279 | total_timesteps 4915.
Path 280 | total_timesteps 4931.
Path 281 | total_timesteps 4944.
Path 282 | total_timesteps 4965.
Path 283 | total_timesteps 4985.
Path 284 | total_timesteps 5012.
Path 285 | total_timesteps 5034.
Path 286 | total_timesteps 5058.
Path 287 | total_timesteps 5080.
Path 288 | total_timesteps 5091.
Path 289 | total_timesteps 5116.
Path 290 | total_timesteps 5128.
Path 291 | total_timesteps 5140.
Path 292 | total_timesteps 5162.
Path 293 | total_timesteps 5174.
Path 294 | total_timesteps 5191.
Path 295 | total_timesteps 5207.
Path 296 | total_timesteps 5224.
Path 297 | total_timesteps 5239.
Path 298 | total_timesteps 5253.
Path 299 | total_timesteps 5267.
Path 300 | total_timesteps 5289.
Path 301 | total_timesteps 5308.
Path 302 | total_timesteps 5327.
Path 303 | total_timesteps 5345.
Path 304 | total_timesteps 5358.
Path 305 | total_timesteps 5370.
Path 306 | total_timesteps 5391.
Path 307 | total_timesteps 5408.
Path 308 | total_timesteps 5428.
Path 309 | total_timesteps 5446.
Path 310 | total_timesteps 5466.
Path 311 | total_timesteps 5491.
Path 312 | total_timesteps 5518.
Path 313 | total_timesteps 5532.
Path 314 | total_timesteps 5545.
Path 315 | total_timesteps 5561.
Path 316 | total_timesteps 5572.
Path 317 | total_timesteps 5602.
Path 318 | total_timesteps 5618.
Path 319 | total_timesteps 5634.
Path 320 | total_timesteps 5643.
Path 321 | total_timesteps 5662.
Path 322 | total_timesteps 5679.
Path 323 | total_timesteps 5694.
Path 324 | total_timesteps 5714.
Path 325 | total_timesteps 5737.
Path 326 | total_timesteps 5750.
Path 327 | total_timesteps 5764.
Path 328 | total_timesteps 5779.
Path 329 | total_timesteps 5801.
Path 330 | total_timesteps 5815.
Path 331 | total_timesteps 5828.
Path 332 | total_timesteps 5845.
Path 333 | total_timesteps 5870.
Path 334 | total_timesteps 5891.
Path 335 | total_timesteps 5927.
Path 336 | total_timesteps 5943.
Path 337 | total_timesteps 5955.
Path 338 | total_timesteps 5976.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -14.2    |
| Iteration     | 23       |
| MaximumReturn | -2.18    |
| MinimumReturn | -23.5    |
| TotalSamples  | 100164   |
----------------------------
itr #24 | 
Fitting dynamics.
Validation loss = 0.0067596714943647385
Validation loss = 0.0062587689608335495
Validation loss = 0.006216854322701693
Validation loss = 0.006439998280256987
Validation loss = 0.006370060611516237
Validation loss = 0.006193406414240599
Validation loss = 0.006261540111154318
Validation loss = 0.0059448229148983955
Validation loss = 0.006606892216950655
Validation loss = 0.006221754476428032
Validation loss = 0.005989480763673782
Validation loss = 0.006274594459682703
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 38.
Path 4 | total_timesteps 53.
Path 5 | total_timesteps 75.
Path 6 | total_timesteps 92.
Path 7 | total_timesteps 104.
Path 8 | total_timesteps 121.
Path 9 | total_timesteps 136.
Path 10 | total_timesteps 153.
Path 11 | total_timesteps 169.
Path 12 | total_timesteps 186.
Path 13 | total_timesteps 199.
Path 14 | total_timesteps 211.
Path 15 | total_timesteps 222.
Path 16 | total_timesteps 240.
Path 17 | total_timesteps 260.
Path 18 | total_timesteps 275.
Path 19 | total_timesteps 291.
Path 20 | total_timesteps 305.
Path 21 | total_timesteps 329.
Path 22 | total_timesteps 345.
Path 23 | total_timesteps 361.
Path 24 | total_timesteps 375.
Path 25 | total_timesteps 393.
Path 26 | total_timesteps 412.
Path 27 | total_timesteps 428.
Path 28 | total_timesteps 448.
Path 29 | total_timesteps 470.
Path 30 | total_timesteps 490.
Path 31 | total_timesteps 510.
Path 32 | total_timesteps 523.
Path 33 | total_timesteps 535.
Path 34 | total_timesteps 552.
Path 35 | total_timesteps 567.
Path 36 | total_timesteps 580.
Path 37 | total_timesteps 597.
Path 38 | total_timesteps 618.
Path 39 | total_timesteps 629.
Path 40 | total_timesteps 647.
Path 41 | total_timesteps 667.
Path 42 | total_timesteps 678.
Path 43 | total_timesteps 692.
Path 44 | total_timesteps 711.
Path 45 | total_timesteps 726.
Path 46 | total_timesteps 744.
Path 47 | total_timesteps 767.
Path 48 | total_timesteps 781.
Path 49 | total_timesteps 795.
Path 50 | total_timesteps 814.
Path 51 | total_timesteps 832.
Path 52 | total_timesteps 842.
Path 53 | total_timesteps 862.
Path 54 | total_timesteps 875.
Path 55 | total_timesteps 890.
Path 56 | total_timesteps 907.
Path 57 | total_timesteps 927.
Path 58 | total_timesteps 948.
Path 59 | total_timesteps 963.
Path 60 | total_timesteps 982.
Path 61 | total_timesteps 1002.
Path 62 | total_timesteps 1018.
Path 63 | total_timesteps 1028.
Path 64 | total_timesteps 1046.
Path 65 | total_timesteps 1060.
Path 66 | total_timesteps 1069.
Path 67 | total_timesteps 1082.
Path 68 | total_timesteps 1097.
Path 69 | total_timesteps 1106.
Path 70 | total_timesteps 1119.
Path 71 | total_timesteps 1137.
Path 72 | total_timesteps 1155.
Path 73 | total_timesteps 1171.
Path 74 | total_timesteps 1195.
Path 75 | total_timesteps 1212.
Path 76 | total_timesteps 1229.
Path 77 | total_timesteps 1239.
Path 78 | total_timesteps 1261.
Path 79 | total_timesteps 1274.
Path 80 | total_timesteps 1292.
Path 81 | total_timesteps 1310.
Path 82 | total_timesteps 1328.
Path 83 | total_timesteps 1341.
Path 84 | total_timesteps 1357.
Path 85 | total_timesteps 1371.
Path 86 | total_timesteps 1386.
Path 87 | total_timesteps 1399.
Path 88 | total_timesteps 1427.
Path 89 | total_timesteps 1444.
Path 90 | total_timesteps 1461.
Path 91 | total_timesteps 1476.
Path 92 | total_timesteps 1492.
Path 93 | total_timesteps 1505.
Path 94 | total_timesteps 1522.
Path 95 | total_timesteps 1538.
Path 96 | total_timesteps 1557.
Path 97 | total_timesteps 1571.
Path 98 | total_timesteps 1583.
Path 99 | total_timesteps 1597.
Path 100 | total_timesteps 1609.
Path 101 | total_timesteps 1625.
Path 102 | total_timesteps 1639.
Path 103 | total_timesteps 1655.
Path 104 | total_timesteps 1670.
Path 105 | total_timesteps 1682.
Path 106 | total_timesteps 1700.
Path 107 | total_timesteps 1711.
Path 108 | total_timesteps 1729.
Path 109 | total_timesteps 1746.
Path 110 | total_timesteps 1759.
Path 111 | total_timesteps 1772.
Path 112 | total_timesteps 1799.
Path 113 | total_timesteps 1806.
Path 114 | total_timesteps 1825.
Path 115 | total_timesteps 1841.
Path 116 | total_timesteps 1854.
Path 117 | total_timesteps 1867.
Path 118 | total_timesteps 1882.
Path 119 | total_timesteps 1896.
Path 120 | total_timesteps 1915.
Path 121 | total_timesteps 1935.
Path 122 | total_timesteps 1961.
Path 123 | total_timesteps 1984.
Path 124 | total_timesteps 2001.
Path 125 | total_timesteps 2020.
Path 126 | total_timesteps 2028.
Path 127 | total_timesteps 2044.
Path 128 | total_timesteps 2064.
Path 129 | total_timesteps 2081.
Path 130 | total_timesteps 2095.
Path 131 | total_timesteps 2111.
Path 132 | total_timesteps 2132.
Path 133 | total_timesteps 2150.
Path 134 | total_timesteps 2164.
Path 135 | total_timesteps 2176.
Path 136 | total_timesteps 2189.
Path 137 | total_timesteps 2206.
Path 138 | total_timesteps 2226.
Path 139 | total_timesteps 2245.
Path 140 | total_timesteps 2262.
Path 141 | total_timesteps 2277.
Path 142 | total_timesteps 2288.
Path 143 | total_timesteps 2301.
Path 144 | total_timesteps 2318.
Path 145 | total_timesteps 2337.
Path 146 | total_timesteps 2352.
Path 147 | total_timesteps 2366.
Path 148 | total_timesteps 2384.
Path 149 | total_timesteps 2405.
Path 150 | total_timesteps 2426.
Path 151 | total_timesteps 2440.
Path 152 | total_timesteps 2458.
Path 153 | total_timesteps 2478.
Path 154 | total_timesteps 2495.
Path 155 | total_timesteps 2514.
Path 156 | total_timesteps 2529.
Path 157 | total_timesteps 2542.
Path 158 | total_timesteps 2558.
Path 159 | total_timesteps 2572.
Path 160 | total_timesteps 2586.
Path 161 | total_timesteps 2601.
Path 162 | total_timesteps 2620.
Path 163 | total_timesteps 2636.
Path 164 | total_timesteps 2652.
Path 165 | total_timesteps 2661.
Path 166 | total_timesteps 2674.
Path 167 | total_timesteps 2690.
Path 168 | total_timesteps 2704.
Path 169 | total_timesteps 2720.
Path 170 | total_timesteps 2735.
Path 171 | total_timesteps 2743.
Path 172 | total_timesteps 2758.
Path 173 | total_timesteps 2778.
Path 174 | total_timesteps 2797.
Path 175 | total_timesteps 2811.
Path 176 | total_timesteps 2819.
Path 177 | total_timesteps 2846.
Path 178 | total_timesteps 2863.
Path 179 | total_timesteps 2883.
Path 180 | total_timesteps 2900.
Path 181 | total_timesteps 2925.
Path 182 | total_timesteps 2937.
Path 183 | total_timesteps 2954.
Path 184 | total_timesteps 2968.
Path 185 | total_timesteps 2989.
Path 186 | total_timesteps 3007.
Path 187 | total_timesteps 3021.
Path 188 | total_timesteps 3049.
Path 189 | total_timesteps 3062.
Path 190 | total_timesteps 3083.
Path 191 | total_timesteps 3096.
Path 192 | total_timesteps 3114.
Path 193 | total_timesteps 3132.
Path 194 | total_timesteps 3152.
Path 195 | total_timesteps 3168.
Path 196 | total_timesteps 3187.
Path 197 | total_timesteps 3203.
Path 198 | total_timesteps 3219.
Path 199 | total_timesteps 3237.
Path 200 | total_timesteps 3257.
Path 201 | total_timesteps 3276.
Path 202 | total_timesteps 3299.
Path 203 | total_timesteps 3310.
Path 204 | total_timesteps 3319.
Path 205 | total_timesteps 3336.
Path 206 | total_timesteps 3351.
Path 207 | total_timesteps 3372.
Path 208 | total_timesteps 3395.
Path 209 | total_timesteps 3415.
Path 210 | total_timesteps 3433.
Path 211 | total_timesteps 3446.
Path 212 | total_timesteps 3474.
Path 213 | total_timesteps 3490.
Path 214 | total_timesteps 3504.
Path 215 | total_timesteps 3524.
Path 216 | total_timesteps 3545.
Path 217 | total_timesteps 3556.
Path 218 | total_timesteps 3571.
Path 219 | total_timesteps 3583.
Path 220 | total_timesteps 3597.
Path 221 | total_timesteps 3615.
Path 222 | total_timesteps 3625.
Path 223 | total_timesteps 3644.
Path 224 | total_timesteps 3667.
Path 225 | total_timesteps 3687.
Path 226 | total_timesteps 3696.
Path 227 | total_timesteps 3712.
Path 228 | total_timesteps 3729.
Path 229 | total_timesteps 3745.
Path 230 | total_timesteps 3763.
Path 231 | total_timesteps 3776.
Path 232 | total_timesteps 3787.
Path 233 | total_timesteps 3803.
Path 234 | total_timesteps 3813.
Path 235 | total_timesteps 3824.
Path 236 | total_timesteps 3843.
Path 237 | total_timesteps 3851.
Path 238 | total_timesteps 3863.
Path 239 | total_timesteps 3879.
Path 240 | total_timesteps 3897.
Path 241 | total_timesteps 3916.
Path 242 | total_timesteps 3936.
Path 243 | total_timesteps 3945.
Path 244 | total_timesteps 3958.
Path 245 | total_timesteps 3973.
Path 246 | total_timesteps 3987.
Path 247 | total_timesteps 4016.
Path 248 | total_timesteps 4034.
Path 249 | total_timesteps 4052.
Path 250 | total_timesteps 4067.
Path 251 | total_timesteps 4087.
Path 252 | total_timesteps 4105.
Path 253 | total_timesteps 4121.
Path 254 | total_timesteps 4137.
Path 255 | total_timesteps 4159.
Path 256 | total_timesteps 4181.
Path 257 | total_timesteps 4196.
Path 258 | total_timesteps 4215.
Path 259 | total_timesteps 4234.
Path 260 | total_timesteps 4246.
Path 261 | total_timesteps 4264.
Path 262 | total_timesteps 4275.
Path 263 | total_timesteps 4289.
Path 264 | total_timesteps 4305.
Path 265 | total_timesteps 4325.
Path 266 | total_timesteps 4345.
Path 267 | total_timesteps 4360.
Path 268 | total_timesteps 4374.
Path 269 | total_timesteps 4395.
Path 270 | total_timesteps 4408.
Path 271 | total_timesteps 4428.
Path 272 | total_timesteps 4441.
Path 273 | total_timesteps 4456.
Path 274 | total_timesteps 4476.
Path 275 | total_timesteps 4489.
Path 276 | total_timesteps 4503.
Path 277 | total_timesteps 4526.
Path 278 | total_timesteps 4547.
Path 279 | total_timesteps 4558.
Path 280 | total_timesteps 4572.
Path 281 | total_timesteps 4583.
Path 282 | total_timesteps 4599.
Path 283 | total_timesteps 4622.
Path 284 | total_timesteps 4634.
Path 285 | total_timesteps 4649.
Path 286 | total_timesteps 4659.
Path 287 | total_timesteps 4673.
Path 288 | total_timesteps 4683.
Path 289 | total_timesteps 4697.
Path 290 | total_timesteps 4707.
Path 291 | total_timesteps 4719.
Path 292 | total_timesteps 4736.
Path 293 | total_timesteps 4758.
Path 294 | total_timesteps 4772.
Path 295 | total_timesteps 4793.
Path 296 | total_timesteps 4811.
Path 297 | total_timesteps 4825.
Path 298 | total_timesteps 4839.
Path 299 | total_timesteps 4851.
Path 300 | total_timesteps 4874.
Path 301 | total_timesteps 4890.
Path 302 | total_timesteps 4906.
Path 303 | total_timesteps 4918.
Path 304 | total_timesteps 4934.
Path 305 | total_timesteps 4949.
Path 306 | total_timesteps 4958.
Path 307 | total_timesteps 4974.
Path 308 | total_timesteps 4995.
Path 309 | total_timesteps 5025.
Path 310 | total_timesteps 5042.
Path 311 | total_timesteps 5065.
Path 312 | total_timesteps 5082.
Path 313 | total_timesteps 5101.
Path 314 | total_timesteps 5115.
Path 315 | total_timesteps 5135.
Path 316 | total_timesteps 5166.
Path 317 | total_timesteps 5185.
Path 318 | total_timesteps 5198.
Path 319 | total_timesteps 5213.
Path 320 | total_timesteps 5233.
Path 321 | total_timesteps 5247.
Path 322 | total_timesteps 5270.
Path 323 | total_timesteps 5294.
Path 324 | total_timesteps 5308.
Path 325 | total_timesteps 5328.
Path 326 | total_timesteps 5341.
Path 327 | total_timesteps 5356.
Path 328 | total_timesteps 5368.
Path 329 | total_timesteps 5388.
Path 330 | total_timesteps 5399.
Path 331 | total_timesteps 5410.
Path 332 | total_timesteps 5447.
Path 333 | total_timesteps 5465.
Path 334 | total_timesteps 5481.
Path 335 | total_timesteps 5499.
Path 336 | total_timesteps 5513.
Path 337 | total_timesteps 5530.
Path 338 | total_timesteps 5542.
Path 339 | total_timesteps 5557.
Path 340 | total_timesteps 5570.
Path 341 | total_timesteps 5591.
Path 342 | total_timesteps 5608.
Path 343 | total_timesteps 5635.
Path 344 | total_timesteps 5648.
Path 345 | total_timesteps 5659.
Path 346 | total_timesteps 5675.
Path 347 | total_timesteps 5698.
Path 348 | total_timesteps 5717.
Path 349 | total_timesteps 5744.
Path 350 | total_timesteps 5763.
Path 351 | total_timesteps 5788.
Path 352 | total_timesteps 5813.
Path 353 | total_timesteps 5831.
Path 354 | total_timesteps 5843.
Path 355 | total_timesteps 5857.
Path 356 | total_timesteps 5877.
Path 357 | total_timesteps 5895.
Path 358 | total_timesteps 5913.
Path 359 | total_timesteps 5936.
Path 360 | total_timesteps 5947.
Path 361 | total_timesteps 5961.
Path 362 | total_timesteps 5971.
Path 363 | total_timesteps 5989.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -13.7    |
| Iteration     | 24       |
| MaximumReturn | -1.3     |
| MinimumReturn | -22.7    |
| TotalSamples  | 104164   |
----------------------------
itr #25 | 
Fitting dynamics.
Validation loss = 0.0062773944810032845
Validation loss = 0.006007892545312643
Validation loss = 0.005965598858892918
Validation loss = 0.006116711534559727
Validation loss = 0.006010320503264666
Validation loss = 0.0063147712498903275
Validation loss = 0.00635311845690012
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 18.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 61.
Path 5 | total_timesteps 75.
Path 6 | total_timesteps 87.
Path 7 | total_timesteps 96.
Path 8 | total_timesteps 117.
Path 9 | total_timesteps 130.
Path 10 | total_timesteps 147.
Path 11 | total_timesteps 160.
Path 12 | total_timesteps 174.
Path 13 | total_timesteps 191.
Path 14 | total_timesteps 210.
Path 15 | total_timesteps 229.
Path 16 | total_timesteps 249.
Path 17 | total_timesteps 267.
Path 18 | total_timesteps 282.
Path 19 | total_timesteps 297.
Path 20 | total_timesteps 316.
Path 21 | total_timesteps 334.
Path 22 | total_timesteps 356.
Path 23 | total_timesteps 370.
Path 24 | total_timesteps 389.
Path 25 | total_timesteps 404.
Path 26 | total_timesteps 422.
Path 27 | total_timesteps 438.
Path 28 | total_timesteps 457.
Path 29 | total_timesteps 480.
Path 30 | total_timesteps 500.
Path 31 | total_timesteps 513.
Path 32 | total_timesteps 526.
Path 33 | total_timesteps 545.
Path 34 | total_timesteps 558.
Path 35 | total_timesteps 569.
Path 36 | total_timesteps 580.
Path 37 | total_timesteps 598.
Path 38 | total_timesteps 613.
Path 39 | total_timesteps 634.
Path 40 | total_timesteps 651.
Path 41 | total_timesteps 667.
Path 42 | total_timesteps 685.
Path 43 | total_timesteps 701.
Path 44 | total_timesteps 715.
Path 45 | total_timesteps 733.
Path 46 | total_timesteps 747.
Path 47 | total_timesteps 759.
Path 48 | total_timesteps 770.
Path 49 | total_timesteps 787.
Path 50 | total_timesteps 800.
Path 51 | total_timesteps 817.
Path 52 | total_timesteps 831.
Path 53 | total_timesteps 851.
Path 54 | total_timesteps 867.
Path 55 | total_timesteps 881.
Path 56 | total_timesteps 896.
Path 57 | total_timesteps 911.
Path 58 | total_timesteps 920.
Path 59 | total_timesteps 936.
Path 60 | total_timesteps 953.
Path 61 | total_timesteps 976.
Path 62 | total_timesteps 992.
Path 63 | total_timesteps 1012.
Path 64 | total_timesteps 1028.
Path 65 | total_timesteps 1043.
Path 66 | total_timesteps 1059.
Path 67 | total_timesteps 1076.
Path 68 | total_timesteps 1095.
Path 69 | total_timesteps 1104.
Path 70 | total_timesteps 1124.
Path 71 | total_timesteps 1139.
Path 72 | total_timesteps 1163.
Path 73 | total_timesteps 1178.
Path 74 | total_timesteps 1195.
Path 75 | total_timesteps 1211.
Path 76 | total_timesteps 1222.
Path 77 | total_timesteps 1233.
Path 78 | total_timesteps 1248.
Path 79 | total_timesteps 1259.
Path 80 | total_timesteps 1281.
Path 81 | total_timesteps 1307.
Path 82 | total_timesteps 1328.
Path 83 | total_timesteps 1341.
Path 84 | total_timesteps 1356.
Path 85 | total_timesteps 1371.
Path 86 | total_timesteps 1387.
Path 87 | total_timesteps 1400.
Path 88 | total_timesteps 1410.
Path 89 | total_timesteps 1423.
Path 90 | total_timesteps 1438.
Path 91 | total_timesteps 1457.
Path 92 | total_timesteps 1472.
Path 93 | total_timesteps 1488.
Path 94 | total_timesteps 1510.
Path 95 | total_timesteps 1528.
Path 96 | total_timesteps 1548.
Path 97 | total_timesteps 1560.
Path 98 | total_timesteps 1570.
Path 99 | total_timesteps 1584.
Path 100 | total_timesteps 1594.
Path 101 | total_timesteps 1616.
Path 102 | total_timesteps 1632.
Path 103 | total_timesteps 1642.
Path 104 | total_timesteps 1663.
Path 105 | total_timesteps 1678.
Path 106 | total_timesteps 1701.
Path 107 | total_timesteps 1723.
Path 108 | total_timesteps 1739.
Path 109 | total_timesteps 1755.
Path 110 | total_timesteps 1778.
Path 111 | total_timesteps 1791.
Path 112 | total_timesteps 1802.
Path 113 | total_timesteps 1816.
Path 114 | total_timesteps 1834.
Path 115 | total_timesteps 1855.
Path 116 | total_timesteps 1869.
Path 117 | total_timesteps 1895.
Path 118 | total_timesteps 1915.
Path 119 | total_timesteps 1928.
Path 120 | total_timesteps 1943.
Path 121 | total_timesteps 1962.
Path 122 | total_timesteps 1977.
Path 123 | total_timesteps 1990.
Path 124 | total_timesteps 2002.
Path 125 | total_timesteps 2010.
Path 126 | total_timesteps 2023.
Path 127 | total_timesteps 2038.
Path 128 | total_timesteps 2051.
Path 129 | total_timesteps 2066.
Path 130 | total_timesteps 2081.
Path 131 | total_timesteps 2110.
Path 132 | total_timesteps 2127.
Path 133 | total_timesteps 2145.
Path 134 | total_timesteps 2160.
Path 135 | total_timesteps 2174.
Path 136 | total_timesteps 2185.
Path 137 | total_timesteps 2211.
Path 138 | total_timesteps 2224.
Path 139 | total_timesteps 2240.
Path 140 | total_timesteps 2250.
Path 141 | total_timesteps 2261.
Path 142 | total_timesteps 2283.
Path 143 | total_timesteps 2298.
Path 144 | total_timesteps 2313.
Path 145 | total_timesteps 2321.
Path 146 | total_timesteps 2336.
Path 147 | total_timesteps 2350.
Path 148 | total_timesteps 2363.
Path 149 | total_timesteps 2382.
Path 150 | total_timesteps 2399.
Path 151 | total_timesteps 2414.
Path 152 | total_timesteps 2429.
Path 153 | total_timesteps 2445.
Path 154 | total_timesteps 2459.
Path 155 | total_timesteps 2473.
Path 156 | total_timesteps 2484.
Path 157 | total_timesteps 2499.
Path 158 | total_timesteps 2510.
Path 159 | total_timesteps 2527.
Path 160 | total_timesteps 2541.
Path 161 | total_timesteps 2563.
Path 162 | total_timesteps 2572.
Path 163 | total_timesteps 2591.
Path 164 | total_timesteps 2621.
Path 165 | total_timesteps 2636.
Path 166 | total_timesteps 2651.
Path 167 | total_timesteps 2663.
Path 168 | total_timesteps 2684.
Path 169 | total_timesteps 2706.
Path 170 | total_timesteps 2723.
Path 171 | total_timesteps 2737.
Path 172 | total_timesteps 2745.
Path 173 | total_timesteps 2762.
Path 174 | total_timesteps 2782.
Path 175 | total_timesteps 2805.
Path 176 | total_timesteps 2820.
Path 177 | total_timesteps 2841.
Path 178 | total_timesteps 2871.
Path 179 | total_timesteps 2887.
Path 180 | total_timesteps 2900.
Path 181 | total_timesteps 2918.
Path 182 | total_timesteps 2977.
Path 183 | total_timesteps 2993.
Path 184 | total_timesteps 3011.
Path 185 | total_timesteps 3030.
Path 186 | total_timesteps 3039.
Path 187 | total_timesteps 3051.
Path 188 | total_timesteps 3080.
Path 189 | total_timesteps 3095.
Path 190 | total_timesteps 3108.
Path 191 | total_timesteps 3126.
Path 192 | total_timesteps 3142.
Path 193 | total_timesteps 3159.
Path 194 | total_timesteps 3175.
Path 195 | total_timesteps 3192.
Path 196 | total_timesteps 3210.
Path 197 | total_timesteps 3228.
Path 198 | total_timesteps 3246.
Path 199 | total_timesteps 3260.
Path 200 | total_timesteps 3279.
Path 201 | total_timesteps 3293.
Path 202 | total_timesteps 3304.
Path 203 | total_timesteps 3315.
Path 204 | total_timesteps 3332.
Path 205 | total_timesteps 3352.
Path 206 | total_timesteps 3374.
Path 207 | total_timesteps 3391.
Path 208 | total_timesteps 3404.
Path 209 | total_timesteps 3423.
Path 210 | total_timesteps 3449.
Path 211 | total_timesteps 3464.
Path 212 | total_timesteps 3475.
Path 213 | total_timesteps 3492.
Path 214 | total_timesteps 3506.
Path 215 | total_timesteps 3525.
Path 216 | total_timesteps 3538.
Path 217 | total_timesteps 3557.
Path 218 | total_timesteps 3570.
Path 219 | total_timesteps 3586.
Path 220 | total_timesteps 3616.
Path 221 | total_timesteps 3631.
Path 222 | total_timesteps 3644.
Path 223 | total_timesteps 3657.
Path 224 | total_timesteps 3673.
Path 225 | total_timesteps 3681.
Path 226 | total_timesteps 3698.
Path 227 | total_timesteps 3712.
Path 228 | total_timesteps 3725.
Path 229 | total_timesteps 3739.
Path 230 | total_timesteps 3752.
Path 231 | total_timesteps 3769.
Path 232 | total_timesteps 3783.
Path 233 | total_timesteps 3804.
Path 234 | total_timesteps 3823.
Path 235 | total_timesteps 3838.
Path 236 | total_timesteps 3858.
Path 237 | total_timesteps 3875.
Path 238 | total_timesteps 3890.
Path 239 | total_timesteps 3903.
Path 240 | total_timesteps 3926.
Path 241 | total_timesteps 3941.
Path 242 | total_timesteps 3964.
Path 243 | total_timesteps 3985.
Path 244 | total_timesteps 4008.
Path 245 | total_timesteps 4027.
Path 246 | total_timesteps 4041.
Path 247 | total_timesteps 4055.
Path 248 | total_timesteps 4076.
Path 249 | total_timesteps 4095.
Path 250 | total_timesteps 4113.
Path 251 | total_timesteps 4131.
Path 252 | total_timesteps 4151.
Path 253 | total_timesteps 4163.
Path 254 | total_timesteps 4177.
Path 255 | total_timesteps 4186.
Path 256 | total_timesteps 4199.
Path 257 | total_timesteps 4216.
Path 258 | total_timesteps 4232.
Path 259 | total_timesteps 4247.
Path 260 | total_timesteps 4264.
Path 261 | total_timesteps 4277.
Path 262 | total_timesteps 4292.
Path 263 | total_timesteps 4305.
Path 264 | total_timesteps 4319.
Path 265 | total_timesteps 4331.
Path 266 | total_timesteps 4348.
Path 267 | total_timesteps 4368.
Path 268 | total_timesteps 4385.
Path 269 | total_timesteps 4401.
Path 270 | total_timesteps 4416.
Path 271 | total_timesteps 4445.
Path 272 | total_timesteps 4464.
Path 273 | total_timesteps 4481.
Path 274 | total_timesteps 4502.
Path 275 | total_timesteps 4522.
Path 276 | total_timesteps 4541.
Path 277 | total_timesteps 4558.
Path 278 | total_timesteps 4573.
Path 279 | total_timesteps 4590.
Path 280 | total_timesteps 4603.
Path 281 | total_timesteps 4618.
Path 282 | total_timesteps 4637.
Path 283 | total_timesteps 4655.
Path 284 | total_timesteps 4673.
Path 285 | total_timesteps 4689.
Path 286 | total_timesteps 4700.
Path 287 | total_timesteps 4715.
Path 288 | total_timesteps 4730.
Path 289 | total_timesteps 4747.
Path 290 | total_timesteps 4760.
Path 291 | total_timesteps 4772.
Path 292 | total_timesteps 4795.
Path 293 | total_timesteps 4813.
Path 294 | total_timesteps 4829.
Path 295 | total_timesteps 4845.
Path 296 | total_timesteps 4859.
Path 297 | total_timesteps 4878.
Path 298 | total_timesteps 4892.
Path 299 | total_timesteps 4908.
Path 300 | total_timesteps 4919.
Path 301 | total_timesteps 4933.
Path 302 | total_timesteps 4954.
Path 303 | total_timesteps 4968.
Path 304 | total_timesteps 4981.
Path 305 | total_timesteps 4999.
Path 306 | total_timesteps 5007.
Path 307 | total_timesteps 5028.
Path 308 | total_timesteps 5042.
Path 309 | total_timesteps 5062.
Path 310 | total_timesteps 5078.
Path 311 | total_timesteps 5091.
Path 312 | total_timesteps 5107.
Path 313 | total_timesteps 5118.
Path 314 | total_timesteps 5132.
Path 315 | total_timesteps 5161.
Path 316 | total_timesteps 5184.
Path 317 | total_timesteps 5196.
Path 318 | total_timesteps 5210.
Path 319 | total_timesteps 5225.
Path 320 | total_timesteps 5244.
Path 321 | total_timesteps 5258.
Path 322 | total_timesteps 5275.
Path 323 | total_timesteps 5297.
Path 324 | total_timesteps 5309.
Path 325 | total_timesteps 5327.
Path 326 | total_timesteps 5342.
Path 327 | total_timesteps 5359.
Path 328 | total_timesteps 5378.
Path 329 | total_timesteps 5402.
Path 330 | total_timesteps 5416.
Path 331 | total_timesteps 5434.
Path 332 | total_timesteps 5453.
Path 333 | total_timesteps 5468.
Path 334 | total_timesteps 5488.
Path 335 | total_timesteps 5502.
Path 336 | total_timesteps 5519.
Path 337 | total_timesteps 5534.
Path 338 | total_timesteps 5550.
Path 339 | total_timesteps 5562.
Path 340 | total_timesteps 5580.
Path 341 | total_timesteps 5592.
Path 342 | total_timesteps 5614.
Path 343 | total_timesteps 5632.
Path 344 | total_timesteps 5643.
Path 345 | total_timesteps 5666.
Path 346 | total_timesteps 5675.
Path 347 | total_timesteps 5695.
Path 348 | total_timesteps 5713.
Path 349 | total_timesteps 5731.
Path 350 | total_timesteps 5745.
Path 351 | total_timesteps 5762.
Path 352 | total_timesteps 5776.
Path 353 | total_timesteps 5796.
Path 354 | total_timesteps 5814.
Path 355 | total_timesteps 5829.
Path 356 | total_timesteps 5844.
Path 357 | total_timesteps 5858.
Path 358 | total_timesteps 5875.
Path 359 | total_timesteps 5887.
Path 360 | total_timesteps 5899.
Path 361 | total_timesteps 5917.
Path 362 | total_timesteps 5933.
Path 363 | total_timesteps 5945.
Path 364 | total_timesteps 5959.
Path 365 | total_timesteps 5980.
Path 366 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -14.2    |
| Iteration     | 25       |
| MaximumReturn | -4.75    |
| MinimumReturn | -21.9    |
| TotalSamples  | 108170   |
----------------------------
itr #26 | 
Fitting dynamics.
Validation loss = 0.006366124842315912
Validation loss = 0.006009026430547237
Validation loss = 0.0062316954135894775
Validation loss = 0.005847256630659103
Validation loss = 0.0062616923823952675
Validation loss = 0.0057288119569420815
Validation loss = 0.005607356783002615
Validation loss = 0.005985155701637268
Validation loss = 0.005897923372685909
Validation loss = 0.005824106279760599
Validation loss = 0.005691013764590025
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 25.
Path 3 | total_timesteps 36.
Path 4 | total_timesteps 51.
Path 5 | total_timesteps 66.
Path 6 | total_timesteps 76.
Path 7 | total_timesteps 98.
Path 8 | total_timesteps 115.
Path 9 | total_timesteps 127.
Path 10 | total_timesteps 140.
Path 11 | total_timesteps 155.
Path 12 | total_timesteps 170.
Path 13 | total_timesteps 186.
Path 14 | total_timesteps 195.
Path 15 | total_timesteps 210.
Path 16 | total_timesteps 226.
Path 17 | total_timesteps 233.
Path 18 | total_timesteps 245.
Path 19 | total_timesteps 265.
Path 20 | total_timesteps 281.
Path 21 | total_timesteps 289.
Path 22 | total_timesteps 303.
Path 23 | total_timesteps 320.
Path 24 | total_timesteps 338.
Path 25 | total_timesteps 351.
Path 26 | total_timesteps 363.
Path 27 | total_timesteps 383.
Path 28 | total_timesteps 396.
Path 29 | total_timesteps 410.
Path 30 | total_timesteps 421.
Path 31 | total_timesteps 437.
Path 32 | total_timesteps 450.
Path 33 | total_timesteps 464.
Path 34 | total_timesteps 477.
Path 35 | total_timesteps 493.
Path 36 | total_timesteps 509.
Path 37 | total_timesteps 523.
Path 38 | total_timesteps 532.
Path 39 | total_timesteps 543.
Path 40 | total_timesteps 553.
Path 41 | total_timesteps 569.
Path 42 | total_timesteps 585.
Path 43 | total_timesteps 598.
Path 44 | total_timesteps 605.
Path 45 | total_timesteps 614.
Path 46 | total_timesteps 626.
Path 47 | total_timesteps 643.
Path 48 | total_timesteps 662.
Path 49 | total_timesteps 676.
Path 50 | total_timesteps 690.
Path 51 | total_timesteps 702.
Path 52 | total_timesteps 712.
Path 53 | total_timesteps 729.
Path 54 | total_timesteps 743.
Path 55 | total_timesteps 765.
Path 56 | total_timesteps 783.
Path 57 | total_timesteps 800.
Path 58 | total_timesteps 809.
Path 59 | total_timesteps 822.
Path 60 | total_timesteps 843.
Path 61 | total_timesteps 853.
Path 62 | total_timesteps 871.
Path 63 | total_timesteps 882.
Path 64 | total_timesteps 894.
Path 65 | total_timesteps 909.
Path 66 | total_timesteps 921.
Path 67 | total_timesteps 932.
Path 68 | total_timesteps 948.
Path 69 | total_timesteps 959.
Path 70 | total_timesteps 973.
Path 71 | total_timesteps 983.
Path 72 | total_timesteps 992.
Path 73 | total_timesteps 1016.
Path 74 | total_timesteps 1031.
Path 75 | total_timesteps 1045.
Path 76 | total_timesteps 1056.
Path 77 | total_timesteps 1071.
Path 78 | total_timesteps 1080.
Path 79 | total_timesteps 1089.
Path 80 | total_timesteps 1100.
Path 81 | total_timesteps 1111.
Path 82 | total_timesteps 1125.
Path 83 | total_timesteps 1149.
Path 84 | total_timesteps 1158.
Path 85 | total_timesteps 1184.
Path 86 | total_timesteps 1200.
Path 87 | total_timesteps 1216.
Path 88 | total_timesteps 1233.
Path 89 | total_timesteps 1248.
Path 90 | total_timesteps 1263.
Path 91 | total_timesteps 1273.
Path 92 | total_timesteps 1285.
Path 93 | total_timesteps 1296.
Path 94 | total_timesteps 1314.
Path 95 | total_timesteps 1332.
Path 96 | total_timesteps 1347.
Path 97 | total_timesteps 1360.
Path 98 | total_timesteps 1376.
Path 99 | total_timesteps 1388.
Path 100 | total_timesteps 1402.
Path 101 | total_timesteps 1415.
Path 102 | total_timesteps 1432.
Path 103 | total_timesteps 1448.
Path 104 | total_timesteps 1466.
Path 105 | total_timesteps 1479.
Path 106 | total_timesteps 1496.
Path 107 | total_timesteps 1514.
Path 108 | total_timesteps 1531.
Path 109 | total_timesteps 1549.
Path 110 | total_timesteps 1569.
Path 111 | total_timesteps 1579.
Path 112 | total_timesteps 1598.
Path 113 | total_timesteps 1614.
Path 114 | total_timesteps 1633.
Path 115 | total_timesteps 1645.
Path 116 | total_timesteps 1655.
Path 117 | total_timesteps 1665.
Path 118 | total_timesteps 1677.
Path 119 | total_timesteps 1696.
Path 120 | total_timesteps 1706.
Path 121 | total_timesteps 1719.
Path 122 | total_timesteps 1731.
Path 123 | total_timesteps 1742.
Path 124 | total_timesteps 1765.
Path 125 | total_timesteps 1785.
Path 126 | total_timesteps 1799.
Path 127 | total_timesteps 1823.
Path 128 | total_timesteps 1840.
Path 129 | total_timesteps 1852.
Path 130 | total_timesteps 1866.
Path 131 | total_timesteps 1881.
Path 132 | total_timesteps 1896.
Path 133 | total_timesteps 1908.
Path 134 | total_timesteps 1924.
Path 135 | total_timesteps 1935.
Path 136 | total_timesteps 1946.
Path 137 | total_timesteps 1955.
Path 138 | total_timesteps 1966.
Path 139 | total_timesteps 1984.
Path 140 | total_timesteps 2002.
Path 141 | total_timesteps 2018.
Path 142 | total_timesteps 2030.
Path 143 | total_timesteps 2048.
Path 144 | total_timesteps 2068.
Path 145 | total_timesteps 2090.
Path 146 | total_timesteps 2103.
Path 147 | total_timesteps 2117.
Path 148 | total_timesteps 2129.
Path 149 | total_timesteps 2140.
Path 150 | total_timesteps 2158.
Path 151 | total_timesteps 2175.
Path 152 | total_timesteps 2187.
Path 153 | total_timesteps 2199.
Path 154 | total_timesteps 2208.
Path 155 | total_timesteps 2219.
Path 156 | total_timesteps 2230.
Path 157 | total_timesteps 2246.
Path 158 | total_timesteps 2256.
Path 159 | total_timesteps 2271.
Path 160 | total_timesteps 2290.
Path 161 | total_timesteps 2306.
Path 162 | total_timesteps 2321.
Path 163 | total_timesteps 2335.
Path 164 | total_timesteps 2344.
Path 165 | total_timesteps 2357.
Path 166 | total_timesteps 2368.
Path 167 | total_timesteps 2384.
Path 168 | total_timesteps 2403.
Path 169 | total_timesteps 2414.
Path 170 | total_timesteps 2424.
Path 171 | total_timesteps 2442.
Path 172 | total_timesteps 2451.
Path 173 | total_timesteps 2465.
Path 174 | total_timesteps 2483.
Path 175 | total_timesteps 2500.
Path 176 | total_timesteps 2517.
Path 177 | total_timesteps 2531.
Path 178 | total_timesteps 2540.
Path 179 | total_timesteps 2549.
Path 180 | total_timesteps 2565.
Path 181 | total_timesteps 2577.
Path 182 | total_timesteps 2595.
Path 183 | total_timesteps 2605.
Path 184 | total_timesteps 2626.
Path 185 | total_timesteps 2644.
Path 186 | total_timesteps 2658.
Path 187 | total_timesteps 2669.
Path 188 | total_timesteps 2682.
Path 189 | total_timesteps 2691.
Path 190 | total_timesteps 2708.
Path 191 | total_timesteps 2720.
Path 192 | total_timesteps 2736.
Path 193 | total_timesteps 2750.
Path 194 | total_timesteps 2761.
Path 195 | total_timesteps 2775.
Path 196 | total_timesteps 2794.
Path 197 | total_timesteps 2813.
Path 198 | total_timesteps 2828.
Path 199 | total_timesteps 2841.
Path 200 | total_timesteps 2850.
Path 201 | total_timesteps 2863.
Path 202 | total_timesteps 2877.
Path 203 | total_timesteps 2884.
Path 204 | total_timesteps 2900.
Path 205 | total_timesteps 2910.
Path 206 | total_timesteps 2928.
Path 207 | total_timesteps 2948.
Path 208 | total_timesteps 2968.
Path 209 | total_timesteps 2988.
Path 210 | total_timesteps 3004.
Path 211 | total_timesteps 3014.
Path 212 | total_timesteps 3029.
Path 213 | total_timesteps 3043.
Path 214 | total_timesteps 3059.
Path 215 | total_timesteps 3069.
Path 216 | total_timesteps 3086.
Path 217 | total_timesteps 3112.
Path 218 | total_timesteps 3132.
Path 219 | total_timesteps 3142.
Path 220 | total_timesteps 3155.
Path 221 | total_timesteps 3169.
Path 222 | total_timesteps 3179.
Path 223 | total_timesteps 3197.
Path 224 | total_timesteps 3209.
Path 225 | total_timesteps 3232.
Path 226 | total_timesteps 3246.
Path 227 | total_timesteps 3259.
Path 228 | total_timesteps 3280.
Path 229 | total_timesteps 3293.
Path 230 | total_timesteps 3307.
Path 231 | total_timesteps 3318.
Path 232 | total_timesteps 3335.
Path 233 | total_timesteps 3350.
Path 234 | total_timesteps 3373.
Path 235 | total_timesteps 3383.
Path 236 | total_timesteps 3399.
Path 237 | total_timesteps 3412.
Path 238 | total_timesteps 3426.
Path 239 | total_timesteps 3442.
Path 240 | total_timesteps 3459.
Path 241 | total_timesteps 3475.
Path 242 | total_timesteps 3487.
Path 243 | total_timesteps 3506.
Path 244 | total_timesteps 3518.
Path 245 | total_timesteps 3539.
Path 246 | total_timesteps 3551.
Path 247 | total_timesteps 3566.
Path 248 | total_timesteps 3588.
Path 249 | total_timesteps 3605.
Path 250 | total_timesteps 3627.
Path 251 | total_timesteps 3640.
Path 252 | total_timesteps 3655.
Path 253 | total_timesteps 3666.
Path 254 | total_timesteps 3677.
Path 255 | total_timesteps 3689.
Path 256 | total_timesteps 3708.
Path 257 | total_timesteps 3725.
Path 258 | total_timesteps 3733.
Path 259 | total_timesteps 3757.
Path 260 | total_timesteps 3767.
Path 261 | total_timesteps 3785.
Path 262 | total_timesteps 3796.
Path 263 | total_timesteps 3807.
Path 264 | total_timesteps 3815.
Path 265 | total_timesteps 3843.
Path 266 | total_timesteps 3854.
Path 267 | total_timesteps 3864.
Path 268 | total_timesteps 3878.
Path 269 | total_timesteps 3898.
Path 270 | total_timesteps 3917.
Path 271 | total_timesteps 3931.
Path 272 | total_timesteps 3941.
Path 273 | total_timesteps 3954.
Path 274 | total_timesteps 3968.
Path 275 | total_timesteps 3987.
Path 276 | total_timesteps 4000.
Path 277 | total_timesteps 4015.
Path 278 | total_timesteps 4032.
Path 279 | total_timesteps 4042.
Path 280 | total_timesteps 4053.
Path 281 | total_timesteps 4065.
Path 282 | total_timesteps 4077.
Path 283 | total_timesteps 4095.
Path 284 | total_timesteps 4107.
Path 285 | total_timesteps 4129.
Path 286 | total_timesteps 4138.
Path 287 | total_timesteps 4149.
Path 288 | total_timesteps 4159.
Path 289 | total_timesteps 4172.
Path 290 | total_timesteps 4193.
Path 291 | total_timesteps 4204.
Path 292 | total_timesteps 4216.
Path 293 | total_timesteps 4236.
Path 294 | total_timesteps 4247.
Path 295 | total_timesteps 4271.
Path 296 | total_timesteps 4284.
Path 297 | total_timesteps 4301.
Path 298 | total_timesteps 4312.
Path 299 | total_timesteps 4323.
Path 300 | total_timesteps 4333.
Path 301 | total_timesteps 4349.
Path 302 | total_timesteps 4367.
Path 303 | total_timesteps 4382.
Path 304 | total_timesteps 4399.
Path 305 | total_timesteps 4419.
Path 306 | total_timesteps 4433.
Path 307 | total_timesteps 4448.
Path 308 | total_timesteps 4465.
Path 309 | total_timesteps 4477.
Path 310 | total_timesteps 4495.
Path 311 | total_timesteps 4518.
Path 312 | total_timesteps 4534.
Path 313 | total_timesteps 4551.
Path 314 | total_timesteps 4565.
Path 315 | total_timesteps 4578.
Path 316 | total_timesteps 4592.
Path 317 | total_timesteps 4609.
Path 318 | total_timesteps 4629.
Path 319 | total_timesteps 4638.
Path 320 | total_timesteps 4651.
Path 321 | total_timesteps 4660.
Path 322 | total_timesteps 4670.
Path 323 | total_timesteps 4685.
Path 324 | total_timesteps 4700.
Path 325 | total_timesteps 4715.
Path 326 | total_timesteps 4730.
Path 327 | total_timesteps 4741.
Path 328 | total_timesteps 4754.
Path 329 | total_timesteps 4775.
Path 330 | total_timesteps 4788.
Path 331 | total_timesteps 4803.
Path 332 | total_timesteps 4823.
Path 333 | total_timesteps 4834.
Path 334 | total_timesteps 4856.
Path 335 | total_timesteps 4867.
Path 336 | total_timesteps 4890.
Path 337 | total_timesteps 4906.
Path 338 | total_timesteps 4916.
Path 339 | total_timesteps 4930.
Path 340 | total_timesteps 4944.
Path 341 | total_timesteps 4961.
Path 342 | total_timesteps 4976.
Path 343 | total_timesteps 4986.
Path 344 | total_timesteps 5000.
Path 345 | total_timesteps 5026.
Path 346 | total_timesteps 5039.
Path 347 | total_timesteps 5057.
Path 348 | total_timesteps 5071.
Path 349 | total_timesteps 5089.
Path 350 | total_timesteps 5100.
Path 351 | total_timesteps 5117.
Path 352 | total_timesteps 5133.
Path 353 | total_timesteps 5145.
Path 354 | total_timesteps 5157.
Path 355 | total_timesteps 5170.
Path 356 | total_timesteps 5186.
Path 357 | total_timesteps 5199.
Path 358 | total_timesteps 5222.
Path 359 | total_timesteps 5234.
Path 360 | total_timesteps 5248.
Path 361 | total_timesteps 5259.
Path 362 | total_timesteps 5267.
Path 363 | total_timesteps 5278.
Path 364 | total_timesteps 5288.
Path 365 | total_timesteps 5302.
Path 366 | total_timesteps 5315.
Path 367 | total_timesteps 5330.
Path 368 | total_timesteps 5343.
Path 369 | total_timesteps 5354.
Path 370 | total_timesteps 5372.
Path 371 | total_timesteps 5386.
Path 372 | total_timesteps 5397.
Path 373 | total_timesteps 5405.
Path 374 | total_timesteps 5414.
Path 375 | total_timesteps 5424.
Path 376 | total_timesteps 5441.
Path 377 | total_timesteps 5451.
Path 378 | total_timesteps 5463.
Path 379 | total_timesteps 5475.
Path 380 | total_timesteps 5483.
Path 381 | total_timesteps 5505.
Path 382 | total_timesteps 5518.
Path 383 | total_timesteps 5530.
Path 384 | total_timesteps 5539.
Path 385 | total_timesteps 5552.
Path 386 | total_timesteps 5574.
Path 387 | total_timesteps 5585.
Path 388 | total_timesteps 5615.
Path 389 | total_timesteps 5633.
Path 390 | total_timesteps 5645.
Path 391 | total_timesteps 5659.
Path 392 | total_timesteps 5676.
Path 393 | total_timesteps 5688.
Path 394 | total_timesteps 5704.
Path 395 | total_timesteps 5720.
Path 396 | total_timesteps 5730.
Path 397 | total_timesteps 5744.
Path 398 | total_timesteps 5756.
Path 399 | total_timesteps 5769.
Path 400 | total_timesteps 5785.
Path 401 | total_timesteps 5800.
Path 402 | total_timesteps 5819.
Path 403 | total_timesteps 5831.
Path 404 | total_timesteps 5843.
Path 405 | total_timesteps 5858.
Path 406 | total_timesteps 5868.
Path 407 | total_timesteps 5883.
Path 408 | total_timesteps 5896.
Path 409 | total_timesteps 5906.
Path 410 | total_timesteps 5919.
Path 411 | total_timesteps 5931.
Path 412 | total_timesteps 5945.
Path 413 | total_timesteps 5955.
Path 414 | total_timesteps 5965.
Path 415 | total_timesteps 5975.
Path 416 | total_timesteps 5987.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -12.6    |
| Iteration     | 26       |
| MaximumReturn | -1.13    |
| MinimumReturn | -24.4    |
| TotalSamples  | 112174   |
----------------------------
itr #27 | 
Fitting dynamics.
Validation loss = 0.005968983750790358
Validation loss = 0.0055297380313277245
Validation loss = 0.0055289980955421925
Validation loss = 0.006087134126573801
Validation loss = 0.005886867642402649
Validation loss = 0.005800164770334959
Validation loss = 0.00561492657288909
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 45.
Path 4 | total_timesteps 62.
Path 5 | total_timesteps 76.
Path 6 | total_timesteps 99.
Path 7 | total_timesteps 108.
Path 8 | total_timesteps 115.
Path 9 | total_timesteps 135.
Path 10 | total_timesteps 150.
Path 11 | total_timesteps 162.
Path 12 | total_timesteps 180.
Path 13 | total_timesteps 195.
Path 14 | total_timesteps 204.
Path 15 | total_timesteps 214.
Path 16 | total_timesteps 229.
Path 17 | total_timesteps 239.
Path 18 | total_timesteps 254.
Path 19 | total_timesteps 272.
Path 20 | total_timesteps 288.
Path 21 | total_timesteps 304.
Path 22 | total_timesteps 318.
Path 23 | total_timesteps 333.
Path 24 | total_timesteps 348.
Path 25 | total_timesteps 369.
Path 26 | total_timesteps 381.
Path 27 | total_timesteps 397.
Path 28 | total_timesteps 410.
Path 29 | total_timesteps 423.
Path 30 | total_timesteps 432.
Path 31 | total_timesteps 444.
Path 32 | total_timesteps 453.
Path 33 | total_timesteps 463.
Path 34 | total_timesteps 473.
Path 35 | total_timesteps 490.
Path 36 | total_timesteps 502.
Path 37 | total_timesteps 514.
Path 38 | total_timesteps 533.
Path 39 | total_timesteps 549.
Path 40 | total_timesteps 564.
Path 41 | total_timesteps 575.
Path 42 | total_timesteps 597.
Path 43 | total_timesteps 606.
Path 44 | total_timesteps 618.
Path 45 | total_timesteps 636.
Path 46 | total_timesteps 647.
Path 47 | total_timesteps 660.
Path 48 | total_timesteps 676.
Path 49 | total_timesteps 690.
Path 50 | total_timesteps 702.
Path 51 | total_timesteps 715.
Path 52 | total_timesteps 729.
Path 53 | total_timesteps 739.
Path 54 | total_timesteps 758.
Path 55 | total_timesteps 773.
Path 56 | total_timesteps 786.
Path 57 | total_timesteps 798.
Path 58 | total_timesteps 809.
Path 59 | total_timesteps 831.
Path 60 | total_timesteps 847.
Path 61 | total_timesteps 861.
Path 62 | total_timesteps 878.
Path 63 | total_timesteps 890.
Path 64 | total_timesteps 900.
Path 65 | total_timesteps 912.
Path 66 | total_timesteps 928.
Path 67 | total_timesteps 942.
Path 68 | total_timesteps 955.
Path 69 | total_timesteps 966.
Path 70 | total_timesteps 975.
Path 71 | total_timesteps 985.
Path 72 | total_timesteps 999.
Path 73 | total_timesteps 1008.
Path 74 | total_timesteps 1029.
Path 75 | total_timesteps 1040.
Path 76 | total_timesteps 1055.
Path 77 | total_timesteps 1083.
Path 78 | total_timesteps 1096.
Path 79 | total_timesteps 1111.
Path 80 | total_timesteps 1125.
Path 81 | total_timesteps 1143.
Path 82 | total_timesteps 1151.
Path 83 | total_timesteps 1160.
Path 84 | total_timesteps 1179.
Path 85 | total_timesteps 1198.
Path 86 | total_timesteps 1212.
Path 87 | total_timesteps 1237.
Path 88 | total_timesteps 1254.
Path 89 | total_timesteps 1265.
Path 90 | total_timesteps 1283.
Path 91 | total_timesteps 1297.
Path 92 | total_timesteps 1309.
Path 93 | total_timesteps 1322.
Path 94 | total_timesteps 1339.
Path 95 | total_timesteps 1351.
Path 96 | total_timesteps 1362.
Path 97 | total_timesteps 1372.
Path 98 | total_timesteps 1392.
Path 99 | total_timesteps 1410.
Path 100 | total_timesteps 1425.
Path 101 | total_timesteps 1444.
Path 102 | total_timesteps 1458.
Path 103 | total_timesteps 1469.
Path 104 | total_timesteps 1483.
Path 105 | total_timesteps 1498.
Path 106 | total_timesteps 1513.
Path 107 | total_timesteps 1526.
Path 108 | total_timesteps 1542.
Path 109 | total_timesteps 1556.
Path 110 | total_timesteps 1583.
Path 111 | total_timesteps 1594.
Path 112 | total_timesteps 1607.
Path 113 | total_timesteps 1620.
Path 114 | total_timesteps 1634.
Path 115 | total_timesteps 1651.
Path 116 | total_timesteps 1669.
Path 117 | total_timesteps 1686.
Path 118 | total_timesteps 1698.
Path 119 | total_timesteps 1715.
Path 120 | total_timesteps 1731.
Path 121 | total_timesteps 1741.
Path 122 | total_timesteps 1750.
Path 123 | total_timesteps 1764.
Path 124 | total_timesteps 1774.
Path 125 | total_timesteps 1788.
Path 126 | total_timesteps 1800.
Path 127 | total_timesteps 1813.
Path 128 | total_timesteps 1832.
Path 129 | total_timesteps 1846.
Path 130 | total_timesteps 1860.
Path 131 | total_timesteps 1875.
Path 132 | total_timesteps 1891.
Path 133 | total_timesteps 1903.
Path 134 | total_timesteps 1914.
Path 135 | total_timesteps 1923.
Path 136 | total_timesteps 1935.
Path 137 | total_timesteps 1949.
Path 138 | total_timesteps 1966.
Path 139 | total_timesteps 1976.
Path 140 | total_timesteps 1990.
Path 141 | total_timesteps 2002.
Path 142 | total_timesteps 2020.
Path 143 | total_timesteps 2029.
Path 144 | total_timesteps 2039.
Path 145 | total_timesteps 2058.
Path 146 | total_timesteps 2073.
Path 147 | total_timesteps 2083.
Path 148 | total_timesteps 2095.
Path 149 | total_timesteps 2106.
Path 150 | total_timesteps 2114.
Path 151 | total_timesteps 2133.
Path 152 | total_timesteps 2156.
Path 153 | total_timesteps 2165.
Path 154 | total_timesteps 2179.
Path 155 | total_timesteps 2191.
Path 156 | total_timesteps 2217.
Path 157 | total_timesteps 2229.
Path 158 | total_timesteps 2243.
Path 159 | total_timesteps 2262.
Path 160 | total_timesteps 2274.
Path 161 | total_timesteps 2291.
Path 162 | total_timesteps 2310.
Path 163 | total_timesteps 2322.
Path 164 | total_timesteps 2337.
Path 165 | total_timesteps 2360.
Path 166 | total_timesteps 2372.
Path 167 | total_timesteps 2385.
Path 168 | total_timesteps 2399.
Path 169 | total_timesteps 2412.
Path 170 | total_timesteps 2428.
Path 171 | total_timesteps 2441.
Path 172 | total_timesteps 2454.
Path 173 | total_timesteps 2464.
Path 174 | total_timesteps 2474.
Path 175 | total_timesteps 2495.
Path 176 | total_timesteps 2511.
Path 177 | total_timesteps 2524.
Path 178 | total_timesteps 2546.
Path 179 | total_timesteps 2560.
Path 180 | total_timesteps 2575.
Path 181 | total_timesteps 2585.
Path 182 | total_timesteps 2598.
Path 183 | total_timesteps 2613.
Path 184 | total_timesteps 2629.
Path 185 | total_timesteps 2640.
Path 186 | total_timesteps 2655.
Path 187 | total_timesteps 2665.
Path 188 | total_timesteps 2675.
Path 189 | total_timesteps 2689.
Path 190 | total_timesteps 2701.
Path 191 | total_timesteps 2716.
Path 192 | total_timesteps 2730.
Path 193 | total_timesteps 2742.
Path 194 | total_timesteps 2760.
Path 195 | total_timesteps 2780.
Path 196 | total_timesteps 2794.
Path 197 | total_timesteps 2809.
Path 198 | total_timesteps 2822.
Path 199 | total_timesteps 2840.
Path 200 | total_timesteps 2860.
Path 201 | total_timesteps 2870.
Path 202 | total_timesteps 2879.
Path 203 | total_timesteps 2889.
Path 204 | total_timesteps 2904.
Path 205 | total_timesteps 2917.
Path 206 | total_timesteps 2930.
Path 207 | total_timesteps 2941.
Path 208 | total_timesteps 2955.
Path 209 | total_timesteps 2969.
Path 210 | total_timesteps 2980.
Path 211 | total_timesteps 2998.
Path 212 | total_timesteps 3014.
Path 213 | total_timesteps 3022.
Path 214 | total_timesteps 3035.
Path 215 | total_timesteps 3054.
Path 216 | total_timesteps 3064.
Path 217 | total_timesteps 3076.
Path 218 | total_timesteps 3093.
Path 219 | total_timesteps 3107.
Path 220 | total_timesteps 3122.
Path 221 | total_timesteps 3137.
Path 222 | total_timesteps 3155.
Path 223 | total_timesteps 3166.
Path 224 | total_timesteps 3180.
Path 225 | total_timesteps 3197.
Path 226 | total_timesteps 3213.
Path 227 | total_timesteps 3227.
Path 228 | total_timesteps 3243.
Path 229 | total_timesteps 3257.
Path 230 | total_timesteps 3275.
Path 231 | total_timesteps 3294.
Path 232 | total_timesteps 3302.
Path 233 | total_timesteps 3320.
Path 234 | total_timesteps 3335.
Path 235 | total_timesteps 3345.
Path 236 | total_timesteps 3361.
Path 237 | total_timesteps 3368.
Path 238 | total_timesteps 3385.
Path 239 | total_timesteps 3402.
Path 240 | total_timesteps 3412.
Path 241 | total_timesteps 3421.
Path 242 | total_timesteps 3437.
Path 243 | total_timesteps 3456.
Path 244 | total_timesteps 3473.
Path 245 | total_timesteps 3486.
Path 246 | total_timesteps 3499.
Path 247 | total_timesteps 3519.
Path 248 | total_timesteps 3527.
Path 249 | total_timesteps 3544.
Path 250 | total_timesteps 3554.
Path 251 | total_timesteps 3570.
Path 252 | total_timesteps 3579.
Path 253 | total_timesteps 3601.
Path 254 | total_timesteps 3613.
Path 255 | total_timesteps 3621.
Path 256 | total_timesteps 3636.
Path 257 | total_timesteps 3649.
Path 258 | total_timesteps 3659.
Path 259 | total_timesteps 3672.
Path 260 | total_timesteps 3681.
Path 261 | total_timesteps 3691.
Path 262 | total_timesteps 3719.
Path 263 | total_timesteps 3732.
Path 264 | total_timesteps 3740.
Path 265 | total_timesteps 3755.
Path 266 | total_timesteps 3772.
Path 267 | total_timesteps 3782.
Path 268 | total_timesteps 3797.
Path 269 | total_timesteps 3806.
Path 270 | total_timesteps 3824.
Path 271 | total_timesteps 3835.
Path 272 | total_timesteps 3850.
Path 273 | total_timesteps 3865.
Path 274 | total_timesteps 3878.
Path 275 | total_timesteps 3896.
Path 276 | total_timesteps 3912.
Path 277 | total_timesteps 3941.
Path 278 | total_timesteps 3949.
Path 279 | total_timesteps 3961.
Path 280 | total_timesteps 3972.
Path 281 | total_timesteps 3983.
Path 282 | total_timesteps 3998.
Path 283 | total_timesteps 4008.
Path 284 | total_timesteps 4026.
Path 285 | total_timesteps 4036.
Path 286 | total_timesteps 4051.
Path 287 | total_timesteps 4073.
Path 288 | total_timesteps 4087.
Path 289 | total_timesteps 4095.
Path 290 | total_timesteps 4110.
Path 291 | total_timesteps 4120.
Path 292 | total_timesteps 4136.
Path 293 | total_timesteps 4146.
Path 294 | total_timesteps 4166.
Path 295 | total_timesteps 4176.
Path 296 | total_timesteps 4185.
Path 297 | total_timesteps 4203.
Path 298 | total_timesteps 4211.
Path 299 | total_timesteps 4225.
Path 300 | total_timesteps 4241.
Path 301 | total_timesteps 4257.
Path 302 | total_timesteps 4267.
Path 303 | total_timesteps 4293.
Path 304 | total_timesteps 4307.
Path 305 | total_timesteps 4318.
Path 306 | total_timesteps 4338.
Path 307 | total_timesteps 4356.
Path 308 | total_timesteps 4371.
Path 309 | total_timesteps 4380.
Path 310 | total_timesteps 4390.
Path 311 | total_timesteps 4405.
Path 312 | total_timesteps 4425.
Path 313 | total_timesteps 4440.
Path 314 | total_timesteps 4456.
Path 315 | total_timesteps 4469.
Path 316 | total_timesteps 4477.
Path 317 | total_timesteps 4487.
Path 318 | total_timesteps 4508.
Path 319 | total_timesteps 4526.
Path 320 | total_timesteps 4541.
Path 321 | total_timesteps 4555.
Path 322 | total_timesteps 4569.
Path 323 | total_timesteps 4583.
Path 324 | total_timesteps 4594.
Path 325 | total_timesteps 4611.
Path 326 | total_timesteps 4629.
Path 327 | total_timesteps 4643.
Path 328 | total_timesteps 4660.
Path 329 | total_timesteps 4674.
Path 330 | total_timesteps 4688.
Path 331 | total_timesteps 4704.
Path 332 | total_timesteps 4715.
Path 333 | total_timesteps 4727.
Path 334 | total_timesteps 4739.
Path 335 | total_timesteps 4753.
Path 336 | total_timesteps 4764.
Path 337 | total_timesteps 4781.
Path 338 | total_timesteps 4794.
Path 339 | total_timesteps 4816.
Path 340 | total_timesteps 4831.
Path 341 | total_timesteps 4842.
Path 342 | total_timesteps 4860.
Path 343 | total_timesteps 4869.
Path 344 | total_timesteps 4884.
Path 345 | total_timesteps 4901.
Path 346 | total_timesteps 4915.
Path 347 | total_timesteps 4930.
Path 348 | total_timesteps 4940.
Path 349 | total_timesteps 4951.
Path 350 | total_timesteps 4965.
Path 351 | total_timesteps 4975.
Path 352 | total_timesteps 4986.
Path 353 | total_timesteps 5001.
Path 354 | total_timesteps 5016.
Path 355 | total_timesteps 5037.
Path 356 | total_timesteps 5044.
Path 357 | total_timesteps 5061.
Path 358 | total_timesteps 5075.
Path 359 | total_timesteps 5090.
Path 360 | total_timesteps 5104.
Path 361 | total_timesteps 5118.
Path 362 | total_timesteps 5129.
Path 363 | total_timesteps 5143.
Path 364 | total_timesteps 5151.
Path 365 | total_timesteps 5164.
Path 366 | total_timesteps 5177.
Path 367 | total_timesteps 5190.
Path 368 | total_timesteps 5200.
Path 369 | total_timesteps 5215.
Path 370 | total_timesteps 5230.
Path 371 | total_timesteps 5243.
Path 372 | total_timesteps 5257.
Path 373 | total_timesteps 5273.
Path 374 | total_timesteps 5288.
Path 375 | total_timesteps 5304.
Path 376 | total_timesteps 5321.
Path 377 | total_timesteps 5337.
Path 378 | total_timesteps 5350.
Path 379 | total_timesteps 5362.
Path 380 | total_timesteps 5376.
Path 381 | total_timesteps 5388.
Path 382 | total_timesteps 5398.
Path 383 | total_timesteps 5415.
Path 384 | total_timesteps 5432.
Path 385 | total_timesteps 5444.
Path 386 | total_timesteps 5459.
Path 387 | total_timesteps 5475.
Path 388 | total_timesteps 5496.
Path 389 | total_timesteps 5509.
Path 390 | total_timesteps 5527.
Path 391 | total_timesteps 5542.
Path 392 | total_timesteps 5553.
Path 393 | total_timesteps 5576.
Path 394 | total_timesteps 5595.
Path 395 | total_timesteps 5615.
Path 396 | total_timesteps 5627.
Path 397 | total_timesteps 5635.
Path 398 | total_timesteps 5650.
Path 399 | total_timesteps 5665.
Path 400 | total_timesteps 5676.
Path 401 | total_timesteps 5694.
Path 402 | total_timesteps 5711.
Path 403 | total_timesteps 5724.
Path 404 | total_timesteps 5733.
Path 405 | total_timesteps 5746.
Path 406 | total_timesteps 5761.
Path 407 | total_timesteps 5772.
Path 408 | total_timesteps 5783.
Path 409 | total_timesteps 5799.
Path 410 | total_timesteps 5813.
Path 411 | total_timesteps 5824.
Path 412 | total_timesteps 5837.
Path 413 | total_timesteps 5850.
Path 414 | total_timesteps 5861.
Path 415 | total_timesteps 5876.
Path 416 | total_timesteps 5887.
Path 417 | total_timesteps 5895.
Path 418 | total_timesteps 5910.
Path 419 | total_timesteps 5928.
Path 420 | total_timesteps 5949.
Path 421 | total_timesteps 5959.
Path 422 | total_timesteps 5970.
Path 423 | total_timesteps 5981.
Path 424 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -12      |
| Iteration     | 27       |
| MaximumReturn | -1.48    |
| MinimumReturn | -21      |
| TotalSamples  | 116180   |
----------------------------
itr #28 | 
Fitting dynamics.
Validation loss = 0.005415939260274172
Validation loss = 0.005531734321266413
Validation loss = 0.005448243580758572
Validation loss = 0.0057572294026613235
Validation loss = 0.005458313040435314
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 25.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 59.
Path 5 | total_timesteps 72.
Path 6 | total_timesteps 84.
Path 7 | total_timesteps 97.
Path 8 | total_timesteps 116.
Path 9 | total_timesteps 132.
Path 10 | total_timesteps 142.
Path 11 | total_timesteps 150.
Path 12 | total_timesteps 158.
Path 13 | total_timesteps 174.
Path 14 | total_timesteps 188.
Path 15 | total_timesteps 201.
Path 16 | total_timesteps 215.
Path 17 | total_timesteps 224.
Path 18 | total_timesteps 235.
Path 19 | total_timesteps 255.
Path 20 | total_timesteps 273.
Path 21 | total_timesteps 283.
Path 22 | total_timesteps 293.
Path 23 | total_timesteps 313.
Path 24 | total_timesteps 329.
Path 25 | total_timesteps 338.
Path 26 | total_timesteps 349.
Path 27 | total_timesteps 370.
Path 28 | total_timesteps 388.
Path 29 | total_timesteps 402.
Path 30 | total_timesteps 413.
Path 31 | total_timesteps 429.
Path 32 | total_timesteps 452.
Path 33 | total_timesteps 463.
Path 34 | total_timesteps 480.
Path 35 | total_timesteps 492.
Path 36 | total_timesteps 504.
Path 37 | total_timesteps 517.
Path 38 | total_timesteps 533.
Path 39 | total_timesteps 550.
Path 40 | total_timesteps 570.
Path 41 | total_timesteps 583.
Path 42 | total_timesteps 597.
Path 43 | total_timesteps 609.
Path 44 | total_timesteps 618.
Path 45 | total_timesteps 628.
Path 46 | total_timesteps 641.
Path 47 | total_timesteps 653.
Path 48 | total_timesteps 665.
Path 49 | total_timesteps 675.
Path 50 | total_timesteps 686.
Path 51 | total_timesteps 695.
Path 52 | total_timesteps 709.
Path 53 | total_timesteps 721.
Path 54 | total_timesteps 730.
Path 55 | total_timesteps 741.
Path 56 | total_timesteps 753.
Path 57 | total_timesteps 763.
Path 58 | total_timesteps 788.
Path 59 | total_timesteps 800.
Path 60 | total_timesteps 810.
Path 61 | total_timesteps 817.
Path 62 | total_timesteps 829.
Path 63 | total_timesteps 839.
Path 64 | total_timesteps 848.
Path 65 | total_timesteps 860.
Path 66 | total_timesteps 873.
Path 67 | total_timesteps 886.
Path 68 | total_timesteps 901.
Path 69 | total_timesteps 950.
Path 70 | total_timesteps 967.
Path 71 | total_timesteps 986.
Path 72 | total_timesteps 998.
Path 73 | total_timesteps 1007.
Path 74 | total_timesteps 1021.
Path 75 | total_timesteps 1031.
Path 76 | total_timesteps 1044.
Path 77 | total_timesteps 1062.
Path 78 | total_timesteps 1075.
Path 79 | total_timesteps 1085.
Path 80 | total_timesteps 1096.
Path 81 | total_timesteps 1108.
Path 82 | total_timesteps 1119.
Path 83 | total_timesteps 1134.
Path 84 | total_timesteps 1146.
Path 85 | total_timesteps 1156.
Path 86 | total_timesteps 1167.
Path 87 | total_timesteps 1176.
Path 88 | total_timesteps 1188.
Path 89 | total_timesteps 1201.
Path 90 | total_timesteps 1215.
Path 91 | total_timesteps 1234.
Path 92 | total_timesteps 1248.
Path 93 | total_timesteps 1261.
Path 94 | total_timesteps 1270.
Path 95 | total_timesteps 1285.
Path 96 | total_timesteps 1302.
Path 97 | total_timesteps 1315.
Path 98 | total_timesteps 1330.
Path 99 | total_timesteps 1342.
Path 100 | total_timesteps 1351.
Path 101 | total_timesteps 1363.
Path 102 | total_timesteps 1377.
Path 103 | total_timesteps 1389.
Path 104 | total_timesteps 1403.
Path 105 | total_timesteps 1424.
Path 106 | total_timesteps 1440.
Path 107 | total_timesteps 1461.
Path 108 | total_timesteps 1478.
Path 109 | total_timesteps 1496.
Path 110 | total_timesteps 1512.
Path 111 | total_timesteps 1526.
Path 112 | total_timesteps 1544.
Path 113 | total_timesteps 1554.
Path 114 | total_timesteps 1565.
Path 115 | total_timesteps 1580.
Path 116 | total_timesteps 1592.
Path 117 | total_timesteps 1608.
Path 118 | total_timesteps 1616.
Path 119 | total_timesteps 1628.
Path 120 | total_timesteps 1637.
Path 121 | total_timesteps 1649.
Path 122 | total_timesteps 1662.
Path 123 | total_timesteps 1675.
Path 124 | total_timesteps 1691.
Path 125 | total_timesteps 1700.
Path 126 | total_timesteps 1711.
Path 127 | total_timesteps 1729.
Path 128 | total_timesteps 1743.
Path 129 | total_timesteps 1760.
Path 130 | total_timesteps 1771.
Path 131 | total_timesteps 1785.
Path 132 | total_timesteps 1805.
Path 133 | total_timesteps 1823.
Path 134 | total_timesteps 1837.
Path 135 | total_timesteps 1846.
Path 136 | total_timesteps 1865.
Path 137 | total_timesteps 1882.
Path 138 | total_timesteps 1893.
Path 139 | total_timesteps 1906.
Path 140 | total_timesteps 1923.
Path 141 | total_timesteps 1938.
Path 142 | total_timesteps 1957.
Path 143 | total_timesteps 1971.
Path 144 | total_timesteps 1982.
Path 145 | total_timesteps 1993.
Path 146 | total_timesteps 2005.
Path 147 | total_timesteps 2017.
Path 148 | total_timesteps 2033.
Path 149 | total_timesteps 2047.
Path 150 | total_timesteps 2065.
Path 151 | total_timesteps 2077.
Path 152 | total_timesteps 2089.
Path 153 | total_timesteps 2099.
Path 154 | total_timesteps 2110.
Path 155 | total_timesteps 2123.
Path 156 | total_timesteps 2136.
Path 157 | total_timesteps 2147.
Path 158 | total_timesteps 2159.
Path 159 | total_timesteps 2172.
Path 160 | total_timesteps 2180.
Path 161 | total_timesteps 2198.
Path 162 | total_timesteps 2208.
Path 163 | total_timesteps 2221.
Path 164 | total_timesteps 2238.
Path 165 | total_timesteps 2253.
Path 166 | total_timesteps 2265.
Path 167 | total_timesteps 2294.
Path 168 | total_timesteps 2310.
Path 169 | total_timesteps 2324.
Path 170 | total_timesteps 2335.
Path 171 | total_timesteps 2346.
Path 172 | total_timesteps 2368.
Path 173 | total_timesteps 2379.
Path 174 | total_timesteps 2389.
Path 175 | total_timesteps 2400.
Path 176 | total_timesteps 2413.
Path 177 | total_timesteps 2423.
Path 178 | total_timesteps 2437.
Path 179 | total_timesteps 2448.
Path 180 | total_timesteps 2464.
Path 181 | total_timesteps 2477.
Path 182 | total_timesteps 2487.
Path 183 | total_timesteps 2502.
Path 184 | total_timesteps 2513.
Path 185 | total_timesteps 2529.
Path 186 | total_timesteps 2542.
Path 187 | total_timesteps 2552.
Path 188 | total_timesteps 2565.
Path 189 | total_timesteps 2580.
Path 190 | total_timesteps 2594.
Path 191 | total_timesteps 2608.
Path 192 | total_timesteps 2620.
Path 193 | total_timesteps 2636.
Path 194 | total_timesteps 2650.
Path 195 | total_timesteps 2660.
Path 196 | total_timesteps 2677.
Path 197 | total_timesteps 2692.
Path 198 | total_timesteps 2706.
Path 199 | total_timesteps 2719.
Path 200 | total_timesteps 2728.
Path 201 | total_timesteps 2744.
Path 202 | total_timesteps 2754.
Path 203 | total_timesteps 2765.
Path 204 | total_timesteps 2777.
Path 205 | total_timesteps 2789.
Path 206 | total_timesteps 2813.
Path 207 | total_timesteps 2819.
Path 208 | total_timesteps 2836.
Path 209 | total_timesteps 2856.
Path 210 | total_timesteps 2874.
Path 211 | total_timesteps 2890.
Path 212 | total_timesteps 2899.
Path 213 | total_timesteps 2910.
Path 214 | total_timesteps 2919.
Path 215 | total_timesteps 2928.
Path 216 | total_timesteps 2939.
Path 217 | total_timesteps 2948.
Path 218 | total_timesteps 2961.
Path 219 | total_timesteps 2977.
Path 220 | total_timesteps 2987.
Path 221 | total_timesteps 3005.
Path 222 | total_timesteps 3020.
Path 223 | total_timesteps 3037.
Path 224 | total_timesteps 3048.
Path 225 | total_timesteps 3057.
Path 226 | total_timesteps 3072.
Path 227 | total_timesteps 3081.
Path 228 | total_timesteps 3102.
Path 229 | total_timesteps 3112.
Path 230 | total_timesteps 3123.
Path 231 | total_timesteps 3132.
Path 232 | total_timesteps 3150.
Path 233 | total_timesteps 3163.
Path 234 | total_timesteps 3176.
Path 235 | total_timesteps 3192.
Path 236 | total_timesteps 3208.
Path 237 | total_timesteps 3224.
Path 238 | total_timesteps 3235.
Path 239 | total_timesteps 3247.
Path 240 | total_timesteps 3258.
Path 241 | total_timesteps 3276.
Path 242 | total_timesteps 3288.
Path 243 | total_timesteps 3300.
Path 244 | total_timesteps 3312.
Path 245 | total_timesteps 3331.
Path 246 | total_timesteps 3340.
Path 247 | total_timesteps 3349.
Path 248 | total_timesteps 3357.
Path 249 | total_timesteps 3367.
Path 250 | total_timesteps 3378.
Path 251 | total_timesteps 3389.
Path 252 | total_timesteps 3401.
Path 253 | total_timesteps 3419.
Path 254 | total_timesteps 3429.
Path 255 | total_timesteps 3442.
Path 256 | total_timesteps 3454.
Path 257 | total_timesteps 3465.
Path 258 | total_timesteps 3483.
Path 259 | total_timesteps 3501.
Path 260 | total_timesteps 3518.
Path 261 | total_timesteps 3533.
Path 262 | total_timesteps 3546.
Path 263 | total_timesteps 3561.
Path 264 | total_timesteps 3576.
Path 265 | total_timesteps 3586.
Path 266 | total_timesteps 3604.
Path 267 | total_timesteps 3619.
Path 268 | total_timesteps 3632.
Path 269 | total_timesteps 3644.
Path 270 | total_timesteps 3658.
Path 271 | total_timesteps 3669.
Path 272 | total_timesteps 3679.
Path 273 | total_timesteps 3696.
Path 274 | total_timesteps 3707.
Path 275 | total_timesteps 3719.
Path 276 | total_timesteps 3729.
Path 277 | total_timesteps 3740.
Path 278 | total_timesteps 3756.
Path 279 | total_timesteps 3771.
Path 280 | total_timesteps 3786.
Path 281 | total_timesteps 3798.
Path 282 | total_timesteps 3809.
Path 283 | total_timesteps 3817.
Path 284 | total_timesteps 3828.
Path 285 | total_timesteps 3834.
Path 286 | total_timesteps 3851.
Path 287 | total_timesteps 3863.
Path 288 | total_timesteps 3877.
Path 289 | total_timesteps 3892.
Path 290 | total_timesteps 3902.
Path 291 | total_timesteps 3919.
Path 292 | total_timesteps 3940.
Path 293 | total_timesteps 3958.
Path 294 | total_timesteps 3971.
Path 295 | total_timesteps 3990.
Path 296 | total_timesteps 4001.
Path 297 | total_timesteps 4013.
Path 298 | total_timesteps 4027.
Path 299 | total_timesteps 4039.
Path 300 | total_timesteps 4049.
Path 301 | total_timesteps 4062.
Path 302 | total_timesteps 4072.
Path 303 | total_timesteps 4089.
Path 304 | total_timesteps 4104.
Path 305 | total_timesteps 4115.
Path 306 | total_timesteps 4132.
Path 307 | total_timesteps 4148.
Path 308 | total_timesteps 4159.
Path 309 | total_timesteps 4171.
Path 310 | total_timesteps 4189.
Path 311 | total_timesteps 4197.
Path 312 | total_timesteps 4212.
Path 313 | total_timesteps 4221.
Path 314 | total_timesteps 4231.
Path 315 | total_timesteps 4241.
Path 316 | total_timesteps 4257.
Path 317 | total_timesteps 4272.
Path 318 | total_timesteps 4284.
Path 319 | total_timesteps 4295.
Path 320 | total_timesteps 4309.
Path 321 | total_timesteps 4317.
Path 322 | total_timesteps 4326.
Path 323 | total_timesteps 4334.
Path 324 | total_timesteps 4345.
Path 325 | total_timesteps 4357.
Path 326 | total_timesteps 4374.
Path 327 | total_timesteps 4392.
Path 328 | total_timesteps 4403.
Path 329 | total_timesteps 4415.
Path 330 | total_timesteps 4427.
Path 331 | total_timesteps 4439.
Path 332 | total_timesteps 4447.
Path 333 | total_timesteps 4458.
Path 334 | total_timesteps 4474.
Path 335 | total_timesteps 4495.
Path 336 | total_timesteps 4508.
Path 337 | total_timesteps 4520.
Path 338 | total_timesteps 4536.
Path 339 | total_timesteps 4545.
Path 340 | total_timesteps 4565.
Path 341 | total_timesteps 4574.
Path 342 | total_timesteps 4584.
Path 343 | total_timesteps 4595.
Path 344 | total_timesteps 4609.
Path 345 | total_timesteps 4628.
Path 346 | total_timesteps 4652.
Path 347 | total_timesteps 4664.
Path 348 | total_timesteps 4680.
Path 349 | total_timesteps 4695.
Path 350 | total_timesteps 4710.
Path 351 | total_timesteps 4722.
Path 352 | total_timesteps 4737.
Path 353 | total_timesteps 4752.
Path 354 | total_timesteps 4767.
Path 355 | total_timesteps 4778.
Path 356 | total_timesteps 4788.
Path 357 | total_timesteps 4799.
Path 358 | total_timesteps 4816.
Path 359 | total_timesteps 4832.
Path 360 | total_timesteps 4843.
Path 361 | total_timesteps 4851.
Path 362 | total_timesteps 4866.
Path 363 | total_timesteps 4875.
Path 364 | total_timesteps 4896.
Path 365 | total_timesteps 4907.
Path 366 | total_timesteps 4924.
Path 367 | total_timesteps 4935.
Path 368 | total_timesteps 4949.
Path 369 | total_timesteps 4965.
Path 370 | total_timesteps 4980.
Path 371 | total_timesteps 4993.
Path 372 | total_timesteps 5001.
Path 373 | total_timesteps 5013.
Path 374 | total_timesteps 5023.
Path 375 | total_timesteps 5040.
Path 376 | total_timesteps 5049.
Path 377 | total_timesteps 5063.
Path 378 | total_timesteps 5078.
Path 379 | total_timesteps 5091.
Path 380 | total_timesteps 5103.
Path 381 | total_timesteps 5110.
Path 382 | total_timesteps 5123.
Path 383 | total_timesteps 5133.
Path 384 | total_timesteps 5143.
Path 385 | total_timesteps 5153.
Path 386 | total_timesteps 5166.
Path 387 | total_timesteps 5177.
Path 388 | total_timesteps 5189.
Path 389 | total_timesteps 5213.
Path 390 | total_timesteps 5225.
Path 391 | total_timesteps 5237.
Path 392 | total_timesteps 5254.
Path 393 | total_timesteps 5266.
Path 394 | total_timesteps 5278.
Path 395 | total_timesteps 5292.
Path 396 | total_timesteps 5302.
Path 397 | total_timesteps 5324.
Path 398 | total_timesteps 5341.
Path 399 | total_timesteps 5355.
Path 400 | total_timesteps 5368.
Path 401 | total_timesteps 5378.
Path 402 | total_timesteps 5394.
Path 403 | total_timesteps 5406.
Path 404 | total_timesteps 5417.
Path 405 | total_timesteps 5432.
Path 406 | total_timesteps 5449.
Path 407 | total_timesteps 5461.
Path 408 | total_timesteps 5478.
Path 409 | total_timesteps 5487.
Path 410 | total_timesteps 5497.
Path 411 | total_timesteps 5508.
Path 412 | total_timesteps 5521.
Path 413 | total_timesteps 5530.
Path 414 | total_timesteps 5548.
Path 415 | total_timesteps 5566.
Path 416 | total_timesteps 5581.
Path 417 | total_timesteps 5592.
Path 418 | total_timesteps 5602.
Path 419 | total_timesteps 5611.
Path 420 | total_timesteps 5624.
Path 421 | total_timesteps 5632.
Path 422 | total_timesteps 5647.
Path 423 | total_timesteps 5666.
Path 424 | total_timesteps 5681.
Path 425 | total_timesteps 5688.
Path 426 | total_timesteps 5700.
Path 427 | total_timesteps 5712.
Path 428 | total_timesteps 5726.
Path 429 | total_timesteps 5737.
Path 430 | total_timesteps 5747.
Path 431 | total_timesteps 5760.
Path 432 | total_timesteps 5775.
Path 433 | total_timesteps 5787.
Path 434 | total_timesteps 5797.
Path 435 | total_timesteps 5808.
Path 436 | total_timesteps 5819.
Path 437 | total_timesteps 5835.
Path 438 | total_timesteps 5848.
Path 439 | total_timesteps 5860.
Path 440 | total_timesteps 5875.
Path 441 | total_timesteps 5887.
Path 442 | total_timesteps 5905.
Path 443 | total_timesteps 5915.
Path 444 | total_timesteps 5922.
Path 445 | total_timesteps 5936.
Path 446 | total_timesteps 5946.
Path 447 | total_timesteps 5962.
Path 448 | total_timesteps 5979.
Path 449 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.3    |
| Iteration     | 28       |
| MaximumReturn | 8.53     |
| MinimumReturn | -20.8    |
| TotalSamples  | 120184   |
----------------------------
itr #29 | 
Fitting dynamics.
Validation loss = 0.005589911248534918
Validation loss = 0.006060861982405186
Validation loss = 0.005844507832080126
Validation loss = 0.005907401442527771
Validation loss = 0.005229960661381483
Validation loss = 0.005468800198286772
Validation loss = 0.005447703879326582
Validation loss = 0.0061444384045898914
Validation loss = 0.005397615022957325
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 35.
Path 4 | total_timesteps 44.
Path 5 | total_timesteps 57.
Path 6 | total_timesteps 74.
Path 7 | total_timesteps 85.
Path 8 | total_timesteps 98.
Path 9 | total_timesteps 119.
Path 10 | total_timesteps 132.
Path 11 | total_timesteps 142.
Path 12 | total_timesteps 155.
Path 13 | total_timesteps 165.
Path 14 | total_timesteps 187.
Path 15 | total_timesteps 199.
Path 16 | total_timesteps 213.
Path 17 | total_timesteps 226.
Path 18 | total_timesteps 242.
Path 19 | total_timesteps 251.
Path 20 | total_timesteps 260.
Path 21 | total_timesteps 274.
Path 22 | total_timesteps 287.
Path 23 | total_timesteps 310.
Path 24 | total_timesteps 328.
Path 25 | total_timesteps 341.
Path 26 | total_timesteps 350.
Path 27 | total_timesteps 364.
Path 28 | total_timesteps 380.
Path 29 | total_timesteps 392.
Path 30 | total_timesteps 410.
Path 31 | total_timesteps 425.
Path 32 | total_timesteps 436.
Path 33 | total_timesteps 452.
Path 34 | total_timesteps 462.
Path 35 | total_timesteps 477.
Path 36 | total_timesteps 490.
Path 37 | total_timesteps 502.
Path 38 | total_timesteps 513.
Path 39 | total_timesteps 525.
Path 40 | total_timesteps 539.
Path 41 | total_timesteps 553.
Path 42 | total_timesteps 571.
Path 43 | total_timesteps 585.
Path 44 | total_timesteps 598.
Path 45 | total_timesteps 611.
Path 46 | total_timesteps 626.
Path 47 | total_timesteps 646.
Path 48 | total_timesteps 657.
Path 49 | total_timesteps 671.
Path 50 | total_timesteps 684.
Path 51 | total_timesteps 692.
Path 52 | total_timesteps 710.
Path 53 | total_timesteps 723.
Path 54 | total_timesteps 736.
Path 55 | total_timesteps 747.
Path 56 | total_timesteps 767.
Path 57 | total_timesteps 782.
Path 58 | total_timesteps 797.
Path 59 | total_timesteps 814.
Path 60 | total_timesteps 823.
Path 61 | total_timesteps 839.
Path 62 | total_timesteps 849.
Path 63 | total_timesteps 862.
Path 64 | total_timesteps 877.
Path 65 | total_timesteps 892.
Path 66 | total_timesteps 905.
Path 67 | total_timesteps 916.
Path 68 | total_timesteps 925.
Path 69 | total_timesteps 945.
Path 70 | total_timesteps 959.
Path 71 | total_timesteps 972.
Path 72 | total_timesteps 986.
Path 73 | total_timesteps 997.
Path 74 | total_timesteps 1015.
Path 75 | total_timesteps 1034.
Path 76 | total_timesteps 1046.
Path 77 | total_timesteps 1057.
Path 78 | total_timesteps 1073.
Path 79 | total_timesteps 1087.
Path 80 | total_timesteps 1098.
Path 81 | total_timesteps 1113.
Path 82 | total_timesteps 1120.
Path 83 | total_timesteps 1135.
Path 84 | total_timesteps 1145.
Path 85 | total_timesteps 1158.
Path 86 | total_timesteps 1171.
Path 87 | total_timesteps 1183.
Path 88 | total_timesteps 1197.
Path 89 | total_timesteps 1209.
Path 90 | total_timesteps 1220.
Path 91 | total_timesteps 1232.
Path 92 | total_timesteps 1240.
Path 93 | total_timesteps 1252.
Path 94 | total_timesteps 1267.
Path 95 | total_timesteps 1287.
Path 96 | total_timesteps 1303.
Path 97 | total_timesteps 1317.
Path 98 | total_timesteps 1329.
Path 99 | total_timesteps 1346.
Path 100 | total_timesteps 1364.
Path 101 | total_timesteps 1374.
Path 102 | total_timesteps 1388.
Path 103 | total_timesteps 1397.
Path 104 | total_timesteps 1408.
Path 105 | total_timesteps 1425.
Path 106 | total_timesteps 1436.
Path 107 | total_timesteps 1452.
Path 108 | total_timesteps 1476.
Path 109 | total_timesteps 1491.
Path 110 | total_timesteps 1503.
Path 111 | total_timesteps 1520.
Path 112 | total_timesteps 1537.
Path 113 | total_timesteps 1554.
Path 114 | total_timesteps 1568.
Path 115 | total_timesteps 1578.
Path 116 | total_timesteps 1595.
Path 117 | total_timesteps 1606.
Path 118 | total_timesteps 1620.
Path 119 | total_timesteps 1633.
Path 120 | total_timesteps 1646.
Path 121 | total_timesteps 1660.
Path 122 | total_timesteps 1676.
Path 123 | total_timesteps 1690.
Path 124 | total_timesteps 1700.
Path 125 | total_timesteps 1710.
Path 126 | total_timesteps 1726.
Path 127 | total_timesteps 1741.
Path 128 | total_timesteps 1754.
Path 129 | total_timesteps 1763.
Path 130 | total_timesteps 1779.
Path 131 | total_timesteps 1790.
Path 132 | total_timesteps 1804.
Path 133 | total_timesteps 1819.
Path 134 | total_timesteps 1836.
Path 135 | total_timesteps 1850.
Path 136 | total_timesteps 1861.
Path 137 | total_timesteps 1875.
Path 138 | total_timesteps 1890.
Path 139 | total_timesteps 1903.
Path 140 | total_timesteps 1918.
Path 141 | total_timesteps 1930.
Path 142 | total_timesteps 1944.
Path 143 | total_timesteps 1957.
Path 144 | total_timesteps 1970.
Path 145 | total_timesteps 1981.
Path 146 | total_timesteps 1995.
Path 147 | total_timesteps 2016.
Path 148 | total_timesteps 2026.
Path 149 | total_timesteps 2039.
Path 150 | total_timesteps 2050.
Path 151 | total_timesteps 2065.
Path 152 | total_timesteps 2078.
Path 153 | total_timesteps 2088.
Path 154 | total_timesteps 2105.
Path 155 | total_timesteps 2118.
Path 156 | total_timesteps 2133.
Path 157 | total_timesteps 2147.
Path 158 | total_timesteps 2161.
Path 159 | total_timesteps 2182.
Path 160 | total_timesteps 2200.
Path 161 | total_timesteps 2212.
Path 162 | total_timesteps 2223.
Path 163 | total_timesteps 2230.
Path 164 | total_timesteps 2240.
Path 165 | total_timesteps 2253.
Path 166 | total_timesteps 2268.
Path 167 | total_timesteps 2286.
Path 168 | total_timesteps 2300.
Path 169 | total_timesteps 2319.
Path 170 | total_timesteps 2331.
Path 171 | total_timesteps 2343.
Path 172 | total_timesteps 2365.
Path 173 | total_timesteps 2408.
Path 174 | total_timesteps 2428.
Path 175 | total_timesteps 2444.
Path 176 | total_timesteps 2456.
Path 177 | total_timesteps 2467.
Path 178 | total_timesteps 2484.
Path 179 | total_timesteps 2496.
Path 180 | total_timesteps 2509.
Path 181 | total_timesteps 2520.
Path 182 | total_timesteps 2533.
Path 183 | total_timesteps 2542.
Path 184 | total_timesteps 2553.
Path 185 | total_timesteps 2566.
Path 186 | total_timesteps 2579.
Path 187 | total_timesteps 2598.
Path 188 | total_timesteps 2610.
Path 189 | total_timesteps 2629.
Path 190 | total_timesteps 2642.
Path 191 | total_timesteps 2655.
Path 192 | total_timesteps 2664.
Path 193 | total_timesteps 2677.
Path 194 | total_timesteps 2697.
Path 195 | total_timesteps 2716.
Path 196 | total_timesteps 2729.
Path 197 | total_timesteps 2745.
Path 198 | total_timesteps 2761.
Path 199 | total_timesteps 2774.
Path 200 | total_timesteps 2787.
Path 201 | total_timesteps 2799.
Path 202 | total_timesteps 2816.
Path 203 | total_timesteps 2832.
Path 204 | total_timesteps 2842.
Path 205 | total_timesteps 2851.
Path 206 | total_timesteps 2864.
Path 207 | total_timesteps 2882.
Path 208 | total_timesteps 2890.
Path 209 | total_timesteps 2905.
Path 210 | total_timesteps 2921.
Path 211 | total_timesteps 2938.
Path 212 | total_timesteps 2952.
Path 213 | total_timesteps 2967.
Path 214 | total_timesteps 2977.
Path 215 | total_timesteps 2988.
Path 216 | total_timesteps 3001.
Path 217 | total_timesteps 3012.
Path 218 | total_timesteps 3042.
Path 219 | total_timesteps 3053.
Path 220 | total_timesteps 3068.
Path 221 | total_timesteps 3077.
Path 222 | total_timesteps 3096.
Path 223 | total_timesteps 3107.
Path 224 | total_timesteps 3121.
Path 225 | total_timesteps 3140.
Path 226 | total_timesteps 3149.
Path 227 | total_timesteps 3165.
Path 228 | total_timesteps 3178.
Path 229 | total_timesteps 3193.
Path 230 | total_timesteps 3207.
Path 231 | total_timesteps 3220.
Path 232 | total_timesteps 3230.
Path 233 | total_timesteps 3246.
Path 234 | total_timesteps 3258.
Path 235 | total_timesteps 3273.
Path 236 | total_timesteps 3287.
Path 237 | total_timesteps 3303.
Path 238 | total_timesteps 3321.
Path 239 | total_timesteps 3337.
Path 240 | total_timesteps 3354.
Path 241 | total_timesteps 3365.
Path 242 | total_timesteps 3373.
Path 243 | total_timesteps 3386.
Path 244 | total_timesteps 3395.
Path 245 | total_timesteps 3408.
Path 246 | total_timesteps 3420.
Path 247 | total_timesteps 3432.
Path 248 | total_timesteps 3448.
Path 249 | total_timesteps 3462.
Path 250 | total_timesteps 3478.
Path 251 | total_timesteps 3489.
Path 252 | total_timesteps 3502.
Path 253 | total_timesteps 3520.
Path 254 | total_timesteps 3533.
Path 255 | total_timesteps 3544.
Path 256 | total_timesteps 3557.
Path 257 | total_timesteps 3571.
Path 258 | total_timesteps 3585.
Path 259 | total_timesteps 3599.
Path 260 | total_timesteps 3611.
Path 261 | total_timesteps 3623.
Path 262 | total_timesteps 3633.
Path 263 | total_timesteps 3652.
Path 264 | total_timesteps 3672.
Path 265 | total_timesteps 3686.
Path 266 | total_timesteps 3703.
Path 267 | total_timesteps 3714.
Path 268 | total_timesteps 3725.
Path 269 | total_timesteps 3742.
Path 270 | total_timesteps 3756.
Path 271 | total_timesteps 3777.
Path 272 | total_timesteps 3796.
Path 273 | total_timesteps 3816.
Path 274 | total_timesteps 3836.
Path 275 | total_timesteps 3856.
Path 276 | total_timesteps 3873.
Path 277 | total_timesteps 3885.
Path 278 | total_timesteps 3902.
Path 279 | total_timesteps 3915.
Path 280 | total_timesteps 3931.
Path 281 | total_timesteps 3948.
Path 282 | total_timesteps 3957.
Path 283 | total_timesteps 3969.
Path 284 | total_timesteps 3984.
Path 285 | total_timesteps 3995.
Path 286 | total_timesteps 4003.
Path 287 | total_timesteps 4015.
Path 288 | total_timesteps 4031.
Path 289 | total_timesteps 4048.
Path 290 | total_timesteps 4059.
Path 291 | total_timesteps 4072.
Path 292 | total_timesteps 4087.
Path 293 | total_timesteps 4104.
Path 294 | total_timesteps 4118.
Path 295 | total_timesteps 4131.
Path 296 | total_timesteps 4151.
Path 297 | total_timesteps 4170.
Path 298 | total_timesteps 4185.
Path 299 | total_timesteps 4198.
Path 300 | total_timesteps 4207.
Path 301 | total_timesteps 4226.
Path 302 | total_timesteps 4238.
Path 303 | total_timesteps 4251.
Path 304 | total_timesteps 4264.
Path 305 | total_timesteps 4274.
Path 306 | total_timesteps 4286.
Path 307 | total_timesteps 4301.
Path 308 | total_timesteps 4317.
Path 309 | total_timesteps 4335.
Path 310 | total_timesteps 4351.
Path 311 | total_timesteps 4367.
Path 312 | total_timesteps 4379.
Path 313 | total_timesteps 4390.
Path 314 | total_timesteps 4404.
Path 315 | total_timesteps 4413.
Path 316 | total_timesteps 4422.
Path 317 | total_timesteps 4436.
Path 318 | total_timesteps 4446.
Path 319 | total_timesteps 4461.
Path 320 | total_timesteps 4469.
Path 321 | total_timesteps 4477.
Path 322 | total_timesteps 4492.
Path 323 | total_timesteps 4509.
Path 324 | total_timesteps 4530.
Path 325 | total_timesteps 4550.
Path 326 | total_timesteps 4568.
Path 327 | total_timesteps 4580.
Path 328 | total_timesteps 4595.
Path 329 | total_timesteps 4612.
Path 330 | total_timesteps 4624.
Path 331 | total_timesteps 4640.
Path 332 | total_timesteps 4652.
Path 333 | total_timesteps 4674.
Path 334 | total_timesteps 4684.
Path 335 | total_timesteps 4696.
Path 336 | total_timesteps 4715.
Path 337 | total_timesteps 4729.
Path 338 | total_timesteps 4746.
Path 339 | total_timesteps 4765.
Path 340 | total_timesteps 4779.
Path 341 | total_timesteps 4799.
Path 342 | total_timesteps 4818.
Path 343 | total_timesteps 4833.
Path 344 | total_timesteps 4845.
Path 345 | total_timesteps 4861.
Path 346 | total_timesteps 4871.
Path 347 | total_timesteps 4887.
Path 348 | total_timesteps 4900.
Path 349 | total_timesteps 4915.
Path 350 | total_timesteps 4932.
Path 351 | total_timesteps 4943.
Path 352 | total_timesteps 4956.
Path 353 | total_timesteps 4968.
Path 354 | total_timesteps 4985.
Path 355 | total_timesteps 4999.
Path 356 | total_timesteps 5008.
Path 357 | total_timesteps 5023.
Path 358 | total_timesteps 5043.
Path 359 | total_timesteps 5061.
Path 360 | total_timesteps 5073.
Path 361 | total_timesteps 5089.
Path 362 | total_timesteps 5106.
Path 363 | total_timesteps 5117.
Path 364 | total_timesteps 5130.
Path 365 | total_timesteps 5144.
Path 366 | total_timesteps 5154.
Path 367 | total_timesteps 5165.
Path 368 | total_timesteps 5180.
Path 369 | total_timesteps 5194.
Path 370 | total_timesteps 5207.
Path 371 | total_timesteps 5220.
Path 372 | total_timesteps 5233.
Path 373 | total_timesteps 5252.
Path 374 | total_timesteps 5267.
Path 375 | total_timesteps 5282.
Path 376 | total_timesteps 5294.
Path 377 | total_timesteps 5307.
Path 378 | total_timesteps 5318.
Path 379 | total_timesteps 5335.
Path 380 | total_timesteps 5347.
Path 381 | total_timesteps 5359.
Path 382 | total_timesteps 5373.
Path 383 | total_timesteps 5385.
Path 384 | total_timesteps 5402.
Path 385 | total_timesteps 5415.
Path 386 | total_timesteps 5427.
Path 387 | total_timesteps 5439.
Path 388 | total_timesteps 5453.
Path 389 | total_timesteps 5467.
Path 390 | total_timesteps 5476.
Path 391 | total_timesteps 5488.
Path 392 | total_timesteps 5506.
Path 393 | total_timesteps 5517.
Path 394 | total_timesteps 5530.
Path 395 | total_timesteps 5547.
Path 396 | total_timesteps 5570.
Path 397 | total_timesteps 5583.
Path 398 | total_timesteps 5603.
Path 399 | total_timesteps 5615.
Path 400 | total_timesteps 5626.
Path 401 | total_timesteps 5641.
Path 402 | total_timesteps 5650.
Path 403 | total_timesteps 5661.
Path 404 | total_timesteps 5672.
Path 405 | total_timesteps 5685.
Path 406 | total_timesteps 5693.
Path 407 | total_timesteps 5705.
Path 408 | total_timesteps 5726.
Path 409 | total_timesteps 5742.
Path 410 | total_timesteps 5767.
Path 411 | total_timesteps 5784.
Path 412 | total_timesteps 5803.
Path 413 | total_timesteps 5812.
Path 414 | total_timesteps 5828.
Path 415 | total_timesteps 5842.
Path 416 | total_timesteps 5855.
Path 417 | total_timesteps 5869.
Path 418 | total_timesteps 5889.
Path 419 | total_timesteps 5905.
Path 420 | total_timesteps 5922.
Path 421 | total_timesteps 5939.
Path 422 | total_timesteps 5961.
Path 423 | total_timesteps 5973.
Path 424 | total_timesteps 5982.
Path 425 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.9    |
| Iteration     | 29       |
| MaximumReturn | 3.32     |
| MinimumReturn | -21      |
| TotalSamples  | 124192   |
----------------------------
itr #30 | 
Fitting dynamics.
Validation loss = 0.006306366063654423
Validation loss = 0.0054295179434120655
Validation loss = 0.005291468929499388
Validation loss = 0.005420309491455555
Validation loss = 0.005253057461231947
Validation loss = 0.00509816175326705
Validation loss = 0.005089910235255957
Validation loss = 0.005107132717967033
Validation loss = 0.0052558802999556065
Validation loss = 0.005374418571591377
Validation loss = 0.005076651927083731
Validation loss = 0.005322624929249287
Validation loss = 0.005242347251623869
Validation loss = 0.005157945677638054
Validation loss = 0.005197306629270315
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 47.
Path 5 | total_timesteps 59.
Path 6 | total_timesteps 73.
Path 7 | total_timesteps 84.
Path 8 | total_timesteps 96.
Path 9 | total_timesteps 106.
Path 10 | total_timesteps 121.
Path 11 | total_timesteps 136.
Path 12 | total_timesteps 151.
Path 13 | total_timesteps 160.
Path 14 | total_timesteps 172.
Path 15 | total_timesteps 189.
Path 16 | total_timesteps 200.
Path 17 | total_timesteps 213.
Path 18 | total_timesteps 223.
Path 19 | total_timesteps 234.
Path 20 | total_timesteps 252.
Path 21 | total_timesteps 269.
Path 22 | total_timesteps 280.
Path 23 | total_timesteps 295.
Path 24 | total_timesteps 310.
Path 25 | total_timesteps 318.
Path 26 | total_timesteps 332.
Path 27 | total_timesteps 348.
Path 28 | total_timesteps 356.
Path 29 | total_timesteps 369.
Path 30 | total_timesteps 385.
Path 31 | total_timesteps 398.
Path 32 | total_timesteps 408.
Path 33 | total_timesteps 421.
Path 34 | total_timesteps 431.
Path 35 | total_timesteps 449.
Path 36 | total_timesteps 466.
Path 37 | total_timesteps 484.
Path 38 | total_timesteps 497.
Path 39 | total_timesteps 510.
Path 40 | total_timesteps 524.
Path 41 | total_timesteps 535.
Path 42 | total_timesteps 556.
Path 43 | total_timesteps 577.
Path 44 | total_timesteps 587.
Path 45 | total_timesteps 597.
Path 46 | total_timesteps 606.
Path 47 | total_timesteps 622.
Path 48 | total_timesteps 634.
Path 49 | total_timesteps 647.
Path 50 | total_timesteps 656.
Path 51 | total_timesteps 673.
Path 52 | total_timesteps 683.
Path 53 | total_timesteps 703.
Path 54 | total_timesteps 713.
Path 55 | total_timesteps 731.
Path 56 | total_timesteps 743.
Path 57 | total_timesteps 766.
Path 58 | total_timesteps 778.
Path 59 | total_timesteps 791.
Path 60 | total_timesteps 804.
Path 61 | total_timesteps 817.
Path 62 | total_timesteps 832.
Path 63 | total_timesteps 843.
Path 64 | total_timesteps 857.
Path 65 | total_timesteps 876.
Path 66 | total_timesteps 892.
Path 67 | total_timesteps 901.
Path 68 | total_timesteps 919.
Path 69 | total_timesteps 935.
Path 70 | total_timesteps 952.
Path 71 | total_timesteps 972.
Path 72 | total_timesteps 989.
Path 73 | total_timesteps 1001.
Path 74 | total_timesteps 1016.
Path 75 | total_timesteps 1034.
Path 76 | total_timesteps 1046.
Path 77 | total_timesteps 1057.
Path 78 | total_timesteps 1071.
Path 79 | total_timesteps 1089.
Path 80 | total_timesteps 1100.
Path 81 | total_timesteps 1115.
Path 82 | total_timesteps 1132.
Path 83 | total_timesteps 1149.
Path 84 | total_timesteps 1161.
Path 85 | total_timesteps 1179.
Path 86 | total_timesteps 1192.
Path 87 | total_timesteps 1203.
Path 88 | total_timesteps 1221.
Path 89 | total_timesteps 1236.
Path 90 | total_timesteps 1250.
Path 91 | total_timesteps 1260.
Path 92 | total_timesteps 1281.
Path 93 | total_timesteps 1293.
Path 94 | total_timesteps 1301.
Path 95 | total_timesteps 1312.
Path 96 | total_timesteps 1324.
Path 97 | total_timesteps 1337.
Path 98 | total_timesteps 1349.
Path 99 | total_timesteps 1363.
Path 100 | total_timesteps 1379.
Path 101 | total_timesteps 1398.
Path 102 | total_timesteps 1409.
Path 103 | total_timesteps 1428.
Path 104 | total_timesteps 1442.
Path 105 | total_timesteps 1457.
Path 106 | total_timesteps 1468.
Path 107 | total_timesteps 1486.
Path 108 | total_timesteps 1497.
Path 109 | total_timesteps 1506.
Path 110 | total_timesteps 1524.
Path 111 | total_timesteps 1537.
Path 112 | total_timesteps 1548.
Path 113 | total_timesteps 1563.
Path 114 | total_timesteps 1579.
Path 115 | total_timesteps 1601.
Path 116 | total_timesteps 1611.
Path 117 | total_timesteps 1622.
Path 118 | total_timesteps 1640.
Path 119 | total_timesteps 1654.
Path 120 | total_timesteps 1662.
Path 121 | total_timesteps 1675.
Path 122 | total_timesteps 1691.
Path 123 | total_timesteps 1704.
Path 124 | total_timesteps 1726.
Path 125 | total_timesteps 1741.
Path 126 | total_timesteps 1748.
Path 127 | total_timesteps 1757.
Path 128 | total_timesteps 1767.
Path 129 | total_timesteps 1778.
Path 130 | total_timesteps 1788.
Path 131 | total_timesteps 1800.
Path 132 | total_timesteps 1816.
Path 133 | total_timesteps 1827.
Path 134 | total_timesteps 1843.
Path 135 | total_timesteps 1857.
Path 136 | total_timesteps 1878.
Path 137 | total_timesteps 1893.
Path 138 | total_timesteps 1905.
Path 139 | total_timesteps 1922.
Path 140 | total_timesteps 1938.
Path 141 | total_timesteps 1954.
Path 142 | total_timesteps 1973.
Path 143 | total_timesteps 1987.
Path 144 | total_timesteps 2002.
Path 145 | total_timesteps 2017.
Path 146 | total_timesteps 2028.
Path 147 | total_timesteps 2041.
Path 148 | total_timesteps 2051.
Path 149 | total_timesteps 2065.
Path 150 | total_timesteps 2076.
Path 151 | total_timesteps 2086.
Path 152 | total_timesteps 2099.
Path 153 | total_timesteps 2115.
Path 154 | total_timesteps 2124.
Path 155 | total_timesteps 2135.
Path 156 | total_timesteps 2144.
Path 157 | total_timesteps 2162.
Path 158 | total_timesteps 2180.
Path 159 | total_timesteps 2192.
Path 160 | total_timesteps 2202.
Path 161 | total_timesteps 2215.
Path 162 | total_timesteps 2225.
Path 163 | total_timesteps 2237.
Path 164 | total_timesteps 2247.
Path 165 | total_timesteps 2258.
Path 166 | total_timesteps 2272.
Path 167 | total_timesteps 2285.
Path 168 | total_timesteps 2298.
Path 169 | total_timesteps 2308.
Path 170 | total_timesteps 2320.
Path 171 | total_timesteps 2337.
Path 172 | total_timesteps 2355.
Path 173 | total_timesteps 2366.
Path 174 | total_timesteps 2375.
Path 175 | total_timesteps 2389.
Path 176 | total_timesteps 2410.
Path 177 | total_timesteps 2429.
Path 178 | total_timesteps 2447.
Path 179 | total_timesteps 2460.
Path 180 | total_timesteps 2488.
Path 181 | total_timesteps 2497.
Path 182 | total_timesteps 2513.
Path 183 | total_timesteps 2524.
Path 184 | total_timesteps 2540.
Path 185 | total_timesteps 2553.
Path 186 | total_timesteps 2566.
Path 187 | total_timesteps 2583.
Path 188 | total_timesteps 2594.
Path 189 | total_timesteps 2605.
Path 190 | total_timesteps 2617.
Path 191 | total_timesteps 2629.
Path 192 | total_timesteps 2641.
Path 193 | total_timesteps 2654.
Path 194 | total_timesteps 2669.
Path 195 | total_timesteps 2681.
Path 196 | total_timesteps 2697.
Path 197 | total_timesteps 2719.
Path 198 | total_timesteps 2735.
Path 199 | total_timesteps 2745.
Path 200 | total_timesteps 2756.
Path 201 | total_timesteps 2770.
Path 202 | total_timesteps 2791.
Path 203 | total_timesteps 2801.
Path 204 | total_timesteps 2814.
Path 205 | total_timesteps 2825.
Path 206 | total_timesteps 2839.
Path 207 | total_timesteps 2853.
Path 208 | total_timesteps 2869.
Path 209 | total_timesteps 2888.
Path 210 | total_timesteps 2907.
Path 211 | total_timesteps 2920.
Path 212 | total_timesteps 2942.
Path 213 | total_timesteps 2952.
Path 214 | total_timesteps 2965.
Path 215 | total_timesteps 2976.
Path 216 | total_timesteps 2993.
Path 217 | total_timesteps 3006.
Path 218 | total_timesteps 3023.
Path 219 | total_timesteps 3032.
Path 220 | total_timesteps 3045.
Path 221 | total_timesteps 3054.
Path 222 | total_timesteps 3065.
Path 223 | total_timesteps 3083.
Path 224 | total_timesteps 3096.
Path 225 | total_timesteps 3106.
Path 226 | total_timesteps 3117.
Path 227 | total_timesteps 3124.
Path 228 | total_timesteps 3135.
Path 229 | total_timesteps 3152.
Path 230 | total_timesteps 3166.
Path 231 | total_timesteps 3176.
Path 232 | total_timesteps 3191.
Path 233 | total_timesteps 3198.
Path 234 | total_timesteps 3214.
Path 235 | total_timesteps 3224.
Path 236 | total_timesteps 3240.
Path 237 | total_timesteps 3253.
Path 238 | total_timesteps 3269.
Path 239 | total_timesteps 3280.
Path 240 | total_timesteps 3290.
Path 241 | total_timesteps 3309.
Path 242 | total_timesteps 3335.
Path 243 | total_timesteps 3343.
Path 244 | total_timesteps 3361.
Path 245 | total_timesteps 3371.
Path 246 | total_timesteps 3384.
Path 247 | total_timesteps 3395.
Path 248 | total_timesteps 3411.
Path 249 | total_timesteps 3432.
Path 250 | total_timesteps 3445.
Path 251 | total_timesteps 3456.
Path 252 | total_timesteps 3470.
Path 253 | total_timesteps 3481.
Path 254 | total_timesteps 3493.
Path 255 | total_timesteps 3503.
Path 256 | total_timesteps 3512.
Path 257 | total_timesteps 3522.
Path 258 | total_timesteps 3538.
Path 259 | total_timesteps 3553.
Path 260 | total_timesteps 3566.
Path 261 | total_timesteps 3579.
Path 262 | total_timesteps 3589.
Path 263 | total_timesteps 3602.
Path 264 | total_timesteps 3617.
Path 265 | total_timesteps 3632.
Path 266 | total_timesteps 3657.
Path 267 | total_timesteps 3676.
Path 268 | total_timesteps 3686.
Path 269 | total_timesteps 3700.
Path 270 | total_timesteps 3715.
Path 271 | total_timesteps 3730.
Path 272 | total_timesteps 3742.
Path 273 | total_timesteps 3752.
Path 274 | total_timesteps 3761.
Path 275 | total_timesteps 3774.
Path 276 | total_timesteps 3788.
Path 277 | total_timesteps 3799.
Path 278 | total_timesteps 3808.
Path 279 | total_timesteps 3819.
Path 280 | total_timesteps 3826.
Path 281 | total_timesteps 3841.
Path 282 | total_timesteps 3863.
Path 283 | total_timesteps 3878.
Path 284 | total_timesteps 3885.
Path 285 | total_timesteps 3899.
Path 286 | total_timesteps 3916.
Path 287 | total_timesteps 3930.
Path 288 | total_timesteps 3945.
Path 289 | total_timesteps 3955.
Path 290 | total_timesteps 3968.
Path 291 | total_timesteps 3984.
Path 292 | total_timesteps 4002.
Path 293 | total_timesteps 4013.
Path 294 | total_timesteps 4032.
Path 295 | total_timesteps 4047.
Path 296 | total_timesteps 4058.
Path 297 | total_timesteps 4072.
Path 298 | total_timesteps 4106.
Path 299 | total_timesteps 4125.
Path 300 | total_timesteps 4136.
Path 301 | total_timesteps 4152.
Path 302 | total_timesteps 4161.
Path 303 | total_timesteps 4176.
Path 304 | total_timesteps 4187.
Path 305 | total_timesteps 4202.
Path 306 | total_timesteps 4209.
Path 307 | total_timesteps 4222.
Path 308 | total_timesteps 4237.
Path 309 | total_timesteps 4248.
Path 310 | total_timesteps 4261.
Path 311 | total_timesteps 4280.
Path 312 | total_timesteps 4298.
Path 313 | total_timesteps 4317.
Path 314 | total_timesteps 4334.
Path 315 | total_timesteps 4348.
Path 316 | total_timesteps 4360.
Path 317 | total_timesteps 4376.
Path 318 | total_timesteps 4390.
Path 319 | total_timesteps 4405.
Path 320 | total_timesteps 4421.
Path 321 | total_timesteps 4434.
Path 322 | total_timesteps 4445.
Path 323 | total_timesteps 4460.
Path 324 | total_timesteps 4474.
Path 325 | total_timesteps 4486.
Path 326 | total_timesteps 4502.
Path 327 | total_timesteps 4511.
Path 328 | total_timesteps 4529.
Path 329 | total_timesteps 4542.
Path 330 | total_timesteps 4558.
Path 331 | total_timesteps 4570.
Path 332 | total_timesteps 4587.
Path 333 | total_timesteps 4601.
Path 334 | total_timesteps 4615.
Path 335 | total_timesteps 4631.
Path 336 | total_timesteps 4647.
Path 337 | total_timesteps 4659.
Path 338 | total_timesteps 4678.
Path 339 | total_timesteps 4693.
Path 340 | total_timesteps 4709.
Path 341 | total_timesteps 4717.
Path 342 | total_timesteps 4731.
Path 343 | total_timesteps 4742.
Path 344 | total_timesteps 4752.
Path 345 | total_timesteps 4766.
Path 346 | total_timesteps 4779.
Path 347 | total_timesteps 4792.
Path 348 | total_timesteps 4809.
Path 349 | total_timesteps 4821.
Path 350 | total_timesteps 4837.
Path 351 | total_timesteps 4849.
Path 352 | total_timesteps 4859.
Path 353 | total_timesteps 4871.
Path 354 | total_timesteps 4885.
Path 355 | total_timesteps 4898.
Path 356 | total_timesteps 4912.
Path 357 | total_timesteps 4931.
Path 358 | total_timesteps 4947.
Path 359 | total_timesteps 4961.
Path 360 | total_timesteps 4974.
Path 361 | total_timesteps 4990.
Path 362 | total_timesteps 5005.
Path 363 | total_timesteps 5019.
Path 364 | total_timesteps 5033.
Path 365 | total_timesteps 5048.
Path 366 | total_timesteps 5066.
Path 367 | total_timesteps 5078.
Path 368 | total_timesteps 5089.
Path 369 | total_timesteps 5102.
Path 370 | total_timesteps 5117.
Path 371 | total_timesteps 5126.
Path 372 | total_timesteps 5146.
Path 373 | total_timesteps 5161.
Path 374 | total_timesteps 5176.
Path 375 | total_timesteps 5185.
Path 376 | total_timesteps 5192.
Path 377 | total_timesteps 5203.
Path 378 | total_timesteps 5219.
Path 379 | total_timesteps 5229.
Path 380 | total_timesteps 5239.
Path 381 | total_timesteps 5250.
Path 382 | total_timesteps 5264.
Path 383 | total_timesteps 5280.
Path 384 | total_timesteps 5300.
Path 385 | total_timesteps 5329.
Path 386 | total_timesteps 5340.
Path 387 | total_timesteps 5355.
Path 388 | total_timesteps 5367.
Path 389 | total_timesteps 5383.
Path 390 | total_timesteps 5397.
Path 391 | total_timesteps 5412.
Path 392 | total_timesteps 5429.
Path 393 | total_timesteps 5448.
Path 394 | total_timesteps 5460.
Path 395 | total_timesteps 5474.
Path 396 | total_timesteps 5492.
Path 397 | total_timesteps 5503.
Path 398 | total_timesteps 5521.
Path 399 | total_timesteps 5541.
Path 400 | total_timesteps 5557.
Path 401 | total_timesteps 5574.
Path 402 | total_timesteps 5591.
Path 403 | total_timesteps 5600.
Path 404 | total_timesteps 5609.
Path 405 | total_timesteps 5618.
Path 406 | total_timesteps 5625.
Path 407 | total_timesteps 5638.
Path 408 | total_timesteps 5653.
Path 409 | total_timesteps 5668.
Path 410 | total_timesteps 5681.
Path 411 | total_timesteps 5696.
Path 412 | total_timesteps 5706.
Path 413 | total_timesteps 5720.
Path 414 | total_timesteps 5732.
Path 415 | total_timesteps 5746.
Path 416 | total_timesteps 5760.
Path 417 | total_timesteps 5776.
Path 418 | total_timesteps 5786.
Path 419 | total_timesteps 5801.
Path 420 | total_timesteps 5814.
Path 421 | total_timesteps 5834.
Path 422 | total_timesteps 5846.
Path 423 | total_timesteps 5860.
Path 424 | total_timesteps 5874.
Path 425 | total_timesteps 5890.
Path 426 | total_timesteps 5900.
Path 427 | total_timesteps 5909.
Path 428 | total_timesteps 5919.
Path 429 | total_timesteps 5930.
Path 430 | total_timesteps 5949.
Path 431 | total_timesteps 5962.
Path 432 | total_timesteps 5974.
Path 433 | total_timesteps 5985.
Path 434 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.5    |
| Iteration     | 30       |
| MaximumReturn | -1.75    |
| MinimumReturn | -22.4    |
| TotalSamples  | 128197   |
----------------------------
itr #31 | 
Fitting dynamics.
Validation loss = 0.005047062411904335
Validation loss = 0.005039956886321306
Validation loss = 0.005028776358813047
Validation loss = 0.005157650448381901
Validation loss = 0.0050014713779091835
Validation loss = 0.004989994689822197
Validation loss = 0.005234570242464542
Validation loss = 0.005006738472729921
Validation loss = 0.0048847496509552
Validation loss = 0.005098133347928524
Validation loss = 0.005154138430953026
Validation loss = 0.0051046088337898254
Validation loss = 0.004949440713971853
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 37.
Path 4 | total_timesteps 52.
Path 5 | total_timesteps 66.
Path 6 | total_timesteps 76.
Path 7 | total_timesteps 92.
Path 8 | total_timesteps 108.
Path 9 | total_timesteps 126.
Path 10 | total_timesteps 134.
Path 11 | total_timesteps 142.
Path 12 | total_timesteps 158.
Path 13 | total_timesteps 167.
Path 14 | total_timesteps 179.
Path 15 | total_timesteps 190.
Path 16 | total_timesteps 207.
Path 17 | total_timesteps 221.
Path 18 | total_timesteps 237.
Path 19 | total_timesteps 248.
Path 20 | total_timesteps 259.
Path 21 | total_timesteps 272.
Path 22 | total_timesteps 293.
Path 23 | total_timesteps 310.
Path 24 | total_timesteps 319.
Path 25 | total_timesteps 336.
Path 26 | total_timesteps 346.
Path 27 | total_timesteps 356.
Path 28 | total_timesteps 368.
Path 29 | total_timesteps 395.
Path 30 | total_timesteps 405.
Path 31 | total_timesteps 421.
Path 32 | total_timesteps 432.
Path 33 | total_timesteps 439.
Path 34 | total_timesteps 455.
Path 35 | total_timesteps 473.
Path 36 | total_timesteps 483.
Path 37 | total_timesteps 496.
Path 38 | total_timesteps 512.
Path 39 | total_timesteps 531.
Path 40 | total_timesteps 546.
Path 41 | total_timesteps 565.
Path 42 | total_timesteps 575.
Path 43 | total_timesteps 584.
Path 44 | total_timesteps 593.
Path 45 | total_timesteps 604.
Path 46 | total_timesteps 621.
Path 47 | total_timesteps 638.
Path 48 | total_timesteps 650.
Path 49 | total_timesteps 661.
Path 50 | total_timesteps 677.
Path 51 | total_timesteps 689.
Path 52 | total_timesteps 704.
Path 53 | total_timesteps 712.
Path 54 | total_timesteps 728.
Path 55 | total_timesteps 740.
Path 56 | total_timesteps 750.
Path 57 | total_timesteps 759.
Path 58 | total_timesteps 770.
Path 59 | total_timesteps 780.
Path 60 | total_timesteps 789.
Path 61 | total_timesteps 800.
Path 62 | total_timesteps 813.
Path 63 | total_timesteps 832.
Path 64 | total_timesteps 847.
Path 65 | total_timesteps 858.
Path 66 | total_timesteps 865.
Path 67 | total_timesteps 880.
Path 68 | total_timesteps 890.
Path 69 | total_timesteps 903.
Path 70 | total_timesteps 916.
Path 71 | total_timesteps 941.
Path 72 | total_timesteps 951.
Path 73 | total_timesteps 964.
Path 74 | total_timesteps 974.
Path 75 | total_timesteps 983.
Path 76 | total_timesteps 1001.
Path 77 | total_timesteps 1011.
Path 78 | total_timesteps 1022.
Path 79 | total_timesteps 1039.
Path 80 | total_timesteps 1053.
Path 81 | total_timesteps 1068.
Path 82 | total_timesteps 1085.
Path 83 | total_timesteps 1103.
Path 84 | total_timesteps 1112.
Path 85 | total_timesteps 1123.
Path 86 | total_timesteps 1137.
Path 87 | total_timesteps 1154.
Path 88 | total_timesteps 1167.
Path 89 | total_timesteps 1176.
Path 90 | total_timesteps 1191.
Path 91 | total_timesteps 1205.
Path 92 | total_timesteps 1224.
Path 93 | total_timesteps 1240.
Path 94 | total_timesteps 1258.
Path 95 | total_timesteps 1275.
Path 96 | total_timesteps 1292.
Path 97 | total_timesteps 1312.
Path 98 | total_timesteps 1331.
Path 99 | total_timesteps 1339.
Path 100 | total_timesteps 1353.
Path 101 | total_timesteps 1366.
Path 102 | total_timesteps 1378.
Path 103 | total_timesteps 1392.
Path 104 | total_timesteps 1400.
Path 105 | total_timesteps 1412.
Path 106 | total_timesteps 1433.
Path 107 | total_timesteps 1446.
Path 108 | total_timesteps 1457.
Path 109 | total_timesteps 1467.
Path 110 | total_timesteps 1478.
Path 111 | total_timesteps 1498.
Path 112 | total_timesteps 1505.
Path 113 | total_timesteps 1519.
Path 114 | total_timesteps 1543.
Path 115 | total_timesteps 1557.
Path 116 | total_timesteps 1566.
Path 117 | total_timesteps 1577.
Path 118 | total_timesteps 1594.
Path 119 | total_timesteps 1607.
Path 120 | total_timesteps 1623.
Path 121 | total_timesteps 1638.
Path 122 | total_timesteps 1656.
Path 123 | total_timesteps 1671.
Path 124 | total_timesteps 1682.
Path 125 | total_timesteps 1695.
Path 126 | total_timesteps 1717.
Path 127 | total_timesteps 1733.
Path 128 | total_timesteps 1748.
Path 129 | total_timesteps 1763.
Path 130 | total_timesteps 1779.
Path 131 | total_timesteps 1792.
Path 132 | total_timesteps 1810.
Path 133 | total_timesteps 1827.
Path 134 | total_timesteps 1842.
Path 135 | total_timesteps 1858.
Path 136 | total_timesteps 1876.
Path 137 | total_timesteps 1887.
Path 138 | total_timesteps 1902.
Path 139 | total_timesteps 1916.
Path 140 | total_timesteps 1932.
Path 141 | total_timesteps 1944.
Path 142 | total_timesteps 1957.
Path 143 | total_timesteps 1973.
Path 144 | total_timesteps 1987.
Path 145 | total_timesteps 1997.
Path 146 | total_timesteps 2008.
Path 147 | total_timesteps 2018.
Path 148 | total_timesteps 2031.
Path 149 | total_timesteps 2045.
Path 150 | total_timesteps 2057.
Path 151 | total_timesteps 2070.
Path 152 | total_timesteps 2085.
Path 153 | total_timesteps 2100.
Path 154 | total_timesteps 2109.
Path 155 | total_timesteps 2120.
Path 156 | total_timesteps 2131.
Path 157 | total_timesteps 2143.
Path 158 | total_timesteps 2155.
Path 159 | total_timesteps 2170.
Path 160 | total_timesteps 2181.
Path 161 | total_timesteps 2196.
Path 162 | total_timesteps 2212.
Path 163 | total_timesteps 2224.
Path 164 | total_timesteps 2240.
Path 165 | total_timesteps 2254.
Path 166 | total_timesteps 2268.
Path 167 | total_timesteps 2279.
Path 168 | total_timesteps 2295.
Path 169 | total_timesteps 2303.
Path 170 | total_timesteps 2314.
Path 171 | total_timesteps 2330.
Path 172 | total_timesteps 2340.
Path 173 | total_timesteps 2353.
Path 174 | total_timesteps 2368.
Path 175 | total_timesteps 2386.
Path 176 | total_timesteps 2403.
Path 177 | total_timesteps 2417.
Path 178 | total_timesteps 2432.
Path 179 | total_timesteps 2446.
Path 180 | total_timesteps 2454.
Path 181 | total_timesteps 2468.
Path 182 | total_timesteps 2477.
Path 183 | total_timesteps 2491.
Path 184 | total_timesteps 2508.
Path 185 | total_timesteps 2521.
Path 186 | total_timesteps 2535.
Path 187 | total_timesteps 2543.
Path 188 | total_timesteps 2560.
Path 189 | total_timesteps 2584.
Path 190 | total_timesteps 2597.
Path 191 | total_timesteps 2608.
Path 192 | total_timesteps 2620.
Path 193 | total_timesteps 2632.
Path 194 | total_timesteps 2647.
Path 195 | total_timesteps 2661.
Path 196 | total_timesteps 2683.
Path 197 | total_timesteps 2700.
Path 198 | total_timesteps 2714.
Path 199 | total_timesteps 2729.
Path 200 | total_timesteps 2739.
Path 201 | total_timesteps 2753.
Path 202 | total_timesteps 2771.
Path 203 | total_timesteps 2787.
Path 204 | total_timesteps 2806.
Path 205 | total_timesteps 2821.
Path 206 | total_timesteps 2836.
Path 207 | total_timesteps 2852.
Path 208 | total_timesteps 2871.
Path 209 | total_timesteps 2880.
Path 210 | total_timesteps 2890.
Path 211 | total_timesteps 2902.
Path 212 | total_timesteps 2921.
Path 213 | total_timesteps 2929.
Path 214 | total_timesteps 2942.
Path 215 | total_timesteps 2962.
Path 216 | total_timesteps 2975.
Path 217 | total_timesteps 2986.
Path 218 | total_timesteps 2996.
Path 219 | total_timesteps 3011.
Path 220 | total_timesteps 3035.
Path 221 | total_timesteps 3049.
Path 222 | total_timesteps 3061.
Path 223 | total_timesteps 3079.
Path 224 | total_timesteps 3100.
Path 225 | total_timesteps 3112.
Path 226 | total_timesteps 3123.
Path 227 | total_timesteps 3136.
Path 228 | total_timesteps 3149.
Path 229 | total_timesteps 3169.
Path 230 | total_timesteps 3185.
Path 231 | total_timesteps 3196.
Path 232 | total_timesteps 3213.
Path 233 | total_timesteps 3228.
Path 234 | total_timesteps 3245.
Path 235 | total_timesteps 3256.
Path 236 | total_timesteps 3271.
Path 237 | total_timesteps 3292.
Path 238 | total_timesteps 3302.
Path 239 | total_timesteps 3314.
Path 240 | total_timesteps 3327.
Path 241 | total_timesteps 3337.
Path 242 | total_timesteps 3351.
Path 243 | total_timesteps 3366.
Path 244 | total_timesteps 3380.
Path 245 | total_timesteps 3389.
Path 246 | total_timesteps 3402.
Path 247 | total_timesteps 3420.
Path 248 | total_timesteps 3435.
Path 249 | total_timesteps 3452.
Path 250 | total_timesteps 3470.
Path 251 | total_timesteps 3477.
Path 252 | total_timesteps 3493.
Path 253 | total_timesteps 3505.
Path 254 | total_timesteps 3524.
Path 255 | total_timesteps 3539.
Path 256 | total_timesteps 3553.
Path 257 | total_timesteps 3563.
Path 258 | total_timesteps 3583.
Path 259 | total_timesteps 3595.
Path 260 | total_timesteps 3607.
Path 261 | total_timesteps 3620.
Path 262 | total_timesteps 3628.
Path 263 | total_timesteps 3642.
Path 264 | total_timesteps 3660.
Path 265 | total_timesteps 3675.
Path 266 | total_timesteps 3692.
Path 267 | total_timesteps 3706.
Path 268 | total_timesteps 3722.
Path 269 | total_timesteps 3736.
Path 270 | total_timesteps 3758.
Path 271 | total_timesteps 3771.
Path 272 | total_timesteps 3789.
Path 273 | total_timesteps 3803.
Path 274 | total_timesteps 3815.
Path 275 | total_timesteps 3827.
Path 276 | total_timesteps 3834.
Path 277 | total_timesteps 3853.
Path 278 | total_timesteps 3870.
Path 279 | total_timesteps 3882.
Path 280 | total_timesteps 3903.
Path 281 | total_timesteps 3919.
Path 282 | total_timesteps 3935.
Path 283 | total_timesteps 3945.
Path 284 | total_timesteps 3955.
Path 285 | total_timesteps 3966.
Path 286 | total_timesteps 3979.
Path 287 | total_timesteps 3994.
Path 288 | total_timesteps 4005.
Path 289 | total_timesteps 4019.
Path 290 | total_timesteps 4038.
Path 291 | total_timesteps 4053.
Path 292 | total_timesteps 4065.
Path 293 | total_timesteps 4080.
Path 294 | total_timesteps 4094.
Path 295 | total_timesteps 4108.
Path 296 | total_timesteps 4120.
Path 297 | total_timesteps 4135.
Path 298 | total_timesteps 4146.
Path 299 | total_timesteps 4157.
Path 300 | total_timesteps 4168.
Path 301 | total_timesteps 4182.
Path 302 | total_timesteps 4194.
Path 303 | total_timesteps 4204.
Path 304 | total_timesteps 4221.
Path 305 | total_timesteps 4233.
Path 306 | total_timesteps 4248.
Path 307 | total_timesteps 4263.
Path 308 | total_timesteps 4272.
Path 309 | total_timesteps 4289.
Path 310 | total_timesteps 4306.
Path 311 | total_timesteps 4317.
Path 312 | total_timesteps 4335.
Path 313 | total_timesteps 4358.
Path 314 | total_timesteps 4366.
Path 315 | total_timesteps 4377.
Path 316 | total_timesteps 4391.
Path 317 | total_timesteps 4402.
Path 318 | total_timesteps 4420.
Path 319 | total_timesteps 4433.
Path 320 | total_timesteps 4446.
Path 321 | total_timesteps 4459.
Path 322 | total_timesteps 4474.
Path 323 | total_timesteps 4486.
Path 324 | total_timesteps 4505.
Path 325 | total_timesteps 4516.
Path 326 | total_timesteps 4531.
Path 327 | total_timesteps 4546.
Path 328 | total_timesteps 4561.
Path 329 | total_timesteps 4586.
Path 330 | total_timesteps 4597.
Path 331 | total_timesteps 4621.
Path 332 | total_timesteps 4635.
Path 333 | total_timesteps 4649.
Path 334 | total_timesteps 4660.
Path 335 | total_timesteps 4672.
Path 336 | total_timesteps 4684.
Path 337 | total_timesteps 4694.
Path 338 | total_timesteps 4707.
Path 339 | total_timesteps 4717.
Path 340 | total_timesteps 4729.
Path 341 | total_timesteps 4742.
Path 342 | total_timesteps 4751.
Path 343 | total_timesteps 4763.
Path 344 | total_timesteps 4775.
Path 345 | total_timesteps 4787.
Path 346 | total_timesteps 4801.
Path 347 | total_timesteps 4813.
Path 348 | total_timesteps 4825.
Path 349 | total_timesteps 4840.
Path 350 | total_timesteps 4852.
Path 351 | total_timesteps 4860.
Path 352 | total_timesteps 4869.
Path 353 | total_timesteps 4882.
Path 354 | total_timesteps 4890.
Path 355 | total_timesteps 4903.
Path 356 | total_timesteps 4921.
Path 357 | total_timesteps 4938.
Path 358 | total_timesteps 4964.
Path 359 | total_timesteps 4974.
Path 360 | total_timesteps 4989.
Path 361 | total_timesteps 4997.
Path 362 | total_timesteps 5009.
Path 363 | total_timesteps 5027.
Path 364 | total_timesteps 5044.
Path 365 | total_timesteps 5052.
Path 366 | total_timesteps 5067.
Path 367 | total_timesteps 5095.
Path 368 | total_timesteps 5112.
Path 369 | total_timesteps 5133.
Path 370 | total_timesteps 5148.
Path 371 | total_timesteps 5162.
Path 372 | total_timesteps 5174.
Path 373 | total_timesteps 5187.
Path 374 | total_timesteps 5199.
Path 375 | total_timesteps 5213.
Path 376 | total_timesteps 5226.
Path 377 | total_timesteps 5243.
Path 378 | total_timesteps 5259.
Path 379 | total_timesteps 5272.
Path 380 | total_timesteps 5286.
Path 381 | total_timesteps 5298.
Path 382 | total_timesteps 5313.
Path 383 | total_timesteps 5328.
Path 384 | total_timesteps 5343.
Path 385 | total_timesteps 5358.
Path 386 | total_timesteps 5376.
Path 387 | total_timesteps 5384.
Path 388 | total_timesteps 5397.
Path 389 | total_timesteps 5410.
Path 390 | total_timesteps 5420.
Path 391 | total_timesteps 5438.
Path 392 | total_timesteps 5447.
Path 393 | total_timesteps 5461.
Path 394 | total_timesteps 5478.
Path 395 | total_timesteps 5496.
Path 396 | total_timesteps 5513.
Path 397 | total_timesteps 5527.
Path 398 | total_timesteps 5539.
Path 399 | total_timesteps 5550.
Path 400 | total_timesteps 5559.
Path 401 | total_timesteps 5572.
Path 402 | total_timesteps 5586.
Path 403 | total_timesteps 5599.
Path 404 | total_timesteps 5609.
Path 405 | total_timesteps 5621.
Path 406 | total_timesteps 5633.
Path 407 | total_timesteps 5648.
Path 408 | total_timesteps 5660.
Path 409 | total_timesteps 5674.
Path 410 | total_timesteps 5688.
Path 411 | total_timesteps 5701.
Path 412 | total_timesteps 5711.
Path 413 | total_timesteps 5721.
Path 414 | total_timesteps 5732.
Path 415 | total_timesteps 5747.
Path 416 | total_timesteps 5755.
Path 417 | total_timesteps 5780.
Path 418 | total_timesteps 5790.
Path 419 | total_timesteps 5799.
Path 420 | total_timesteps 5817.
Path 421 | total_timesteps 5832.
Path 422 | total_timesteps 5860.
Path 423 | total_timesteps 5883.
Path 424 | total_timesteps 5897.
Path 425 | total_timesteps 5918.
Path 426 | total_timesteps 5937.
Path 427 | total_timesteps 5949.
Path 428 | total_timesteps 5960.
Path 429 | total_timesteps 5970.
Path 430 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.7    |
| Iteration     | 31       |
| MaximumReturn | -3.25    |
| MinimumReturn | -22      |
| TotalSamples  | 132202   |
----------------------------
itr #32 | 
Fitting dynamics.
Validation loss = 0.004986970219761133
Validation loss = 0.005352803040295839
Validation loss = 0.0050733983516693115
Validation loss = 0.005180836655199528
Validation loss = 0.005132361780852079
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 36.
Path 4 | total_timesteps 50.
Path 5 | total_timesteps 62.
Path 6 | total_timesteps 75.
Path 7 | total_timesteps 86.
Path 8 | total_timesteps 99.
Path 9 | total_timesteps 113.
Path 10 | total_timesteps 131.
Path 11 | total_timesteps 148.
Path 12 | total_timesteps 160.
Path 13 | total_timesteps 171.
Path 14 | total_timesteps 182.
Path 15 | total_timesteps 189.
Path 16 | total_timesteps 204.
Path 17 | total_timesteps 217.
Path 18 | total_timesteps 227.
Path 19 | total_timesteps 242.
Path 20 | total_timesteps 253.
Path 21 | total_timesteps 264.
Path 22 | total_timesteps 275.
Path 23 | total_timesteps 284.
Path 24 | total_timesteps 297.
Path 25 | total_timesteps 309.
Path 26 | total_timesteps 325.
Path 27 | total_timesteps 339.
Path 28 | total_timesteps 351.
Path 29 | total_timesteps 363.
Path 30 | total_timesteps 384.
Path 31 | total_timesteps 396.
Path 32 | total_timesteps 404.
Path 33 | total_timesteps 424.
Path 34 | total_timesteps 434.
Path 35 | total_timesteps 446.
Path 36 | total_timesteps 458.
Path 37 | total_timesteps 474.
Path 38 | total_timesteps 486.
Path 39 | total_timesteps 495.
Path 40 | total_timesteps 505.
Path 41 | total_timesteps 520.
Path 42 | total_timesteps 531.
Path 43 | total_timesteps 539.
Path 44 | total_timesteps 549.
Path 45 | total_timesteps 567.
Path 46 | total_timesteps 583.
Path 47 | total_timesteps 598.
Path 48 | total_timesteps 613.
Path 49 | total_timesteps 632.
Path 50 | total_timesteps 647.
Path 51 | total_timesteps 657.
Path 52 | total_timesteps 668.
Path 53 | total_timesteps 680.
Path 54 | total_timesteps 697.
Path 55 | total_timesteps 709.
Path 56 | total_timesteps 724.
Path 57 | total_timesteps 737.
Path 58 | total_timesteps 757.
Path 59 | total_timesteps 772.
Path 60 | total_timesteps 787.
Path 61 | total_timesteps 798.
Path 62 | total_timesteps 810.
Path 63 | total_timesteps 832.
Path 64 | total_timesteps 851.
Path 65 | total_timesteps 862.
Path 66 | total_timesteps 879.
Path 67 | total_timesteps 888.
Path 68 | total_timesteps 902.
Path 69 | total_timesteps 912.
Path 70 | total_timesteps 921.
Path 71 | total_timesteps 935.
Path 72 | total_timesteps 949.
Path 73 | total_timesteps 963.
Path 74 | total_timesteps 976.
Path 75 | total_timesteps 988.
Path 76 | total_timesteps 999.
Path 77 | total_timesteps 1018.
Path 78 | total_timesteps 1029.
Path 79 | total_timesteps 1041.
Path 80 | total_timesteps 1058.
Path 81 | total_timesteps 1082.
Path 82 | total_timesteps 1096.
Path 83 | total_timesteps 1110.
Path 84 | total_timesteps 1126.
Path 85 | total_timesteps 1137.
Path 86 | total_timesteps 1148.
Path 87 | total_timesteps 1157.
Path 88 | total_timesteps 1168.
Path 89 | total_timesteps 1181.
Path 90 | total_timesteps 1190.
Path 91 | total_timesteps 1202.
Path 92 | total_timesteps 1213.
Path 93 | total_timesteps 1225.
Path 94 | total_timesteps 1237.
Path 95 | total_timesteps 1253.
Path 96 | total_timesteps 1262.
Path 97 | total_timesteps 1273.
Path 98 | total_timesteps 1284.
Path 99 | total_timesteps 1294.
Path 100 | total_timesteps 1317.
Path 101 | total_timesteps 1330.
Path 102 | total_timesteps 1345.
Path 103 | total_timesteps 1356.
Path 104 | total_timesteps 1366.
Path 105 | total_timesteps 1378.
Path 106 | total_timesteps 1389.
Path 107 | total_timesteps 1404.
Path 108 | total_timesteps 1426.
Path 109 | total_timesteps 1439.
Path 110 | total_timesteps 1454.
Path 111 | total_timesteps 1470.
Path 112 | total_timesteps 1480.
Path 113 | total_timesteps 1491.
Path 114 | total_timesteps 1504.
Path 115 | total_timesteps 1517.
Path 116 | total_timesteps 1536.
Path 117 | total_timesteps 1548.
Path 118 | total_timesteps 1558.
Path 119 | total_timesteps 1574.
Path 120 | total_timesteps 1590.
Path 121 | total_timesteps 1602.
Path 122 | total_timesteps 1611.
Path 123 | total_timesteps 1624.
Path 124 | total_timesteps 1642.
Path 125 | total_timesteps 1654.
Path 126 | total_timesteps 1664.
Path 127 | total_timesteps 1678.
Path 128 | total_timesteps 1693.
Path 129 | total_timesteps 1704.
Path 130 | total_timesteps 1716.
Path 131 | total_timesteps 1725.
Path 132 | total_timesteps 1739.
Path 133 | total_timesteps 1747.
Path 134 | total_timesteps 1761.
Path 135 | total_timesteps 1769.
Path 136 | total_timesteps 1783.
Path 137 | total_timesteps 1800.
Path 138 | total_timesteps 1809.
Path 139 | total_timesteps 1822.
Path 140 | total_timesteps 1833.
Path 141 | total_timesteps 1849.
Path 142 | total_timesteps 1867.
Path 143 | total_timesteps 1878.
Path 144 | total_timesteps 1890.
Path 145 | total_timesteps 1910.
Path 146 | total_timesteps 1918.
Path 147 | total_timesteps 1932.
Path 148 | total_timesteps 1948.
Path 149 | total_timesteps 1961.
Path 150 | total_timesteps 1970.
Path 151 | total_timesteps 1981.
Path 152 | total_timesteps 1994.
Path 153 | total_timesteps 2007.
Path 154 | total_timesteps 2020.
Path 155 | total_timesteps 2036.
Path 156 | total_timesteps 2056.
Path 157 | total_timesteps 2065.
Path 158 | total_timesteps 2076.
Path 159 | total_timesteps 2088.
Path 160 | total_timesteps 2096.
Path 161 | total_timesteps 2114.
Path 162 | total_timesteps 2131.
Path 163 | total_timesteps 2154.
Path 164 | total_timesteps 2174.
Path 165 | total_timesteps 2190.
Path 166 | total_timesteps 2202.
Path 167 | total_timesteps 2215.
Path 168 | total_timesteps 2226.
Path 169 | total_timesteps 2239.
Path 170 | total_timesteps 2254.
Path 171 | total_timesteps 2265.
Path 172 | total_timesteps 2282.
Path 173 | total_timesteps 2293.
Path 174 | total_timesteps 2302.
Path 175 | total_timesteps 2315.
Path 176 | total_timesteps 2325.
Path 177 | total_timesteps 2344.
Path 178 | total_timesteps 2361.
Path 179 | total_timesteps 2377.
Path 180 | total_timesteps 2388.
Path 181 | total_timesteps 2400.
Path 182 | total_timesteps 2413.
Path 183 | total_timesteps 2430.
Path 184 | total_timesteps 2439.
Path 185 | total_timesteps 2449.
Path 186 | total_timesteps 2471.
Path 187 | total_timesteps 2484.
Path 188 | total_timesteps 2494.
Path 189 | total_timesteps 2504.
Path 190 | total_timesteps 2515.
Path 191 | total_timesteps 2527.
Path 192 | total_timesteps 2537.
Path 193 | total_timesteps 2546.
Path 194 | total_timesteps 2563.
Path 195 | total_timesteps 2574.
Path 196 | total_timesteps 2591.
Path 197 | total_timesteps 2602.
Path 198 | total_timesteps 2616.
Path 199 | total_timesteps 2632.
Path 200 | total_timesteps 2648.
Path 201 | total_timesteps 2663.
Path 202 | total_timesteps 2674.
Path 203 | total_timesteps 2687.
Path 204 | total_timesteps 2698.
Path 205 | total_timesteps 2709.
Path 206 | total_timesteps 2721.
Path 207 | total_timesteps 2741.
Path 208 | total_timesteps 2753.
Path 209 | total_timesteps 2766.
Path 210 | total_timesteps 2776.
Path 211 | total_timesteps 2791.
Path 212 | total_timesteps 2804.
Path 213 | total_timesteps 2816.
Path 214 | total_timesteps 2831.
Path 215 | total_timesteps 2843.
Path 216 | total_timesteps 2860.
Path 217 | total_timesteps 2872.
Path 218 | total_timesteps 2884.
Path 219 | total_timesteps 2893.
Path 220 | total_timesteps 2908.
Path 221 | total_timesteps 2919.
Path 222 | total_timesteps 2929.
Path 223 | total_timesteps 2940.
Path 224 | total_timesteps 2949.
Path 225 | total_timesteps 2966.
Path 226 | total_timesteps 2983.
Path 227 | total_timesteps 2996.
Path 228 | total_timesteps 3004.
Path 229 | total_timesteps 3013.
Path 230 | total_timesteps 3033.
Path 231 | total_timesteps 3042.
Path 232 | total_timesteps 3060.
Path 233 | total_timesteps 3071.
Path 234 | total_timesteps 3083.
Path 235 | total_timesteps 3104.
Path 236 | total_timesteps 3113.
Path 237 | total_timesteps 3134.
Path 238 | total_timesteps 3153.
Path 239 | total_timesteps 3166.
Path 240 | total_timesteps 3182.
Path 241 | total_timesteps 3193.
Path 242 | total_timesteps 3226.
Path 243 | total_timesteps 3235.
Path 244 | total_timesteps 3248.
Path 245 | total_timesteps 3257.
Path 246 | total_timesteps 3268.
Path 247 | total_timesteps 3287.
Path 248 | total_timesteps 3294.
Path 249 | total_timesteps 3309.
Path 250 | total_timesteps 3317.
Path 251 | total_timesteps 3329.
Path 252 | total_timesteps 3348.
Path 253 | total_timesteps 3365.
Path 254 | total_timesteps 3380.
Path 255 | total_timesteps 3389.
Path 256 | total_timesteps 3400.
Path 257 | total_timesteps 3412.
Path 258 | total_timesteps 3424.
Path 259 | total_timesteps 3433.
Path 260 | total_timesteps 3450.
Path 261 | total_timesteps 3476.
Path 262 | total_timesteps 3492.
Path 263 | total_timesteps 3502.
Path 264 | total_timesteps 3522.
Path 265 | total_timesteps 3536.
Path 266 | total_timesteps 3549.
Path 267 | total_timesteps 3561.
Path 268 | total_timesteps 3579.
Path 269 | total_timesteps 3589.
Path 270 | total_timesteps 3598.
Path 271 | total_timesteps 3614.
Path 272 | total_timesteps 3624.
Path 273 | total_timesteps 3633.
Path 274 | total_timesteps 3642.
Path 275 | total_timesteps 3655.
Path 276 | total_timesteps 3667.
Path 277 | total_timesteps 3684.
Path 278 | total_timesteps 3702.
Path 279 | total_timesteps 3721.
Path 280 | total_timesteps 3736.
Path 281 | total_timesteps 3747.
Path 282 | total_timesteps 3760.
Path 283 | total_timesteps 3770.
Path 284 | total_timesteps 3792.
Path 285 | total_timesteps 3806.
Path 286 | total_timesteps 3816.
Path 287 | total_timesteps 3830.
Path 288 | total_timesteps 3841.
Path 289 | total_timesteps 3851.
Path 290 | total_timesteps 3866.
Path 291 | total_timesteps 3877.
Path 292 | total_timesteps 3896.
Path 293 | total_timesteps 3914.
Path 294 | total_timesteps 3932.
Path 295 | total_timesteps 3950.
Path 296 | total_timesteps 3964.
Path 297 | total_timesteps 3979.
Path 298 | total_timesteps 3990.
Path 299 | total_timesteps 4000.
Path 300 | total_timesteps 4010.
Path 301 | total_timesteps 4018.
Path 302 | total_timesteps 4029.
Path 303 | total_timesteps 4045.
Path 304 | total_timesteps 4060.
Path 305 | total_timesteps 4074.
Path 306 | total_timesteps 4088.
Path 307 | total_timesteps 4104.
Path 308 | total_timesteps 4116.
Path 309 | total_timesteps 4128.
Path 310 | total_timesteps 4142.
Path 311 | total_timesteps 4156.
Path 312 | total_timesteps 4169.
Path 313 | total_timesteps 4182.
Path 314 | total_timesteps 4194.
Path 315 | total_timesteps 4207.
Path 316 | total_timesteps 4218.
Path 317 | total_timesteps 4229.
Path 318 | total_timesteps 4241.
Path 319 | total_timesteps 4249.
Path 320 | total_timesteps 4261.
Path 321 | total_timesteps 4278.
Path 322 | total_timesteps 4294.
Path 323 | total_timesteps 4309.
Path 324 | total_timesteps 4320.
Path 325 | total_timesteps 4330.
Path 326 | total_timesteps 4342.
Path 327 | total_timesteps 4350.
Path 328 | total_timesteps 4363.
Path 329 | total_timesteps 4374.
Path 330 | total_timesteps 4382.
Path 331 | total_timesteps 4402.
Path 332 | total_timesteps 4415.
Path 333 | total_timesteps 4424.
Path 334 | total_timesteps 4439.
Path 335 | total_timesteps 4452.
Path 336 | total_timesteps 4466.
Path 337 | total_timesteps 4480.
Path 338 | total_timesteps 4493.
Path 339 | total_timesteps 4505.
Path 340 | total_timesteps 4515.
Path 341 | total_timesteps 4534.
Path 342 | total_timesteps 4545.
Path 343 | total_timesteps 4561.
Path 344 | total_timesteps 4576.
Path 345 | total_timesteps 4584.
Path 346 | total_timesteps 4600.
Path 347 | total_timesteps 4614.
Path 348 | total_timesteps 4625.
Path 349 | total_timesteps 4640.
Path 350 | total_timesteps 4649.
Path 351 | total_timesteps 4661.
Path 352 | total_timesteps 4673.
Path 353 | total_timesteps 4689.
Path 354 | total_timesteps 4699.
Path 355 | total_timesteps 4716.
Path 356 | total_timesteps 4727.
Path 357 | total_timesteps 4736.
Path 358 | total_timesteps 4747.
Path 359 | total_timesteps 4759.
Path 360 | total_timesteps 4768.
Path 361 | total_timesteps 4785.
Path 362 | total_timesteps 4797.
Path 363 | total_timesteps 4811.
Path 364 | total_timesteps 4823.
Path 365 | total_timesteps 4838.
Path 366 | total_timesteps 4855.
Path 367 | total_timesteps 4868.
Path 368 | total_timesteps 4885.
Path 369 | total_timesteps 4894.
Path 370 | total_timesteps 4905.
Path 371 | total_timesteps 4920.
Path 372 | total_timesteps 4933.
Path 373 | total_timesteps 4941.
Path 374 | total_timesteps 4951.
Path 375 | total_timesteps 4965.
Path 376 | total_timesteps 4979.
Path 377 | total_timesteps 4986.
Path 378 | total_timesteps 4996.
Path 379 | total_timesteps 5005.
Path 380 | total_timesteps 5020.
Path 381 | total_timesteps 5034.
Path 382 | total_timesteps 5049.
Path 383 | total_timesteps 5059.
Path 384 | total_timesteps 5068.
Path 385 | total_timesteps 5081.
Path 386 | total_timesteps 5092.
Path 387 | total_timesteps 5106.
Path 388 | total_timesteps 5123.
Path 389 | total_timesteps 5133.
Path 390 | total_timesteps 5152.
Path 391 | total_timesteps 5164.
Path 392 | total_timesteps 5175.
Path 393 | total_timesteps 5190.
Path 394 | total_timesteps 5209.
Path 395 | total_timesteps 5217.
Path 396 | total_timesteps 5235.
Path 397 | total_timesteps 5248.
Path 398 | total_timesteps 5258.
Path 399 | total_timesteps 5273.
Path 400 | total_timesteps 5286.
Path 401 | total_timesteps 5297.
Path 402 | total_timesteps 5311.
Path 403 | total_timesteps 5323.
Path 404 | total_timesteps 5336.
Path 405 | total_timesteps 5353.
Path 406 | total_timesteps 5364.
Path 407 | total_timesteps 5372.
Path 408 | total_timesteps 5389.
Path 409 | total_timesteps 5402.
Path 410 | total_timesteps 5420.
Path 411 | total_timesteps 5433.
Path 412 | total_timesteps 5447.
Path 413 | total_timesteps 5459.
Path 414 | total_timesteps 5473.
Path 415 | total_timesteps 5483.
Path 416 | total_timesteps 5498.
Path 417 | total_timesteps 5515.
Path 418 | total_timesteps 5526.
Path 419 | total_timesteps 5537.
Path 420 | total_timesteps 5550.
Path 421 | total_timesteps 5569.
Path 422 | total_timesteps 5580.
Path 423 | total_timesteps 5591.
Path 424 | total_timesteps 5603.
Path 425 | total_timesteps 5615.
Path 426 | total_timesteps 5628.
Path 427 | total_timesteps 5637.
Path 428 | total_timesteps 5649.
Path 429 | total_timesteps 5663.
Path 430 | total_timesteps 5679.
Path 431 | total_timesteps 5697.
Path 432 | total_timesteps 5713.
Path 433 | total_timesteps 5727.
Path 434 | total_timesteps 5748.
Path 435 | total_timesteps 5763.
Path 436 | total_timesteps 5772.
Path 437 | total_timesteps 5793.
Path 438 | total_timesteps 5803.
Path 439 | total_timesteps 5814.
Path 440 | total_timesteps 5827.
Path 441 | total_timesteps 5846.
Path 442 | total_timesteps 5856.
Path 443 | total_timesteps 5868.
Path 444 | total_timesteps 5885.
Path 445 | total_timesteps 5898.
Path 446 | total_timesteps 5907.
Path 447 | total_timesteps 5919.
Path 448 | total_timesteps 5931.
Path 449 | total_timesteps 5944.
Path 450 | total_timesteps 5960.
Path 451 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.4    |
| Iteration     | 32       |
| MaximumReturn | -2.32    |
| MinimumReturn | -23.7    |
| TotalSamples  | 136209   |
----------------------------
