Logging to experiments/gym_fwalker2d/Wa01/Mon-07-Nov-2022-10-29-40-AM-CST_gym_fwalker2d_trpo_iteration_20_seed3214
Print configuration .....
{'env_name': 'gym_fwalker2d', 'random_seeds': [3214, 2431, 2531, 2231], 'save_variables': False, 'model_save_dir': '/tmp/gym_fwalker2d_models/', 'restore_variables': False, 'start_onpol_iter': 0, 'onpol_iters': 33, 'num_path_random': 6, 'num_path_onpol': 6, 'env_horizon': 1000, 'max_train_data': 200000, 'max_val_data': 100000, 'discard_ratio': 0.0, 'dynamics': {'pre_training': {'mode': 'intrinsic_reward', 'itr': 0, 'policy_itr': 20}, 'model': 'nn', 'ensemble': False, 'ensemble_model_count': 5, 'enable_particle_ensemble': True, 'particles': 5, 'obs_var': 1.0, 'intrinsic_reward_coeff': 1.0, 'ita': 1.0, 'mode': 'random', 'val': True, 'n_layers': 4, 'hidden_size': 1000, 'activation': 'relu', 'batch_size': 1000, 'learning_rate': 0.001, 'reg_coeff': 0.0, 'epochs': 200, 'kfac_params': {'learning_rate': 0.1, 'damping': 0.001, 'momentum': 0.9, 'kl_clip': 0.0001, 'cov_ema_decay': 0.99}}, 'policy': {'network_shape': [64, 64], 'init_logstd': 0.0, 'activation': 'tanh', 'reinitialize_every_itr': False}, 'trpo': {'horizon': 1000, 'gamma': 0.99, 'step_size': 0.01, 'iterations': 20, 'batch_size': 50000, 'gae': 0.95, 'visualization': False, 'visualize_iterations': [0]}, 'algo': 'trpo'}
Generating random rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 52.
Path 3 | total_timesteps 62.
Path 4 | total_timesteps 77.
Path 5 | total_timesteps 91.
Path 6 | total_timesteps 109.
Path 7 | total_timesteps 137.
Path 8 | total_timesteps 152.
Path 9 | total_timesteps 173.
Path 10 | total_timesteps 196.
Path 11 | total_timesteps 219.
Path 12 | total_timesteps 259.
Path 13 | total_timesteps 271.
Path 14 | total_timesteps 293.
Path 15 | total_timesteps 320.
Path 16 | total_timesteps 337.
Path 17 | total_timesteps 355.
Path 18 | total_timesteps 395.
Path 19 | total_timesteps 423.
Path 20 | total_timesteps 439.
Path 21 | total_timesteps 455.
Path 22 | total_timesteps 465.
Path 23 | total_timesteps 500.
Path 24 | total_timesteps 515.
Path 25 | total_timesteps 539.
Path 26 | total_timesteps 558.
Path 27 | total_timesteps 580.
Path 28 | total_timesteps 603.
Path 29 | total_timesteps 623.
Path 30 | total_timesteps 644.
Path 31 | total_timesteps 658.
Path 32 | total_timesteps 670.
Path 33 | total_timesteps 689.
Path 34 | total_timesteps 715.
Path 35 | total_timesteps 734.
Path 36 | total_timesteps 769.
Path 37 | total_timesteps 787.
Path 38 | total_timesteps 800.
Path 39 | total_timesteps 810.
Path 40 | total_timesteps 830.
Path 41 | total_timesteps 845.
Path 42 | total_timesteps 859.
Path 43 | total_timesteps 870.
Path 44 | total_timesteps 894.
Path 45 | total_timesteps 908.
Path 46 | total_timesteps 923.
Path 47 | total_timesteps 940.
Path 48 | total_timesteps 957.
Path 49 | total_timesteps 976.
Path 50 | total_timesteps 990.
Path 51 | total_timesteps 1003.
Path 52 | total_timesteps 1020.
Path 53 | total_timesteps 1051.
Path 54 | total_timesteps 1072.
Path 55 | total_timesteps 1089.
Path 56 | total_timesteps 1110.
Path 57 | total_timesteps 1130.
Path 58 | total_timesteps 1146.
Path 59 | total_timesteps 1162.
Path 60 | total_timesteps 1189.
Path 61 | total_timesteps 1200.
Path 62 | total_timesteps 1214.
Path 63 | total_timesteps 1254.
Path 64 | total_timesteps 1269.
Path 65 | total_timesteps 1291.
Path 66 | total_timesteps 1313.
Path 67 | total_timesteps 1321.
Path 68 | total_timesteps 1337.
Path 69 | total_timesteps 1349.
Path 70 | total_timesteps 1371.
Path 71 | total_timesteps 1383.
Path 72 | total_timesteps 1394.
Path 73 | total_timesteps 1418.
Path 74 | total_timesteps 1427.
Path 75 | total_timesteps 1459.
Path 76 | total_timesteps 1491.
Path 77 | total_timesteps 1505.
Path 78 | total_timesteps 1522.
Path 79 | total_timesteps 1538.
Path 80 | total_timesteps 1546.
Path 81 | total_timesteps 1571.
Path 82 | total_timesteps 1595.
Path 83 | total_timesteps 1610.
Path 84 | total_timesteps 1662.
Path 85 | total_timesteps 1688.
Path 86 | total_timesteps 1709.
Path 87 | total_timesteps 1729.
Path 88 | total_timesteps 1757.
Path 89 | total_timesteps 1777.
Path 90 | total_timesteps 1791.
Path 91 | total_timesteps 1816.
Path 92 | total_timesteps 1837.
Path 93 | total_timesteps 1851.
Path 94 | total_timesteps 1880.
Path 95 | total_timesteps 1895.
Path 96 | total_timesteps 1914.
Path 97 | total_timesteps 1940.
Path 98 | total_timesteps 1962.
Path 99 | total_timesteps 1982.
Path 100 | total_timesteps 1992.
Path 101 | total_timesteps 2015.
Path 102 | total_timesteps 2029.
Path 103 | total_timesteps 2046.
Path 104 | total_timesteps 2061.
Path 105 | total_timesteps 2084.
Path 106 | total_timesteps 2096.
Path 107 | total_timesteps 2113.
Path 108 | total_timesteps 2125.
Path 109 | total_timesteps 2147.
Path 110 | total_timesteps 2174.
Path 111 | total_timesteps 2193.
Path 112 | total_timesteps 2209.
Path 113 | total_timesteps 2233.
Path 114 | total_timesteps 2254.
Path 115 | total_timesteps 2264.
Path 116 | total_timesteps 2282.
Path 117 | total_timesteps 2309.
Path 118 | total_timesteps 2328.
Path 119 | total_timesteps 2343.
Path 120 | total_timesteps 2382.
Path 121 | total_timesteps 2400.
Path 122 | total_timesteps 2413.
Path 123 | total_timesteps 2438.
Path 124 | total_timesteps 2450.
Path 125 | total_timesteps 2474.
Path 126 | total_timesteps 2489.
Path 127 | total_timesteps 2508.
Path 128 | total_timesteps 2539.
Path 129 | total_timesteps 2555.
Path 130 | total_timesteps 2572.
Path 131 | total_timesteps 2587.
Path 132 | total_timesteps 2608.
Path 133 | total_timesteps 2625.
Path 134 | total_timesteps 2637.
Path 135 | total_timesteps 2654.
Path 136 | total_timesteps 2672.
Path 137 | total_timesteps 2705.
Path 138 | total_timesteps 2726.
Path 139 | total_timesteps 2761.
Path 140 | total_timesteps 2776.
Path 141 | total_timesteps 2792.
Path 142 | total_timesteps 2814.
Path 143 | total_timesteps 2824.
Path 144 | total_timesteps 2849.
Path 145 | total_timesteps 2871.
Path 146 | total_timesteps 2893.
Path 147 | total_timesteps 2924.
Path 148 | total_timesteps 2937.
Path 149 | total_timesteps 2952.
Path 150 | total_timesteps 2981.
Path 151 | total_timesteps 3011.
Path 152 | total_timesteps 3030.
Path 153 | total_timesteps 3053.
Path 154 | total_timesteps 3068.
Path 155 | total_timesteps 3079.
Path 156 | total_timesteps 3088.
Path 157 | total_timesteps 3106.
Path 158 | total_timesteps 3124.
Path 159 | total_timesteps 3150.
Path 160 | total_timesteps 3164.
Path 161 | total_timesteps 3197.
Path 162 | total_timesteps 3212.
Path 163 | total_timesteps 3233.
Path 164 | total_timesteps 3248.
Path 165 | total_timesteps 3265.
Path 166 | total_timesteps 3276.
Path 167 | total_timesteps 3298.
Path 168 | total_timesteps 3320.
Path 169 | total_timesteps 3335.
Path 170 | total_timesteps 3345.
Path 171 | total_timesteps 3373.
Path 172 | total_timesteps 3386.
Path 173 | total_timesteps 3403.
Path 174 | total_timesteps 3417.
Path 175 | total_timesteps 3433.
Path 176 | total_timesteps 3452.
Path 177 | total_timesteps 3480.
Path 178 | total_timesteps 3502.
Path 179 | total_timesteps 3519.
Path 180 | total_timesteps 3548.
Path 181 | total_timesteps 3567.
Path 182 | total_timesteps 3586.
Path 183 | total_timesteps 3609.
Path 184 | total_timesteps 3631.
Path 185 | total_timesteps 3643.
Path 186 | total_timesteps 3655.
Path 187 | total_timesteps 3681.
Path 188 | total_timesteps 3704.
Path 189 | total_timesteps 3728.
Path 190 | total_timesteps 3747.
Path 191 | total_timesteps 3763.
Path 192 | total_timesteps 3775.
Path 193 | total_timesteps 3796.
Path 194 | total_timesteps 3816.
Path 195 | total_timesteps 3839.
Path 196 | total_timesteps 3864.
Path 197 | total_timesteps 3883.
Path 198 | total_timesteps 3894.
Path 199 | total_timesteps 3907.
Path 200 | total_timesteps 3934.
Path 201 | total_timesteps 3952.
Path 202 | total_timesteps 3963.
Path 203 | total_timesteps 3977.
Path 204 | total_timesteps 3995.
Path 205 | total_timesteps 4019.
Path 206 | total_timesteps 4041.
Path 207 | total_timesteps 4061.
Path 208 | total_timesteps 4077.
Path 209 | total_timesteps 4095.
Path 210 | total_timesteps 4119.
Path 211 | total_timesteps 4147.
Path 212 | total_timesteps 4160.
Path 213 | total_timesteps 4187.
Path 214 | total_timesteps 4194.
Path 215 | total_timesteps 4218.
Path 216 | total_timesteps 4235.
Path 217 | total_timesteps 4257.
Path 218 | total_timesteps 4273.
Path 219 | total_timesteps 4297.
Path 220 | total_timesteps 4315.
Path 221 | total_timesteps 4330.
Path 222 | total_timesteps 4371.
Path 223 | total_timesteps 4396.
Path 224 | total_timesteps 4426.
Path 225 | total_timesteps 4438.
Path 226 | total_timesteps 4454.
Path 227 | total_timesteps 4466.
Path 228 | total_timesteps 4509.
Path 229 | total_timesteps 4531.
Path 230 | total_timesteps 4563.
Path 231 | total_timesteps 4579.
Path 232 | total_timesteps 4604.
Path 233 | total_timesteps 4621.
Path 234 | total_timesteps 4650.
Path 235 | total_timesteps 4669.
Path 236 | total_timesteps 4683.
Path 237 | total_timesteps 4695.
Path 238 | total_timesteps 4712.
Path 239 | total_timesteps 4755.
Path 240 | total_timesteps 4792.
Path 241 | total_timesteps 4815.
Path 242 | total_timesteps 4831.
Path 243 | total_timesteps 4841.
Path 244 | total_timesteps 4853.
Path 245 | total_timesteps 4873.
Path 246 | total_timesteps 4897.
Path 247 | total_timesteps 4923.
Path 248 | total_timesteps 4942.
Path 249 | total_timesteps 4959.
Path 250 | total_timesteps 4994.
Path 251 | total_timesteps 5016.
Path 252 | total_timesteps 5029.
Path 253 | total_timesteps 5045.
Path 254 | total_timesteps 5055.
Path 255 | total_timesteps 5070.
Path 256 | total_timesteps 5082.
Path 257 | total_timesteps 5105.
Path 258 | total_timesteps 5115.
Path 259 | total_timesteps 5137.
Path 260 | total_timesteps 5155.
Path 261 | total_timesteps 5167.
Path 262 | total_timesteps 5194.
Path 263 | total_timesteps 5220.
Path 264 | total_timesteps 5233.
Path 265 | total_timesteps 5260.
Path 266 | total_timesteps 5280.
Path 267 | total_timesteps 5303.
Path 268 | total_timesteps 5315.
Path 269 | total_timesteps 5338.
Path 270 | total_timesteps 5369.
Path 271 | total_timesteps 5391.
Path 272 | total_timesteps 5424.
Path 273 | total_timesteps 5439.
Path 274 | total_timesteps 5455.
Path 275 | total_timesteps 5477.
Path 276 | total_timesteps 5493.
Path 277 | total_timesteps 5511.
Path 278 | total_timesteps 5531.
Path 279 | total_timesteps 5561.
Path 280 | total_timesteps 5585.
Path 281 | total_timesteps 5604.
Path 282 | total_timesteps 5643.
Path 283 | total_timesteps 5658.
Path 284 | total_timesteps 5673.
Path 285 | total_timesteps 5697.
Path 286 | total_timesteps 5711.
Path 287 | total_timesteps 5729.
Path 288 | total_timesteps 5749.
Path 289 | total_timesteps 5781.
Path 290 | total_timesteps 5798.
Path 291 | total_timesteps 5821.
Path 292 | total_timesteps 5848.
Path 293 | total_timesteps 5877.
Path 294 | total_timesteps 5892.
Path 295 | total_timesteps 5921.
Path 296 | total_timesteps 5939.
Path 297 | total_timesteps 5958.
Path 298 | total_timesteps 5975.
Path 299 | total_timesteps 5983.
Path 300 | total_timesteps 5999.
Done generating random rollouts.
Creating normalization for training data.
Done creating normalization for training data.
Train dynamics model with intrinsic reward only? False
Pre-training enabled. Using only intrinsic reward.
Pre-training dynamics model for 0 iterations...
Done pre-training dynamics model.
Using external reward only.
itr #0 | 
Fitting dynamics.
Validation loss = 0.3866257071495056
Validation loss = 0.13841427862644196
Validation loss = 0.1030324175953865
Validation loss = 0.08786290884017944
Validation loss = 0.0811980813741684
Validation loss = 0.0774623304605484
Validation loss = 0.06967055797576904
Validation loss = 0.0668637752532959
Validation loss = 0.06445405632257462
Validation loss = 0.06264586001634598
Validation loss = 0.06676796078681946
Validation loss = 0.05637902766466141
Validation loss = 0.05432085692882538
Validation loss = 0.05869355797767639
Validation loss = 0.058076608926057816
Validation loss = 0.053954318165779114
Validation loss = 0.0540584996342659
Validation loss = 0.06827184557914734
Validation loss = 0.0524069145321846
Validation loss = 0.05272854119539261
Validation loss = 0.05275779962539673
Validation loss = 0.0676792562007904
Validation loss = 0.05205947533249855
Validation loss = 0.05370191112160683
Validation loss = 0.04950703680515289
Validation loss = 0.0517628975212574
Validation loss = 0.053098514676094055
Validation loss = 0.04899889975786209
Validation loss = 0.05879446864128113
Validation loss = 0.04952167347073555
Validation loss = 0.052925486117601395
Validation loss = 0.04911040514707565
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 20.
Path 2 | total_timesteps 44.
Path 3 | total_timesteps 65.
Path 4 | total_timesteps 77.
Path 5 | total_timesteps 113.
Path 6 | total_timesteps 127.
Path 7 | total_timesteps 143.
Path 8 | total_timesteps 163.
Path 9 | total_timesteps 175.
Path 10 | total_timesteps 189.
Path 11 | total_timesteps 210.
Path 12 | total_timesteps 220.
Path 13 | total_timesteps 248.
Path 14 | total_timesteps 265.
Path 15 | total_timesteps 275.
Path 16 | total_timesteps 297.
Path 17 | total_timesteps 314.
Path 18 | total_timesteps 328.
Path 19 | total_timesteps 339.
Path 20 | total_timesteps 366.
Path 21 | total_timesteps 380.
Path 22 | total_timesteps 391.
Path 23 | total_timesteps 399.
Path 24 | total_timesteps 429.
Path 25 | total_timesteps 447.
Path 26 | total_timesteps 457.
Path 27 | total_timesteps 476.
Path 28 | total_timesteps 499.
Path 29 | total_timesteps 523.
Path 30 | total_timesteps 537.
Path 31 | total_timesteps 551.
Path 32 | total_timesteps 571.
Path 33 | total_timesteps 590.
Path 34 | total_timesteps 603.
Path 35 | total_timesteps 618.
Path 36 | total_timesteps 634.
Path 37 | total_timesteps 658.
Path 38 | total_timesteps 676.
Path 39 | total_timesteps 697.
Path 40 | total_timesteps 714.
Path 41 | total_timesteps 738.
Path 42 | total_timesteps 766.
Path 43 | total_timesteps 786.
Path 44 | total_timesteps 794.
Path 45 | total_timesteps 809.
Path 46 | total_timesteps 823.
Path 47 | total_timesteps 838.
Path 48 | total_timesteps 850.
Path 49 | total_timesteps 870.
Path 50 | total_timesteps 888.
Path 51 | total_timesteps 906.
Path 52 | total_timesteps 919.
Path 53 | total_timesteps 938.
Path 54 | total_timesteps 962.
Path 55 | total_timesteps 985.
Path 56 | total_timesteps 1004.
Path 57 | total_timesteps 1012.
Path 58 | total_timesteps 1026.
Path 59 | total_timesteps 1042.
Path 60 | total_timesteps 1062.
Path 61 | total_timesteps 1077.
Path 62 | total_timesteps 1100.
Path 63 | total_timesteps 1116.
Path 64 | total_timesteps 1127.
Path 65 | total_timesteps 1139.
Path 66 | total_timesteps 1158.
Path 67 | total_timesteps 1167.
Path 68 | total_timesteps 1192.
Path 69 | total_timesteps 1205.
Path 70 | total_timesteps 1221.
Path 71 | total_timesteps 1229.
Path 72 | total_timesteps 1252.
Path 73 | total_timesteps 1282.
Path 74 | total_timesteps 1308.
Path 75 | total_timesteps 1331.
Path 76 | total_timesteps 1339.
Path 77 | total_timesteps 1361.
Path 78 | total_timesteps 1382.
Path 79 | total_timesteps 1397.
Path 80 | total_timesteps 1424.
Path 81 | total_timesteps 1433.
Path 82 | total_timesteps 1449.
Path 83 | total_timesteps 1472.
Path 84 | total_timesteps 1482.
Path 85 | total_timesteps 1495.
Path 86 | total_timesteps 1522.
Path 87 | total_timesteps 1557.
Path 88 | total_timesteps 1578.
Path 89 | total_timesteps 1590.
Path 90 | total_timesteps 1601.
Path 91 | total_timesteps 1620.
Path 92 | total_timesteps 1653.
Path 93 | total_timesteps 1672.
Path 94 | total_timesteps 1687.
Path 95 | total_timesteps 1697.
Path 96 | total_timesteps 1727.
Path 97 | total_timesteps 1736.
Path 98 | total_timesteps 1755.
Path 99 | total_timesteps 1773.
Path 100 | total_timesteps 1790.
Path 101 | total_timesteps 1805.
Path 102 | total_timesteps 1823.
Path 103 | total_timesteps 1841.
Path 104 | total_timesteps 1859.
Path 105 | total_timesteps 1870.
Path 106 | total_timesteps 1895.
Path 107 | total_timesteps 1944.
Path 108 | total_timesteps 1959.
Path 109 | total_timesteps 1978.
Path 110 | total_timesteps 1998.
Path 111 | total_timesteps 2019.
Path 112 | total_timesteps 2033.
Path 113 | total_timesteps 2053.
Path 114 | total_timesteps 2080.
Path 115 | total_timesteps 2098.
Path 116 | total_timesteps 2137.
Path 117 | total_timesteps 2158.
Path 118 | total_timesteps 2177.
Path 119 | total_timesteps 2204.
Path 120 | total_timesteps 2229.
Path 121 | total_timesteps 2244.
Path 122 | total_timesteps 2259.
Path 123 | total_timesteps 2291.
Path 124 | total_timesteps 2309.
Path 125 | total_timesteps 2319.
Path 126 | total_timesteps 2351.
Path 127 | total_timesteps 2374.
Path 128 | total_timesteps 2386.
Path 129 | total_timesteps 2407.
Path 130 | total_timesteps 2422.
Path 131 | total_timesteps 2440.
Path 132 | total_timesteps 2452.
Path 133 | total_timesteps 2461.
Path 134 | total_timesteps 2477.
Path 135 | total_timesteps 2513.
Path 136 | total_timesteps 2545.
Path 137 | total_timesteps 2562.
Path 138 | total_timesteps 2583.
Path 139 | total_timesteps 2604.
Path 140 | total_timesteps 2641.
Path 141 | total_timesteps 2654.
Path 142 | total_timesteps 2666.
Path 143 | total_timesteps 2699.
Path 144 | total_timesteps 2715.
Path 145 | total_timesteps 2726.
Path 146 | total_timesteps 2755.
Path 147 | total_timesteps 2779.
Path 148 | total_timesteps 2792.
Path 149 | total_timesteps 2804.
Path 150 | total_timesteps 2854.
Path 151 | total_timesteps 2878.
Path 152 | total_timesteps 2887.
Path 153 | total_timesteps 2904.
Path 154 | total_timesteps 2922.
Path 155 | total_timesteps 2936.
Path 156 | total_timesteps 2948.
Path 157 | total_timesteps 2959.
Path 158 | total_timesteps 2976.
Path 159 | total_timesteps 2986.
Path 160 | total_timesteps 2994.
Path 161 | total_timesteps 3011.
Path 162 | total_timesteps 3028.
Path 163 | total_timesteps 3056.
Path 164 | total_timesteps 3081.
Path 165 | total_timesteps 3107.
Path 166 | total_timesteps 3127.
Path 167 | total_timesteps 3147.
Path 168 | total_timesteps 3175.
Path 169 | total_timesteps 3216.
Path 170 | total_timesteps 3233.
Path 171 | total_timesteps 3248.
Path 172 | total_timesteps 3283.
Path 173 | total_timesteps 3305.
Path 174 | total_timesteps 3335.
Path 175 | total_timesteps 3364.
Path 176 | total_timesteps 3391.
Path 177 | total_timesteps 3403.
Path 178 | total_timesteps 3423.
Path 179 | total_timesteps 3441.
Path 180 | total_timesteps 3453.
Path 181 | total_timesteps 3470.
Path 182 | total_timesteps 3483.
Path 183 | total_timesteps 3499.
Path 184 | total_timesteps 3526.
Path 185 | total_timesteps 3540.
Path 186 | total_timesteps 3552.
Path 187 | total_timesteps 3580.
Path 188 | total_timesteps 3594.
Path 189 | total_timesteps 3602.
Path 190 | total_timesteps 3621.
Path 191 | total_timesteps 3644.
Path 192 | total_timesteps 3663.
Path 193 | total_timesteps 3685.
Path 194 | total_timesteps 3723.
Path 195 | total_timesteps 3742.
Path 196 | total_timesteps 3756.
Path 197 | total_timesteps 3780.
Path 198 | total_timesteps 3795.
Path 199 | total_timesteps 3819.
Path 200 | total_timesteps 3829.
Path 201 | total_timesteps 3842.
Path 202 | total_timesteps 3860.
Path 203 | total_timesteps 3879.
Path 204 | total_timesteps 3906.
Path 205 | total_timesteps 3939.
Path 206 | total_timesteps 3969.
Path 207 | total_timesteps 4033.
Path 208 | total_timesteps 4051.
Path 209 | total_timesteps 4091.
Path 210 | total_timesteps 4107.
Path 211 | total_timesteps 4118.
Path 212 | total_timesteps 4136.
Path 213 | total_timesteps 4151.
Path 214 | total_timesteps 4161.
Path 215 | total_timesteps 4181.
Path 216 | total_timesteps 4215.
Path 217 | total_timesteps 4233.
Path 218 | total_timesteps 4249.
Path 219 | total_timesteps 4266.
Path 220 | total_timesteps 4281.
Path 221 | total_timesteps 4292.
Path 222 | total_timesteps 4302.
Path 223 | total_timesteps 4324.
Path 224 | total_timesteps 4335.
Path 225 | total_timesteps 4349.
Path 226 | total_timesteps 4359.
Path 227 | total_timesteps 4380.
Path 228 | total_timesteps 4401.
Path 229 | total_timesteps 4416.
Path 230 | total_timesteps 4433.
Path 231 | total_timesteps 4449.
Path 232 | total_timesteps 4468.
Path 233 | total_timesteps 4494.
Path 234 | total_timesteps 4521.
Path 235 | total_timesteps 4546.
Path 236 | total_timesteps 4601.
Path 237 | total_timesteps 4634.
Path 238 | total_timesteps 4647.
Path 239 | total_timesteps 4660.
Path 240 | total_timesteps 4669.
Path 241 | total_timesteps 4695.
Path 242 | total_timesteps 4727.
Path 243 | total_timesteps 4747.
Path 244 | total_timesteps 4760.
Path 245 | total_timesteps 4784.
Path 246 | total_timesteps 4803.
Path 247 | total_timesteps 4814.
Path 248 | total_timesteps 4834.
Path 249 | total_timesteps 4847.
Path 250 | total_timesteps 4857.
Path 251 | total_timesteps 4883.
Path 252 | total_timesteps 4907.
Path 253 | total_timesteps 4924.
Path 254 | total_timesteps 4938.
Path 255 | total_timesteps 4952.
Path 256 | total_timesteps 4968.
Path 257 | total_timesteps 4979.
Path 258 | total_timesteps 5000.
Path 259 | total_timesteps 5022.
Path 260 | total_timesteps 5043.
Path 261 | total_timesteps 5056.
Path 262 | total_timesteps 5071.
Path 263 | total_timesteps 5104.
Path 264 | total_timesteps 5112.
Path 265 | total_timesteps 5123.
Path 266 | total_timesteps 5131.
Path 267 | total_timesteps 5151.
Path 268 | total_timesteps 5162.
Path 269 | total_timesteps 5178.
Path 270 | total_timesteps 5190.
Path 271 | total_timesteps 5204.
Path 272 | total_timesteps 5225.
Path 273 | total_timesteps 5254.
Path 274 | total_timesteps 5286.
Path 275 | total_timesteps 5310.
Path 276 | total_timesteps 5320.
Path 277 | total_timesteps 5328.
Path 278 | total_timesteps 5362.
Path 279 | total_timesteps 5387.
Path 280 | total_timesteps 5397.
Path 281 | total_timesteps 5409.
Path 282 | total_timesteps 5434.
Path 283 | total_timesteps 5447.
Path 284 | total_timesteps 5480.
Path 285 | total_timesteps 5495.
Path 286 | total_timesteps 5512.
Path 287 | total_timesteps 5525.
Path 288 | total_timesteps 5571.
Path 289 | total_timesteps 5588.
Path 290 | total_timesteps 5607.
Path 291 | total_timesteps 5629.
Path 292 | total_timesteps 5658.
Path 293 | total_timesteps 5671.
Path 294 | total_timesteps 5686.
Path 295 | total_timesteps 5704.
Path 296 | total_timesteps 5722.
Path 297 | total_timesteps 5735.
Path 298 | total_timesteps 5762.
Path 299 | total_timesteps 5778.
Path 300 | total_timesteps 5831.
Path 301 | total_timesteps 5851.
Path 302 | total_timesteps 5862.
Path 303 | total_timesteps 5876.
Path 304 | total_timesteps 5888.
Path 305 | total_timesteps 5903.
Path 306 | total_timesteps 5924.
Path 307 | total_timesteps 5943.
Path 308 | total_timesteps 5954.
Path 309 | total_timesteps 5973.
Path 310 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.73    |
| Iteration     | 0        |
| MaximumReturn | 16.3     |
| MinimumReturn | -24.7    |
| TotalSamples  | 8022     |
----------------------------
itr #1 | 
Fitting dynamics.
Validation loss = 0.11829306185245514
Validation loss = 0.06408888846635818
Validation loss = 0.056098572909832
Validation loss = 0.05360030755400658
Validation loss = 0.055833008140325546
Validation loss = 0.044517938047647476
Validation loss = 0.050635676831007004
Validation loss = 0.04193989932537079
Validation loss = 0.04138902202248573
Validation loss = 0.04297619313001633
Validation loss = 0.042110882699489594
Validation loss = 0.04101695865392685
Validation loss = 0.041598208248615265
Validation loss = 0.04141581431031227
Validation loss = 0.04249786585569382
Validation loss = 0.03993857279419899
Validation loss = 0.03874795138835907
Validation loss = 0.04220438003540039
Validation loss = 0.0390310063958168
Validation loss = 0.045318685472011566
Validation loss = 0.03790035843849182
Validation loss = 0.04160526394844055
Validation loss = 0.04273400455713272
Validation loss = 0.037626560777425766
Validation loss = 0.037828490138053894
Validation loss = 0.04047417268157005
Validation loss = 0.04013039916753769
Validation loss = 0.038258448243141174
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 29.
Path 4 | total_timesteps 37.
Path 5 | total_timesteps 56.
Path 6 | total_timesteps 77.
Path 7 | total_timesteps 89.
Path 8 | total_timesteps 103.
Path 9 | total_timesteps 115.
Path 10 | total_timesteps 123.
Path 11 | total_timesteps 135.
Path 12 | total_timesteps 154.
Path 13 | total_timesteps 161.
Path 14 | total_timesteps 170.
Path 15 | total_timesteps 182.
Path 16 | total_timesteps 193.
Path 17 | total_timesteps 203.
Path 18 | total_timesteps 213.
Path 19 | total_timesteps 223.
Path 20 | total_timesteps 234.
Path 21 | total_timesteps 242.
Path 22 | total_timesteps 267.
Path 23 | total_timesteps 287.
Path 24 | total_timesteps 298.
Path 25 | total_timesteps 313.
Path 26 | total_timesteps 324.
Path 27 | total_timesteps 332.
Path 28 | total_timesteps 367.
Path 29 | total_timesteps 378.
Path 30 | total_timesteps 386.
Path 31 | total_timesteps 398.
Path 32 | total_timesteps 407.
Path 33 | total_timesteps 427.
Path 34 | total_timesteps 442.
Path 35 | total_timesteps 462.
Path 36 | total_timesteps 470.
Path 37 | total_timesteps 484.
Path 38 | total_timesteps 494.
Path 39 | total_timesteps 501.
Path 40 | total_timesteps 512.
Path 41 | total_timesteps 522.
Path 42 | total_timesteps 532.
Path 43 | total_timesteps 541.
Path 44 | total_timesteps 548.
Path 45 | total_timesteps 563.
Path 46 | total_timesteps 584.
Path 47 | total_timesteps 591.
Path 48 | total_timesteps 601.
Path 49 | total_timesteps 612.
Path 50 | total_timesteps 626.
Path 51 | total_timesteps 635.
Path 52 | total_timesteps 643.
Path 53 | total_timesteps 659.
Path 54 | total_timesteps 670.
Path 55 | total_timesteps 680.
Path 56 | total_timesteps 689.
Path 57 | total_timesteps 708.
Path 58 | total_timesteps 724.
Path 59 | total_timesteps 736.
Path 60 | total_timesteps 756.
Path 61 | total_timesteps 767.
Path 62 | total_timesteps 775.
Path 63 | total_timesteps 797.
Path 64 | total_timesteps 813.
Path 65 | total_timesteps 827.
Path 66 | total_timesteps 836.
Path 67 | total_timesteps 846.
Path 68 | total_timesteps 857.
Path 69 | total_timesteps 866.
Path 70 | total_timesteps 875.
Path 71 | total_timesteps 892.
Path 72 | total_timesteps 900.
Path 73 | total_timesteps 909.
Path 74 | total_timesteps 922.
Path 75 | total_timesteps 931.
Path 76 | total_timesteps 943.
Path 77 | total_timesteps 965.
Path 78 | total_timesteps 972.
Path 79 | total_timesteps 982.
Path 80 | total_timesteps 990.
Path 81 | total_timesteps 1003.
Path 82 | total_timesteps 1012.
Path 83 | total_timesteps 1024.
Path 84 | total_timesteps 1037.
Path 85 | total_timesteps 1045.
Path 86 | total_timesteps 1056.
Path 87 | total_timesteps 1073.
Path 88 | total_timesteps 1082.
Path 89 | total_timesteps 1093.
Path 90 | total_timesteps 1108.
Path 91 | total_timesteps 1114.
Path 92 | total_timesteps 1122.
Path 93 | total_timesteps 1130.
Path 94 | total_timesteps 1143.
Path 95 | total_timesteps 1158.
Path 96 | total_timesteps 1168.
Path 97 | total_timesteps 1174.
Path 98 | total_timesteps 1193.
Path 99 | total_timesteps 1201.
Path 100 | total_timesteps 1213.
Path 101 | total_timesteps 1228.
Path 102 | total_timesteps 1239.
Path 103 | total_timesteps 1248.
Path 104 | total_timesteps 1266.
Path 105 | total_timesteps 1293.
Path 106 | total_timesteps 1317.
Path 107 | total_timesteps 1332.
Path 108 | total_timesteps 1343.
Path 109 | total_timesteps 1352.
Path 110 | total_timesteps 1364.
Path 111 | total_timesteps 1373.
Path 112 | total_timesteps 1383.
Path 113 | total_timesteps 1391.
Path 114 | total_timesteps 1404.
Path 115 | total_timesteps 1420.
Path 116 | total_timesteps 1433.
Path 117 | total_timesteps 1444.
Path 118 | total_timesteps 1462.
Path 119 | total_timesteps 1472.
Path 120 | total_timesteps 1486.
Path 121 | total_timesteps 1497.
Path 122 | total_timesteps 1509.
Path 123 | total_timesteps 1523.
Path 124 | total_timesteps 1543.
Path 125 | total_timesteps 1566.
Path 126 | total_timesteps 1578.
Path 127 | total_timesteps 1586.
Path 128 | total_timesteps 1598.
Path 129 | total_timesteps 1610.
Path 130 | total_timesteps 1624.
Path 131 | total_timesteps 1632.
Path 132 | total_timesteps 1644.
Path 133 | total_timesteps 1658.
Path 134 | total_timesteps 1667.
Path 135 | total_timesteps 1678.
Path 136 | total_timesteps 1691.
Path 137 | total_timesteps 1702.
Path 138 | total_timesteps 1713.
Path 139 | total_timesteps 1725.
Path 140 | total_timesteps 1733.
Path 141 | total_timesteps 1740.
Path 142 | total_timesteps 1753.
Path 143 | total_timesteps 1762.
Path 144 | total_timesteps 1773.
Path 145 | total_timesteps 1784.
Path 146 | total_timesteps 1794.
Path 147 | total_timesteps 1809.
Path 148 | total_timesteps 1823.
Path 149 | total_timesteps 1838.
Path 150 | total_timesteps 1849.
Path 151 | total_timesteps 1863.
Path 152 | total_timesteps 1873.
Path 153 | total_timesteps 1885.
Path 154 | total_timesteps 1892.
Path 155 | total_timesteps 1906.
Path 156 | total_timesteps 1917.
Path 157 | total_timesteps 1927.
Path 158 | total_timesteps 1935.
Path 159 | total_timesteps 1944.
Path 160 | total_timesteps 1954.
Path 161 | total_timesteps 1962.
Path 162 | total_timesteps 1975.
Path 163 | total_timesteps 1983.
Path 164 | total_timesteps 1992.
Path 165 | total_timesteps 2025.
Path 166 | total_timesteps 2042.
Path 167 | total_timesteps 2054.
Path 168 | total_timesteps 2062.
Path 169 | total_timesteps 2076.
Path 170 | total_timesteps 2093.
Path 171 | total_timesteps 2106.
Path 172 | total_timesteps 2118.
Path 173 | total_timesteps 2125.
Path 174 | total_timesteps 2135.
Path 175 | total_timesteps 2146.
Path 176 | total_timesteps 2157.
Path 177 | total_timesteps 2169.
Path 178 | total_timesteps 2195.
Path 179 | total_timesteps 2212.
Path 180 | total_timesteps 2224.
Path 181 | total_timesteps 2233.
Path 182 | total_timesteps 2242.
Path 183 | total_timesteps 2262.
Path 184 | total_timesteps 2276.
Path 185 | total_timesteps 2288.
Path 186 | total_timesteps 2318.
Path 187 | total_timesteps 2327.
Path 188 | total_timesteps 2335.
Path 189 | total_timesteps 2346.
Path 190 | total_timesteps 2366.
Path 191 | total_timesteps 2382.
Path 192 | total_timesteps 2390.
Path 193 | total_timesteps 2409.
Path 194 | total_timesteps 2420.
Path 195 | total_timesteps 2447.
Path 196 | total_timesteps 2461.
Path 197 | total_timesteps 2472.
Path 198 | total_timesteps 2483.
Path 199 | total_timesteps 2509.
Path 200 | total_timesteps 2528.
Path 201 | total_timesteps 2541.
Path 202 | total_timesteps 2551.
Path 203 | total_timesteps 2565.
Path 204 | total_timesteps 2588.
Path 205 | total_timesteps 2597.
Path 206 | total_timesteps 2621.
Path 207 | total_timesteps 2632.
Path 208 | total_timesteps 2643.
Path 209 | total_timesteps 2657.
Path 210 | total_timesteps 2667.
Path 211 | total_timesteps 2686.
Path 212 | total_timesteps 2696.
Path 213 | total_timesteps 2709.
Path 214 | total_timesteps 2720.
Path 215 | total_timesteps 2734.
Path 216 | total_timesteps 2751.
Path 217 | total_timesteps 2762.
Path 218 | total_timesteps 2775.
Path 219 | total_timesteps 2784.
Path 220 | total_timesteps 2791.
Path 221 | total_timesteps 2807.
Path 222 | total_timesteps 2819.
Path 223 | total_timesteps 2836.
Path 224 | total_timesteps 2858.
Path 225 | total_timesteps 2876.
Path 226 | total_timesteps 2892.
Path 227 | total_timesteps 2903.
Path 228 | total_timesteps 2912.
Path 229 | total_timesteps 2923.
Path 230 | total_timesteps 2936.
Path 231 | total_timesteps 2958.
Path 232 | total_timesteps 2970.
Path 233 | total_timesteps 2982.
Path 234 | total_timesteps 3007.
Path 235 | total_timesteps 3015.
Path 236 | total_timesteps 3028.
Path 237 | total_timesteps 3039.
Path 238 | total_timesteps 3056.
Path 239 | total_timesteps 3067.
Path 240 | total_timesteps 3075.
Path 241 | total_timesteps 3083.
Path 242 | total_timesteps 3093.
Path 243 | total_timesteps 3101.
Path 244 | total_timesteps 3117.
Path 245 | total_timesteps 3131.
Path 246 | total_timesteps 3145.
Path 247 | total_timesteps 3157.
Path 248 | total_timesteps 3167.
Path 249 | total_timesteps 3174.
Path 250 | total_timesteps 3184.
Path 251 | total_timesteps 3198.
Path 252 | total_timesteps 3206.
Path 253 | total_timesteps 3223.
Path 254 | total_timesteps 3235.
Path 255 | total_timesteps 3245.
Path 256 | total_timesteps 3260.
Path 257 | total_timesteps 3276.
Path 258 | total_timesteps 3305.
Path 259 | total_timesteps 3316.
Path 260 | total_timesteps 3336.
Path 261 | total_timesteps 3346.
Path 262 | total_timesteps 3356.
Path 263 | total_timesteps 3373.
Path 264 | total_timesteps 3388.
Path 265 | total_timesteps 3398.
Path 266 | total_timesteps 3406.
Path 267 | total_timesteps 3417.
Path 268 | total_timesteps 3435.
Path 269 | total_timesteps 3445.
Path 270 | total_timesteps 3457.
Path 271 | total_timesteps 3468.
Path 272 | total_timesteps 3477.
Path 273 | total_timesteps 3500.
Path 274 | total_timesteps 3521.
Path 275 | total_timesteps 3531.
Path 276 | total_timesteps 3543.
Path 277 | total_timesteps 3550.
Path 278 | total_timesteps 3560.
Path 279 | total_timesteps 3572.
Path 280 | total_timesteps 3580.
Path 281 | total_timesteps 3590.
Path 282 | total_timesteps 3599.
Path 283 | total_timesteps 3608.
Path 284 | total_timesteps 3616.
Path 285 | total_timesteps 3631.
Path 286 | total_timesteps 3643.
Path 287 | total_timesteps 3659.
Path 288 | total_timesteps 3672.
Path 289 | total_timesteps 3684.
Path 290 | total_timesteps 3692.
Path 291 | total_timesteps 3706.
Path 292 | total_timesteps 3721.
Path 293 | total_timesteps 3731.
Path 294 | total_timesteps 3748.
Path 295 | total_timesteps 3757.
Path 296 | total_timesteps 3769.
Path 297 | total_timesteps 3779.
Path 298 | total_timesteps 3793.
Path 299 | total_timesteps 3800.
Path 300 | total_timesteps 3808.
Path 301 | total_timesteps 3818.
Path 302 | total_timesteps 3828.
Path 303 | total_timesteps 3838.
Path 304 | total_timesteps 3855.
Path 305 | total_timesteps 3864.
Path 306 | total_timesteps 3883.
Path 307 | total_timesteps 3900.
Path 308 | total_timesteps 3916.
Path 309 | total_timesteps 3928.
Path 310 | total_timesteps 3948.
Path 311 | total_timesteps 3960.
Path 312 | total_timesteps 3968.
Path 313 | total_timesteps 3976.
Path 314 | total_timesteps 3989.
Path 315 | total_timesteps 4014.
Path 316 | total_timesteps 4029.
Path 317 | total_timesteps 4041.
Path 318 | total_timesteps 4052.
Path 319 | total_timesteps 4062.
Path 320 | total_timesteps 4082.
Path 321 | total_timesteps 4090.
Path 322 | total_timesteps 4101.
Path 323 | total_timesteps 4113.
Path 324 | total_timesteps 4122.
Path 325 | total_timesteps 4130.
Path 326 | total_timesteps 4141.
Path 327 | total_timesteps 4151.
Path 328 | total_timesteps 4159.
Path 329 | total_timesteps 4166.
Path 330 | total_timesteps 4180.
Path 331 | total_timesteps 4199.
Path 332 | total_timesteps 4212.
Path 333 | total_timesteps 4223.
Path 334 | total_timesteps 4239.
Path 335 | total_timesteps 4251.
Path 336 | total_timesteps 4261.
Path 337 | total_timesteps 4271.
Path 338 | total_timesteps 4287.
Path 339 | total_timesteps 4307.
Path 340 | total_timesteps 4317.
Path 341 | total_timesteps 4329.
Path 342 | total_timesteps 4344.
Path 343 | total_timesteps 4358.
Path 344 | total_timesteps 4368.
Path 345 | total_timesteps 4381.
Path 346 | total_timesteps 4411.
Path 347 | total_timesteps 4423.
Path 348 | total_timesteps 4440.
Path 349 | total_timesteps 4452.
Path 350 | total_timesteps 4473.
Path 351 | total_timesteps 4481.
Path 352 | total_timesteps 4490.
Path 353 | total_timesteps 4506.
Path 354 | total_timesteps 4517.
Path 355 | total_timesteps 4530.
Path 356 | total_timesteps 4545.
Path 357 | total_timesteps 4554.
Path 358 | total_timesteps 4580.
Path 359 | total_timesteps 4602.
Path 360 | total_timesteps 4615.
Path 361 | total_timesteps 4640.
Path 362 | total_timesteps 4654.
Path 363 | total_timesteps 4668.
Path 364 | total_timesteps 4682.
Path 365 | total_timesteps 4693.
Path 366 | total_timesteps 4701.
Path 367 | total_timesteps 4714.
Path 368 | total_timesteps 4726.
Path 369 | total_timesteps 4739.
Path 370 | total_timesteps 4755.
Path 371 | total_timesteps 4770.
Path 372 | total_timesteps 4779.
Path 373 | total_timesteps 4787.
Path 374 | total_timesteps 4803.
Path 375 | total_timesteps 4825.
Path 376 | total_timesteps 4838.
Path 377 | total_timesteps 4850.
Path 378 | total_timesteps 4860.
Path 379 | total_timesteps 4874.
Path 380 | total_timesteps 4885.
Path 381 | total_timesteps 4896.
Path 382 | total_timesteps 4912.
Path 383 | total_timesteps 4924.
Path 384 | total_timesteps 4933.
Path 385 | total_timesteps 4940.
Path 386 | total_timesteps 4952.
Path 387 | total_timesteps 4969.
Path 388 | total_timesteps 4981.
Path 389 | total_timesteps 4991.
Path 390 | total_timesteps 4998.
Path 391 | total_timesteps 5006.
Path 392 | total_timesteps 5016.
Path 393 | total_timesteps 5025.
Path 394 | total_timesteps 5037.
Path 395 | total_timesteps 5049.
Path 396 | total_timesteps 5060.
Path 397 | total_timesteps 5070.
Path 398 | total_timesteps 5079.
Path 399 | total_timesteps 5089.
Path 400 | total_timesteps 5100.
Path 401 | total_timesteps 5114.
Path 402 | total_timesteps 5133.
Path 403 | total_timesteps 5148.
Path 404 | total_timesteps 5159.
Path 405 | total_timesteps 5172.
Path 406 | total_timesteps 5183.
Path 407 | total_timesteps 5190.
Path 408 | total_timesteps 5203.
Path 409 | total_timesteps 5218.
Path 410 | total_timesteps 5225.
Path 411 | total_timesteps 5234.
Path 412 | total_timesteps 5243.
Path 413 | total_timesteps 5256.
Path 414 | total_timesteps 5267.
Path 415 | total_timesteps 5279.
Path 416 | total_timesteps 5292.
Path 417 | total_timesteps 5302.
Path 418 | total_timesteps 5320.
Path 419 | total_timesteps 5328.
Path 420 | total_timesteps 5336.
Path 421 | total_timesteps 5353.
Path 422 | total_timesteps 5366.
Path 423 | total_timesteps 5379.
Path 424 | total_timesteps 5390.
Path 425 | total_timesteps 5401.
Path 426 | total_timesteps 5413.
Path 427 | total_timesteps 5423.
Path 428 | total_timesteps 5431.
Path 429 | total_timesteps 5444.
Path 430 | total_timesteps 5459.
Path 431 | total_timesteps 5468.
Path 432 | total_timesteps 5480.
Path 433 | total_timesteps 5488.
Path 434 | total_timesteps 5496.
Path 435 | total_timesteps 5509.
Path 436 | total_timesteps 5522.
Path 437 | total_timesteps 5543.
Path 438 | total_timesteps 5553.
Path 439 | total_timesteps 5560.
Path 440 | total_timesteps 5569.
Path 441 | total_timesteps 5579.
Path 442 | total_timesteps 5603.
Path 443 | total_timesteps 5611.
Path 444 | total_timesteps 5620.
Path 445 | total_timesteps 5630.
Path 446 | total_timesteps 5644.
Path 447 | total_timesteps 5655.
Path 448 | total_timesteps 5663.
Path 449 | total_timesteps 5673.
Path 450 | total_timesteps 5684.
Path 451 | total_timesteps 5699.
Path 452 | total_timesteps 5708.
Path 453 | total_timesteps 5718.
Path 454 | total_timesteps 5735.
Path 455 | total_timesteps 5751.
Path 456 | total_timesteps 5759.
Path 457 | total_timesteps 5773.
Path 458 | total_timesteps 5781.
Path 459 | total_timesteps 5794.
Path 460 | total_timesteps 5802.
Path 461 | total_timesteps 5811.
Path 462 | total_timesteps 5821.
Path 463 | total_timesteps 5840.
Path 464 | total_timesteps 5849.
Path 465 | total_timesteps 5857.
Path 466 | total_timesteps 5873.
Path 467 | total_timesteps 5882.
Path 468 | total_timesteps 5893.
Path 469 | total_timesteps 5901.
Path 470 | total_timesteps 5913.
Path 471 | total_timesteps 5924.
Path 472 | total_timesteps 5945.
Path 473 | total_timesteps 5956.
Path 474 | total_timesteps 5970.
Path 475 | total_timesteps 5981.
Path 476 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.96    |
| Iteration     | 1        |
| MaximumReturn | 0.252    |
| MinimumReturn | -24.5    |
| TotalSamples  | 12023    |
----------------------------
itr #2 | 
Fitting dynamics.
Validation loss = 0.047712262719869614
Validation loss = 0.031009582802653313
Validation loss = 0.030725890770554543
Validation loss = 0.030439650639891624
Validation loss = 0.02771853096783161
Validation loss = 0.029278650879859924
Validation loss = 0.029176607728004456
Validation loss = 0.028249194845557213
Validation loss = 0.03300775587558746
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 32.
Path 4 | total_timesteps 41.
Path 5 | total_timesteps 50.
Path 6 | total_timesteps 63.
Path 7 | total_timesteps 82.
Path 8 | total_timesteps 94.
Path 9 | total_timesteps 101.
Path 10 | total_timesteps 108.
Path 11 | total_timesteps 128.
Path 12 | total_timesteps 138.
Path 13 | total_timesteps 151.
Path 14 | total_timesteps 162.
Path 15 | total_timesteps 174.
Path 16 | total_timesteps 182.
Path 17 | total_timesteps 201.
Path 18 | total_timesteps 214.
Path 19 | total_timesteps 224.
Path 20 | total_timesteps 252.
Path 21 | total_timesteps 264.
Path 22 | total_timesteps 281.
Path 23 | total_timesteps 292.
Path 24 | total_timesteps 305.
Path 25 | total_timesteps 315.
Path 26 | total_timesteps 326.
Path 27 | total_timesteps 342.
Path 28 | total_timesteps 351.
Path 29 | total_timesteps 359.
Path 30 | total_timesteps 374.
Path 31 | total_timesteps 397.
Path 32 | total_timesteps 410.
Path 33 | total_timesteps 419.
Path 34 | total_timesteps 426.
Path 35 | total_timesteps 435.
Path 36 | total_timesteps 455.
Path 37 | total_timesteps 465.
Path 38 | total_timesteps 473.
Path 39 | total_timesteps 484.
Path 40 | total_timesteps 500.
Path 41 | total_timesteps 515.
Path 42 | total_timesteps 528.
Path 43 | total_timesteps 538.
Path 44 | total_timesteps 548.
Path 45 | total_timesteps 560.
Path 46 | total_timesteps 573.
Path 47 | total_timesteps 588.
Path 48 | total_timesteps 598.
Path 49 | total_timesteps 619.
Path 50 | total_timesteps 629.
Path 51 | total_timesteps 638.
Path 52 | total_timesteps 646.
Path 53 | total_timesteps 655.
Path 54 | total_timesteps 663.
Path 55 | total_timesteps 676.
Path 56 | total_timesteps 685.
Path 57 | total_timesteps 695.
Path 58 | total_timesteps 712.
Path 59 | total_timesteps 735.
Path 60 | total_timesteps 750.
Path 61 | total_timesteps 768.
Path 62 | total_timesteps 781.
Path 63 | total_timesteps 789.
Path 64 | total_timesteps 802.
Path 65 | total_timesteps 820.
Path 66 | total_timesteps 833.
Path 67 | total_timesteps 843.
Path 68 | total_timesteps 852.
Path 69 | total_timesteps 867.
Path 70 | total_timesteps 877.
Path 71 | total_timesteps 885.
Path 72 | total_timesteps 894.
Path 73 | total_timesteps 904.
Path 74 | total_timesteps 916.
Path 75 | total_timesteps 930.
Path 76 | total_timesteps 942.
Path 77 | total_timesteps 953.
Path 78 | total_timesteps 964.
Path 79 | total_timesteps 976.
Path 80 | total_timesteps 986.
Path 81 | total_timesteps 997.
Path 82 | total_timesteps 1010.
Path 83 | total_timesteps 1018.
Path 84 | total_timesteps 1026.
Path 85 | total_timesteps 1044.
Path 86 | total_timesteps 1054.
Path 87 | total_timesteps 1073.
Path 88 | total_timesteps 1081.
Path 89 | total_timesteps 1094.
Path 90 | total_timesteps 1103.
Path 91 | total_timesteps 1116.
Path 92 | total_timesteps 1131.
Path 93 | total_timesteps 1141.
Path 94 | total_timesteps 1158.
Path 95 | total_timesteps 1168.
Path 96 | total_timesteps 1183.
Path 97 | total_timesteps 1192.
Path 98 | total_timesteps 1202.
Path 99 | total_timesteps 1212.
Path 100 | total_timesteps 1222.
Path 101 | total_timesteps 1229.
Path 102 | total_timesteps 1241.
Path 103 | total_timesteps 1254.
Path 104 | total_timesteps 1265.
Path 105 | total_timesteps 1274.
Path 106 | total_timesteps 1289.
Path 107 | total_timesteps 1314.
Path 108 | total_timesteps 1338.
Path 109 | total_timesteps 1347.
Path 110 | total_timesteps 1363.
Path 111 | total_timesteps 1377.
Path 112 | total_timesteps 1391.
Path 113 | total_timesteps 1401.
Path 114 | total_timesteps 1409.
Path 115 | total_timesteps 1420.
Path 116 | total_timesteps 1440.
Path 117 | total_timesteps 1452.
Path 118 | total_timesteps 1462.
Path 119 | total_timesteps 1472.
Path 120 | total_timesteps 1483.
Path 121 | total_timesteps 1491.
Path 122 | total_timesteps 1505.
Path 123 | total_timesteps 1524.
Path 124 | total_timesteps 1533.
Path 125 | total_timesteps 1541.
Path 126 | total_timesteps 1552.
Path 127 | total_timesteps 1561.
Path 128 | total_timesteps 1571.
Path 129 | total_timesteps 1581.
Path 130 | total_timesteps 1595.
Path 131 | total_timesteps 1603.
Path 132 | total_timesteps 1615.
Path 133 | total_timesteps 1625.
Path 134 | total_timesteps 1636.
Path 135 | total_timesteps 1644.
Path 136 | total_timesteps 1653.
Path 137 | total_timesteps 1669.
Path 138 | total_timesteps 1682.
Path 139 | total_timesteps 1691.
Path 140 | total_timesteps 1698.
Path 141 | total_timesteps 1710.
Path 142 | total_timesteps 1720.
Path 143 | total_timesteps 1728.
Path 144 | total_timesteps 1740.
Path 145 | total_timesteps 1751.
Path 146 | total_timesteps 1767.
Path 147 | total_timesteps 1778.
Path 148 | total_timesteps 1796.
Path 149 | total_timesteps 1810.
Path 150 | total_timesteps 1822.
Path 151 | total_timesteps 1833.
Path 152 | total_timesteps 1849.
Path 153 | total_timesteps 1857.
Path 154 | total_timesteps 1866.
Path 155 | total_timesteps 1877.
Path 156 | total_timesteps 1887.
Path 157 | total_timesteps 1903.
Path 158 | total_timesteps 1916.
Path 159 | total_timesteps 1924.
Path 160 | total_timesteps 1933.
Path 161 | total_timesteps 1944.
Path 162 | total_timesteps 1954.
Path 163 | total_timesteps 1967.
Path 164 | total_timesteps 1976.
Path 165 | total_timesteps 1984.
Path 166 | total_timesteps 1995.
Path 167 | total_timesteps 2008.
Path 168 | total_timesteps 2019.
Path 169 | total_timesteps 2034.
Path 170 | total_timesteps 2044.
Path 171 | total_timesteps 2070.
Path 172 | total_timesteps 2083.
Path 173 | total_timesteps 2095.
Path 174 | total_timesteps 2122.
Path 175 | total_timesteps 2139.
Path 176 | total_timesteps 2149.
Path 177 | total_timesteps 2159.
Path 178 | total_timesteps 2168.
Path 179 | total_timesteps 2175.
Path 180 | total_timesteps 2188.
Path 181 | total_timesteps 2197.
Path 182 | total_timesteps 2210.
Path 183 | total_timesteps 2219.
Path 184 | total_timesteps 2229.
Path 185 | total_timesteps 2243.
Path 186 | total_timesteps 2254.
Path 187 | total_timesteps 2264.
Path 188 | total_timesteps 2283.
Path 189 | total_timesteps 2297.
Path 190 | total_timesteps 2308.
Path 191 | total_timesteps 2319.
Path 192 | total_timesteps 2333.
Path 193 | total_timesteps 2341.
Path 194 | total_timesteps 2349.
Path 195 | total_timesteps 2362.
Path 196 | total_timesteps 2374.
Path 197 | total_timesteps 2386.
Path 198 | total_timesteps 2399.
Path 199 | total_timesteps 2413.
Path 200 | total_timesteps 2425.
Path 201 | total_timesteps 2437.
Path 202 | total_timesteps 2450.
Path 203 | total_timesteps 2459.
Path 204 | total_timesteps 2470.
Path 205 | total_timesteps 2490.
Path 206 | total_timesteps 2498.
Path 207 | total_timesteps 2520.
Path 208 | total_timesteps 2532.
Path 209 | total_timesteps 2549.
Path 210 | total_timesteps 2564.
Path 211 | total_timesteps 2581.
Path 212 | total_timesteps 2592.
Path 213 | total_timesteps 2610.
Path 214 | total_timesteps 2620.
Path 215 | total_timesteps 2637.
Path 216 | total_timesteps 2644.
Path 217 | total_timesteps 2658.
Path 218 | total_timesteps 2667.
Path 219 | total_timesteps 2675.
Path 220 | total_timesteps 2683.
Path 221 | total_timesteps 2693.
Path 222 | total_timesteps 2704.
Path 223 | total_timesteps 2733.
Path 224 | total_timesteps 2748.
Path 225 | total_timesteps 2757.
Path 226 | total_timesteps 2770.
Path 227 | total_timesteps 2794.
Path 228 | total_timesteps 2810.
Path 229 | total_timesteps 2820.
Path 230 | total_timesteps 2831.
Path 231 | total_timesteps 2841.
Path 232 | total_timesteps 2851.
Path 233 | total_timesteps 2876.
Path 234 | total_timesteps 2893.
Path 235 | total_timesteps 2908.
Path 236 | total_timesteps 2917.
Path 237 | total_timesteps 2931.
Path 238 | total_timesteps 2943.
Path 239 | total_timesteps 2953.
Path 240 | total_timesteps 2964.
Path 241 | total_timesteps 2977.
Path 242 | total_timesteps 2986.
Path 243 | total_timesteps 3000.
Path 244 | total_timesteps 3009.
Path 245 | total_timesteps 3018.
Path 246 | total_timesteps 3028.
Path 247 | total_timesteps 3036.
Path 248 | total_timesteps 3043.
Path 249 | total_timesteps 3055.
Path 250 | total_timesteps 3065.
Path 251 | total_timesteps 3081.
Path 252 | total_timesteps 3095.
Path 253 | total_timesteps 3106.
Path 254 | total_timesteps 3119.
Path 255 | total_timesteps 3135.
Path 256 | total_timesteps 3144.
Path 257 | total_timesteps 3151.
Path 258 | total_timesteps 3167.
Path 259 | total_timesteps 3177.
Path 260 | total_timesteps 3193.
Path 261 | total_timesteps 3208.
Path 262 | total_timesteps 3220.
Path 263 | total_timesteps 3233.
Path 264 | total_timesteps 3250.
Path 265 | total_timesteps 3273.
Path 266 | total_timesteps 3290.
Path 267 | total_timesteps 3326.
Path 268 | total_timesteps 3338.
Path 269 | total_timesteps 3356.
Path 270 | total_timesteps 3367.
Path 271 | total_timesteps 3377.
Path 272 | total_timesteps 3384.
Path 273 | total_timesteps 3396.
Path 274 | total_timesteps 3412.
Path 275 | total_timesteps 3422.
Path 276 | total_timesteps 3433.
Path 277 | total_timesteps 3442.
Path 278 | total_timesteps 3455.
Path 279 | total_timesteps 3468.
Path 280 | total_timesteps 3486.
Path 281 | total_timesteps 3498.
Path 282 | total_timesteps 3508.
Path 283 | total_timesteps 3518.
Path 284 | total_timesteps 3528.
Path 285 | total_timesteps 3540.
Path 286 | total_timesteps 3552.
Path 287 | total_timesteps 3561.
Path 288 | total_timesteps 3576.
Path 289 | total_timesteps 3588.
Path 290 | total_timesteps 3596.
Path 291 | total_timesteps 3607.
Path 292 | total_timesteps 3618.
Path 293 | total_timesteps 3628.
Path 294 | total_timesteps 3637.
Path 295 | total_timesteps 3645.
Path 296 | total_timesteps 3655.
Path 297 | total_timesteps 3665.
Path 298 | total_timesteps 3678.
Path 299 | total_timesteps 3686.
Path 300 | total_timesteps 3698.
Path 301 | total_timesteps 3709.
Path 302 | total_timesteps 3718.
Path 303 | total_timesteps 3726.
Path 304 | total_timesteps 3736.
Path 305 | total_timesteps 3747.
Path 306 | total_timesteps 3759.
Path 307 | total_timesteps 3769.
Path 308 | total_timesteps 3779.
Path 309 | total_timesteps 3802.
Path 310 | total_timesteps 3819.
Path 311 | total_timesteps 3836.
Path 312 | total_timesteps 3848.
Path 313 | total_timesteps 3857.
Path 314 | total_timesteps 3866.
Path 315 | total_timesteps 3874.
Path 316 | total_timesteps 3884.
Path 317 | total_timesteps 3896.
Path 318 | total_timesteps 3907.
Path 319 | total_timesteps 3919.
Path 320 | total_timesteps 3938.
Path 321 | total_timesteps 3961.
Path 322 | total_timesteps 3971.
Path 323 | total_timesteps 3980.
Path 324 | total_timesteps 3990.
Path 325 | total_timesteps 3997.
Path 326 | total_timesteps 4010.
Path 327 | total_timesteps 4019.
Path 328 | total_timesteps 4027.
Path 329 | total_timesteps 4041.
Path 330 | total_timesteps 4051.
Path 331 | total_timesteps 4059.
Path 332 | total_timesteps 4068.
Path 333 | total_timesteps 4081.
Path 334 | total_timesteps 4090.
Path 335 | total_timesteps 4101.
Path 336 | total_timesteps 4111.
Path 337 | total_timesteps 4122.
Path 338 | total_timesteps 4129.
Path 339 | total_timesteps 4141.
Path 340 | total_timesteps 4151.
Path 341 | total_timesteps 4160.
Path 342 | total_timesteps 4170.
Path 343 | total_timesteps 4185.
Path 344 | total_timesteps 4200.
Path 345 | total_timesteps 4210.
Path 346 | total_timesteps 4220.
Path 347 | total_timesteps 4243.
Path 348 | total_timesteps 4253.
Path 349 | total_timesteps 4267.
Path 350 | total_timesteps 4277.
Path 351 | total_timesteps 4287.
Path 352 | total_timesteps 4295.
Path 353 | total_timesteps 4307.
Path 354 | total_timesteps 4314.
Path 355 | total_timesteps 4335.
Path 356 | total_timesteps 4347.
Path 357 | total_timesteps 4359.
Path 358 | total_timesteps 4375.
Path 359 | total_timesteps 4384.
Path 360 | total_timesteps 4408.
Path 361 | total_timesteps 4419.
Path 362 | total_timesteps 4432.
Path 363 | total_timesteps 4440.
Path 364 | total_timesteps 4456.
Path 365 | total_timesteps 4469.
Path 366 | total_timesteps 4478.
Path 367 | total_timesteps 4492.
Path 368 | total_timesteps 4506.
Path 369 | total_timesteps 4515.
Path 370 | total_timesteps 4525.
Path 371 | total_timesteps 4546.
Path 372 | total_timesteps 4562.
Path 373 | total_timesteps 4572.
Path 374 | total_timesteps 4582.
Path 375 | total_timesteps 4599.
Path 376 | total_timesteps 4608.
Path 377 | total_timesteps 4624.
Path 378 | total_timesteps 4632.
Path 379 | total_timesteps 4645.
Path 380 | total_timesteps 4656.
Path 381 | total_timesteps 4667.
Path 382 | total_timesteps 4675.
Path 383 | total_timesteps 4687.
Path 384 | total_timesteps 4696.
Path 385 | total_timesteps 4705.
Path 386 | total_timesteps 4717.
Path 387 | total_timesteps 4730.
Path 388 | total_timesteps 4743.
Path 389 | total_timesteps 4752.
Path 390 | total_timesteps 4763.
Path 391 | total_timesteps 4771.
Path 392 | total_timesteps 4779.
Path 393 | total_timesteps 4788.
Path 394 | total_timesteps 4799.
Path 395 | total_timesteps 4809.
Path 396 | total_timesteps 4827.
Path 397 | total_timesteps 4841.
Path 398 | total_timesteps 4850.
Path 399 | total_timesteps 4869.
Path 400 | total_timesteps 4882.
Path 401 | total_timesteps 4894.
Path 402 | total_timesteps 4903.
Path 403 | total_timesteps 4911.
Path 404 | total_timesteps 4928.
Path 405 | total_timesteps 4941.
Path 406 | total_timesteps 4965.
Path 407 | total_timesteps 4979.
Path 408 | total_timesteps 4989.
Path 409 | total_timesteps 5001.
Path 410 | total_timesteps 5007.
Path 411 | total_timesteps 5024.
Path 412 | total_timesteps 5035.
Path 413 | total_timesteps 5046.
Path 414 | total_timesteps 5055.
Path 415 | total_timesteps 5064.
Path 416 | total_timesteps 5088.
Path 417 | total_timesteps 5099.
Path 418 | total_timesteps 5113.
Path 419 | total_timesteps 5126.
Path 420 | total_timesteps 5139.
Path 421 | total_timesteps 5156.
Path 422 | total_timesteps 5174.
Path 423 | total_timesteps 5184.
Path 424 | total_timesteps 5202.
Path 425 | total_timesteps 5209.
Path 426 | total_timesteps 5218.
Path 427 | total_timesteps 5228.
Path 428 | total_timesteps 5242.
Path 429 | total_timesteps 5252.
Path 430 | total_timesteps 5259.
Path 431 | total_timesteps 5270.
Path 432 | total_timesteps 5281.
Path 433 | total_timesteps 5297.
Path 434 | total_timesteps 5306.
Path 435 | total_timesteps 5319.
Path 436 | total_timesteps 5335.
Path 437 | total_timesteps 5343.
Path 438 | total_timesteps 5356.
Path 439 | total_timesteps 5376.
Path 440 | total_timesteps 5397.
Path 441 | total_timesteps 5409.
Path 442 | total_timesteps 5427.
Path 443 | total_timesteps 5438.
Path 444 | total_timesteps 5458.
Path 445 | total_timesteps 5480.
Path 446 | total_timesteps 5491.
Path 447 | total_timesteps 5508.
Path 448 | total_timesteps 5516.
Path 449 | total_timesteps 5527.
Path 450 | total_timesteps 5538.
Path 451 | total_timesteps 5551.
Path 452 | total_timesteps 5561.
Path 453 | total_timesteps 5577.
Path 454 | total_timesteps 5586.
Path 455 | total_timesteps 5600.
Path 456 | total_timesteps 5614.
Path 457 | total_timesteps 5636.
Path 458 | total_timesteps 5651.
Path 459 | total_timesteps 5661.
Path 460 | total_timesteps 5680.
Path 461 | total_timesteps 5689.
Path 462 | total_timesteps 5702.
Path 463 | total_timesteps 5717.
Path 464 | total_timesteps 5724.
Path 465 | total_timesteps 5734.
Path 466 | total_timesteps 5744.
Path 467 | total_timesteps 5754.
Path 468 | total_timesteps 5764.
Path 469 | total_timesteps 5784.
Path 470 | total_timesteps 5796.
Path 471 | total_timesteps 5812.
Path 472 | total_timesteps 5822.
Path 473 | total_timesteps 5837.
Path 474 | total_timesteps 5847.
Path 475 | total_timesteps 5859.
Path 476 | total_timesteps 5869.
Path 477 | total_timesteps 5884.
Path 478 | total_timesteps 5897.
Path 479 | total_timesteps 5905.
Path 480 | total_timesteps 5916.
Path 481 | total_timesteps 5927.
Path 482 | total_timesteps 5941.
Path 483 | total_timesteps 5971.
Path 484 | total_timesteps 5980.
Path 485 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.78    |
| Iteration     | 2        |
| MaximumReturn | -0.0912  |
| MinimumReturn | -23.8    |
| TotalSamples  | 16023    |
----------------------------
itr #3 | 
Fitting dynamics.
Validation loss = 0.03102719411253929
Validation loss = 0.02528722956776619
Validation loss = 0.023114632815122604
Validation loss = 0.022961048409342766
Validation loss = 0.02239210531115532
Validation loss = 0.024069715291261673
Validation loss = 0.022668834775686264
Validation loss = 0.02306366339325905
Validation loss = 0.022964924573898315
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 23.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 61.
Path 6 | total_timesteps 77.
Path 7 | total_timesteps 87.
Path 8 | total_timesteps 99.
Path 9 | total_timesteps 116.
Path 10 | total_timesteps 126.
Path 11 | total_timesteps 135.
Path 12 | total_timesteps 149.
Path 13 | total_timesteps 156.
Path 14 | total_timesteps 167.
Path 15 | total_timesteps 179.
Path 16 | total_timesteps 190.
Path 17 | total_timesteps 206.
Path 18 | total_timesteps 216.
Path 19 | total_timesteps 225.
Path 20 | total_timesteps 234.
Path 21 | total_timesteps 244.
Path 22 | total_timesteps 253.
Path 23 | total_timesteps 265.
Path 24 | total_timesteps 274.
Path 25 | total_timesteps 291.
Path 26 | total_timesteps 302.
Path 27 | total_timesteps 319.
Path 28 | total_timesteps 326.
Path 29 | total_timesteps 336.
Path 30 | total_timesteps 347.
Path 31 | total_timesteps 356.
Path 32 | total_timesteps 366.
Path 33 | total_timesteps 377.
Path 34 | total_timesteps 385.
Path 35 | total_timesteps 393.
Path 36 | total_timesteps 405.
Path 37 | total_timesteps 414.
Path 38 | total_timesteps 428.
Path 39 | total_timesteps 440.
Path 40 | total_timesteps 456.
Path 41 | total_timesteps 462.
Path 42 | total_timesteps 474.
Path 43 | total_timesteps 486.
Path 44 | total_timesteps 499.
Path 45 | total_timesteps 509.
Path 46 | total_timesteps 519.
Path 47 | total_timesteps 531.
Path 48 | total_timesteps 540.
Path 49 | total_timesteps 548.
Path 50 | total_timesteps 566.
Path 51 | total_timesteps 574.
Path 52 | total_timesteps 585.
Path 53 | total_timesteps 600.
Path 54 | total_timesteps 610.
Path 55 | total_timesteps 621.
Path 56 | total_timesteps 632.
Path 57 | total_timesteps 645.
Path 58 | total_timesteps 653.
Path 59 | total_timesteps 663.
Path 60 | total_timesteps 673.
Path 61 | total_timesteps 688.
Path 62 | total_timesteps 700.
Path 63 | total_timesteps 712.
Path 64 | total_timesteps 721.
Path 65 | total_timesteps 728.
Path 66 | total_timesteps 741.
Path 67 | total_timesteps 754.
Path 68 | total_timesteps 768.
Path 69 | total_timesteps 776.
Path 70 | total_timesteps 788.
Path 71 | total_timesteps 803.
Path 72 | total_timesteps 819.
Path 73 | total_timesteps 833.
Path 74 | total_timesteps 841.
Path 75 | total_timesteps 848.
Path 76 | total_timesteps 859.
Path 77 | total_timesteps 869.
Path 78 | total_timesteps 879.
Path 79 | total_timesteps 891.
Path 80 | total_timesteps 901.
Path 81 | total_timesteps 912.
Path 82 | total_timesteps 919.
Path 83 | total_timesteps 928.
Path 84 | total_timesteps 937.
Path 85 | total_timesteps 951.
Path 86 | total_timesteps 961.
Path 87 | total_timesteps 969.
Path 88 | total_timesteps 982.
Path 89 | total_timesteps 991.
Path 90 | total_timesteps 999.
Path 91 | total_timesteps 1008.
Path 92 | total_timesteps 1017.
Path 93 | total_timesteps 1027.
Path 94 | total_timesteps 1038.
Path 95 | total_timesteps 1049.
Path 96 | total_timesteps 1057.
Path 97 | total_timesteps 1065.
Path 98 | total_timesteps 1078.
Path 99 | total_timesteps 1088.
Path 100 | total_timesteps 1106.
Path 101 | total_timesteps 1121.
Path 102 | total_timesteps 1134.
Path 103 | total_timesteps 1141.
Path 104 | total_timesteps 1154.
Path 105 | total_timesteps 1167.
Path 106 | total_timesteps 1184.
Path 107 | total_timesteps 1193.
Path 108 | total_timesteps 1201.
Path 109 | total_timesteps 1212.
Path 110 | total_timesteps 1224.
Path 111 | total_timesteps 1235.
Path 112 | total_timesteps 1242.
Path 113 | total_timesteps 1254.
Path 114 | total_timesteps 1261.
Path 115 | total_timesteps 1270.
Path 116 | total_timesteps 1277.
Path 117 | total_timesteps 1286.
Path 118 | total_timesteps 1297.
Path 119 | total_timesteps 1309.
Path 120 | total_timesteps 1320.
Path 121 | total_timesteps 1337.
Path 122 | total_timesteps 1345.
Path 123 | total_timesteps 1360.
Path 124 | total_timesteps 1366.
Path 125 | total_timesteps 1377.
Path 126 | total_timesteps 1384.
Path 127 | total_timesteps 1406.
Path 128 | total_timesteps 1416.
Path 129 | total_timesteps 1425.
Path 130 | total_timesteps 1435.
Path 131 | total_timesteps 1444.
Path 132 | total_timesteps 1453.
Path 133 | total_timesteps 1473.
Path 134 | total_timesteps 1481.
Path 135 | total_timesteps 1492.
Path 136 | total_timesteps 1501.
Path 137 | total_timesteps 1510.
Path 138 | total_timesteps 1519.
Path 139 | total_timesteps 1529.
Path 140 | total_timesteps 1535.
Path 141 | total_timesteps 1543.
Path 142 | total_timesteps 1554.
Path 143 | total_timesteps 1564.
Path 144 | total_timesteps 1576.
Path 145 | total_timesteps 1602.
Path 146 | total_timesteps 1612.
Path 147 | total_timesteps 1620.
Path 148 | total_timesteps 1632.
Path 149 | total_timesteps 1647.
Path 150 | total_timesteps 1657.
Path 151 | total_timesteps 1665.
Path 152 | total_timesteps 1673.
Path 153 | total_timesteps 1687.
Path 154 | total_timesteps 1697.
Path 155 | total_timesteps 1709.
Path 156 | total_timesteps 1721.
Path 157 | total_timesteps 1731.
Path 158 | total_timesteps 1742.
Path 159 | total_timesteps 1749.
Path 160 | total_timesteps 1760.
Path 161 | total_timesteps 1774.
Path 162 | total_timesteps 1783.
Path 163 | total_timesteps 1794.
Path 164 | total_timesteps 1812.
Path 165 | total_timesteps 1820.
Path 166 | total_timesteps 1828.
Path 167 | total_timesteps 1841.
Path 168 | total_timesteps 1851.
Path 169 | total_timesteps 1859.
Path 170 | total_timesteps 1876.
Path 171 | total_timesteps 1887.
Path 172 | total_timesteps 1899.
Path 173 | total_timesteps 1911.
Path 174 | total_timesteps 1921.
Path 175 | total_timesteps 1930.
Path 176 | total_timesteps 1942.
Path 177 | total_timesteps 1956.
Path 178 | total_timesteps 1963.
Path 179 | total_timesteps 1971.
Path 180 | total_timesteps 1984.
Path 181 | total_timesteps 1998.
Path 182 | total_timesteps 2007.
Path 183 | total_timesteps 2019.
Path 184 | total_timesteps 2026.
Path 185 | total_timesteps 2034.
Path 186 | total_timesteps 2042.
Path 187 | total_timesteps 2051.
Path 188 | total_timesteps 2059.
Path 189 | total_timesteps 2069.
Path 190 | total_timesteps 2076.
Path 191 | total_timesteps 2086.
Path 192 | total_timesteps 2093.
Path 193 | total_timesteps 2101.
Path 194 | total_timesteps 2115.
Path 195 | total_timesteps 2123.
Path 196 | total_timesteps 2132.
Path 197 | total_timesteps 2143.
Path 198 | total_timesteps 2160.
Path 199 | total_timesteps 2173.
Path 200 | total_timesteps 2183.
Path 201 | total_timesteps 2194.
Path 202 | total_timesteps 2204.
Path 203 | total_timesteps 2215.
Path 204 | total_timesteps 2224.
Path 205 | total_timesteps 2240.
Path 206 | total_timesteps 2249.
Path 207 | total_timesteps 2261.
Path 208 | total_timesteps 2271.
Path 209 | total_timesteps 2281.
Path 210 | total_timesteps 2291.
Path 211 | total_timesteps 2304.
Path 212 | total_timesteps 2311.
Path 213 | total_timesteps 2320.
Path 214 | total_timesteps 2330.
Path 215 | total_timesteps 2337.
Path 216 | total_timesteps 2346.
Path 217 | total_timesteps 2355.
Path 218 | total_timesteps 2375.
Path 219 | total_timesteps 2386.
Path 220 | total_timesteps 2393.
Path 221 | total_timesteps 2401.
Path 222 | total_timesteps 2410.
Path 223 | total_timesteps 2426.
Path 224 | total_timesteps 2445.
Path 225 | total_timesteps 2455.
Path 226 | total_timesteps 2464.
Path 227 | total_timesteps 2473.
Path 228 | total_timesteps 2483.
Path 229 | total_timesteps 2500.
Path 230 | total_timesteps 2511.
Path 231 | total_timesteps 2528.
Path 232 | total_timesteps 2537.
Path 233 | total_timesteps 2546.
Path 234 | total_timesteps 2554.
Path 235 | total_timesteps 2564.
Path 236 | total_timesteps 2582.
Path 237 | total_timesteps 2591.
Path 238 | total_timesteps 2609.
Path 239 | total_timesteps 2621.
Path 240 | total_timesteps 2628.
Path 241 | total_timesteps 2637.
Path 242 | total_timesteps 2652.
Path 243 | total_timesteps 2665.
Path 244 | total_timesteps 2675.
Path 245 | total_timesteps 2685.
Path 246 | total_timesteps 2700.
Path 247 | total_timesteps 2709.
Path 248 | total_timesteps 2719.
Path 249 | total_timesteps 2728.
Path 250 | total_timesteps 2737.
Path 251 | total_timesteps 2748.
Path 252 | total_timesteps 2757.
Path 253 | total_timesteps 2769.
Path 254 | total_timesteps 2785.
Path 255 | total_timesteps 2794.
Path 256 | total_timesteps 2804.
Path 257 | total_timesteps 2813.
Path 258 | total_timesteps 2825.
Path 259 | total_timesteps 2835.
Path 260 | total_timesteps 2844.
Path 261 | total_timesteps 2858.
Path 262 | total_timesteps 2868.
Path 263 | total_timesteps 2876.
Path 264 | total_timesteps 2890.
Path 265 | total_timesteps 2901.
Path 266 | total_timesteps 2914.
Path 267 | total_timesteps 2925.
Path 268 | total_timesteps 2936.
Path 269 | total_timesteps 2952.
Path 270 | total_timesteps 2963.
Path 271 | total_timesteps 2976.
Path 272 | total_timesteps 2988.
Path 273 | total_timesteps 2997.
Path 274 | total_timesteps 3009.
Path 275 | total_timesteps 3017.
Path 276 | total_timesteps 3032.
Path 277 | total_timesteps 3040.
Path 278 | total_timesteps 3060.
Path 279 | total_timesteps 3072.
Path 280 | total_timesteps 3082.
Path 281 | total_timesteps 3094.
Path 282 | total_timesteps 3103.
Path 283 | total_timesteps 3115.
Path 284 | total_timesteps 3123.
Path 285 | total_timesteps 3132.
Path 286 | total_timesteps 3148.
Path 287 | total_timesteps 3163.
Path 288 | total_timesteps 3173.
Path 289 | total_timesteps 3184.
Path 290 | total_timesteps 3190.
Path 291 | total_timesteps 3200.
Path 292 | total_timesteps 3207.
Path 293 | total_timesteps 3220.
Path 294 | total_timesteps 3235.
Path 295 | total_timesteps 3243.
Path 296 | total_timesteps 3252.
Path 297 | total_timesteps 3264.
Path 298 | total_timesteps 3275.
Path 299 | total_timesteps 3293.
Path 300 | total_timesteps 3304.
Path 301 | total_timesteps 3314.
Path 302 | total_timesteps 3326.
Path 303 | total_timesteps 3343.
Path 304 | total_timesteps 3356.
Path 305 | total_timesteps 3365.
Path 306 | total_timesteps 3373.
Path 307 | total_timesteps 3395.
Path 308 | total_timesteps 3405.
Path 309 | total_timesteps 3422.
Path 310 | total_timesteps 3433.
Path 311 | total_timesteps 3443.
Path 312 | total_timesteps 3452.
Path 313 | total_timesteps 3465.
Path 314 | total_timesteps 3478.
Path 315 | total_timesteps 3485.
Path 316 | total_timesteps 3500.
Path 317 | total_timesteps 3514.
Path 318 | total_timesteps 3522.
Path 319 | total_timesteps 3532.
Path 320 | total_timesteps 3552.
Path 321 | total_timesteps 3563.
Path 322 | total_timesteps 3572.
Path 323 | total_timesteps 3581.
Path 324 | total_timesteps 3590.
Path 325 | total_timesteps 3598.
Path 326 | total_timesteps 3609.
Path 327 | total_timesteps 3625.
Path 328 | total_timesteps 3632.
Path 329 | total_timesteps 3641.
Path 330 | total_timesteps 3648.
Path 331 | total_timesteps 3656.
Path 332 | total_timesteps 3667.
Path 333 | total_timesteps 3675.
Path 334 | total_timesteps 3686.
Path 335 | total_timesteps 3696.
Path 336 | total_timesteps 3705.
Path 337 | total_timesteps 3714.
Path 338 | total_timesteps 3725.
Path 339 | total_timesteps 3736.
Path 340 | total_timesteps 3746.
Path 341 | total_timesteps 3760.
Path 342 | total_timesteps 3776.
Path 343 | total_timesteps 3795.
Path 344 | total_timesteps 3805.
Path 345 | total_timesteps 3813.
Path 346 | total_timesteps 3824.
Path 347 | total_timesteps 3833.
Path 348 | total_timesteps 3842.
Path 349 | total_timesteps 3853.
Path 350 | total_timesteps 3862.
Path 351 | total_timesteps 3872.
Path 352 | total_timesteps 3879.
Path 353 | total_timesteps 3890.
Path 354 | total_timesteps 3908.
Path 355 | total_timesteps 3918.
Path 356 | total_timesteps 3925.
Path 357 | total_timesteps 3933.
Path 358 | total_timesteps 3947.
Path 359 | total_timesteps 3961.
Path 360 | total_timesteps 3969.
Path 361 | total_timesteps 3985.
Path 362 | total_timesteps 3995.
Path 363 | total_timesteps 4004.
Path 364 | total_timesteps 4014.
Path 365 | total_timesteps 4021.
Path 366 | total_timesteps 4032.
Path 367 | total_timesteps 4044.
Path 368 | total_timesteps 4070.
Path 369 | total_timesteps 4079.
Path 370 | total_timesteps 4095.
Path 371 | total_timesteps 4103.
Path 372 | total_timesteps 4119.
Path 373 | total_timesteps 4127.
Path 374 | total_timesteps 4137.
Path 375 | total_timesteps 4150.
Path 376 | total_timesteps 4163.
Path 377 | total_timesteps 4172.
Path 378 | total_timesteps 4184.
Path 379 | total_timesteps 4195.
Path 380 | total_timesteps 4204.
Path 381 | total_timesteps 4211.
Path 382 | total_timesteps 4226.
Path 383 | total_timesteps 4241.
Path 384 | total_timesteps 4252.
Path 385 | total_timesteps 4262.
Path 386 | total_timesteps 4271.
Path 387 | total_timesteps 4282.
Path 388 | total_timesteps 4290.
Path 389 | total_timesteps 4303.
Path 390 | total_timesteps 4318.
Path 391 | total_timesteps 4331.
Path 392 | total_timesteps 4342.
Path 393 | total_timesteps 4353.
Path 394 | total_timesteps 4362.
Path 395 | total_timesteps 4391.
Path 396 | total_timesteps 4402.
Path 397 | total_timesteps 4412.
Path 398 | total_timesteps 4425.
Path 399 | total_timesteps 4435.
Path 400 | total_timesteps 4445.
Path 401 | total_timesteps 4457.
Path 402 | total_timesteps 4467.
Path 403 | total_timesteps 4477.
Path 404 | total_timesteps 4490.
Path 405 | total_timesteps 4503.
Path 406 | total_timesteps 4511.
Path 407 | total_timesteps 4521.
Path 408 | total_timesteps 4537.
Path 409 | total_timesteps 4555.
Path 410 | total_timesteps 4567.
Path 411 | total_timesteps 4576.
Path 412 | total_timesteps 4587.
Path 413 | total_timesteps 4598.
Path 414 | total_timesteps 4610.
Path 415 | total_timesteps 4620.
Path 416 | total_timesteps 4643.
Path 417 | total_timesteps 4657.
Path 418 | total_timesteps 4666.
Path 419 | total_timesteps 4677.
Path 420 | total_timesteps 4690.
Path 421 | total_timesteps 4702.
Path 422 | total_timesteps 4721.
Path 423 | total_timesteps 4730.
Path 424 | total_timesteps 4737.
Path 425 | total_timesteps 4750.
Path 426 | total_timesteps 4762.
Path 427 | total_timesteps 4772.
Path 428 | total_timesteps 4786.
Path 429 | total_timesteps 4795.
Path 430 | total_timesteps 4803.
Path 431 | total_timesteps 4811.
Path 432 | total_timesteps 4826.
Path 433 | total_timesteps 4835.
Path 434 | total_timesteps 4853.
Path 435 | total_timesteps 4864.
Path 436 | total_timesteps 4879.
Path 437 | total_timesteps 4889.
Path 438 | total_timesteps 4897.
Path 439 | total_timesteps 4904.
Path 440 | total_timesteps 4911.
Path 441 | total_timesteps 4922.
Path 442 | total_timesteps 4931.
Path 443 | total_timesteps 4940.
Path 444 | total_timesteps 4951.
Path 445 | total_timesteps 4960.
Path 446 | total_timesteps 4969.
Path 447 | total_timesteps 4984.
Path 448 | total_timesteps 4993.
Path 449 | total_timesteps 5006.
Path 450 | total_timesteps 5016.
Path 451 | total_timesteps 5026.
Path 452 | total_timesteps 5037.
Path 453 | total_timesteps 5047.
Path 454 | total_timesteps 5053.
Path 455 | total_timesteps 5063.
Path 456 | total_timesteps 5070.
Path 457 | total_timesteps 5082.
Path 458 | total_timesteps 5092.
Path 459 | total_timesteps 5102.
Path 460 | total_timesteps 5111.
Path 461 | total_timesteps 5120.
Path 462 | total_timesteps 5129.
Path 463 | total_timesteps 5145.
Path 464 | total_timesteps 5157.
Path 465 | total_timesteps 5168.
Path 466 | total_timesteps 5176.
Path 467 | total_timesteps 5182.
Path 468 | total_timesteps 5189.
Path 469 | total_timesteps 5199.
Path 470 | total_timesteps 5207.
Path 471 | total_timesteps 5217.
Path 472 | total_timesteps 5225.
Path 473 | total_timesteps 5234.
Path 474 | total_timesteps 5240.
Path 475 | total_timesteps 5249.
Path 476 | total_timesteps 5261.
Path 477 | total_timesteps 5269.
Path 478 | total_timesteps 5277.
Path 479 | total_timesteps 5291.
Path 480 | total_timesteps 5299.
Path 481 | total_timesteps 5312.
Path 482 | total_timesteps 5322.
Path 483 | total_timesteps 5331.
Path 484 | total_timesteps 5340.
Path 485 | total_timesteps 5349.
Path 486 | total_timesteps 5361.
Path 487 | total_timesteps 5370.
Path 488 | total_timesteps 5383.
Path 489 | total_timesteps 5393.
Path 490 | total_timesteps 5408.
Path 491 | total_timesteps 5435.
Path 492 | total_timesteps 5447.
Path 493 | total_timesteps 5466.
Path 494 | total_timesteps 5476.
Path 495 | total_timesteps 5489.
Path 496 | total_timesteps 5502.
Path 497 | total_timesteps 5515.
Path 498 | total_timesteps 5526.
Path 499 | total_timesteps 5538.
Path 500 | total_timesteps 5550.
Path 501 | total_timesteps 5561.
Path 502 | total_timesteps 5573.
Path 503 | total_timesteps 5580.
Path 504 | total_timesteps 5593.
Path 505 | total_timesteps 5606.
Path 506 | total_timesteps 5618.
Path 507 | total_timesteps 5630.
Path 508 | total_timesteps 5639.
Path 509 | total_timesteps 5649.
Path 510 | total_timesteps 5661.
Path 511 | total_timesteps 5679.
Path 512 | total_timesteps 5691.
Path 513 | total_timesteps 5699.
Path 514 | total_timesteps 5714.
Path 515 | total_timesteps 5722.
Path 516 | total_timesteps 5735.
Path 517 | total_timesteps 5749.
Path 518 | total_timesteps 5758.
Path 519 | total_timesteps 5770.
Path 520 | total_timesteps 5781.
Path 521 | total_timesteps 5793.
Path 522 | total_timesteps 5801.
Path 523 | total_timesteps 5807.
Path 524 | total_timesteps 5815.
Path 525 | total_timesteps 5823.
Path 526 | total_timesteps 5835.
Path 527 | total_timesteps 5845.
Path 528 | total_timesteps 5854.
Path 529 | total_timesteps 5865.
Path 530 | total_timesteps 5876.
Path 531 | total_timesteps 5884.
Path 532 | total_timesteps 5890.
Path 533 | total_timesteps 5900.
Path 534 | total_timesteps 5911.
Path 535 | total_timesteps 5919.
Path 536 | total_timesteps 5931.
Path 537 | total_timesteps 5943.
Path 538 | total_timesteps 5959.
Path 539 | total_timesteps 5971.
Path 540 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.64    |
| Iteration     | 3        |
| MaximumReturn | 0.167    |
| MinimumReturn | -19.8    |
| TotalSamples  | 20025    |
----------------------------
itr #4 | 
Fitting dynamics.
Validation loss = 0.034636180847883224
Validation loss = 0.019263513386249542
Validation loss = 0.020684488117694855
Validation loss = 0.021449986845254898
Validation loss = 0.019314389675855637
Validation loss = 0.01882965676486492
Validation loss = 0.018121475353837013
Validation loss = 0.01738656684756279
Validation loss = 0.021055374294519424
Validation loss = 0.019027169793844223
Validation loss = 0.017659271135926247
Validation loss = 0.016809802502393723
Validation loss = 0.017975781112909317
Validation loss = 0.017297565937042236
Validation loss = 0.018597455695271492
Validation loss = 0.016996268182992935
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 44.
Path 5 | total_timesteps 52.
Path 6 | total_timesteps 66.
Path 7 | total_timesteps 87.
Path 8 | total_timesteps 97.
Path 9 | total_timesteps 109.
Path 10 | total_timesteps 120.
Path 11 | total_timesteps 128.
Path 12 | total_timesteps 136.
Path 13 | total_timesteps 159.
Path 14 | total_timesteps 169.
Path 15 | total_timesteps 181.
Path 16 | total_timesteps 191.
Path 17 | total_timesteps 205.
Path 18 | total_timesteps 216.
Path 19 | total_timesteps 225.
Path 20 | total_timesteps 233.
Path 21 | total_timesteps 245.
Path 22 | total_timesteps 256.
Path 23 | total_timesteps 267.
Path 24 | total_timesteps 277.
Path 25 | total_timesteps 286.
Path 26 | total_timesteps 305.
Path 27 | total_timesteps 315.
Path 28 | total_timesteps 329.
Path 29 | total_timesteps 338.
Path 30 | total_timesteps 345.
Path 31 | total_timesteps 353.
Path 32 | total_timesteps 368.
Path 33 | total_timesteps 377.
Path 34 | total_timesteps 388.
Path 35 | total_timesteps 399.
Path 36 | total_timesteps 412.
Path 37 | total_timesteps 424.
Path 38 | total_timesteps 437.
Path 39 | total_timesteps 444.
Path 40 | total_timesteps 453.
Path 41 | total_timesteps 460.
Path 42 | total_timesteps 469.
Path 43 | total_timesteps 479.
Path 44 | total_timesteps 489.
Path 45 | total_timesteps 497.
Path 46 | total_timesteps 511.
Path 47 | total_timesteps 521.
Path 48 | total_timesteps 531.
Path 49 | total_timesteps 543.
Path 50 | total_timesteps 551.
Path 51 | total_timesteps 562.
Path 52 | total_timesteps 571.
Path 53 | total_timesteps 582.
Path 54 | total_timesteps 594.
Path 55 | total_timesteps 611.
Path 56 | total_timesteps 621.
Path 57 | total_timesteps 630.
Path 58 | total_timesteps 641.
Path 59 | total_timesteps 649.
Path 60 | total_timesteps 660.
Path 61 | total_timesteps 670.
Path 62 | total_timesteps 680.
Path 63 | total_timesteps 687.
Path 64 | total_timesteps 701.
Path 65 | total_timesteps 708.
Path 66 | total_timesteps 718.
Path 67 | total_timesteps 726.
Path 68 | total_timesteps 733.
Path 69 | total_timesteps 740.
Path 70 | total_timesteps 749.
Path 71 | total_timesteps 759.
Path 72 | total_timesteps 771.
Path 73 | total_timesteps 785.
Path 74 | total_timesteps 796.
Path 75 | total_timesteps 811.
Path 76 | total_timesteps 821.
Path 77 | total_timesteps 828.
Path 78 | total_timesteps 837.
Path 79 | total_timesteps 846.
Path 80 | total_timesteps 858.
Path 81 | total_timesteps 880.
Path 82 | total_timesteps 889.
Path 83 | total_timesteps 901.
Path 84 | total_timesteps 909.
Path 85 | total_timesteps 918.
Path 86 | total_timesteps 930.
Path 87 | total_timesteps 942.
Path 88 | total_timesteps 950.
Path 89 | total_timesteps 958.
Path 90 | total_timesteps 972.
Path 91 | total_timesteps 988.
Path 92 | total_timesteps 1006.
Path 93 | total_timesteps 1016.
Path 94 | total_timesteps 1028.
Path 95 | total_timesteps 1037.
Path 96 | total_timesteps 1045.
Path 97 | total_timesteps 1058.
Path 98 | total_timesteps 1070.
Path 99 | total_timesteps 1077.
Path 100 | total_timesteps 1088.
Path 101 | total_timesteps 1101.
Path 102 | total_timesteps 1111.
Path 103 | total_timesteps 1120.
Path 104 | total_timesteps 1137.
Path 105 | total_timesteps 1145.
Path 106 | total_timesteps 1154.
Path 107 | total_timesteps 1163.
Path 108 | total_timesteps 1174.
Path 109 | total_timesteps 1190.
Path 110 | total_timesteps 1198.
Path 111 | total_timesteps 1210.
Path 112 | total_timesteps 1219.
Path 113 | total_timesteps 1229.
Path 114 | total_timesteps 1238.
Path 115 | total_timesteps 1244.
Path 116 | total_timesteps 1251.
Path 117 | total_timesteps 1266.
Path 118 | total_timesteps 1275.
Path 119 | total_timesteps 1286.
Path 120 | total_timesteps 1294.
Path 121 | total_timesteps 1304.
Path 122 | total_timesteps 1312.
Path 123 | total_timesteps 1319.
Path 124 | total_timesteps 1330.
Path 125 | total_timesteps 1339.
Path 126 | total_timesteps 1349.
Path 127 | total_timesteps 1359.
Path 128 | total_timesteps 1371.
Path 129 | total_timesteps 1388.
Path 130 | total_timesteps 1398.
Path 131 | total_timesteps 1409.
Path 132 | total_timesteps 1416.
Path 133 | total_timesteps 1433.
Path 134 | total_timesteps 1441.
Path 135 | total_timesteps 1454.
Path 136 | total_timesteps 1463.
Path 137 | total_timesteps 1472.
Path 138 | total_timesteps 1479.
Path 139 | total_timesteps 1488.
Path 140 | total_timesteps 1499.
Path 141 | total_timesteps 1512.
Path 142 | total_timesteps 1520.
Path 143 | total_timesteps 1531.
Path 144 | total_timesteps 1543.
Path 145 | total_timesteps 1569.
Path 146 | total_timesteps 1578.
Path 147 | total_timesteps 1598.
Path 148 | total_timesteps 1605.
Path 149 | total_timesteps 1616.
Path 150 | total_timesteps 1628.
Path 151 | total_timesteps 1636.
Path 152 | total_timesteps 1647.
Path 153 | total_timesteps 1659.
Path 154 | total_timesteps 1667.
Path 155 | total_timesteps 1674.
Path 156 | total_timesteps 1683.
Path 157 | total_timesteps 1693.
Path 158 | total_timesteps 1703.
Path 159 | total_timesteps 1711.
Path 160 | total_timesteps 1721.
Path 161 | total_timesteps 1730.
Path 162 | total_timesteps 1739.
Path 163 | total_timesteps 1752.
Path 164 | total_timesteps 1767.
Path 165 | total_timesteps 1777.
Path 166 | total_timesteps 1786.
Path 167 | total_timesteps 1799.
Path 168 | total_timesteps 1808.
Path 169 | total_timesteps 1823.
Path 170 | total_timesteps 1832.
Path 171 | total_timesteps 1838.
Path 172 | total_timesteps 1845.
Path 173 | total_timesteps 1856.
Path 174 | total_timesteps 1862.
Path 175 | total_timesteps 1869.
Path 176 | total_timesteps 1883.
Path 177 | total_timesteps 1890.
Path 178 | total_timesteps 1899.
Path 179 | total_timesteps 1911.
Path 180 | total_timesteps 1925.
Path 181 | total_timesteps 1940.
Path 182 | total_timesteps 1949.
Path 183 | total_timesteps 1955.
Path 184 | total_timesteps 1965.
Path 185 | total_timesteps 1971.
Path 186 | total_timesteps 1982.
Path 187 | total_timesteps 1990.
Path 188 | total_timesteps 1998.
Path 189 | total_timesteps 2007.
Path 190 | total_timesteps 2015.
Path 191 | total_timesteps 2022.
Path 192 | total_timesteps 2032.
Path 193 | total_timesteps 2043.
Path 194 | total_timesteps 2056.
Path 195 | total_timesteps 2067.
Path 196 | total_timesteps 2075.
Path 197 | total_timesteps 2084.
Path 198 | total_timesteps 2092.
Path 199 | total_timesteps 2102.
Path 200 | total_timesteps 2110.
Path 201 | total_timesteps 2123.
Path 202 | total_timesteps 2131.
Path 203 | total_timesteps 2140.
Path 204 | total_timesteps 2150.
Path 205 | total_timesteps 2160.
Path 206 | total_timesteps 2173.
Path 207 | total_timesteps 2191.
Path 208 | total_timesteps 2206.
Path 209 | total_timesteps 2214.
Path 210 | total_timesteps 2232.
Path 211 | total_timesteps 2240.
Path 212 | total_timesteps 2252.
Path 213 | total_timesteps 2267.
Path 214 | total_timesteps 2276.
Path 215 | total_timesteps 2290.
Path 216 | total_timesteps 2298.
Path 217 | total_timesteps 2307.
Path 218 | total_timesteps 2320.
Path 219 | total_timesteps 2333.
Path 220 | total_timesteps 2343.
Path 221 | total_timesteps 2356.
Path 222 | total_timesteps 2375.
Path 223 | total_timesteps 2384.
Path 224 | total_timesteps 2399.
Path 225 | total_timesteps 2416.
Path 226 | total_timesteps 2426.
Path 227 | total_timesteps 2436.
Path 228 | total_timesteps 2450.
Path 229 | total_timesteps 2460.
Path 230 | total_timesteps 2470.
Path 231 | total_timesteps 2477.
Path 232 | total_timesteps 2492.
Path 233 | total_timesteps 2502.
Path 234 | total_timesteps 2512.
Path 235 | total_timesteps 2518.
Path 236 | total_timesteps 2526.
Path 237 | total_timesteps 2545.
Path 238 | total_timesteps 2553.
Path 239 | total_timesteps 2563.
Path 240 | total_timesteps 2571.
Path 241 | total_timesteps 2579.
Path 242 | total_timesteps 2592.
Path 243 | total_timesteps 2605.
Path 244 | total_timesteps 2616.
Path 245 | total_timesteps 2628.
Path 246 | total_timesteps 2639.
Path 247 | total_timesteps 2651.
Path 248 | total_timesteps 2667.
Path 249 | total_timesteps 2675.
Path 250 | total_timesteps 2684.
Path 251 | total_timesteps 2691.
Path 252 | total_timesteps 2699.
Path 253 | total_timesteps 2712.
Path 254 | total_timesteps 2722.
Path 255 | total_timesteps 2728.
Path 256 | total_timesteps 2736.
Path 257 | total_timesteps 2744.
Path 258 | total_timesteps 2751.
Path 259 | total_timesteps 2759.
Path 260 | total_timesteps 2768.
Path 261 | total_timesteps 2780.
Path 262 | total_timesteps 2791.
Path 263 | total_timesteps 2800.
Path 264 | total_timesteps 2810.
Path 265 | total_timesteps 2819.
Path 266 | total_timesteps 2826.
Path 267 | total_timesteps 2836.
Path 268 | total_timesteps 2845.
Path 269 | total_timesteps 2854.
Path 270 | total_timesteps 2868.
Path 271 | total_timesteps 2875.
Path 272 | total_timesteps 2884.
Path 273 | total_timesteps 2896.
Path 274 | total_timesteps 2908.
Path 275 | total_timesteps 2914.
Path 276 | total_timesteps 2925.
Path 277 | total_timesteps 2935.
Path 278 | total_timesteps 2951.
Path 279 | total_timesteps 2968.
Path 280 | total_timesteps 2978.
Path 281 | total_timesteps 2985.
Path 282 | total_timesteps 2991.
Path 283 | total_timesteps 2999.
Path 284 | total_timesteps 3008.
Path 285 | total_timesteps 3017.
Path 286 | total_timesteps 3026.
Path 287 | total_timesteps 3039.
Path 288 | total_timesteps 3047.
Path 289 | total_timesteps 3055.
Path 290 | total_timesteps 3070.
Path 291 | total_timesteps 3082.
Path 292 | total_timesteps 3102.
Path 293 | total_timesteps 3112.
Path 294 | total_timesteps 3130.
Path 295 | total_timesteps 3147.
Path 296 | total_timesteps 3160.
Path 297 | total_timesteps 3168.
Path 298 | total_timesteps 3184.
Path 299 | total_timesteps 3193.
Path 300 | total_timesteps 3203.
Path 301 | total_timesteps 3211.
Path 302 | total_timesteps 3220.
Path 303 | total_timesteps 3228.
Path 304 | total_timesteps 3239.
Path 305 | total_timesteps 3248.
Path 306 | total_timesteps 3261.
Path 307 | total_timesteps 3270.
Path 308 | total_timesteps 3278.
Path 309 | total_timesteps 3287.
Path 310 | total_timesteps 3298.
Path 311 | total_timesteps 3316.
Path 312 | total_timesteps 3327.
Path 313 | total_timesteps 3335.
Path 314 | total_timesteps 3344.
Path 315 | total_timesteps 3355.
Path 316 | total_timesteps 3367.
Path 317 | total_timesteps 3375.
Path 318 | total_timesteps 3383.
Path 319 | total_timesteps 3393.
Path 320 | total_timesteps 3408.
Path 321 | total_timesteps 3418.
Path 322 | total_timesteps 3431.
Path 323 | total_timesteps 3443.
Path 324 | total_timesteps 3457.
Path 325 | total_timesteps 3467.
Path 326 | total_timesteps 3475.
Path 327 | total_timesteps 3488.
Path 328 | total_timesteps 3496.
Path 329 | total_timesteps 3506.
Path 330 | total_timesteps 3515.
Path 331 | total_timesteps 3524.
Path 332 | total_timesteps 3532.
Path 333 | total_timesteps 3540.
Path 334 | total_timesteps 3548.
Path 335 | total_timesteps 3557.
Path 336 | total_timesteps 3570.
Path 337 | total_timesteps 3584.
Path 338 | total_timesteps 3593.
Path 339 | total_timesteps 3615.
Path 340 | total_timesteps 3623.
Path 341 | total_timesteps 3631.
Path 342 | total_timesteps 3639.
Path 343 | total_timesteps 3648.
Path 344 | total_timesteps 3661.
Path 345 | total_timesteps 3673.
Path 346 | total_timesteps 3682.
Path 347 | total_timesteps 3690.
Path 348 | total_timesteps 3699.
Path 349 | total_timesteps 3710.
Path 350 | total_timesteps 3722.
Path 351 | total_timesteps 3738.
Path 352 | total_timesteps 3751.
Path 353 | total_timesteps 3758.
Path 354 | total_timesteps 3770.
Path 355 | total_timesteps 3782.
Path 356 | total_timesteps 3789.
Path 357 | total_timesteps 3797.
Path 358 | total_timesteps 3804.
Path 359 | total_timesteps 3814.
Path 360 | total_timesteps 3827.
Path 361 | total_timesteps 3836.
Path 362 | total_timesteps 3845.
Path 363 | total_timesteps 3856.
Path 364 | total_timesteps 3867.
Path 365 | total_timesteps 3876.
Path 366 | total_timesteps 3885.
Path 367 | total_timesteps 3899.
Path 368 | total_timesteps 3907.
Path 369 | total_timesteps 3921.
Path 370 | total_timesteps 3931.
Path 371 | total_timesteps 3945.
Path 372 | total_timesteps 3955.
Path 373 | total_timesteps 3964.
Path 374 | total_timesteps 3973.
Path 375 | total_timesteps 3981.
Path 376 | total_timesteps 3991.
Path 377 | total_timesteps 4008.
Path 378 | total_timesteps 4021.
Path 379 | total_timesteps 4030.
Path 380 | total_timesteps 4042.
Path 381 | total_timesteps 4050.
Path 382 | total_timesteps 4067.
Path 383 | total_timesteps 4081.
Path 384 | total_timesteps 4091.
Path 385 | total_timesteps 4097.
Path 386 | total_timesteps 4106.
Path 387 | total_timesteps 4121.
Path 388 | total_timesteps 4133.
Path 389 | total_timesteps 4141.
Path 390 | total_timesteps 4151.
Path 391 | total_timesteps 4158.
Path 392 | total_timesteps 4165.
Path 393 | total_timesteps 4173.
Path 394 | total_timesteps 4183.
Path 395 | total_timesteps 4194.
Path 396 | total_timesteps 4200.
Path 397 | total_timesteps 4214.
Path 398 | total_timesteps 4224.
Path 399 | total_timesteps 4241.
Path 400 | total_timesteps 4249.
Path 401 | total_timesteps 4264.
Path 402 | total_timesteps 4283.
Path 403 | total_timesteps 4290.
Path 404 | total_timesteps 4300.
Path 405 | total_timesteps 4307.
Path 406 | total_timesteps 4314.
Path 407 | total_timesteps 4324.
Path 408 | total_timesteps 4332.
Path 409 | total_timesteps 4340.
Path 410 | total_timesteps 4348.
Path 411 | total_timesteps 4359.
Path 412 | total_timesteps 4372.
Path 413 | total_timesteps 4388.
Path 414 | total_timesteps 4402.
Path 415 | total_timesteps 4413.
Path 416 | total_timesteps 4427.
Path 417 | total_timesteps 4438.
Path 418 | total_timesteps 4453.
Path 419 | total_timesteps 4465.
Path 420 | total_timesteps 4477.
Path 421 | total_timesteps 4483.
Path 422 | total_timesteps 4493.
Path 423 | total_timesteps 4502.
Path 424 | total_timesteps 4510.
Path 425 | total_timesteps 4518.
Path 426 | total_timesteps 4529.
Path 427 | total_timesteps 4544.
Path 428 | total_timesteps 4553.
Path 429 | total_timesteps 4572.
Path 430 | total_timesteps 4580.
Path 431 | total_timesteps 4587.
Path 432 | total_timesteps 4594.
Path 433 | total_timesteps 4601.
Path 434 | total_timesteps 4610.
Path 435 | total_timesteps 4627.
Path 436 | total_timesteps 4634.
Path 437 | total_timesteps 4643.
Path 438 | total_timesteps 4657.
Path 439 | total_timesteps 4677.
Path 440 | total_timesteps 4690.
Path 441 | total_timesteps 4704.
Path 442 | total_timesteps 4713.
Path 443 | total_timesteps 4721.
Path 444 | total_timesteps 4729.
Path 445 | total_timesteps 4735.
Path 446 | total_timesteps 4745.
Path 447 | total_timesteps 4762.
Path 448 | total_timesteps 4771.
Path 449 | total_timesteps 4781.
Path 450 | total_timesteps 4789.
Path 451 | total_timesteps 4806.
Path 452 | total_timesteps 4814.
Path 453 | total_timesteps 4828.
Path 454 | total_timesteps 4840.
Path 455 | total_timesteps 4849.
Path 456 | total_timesteps 4861.
Path 457 | total_timesteps 4869.
Path 458 | total_timesteps 4880.
Path 459 | total_timesteps 4889.
Path 460 | total_timesteps 4900.
Path 461 | total_timesteps 4908.
Path 462 | total_timesteps 4916.
Path 463 | total_timesteps 4924.
Path 464 | total_timesteps 4935.
Path 465 | total_timesteps 4943.
Path 466 | total_timesteps 4950.
Path 467 | total_timesteps 4960.
Path 468 | total_timesteps 4974.
Path 469 | total_timesteps 4983.
Path 470 | total_timesteps 4991.
Path 471 | total_timesteps 5003.
Path 472 | total_timesteps 5013.
Path 473 | total_timesteps 5021.
Path 474 | total_timesteps 5028.
Path 475 | total_timesteps 5037.
Path 476 | total_timesteps 5045.
Path 477 | total_timesteps 5052.
Path 478 | total_timesteps 5061.
Path 479 | total_timesteps 5072.
Path 480 | total_timesteps 5087.
Path 481 | total_timesteps 5099.
Path 482 | total_timesteps 5109.
Path 483 | total_timesteps 5116.
Path 484 | total_timesteps 5124.
Path 485 | total_timesteps 5132.
Path 486 | total_timesteps 5139.
Path 487 | total_timesteps 5150.
Path 488 | total_timesteps 5169.
Path 489 | total_timesteps 5182.
Path 490 | total_timesteps 5193.
Path 491 | total_timesteps 5203.
Path 492 | total_timesteps 5212.
Path 493 | total_timesteps 5220.
Path 494 | total_timesteps 5230.
Path 495 | total_timesteps 5240.
Path 496 | total_timesteps 5251.
Path 497 | total_timesteps 5264.
Path 498 | total_timesteps 5276.
Path 499 | total_timesteps 5286.
Path 500 | total_timesteps 5298.
Path 501 | total_timesteps 5311.
Path 502 | total_timesteps 5323.
Path 503 | total_timesteps 5334.
Path 504 | total_timesteps 5348.
Path 505 | total_timesteps 5359.
Path 506 | total_timesteps 5371.
Path 507 | total_timesteps 5388.
Path 508 | total_timesteps 5405.
Path 509 | total_timesteps 5418.
Path 510 | total_timesteps 5433.
Path 511 | total_timesteps 5441.
Path 512 | total_timesteps 5448.
Path 513 | total_timesteps 5467.
Path 514 | total_timesteps 5477.
Path 515 | total_timesteps 5485.
Path 516 | total_timesteps 5495.
Path 517 | total_timesteps 5510.
Path 518 | total_timesteps 5522.
Path 519 | total_timesteps 5532.
Path 520 | total_timesteps 5543.
Path 521 | total_timesteps 5550.
Path 522 | total_timesteps 5562.
Path 523 | total_timesteps 5575.
Path 524 | total_timesteps 5587.
Path 525 | total_timesteps 5593.
Path 526 | total_timesteps 5603.
Path 527 | total_timesteps 5613.
Path 528 | total_timesteps 5620.
Path 529 | total_timesteps 5635.
Path 530 | total_timesteps 5643.
Path 531 | total_timesteps 5659.
Path 532 | total_timesteps 5671.
Path 533 | total_timesteps 5687.
Path 534 | total_timesteps 5695.
Path 535 | total_timesteps 5707.
Path 536 | total_timesteps 5719.
Path 537 | total_timesteps 5734.
Path 538 | total_timesteps 5742.
Path 539 | total_timesteps 5755.
Path 540 | total_timesteps 5763.
Path 541 | total_timesteps 5777.
Path 542 | total_timesteps 5790.
Path 543 | total_timesteps 5798.
Path 544 | total_timesteps 5808.
Path 545 | total_timesteps 5816.
Path 546 | total_timesteps 5825.
Path 547 | total_timesteps 5835.
Path 548 | total_timesteps 5846.
Path 549 | total_timesteps 5855.
Path 550 | total_timesteps 5868.
Path 551 | total_timesteps 5876.
Path 552 | total_timesteps 5884.
Path 553 | total_timesteps 5895.
Path 554 | total_timesteps 5905.
Path 555 | total_timesteps 5922.
Path 556 | total_timesteps 5942.
Path 557 | total_timesteps 5956.
Path 558 | total_timesteps 5965.
Path 559 | total_timesteps 5974.
Path 560 | total_timesteps 5984.
Path 561 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.25    |
| Iteration     | 4        |
| MaximumReturn | 3.12     |
| MinimumReturn | -23.2    |
| TotalSamples  | 24025    |
----------------------------
itr #5 | 
Fitting dynamics.
Validation loss = 0.01677650213241577
Validation loss = 0.015312422066926956
Validation loss = 0.018660273402929306
Validation loss = 0.01678393967449665
Validation loss = 0.01520098838955164
Validation loss = 0.01554124802350998
Validation loss = 0.015157990157604218
Validation loss = 0.016442082822322845
Validation loss = 0.015552918426692486
Validation loss = 0.014443453401327133
Validation loss = 0.017872465774416924
Validation loss = 0.013736906461417675
Validation loss = 0.013722800649702549
Validation loss = 0.014023476280272007
Validation loss = 0.014698445796966553
Validation loss = 0.014875766821205616
Validation loss = 0.013658841140568256
Validation loss = 0.016637230291962624
Validation loss = 0.013664290308952332
Validation loss = 0.013801218010485172
Validation loss = 0.015295824967324734
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 29.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 57.
Path 6 | total_timesteps 74.
Path 7 | total_timesteps 80.
Path 8 | total_timesteps 98.
Path 9 | total_timesteps 107.
Path 10 | total_timesteps 114.
Path 11 | total_timesteps 127.
Path 12 | total_timesteps 140.
Path 13 | total_timesteps 148.
Path 14 | total_timesteps 162.
Path 15 | total_timesteps 175.
Path 16 | total_timesteps 186.
Path 17 | total_timesteps 198.
Path 18 | total_timesteps 207.
Path 19 | total_timesteps 214.
Path 20 | total_timesteps 226.
Path 21 | total_timesteps 241.
Path 22 | total_timesteps 251.
Path 23 | total_timesteps 261.
Path 24 | total_timesteps 272.
Path 25 | total_timesteps 281.
Path 26 | total_timesteps 292.
Path 27 | total_timesteps 299.
Path 28 | total_timesteps 312.
Path 29 | total_timesteps 321.
Path 30 | total_timesteps 328.
Path 31 | total_timesteps 336.
Path 32 | total_timesteps 350.
Path 33 | total_timesteps 360.
Path 34 | total_timesteps 373.
Path 35 | total_timesteps 384.
Path 36 | total_timesteps 403.
Path 37 | total_timesteps 413.
Path 38 | total_timesteps 429.
Path 39 | total_timesteps 439.
Path 40 | total_timesteps 449.
Path 41 | total_timesteps 465.
Path 42 | total_timesteps 476.
Path 43 | total_timesteps 485.
Path 44 | total_timesteps 496.
Path 45 | total_timesteps 504.
Path 46 | total_timesteps 515.
Path 47 | total_timesteps 522.
Path 48 | total_timesteps 537.
Path 49 | total_timesteps 548.
Path 50 | total_timesteps 556.
Path 51 | total_timesteps 563.
Path 52 | total_timesteps 572.
Path 53 | total_timesteps 582.
Path 54 | total_timesteps 591.
Path 55 | total_timesteps 600.
Path 56 | total_timesteps 624.
Path 57 | total_timesteps 633.
Path 58 | total_timesteps 642.
Path 59 | total_timesteps 650.
Path 60 | total_timesteps 657.
Path 61 | total_timesteps 667.
Path 62 | total_timesteps 681.
Path 63 | total_timesteps 691.
Path 64 | total_timesteps 701.
Path 65 | total_timesteps 716.
Path 66 | total_timesteps 723.
Path 67 | total_timesteps 729.
Path 68 | total_timesteps 743.
Path 69 | total_timesteps 756.
Path 70 | total_timesteps 766.
Path 71 | total_timesteps 776.
Path 72 | total_timesteps 786.
Path 73 | total_timesteps 797.
Path 74 | total_timesteps 805.
Path 75 | total_timesteps 815.
Path 76 | total_timesteps 826.
Path 77 | total_timesteps 842.
Path 78 | total_timesteps 853.
Path 79 | total_timesteps 861.
Path 80 | total_timesteps 870.
Path 81 | total_timesteps 878.
Path 82 | total_timesteps 887.
Path 83 | total_timesteps 899.
Path 84 | total_timesteps 909.
Path 85 | total_timesteps 919.
Path 86 | total_timesteps 931.
Path 87 | total_timesteps 940.
Path 88 | total_timesteps 958.
Path 89 | total_timesteps 972.
Path 90 | total_timesteps 984.
Path 91 | total_timesteps 994.
Path 92 | total_timesteps 1003.
Path 93 | total_timesteps 1010.
Path 94 | total_timesteps 1017.
Path 95 | total_timesteps 1031.
Path 96 | total_timesteps 1044.
Path 97 | total_timesteps 1052.
Path 98 | total_timesteps 1065.
Path 99 | total_timesteps 1078.
Path 100 | total_timesteps 1092.
Path 101 | total_timesteps 1099.
Path 102 | total_timesteps 1111.
Path 103 | total_timesteps 1121.
Path 104 | total_timesteps 1133.
Path 105 | total_timesteps 1145.
Path 106 | total_timesteps 1162.
Path 107 | total_timesteps 1171.
Path 108 | total_timesteps 1184.
Path 109 | total_timesteps 1194.
Path 110 | total_timesteps 1205.
Path 111 | total_timesteps 1212.
Path 112 | total_timesteps 1230.
Path 113 | total_timesteps 1244.
Path 114 | total_timesteps 1251.
Path 115 | total_timesteps 1258.
Path 116 | total_timesteps 1268.
Path 117 | total_timesteps 1276.
Path 118 | total_timesteps 1285.
Path 119 | total_timesteps 1296.
Path 120 | total_timesteps 1303.
Path 121 | total_timesteps 1311.
Path 122 | total_timesteps 1319.
Path 123 | total_timesteps 1332.
Path 124 | total_timesteps 1341.
Path 125 | total_timesteps 1355.
Path 126 | total_timesteps 1369.
Path 127 | total_timesteps 1378.
Path 128 | total_timesteps 1390.
Path 129 | total_timesteps 1396.
Path 130 | total_timesteps 1407.
Path 131 | total_timesteps 1421.
Path 132 | total_timesteps 1433.
Path 133 | total_timesteps 1447.
Path 134 | total_timesteps 1463.
Path 135 | total_timesteps 1472.
Path 136 | total_timesteps 1483.
Path 137 | total_timesteps 1490.
Path 138 | total_timesteps 1502.
Path 139 | total_timesteps 1515.
Path 140 | total_timesteps 1524.
Path 141 | total_timesteps 1536.
Path 142 | total_timesteps 1545.
Path 143 | total_timesteps 1560.
Path 144 | total_timesteps 1568.
Path 145 | total_timesteps 1577.
Path 146 | total_timesteps 1591.
Path 147 | total_timesteps 1601.
Path 148 | total_timesteps 1615.
Path 149 | total_timesteps 1628.
Path 150 | total_timesteps 1659.
Path 151 | total_timesteps 1668.
Path 152 | total_timesteps 1691.
Path 153 | total_timesteps 1704.
Path 154 | total_timesteps 1712.
Path 155 | total_timesteps 1718.
Path 156 | total_timesteps 1729.
Path 157 | total_timesteps 1743.
Path 158 | total_timesteps 1752.
Path 159 | total_timesteps 1764.
Path 160 | total_timesteps 1783.
Path 161 | total_timesteps 1803.
Path 162 | total_timesteps 1814.
Path 163 | total_timesteps 1823.
Path 164 | total_timesteps 1839.
Path 165 | total_timesteps 1849.
Path 166 | total_timesteps 1858.
Path 167 | total_timesteps 1869.
Path 168 | total_timesteps 1878.
Path 169 | total_timesteps 1888.
Path 170 | total_timesteps 1899.
Path 171 | total_timesteps 1911.
Path 172 | total_timesteps 1936.
Path 173 | total_timesteps 1944.
Path 174 | total_timesteps 1951.
Path 175 | total_timesteps 1983.
Path 176 | total_timesteps 1992.
Path 177 | total_timesteps 2000.
Path 178 | total_timesteps 2012.
Path 179 | total_timesteps 2025.
Path 180 | total_timesteps 2036.
Path 181 | total_timesteps 2050.
Path 182 | total_timesteps 2059.
Path 183 | total_timesteps 2076.
Path 184 | total_timesteps 2086.
Path 185 | total_timesteps 2095.
Path 186 | total_timesteps 2106.
Path 187 | total_timesteps 2123.
Path 188 | total_timesteps 2130.
Path 189 | total_timesteps 2139.
Path 190 | total_timesteps 2148.
Path 191 | total_timesteps 2156.
Path 192 | total_timesteps 2170.
Path 193 | total_timesteps 2180.
Path 194 | total_timesteps 2190.
Path 195 | total_timesteps 2202.
Path 196 | total_timesteps 2212.
Path 197 | total_timesteps 2233.
Path 198 | total_timesteps 2245.
Path 199 | total_timesteps 2254.
Path 200 | total_timesteps 2264.
Path 201 | total_timesteps 2275.
Path 202 | total_timesteps 2286.
Path 203 | total_timesteps 2304.
Path 204 | total_timesteps 2312.
Path 205 | total_timesteps 2322.
Path 206 | total_timesteps 2344.
Path 207 | total_timesteps 2354.
Path 208 | total_timesteps 2365.
Path 209 | total_timesteps 2383.
Path 210 | total_timesteps 2391.
Path 211 | total_timesteps 2406.
Path 212 | total_timesteps 2419.
Path 213 | total_timesteps 2428.
Path 214 | total_timesteps 2440.
Path 215 | total_timesteps 2447.
Path 216 | total_timesteps 2461.
Path 217 | total_timesteps 2469.
Path 218 | total_timesteps 2482.
Path 219 | total_timesteps 2499.
Path 220 | total_timesteps 2512.
Path 221 | total_timesteps 2523.
Path 222 | total_timesteps 2533.
Path 223 | total_timesteps 2547.
Path 224 | total_timesteps 2555.
Path 225 | total_timesteps 2566.
Path 226 | total_timesteps 2575.
Path 227 | total_timesteps 2582.
Path 228 | total_timesteps 2593.
Path 229 | total_timesteps 2601.
Path 230 | total_timesteps 2611.
Path 231 | total_timesteps 2623.
Path 232 | total_timesteps 2633.
Path 233 | total_timesteps 2643.
Path 234 | total_timesteps 2657.
Path 235 | total_timesteps 2666.
Path 236 | total_timesteps 2678.
Path 237 | total_timesteps 2688.
Path 238 | total_timesteps 2699.
Path 239 | total_timesteps 2715.
Path 240 | total_timesteps 2722.
Path 241 | total_timesteps 2730.
Path 242 | total_timesteps 2738.
Path 243 | total_timesteps 2746.
Path 244 | total_timesteps 2759.
Path 245 | total_timesteps 2765.
Path 246 | total_timesteps 2778.
Path 247 | total_timesteps 2789.
Path 248 | total_timesteps 2809.
Path 249 | total_timesteps 2824.
Path 250 | total_timesteps 2834.
Path 251 | total_timesteps 2841.
Path 252 | total_timesteps 2852.
Path 253 | total_timesteps 2861.
Path 254 | total_timesteps 2873.
Path 255 | total_timesteps 2886.
Path 256 | total_timesteps 2904.
Path 257 | total_timesteps 2917.
Path 258 | total_timesteps 2924.
Path 259 | total_timesteps 2939.
Path 260 | total_timesteps 2950.
Path 261 | total_timesteps 2959.
Path 262 | total_timesteps 2972.
Path 263 | total_timesteps 2984.
Path 264 | total_timesteps 2990.
Path 265 | total_timesteps 3015.
Path 266 | total_timesteps 3024.
Path 267 | total_timesteps 3032.
Path 268 | total_timesteps 3041.
Path 269 | total_timesteps 3053.
Path 270 | total_timesteps 3061.
Path 271 | total_timesteps 3069.
Path 272 | total_timesteps 3080.
Path 273 | total_timesteps 3092.
Path 274 | total_timesteps 3109.
Path 275 | total_timesteps 3124.
Path 276 | total_timesteps 3133.
Path 277 | total_timesteps 3142.
Path 278 | total_timesteps 3155.
Path 279 | total_timesteps 3163.
Path 280 | total_timesteps 3176.
Path 281 | total_timesteps 3188.
Path 282 | total_timesteps 3200.
Path 283 | total_timesteps 3210.
Path 284 | total_timesteps 3218.
Path 285 | total_timesteps 3228.
Path 286 | total_timesteps 3238.
Path 287 | total_timesteps 3250.
Path 288 | total_timesteps 3264.
Path 289 | total_timesteps 3273.
Path 290 | total_timesteps 3280.
Path 291 | total_timesteps 3294.
Path 292 | total_timesteps 3302.
Path 293 | total_timesteps 3309.
Path 294 | total_timesteps 3322.
Path 295 | total_timesteps 3329.
Path 296 | total_timesteps 3340.
Path 297 | total_timesteps 3348.
Path 298 | total_timesteps 3356.
Path 299 | total_timesteps 3367.
Path 300 | total_timesteps 3378.
Path 301 | total_timesteps 3387.
Path 302 | total_timesteps 3400.
Path 303 | total_timesteps 3413.
Path 304 | total_timesteps 3423.
Path 305 | total_timesteps 3432.
Path 306 | total_timesteps 3441.
Path 307 | total_timesteps 3452.
Path 308 | total_timesteps 3462.
Path 309 | total_timesteps 3470.
Path 310 | total_timesteps 3482.
Path 311 | total_timesteps 3489.
Path 312 | total_timesteps 3506.
Path 313 | total_timesteps 3521.
Path 314 | total_timesteps 3531.
Path 315 | total_timesteps 3559.
Path 316 | total_timesteps 3569.
Path 317 | total_timesteps 3579.
Path 318 | total_timesteps 3591.
Path 319 | total_timesteps 3602.
Path 320 | total_timesteps 3616.
Path 321 | total_timesteps 3627.
Path 322 | total_timesteps 3641.
Path 323 | total_timesteps 3650.
Path 324 | total_timesteps 3661.
Path 325 | total_timesteps 3671.
Path 326 | total_timesteps 3690.
Path 327 | total_timesteps 3706.
Path 328 | total_timesteps 3713.
Path 329 | total_timesteps 3721.
Path 330 | total_timesteps 3731.
Path 331 | total_timesteps 3741.
Path 332 | total_timesteps 3749.
Path 333 | total_timesteps 3761.
Path 334 | total_timesteps 3780.
Path 335 | total_timesteps 3791.
Path 336 | total_timesteps 3799.
Path 337 | total_timesteps 3809.
Path 338 | total_timesteps 3821.
Path 339 | total_timesteps 3833.
Path 340 | total_timesteps 3842.
Path 341 | total_timesteps 3853.
Path 342 | total_timesteps 3862.
Path 343 | total_timesteps 3879.
Path 344 | total_timesteps 3891.
Path 345 | total_timesteps 3904.
Path 346 | total_timesteps 3914.
Path 347 | total_timesteps 3924.
Path 348 | total_timesteps 3939.
Path 349 | total_timesteps 3948.
Path 350 | total_timesteps 3955.
Path 351 | total_timesteps 3970.
Path 352 | total_timesteps 3980.
Path 353 | total_timesteps 3986.
Path 354 | total_timesteps 3995.
Path 355 | total_timesteps 4005.
Path 356 | total_timesteps 4012.
Path 357 | total_timesteps 4019.
Path 358 | total_timesteps 4027.
Path 359 | total_timesteps 4037.
Path 360 | total_timesteps 4052.
Path 361 | total_timesteps 4064.
Path 362 | total_timesteps 4071.
Path 363 | total_timesteps 4079.
Path 364 | total_timesteps 4091.
Path 365 | total_timesteps 4102.
Path 366 | total_timesteps 4112.
Path 367 | total_timesteps 4124.
Path 368 | total_timesteps 4137.
Path 369 | total_timesteps 4153.
Path 370 | total_timesteps 4165.
Path 371 | total_timesteps 4176.
Path 372 | total_timesteps 4186.
Path 373 | total_timesteps 4200.
Path 374 | total_timesteps 4212.
Path 375 | total_timesteps 4223.
Path 376 | total_timesteps 4232.
Path 377 | total_timesteps 4240.
Path 378 | total_timesteps 4247.
Path 379 | total_timesteps 4259.
Path 380 | total_timesteps 4280.
Path 381 | total_timesteps 4289.
Path 382 | total_timesteps 4297.
Path 383 | total_timesteps 4305.
Path 384 | total_timesteps 4313.
Path 385 | total_timesteps 4320.
Path 386 | total_timesteps 4332.
Path 387 | total_timesteps 4344.
Path 388 | total_timesteps 4355.
Path 389 | total_timesteps 4363.
Path 390 | total_timesteps 4376.
Path 391 | total_timesteps 4385.
Path 392 | total_timesteps 4393.
Path 393 | total_timesteps 4401.
Path 394 | total_timesteps 4409.
Path 395 | total_timesteps 4418.
Path 396 | total_timesteps 4427.
Path 397 | total_timesteps 4440.
Path 398 | total_timesteps 4453.
Path 399 | total_timesteps 4465.
Path 400 | total_timesteps 4476.
Path 401 | total_timesteps 4487.
Path 402 | total_timesteps 4499.
Path 403 | total_timesteps 4507.
Path 404 | total_timesteps 4518.
Path 405 | total_timesteps 4525.
Path 406 | total_timesteps 4533.
Path 407 | total_timesteps 4544.
Path 408 | total_timesteps 4554.
Path 409 | total_timesteps 4563.
Path 410 | total_timesteps 4577.
Path 411 | total_timesteps 4584.
Path 412 | total_timesteps 4591.
Path 413 | total_timesteps 4600.
Path 414 | total_timesteps 4611.
Path 415 | total_timesteps 4624.
Path 416 | total_timesteps 4634.
Path 417 | total_timesteps 4646.
Path 418 | total_timesteps 4657.
Path 419 | total_timesteps 4667.
Path 420 | total_timesteps 4677.
Path 421 | total_timesteps 4686.
Path 422 | total_timesteps 4693.
Path 423 | total_timesteps 4702.
Path 424 | total_timesteps 4710.
Path 425 | total_timesteps 4717.
Path 426 | total_timesteps 4731.
Path 427 | total_timesteps 4737.
Path 428 | total_timesteps 4750.
Path 429 | total_timesteps 4764.
Path 430 | total_timesteps 4779.
Path 431 | total_timesteps 4796.
Path 432 | total_timesteps 4803.
Path 433 | total_timesteps 4812.
Path 434 | total_timesteps 4824.
Path 435 | total_timesteps 4842.
Path 436 | total_timesteps 4852.
Path 437 | total_timesteps 4862.
Path 438 | total_timesteps 4872.
Path 439 | total_timesteps 4885.
Path 440 | total_timesteps 4899.
Path 441 | total_timesteps 4909.
Path 442 | total_timesteps 4916.
Path 443 | total_timesteps 4925.
Path 444 | total_timesteps 4945.
Path 445 | total_timesteps 4954.
Path 446 | total_timesteps 4961.
Path 447 | total_timesteps 4970.
Path 448 | total_timesteps 4981.
Path 449 | total_timesteps 4992.
Path 450 | total_timesteps 5002.
Path 451 | total_timesteps 5014.
Path 452 | total_timesteps 5026.
Path 453 | total_timesteps 5038.
Path 454 | total_timesteps 5047.
Path 455 | total_timesteps 5058.
Path 456 | total_timesteps 5065.
Path 457 | total_timesteps 5080.
Path 458 | total_timesteps 5090.
Path 459 | total_timesteps 5101.
Path 460 | total_timesteps 5117.
Path 461 | total_timesteps 5134.
Path 462 | total_timesteps 5145.
Path 463 | total_timesteps 5158.
Path 464 | total_timesteps 5166.
Path 465 | total_timesteps 5174.
Path 466 | total_timesteps 5183.
Path 467 | total_timesteps 5194.
Path 468 | total_timesteps 5216.
Path 469 | total_timesteps 5224.
Path 470 | total_timesteps 5232.
Path 471 | total_timesteps 5244.
Path 472 | total_timesteps 5256.
Path 473 | total_timesteps 5265.
Path 474 | total_timesteps 5286.
Path 475 | total_timesteps 5295.
Path 476 | total_timesteps 5302.
Path 477 | total_timesteps 5309.
Path 478 | total_timesteps 5321.
Path 479 | total_timesteps 5333.
Path 480 | total_timesteps 5343.
Path 481 | total_timesteps 5353.
Path 482 | total_timesteps 5362.
Path 483 | total_timesteps 5370.
Path 484 | total_timesteps 5380.
Path 485 | total_timesteps 5389.
Path 486 | total_timesteps 5399.
Path 487 | total_timesteps 5411.
Path 488 | total_timesteps 5423.
Path 489 | total_timesteps 5433.
Path 490 | total_timesteps 5441.
Path 491 | total_timesteps 5452.
Path 492 | total_timesteps 5460.
Path 493 | total_timesteps 5478.
Path 494 | total_timesteps 5496.
Path 495 | total_timesteps 5505.
Path 496 | total_timesteps 5515.
Path 497 | total_timesteps 5523.
Path 498 | total_timesteps 5531.
Path 499 | total_timesteps 5538.
Path 500 | total_timesteps 5549.
Path 501 | total_timesteps 5556.
Path 502 | total_timesteps 5566.
Path 503 | total_timesteps 5579.
Path 504 | total_timesteps 5588.
Path 505 | total_timesteps 5605.
Path 506 | total_timesteps 5615.
Path 507 | total_timesteps 5623.
Path 508 | total_timesteps 5633.
Path 509 | total_timesteps 5640.
Path 510 | total_timesteps 5660.
Path 511 | total_timesteps 5667.
Path 512 | total_timesteps 5681.
Path 513 | total_timesteps 5689.
Path 514 | total_timesteps 5705.
Path 515 | total_timesteps 5718.
Path 516 | total_timesteps 5731.
Path 517 | total_timesteps 5738.
Path 518 | total_timesteps 5746.
Path 519 | total_timesteps 5757.
Path 520 | total_timesteps 5764.
Path 521 | total_timesteps 5778.
Path 522 | total_timesteps 5789.
Path 523 | total_timesteps 5796.
Path 524 | total_timesteps 5805.
Path 525 | total_timesteps 5823.
Path 526 | total_timesteps 5839.
Path 527 | total_timesteps 5863.
Path 528 | total_timesteps 5874.
Path 529 | total_timesteps 5886.
Path 530 | total_timesteps 5908.
Path 531 | total_timesteps 5917.
Path 532 | total_timesteps 5933.
Path 533 | total_timesteps 5944.
Path 534 | total_timesteps 5961.
Path 535 | total_timesteps 5968.
Path 536 | total_timesteps 5978.
Path 537 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.77    |
| Iteration     | 5        |
| MaximumReturn | 0.67     |
| MinimumReturn | -23.5    |
| TotalSamples  | 28028    |
----------------------------
itr #6 | 
Fitting dynamics.
Validation loss = 0.015268410556018353
Validation loss = 0.01357400231063366
Validation loss = 0.013032048009335995
Validation loss = 0.013777029700577259
Validation loss = 0.012889995239675045
Validation loss = 0.013678382150828838
Validation loss = 0.01323731243610382
Validation loss = 0.012307317927479744
Validation loss = 0.012831328436732292
Validation loss = 0.016719335690140724
Validation loss = 0.01199749018996954
Validation loss = 0.01335512287914753
Validation loss = 0.014526695013046265
Validation loss = 0.012808169238269329
Validation loss = 0.014009167440235615
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 38.
Path 5 | total_timesteps 50.
Path 6 | total_timesteps 58.
Path 7 | total_timesteps 66.
Path 8 | total_timesteps 76.
Path 9 | total_timesteps 89.
Path 10 | total_timesteps 101.
Path 11 | total_timesteps 109.
Path 12 | total_timesteps 119.
Path 13 | total_timesteps 127.
Path 14 | total_timesteps 135.
Path 15 | total_timesteps 145.
Path 16 | total_timesteps 165.
Path 17 | total_timesteps 186.
Path 18 | total_timesteps 193.
Path 19 | total_timesteps 204.
Path 20 | total_timesteps 211.
Path 21 | total_timesteps 224.
Path 22 | total_timesteps 232.
Path 23 | total_timesteps 242.
Path 24 | total_timesteps 249.
Path 25 | total_timesteps 256.
Path 26 | total_timesteps 268.
Path 27 | total_timesteps 277.
Path 28 | total_timesteps 288.
Path 29 | total_timesteps 295.
Path 30 | total_timesteps 314.
Path 31 | total_timesteps 321.
Path 32 | total_timesteps 335.
Path 33 | total_timesteps 352.
Path 34 | total_timesteps 369.
Path 35 | total_timesteps 378.
Path 36 | total_timesteps 395.
Path 37 | total_timesteps 402.
Path 38 | total_timesteps 411.
Path 39 | total_timesteps 420.
Path 40 | total_timesteps 430.
Path 41 | total_timesteps 437.
Path 42 | total_timesteps 444.
Path 43 | total_timesteps 452.
Path 44 | total_timesteps 460.
Path 45 | total_timesteps 479.
Path 46 | total_timesteps 494.
Path 47 | total_timesteps 503.
Path 48 | total_timesteps 514.
Path 49 | total_timesteps 524.
Path 50 | total_timesteps 539.
Path 51 | total_timesteps 554.
Path 52 | total_timesteps 561.
Path 53 | total_timesteps 572.
Path 54 | total_timesteps 581.
Path 55 | total_timesteps 590.
Path 56 | total_timesteps 600.
Path 57 | total_timesteps 609.
Path 58 | total_timesteps 633.
Path 59 | total_timesteps 644.
Path 60 | total_timesteps 653.
Path 61 | total_timesteps 666.
Path 62 | total_timesteps 675.
Path 63 | total_timesteps 687.
Path 64 | total_timesteps 701.
Path 65 | total_timesteps 719.
Path 66 | total_timesteps 729.
Path 67 | total_timesteps 735.
Path 68 | total_timesteps 747.
Path 69 | total_timesteps 760.
Path 70 | total_timesteps 773.
Path 71 | total_timesteps 784.
Path 72 | total_timesteps 792.
Path 73 | total_timesteps 804.
Path 74 | total_timesteps 811.
Path 75 | total_timesteps 822.
Path 76 | total_timesteps 832.
Path 77 | total_timesteps 839.
Path 78 | total_timesteps 847.
Path 79 | total_timesteps 858.
Path 80 | total_timesteps 871.
Path 81 | total_timesteps 879.
Path 82 | total_timesteps 887.
Path 83 | total_timesteps 898.
Path 84 | total_timesteps 906.
Path 85 | total_timesteps 918.
Path 86 | total_timesteps 926.
Path 87 | total_timesteps 938.
Path 88 | total_timesteps 946.
Path 89 | total_timesteps 957.
Path 90 | total_timesteps 973.
Path 91 | total_timesteps 981.
Path 92 | total_timesteps 996.
Path 93 | total_timesteps 1004.
Path 94 | total_timesteps 1011.
Path 95 | total_timesteps 1018.
Path 96 | total_timesteps 1029.
Path 97 | total_timesteps 1038.
Path 98 | total_timesteps 1048.
Path 99 | total_timesteps 1055.
Path 100 | total_timesteps 1071.
Path 101 | total_timesteps 1082.
Path 102 | total_timesteps 1091.
Path 103 | total_timesteps 1099.
Path 104 | total_timesteps 1108.
Path 105 | total_timesteps 1122.
Path 106 | total_timesteps 1134.
Path 107 | total_timesteps 1151.
Path 108 | total_timesteps 1162.
Path 109 | total_timesteps 1173.
Path 110 | total_timesteps 1186.
Path 111 | total_timesteps 1196.
Path 112 | total_timesteps 1208.
Path 113 | total_timesteps 1230.
Path 114 | total_timesteps 1245.
Path 115 | total_timesteps 1255.
Path 116 | total_timesteps 1266.
Path 117 | total_timesteps 1275.
Path 118 | total_timesteps 1284.
Path 119 | total_timesteps 1294.
Path 120 | total_timesteps 1304.
Path 121 | total_timesteps 1316.
Path 122 | total_timesteps 1328.
Path 123 | total_timesteps 1339.
Path 124 | total_timesteps 1351.
Path 125 | total_timesteps 1362.
Path 126 | total_timesteps 1374.
Path 127 | total_timesteps 1390.
Path 128 | total_timesteps 1402.
Path 129 | total_timesteps 1413.
Path 130 | total_timesteps 1425.
Path 131 | total_timesteps 1433.
Path 132 | total_timesteps 1444.
Path 133 | total_timesteps 1453.
Path 134 | total_timesteps 1461.
Path 135 | total_timesteps 1470.
Path 136 | total_timesteps 1479.
Path 137 | total_timesteps 1486.
Path 138 | total_timesteps 1495.
Path 139 | total_timesteps 1517.
Path 140 | total_timesteps 1525.
Path 141 | total_timesteps 1537.
Path 142 | total_timesteps 1548.
Path 143 | total_timesteps 1557.
Path 144 | total_timesteps 1566.
Path 145 | total_timesteps 1575.
Path 146 | total_timesteps 1589.
Path 147 | total_timesteps 1602.
Path 148 | total_timesteps 1615.
Path 149 | total_timesteps 1629.
Path 150 | total_timesteps 1640.
Path 151 | total_timesteps 1649.
Path 152 | total_timesteps 1661.
Path 153 | total_timesteps 1675.
Path 154 | total_timesteps 1688.
Path 155 | total_timesteps 1698.
Path 156 | total_timesteps 1710.
Path 157 | total_timesteps 1721.
Path 158 | total_timesteps 1738.
Path 159 | total_timesteps 1748.
Path 160 | total_timesteps 1757.
Path 161 | total_timesteps 1773.
Path 162 | total_timesteps 1784.
Path 163 | total_timesteps 1790.
Path 164 | total_timesteps 1800.
Path 165 | total_timesteps 1809.
Path 166 | total_timesteps 1819.
Path 167 | total_timesteps 1829.
Path 168 | total_timesteps 1844.
Path 169 | total_timesteps 1854.
Path 170 | total_timesteps 1863.
Path 171 | total_timesteps 1871.
Path 172 | total_timesteps 1881.
Path 173 | total_timesteps 1891.
Path 174 | total_timesteps 1906.
Path 175 | total_timesteps 1914.
Path 176 | total_timesteps 1923.
Path 177 | total_timesteps 1932.
Path 178 | total_timesteps 1944.
Path 179 | total_timesteps 1960.
Path 180 | total_timesteps 1967.
Path 181 | total_timesteps 1978.
Path 182 | total_timesteps 1986.
Path 183 | total_timesteps 1994.
Path 184 | total_timesteps 2009.
Path 185 | total_timesteps 2021.
Path 186 | total_timesteps 2028.
Path 187 | total_timesteps 2037.
Path 188 | total_timesteps 2048.
Path 189 | total_timesteps 2057.
Path 190 | total_timesteps 2068.
Path 191 | total_timesteps 2076.
Path 192 | total_timesteps 2086.
Path 193 | total_timesteps 2095.
Path 194 | total_timesteps 2106.
Path 195 | total_timesteps 2113.
Path 196 | total_timesteps 2127.
Path 197 | total_timesteps 2135.
Path 198 | total_timesteps 2156.
Path 199 | total_timesteps 2166.
Path 200 | total_timesteps 2178.
Path 201 | total_timesteps 2188.
Path 202 | total_timesteps 2205.
Path 203 | total_timesteps 2216.
Path 204 | total_timesteps 2225.
Path 205 | total_timesteps 2235.
Path 206 | total_timesteps 2250.
Path 207 | total_timesteps 2259.
Path 208 | total_timesteps 2267.
Path 209 | total_timesteps 2275.
Path 210 | total_timesteps 2288.
Path 211 | total_timesteps 2295.
Path 212 | total_timesteps 2304.
Path 213 | total_timesteps 2311.
Path 214 | total_timesteps 2326.
Path 215 | total_timesteps 2337.
Path 216 | total_timesteps 2345.
Path 217 | total_timesteps 2357.
Path 218 | total_timesteps 2368.
Path 219 | total_timesteps 2377.
Path 220 | total_timesteps 2387.
Path 221 | total_timesteps 2395.
Path 222 | total_timesteps 2409.
Path 223 | total_timesteps 2420.
Path 224 | total_timesteps 2428.
Path 225 | total_timesteps 2445.
Path 226 | total_timesteps 2453.
Path 227 | total_timesteps 2462.
Path 228 | total_timesteps 2470.
Path 229 | total_timesteps 2480.
Path 230 | total_timesteps 2499.
Path 231 | total_timesteps 2510.
Path 232 | total_timesteps 2520.
Path 233 | total_timesteps 2528.
Path 234 | total_timesteps 2538.
Path 235 | total_timesteps 2547.
Path 236 | total_timesteps 2555.
Path 237 | total_timesteps 2569.
Path 238 | total_timesteps 2577.
Path 239 | total_timesteps 2593.
Path 240 | total_timesteps 2605.
Path 241 | total_timesteps 2620.
Path 242 | total_timesteps 2638.
Path 243 | total_timesteps 2652.
Path 244 | total_timesteps 2659.
Path 245 | total_timesteps 2673.
Path 246 | total_timesteps 2680.
Path 247 | total_timesteps 2697.
Path 248 | total_timesteps 2714.
Path 249 | total_timesteps 2722.
Path 250 | total_timesteps 2735.
Path 251 | total_timesteps 2746.
Path 252 | total_timesteps 2755.
Path 253 | total_timesteps 2765.
Path 254 | total_timesteps 2776.
Path 255 | total_timesteps 2785.
Path 256 | total_timesteps 2806.
Path 257 | total_timesteps 2815.
Path 258 | total_timesteps 2826.
Path 259 | total_timesteps 2834.
Path 260 | total_timesteps 2841.
Path 261 | total_timesteps 2851.
Path 262 | total_timesteps 2861.
Path 263 | total_timesteps 2869.
Path 264 | total_timesteps 2878.
Path 265 | total_timesteps 2887.
Path 266 | total_timesteps 2898.
Path 267 | total_timesteps 2913.
Path 268 | total_timesteps 2922.
Path 269 | total_timesteps 2931.
Path 270 | total_timesteps 2942.
Path 271 | total_timesteps 2949.
Path 272 | total_timesteps 2957.
Path 273 | total_timesteps 2967.
Path 274 | total_timesteps 2978.
Path 275 | total_timesteps 2986.
Path 276 | total_timesteps 2994.
Path 277 | total_timesteps 3018.
Path 278 | total_timesteps 3026.
Path 279 | total_timesteps 3034.
Path 280 | total_timesteps 3044.
Path 281 | total_timesteps 3053.
Path 282 | total_timesteps 3062.
Path 283 | total_timesteps 3069.
Path 284 | total_timesteps 3078.
Path 285 | total_timesteps 3094.
Path 286 | total_timesteps 3106.
Path 287 | total_timesteps 3114.
Path 288 | total_timesteps 3122.
Path 289 | total_timesteps 3136.
Path 290 | total_timesteps 3145.
Path 291 | total_timesteps 3157.
Path 292 | total_timesteps 3169.
Path 293 | total_timesteps 3178.
Path 294 | total_timesteps 3187.
Path 295 | total_timesteps 3198.
Path 296 | total_timesteps 3208.
Path 297 | total_timesteps 3216.
Path 298 | total_timesteps 3222.
Path 299 | total_timesteps 3237.
Path 300 | total_timesteps 3246.
Path 301 | total_timesteps 3253.
Path 302 | total_timesteps 3267.
Path 303 | total_timesteps 3280.
Path 304 | total_timesteps 3289.
Path 305 | total_timesteps 3299.
Path 306 | total_timesteps 3309.
Path 307 | total_timesteps 3322.
Path 308 | total_timesteps 3329.
Path 309 | total_timesteps 3340.
Path 310 | total_timesteps 3352.
Path 311 | total_timesteps 3362.
Path 312 | total_timesteps 3368.
Path 313 | total_timesteps 3383.
Path 314 | total_timesteps 3393.
Path 315 | total_timesteps 3413.
Path 316 | total_timesteps 3426.
Path 317 | total_timesteps 3445.
Path 318 | total_timesteps 3452.
Path 319 | total_timesteps 3460.
Path 320 | total_timesteps 3473.
Path 321 | total_timesteps 3488.
Path 322 | total_timesteps 3505.
Path 323 | total_timesteps 3520.
Path 324 | total_timesteps 3529.
Path 325 | total_timesteps 3537.
Path 326 | total_timesteps 3549.
Path 327 | total_timesteps 3564.
Path 328 | total_timesteps 3582.
Path 329 | total_timesteps 3590.
Path 330 | total_timesteps 3597.
Path 331 | total_timesteps 3606.
Path 332 | total_timesteps 3614.
Path 333 | total_timesteps 3628.
Path 334 | total_timesteps 3640.
Path 335 | total_timesteps 3649.
Path 336 | total_timesteps 3658.
Path 337 | total_timesteps 3670.
Path 338 | total_timesteps 3686.
Path 339 | total_timesteps 3694.
Path 340 | total_timesteps 3703.
Path 341 | total_timesteps 3710.
Path 342 | total_timesteps 3719.
Path 343 | total_timesteps 3727.
Path 344 | total_timesteps 3739.
Path 345 | total_timesteps 3747.
Path 346 | total_timesteps 3763.
Path 347 | total_timesteps 3772.
Path 348 | total_timesteps 3782.
Path 349 | total_timesteps 3796.
Path 350 | total_timesteps 3806.
Path 351 | total_timesteps 3815.
Path 352 | total_timesteps 3822.
Path 353 | total_timesteps 3832.
Path 354 | total_timesteps 3845.
Path 355 | total_timesteps 3856.
Path 356 | total_timesteps 3864.
Path 357 | total_timesteps 3881.
Path 358 | total_timesteps 3889.
Path 359 | total_timesteps 3909.
Path 360 | total_timesteps 3921.
Path 361 | total_timesteps 3930.
Path 362 | total_timesteps 3943.
Path 363 | total_timesteps 3950.
Path 364 | total_timesteps 3964.
Path 365 | total_timesteps 3972.
Path 366 | total_timesteps 3987.
Path 367 | total_timesteps 3994.
Path 368 | total_timesteps 4002.
Path 369 | total_timesteps 4011.
Path 370 | total_timesteps 4024.
Path 371 | total_timesteps 4032.
Path 372 | total_timesteps 4040.
Path 373 | total_timesteps 4048.
Path 374 | total_timesteps 4061.
Path 375 | total_timesteps 4072.
Path 376 | total_timesteps 4085.
Path 377 | total_timesteps 4093.
Path 378 | total_timesteps 4107.
Path 379 | total_timesteps 4123.
Path 380 | total_timesteps 4129.
Path 381 | total_timesteps 4139.
Path 382 | total_timesteps 4150.
Path 383 | total_timesteps 4157.
Path 384 | total_timesteps 4170.
Path 385 | total_timesteps 4180.
Path 386 | total_timesteps 4191.
Path 387 | total_timesteps 4199.
Path 388 | total_timesteps 4208.
Path 389 | total_timesteps 4215.
Path 390 | total_timesteps 4225.
Path 391 | total_timesteps 4237.
Path 392 | total_timesteps 4245.
Path 393 | total_timesteps 4255.
Path 394 | total_timesteps 4265.
Path 395 | total_timesteps 4284.
Path 396 | total_timesteps 4294.
Path 397 | total_timesteps 4305.
Path 398 | total_timesteps 4321.
Path 399 | total_timesteps 4334.
Path 400 | total_timesteps 4344.
Path 401 | total_timesteps 4352.
Path 402 | total_timesteps 4362.
Path 403 | total_timesteps 4370.
Path 404 | total_timesteps 4380.
Path 405 | total_timesteps 4388.
Path 406 | total_timesteps 4399.
Path 407 | total_timesteps 4414.
Path 408 | total_timesteps 4423.
Path 409 | total_timesteps 4434.
Path 410 | total_timesteps 4443.
Path 411 | total_timesteps 4451.
Path 412 | total_timesteps 4459.
Path 413 | total_timesteps 4471.
Path 414 | total_timesteps 4492.
Path 415 | total_timesteps 4501.
Path 416 | total_timesteps 4514.
Path 417 | total_timesteps 4527.
Path 418 | total_timesteps 4542.
Path 419 | total_timesteps 4549.
Path 420 | total_timesteps 4561.
Path 421 | total_timesteps 4567.
Path 422 | total_timesteps 4585.
Path 423 | total_timesteps 4595.
Path 424 | total_timesteps 4610.
Path 425 | total_timesteps 4629.
Path 426 | total_timesteps 4637.
Path 427 | total_timesteps 4647.
Path 428 | total_timesteps 4657.
Path 429 | total_timesteps 4668.
Path 430 | total_timesteps 4676.
Path 431 | total_timesteps 4687.
Path 432 | total_timesteps 4698.
Path 433 | total_timesteps 4707.
Path 434 | total_timesteps 4721.
Path 435 | total_timesteps 4733.
Path 436 | total_timesteps 4743.
Path 437 | total_timesteps 4756.
Path 438 | total_timesteps 4763.
Path 439 | total_timesteps 4780.
Path 440 | total_timesteps 4792.
Path 441 | total_timesteps 4801.
Path 442 | total_timesteps 4808.
Path 443 | total_timesteps 4816.
Path 444 | total_timesteps 4828.
Path 445 | total_timesteps 4835.
Path 446 | total_timesteps 4847.
Path 447 | total_timesteps 4857.
Path 448 | total_timesteps 4867.
Path 449 | total_timesteps 4878.
Path 450 | total_timesteps 4895.
Path 451 | total_timesteps 4905.
Path 452 | total_timesteps 4916.
Path 453 | total_timesteps 4931.
Path 454 | total_timesteps 4941.
Path 455 | total_timesteps 4953.
Path 456 | total_timesteps 4961.
Path 457 | total_timesteps 4969.
Path 458 | total_timesteps 4978.
Path 459 | total_timesteps 4987.
Path 460 | total_timesteps 4998.
Path 461 | total_timesteps 5006.
Path 462 | total_timesteps 5019.
Path 463 | total_timesteps 5030.
Path 464 | total_timesteps 5044.
Path 465 | total_timesteps 5052.
Path 466 | total_timesteps 5067.
Path 467 | total_timesteps 5076.
Path 468 | total_timesteps 5088.
Path 469 | total_timesteps 5102.
Path 470 | total_timesteps 5113.
Path 471 | total_timesteps 5120.
Path 472 | total_timesteps 5132.
Path 473 | total_timesteps 5142.
Path 474 | total_timesteps 5154.
Path 475 | total_timesteps 5165.
Path 476 | total_timesteps 5175.
Path 477 | total_timesteps 5187.
Path 478 | total_timesteps 5195.
Path 479 | total_timesteps 5216.
Path 480 | total_timesteps 5228.
Path 481 | total_timesteps 5239.
Path 482 | total_timesteps 5246.
Path 483 | total_timesteps 5258.
Path 484 | total_timesteps 5265.
Path 485 | total_timesteps 5288.
Path 486 | total_timesteps 5294.
Path 487 | total_timesteps 5304.
Path 488 | total_timesteps 5322.
Path 489 | total_timesteps 5331.
Path 490 | total_timesteps 5341.
Path 491 | total_timesteps 5350.
Path 492 | total_timesteps 5361.
Path 493 | total_timesteps 5370.
Path 494 | total_timesteps 5379.
Path 495 | total_timesteps 5388.
Path 496 | total_timesteps 5398.
Path 497 | total_timesteps 5407.
Path 498 | total_timesteps 5421.
Path 499 | total_timesteps 5431.
Path 500 | total_timesteps 5442.
Path 501 | total_timesteps 5452.
Path 502 | total_timesteps 5463.
Path 503 | total_timesteps 5475.
Path 504 | total_timesteps 5482.
Path 505 | total_timesteps 5492.
Path 506 | total_timesteps 5503.
Path 507 | total_timesteps 5511.
Path 508 | total_timesteps 5521.
Path 509 | total_timesteps 5528.
Path 510 | total_timesteps 5539.
Path 511 | total_timesteps 5558.
Path 512 | total_timesteps 5568.
Path 513 | total_timesteps 5584.
Path 514 | total_timesteps 5590.
Path 515 | total_timesteps 5600.
Path 516 | total_timesteps 5609.
Path 517 | total_timesteps 5620.
Path 518 | total_timesteps 5629.
Path 519 | total_timesteps 5642.
Path 520 | total_timesteps 5651.
Path 521 | total_timesteps 5659.
Path 522 | total_timesteps 5668.
Path 523 | total_timesteps 5677.
Path 524 | total_timesteps 5685.
Path 525 | total_timesteps 5694.
Path 526 | total_timesteps 5709.
Path 527 | total_timesteps 5718.
Path 528 | total_timesteps 5728.
Path 529 | total_timesteps 5735.
Path 530 | total_timesteps 5746.
Path 531 | total_timesteps 5757.
Path 532 | total_timesteps 5769.
Path 533 | total_timesteps 5778.
Path 534 | total_timesteps 5792.
Path 535 | total_timesteps 5804.
Path 536 | total_timesteps 5816.
Path 537 | total_timesteps 5828.
Path 538 | total_timesteps 5835.
Path 539 | total_timesteps 5844.
Path 540 | total_timesteps 5852.
Path 541 | total_timesteps 5859.
Path 542 | total_timesteps 5866.
Path 543 | total_timesteps 5875.
Path 544 | total_timesteps 5886.
Path 545 | total_timesteps 5897.
Path 546 | total_timesteps 5910.
Path 547 | total_timesteps 5917.
Path 548 | total_timesteps 5929.
Path 549 | total_timesteps 5938.
Path 550 | total_timesteps 5950.
Path 551 | total_timesteps 5961.
Path 552 | total_timesteps 5972.
Path 553 | total_timesteps 5982.
Path 554 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.41    |
| Iteration     | 6        |
| MaximumReturn | -0.759   |
| MinimumReturn | -21.9    |
| TotalSamples  | 32028    |
----------------------------
itr #7 | 
Fitting dynamics.
Validation loss = 0.013709679245948792
Validation loss = 0.01487263385206461
Validation loss = 0.012157989665865898
Validation loss = 0.011205865070223808
Validation loss = 0.01236073486506939
Validation loss = 0.01086082961410284
Validation loss = 0.011317483149468899
Validation loss = 0.0107127595692873
Validation loss = 0.010797305032610893
Validation loss = 0.011787453666329384
Validation loss = 0.010261839255690575
Validation loss = 0.011668773368000984
Validation loss = 0.01025710254907608
Validation loss = 0.010736276395618916
Validation loss = 0.01205920334905386
Validation loss = 0.011729992926120758
Validation loss = 0.010114847682416439
Validation loss = 0.01084049791097641
Validation loss = 0.01075496431440115
Validation loss = 0.011907866224646568
Validation loss = 0.010360504500567913
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 53.
Path 6 | total_timesteps 62.
Path 7 | total_timesteps 72.
Path 8 | total_timesteps 85.
Path 9 | total_timesteps 92.
Path 10 | total_timesteps 106.
Path 11 | total_timesteps 115.
Path 12 | total_timesteps 127.
Path 13 | total_timesteps 133.
Path 14 | total_timesteps 145.
Path 15 | total_timesteps 155.
Path 16 | total_timesteps 162.
Path 17 | total_timesteps 169.
Path 18 | total_timesteps 176.
Path 19 | total_timesteps 190.
Path 20 | total_timesteps 198.
Path 21 | total_timesteps 209.
Path 22 | total_timesteps 221.
Path 23 | total_timesteps 228.
Path 24 | total_timesteps 242.
Path 25 | total_timesteps 250.
Path 26 | total_timesteps 263.
Path 27 | total_timesteps 271.
Path 28 | total_timesteps 280.
Path 29 | total_timesteps 306.
Path 30 | total_timesteps 317.
Path 31 | total_timesteps 327.
Path 32 | total_timesteps 337.
Path 33 | total_timesteps 344.
Path 34 | total_timesteps 358.
Path 35 | total_timesteps 369.
Path 36 | total_timesteps 379.
Path 37 | total_timesteps 389.
Path 38 | total_timesteps 412.
Path 39 | total_timesteps 423.
Path 40 | total_timesteps 445.
Path 41 | total_timesteps 455.
Path 42 | total_timesteps 462.
Path 43 | total_timesteps 468.
Path 44 | total_timesteps 475.
Path 45 | total_timesteps 481.
Path 46 | total_timesteps 492.
Path 47 | total_timesteps 503.
Path 48 | total_timesteps 516.
Path 49 | total_timesteps 527.
Path 50 | total_timesteps 540.
Path 51 | total_timesteps 552.
Path 52 | total_timesteps 566.
Path 53 | total_timesteps 579.
Path 54 | total_timesteps 591.
Path 55 | total_timesteps 601.
Path 56 | total_timesteps 612.
Path 57 | total_timesteps 621.
Path 58 | total_timesteps 632.
Path 59 | total_timesteps 642.
Path 60 | total_timesteps 656.
Path 61 | total_timesteps 665.
Path 62 | total_timesteps 680.
Path 63 | total_timesteps 688.
Path 64 | total_timesteps 696.
Path 65 | total_timesteps 722.
Path 66 | total_timesteps 735.
Path 67 | total_timesteps 744.
Path 68 | total_timesteps 750.
Path 69 | total_timesteps 763.
Path 70 | total_timesteps 773.
Path 71 | total_timesteps 784.
Path 72 | total_timesteps 793.
Path 73 | total_timesteps 800.
Path 74 | total_timesteps 815.
Path 75 | total_timesteps 821.
Path 76 | total_timesteps 829.
Path 77 | total_timesteps 849.
Path 78 | total_timesteps 863.
Path 79 | total_timesteps 874.
Path 80 | total_timesteps 883.
Path 81 | total_timesteps 900.
Path 82 | total_timesteps 911.
Path 83 | total_timesteps 919.
Path 84 | total_timesteps 925.
Path 85 | total_timesteps 936.
Path 86 | total_timesteps 943.
Path 87 | total_timesteps 950.
Path 88 | total_timesteps 958.
Path 89 | total_timesteps 975.
Path 90 | total_timesteps 985.
Path 91 | total_timesteps 994.
Path 92 | total_timesteps 1006.
Path 93 | total_timesteps 1014.
Path 94 | total_timesteps 1022.
Path 95 | total_timesteps 1032.
Path 96 | total_timesteps 1041.
Path 97 | total_timesteps 1048.
Path 98 | total_timesteps 1058.
Path 99 | total_timesteps 1068.
Path 100 | total_timesteps 1082.
Path 101 | total_timesteps 1091.
Path 102 | total_timesteps 1099.
Path 103 | total_timesteps 1106.
Path 104 | total_timesteps 1113.
Path 105 | total_timesteps 1125.
Path 106 | total_timesteps 1134.
Path 107 | total_timesteps 1147.
Path 108 | total_timesteps 1155.
Path 109 | total_timesteps 1161.
Path 110 | total_timesteps 1171.
Path 111 | total_timesteps 1178.
Path 112 | total_timesteps 1185.
Path 113 | total_timesteps 1197.
Path 114 | total_timesteps 1210.
Path 115 | total_timesteps 1222.
Path 116 | total_timesteps 1231.
Path 117 | total_timesteps 1249.
Path 118 | total_timesteps 1266.
Path 119 | total_timesteps 1277.
Path 120 | total_timesteps 1285.
Path 121 | total_timesteps 1297.
Path 122 | total_timesteps 1308.
Path 123 | total_timesteps 1320.
Path 124 | total_timesteps 1332.
Path 125 | total_timesteps 1341.
Path 126 | total_timesteps 1349.
Path 127 | total_timesteps 1357.
Path 128 | total_timesteps 1366.
Path 129 | total_timesteps 1373.
Path 130 | total_timesteps 1381.
Path 131 | total_timesteps 1389.
Path 132 | total_timesteps 1396.
Path 133 | total_timesteps 1403.
Path 134 | total_timesteps 1414.
Path 135 | total_timesteps 1424.
Path 136 | total_timesteps 1432.
Path 137 | total_timesteps 1442.
Path 138 | total_timesteps 1455.
Path 139 | total_timesteps 1465.
Path 140 | total_timesteps 1484.
Path 141 | total_timesteps 1497.
Path 142 | total_timesteps 1505.
Path 143 | total_timesteps 1516.
Path 144 | total_timesteps 1524.
Path 145 | total_timesteps 1534.
Path 146 | total_timesteps 1541.
Path 147 | total_timesteps 1552.
Path 148 | total_timesteps 1564.
Path 149 | total_timesteps 1575.
Path 150 | total_timesteps 1585.
Path 151 | total_timesteps 1594.
Path 152 | total_timesteps 1603.
Path 153 | total_timesteps 1614.
Path 154 | total_timesteps 1622.
Path 155 | total_timesteps 1630.
Path 156 | total_timesteps 1641.
Path 157 | total_timesteps 1649.
Path 158 | total_timesteps 1661.
Path 159 | total_timesteps 1669.
Path 160 | total_timesteps 1680.
Path 161 | total_timesteps 1686.
Path 162 | total_timesteps 1694.
Path 163 | total_timesteps 1701.
Path 164 | total_timesteps 1712.
Path 165 | total_timesteps 1724.
Path 166 | total_timesteps 1732.
Path 167 | total_timesteps 1740.
Path 168 | total_timesteps 1747.
Path 169 | total_timesteps 1755.
Path 170 | total_timesteps 1763.
Path 171 | total_timesteps 1770.
Path 172 | total_timesteps 1777.
Path 173 | total_timesteps 1794.
Path 174 | total_timesteps 1811.
Path 175 | total_timesteps 1819.
Path 176 | total_timesteps 1827.
Path 177 | total_timesteps 1835.
Path 178 | total_timesteps 1849.
Path 179 | total_timesteps 1858.
Path 180 | total_timesteps 1870.
Path 181 | total_timesteps 1879.
Path 182 | total_timesteps 1887.
Path 183 | total_timesteps 1898.
Path 184 | total_timesteps 1914.
Path 185 | total_timesteps 1923.
Path 186 | total_timesteps 1931.
Path 187 | total_timesteps 1941.
Path 188 | total_timesteps 1950.
Path 189 | total_timesteps 1965.
Path 190 | total_timesteps 1972.
Path 191 | total_timesteps 1981.
Path 192 | total_timesteps 1991.
Path 193 | total_timesteps 2000.
Path 194 | total_timesteps 2010.
Path 195 | total_timesteps 2019.
Path 196 | total_timesteps 2029.
Path 197 | total_timesteps 2036.
Path 198 | total_timesteps 2050.
Path 199 | total_timesteps 2064.
Path 200 | total_timesteps 2073.
Path 201 | total_timesteps 2084.
Path 202 | total_timesteps 2091.
Path 203 | total_timesteps 2099.
Path 204 | total_timesteps 2108.
Path 205 | total_timesteps 2117.
Path 206 | total_timesteps 2126.
Path 207 | total_timesteps 2146.
Path 208 | total_timesteps 2158.
Path 209 | total_timesteps 2168.
Path 210 | total_timesteps 2174.
Path 211 | total_timesteps 2182.
Path 212 | total_timesteps 2194.
Path 213 | total_timesteps 2202.
Path 214 | total_timesteps 2209.
Path 215 | total_timesteps 2217.
Path 216 | total_timesteps 2232.
Path 217 | total_timesteps 2239.
Path 218 | total_timesteps 2251.
Path 219 | total_timesteps 2265.
Path 220 | total_timesteps 2273.
Path 221 | total_timesteps 2288.
Path 222 | total_timesteps 2299.
Path 223 | total_timesteps 2312.
Path 224 | total_timesteps 2318.
Path 225 | total_timesteps 2333.
Path 226 | total_timesteps 2341.
Path 227 | total_timesteps 2347.
Path 228 | total_timesteps 2353.
Path 229 | total_timesteps 2362.
Path 230 | total_timesteps 2382.
Path 231 | total_timesteps 2392.
Path 232 | total_timesteps 2406.
Path 233 | total_timesteps 2414.
Path 234 | total_timesteps 2422.
Path 235 | total_timesteps 2432.
Path 236 | total_timesteps 2440.
Path 237 | total_timesteps 2450.
Path 238 | total_timesteps 2465.
Path 239 | total_timesteps 2477.
Path 240 | total_timesteps 2485.
Path 241 | total_timesteps 2494.
Path 242 | total_timesteps 2503.
Path 243 | total_timesteps 2512.
Path 244 | total_timesteps 2521.
Path 245 | total_timesteps 2531.
Path 246 | total_timesteps 2539.
Path 247 | total_timesteps 2551.
Path 248 | total_timesteps 2561.
Path 249 | total_timesteps 2571.
Path 250 | total_timesteps 2583.
Path 251 | total_timesteps 2591.
Path 252 | total_timesteps 2609.
Path 253 | total_timesteps 2618.
Path 254 | total_timesteps 2626.
Path 255 | total_timesteps 2633.
Path 256 | total_timesteps 2644.
Path 257 | total_timesteps 2660.
Path 258 | total_timesteps 2674.
Path 259 | total_timesteps 2686.
Path 260 | total_timesteps 2695.
Path 261 | total_timesteps 2703.
Path 262 | total_timesteps 2710.
Path 263 | total_timesteps 2720.
Path 264 | total_timesteps 2727.
Path 265 | total_timesteps 2736.
Path 266 | total_timesteps 2746.
Path 267 | total_timesteps 2759.
Path 268 | total_timesteps 2774.
Path 269 | total_timesteps 2782.
Path 270 | total_timesteps 2793.
Path 271 | total_timesteps 2803.
Path 272 | total_timesteps 2811.
Path 273 | total_timesteps 2821.
Path 274 | total_timesteps 2830.
Path 275 | total_timesteps 2848.
Path 276 | total_timesteps 2855.
Path 277 | total_timesteps 2863.
Path 278 | total_timesteps 2873.
Path 279 | total_timesteps 2881.
Path 280 | total_timesteps 2892.
Path 281 | total_timesteps 2902.
Path 282 | total_timesteps 2914.
Path 283 | total_timesteps 2924.
Path 284 | total_timesteps 2938.
Path 285 | total_timesteps 2951.
Path 286 | total_timesteps 2964.
Path 287 | total_timesteps 2973.
Path 288 | total_timesteps 2987.
Path 289 | total_timesteps 2997.
Path 290 | total_timesteps 3005.
Path 291 | total_timesteps 3013.
Path 292 | total_timesteps 3021.
Path 293 | total_timesteps 3031.
Path 294 | total_timesteps 3046.
Path 295 | total_timesteps 3056.
Path 296 | total_timesteps 3070.
Path 297 | total_timesteps 3081.
Path 298 | total_timesteps 3089.
Path 299 | total_timesteps 3097.
Path 300 | total_timesteps 3105.
Path 301 | total_timesteps 3120.
Path 302 | total_timesteps 3131.
Path 303 | total_timesteps 3139.
Path 304 | total_timesteps 3161.
Path 305 | total_timesteps 3174.
Path 306 | total_timesteps 3183.
Path 307 | total_timesteps 3193.
Path 308 | total_timesteps 3210.
Path 309 | total_timesteps 3220.
Path 310 | total_timesteps 3228.
Path 311 | total_timesteps 3237.
Path 312 | total_timesteps 3245.
Path 313 | total_timesteps 3253.
Path 314 | total_timesteps 3265.
Path 315 | total_timesteps 3275.
Path 316 | total_timesteps 3284.
Path 317 | total_timesteps 3293.
Path 318 | total_timesteps 3304.
Path 319 | total_timesteps 3310.
Path 320 | total_timesteps 3327.
Path 321 | total_timesteps 3337.
Path 322 | total_timesteps 3348.
Path 323 | total_timesteps 3355.
Path 324 | total_timesteps 3367.
Path 325 | total_timesteps 3374.
Path 326 | total_timesteps 3389.
Path 327 | total_timesteps 3399.
Path 328 | total_timesteps 3408.
Path 329 | total_timesteps 3416.
Path 330 | total_timesteps 3427.
Path 331 | total_timesteps 3435.
Path 332 | total_timesteps 3444.
Path 333 | total_timesteps 3456.
Path 334 | total_timesteps 3466.
Path 335 | total_timesteps 3475.
Path 336 | total_timesteps 3488.
Path 337 | total_timesteps 3496.
Path 338 | total_timesteps 3506.
Path 339 | total_timesteps 3513.
Path 340 | total_timesteps 3524.
Path 341 | total_timesteps 3532.
Path 342 | total_timesteps 3542.
Path 343 | total_timesteps 3548.
Path 344 | total_timesteps 3558.
Path 345 | total_timesteps 3566.
Path 346 | total_timesteps 3583.
Path 347 | total_timesteps 3592.
Path 348 | total_timesteps 3602.
Path 349 | total_timesteps 3610.
Path 350 | total_timesteps 3621.
Path 351 | total_timesteps 3630.
Path 352 | total_timesteps 3638.
Path 353 | total_timesteps 3648.
Path 354 | total_timesteps 3659.
Path 355 | total_timesteps 3666.
Path 356 | total_timesteps 3676.
Path 357 | total_timesteps 3691.
Path 358 | total_timesteps 3701.
Path 359 | total_timesteps 3708.
Path 360 | total_timesteps 3717.
Path 361 | total_timesteps 3725.
Path 362 | total_timesteps 3738.
Path 363 | total_timesteps 3749.
Path 364 | total_timesteps 3759.
Path 365 | total_timesteps 3771.
Path 366 | total_timesteps 3780.
Path 367 | total_timesteps 3787.
Path 368 | total_timesteps 3795.
Path 369 | total_timesteps 3803.
Path 370 | total_timesteps 3813.
Path 371 | total_timesteps 3824.
Path 372 | total_timesteps 3837.
Path 373 | total_timesteps 3844.
Path 374 | total_timesteps 3854.
Path 375 | total_timesteps 3864.
Path 376 | total_timesteps 3873.
Path 377 | total_timesteps 3880.
Path 378 | total_timesteps 3892.
Path 379 | total_timesteps 3901.
Path 380 | total_timesteps 3910.
Path 381 | total_timesteps 3920.
Path 382 | total_timesteps 3932.
Path 383 | total_timesteps 3940.
Path 384 | total_timesteps 3952.
Path 385 | total_timesteps 3959.
Path 386 | total_timesteps 3968.
Path 387 | total_timesteps 3977.
Path 388 | total_timesteps 3986.
Path 389 | total_timesteps 3995.
Path 390 | total_timesteps 4006.
Path 391 | total_timesteps 4027.
Path 392 | total_timesteps 4041.
Path 393 | total_timesteps 4052.
Path 394 | total_timesteps 4066.
Path 395 | total_timesteps 4073.
Path 396 | total_timesteps 4081.
Path 397 | total_timesteps 4092.
Path 398 | total_timesteps 4101.
Path 399 | total_timesteps 4118.
Path 400 | total_timesteps 4126.
Path 401 | total_timesteps 4137.
Path 402 | total_timesteps 4146.
Path 403 | total_timesteps 4155.
Path 404 | total_timesteps 4167.
Path 405 | total_timesteps 4177.
Path 406 | total_timesteps 4188.
Path 407 | total_timesteps 4201.
Path 408 | total_timesteps 4210.
Path 409 | total_timesteps 4219.
Path 410 | total_timesteps 4228.
Path 411 | total_timesteps 4238.
Path 412 | total_timesteps 4248.
Path 413 | total_timesteps 4263.
Path 414 | total_timesteps 4274.
Path 415 | total_timesteps 4281.
Path 416 | total_timesteps 4289.
Path 417 | total_timesteps 4300.
Path 418 | total_timesteps 4310.
Path 419 | total_timesteps 4321.
Path 420 | total_timesteps 4331.
Path 421 | total_timesteps 4343.
Path 422 | total_timesteps 4350.
Path 423 | total_timesteps 4358.
Path 424 | total_timesteps 4372.
Path 425 | total_timesteps 4379.
Path 426 | total_timesteps 4386.
Path 427 | total_timesteps 4396.
Path 428 | total_timesteps 4406.
Path 429 | total_timesteps 4414.
Path 430 | total_timesteps 4422.
Path 431 | total_timesteps 4431.
Path 432 | total_timesteps 4440.
Path 433 | total_timesteps 4451.
Path 434 | total_timesteps 4461.
Path 435 | total_timesteps 4473.
Path 436 | total_timesteps 4484.
Path 437 | total_timesteps 4494.
Path 438 | total_timesteps 4503.
Path 439 | total_timesteps 4518.
Path 440 | total_timesteps 4526.
Path 441 | total_timesteps 4534.
Path 442 | total_timesteps 4542.
Path 443 | total_timesteps 4552.
Path 444 | total_timesteps 4568.
Path 445 | total_timesteps 4574.
Path 446 | total_timesteps 4582.
Path 447 | total_timesteps 4601.
Path 448 | total_timesteps 4609.
Path 449 | total_timesteps 4620.
Path 450 | total_timesteps 4632.
Path 451 | total_timesteps 4644.
Path 452 | total_timesteps 4659.
Path 453 | total_timesteps 4667.
Path 454 | total_timesteps 4675.
Path 455 | total_timesteps 4690.
Path 456 | total_timesteps 4703.
Path 457 | total_timesteps 4710.
Path 458 | total_timesteps 4720.
Path 459 | total_timesteps 4728.
Path 460 | total_timesteps 4741.
Path 461 | total_timesteps 4753.
Path 462 | total_timesteps 4767.
Path 463 | total_timesteps 4777.
Path 464 | total_timesteps 4786.
Path 465 | total_timesteps 4796.
Path 466 | total_timesteps 4810.
Path 467 | total_timesteps 4820.
Path 468 | total_timesteps 4831.
Path 469 | total_timesteps 4843.
Path 470 | total_timesteps 4853.
Path 471 | total_timesteps 4861.
Path 472 | total_timesteps 4870.
Path 473 | total_timesteps 4884.
Path 474 | total_timesteps 4896.
Path 475 | total_timesteps 4904.
Path 476 | total_timesteps 4911.
Path 477 | total_timesteps 4920.
Path 478 | total_timesteps 4929.
Path 479 | total_timesteps 4942.
Path 480 | total_timesteps 4953.
Path 481 | total_timesteps 4961.
Path 482 | total_timesteps 4968.
Path 483 | total_timesteps 4978.
Path 484 | total_timesteps 4986.
Path 485 | total_timesteps 4998.
Path 486 | total_timesteps 5011.
Path 487 | total_timesteps 5020.
Path 488 | total_timesteps 5030.
Path 489 | total_timesteps 5041.
Path 490 | total_timesteps 5055.
Path 491 | total_timesteps 5065.
Path 492 | total_timesteps 5078.
Path 493 | total_timesteps 5087.
Path 494 | total_timesteps 5098.
Path 495 | total_timesteps 5106.
Path 496 | total_timesteps 5114.
Path 497 | total_timesteps 5123.
Path 498 | total_timesteps 5132.
Path 499 | total_timesteps 5147.
Path 500 | total_timesteps 5158.
Path 501 | total_timesteps 5165.
Path 502 | total_timesteps 5183.
Path 503 | total_timesteps 5195.
Path 504 | total_timesteps 5206.
Path 505 | total_timesteps 5215.
Path 506 | total_timesteps 5222.
Path 507 | total_timesteps 5231.
Path 508 | total_timesteps 5242.
Path 509 | total_timesteps 5251.
Path 510 | total_timesteps 5265.
Path 511 | total_timesteps 5284.
Path 512 | total_timesteps 5291.
Path 513 | total_timesteps 5297.
Path 514 | total_timesteps 5310.
Path 515 | total_timesteps 5318.
Path 516 | total_timesteps 5327.
Path 517 | total_timesteps 5335.
Path 518 | total_timesteps 5344.
Path 519 | total_timesteps 5354.
Path 520 | total_timesteps 5365.
Path 521 | total_timesteps 5372.
Path 522 | total_timesteps 5379.
Path 523 | total_timesteps 5389.
Path 524 | total_timesteps 5404.
Path 525 | total_timesteps 5417.
Path 526 | total_timesteps 5423.
Path 527 | total_timesteps 5432.
Path 528 | total_timesteps 5443.
Path 529 | total_timesteps 5455.
Path 530 | total_timesteps 5465.
Path 531 | total_timesteps 5473.
Path 532 | total_timesteps 5480.
Path 533 | total_timesteps 5486.
Path 534 | total_timesteps 5495.
Path 535 | total_timesteps 5504.
Path 536 | total_timesteps 5516.
Path 537 | total_timesteps 5525.
Path 538 | total_timesteps 5535.
Path 539 | total_timesteps 5546.
Path 540 | total_timesteps 5554.
Path 541 | total_timesteps 5562.
Path 542 | total_timesteps 5570.
Path 543 | total_timesteps 5580.
Path 544 | total_timesteps 5588.
Path 545 | total_timesteps 5600.
Path 546 | total_timesteps 5617.
Path 547 | total_timesteps 5624.
Path 548 | total_timesteps 5632.
Path 549 | total_timesteps 5644.
Path 550 | total_timesteps 5653.
Path 551 | total_timesteps 5668.
Path 552 | total_timesteps 5678.
Path 553 | total_timesteps 5689.
Path 554 | total_timesteps 5701.
Path 555 | total_timesteps 5709.
Path 556 | total_timesteps 5718.
Path 557 | total_timesteps 5728.
Path 558 | total_timesteps 5740.
Path 559 | total_timesteps 5750.
Path 560 | total_timesteps 5765.
Path 561 | total_timesteps 5775.
Path 562 | total_timesteps 5784.
Path 563 | total_timesteps 5792.
Path 564 | total_timesteps 5816.
Path 565 | total_timesteps 5822.
Path 566 | total_timesteps 5833.
Path 567 | total_timesteps 5842.
Path 568 | total_timesteps 5850.
Path 569 | total_timesteps 5859.
Path 570 | total_timesteps 5868.
Path 571 | total_timesteps 5875.
Path 572 | total_timesteps 5883.
Path 573 | total_timesteps 5893.
Path 574 | total_timesteps 5902.
Path 575 | total_timesteps 5918.
Path 576 | total_timesteps 5927.
Path 577 | total_timesteps 5936.
Path 578 | total_timesteps 5944.
Path 579 | total_timesteps 5960.
Path 580 | total_timesteps 5967.
Path 581 | total_timesteps 5976.
Path 582 | total_timesteps 5989.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.29    |
| Iteration     | 7        |
| MaximumReturn | -0.835   |
| MinimumReturn | -22.3    |
| TotalSamples  | 36028    |
----------------------------
itr #8 | 
Fitting dynamics.
Validation loss = 0.01079455204308033
Validation loss = 0.009920873679220676
Validation loss = 0.009720955044031143
Validation loss = 0.009561502374708652
Validation loss = 0.010546727105975151
Validation loss = 0.009619119577109814
Validation loss = 0.009727712720632553
Validation loss = 0.009281205013394356
Validation loss = 0.009637068957090378
Validation loss = 0.010081514716148376
Validation loss = 0.010400957427918911
Validation loss = 0.009653806686401367
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 29.
Path 3 | total_timesteps 38.
Path 4 | total_timesteps 48.
Path 5 | total_timesteps 61.
Path 6 | total_timesteps 72.
Path 7 | total_timesteps 81.
Path 8 | total_timesteps 92.
Path 9 | total_timesteps 102.
Path 10 | total_timesteps 110.
Path 11 | total_timesteps 121.
Path 12 | total_timesteps 135.
Path 13 | total_timesteps 145.
Path 14 | total_timesteps 155.
Path 15 | total_timesteps 170.
Path 16 | total_timesteps 181.
Path 17 | total_timesteps 189.
Path 18 | total_timesteps 198.
Path 19 | total_timesteps 206.
Path 20 | total_timesteps 219.
Path 21 | total_timesteps 229.
Path 22 | total_timesteps 237.
Path 23 | total_timesteps 247.
Path 24 | total_timesteps 256.
Path 25 | total_timesteps 265.
Path 26 | total_timesteps 274.
Path 27 | total_timesteps 282.
Path 28 | total_timesteps 294.
Path 29 | total_timesteps 306.
Path 30 | total_timesteps 315.
Path 31 | total_timesteps 329.
Path 32 | total_timesteps 337.
Path 33 | total_timesteps 351.
Path 34 | total_timesteps 364.
Path 35 | total_timesteps 374.
Path 36 | total_timesteps 387.
Path 37 | total_timesteps 397.
Path 38 | total_timesteps 412.
Path 39 | total_timesteps 420.
Path 40 | total_timesteps 430.
Path 41 | total_timesteps 441.
Path 42 | total_timesteps 455.
Path 43 | total_timesteps 465.
Path 44 | total_timesteps 474.
Path 45 | total_timesteps 485.
Path 46 | total_timesteps 502.
Path 47 | total_timesteps 520.
Path 48 | total_timesteps 527.
Path 49 | total_timesteps 535.
Path 50 | total_timesteps 543.
Path 51 | total_timesteps 549.
Path 52 | total_timesteps 559.
Path 53 | total_timesteps 567.
Path 54 | total_timesteps 576.
Path 55 | total_timesteps 583.
Path 56 | total_timesteps 604.
Path 57 | total_timesteps 616.
Path 58 | total_timesteps 626.
Path 59 | total_timesteps 638.
Path 60 | total_timesteps 649.
Path 61 | total_timesteps 658.
Path 62 | total_timesteps 669.
Path 63 | total_timesteps 681.
Path 64 | total_timesteps 689.
Path 65 | total_timesteps 697.
Path 66 | total_timesteps 708.
Path 67 | total_timesteps 719.
Path 68 | total_timesteps 728.
Path 69 | total_timesteps 737.
Path 70 | total_timesteps 745.
Path 71 | total_timesteps 757.
Path 72 | total_timesteps 766.
Path 73 | total_timesteps 777.
Path 74 | total_timesteps 792.
Path 75 | total_timesteps 799.
Path 76 | total_timesteps 806.
Path 77 | total_timesteps 813.
Path 78 | total_timesteps 825.
Path 79 | total_timesteps 833.
Path 80 | total_timesteps 840.
Path 81 | total_timesteps 850.
Path 82 | total_timesteps 861.
Path 83 | total_timesteps 873.
Path 84 | total_timesteps 880.
Path 85 | total_timesteps 888.
Path 86 | total_timesteps 898.
Path 87 | total_timesteps 908.
Path 88 | total_timesteps 916.
Path 89 | total_timesteps 924.
Path 90 | total_timesteps 933.
Path 91 | total_timesteps 941.
Path 92 | total_timesteps 952.
Path 93 | total_timesteps 963.
Path 94 | total_timesteps 974.
Path 95 | total_timesteps 983.
Path 96 | total_timesteps 993.
Path 97 | total_timesteps 1007.
Path 98 | total_timesteps 1029.
Path 99 | total_timesteps 1036.
Path 100 | total_timesteps 1044.
Path 101 | total_timesteps 1059.
Path 102 | total_timesteps 1067.
Path 103 | total_timesteps 1079.
Path 104 | total_timesteps 1087.
Path 105 | total_timesteps 1094.
Path 106 | total_timesteps 1106.
Path 107 | total_timesteps 1113.
Path 108 | total_timesteps 1121.
Path 109 | total_timesteps 1135.
Path 110 | total_timesteps 1147.
Path 111 | total_timesteps 1162.
Path 112 | total_timesteps 1170.
Path 113 | total_timesteps 1181.
Path 114 | total_timesteps 1190.
Path 115 | total_timesteps 1198.
Path 116 | total_timesteps 1209.
Path 117 | total_timesteps 1217.
Path 118 | total_timesteps 1229.
Path 119 | total_timesteps 1244.
Path 120 | total_timesteps 1251.
Path 121 | total_timesteps 1269.
Path 122 | total_timesteps 1280.
Path 123 | total_timesteps 1296.
Path 124 | total_timesteps 1303.
Path 125 | total_timesteps 1319.
Path 126 | total_timesteps 1334.
Path 127 | total_timesteps 1343.
Path 128 | total_timesteps 1355.
Path 129 | total_timesteps 1369.
Path 130 | total_timesteps 1380.
Path 131 | total_timesteps 1389.
Path 132 | total_timesteps 1398.
Path 133 | total_timesteps 1408.
Path 134 | total_timesteps 1416.
Path 135 | total_timesteps 1435.
Path 136 | total_timesteps 1443.
Path 137 | total_timesteps 1453.
Path 138 | total_timesteps 1462.
Path 139 | total_timesteps 1471.
Path 140 | total_timesteps 1480.
Path 141 | total_timesteps 1490.
Path 142 | total_timesteps 1497.
Path 143 | total_timesteps 1514.
Path 144 | total_timesteps 1528.
Path 145 | total_timesteps 1538.
Path 146 | total_timesteps 1550.
Path 147 | total_timesteps 1569.
Path 148 | total_timesteps 1578.
Path 149 | total_timesteps 1586.
Path 150 | total_timesteps 1594.
Path 151 | total_timesteps 1604.
Path 152 | total_timesteps 1614.
Path 153 | total_timesteps 1632.
Path 154 | total_timesteps 1641.
Path 155 | total_timesteps 1652.
Path 156 | total_timesteps 1660.
Path 157 | total_timesteps 1668.
Path 158 | total_timesteps 1679.
Path 159 | total_timesteps 1691.
Path 160 | total_timesteps 1699.
Path 161 | total_timesteps 1707.
Path 162 | total_timesteps 1717.
Path 163 | total_timesteps 1728.
Path 164 | total_timesteps 1738.
Path 165 | total_timesteps 1756.
Path 166 | total_timesteps 1764.
Path 167 | total_timesteps 1773.
Path 168 | total_timesteps 1785.
Path 169 | total_timesteps 1805.
Path 170 | total_timesteps 1816.
Path 171 | total_timesteps 1828.
Path 172 | total_timesteps 1841.
Path 173 | total_timesteps 1855.
Path 174 | total_timesteps 1862.
Path 175 | total_timesteps 1873.
Path 176 | total_timesteps 1881.
Path 177 | total_timesteps 1887.
Path 178 | total_timesteps 1895.
Path 179 | total_timesteps 1910.
Path 180 | total_timesteps 1929.
Path 181 | total_timesteps 1944.
Path 182 | total_timesteps 1953.
Path 183 | total_timesteps 1966.
Path 184 | total_timesteps 1973.
Path 185 | total_timesteps 1980.
Path 186 | total_timesteps 1987.
Path 187 | total_timesteps 1997.
Path 188 | total_timesteps 2009.
Path 189 | total_timesteps 2019.
Path 190 | total_timesteps 2028.
Path 191 | total_timesteps 2036.
Path 192 | total_timesteps 2049.
Path 193 | total_timesteps 2061.
Path 194 | total_timesteps 2069.
Path 195 | total_timesteps 2080.
Path 196 | total_timesteps 2090.
Path 197 | total_timesteps 2100.
Path 198 | total_timesteps 2111.
Path 199 | total_timesteps 2117.
Path 200 | total_timesteps 2129.
Path 201 | total_timesteps 2140.
Path 202 | total_timesteps 2150.
Path 203 | total_timesteps 2161.
Path 204 | total_timesteps 2168.
Path 205 | total_timesteps 2178.
Path 206 | total_timesteps 2185.
Path 207 | total_timesteps 2195.
Path 208 | total_timesteps 2218.
Path 209 | total_timesteps 2230.
Path 210 | total_timesteps 2240.
Path 211 | total_timesteps 2248.
Path 212 | total_timesteps 2256.
Path 213 | total_timesteps 2268.
Path 214 | total_timesteps 2278.
Path 215 | total_timesteps 2289.
Path 216 | total_timesteps 2300.
Path 217 | total_timesteps 2308.
Path 218 | total_timesteps 2319.
Path 219 | total_timesteps 2326.
Path 220 | total_timesteps 2333.
Path 221 | total_timesteps 2341.
Path 222 | total_timesteps 2349.
Path 223 | total_timesteps 2361.
Path 224 | total_timesteps 2378.
Path 225 | total_timesteps 2392.
Path 226 | total_timesteps 2411.
Path 227 | total_timesteps 2418.
Path 228 | total_timesteps 2429.
Path 229 | total_timesteps 2442.
Path 230 | total_timesteps 2449.
Path 231 | total_timesteps 2462.
Path 232 | total_timesteps 2475.
Path 233 | total_timesteps 2485.
Path 234 | total_timesteps 2499.
Path 235 | total_timesteps 2506.
Path 236 | total_timesteps 2514.
Path 237 | total_timesteps 2529.
Path 238 | total_timesteps 2535.
Path 239 | total_timesteps 2545.
Path 240 | total_timesteps 2557.
Path 241 | total_timesteps 2565.
Path 242 | total_timesteps 2575.
Path 243 | total_timesteps 2582.
Path 244 | total_timesteps 2595.
Path 245 | total_timesteps 2605.
Path 246 | total_timesteps 2614.
Path 247 | total_timesteps 2626.
Path 248 | total_timesteps 2632.
Path 249 | total_timesteps 2647.
Path 250 | total_timesteps 2657.
Path 251 | total_timesteps 2671.
Path 252 | total_timesteps 2678.
Path 253 | total_timesteps 2689.
Path 254 | total_timesteps 2701.
Path 255 | total_timesteps 2711.
Path 256 | total_timesteps 2721.
Path 257 | total_timesteps 2729.
Path 258 | total_timesteps 2740.
Path 259 | total_timesteps 2753.
Path 260 | total_timesteps 2763.
Path 261 | total_timesteps 2771.
Path 262 | total_timesteps 2778.
Path 263 | total_timesteps 2788.
Path 264 | total_timesteps 2799.
Path 265 | total_timesteps 2811.
Path 266 | total_timesteps 2820.
Path 267 | total_timesteps 2827.
Path 268 | total_timesteps 2842.
Path 269 | total_timesteps 2849.
Path 270 | total_timesteps 2857.
Path 271 | total_timesteps 2865.
Path 272 | total_timesteps 2877.
Path 273 | total_timesteps 2884.
Path 274 | total_timesteps 2891.
Path 275 | total_timesteps 2898.
Path 276 | total_timesteps 2906.
Path 277 | total_timesteps 2918.
Path 278 | total_timesteps 2928.
Path 279 | total_timesteps 2938.
Path 280 | total_timesteps 2946.
Path 281 | total_timesteps 2957.
Path 282 | total_timesteps 2968.
Path 283 | total_timesteps 2979.
Path 284 | total_timesteps 2990.
Path 285 | total_timesteps 2999.
Path 286 | total_timesteps 3006.
Path 287 | total_timesteps 3016.
Path 288 | total_timesteps 3026.
Path 289 | total_timesteps 3033.
Path 290 | total_timesteps 3042.
Path 291 | total_timesteps 3053.
Path 292 | total_timesteps 3062.
Path 293 | total_timesteps 3076.
Path 294 | total_timesteps 3088.
Path 295 | total_timesteps 3097.
Path 296 | total_timesteps 3106.
Path 297 | total_timesteps 3124.
Path 298 | total_timesteps 3136.
Path 299 | total_timesteps 3144.
Path 300 | total_timesteps 3157.
Path 301 | total_timesteps 3164.
Path 302 | total_timesteps 3181.
Path 303 | total_timesteps 3192.
Path 304 | total_timesteps 3198.
Path 305 | total_timesteps 3208.
Path 306 | total_timesteps 3215.
Path 307 | total_timesteps 3222.
Path 308 | total_timesteps 3234.
Path 309 | total_timesteps 3244.
Path 310 | total_timesteps 3254.
Path 311 | total_timesteps 3262.
Path 312 | total_timesteps 3270.
Path 313 | total_timesteps 3280.
Path 314 | total_timesteps 3297.
Path 315 | total_timesteps 3308.
Path 316 | total_timesteps 3318.
Path 317 | total_timesteps 3337.
Path 318 | total_timesteps 3351.
Path 319 | total_timesteps 3363.
Path 320 | total_timesteps 3371.
Path 321 | total_timesteps 3388.
Path 322 | total_timesteps 3396.
Path 323 | total_timesteps 3406.
Path 324 | total_timesteps 3417.
Path 325 | total_timesteps 3426.
Path 326 | total_timesteps 3436.
Path 327 | total_timesteps 3451.
Path 328 | total_timesteps 3459.
Path 329 | total_timesteps 3468.
Path 330 | total_timesteps 3477.
Path 331 | total_timesteps 3486.
Path 332 | total_timesteps 3494.
Path 333 | total_timesteps 3500.
Path 334 | total_timesteps 3509.
Path 335 | total_timesteps 3520.
Path 336 | total_timesteps 3527.
Path 337 | total_timesteps 3537.
Path 338 | total_timesteps 3552.
Path 339 | total_timesteps 3564.
Path 340 | total_timesteps 3572.
Path 341 | total_timesteps 3579.
Path 342 | total_timesteps 3589.
Path 343 | total_timesteps 3602.
Path 344 | total_timesteps 3615.
Path 345 | total_timesteps 3625.
Path 346 | total_timesteps 3635.
Path 347 | total_timesteps 3644.
Path 348 | total_timesteps 3652.
Path 349 | total_timesteps 3660.
Path 350 | total_timesteps 3668.
Path 351 | total_timesteps 3679.
Path 352 | total_timesteps 3686.
Path 353 | total_timesteps 3693.
Path 354 | total_timesteps 3704.
Path 355 | total_timesteps 3714.
Path 356 | total_timesteps 3726.
Path 357 | total_timesteps 3738.
Path 358 | total_timesteps 3748.
Path 359 | total_timesteps 3757.
Path 360 | total_timesteps 3768.
Path 361 | total_timesteps 3778.
Path 362 | total_timesteps 3786.
Path 363 | total_timesteps 3794.
Path 364 | total_timesteps 3815.
Path 365 | total_timesteps 3824.
Path 366 | total_timesteps 3834.
Path 367 | total_timesteps 3846.
Path 368 | total_timesteps 3855.
Path 369 | total_timesteps 3863.
Path 370 | total_timesteps 3875.
Path 371 | total_timesteps 3882.
Path 372 | total_timesteps 3892.
Path 373 | total_timesteps 3901.
Path 374 | total_timesteps 3914.
Path 375 | total_timesteps 3922.
Path 376 | total_timesteps 3934.
Path 377 | total_timesteps 3945.
Path 378 | total_timesteps 3965.
Path 379 | total_timesteps 3974.
Path 380 | total_timesteps 3984.
Path 381 | total_timesteps 3995.
Path 382 | total_timesteps 4004.
Path 383 | total_timesteps 4012.
Path 384 | total_timesteps 4021.
Path 385 | total_timesteps 4038.
Path 386 | total_timesteps 4047.
Path 387 | total_timesteps 4058.
Path 388 | total_timesteps 4069.
Path 389 | total_timesteps 4086.
Path 390 | total_timesteps 4095.
Path 391 | total_timesteps 4103.
Path 392 | total_timesteps 4113.
Path 393 | total_timesteps 4126.
Path 394 | total_timesteps 4137.
Path 395 | total_timesteps 4151.
Path 396 | total_timesteps 4166.
Path 397 | total_timesteps 4182.
Path 398 | total_timesteps 4192.
Path 399 | total_timesteps 4202.
Path 400 | total_timesteps 4220.
Path 401 | total_timesteps 4231.
Path 402 | total_timesteps 4241.
Path 403 | total_timesteps 4256.
Path 404 | total_timesteps 4267.
Path 405 | total_timesteps 4282.
Path 406 | total_timesteps 4290.
Path 407 | total_timesteps 4297.
Path 408 | total_timesteps 4307.
Path 409 | total_timesteps 4314.
Path 410 | total_timesteps 4323.
Path 411 | total_timesteps 4336.
Path 412 | total_timesteps 4357.
Path 413 | total_timesteps 4365.
Path 414 | total_timesteps 4374.
Path 415 | total_timesteps 4387.
Path 416 | total_timesteps 4396.
Path 417 | total_timesteps 4407.
Path 418 | total_timesteps 4414.
Path 419 | total_timesteps 4429.
Path 420 | total_timesteps 4439.
Path 421 | total_timesteps 4447.
Path 422 | total_timesteps 4458.
Path 423 | total_timesteps 4466.
Path 424 | total_timesteps 4475.
Path 425 | total_timesteps 4483.
Path 426 | total_timesteps 4492.
Path 427 | total_timesteps 4507.
Path 428 | total_timesteps 4518.
Path 429 | total_timesteps 4531.
Path 430 | total_timesteps 4543.
Path 431 | total_timesteps 4552.
Path 432 | total_timesteps 4559.
Path 433 | total_timesteps 4568.
Path 434 | total_timesteps 4577.
Path 435 | total_timesteps 4584.
Path 436 | total_timesteps 4596.
Path 437 | total_timesteps 4606.
Path 438 | total_timesteps 4627.
Path 439 | total_timesteps 4634.
Path 440 | total_timesteps 4644.
Path 441 | total_timesteps 4653.
Path 442 | total_timesteps 4668.
Path 443 | total_timesteps 4679.
Path 444 | total_timesteps 4688.
Path 445 | total_timesteps 4699.
Path 446 | total_timesteps 4711.
Path 447 | total_timesteps 4719.
Path 448 | total_timesteps 4726.
Path 449 | total_timesteps 4733.
Path 450 | total_timesteps 4742.
Path 451 | total_timesteps 4752.
Path 452 | total_timesteps 4772.
Path 453 | total_timesteps 4789.
Path 454 | total_timesteps 4797.
Path 455 | total_timesteps 4804.
Path 456 | total_timesteps 4814.
Path 457 | total_timesteps 4832.
Path 458 | total_timesteps 4839.
Path 459 | total_timesteps 4848.
Path 460 | total_timesteps 4856.
Path 461 | total_timesteps 4864.
Path 462 | total_timesteps 4881.
Path 463 | total_timesteps 4893.
Path 464 | total_timesteps 4901.
Path 465 | total_timesteps 4913.
Path 466 | total_timesteps 4936.
Path 467 | total_timesteps 4943.
Path 468 | total_timesteps 4954.
Path 469 | total_timesteps 4963.
Path 470 | total_timesteps 4975.
Path 471 | total_timesteps 4985.
Path 472 | total_timesteps 4995.
Path 473 | total_timesteps 5009.
Path 474 | total_timesteps 5017.
Path 475 | total_timesteps 5027.
Path 476 | total_timesteps 5040.
Path 477 | total_timesteps 5050.
Path 478 | total_timesteps 5067.
Path 479 | total_timesteps 5077.
Path 480 | total_timesteps 5086.
Path 481 | total_timesteps 5107.
Path 482 | total_timesteps 5120.
Path 483 | total_timesteps 5130.
Path 484 | total_timesteps 5146.
Path 485 | total_timesteps 5153.
Path 486 | total_timesteps 5161.
Path 487 | total_timesteps 5168.
Path 488 | total_timesteps 5175.
Path 489 | total_timesteps 5183.
Path 490 | total_timesteps 5190.
Path 491 | total_timesteps 5208.
Path 492 | total_timesteps 5218.
Path 493 | total_timesteps 5229.
Path 494 | total_timesteps 5236.
Path 495 | total_timesteps 5245.
Path 496 | total_timesteps 5253.
Path 497 | total_timesteps 5264.
Path 498 | total_timesteps 5278.
Path 499 | total_timesteps 5285.
Path 500 | total_timesteps 5296.
Path 501 | total_timesteps 5303.
Path 502 | total_timesteps 5311.
Path 503 | total_timesteps 5318.
Path 504 | total_timesteps 5331.
Path 505 | total_timesteps 5338.
Path 506 | total_timesteps 5354.
Path 507 | total_timesteps 5365.
Path 508 | total_timesteps 5380.
Path 509 | total_timesteps 5392.
Path 510 | total_timesteps 5402.
Path 511 | total_timesteps 5411.
Path 512 | total_timesteps 5418.
Path 513 | total_timesteps 5429.
Path 514 | total_timesteps 5438.
Path 515 | total_timesteps 5449.
Path 516 | total_timesteps 5458.
Path 517 | total_timesteps 5484.
Path 518 | total_timesteps 5495.
Path 519 | total_timesteps 5512.
Path 520 | total_timesteps 5521.
Path 521 | total_timesteps 5532.
Path 522 | total_timesteps 5542.
Path 523 | total_timesteps 5550.
Path 524 | total_timesteps 5559.
Path 525 | total_timesteps 5567.
Path 526 | total_timesteps 5579.
Path 527 | total_timesteps 5589.
Path 528 | total_timesteps 5616.
Path 529 | total_timesteps 5626.
Path 530 | total_timesteps 5639.
Path 531 | total_timesteps 5647.
Path 532 | total_timesteps 5655.
Path 533 | total_timesteps 5665.
Path 534 | total_timesteps 5677.
Path 535 | total_timesteps 5687.
Path 536 | total_timesteps 5695.
Path 537 | total_timesteps 5705.
Path 538 | total_timesteps 5714.
Path 539 | total_timesteps 5722.
Path 540 | total_timesteps 5731.
Path 541 | total_timesteps 5743.
Path 542 | total_timesteps 5756.
Path 543 | total_timesteps 5766.
Path 544 | total_timesteps 5775.
Path 545 | total_timesteps 5786.
Path 546 | total_timesteps 5792.
Path 547 | total_timesteps 5807.
Path 548 | total_timesteps 5815.
Path 549 | total_timesteps 5824.
Path 550 | total_timesteps 5832.
Path 551 | total_timesteps 5840.
Path 552 | total_timesteps 5851.
Path 553 | total_timesteps 5862.
Path 554 | total_timesteps 5870.
Path 555 | total_timesteps 5882.
Path 556 | total_timesteps 5891.
Path 557 | total_timesteps 5898.
Path 558 | total_timesteps 5908.
Path 559 | total_timesteps 5916.
Path 560 | total_timesteps 5925.
Path 561 | total_timesteps 5948.
Path 562 | total_timesteps 5956.
Path 563 | total_timesteps 5964.
Path 564 | total_timesteps 5972.
Path 565 | total_timesteps 5982.
Path 566 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.22    |
| Iteration     | 8        |
| MaximumReturn | 1.72     |
| MinimumReturn | -22.8    |
| TotalSamples  | 40028    |
----------------------------
itr #9 | 
Fitting dynamics.
Validation loss = 0.009990056976675987
Validation loss = 0.011291625909507275
Validation loss = 0.010227937251329422
Validation loss = 0.010190906003117561
Validation loss = 0.008933660574257374
Validation loss = 0.009348890744149685
Validation loss = 0.009819087572395802
Validation loss = 0.00944376178085804
Validation loss = 0.008836379274725914
Validation loss = 0.009282425977289677
Validation loss = 0.008530883118510246
Validation loss = 0.008812591433525085
Validation loss = 0.009821969084441662
Validation loss = 0.009365489706397057
Validation loss = 0.008670670911669731
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 55.
Path 6 | total_timesteps 63.
Path 7 | total_timesteps 71.
Path 8 | total_timesteps 80.
Path 9 | total_timesteps 92.
Path 10 | total_timesteps 106.
Path 11 | total_timesteps 115.
Path 12 | total_timesteps 131.
Path 13 | total_timesteps 144.
Path 14 | total_timesteps 156.
Path 15 | total_timesteps 167.
Path 16 | total_timesteps 176.
Path 17 | total_timesteps 193.
Path 18 | total_timesteps 202.
Path 19 | total_timesteps 210.
Path 20 | total_timesteps 216.
Path 21 | total_timesteps 229.
Path 22 | total_timesteps 239.
Path 23 | total_timesteps 252.
Path 24 | total_timesteps 260.
Path 25 | total_timesteps 270.
Path 26 | total_timesteps 280.
Path 27 | total_timesteps 296.
Path 28 | total_timesteps 305.
Path 29 | total_timesteps 312.
Path 30 | total_timesteps 322.
Path 31 | total_timesteps 330.
Path 32 | total_timesteps 344.
Path 33 | total_timesteps 354.
Path 34 | total_timesteps 377.
Path 35 | total_timesteps 387.
Path 36 | total_timesteps 398.
Path 37 | total_timesteps 410.
Path 38 | total_timesteps 426.
Path 39 | total_timesteps 438.
Path 40 | total_timesteps 455.
Path 41 | total_timesteps 463.
Path 42 | total_timesteps 476.
Path 43 | total_timesteps 484.
Path 44 | total_timesteps 494.
Path 45 | total_timesteps 506.
Path 46 | total_timesteps 514.
Path 47 | total_timesteps 526.
Path 48 | total_timesteps 533.
Path 49 | total_timesteps 545.
Path 50 | total_timesteps 551.
Path 51 | total_timesteps 559.
Path 52 | total_timesteps 568.
Path 53 | total_timesteps 580.
Path 54 | total_timesteps 593.
Path 55 | total_timesteps 604.
Path 56 | total_timesteps 614.
Path 57 | total_timesteps 626.
Path 58 | total_timesteps 646.
Path 59 | total_timesteps 657.
Path 60 | total_timesteps 667.
Path 61 | total_timesteps 682.
Path 62 | total_timesteps 696.
Path 63 | total_timesteps 704.
Path 64 | total_timesteps 715.
Path 65 | total_timesteps 727.
Path 66 | total_timesteps 736.
Path 67 | total_timesteps 751.
Path 68 | total_timesteps 757.
Path 69 | total_timesteps 773.
Path 70 | total_timesteps 794.
Path 71 | total_timesteps 804.
Path 72 | total_timesteps 816.
Path 73 | total_timesteps 837.
Path 74 | total_timesteps 852.
Path 75 | total_timesteps 859.
Path 76 | total_timesteps 872.
Path 77 | total_timesteps 881.
Path 78 | total_timesteps 896.
Path 79 | total_timesteps 905.
Path 80 | total_timesteps 916.
Path 81 | total_timesteps 929.
Path 82 | total_timesteps 941.
Path 83 | total_timesteps 951.
Path 84 | total_timesteps 964.
Path 85 | total_timesteps 979.
Path 86 | total_timesteps 989.
Path 87 | total_timesteps 1000.
Path 88 | total_timesteps 1017.
Path 89 | total_timesteps 1028.
Path 90 | total_timesteps 1037.
Path 91 | total_timesteps 1045.
Path 92 | total_timesteps 1059.
Path 93 | total_timesteps 1065.
Path 94 | total_timesteps 1078.
Path 95 | total_timesteps 1094.
Path 96 | total_timesteps 1102.
Path 97 | total_timesteps 1114.
Path 98 | total_timesteps 1124.
Path 99 | total_timesteps 1132.
Path 100 | total_timesteps 1154.
Path 101 | total_timesteps 1168.
Path 102 | total_timesteps 1181.
Path 103 | total_timesteps 1191.
Path 104 | total_timesteps 1205.
Path 105 | total_timesteps 1213.
Path 106 | total_timesteps 1222.
Path 107 | total_timesteps 1234.
Path 108 | total_timesteps 1242.
Path 109 | total_timesteps 1256.
Path 110 | total_timesteps 1264.
Path 111 | total_timesteps 1271.
Path 112 | total_timesteps 1278.
Path 113 | total_timesteps 1301.
Path 114 | total_timesteps 1311.
Path 115 | total_timesteps 1321.
Path 116 | total_timesteps 1334.
Path 117 | total_timesteps 1344.
Path 118 | total_timesteps 1352.
Path 119 | total_timesteps 1361.
Path 120 | total_timesteps 1368.
Path 121 | total_timesteps 1376.
Path 122 | total_timesteps 1391.
Path 123 | total_timesteps 1400.
Path 124 | total_timesteps 1412.
Path 125 | total_timesteps 1421.
Path 126 | total_timesteps 1434.
Path 127 | total_timesteps 1447.
Path 128 | total_timesteps 1454.
Path 129 | total_timesteps 1462.
Path 130 | total_timesteps 1471.
Path 131 | total_timesteps 1482.
Path 132 | total_timesteps 1492.
Path 133 | total_timesteps 1501.
Path 134 | total_timesteps 1511.
Path 135 | total_timesteps 1541.
Path 136 | total_timesteps 1554.
Path 137 | total_timesteps 1562.
Path 138 | total_timesteps 1569.
Path 139 | total_timesteps 1580.
Path 140 | total_timesteps 1588.
Path 141 | total_timesteps 1594.
Path 142 | total_timesteps 1609.
Path 143 | total_timesteps 1621.
Path 144 | total_timesteps 1629.
Path 145 | total_timesteps 1641.
Path 146 | total_timesteps 1652.
Path 147 | total_timesteps 1661.
Path 148 | total_timesteps 1669.
Path 149 | total_timesteps 1678.
Path 150 | total_timesteps 1685.
Path 151 | total_timesteps 1698.
Path 152 | total_timesteps 1707.
Path 153 | total_timesteps 1717.
Path 154 | total_timesteps 1728.
Path 155 | total_timesteps 1747.
Path 156 | total_timesteps 1755.
Path 157 | total_timesteps 1771.
Path 158 | total_timesteps 1782.
Path 159 | total_timesteps 1796.
Path 160 | total_timesteps 1817.
Path 161 | total_timesteps 1825.
Path 162 | total_timesteps 1835.
Path 163 | total_timesteps 1846.
Path 164 | total_timesteps 1863.
Path 165 | total_timesteps 1875.
Path 166 | total_timesteps 1881.
Path 167 | total_timesteps 1892.
Path 168 | total_timesteps 1906.
Path 169 | total_timesteps 1916.
Path 170 | total_timesteps 1925.
Path 171 | total_timesteps 1939.
Path 172 | total_timesteps 1950.
Path 173 | total_timesteps 1961.
Path 174 | total_timesteps 1973.
Path 175 | total_timesteps 1981.
Path 176 | total_timesteps 1993.
Path 177 | total_timesteps 2006.
Path 178 | total_timesteps 2016.
Path 179 | total_timesteps 2024.
Path 180 | total_timesteps 2036.
Path 181 | total_timesteps 2057.
Path 182 | total_timesteps 2067.
Path 183 | total_timesteps 2077.
Path 184 | total_timesteps 2087.
Path 185 | total_timesteps 2099.
Path 186 | total_timesteps 2112.
Path 187 | total_timesteps 2126.
Path 188 | total_timesteps 2136.
Path 189 | total_timesteps 2149.
Path 190 | total_timesteps 2163.
Path 191 | total_timesteps 2175.
Path 192 | total_timesteps 2185.
Path 193 | total_timesteps 2197.
Path 194 | total_timesteps 2205.
Path 195 | total_timesteps 2228.
Path 196 | total_timesteps 2238.
Path 197 | total_timesteps 2247.
Path 198 | total_timesteps 2260.
Path 199 | total_timesteps 2276.
Path 200 | total_timesteps 2288.
Path 201 | total_timesteps 2302.
Path 202 | total_timesteps 2314.
Path 203 | total_timesteps 2324.
Path 204 | total_timesteps 2336.
Path 205 | total_timesteps 2348.
Path 206 | total_timesteps 2355.
Path 207 | total_timesteps 2362.
Path 208 | total_timesteps 2375.
Path 209 | total_timesteps 2385.
Path 210 | total_timesteps 2402.
Path 211 | total_timesteps 2408.
Path 212 | total_timesteps 2419.
Path 213 | total_timesteps 2452.
Path 214 | total_timesteps 2468.
Path 215 | total_timesteps 2477.
Path 216 | total_timesteps 2491.
Path 217 | total_timesteps 2500.
Path 218 | total_timesteps 2508.
Path 219 | total_timesteps 2529.
Path 220 | total_timesteps 2539.
Path 221 | total_timesteps 2548.
Path 222 | total_timesteps 2555.
Path 223 | total_timesteps 2564.
Path 224 | total_timesteps 2575.
Path 225 | total_timesteps 2586.
Path 226 | total_timesteps 2603.
Path 227 | total_timesteps 2615.
Path 228 | total_timesteps 2628.
Path 229 | total_timesteps 2655.
Path 230 | total_timesteps 2663.
Path 231 | total_timesteps 2683.
Path 232 | total_timesteps 2700.
Path 233 | total_timesteps 2712.
Path 234 | total_timesteps 2726.
Path 235 | total_timesteps 2746.
Path 236 | total_timesteps 2763.
Path 237 | total_timesteps 2772.
Path 238 | total_timesteps 2783.
Path 239 | total_timesteps 2798.
Path 240 | total_timesteps 2816.
Path 241 | total_timesteps 2832.
Path 242 | total_timesteps 2844.
Path 243 | total_timesteps 2851.
Path 244 | total_timesteps 2859.
Path 245 | total_timesteps 2873.
Path 246 | total_timesteps 2887.
Path 247 | total_timesteps 2898.
Path 248 | total_timesteps 2908.
Path 249 | total_timesteps 2919.
Path 250 | total_timesteps 2941.
Path 251 | total_timesteps 2949.
Path 252 | total_timesteps 2963.
Path 253 | total_timesteps 2974.
Path 254 | total_timesteps 2985.
Path 255 | total_timesteps 2994.
Path 256 | total_timesteps 3003.
Path 257 | total_timesteps 3011.
Path 258 | total_timesteps 3022.
Path 259 | total_timesteps 3030.
Path 260 | total_timesteps 3040.
Path 261 | total_timesteps 3047.
Path 262 | total_timesteps 3059.
Path 263 | total_timesteps 3070.
Path 264 | total_timesteps 3087.
Path 265 | total_timesteps 3095.
Path 266 | total_timesteps 3109.
Path 267 | total_timesteps 3117.
Path 268 | total_timesteps 3126.
Path 269 | total_timesteps 3141.
Path 270 | total_timesteps 3151.
Path 271 | total_timesteps 3165.
Path 272 | total_timesteps 3175.
Path 273 | total_timesteps 3189.
Path 274 | total_timesteps 3199.
Path 275 | total_timesteps 3205.
Path 276 | total_timesteps 3217.
Path 277 | total_timesteps 3225.
Path 278 | total_timesteps 3235.
Path 279 | total_timesteps 3249.
Path 280 | total_timesteps 3260.
Path 281 | total_timesteps 3271.
Path 282 | total_timesteps 3279.
Path 283 | total_timesteps 3289.
Path 284 | total_timesteps 3299.
Path 285 | total_timesteps 3317.
Path 286 | total_timesteps 3326.
Path 287 | total_timesteps 3337.
Path 288 | total_timesteps 3349.
Path 289 | total_timesteps 3358.
Path 290 | total_timesteps 3375.
Path 291 | total_timesteps 3386.
Path 292 | total_timesteps 3399.
Path 293 | total_timesteps 3409.
Path 294 | total_timesteps 3421.
Path 295 | total_timesteps 3437.
Path 296 | total_timesteps 3445.
Path 297 | total_timesteps 3454.
Path 298 | total_timesteps 3471.
Path 299 | total_timesteps 3486.
Path 300 | total_timesteps 3499.
Path 301 | total_timesteps 3508.
Path 302 | total_timesteps 3520.
Path 303 | total_timesteps 3528.
Path 304 | total_timesteps 3542.
Path 305 | total_timesteps 3552.
Path 306 | total_timesteps 3567.
Path 307 | total_timesteps 3575.
Path 308 | total_timesteps 3594.
Path 309 | total_timesteps 3603.
Path 310 | total_timesteps 3613.
Path 311 | total_timesteps 3629.
Path 312 | total_timesteps 3638.
Path 313 | total_timesteps 3649.
Path 314 | total_timesteps 3661.
Path 315 | total_timesteps 3675.
Path 316 | total_timesteps 3687.
Path 317 | total_timesteps 3699.
Path 318 | total_timesteps 3707.
Path 319 | total_timesteps 3714.
Path 320 | total_timesteps 3721.
Path 321 | total_timesteps 3734.
Path 322 | total_timesteps 3742.
Path 323 | total_timesteps 3749.
Path 324 | total_timesteps 3761.
Path 325 | total_timesteps 3771.
Path 326 | total_timesteps 3780.
Path 327 | total_timesteps 3795.
Path 328 | total_timesteps 3804.
Path 329 | total_timesteps 3823.
Path 330 | total_timesteps 3857.
Path 331 | total_timesteps 3865.
Path 332 | total_timesteps 3874.
Path 333 | total_timesteps 3882.
Path 334 | total_timesteps 3891.
Path 335 | total_timesteps 3900.
Path 336 | total_timesteps 3909.
Path 337 | total_timesteps 3922.
Path 338 | total_timesteps 3933.
Path 339 | total_timesteps 3952.
Path 340 | total_timesteps 3965.
Path 341 | total_timesteps 3973.
Path 342 | total_timesteps 3984.
Path 343 | total_timesteps 3993.
Path 344 | total_timesteps 4002.
Path 345 | total_timesteps 4013.
Path 346 | total_timesteps 4026.
Path 347 | total_timesteps 4038.
Path 348 | total_timesteps 4049.
Path 349 | total_timesteps 4075.
Path 350 | total_timesteps 4084.
Path 351 | total_timesteps 4096.
Path 352 | total_timesteps 4105.
Path 353 | total_timesteps 4113.
Path 354 | total_timesteps 4123.
Path 355 | total_timesteps 4132.
Path 356 | total_timesteps 4139.
Path 357 | total_timesteps 4147.
Path 358 | total_timesteps 4159.
Path 359 | total_timesteps 4171.
Path 360 | total_timesteps 4179.
Path 361 | total_timesteps 4188.
Path 362 | total_timesteps 4201.
Path 363 | total_timesteps 4208.
Path 364 | total_timesteps 4223.
Path 365 | total_timesteps 4238.
Path 366 | total_timesteps 4249.
Path 367 | total_timesteps 4260.
Path 368 | total_timesteps 4269.
Path 369 | total_timesteps 4280.
Path 370 | total_timesteps 4288.
Path 371 | total_timesteps 4300.
Path 372 | total_timesteps 4317.
Path 373 | total_timesteps 4330.
Path 374 | total_timesteps 4341.
Path 375 | total_timesteps 4353.
Path 376 | total_timesteps 4363.
Path 377 | total_timesteps 4385.
Path 378 | total_timesteps 4393.
Path 379 | total_timesteps 4402.
Path 380 | total_timesteps 4416.
Path 381 | total_timesteps 4431.
Path 382 | total_timesteps 4448.
Path 383 | total_timesteps 4457.
Path 384 | total_timesteps 4465.
Path 385 | total_timesteps 4473.
Path 386 | total_timesteps 4486.
Path 387 | total_timesteps 4500.
Path 388 | total_timesteps 4518.
Path 389 | total_timesteps 4530.
Path 390 | total_timesteps 4547.
Path 391 | total_timesteps 4567.
Path 392 | total_timesteps 4577.
Path 393 | total_timesteps 4587.
Path 394 | total_timesteps 4595.
Path 395 | total_timesteps 4609.
Path 396 | total_timesteps 4625.
Path 397 | total_timesteps 4637.
Path 398 | total_timesteps 4647.
Path 399 | total_timesteps 4660.
Path 400 | total_timesteps 4667.
Path 401 | total_timesteps 4675.
Path 402 | total_timesteps 4688.
Path 403 | total_timesteps 4695.
Path 404 | total_timesteps 4709.
Path 405 | total_timesteps 4716.
Path 406 | total_timesteps 4723.
Path 407 | total_timesteps 4737.
Path 408 | total_timesteps 4749.
Path 409 | total_timesteps 4760.
Path 410 | total_timesteps 4769.
Path 411 | total_timesteps 4776.
Path 412 | total_timesteps 4791.
Path 413 | total_timesteps 4803.
Path 414 | total_timesteps 4819.
Path 415 | total_timesteps 4829.
Path 416 | total_timesteps 4846.
Path 417 | total_timesteps 4856.
Path 418 | total_timesteps 4866.
Path 419 | total_timesteps 4877.
Path 420 | total_timesteps 4884.
Path 421 | total_timesteps 4899.
Path 422 | total_timesteps 4908.
Path 423 | total_timesteps 4917.
Path 424 | total_timesteps 4927.
Path 425 | total_timesteps 4941.
Path 426 | total_timesteps 4964.
Path 427 | total_timesteps 4973.
Path 428 | total_timesteps 4987.
Path 429 | total_timesteps 4994.
Path 430 | total_timesteps 5005.
Path 431 | total_timesteps 5015.
Path 432 | total_timesteps 5022.
Path 433 | total_timesteps 5031.
Path 434 | total_timesteps 5041.
Path 435 | total_timesteps 5048.
Path 436 | total_timesteps 5056.
Path 437 | total_timesteps 5075.
Path 438 | total_timesteps 5086.
Path 439 | total_timesteps 5092.
Path 440 | total_timesteps 5108.
Path 441 | total_timesteps 5121.
Path 442 | total_timesteps 5134.
Path 443 | total_timesteps 5144.
Path 444 | total_timesteps 5154.
Path 445 | total_timesteps 5165.
Path 446 | total_timesteps 5189.
Path 447 | total_timesteps 5204.
Path 448 | total_timesteps 5217.
Path 449 | total_timesteps 5225.
Path 450 | total_timesteps 5236.
Path 451 | total_timesteps 5248.
Path 452 | total_timesteps 5256.
Path 453 | total_timesteps 5279.
Path 454 | total_timesteps 5288.
Path 455 | total_timesteps 5300.
Path 456 | total_timesteps 5308.
Path 457 | total_timesteps 5317.
Path 458 | total_timesteps 5329.
Path 459 | total_timesteps 5344.
Path 460 | total_timesteps 5352.
Path 461 | total_timesteps 5363.
Path 462 | total_timesteps 5374.
Path 463 | total_timesteps 5384.
Path 464 | total_timesteps 5394.
Path 465 | total_timesteps 5402.
Path 466 | total_timesteps 5410.
Path 467 | total_timesteps 5418.
Path 468 | total_timesteps 5427.
Path 469 | total_timesteps 5437.
Path 470 | total_timesteps 5445.
Path 471 | total_timesteps 5452.
Path 472 | total_timesteps 5461.
Path 473 | total_timesteps 5484.
Path 474 | total_timesteps 5492.
Path 475 | total_timesteps 5501.
Path 476 | total_timesteps 5508.
Path 477 | total_timesteps 5516.
Path 478 | total_timesteps 5529.
Path 479 | total_timesteps 5540.
Path 480 | total_timesteps 5552.
Path 481 | total_timesteps 5560.
Path 482 | total_timesteps 5568.
Path 483 | total_timesteps 5584.
Path 484 | total_timesteps 5591.
Path 485 | total_timesteps 5603.
Path 486 | total_timesteps 5617.
Path 487 | total_timesteps 5630.
Path 488 | total_timesteps 5644.
Path 489 | total_timesteps 5660.
Path 490 | total_timesteps 5670.
Path 491 | total_timesteps 5680.
Path 492 | total_timesteps 5694.
Path 493 | total_timesteps 5705.
Path 494 | total_timesteps 5713.
Path 495 | total_timesteps 5729.
Path 496 | total_timesteps 5737.
Path 497 | total_timesteps 5745.
Path 498 | total_timesteps 5759.
Path 499 | total_timesteps 5769.
Path 500 | total_timesteps 5784.
Path 501 | total_timesteps 5797.
Path 502 | total_timesteps 5809.
Path 503 | total_timesteps 5825.
Path 504 | total_timesteps 5842.
Path 505 | total_timesteps 5852.
Path 506 | total_timesteps 5865.
Path 507 | total_timesteps 5874.
Path 508 | total_timesteps 5883.
Path 509 | total_timesteps 5894.
Path 510 | total_timesteps 5902.
Path 511 | total_timesteps 5912.
Path 512 | total_timesteps 5924.
Path 513 | total_timesteps 5932.
Path 514 | total_timesteps 5943.
Path 515 | total_timesteps 5953.
Path 516 | total_timesteps 5961.
Path 517 | total_timesteps 5969.
Path 518 | total_timesteps 5979.
Path 519 | total_timesteps 5989.
Path 520 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.73    |
| Iteration     | 9        |
| MaximumReturn | 4.08     |
| MinimumReturn | -23.9    |
| TotalSamples  | 44033    |
----------------------------
itr #10 | 
Fitting dynamics.
Validation loss = 0.008655030280351639
Validation loss = 0.008870158344507217
Validation loss = 0.008368664421141148
Validation loss = 0.009036346338689327
Validation loss = 0.011386444792151451
Validation loss = 0.008556434884667397
Validation loss = 0.008688068948686123
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 29.
Path 3 | total_timesteps 40.
Path 4 | total_timesteps 54.
Path 5 | total_timesteps 65.
Path 6 | total_timesteps 76.
Path 7 | total_timesteps 87.
Path 8 | total_timesteps 97.
Path 9 | total_timesteps 105.
Path 10 | total_timesteps 113.
Path 11 | total_timesteps 132.
Path 12 | total_timesteps 142.
Path 13 | total_timesteps 152.
Path 14 | total_timesteps 159.
Path 15 | total_timesteps 173.
Path 16 | total_timesteps 196.
Path 17 | total_timesteps 218.
Path 18 | total_timesteps 233.
Path 19 | total_timesteps 244.
Path 20 | total_timesteps 253.
Path 21 | total_timesteps 266.
Path 22 | total_timesteps 273.
Path 23 | total_timesteps 282.
Path 24 | total_timesteps 291.
Path 25 | total_timesteps 298.
Path 26 | total_timesteps 311.
Path 27 | total_timesteps 323.
Path 28 | total_timesteps 335.
Path 29 | total_timesteps 346.
Path 30 | total_timesteps 354.
Path 31 | total_timesteps 364.
Path 32 | total_timesteps 371.
Path 33 | total_timesteps 380.
Path 34 | total_timesteps 396.
Path 35 | total_timesteps 406.
Path 36 | total_timesteps 414.
Path 37 | total_timesteps 424.
Path 38 | total_timesteps 433.
Path 39 | total_timesteps 443.
Path 40 | total_timesteps 454.
Path 41 | total_timesteps 463.
Path 42 | total_timesteps 474.
Path 43 | total_timesteps 486.
Path 44 | total_timesteps 506.
Path 45 | total_timesteps 515.
Path 46 | total_timesteps 523.
Path 47 | total_timesteps 532.
Path 48 | total_timesteps 541.
Path 49 | total_timesteps 549.
Path 50 | total_timesteps 557.
Path 51 | total_timesteps 568.
Path 52 | total_timesteps 581.
Path 53 | total_timesteps 590.
Path 54 | total_timesteps 597.
Path 55 | total_timesteps 608.
Path 56 | total_timesteps 621.
Path 57 | total_timesteps 631.
Path 58 | total_timesteps 646.
Path 59 | total_timesteps 660.
Path 60 | total_timesteps 669.
Path 61 | total_timesteps 679.
Path 62 | total_timesteps 693.
Path 63 | total_timesteps 700.
Path 64 | total_timesteps 713.
Path 65 | total_timesteps 722.
Path 66 | total_timesteps 731.
Path 67 | total_timesteps 744.
Path 68 | total_timesteps 752.
Path 69 | total_timesteps 759.
Path 70 | total_timesteps 769.
Path 71 | total_timesteps 778.
Path 72 | total_timesteps 787.
Path 73 | total_timesteps 802.
Path 74 | total_timesteps 831.
Path 75 | total_timesteps 851.
Path 76 | total_timesteps 859.
Path 77 | total_timesteps 867.
Path 78 | total_timesteps 882.
Path 79 | total_timesteps 893.
Path 80 | total_timesteps 900.
Path 81 | total_timesteps 910.
Path 82 | total_timesteps 931.
Path 83 | total_timesteps 947.
Path 84 | total_timesteps 956.
Path 85 | total_timesteps 966.
Path 86 | total_timesteps 974.
Path 87 | total_timesteps 984.
Path 88 | total_timesteps 995.
Path 89 | total_timesteps 1006.
Path 90 | total_timesteps 1018.
Path 91 | total_timesteps 1026.
Path 92 | total_timesteps 1037.
Path 93 | total_timesteps 1047.
Path 94 | total_timesteps 1057.
Path 95 | total_timesteps 1069.
Path 96 | total_timesteps 1079.
Path 97 | total_timesteps 1090.
Path 98 | total_timesteps 1101.
Path 99 | total_timesteps 1111.
Path 100 | total_timesteps 1119.
Path 101 | total_timesteps 1134.
Path 102 | total_timesteps 1143.
Path 103 | total_timesteps 1150.
Path 104 | total_timesteps 1159.
Path 105 | total_timesteps 1167.
Path 106 | total_timesteps 1185.
Path 107 | total_timesteps 1201.
Path 108 | total_timesteps 1211.
Path 109 | total_timesteps 1220.
Path 110 | total_timesteps 1231.
Path 111 | total_timesteps 1239.
Path 112 | total_timesteps 1246.
Path 113 | total_timesteps 1257.
Path 114 | total_timesteps 1279.
Path 115 | total_timesteps 1287.
Path 116 | total_timesteps 1299.
Path 117 | total_timesteps 1318.
Path 118 | total_timesteps 1325.
Path 119 | total_timesteps 1332.
Path 120 | total_timesteps 1339.
Path 121 | total_timesteps 1349.
Path 122 | total_timesteps 1362.
Path 123 | total_timesteps 1371.
Path 124 | total_timesteps 1380.
Path 125 | total_timesteps 1396.
Path 126 | total_timesteps 1408.
Path 127 | total_timesteps 1418.
Path 128 | total_timesteps 1430.
Path 129 | total_timesteps 1438.
Path 130 | total_timesteps 1449.
Path 131 | total_timesteps 1457.
Path 132 | total_timesteps 1473.
Path 133 | total_timesteps 1483.
Path 134 | total_timesteps 1493.
Path 135 | total_timesteps 1504.
Path 136 | total_timesteps 1520.
Path 137 | total_timesteps 1534.
Path 138 | total_timesteps 1543.
Path 139 | total_timesteps 1550.
Path 140 | total_timesteps 1559.
Path 141 | total_timesteps 1568.
Path 142 | total_timesteps 1582.
Path 143 | total_timesteps 1600.
Path 144 | total_timesteps 1613.
Path 145 | total_timesteps 1625.
Path 146 | total_timesteps 1639.
Path 147 | total_timesteps 1646.
Path 148 | total_timesteps 1657.
Path 149 | total_timesteps 1680.
Path 150 | total_timesteps 1692.
Path 151 | total_timesteps 1701.
Path 152 | total_timesteps 1710.
Path 153 | total_timesteps 1718.
Path 154 | total_timesteps 1727.
Path 155 | total_timesteps 1735.
Path 156 | total_timesteps 1746.
Path 157 | total_timesteps 1755.
Path 158 | total_timesteps 1765.
Path 159 | total_timesteps 1779.
Path 160 | total_timesteps 1787.
Path 161 | total_timesteps 1801.
Path 162 | total_timesteps 1808.
Path 163 | total_timesteps 1821.
Path 164 | total_timesteps 1833.
Path 165 | total_timesteps 1843.
Path 166 | total_timesteps 1853.
Path 167 | total_timesteps 1871.
Path 168 | total_timesteps 1880.
Path 169 | total_timesteps 1893.
Path 170 | total_timesteps 1901.
Path 171 | total_timesteps 1908.
Path 172 | total_timesteps 1919.
Path 173 | total_timesteps 1929.
Path 174 | total_timesteps 1940.
Path 175 | total_timesteps 1949.
Path 176 | total_timesteps 1960.
Path 177 | total_timesteps 1970.
Path 178 | total_timesteps 1978.
Path 179 | total_timesteps 1990.
Path 180 | total_timesteps 2001.
Path 181 | total_timesteps 2009.
Path 182 | total_timesteps 2017.
Path 183 | total_timesteps 2031.
Path 184 | total_timesteps 2040.
Path 185 | total_timesteps 2051.
Path 186 | total_timesteps 2072.
Path 187 | total_timesteps 2082.
Path 188 | total_timesteps 2091.
Path 189 | total_timesteps 2102.
Path 190 | total_timesteps 2111.
Path 191 | total_timesteps 2127.
Path 192 | total_timesteps 2136.
Path 193 | total_timesteps 2149.
Path 194 | total_timesteps 2161.
Path 195 | total_timesteps 2174.
Path 196 | total_timesteps 2181.
Path 197 | total_timesteps 2191.
Path 198 | total_timesteps 2199.
Path 199 | total_timesteps 2213.
Path 200 | total_timesteps 2228.
Path 201 | total_timesteps 2236.
Path 202 | total_timesteps 2245.
Path 203 | total_timesteps 2255.
Path 204 | total_timesteps 2266.
Path 205 | total_timesteps 2276.
Path 206 | total_timesteps 2289.
Path 207 | total_timesteps 2298.
Path 208 | total_timesteps 2306.
Path 209 | total_timesteps 2315.
Path 210 | total_timesteps 2330.
Path 211 | total_timesteps 2343.
Path 212 | total_timesteps 2352.
Path 213 | total_timesteps 2362.
Path 214 | total_timesteps 2370.
Path 215 | total_timesteps 2380.
Path 216 | total_timesteps 2401.
Path 217 | total_timesteps 2410.
Path 218 | total_timesteps 2424.
Path 219 | total_timesteps 2434.
Path 220 | total_timesteps 2447.
Path 221 | total_timesteps 2459.
Path 222 | total_timesteps 2469.
Path 223 | total_timesteps 2480.
Path 224 | total_timesteps 2490.
Path 225 | total_timesteps 2501.
Path 226 | total_timesteps 2509.
Path 227 | total_timesteps 2518.
Path 228 | total_timesteps 2525.
Path 229 | total_timesteps 2532.
Path 230 | total_timesteps 2542.
Path 231 | total_timesteps 2553.
Path 232 | total_timesteps 2564.
Path 233 | total_timesteps 2578.
Path 234 | total_timesteps 2586.
Path 235 | total_timesteps 2600.
Path 236 | total_timesteps 2608.
Path 237 | total_timesteps 2617.
Path 238 | total_timesteps 2629.
Path 239 | total_timesteps 2637.
Path 240 | total_timesteps 2648.
Path 241 | total_timesteps 2662.
Path 242 | total_timesteps 2679.
Path 243 | total_timesteps 2689.
Path 244 | total_timesteps 2698.
Path 245 | total_timesteps 2708.
Path 246 | total_timesteps 2716.
Path 247 | total_timesteps 2723.
Path 248 | total_timesteps 2731.
Path 249 | total_timesteps 2744.
Path 250 | total_timesteps 2756.
Path 251 | total_timesteps 2771.
Path 252 | total_timesteps 2784.
Path 253 | total_timesteps 2795.
Path 254 | total_timesteps 2802.
Path 255 | total_timesteps 2810.
Path 256 | total_timesteps 2825.
Path 257 | total_timesteps 2838.
Path 258 | total_timesteps 2847.
Path 259 | total_timesteps 2856.
Path 260 | total_timesteps 2863.
Path 261 | total_timesteps 2871.
Path 262 | total_timesteps 2883.
Path 263 | total_timesteps 2891.
Path 264 | total_timesteps 2898.
Path 265 | total_timesteps 2910.
Path 266 | total_timesteps 2920.
Path 267 | total_timesteps 2928.
Path 268 | total_timesteps 2938.
Path 269 | total_timesteps 2948.
Path 270 | total_timesteps 2956.
Path 271 | total_timesteps 2974.
Path 272 | total_timesteps 2982.
Path 273 | total_timesteps 2994.
Path 274 | total_timesteps 3001.
Path 275 | total_timesteps 3009.
Path 276 | total_timesteps 3017.
Path 277 | total_timesteps 3033.
Path 278 | total_timesteps 3040.
Path 279 | total_timesteps 3048.
Path 280 | total_timesteps 3066.
Path 281 | total_timesteps 3076.
Path 282 | total_timesteps 3084.
Path 283 | total_timesteps 3108.
Path 284 | total_timesteps 3116.
Path 285 | total_timesteps 3127.
Path 286 | total_timesteps 3139.
Path 287 | total_timesteps 3148.
Path 288 | total_timesteps 3161.
Path 289 | total_timesteps 3171.
Path 290 | total_timesteps 3177.
Path 291 | total_timesteps 3188.
Path 292 | total_timesteps 3203.
Path 293 | total_timesteps 3225.
Path 294 | total_timesteps 3239.
Path 295 | total_timesteps 3253.
Path 296 | total_timesteps 3266.
Path 297 | total_timesteps 3276.
Path 298 | total_timesteps 3290.
Path 299 | total_timesteps 3304.
Path 300 | total_timesteps 3311.
Path 301 | total_timesteps 3332.
Path 302 | total_timesteps 3342.
Path 303 | total_timesteps 3354.
Path 304 | total_timesteps 3373.
Path 305 | total_timesteps 3386.
Path 306 | total_timesteps 3394.
Path 307 | total_timesteps 3403.
Path 308 | total_timesteps 3415.
Path 309 | total_timesteps 3423.
Path 310 | total_timesteps 3435.
Path 311 | total_timesteps 3453.
Path 312 | total_timesteps 3463.
Path 313 | total_timesteps 3473.
Path 314 | total_timesteps 3481.
Path 315 | total_timesteps 3491.
Path 316 | total_timesteps 3504.
Path 317 | total_timesteps 3526.
Path 318 | total_timesteps 3535.
Path 319 | total_timesteps 3550.
Path 320 | total_timesteps 3560.
Path 321 | total_timesteps 3573.
Path 322 | total_timesteps 3585.
Path 323 | total_timesteps 3594.
Path 324 | total_timesteps 3605.
Path 325 | total_timesteps 3612.
Path 326 | total_timesteps 3621.
Path 327 | total_timesteps 3629.
Path 328 | total_timesteps 3638.
Path 329 | total_timesteps 3645.
Path 330 | total_timesteps 3654.
Path 331 | total_timesteps 3666.
Path 332 | total_timesteps 3680.
Path 333 | total_timesteps 3692.
Path 334 | total_timesteps 3706.
Path 335 | total_timesteps 3716.
Path 336 | total_timesteps 3728.
Path 337 | total_timesteps 3737.
Path 338 | total_timesteps 3747.
Path 339 | total_timesteps 3756.
Path 340 | total_timesteps 3765.
Path 341 | total_timesteps 3776.
Path 342 | total_timesteps 3790.
Path 343 | total_timesteps 3812.
Path 344 | total_timesteps 3827.
Path 345 | total_timesteps 3841.
Path 346 | total_timesteps 3856.
Path 347 | total_timesteps 3863.
Path 348 | total_timesteps 3870.
Path 349 | total_timesteps 3878.
Path 350 | total_timesteps 3889.
Path 351 | total_timesteps 3901.
Path 352 | total_timesteps 3913.
Path 353 | total_timesteps 3923.
Path 354 | total_timesteps 3930.
Path 355 | total_timesteps 3939.
Path 356 | total_timesteps 3947.
Path 357 | total_timesteps 3954.
Path 358 | total_timesteps 3976.
Path 359 | total_timesteps 3988.
Path 360 | total_timesteps 4004.
Path 361 | total_timesteps 4014.
Path 362 | total_timesteps 4023.
Path 363 | total_timesteps 4030.
Path 364 | total_timesteps 4045.
Path 365 | total_timesteps 4057.
Path 366 | total_timesteps 4066.
Path 367 | total_timesteps 4079.
Path 368 | total_timesteps 4089.
Path 369 | total_timesteps 4098.
Path 370 | total_timesteps 4105.
Path 371 | total_timesteps 4114.
Path 372 | total_timesteps 4124.
Path 373 | total_timesteps 4138.
Path 374 | total_timesteps 4148.
Path 375 | total_timesteps 4158.
Path 376 | total_timesteps 4169.
Path 377 | total_timesteps 4178.
Path 378 | total_timesteps 4187.
Path 379 | total_timesteps 4195.
Path 380 | total_timesteps 4205.
Path 381 | total_timesteps 4237.
Path 382 | total_timesteps 4246.
Path 383 | total_timesteps 4267.
Path 384 | total_timesteps 4276.
Path 385 | total_timesteps 4292.
Path 386 | total_timesteps 4311.
Path 387 | total_timesteps 4324.
Path 388 | total_timesteps 4333.
Path 389 | total_timesteps 4354.
Path 390 | total_timesteps 4363.
Path 391 | total_timesteps 4372.
Path 392 | total_timesteps 4384.
Path 393 | total_timesteps 4395.
Path 394 | total_timesteps 4406.
Path 395 | total_timesteps 4418.
Path 396 | total_timesteps 4426.
Path 397 | total_timesteps 4436.
Path 398 | total_timesteps 4447.
Path 399 | total_timesteps 4458.
Path 400 | total_timesteps 4469.
Path 401 | total_timesteps 4483.
Path 402 | total_timesteps 4495.
Path 403 | total_timesteps 4508.
Path 404 | total_timesteps 4515.
Path 405 | total_timesteps 4524.
Path 406 | total_timesteps 4535.
Path 407 | total_timesteps 4544.
Path 408 | total_timesteps 4551.
Path 409 | total_timesteps 4566.
Path 410 | total_timesteps 4586.
Path 411 | total_timesteps 4595.
Path 412 | total_timesteps 4606.
Path 413 | total_timesteps 4615.
Path 414 | total_timesteps 4625.
Path 415 | total_timesteps 4636.
Path 416 | total_timesteps 4650.
Path 417 | total_timesteps 4662.
Path 418 | total_timesteps 4669.
Path 419 | total_timesteps 4679.
Path 420 | total_timesteps 4688.
Path 421 | total_timesteps 4695.
Path 422 | total_timesteps 4702.
Path 423 | total_timesteps 4712.
Path 424 | total_timesteps 4721.
Path 425 | total_timesteps 4731.
Path 426 | total_timesteps 4745.
Path 427 | total_timesteps 4757.
Path 428 | total_timesteps 4764.
Path 429 | total_timesteps 4780.
Path 430 | total_timesteps 4791.
Path 431 | total_timesteps 4802.
Path 432 | total_timesteps 4811.
Path 433 | total_timesteps 4825.
Path 434 | total_timesteps 4832.
Path 435 | total_timesteps 4846.
Path 436 | total_timesteps 4856.
Path 437 | total_timesteps 4863.
Path 438 | total_timesteps 4871.
Path 439 | total_timesteps 4879.
Path 440 | total_timesteps 4887.
Path 441 | total_timesteps 4897.
Path 442 | total_timesteps 4908.
Path 443 | total_timesteps 4921.
Path 444 | total_timesteps 4930.
Path 445 | total_timesteps 4945.
Path 446 | total_timesteps 4954.
Path 447 | total_timesteps 4961.
Path 448 | total_timesteps 4973.
Path 449 | total_timesteps 4983.
Path 450 | total_timesteps 4997.
Path 451 | total_timesteps 5006.
Path 452 | total_timesteps 5013.
Path 453 | total_timesteps 5029.
Path 454 | total_timesteps 5036.
Path 455 | total_timesteps 5057.
Path 456 | total_timesteps 5069.
Path 457 | total_timesteps 5080.
Path 458 | total_timesteps 5089.
Path 459 | total_timesteps 5099.
Path 460 | total_timesteps 5110.
Path 461 | total_timesteps 5119.
Path 462 | total_timesteps 5134.
Path 463 | total_timesteps 5149.
Path 464 | total_timesteps 5159.
Path 465 | total_timesteps 5172.
Path 466 | total_timesteps 5185.
Path 467 | total_timesteps 5193.
Path 468 | total_timesteps 5202.
Path 469 | total_timesteps 5211.
Path 470 | total_timesteps 5229.
Path 471 | total_timesteps 5237.
Path 472 | total_timesteps 5245.
Path 473 | total_timesteps 5258.
Path 474 | total_timesteps 5270.
Path 475 | total_timesteps 5279.
Path 476 | total_timesteps 5287.
Path 477 | total_timesteps 5294.
Path 478 | total_timesteps 5307.
Path 479 | total_timesteps 5325.
Path 480 | total_timesteps 5335.
Path 481 | total_timesteps 5345.
Path 482 | total_timesteps 5356.
Path 483 | total_timesteps 5367.
Path 484 | total_timesteps 5378.
Path 485 | total_timesteps 5387.
Path 486 | total_timesteps 5396.
Path 487 | total_timesteps 5405.
Path 488 | total_timesteps 5421.
Path 489 | total_timesteps 5431.
Path 490 | total_timesteps 5439.
Path 491 | total_timesteps 5458.
Path 492 | total_timesteps 5468.
Path 493 | total_timesteps 5479.
Path 494 | total_timesteps 5492.
Path 495 | total_timesteps 5503.
Path 496 | total_timesteps 5513.
Path 497 | total_timesteps 5524.
Path 498 | total_timesteps 5546.
Path 499 | total_timesteps 5555.
Path 500 | total_timesteps 5566.
Path 501 | total_timesteps 5575.
Path 502 | total_timesteps 5588.
Path 503 | total_timesteps 5596.
Path 504 | total_timesteps 5605.
Path 505 | total_timesteps 5612.
Path 506 | total_timesteps 5623.
Path 507 | total_timesteps 5635.
Path 508 | total_timesteps 5645.
Path 509 | total_timesteps 5654.
Path 510 | total_timesteps 5665.
Path 511 | total_timesteps 5673.
Path 512 | total_timesteps 5688.
Path 513 | total_timesteps 5696.
Path 514 | total_timesteps 5705.
Path 515 | total_timesteps 5716.
Path 516 | total_timesteps 5732.
Path 517 | total_timesteps 5740.
Path 518 | total_timesteps 5748.
Path 519 | total_timesteps 5756.
Path 520 | total_timesteps 5766.
Path 521 | total_timesteps 5776.
Path 522 | total_timesteps 5784.
Path 523 | total_timesteps 5795.
Path 524 | total_timesteps 5802.
Path 525 | total_timesteps 5811.
Path 526 | total_timesteps 5823.
Path 527 | total_timesteps 5833.
Path 528 | total_timesteps 5845.
Path 529 | total_timesteps 5853.
Path 530 | total_timesteps 5862.
Path 531 | total_timesteps 5873.
Path 532 | total_timesteps 5884.
Path 533 | total_timesteps 5894.
Path 534 | total_timesteps 5901.
Path 535 | total_timesteps 5911.
Path 536 | total_timesteps 5919.
Path 537 | total_timesteps 5930.
Path 538 | total_timesteps 5940.
Path 539 | total_timesteps 5959.
Path 540 | total_timesteps 5970.
Path 541 | total_timesteps 5979.
Path 542 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.27    |
| Iteration     | 10       |
| MaximumReturn | 0.575    |
| MinimumReturn | -22.3    |
| TotalSamples  | 48039    |
----------------------------
itr #11 | 
Fitting dynamics.
Validation loss = 0.00941400695592165
Validation loss = 0.008972248062491417
Validation loss = 0.008270282298326492
Validation loss = 0.008002745918929577
Validation loss = 0.007816468365490437
Validation loss = 0.008397546596825123
Validation loss = 0.009493431076407433
Validation loss = 0.008436989039182663
Validation loss = 0.007794596254825592
Validation loss = 0.008133848197758198
Validation loss = 0.008868558332324028
Validation loss = 0.008297870866954327
Validation loss = 0.007639778777956963
Validation loss = 0.008156933821737766
Validation loss = 0.00848762784153223
Validation loss = 0.008765571750700474
Validation loss = 0.008639422245323658
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 44.
Path 4 | total_timesteps 55.
Path 5 | total_timesteps 66.
Path 6 | total_timesteps 75.
Path 7 | total_timesteps 86.
Path 8 | total_timesteps 98.
Path 9 | total_timesteps 106.
Path 10 | total_timesteps 114.
Path 11 | total_timesteps 126.
Path 12 | total_timesteps 142.
Path 13 | total_timesteps 153.
Path 14 | total_timesteps 160.
Path 15 | total_timesteps 167.
Path 16 | total_timesteps 175.
Path 17 | total_timesteps 189.
Path 18 | total_timesteps 198.
Path 19 | total_timesteps 205.
Path 20 | total_timesteps 216.
Path 21 | total_timesteps 227.
Path 22 | total_timesteps 234.
Path 23 | total_timesteps 244.
Path 24 | total_timesteps 256.
Path 25 | total_timesteps 270.
Path 26 | total_timesteps 279.
Path 27 | total_timesteps 285.
Path 28 | total_timesteps 299.
Path 29 | total_timesteps 308.
Path 30 | total_timesteps 321.
Path 31 | total_timesteps 335.
Path 32 | total_timesteps 343.
Path 33 | total_timesteps 355.
Path 34 | total_timesteps 372.
Path 35 | total_timesteps 382.
Path 36 | total_timesteps 395.
Path 37 | total_timesteps 406.
Path 38 | total_timesteps 421.
Path 39 | total_timesteps 434.
Path 40 | total_timesteps 445.
Path 41 | total_timesteps 455.
Path 42 | total_timesteps 468.
Path 43 | total_timesteps 483.
Path 44 | total_timesteps 492.
Path 45 | total_timesteps 501.
Path 46 | total_timesteps 514.
Path 47 | total_timesteps 535.
Path 48 | total_timesteps 542.
Path 49 | total_timesteps 553.
Path 50 | total_timesteps 561.
Path 51 | total_timesteps 569.
Path 52 | total_timesteps 580.
Path 53 | total_timesteps 591.
Path 54 | total_timesteps 602.
Path 55 | total_timesteps 612.
Path 56 | total_timesteps 624.
Path 57 | total_timesteps 635.
Path 58 | total_timesteps 650.
Path 59 | total_timesteps 657.
Path 60 | total_timesteps 667.
Path 61 | total_timesteps 677.
Path 62 | total_timesteps 686.
Path 63 | total_timesteps 694.
Path 64 | total_timesteps 702.
Path 65 | total_timesteps 722.
Path 66 | total_timesteps 729.
Path 67 | total_timesteps 741.
Path 68 | total_timesteps 753.
Path 69 | total_timesteps 763.
Path 70 | total_timesteps 772.
Path 71 | total_timesteps 788.
Path 72 | total_timesteps 800.
Path 73 | total_timesteps 809.
Path 74 | total_timesteps 829.
Path 75 | total_timesteps 836.
Path 76 | total_timesteps 852.
Path 77 | total_timesteps 863.
Path 78 | total_timesteps 870.
Path 79 | total_timesteps 882.
Path 80 | total_timesteps 889.
Path 81 | total_timesteps 898.
Path 82 | total_timesteps 910.
Path 83 | total_timesteps 918.
Path 84 | total_timesteps 927.
Path 85 | total_timesteps 938.
Path 86 | total_timesteps 952.
Path 87 | total_timesteps 961.
Path 88 | total_timesteps 977.
Path 89 | total_timesteps 990.
Path 90 | total_timesteps 998.
Path 91 | total_timesteps 1005.
Path 92 | total_timesteps 1012.
Path 93 | total_timesteps 1021.
Path 94 | total_timesteps 1030.
Path 95 | total_timesteps 1036.
Path 96 | total_timesteps 1044.
Path 97 | total_timesteps 1055.
Path 98 | total_timesteps 1066.
Path 99 | total_timesteps 1077.
Path 100 | total_timesteps 1089.
Path 101 | total_timesteps 1102.
Path 102 | total_timesteps 1117.
Path 103 | total_timesteps 1129.
Path 104 | total_timesteps 1138.
Path 105 | total_timesteps 1148.
Path 106 | total_timesteps 1157.
Path 107 | total_timesteps 1166.
Path 108 | total_timesteps 1175.
Path 109 | total_timesteps 1184.
Path 110 | total_timesteps 1195.
Path 111 | total_timesteps 1204.
Path 112 | total_timesteps 1212.
Path 113 | total_timesteps 1223.
Path 114 | total_timesteps 1236.
Path 115 | total_timesteps 1251.
Path 116 | total_timesteps 1264.
Path 117 | total_timesteps 1280.
Path 118 | total_timesteps 1290.
Path 119 | total_timesteps 1298.
Path 120 | total_timesteps 1307.
Path 121 | total_timesteps 1316.
Path 122 | total_timesteps 1323.
Path 123 | total_timesteps 1334.
Path 124 | total_timesteps 1343.
Path 125 | total_timesteps 1354.
Path 126 | total_timesteps 1364.
Path 127 | total_timesteps 1375.
Path 128 | total_timesteps 1385.
Path 129 | total_timesteps 1399.
Path 130 | total_timesteps 1414.
Path 131 | total_timesteps 1425.
Path 132 | total_timesteps 1436.
Path 133 | total_timesteps 1444.
Path 134 | total_timesteps 1454.
Path 135 | total_timesteps 1466.
Path 136 | total_timesteps 1482.
Path 137 | total_timesteps 1491.
Path 138 | total_timesteps 1503.
Path 139 | total_timesteps 1521.
Path 140 | total_timesteps 1532.
Path 141 | total_timesteps 1543.
Path 142 | total_timesteps 1552.
Path 143 | total_timesteps 1563.
Path 144 | total_timesteps 1574.
Path 145 | total_timesteps 1582.
Path 146 | total_timesteps 1592.
Path 147 | total_timesteps 1600.
Path 148 | total_timesteps 1608.
Path 149 | total_timesteps 1620.
Path 150 | total_timesteps 1631.
Path 151 | total_timesteps 1637.
Path 152 | total_timesteps 1647.
Path 153 | total_timesteps 1655.
Path 154 | total_timesteps 1665.
Path 155 | total_timesteps 1676.
Path 156 | total_timesteps 1686.
Path 157 | total_timesteps 1696.
Path 158 | total_timesteps 1708.
Path 159 | total_timesteps 1718.
Path 160 | total_timesteps 1733.
Path 161 | total_timesteps 1743.
Path 162 | total_timesteps 1753.
Path 163 | total_timesteps 1762.
Path 164 | total_timesteps 1772.
Path 165 | total_timesteps 1786.
Path 166 | total_timesteps 1796.
Path 167 | total_timesteps 1811.
Path 168 | total_timesteps 1824.
Path 169 | total_timesteps 1832.
Path 170 | total_timesteps 1842.
Path 171 | total_timesteps 1849.
Path 172 | total_timesteps 1857.
Path 173 | total_timesteps 1869.
Path 174 | total_timesteps 1881.
Path 175 | total_timesteps 1890.
Path 176 | total_timesteps 1901.
Path 177 | total_timesteps 1910.
Path 178 | total_timesteps 1920.
Path 179 | total_timesteps 1930.
Path 180 | total_timesteps 1939.
Path 181 | total_timesteps 1951.
Path 182 | total_timesteps 1959.
Path 183 | total_timesteps 1973.
Path 184 | total_timesteps 1981.
Path 185 | total_timesteps 1987.
Path 186 | total_timesteps 1999.
Path 187 | total_timesteps 2021.
Path 188 | total_timesteps 2037.
Path 189 | total_timesteps 2045.
Path 190 | total_timesteps 2051.
Path 191 | total_timesteps 2063.
Path 192 | total_timesteps 2074.
Path 193 | total_timesteps 2086.
Path 194 | total_timesteps 2098.
Path 195 | total_timesteps 2108.
Path 196 | total_timesteps 2117.
Path 197 | total_timesteps 2130.
Path 198 | total_timesteps 2148.
Path 199 | total_timesteps 2154.
Path 200 | total_timesteps 2165.
Path 201 | total_timesteps 2181.
Path 202 | total_timesteps 2190.
Path 203 | total_timesteps 2199.
Path 204 | total_timesteps 2213.
Path 205 | total_timesteps 2221.
Path 206 | total_timesteps 2249.
Path 207 | total_timesteps 2263.
Path 208 | total_timesteps 2276.
Path 209 | total_timesteps 2299.
Path 210 | total_timesteps 2309.
Path 211 | total_timesteps 2316.
Path 212 | total_timesteps 2332.
Path 213 | total_timesteps 2340.
Path 214 | total_timesteps 2347.
Path 215 | total_timesteps 2357.
Path 216 | total_timesteps 2373.
Path 217 | total_timesteps 2382.
Path 218 | total_timesteps 2391.
Path 219 | total_timesteps 2401.
Path 220 | total_timesteps 2412.
Path 221 | total_timesteps 2422.
Path 222 | total_timesteps 2443.
Path 223 | total_timesteps 2451.
Path 224 | total_timesteps 2460.
Path 225 | total_timesteps 2471.
Path 226 | total_timesteps 2482.
Path 227 | total_timesteps 2493.
Path 228 | total_timesteps 2502.
Path 229 | total_timesteps 2510.
Path 230 | total_timesteps 2522.
Path 231 | total_timesteps 2535.
Path 232 | total_timesteps 2547.
Path 233 | total_timesteps 2557.
Path 234 | total_timesteps 2564.
Path 235 | total_timesteps 2573.
Path 236 | total_timesteps 2583.
Path 237 | total_timesteps 2594.
Path 238 | total_timesteps 2605.
Path 239 | total_timesteps 2615.
Path 240 | total_timesteps 2625.
Path 241 | total_timesteps 2639.
Path 242 | total_timesteps 2650.
Path 243 | total_timesteps 2668.
Path 244 | total_timesteps 2686.
Path 245 | total_timesteps 2695.
Path 246 | total_timesteps 2705.
Path 247 | total_timesteps 2720.
Path 248 | total_timesteps 2731.
Path 249 | total_timesteps 2739.
Path 250 | total_timesteps 2749.
Path 251 | total_timesteps 2760.
Path 252 | total_timesteps 2779.
Path 253 | total_timesteps 2788.
Path 254 | total_timesteps 2798.
Path 255 | total_timesteps 2816.
Path 256 | total_timesteps 2828.
Path 257 | total_timesteps 2837.
Path 258 | total_timesteps 2845.
Path 259 | total_timesteps 2858.
Path 260 | total_timesteps 2866.
Path 261 | total_timesteps 2875.
Path 262 | total_timesteps 2886.
Path 263 | total_timesteps 2902.
Path 264 | total_timesteps 2909.
Path 265 | total_timesteps 2928.
Path 266 | total_timesteps 2935.
Path 267 | total_timesteps 2953.
Path 268 | total_timesteps 2961.
Path 269 | total_timesteps 2971.
Path 270 | total_timesteps 2981.
Path 271 | total_timesteps 2989.
Path 272 | total_timesteps 3000.
Path 273 | total_timesteps 3011.
Path 274 | total_timesteps 3025.
Path 275 | total_timesteps 3033.
Path 276 | total_timesteps 3044.
Path 277 | total_timesteps 3051.
Path 278 | total_timesteps 3060.
Path 279 | total_timesteps 3069.
Path 280 | total_timesteps 3077.
Path 281 | total_timesteps 3086.
Path 282 | total_timesteps 3100.
Path 283 | total_timesteps 3107.
Path 284 | total_timesteps 3115.
Path 285 | total_timesteps 3132.
Path 286 | total_timesteps 3139.
Path 287 | total_timesteps 3147.
Path 288 | total_timesteps 3158.
Path 289 | total_timesteps 3166.
Path 290 | total_timesteps 3175.
Path 291 | total_timesteps 3184.
Path 292 | total_timesteps 3196.
Path 293 | total_timesteps 3204.
Path 294 | total_timesteps 3216.
Path 295 | total_timesteps 3226.
Path 296 | total_timesteps 3238.
Path 297 | total_timesteps 3251.
Path 298 | total_timesteps 3260.
Path 299 | total_timesteps 3277.
Path 300 | total_timesteps 3286.
Path 301 | total_timesteps 3295.
Path 302 | total_timesteps 3304.
Path 303 | total_timesteps 3324.
Path 304 | total_timesteps 3334.
Path 305 | total_timesteps 3346.
Path 306 | total_timesteps 3353.
Path 307 | total_timesteps 3365.
Path 308 | total_timesteps 3374.
Path 309 | total_timesteps 3382.
Path 310 | total_timesteps 3391.
Path 311 | total_timesteps 3399.
Path 312 | total_timesteps 3413.
Path 313 | total_timesteps 3419.
Path 314 | total_timesteps 3427.
Path 315 | total_timesteps 3436.
Path 316 | total_timesteps 3456.
Path 317 | total_timesteps 3464.
Path 318 | total_timesteps 3484.
Path 319 | total_timesteps 3494.
Path 320 | total_timesteps 3502.
Path 321 | total_timesteps 3509.
Path 322 | total_timesteps 3522.
Path 323 | total_timesteps 3536.
Path 324 | total_timesteps 3548.
Path 325 | total_timesteps 3561.
Path 326 | total_timesteps 3569.
Path 327 | total_timesteps 3579.
Path 328 | total_timesteps 3592.
Path 329 | total_timesteps 3601.
Path 330 | total_timesteps 3608.
Path 331 | total_timesteps 3621.
Path 332 | total_timesteps 3628.
Path 333 | total_timesteps 3639.
Path 334 | total_timesteps 3654.
Path 335 | total_timesteps 3665.
Path 336 | total_timesteps 3672.
Path 337 | total_timesteps 3679.
Path 338 | total_timesteps 3690.
Path 339 | total_timesteps 3701.
Path 340 | total_timesteps 3713.
Path 341 | total_timesteps 3722.
Path 342 | total_timesteps 3730.
Path 343 | total_timesteps 3741.
Path 344 | total_timesteps 3749.
Path 345 | total_timesteps 3757.
Path 346 | total_timesteps 3766.
Path 347 | total_timesteps 3773.
Path 348 | total_timesteps 3783.
Path 349 | total_timesteps 3794.
Path 350 | total_timesteps 3808.
Path 351 | total_timesteps 3816.
Path 352 | total_timesteps 3825.
Path 353 | total_timesteps 3837.
Path 354 | total_timesteps 3848.
Path 355 | total_timesteps 3856.
Path 356 | total_timesteps 3864.
Path 357 | total_timesteps 3875.
Path 358 | total_timesteps 3889.
Path 359 | total_timesteps 3898.
Path 360 | total_timesteps 3908.
Path 361 | total_timesteps 3917.
Path 362 | total_timesteps 3925.
Path 363 | total_timesteps 3939.
Path 364 | total_timesteps 3950.
Path 365 | total_timesteps 3968.
Path 366 | total_timesteps 3976.
Path 367 | total_timesteps 3986.
Path 368 | total_timesteps 3993.
Path 369 | total_timesteps 4002.
Path 370 | total_timesteps 4021.
Path 371 | total_timesteps 4030.
Path 372 | total_timesteps 4038.
Path 373 | total_timesteps 4047.
Path 374 | total_timesteps 4054.
Path 375 | total_timesteps 4060.
Path 376 | total_timesteps 4068.
Path 377 | total_timesteps 4075.
Path 378 | total_timesteps 4084.
Path 379 | total_timesteps 4092.
Path 380 | total_timesteps 4102.
Path 381 | total_timesteps 4113.
Path 382 | total_timesteps 4125.
Path 383 | total_timesteps 4133.
Path 384 | total_timesteps 4144.
Path 385 | total_timesteps 4160.
Path 386 | total_timesteps 4167.
Path 387 | total_timesteps 4178.
Path 388 | total_timesteps 4186.
Path 389 | total_timesteps 4196.
Path 390 | total_timesteps 4208.
Path 391 | total_timesteps 4222.
Path 392 | total_timesteps 4229.
Path 393 | total_timesteps 4238.
Path 394 | total_timesteps 4245.
Path 395 | total_timesteps 4255.
Path 396 | total_timesteps 4265.
Path 397 | total_timesteps 4275.
Path 398 | total_timesteps 4284.
Path 399 | total_timesteps 4292.
Path 400 | total_timesteps 4302.
Path 401 | total_timesteps 4311.
Path 402 | total_timesteps 4323.
Path 403 | total_timesteps 4332.
Path 404 | total_timesteps 4340.
Path 405 | total_timesteps 4357.
Path 406 | total_timesteps 4369.
Path 407 | total_timesteps 4381.
Path 408 | total_timesteps 4393.
Path 409 | total_timesteps 4405.
Path 410 | total_timesteps 4425.
Path 411 | total_timesteps 4436.
Path 412 | total_timesteps 4449.
Path 413 | total_timesteps 4466.
Path 414 | total_timesteps 4483.
Path 415 | total_timesteps 4492.
Path 416 | total_timesteps 4508.
Path 417 | total_timesteps 4523.
Path 418 | total_timesteps 4532.
Path 419 | total_timesteps 4542.
Path 420 | total_timesteps 4556.
Path 421 | total_timesteps 4568.
Path 422 | total_timesteps 4578.
Path 423 | total_timesteps 4591.
Path 424 | total_timesteps 4599.
Path 425 | total_timesteps 4605.
Path 426 | total_timesteps 4621.
Path 427 | total_timesteps 4629.
Path 428 | total_timesteps 4639.
Path 429 | total_timesteps 4650.
Path 430 | total_timesteps 4657.
Path 431 | total_timesteps 4667.
Path 432 | total_timesteps 4676.
Path 433 | total_timesteps 4692.
Path 434 | total_timesteps 4701.
Path 435 | total_timesteps 4712.
Path 436 | total_timesteps 4724.
Path 437 | total_timesteps 4739.
Path 438 | total_timesteps 4747.
Path 439 | total_timesteps 4761.
Path 440 | total_timesteps 4776.
Path 441 | total_timesteps 4785.
Path 442 | total_timesteps 4792.
Path 443 | total_timesteps 4803.
Path 444 | total_timesteps 4814.
Path 445 | total_timesteps 4832.
Path 446 | total_timesteps 4842.
Path 447 | total_timesteps 4850.
Path 448 | total_timesteps 4861.
Path 449 | total_timesteps 4873.
Path 450 | total_timesteps 4881.
Path 451 | total_timesteps 4891.
Path 452 | total_timesteps 4905.
Path 453 | total_timesteps 4917.
Path 454 | total_timesteps 4932.
Path 455 | total_timesteps 4940.
Path 456 | total_timesteps 4949.
Path 457 | total_timesteps 4960.
Path 458 | total_timesteps 4970.
Path 459 | total_timesteps 4979.
Path 460 | total_timesteps 4985.
Path 461 | total_timesteps 5002.
Path 462 | total_timesteps 5012.
Path 463 | total_timesteps 5021.
Path 464 | total_timesteps 5030.
Path 465 | total_timesteps 5039.
Path 466 | total_timesteps 5052.
Path 467 | total_timesteps 5061.
Path 468 | total_timesteps 5070.
Path 469 | total_timesteps 5080.
Path 470 | total_timesteps 5090.
Path 471 | total_timesteps 5106.
Path 472 | total_timesteps 5118.
Path 473 | total_timesteps 5146.
Path 474 | total_timesteps 5154.
Path 475 | total_timesteps 5172.
Path 476 | total_timesteps 5185.
Path 477 | total_timesteps 5199.
Path 478 | total_timesteps 5205.
Path 479 | total_timesteps 5215.
Path 480 | total_timesteps 5223.
Path 481 | total_timesteps 5240.
Path 482 | total_timesteps 5252.
Path 483 | total_timesteps 5259.
Path 484 | total_timesteps 5267.
Path 485 | total_timesteps 5291.
Path 486 | total_timesteps 5299.
Path 487 | total_timesteps 5309.
Path 488 | total_timesteps 5317.
Path 489 | total_timesteps 5332.
Path 490 | total_timesteps 5340.
Path 491 | total_timesteps 5351.
Path 492 | total_timesteps 5362.
Path 493 | total_timesteps 5369.
Path 494 | total_timesteps 5378.
Path 495 | total_timesteps 5390.
Path 496 | total_timesteps 5398.
Path 497 | total_timesteps 5412.
Path 498 | total_timesteps 5422.
Path 499 | total_timesteps 5431.
Path 500 | total_timesteps 5441.
Path 501 | total_timesteps 5448.
Path 502 | total_timesteps 5459.
Path 503 | total_timesteps 5468.
Path 504 | total_timesteps 5477.
Path 505 | total_timesteps 5489.
Path 506 | total_timesteps 5504.
Path 507 | total_timesteps 5516.
Path 508 | total_timesteps 5526.
Path 509 | total_timesteps 5540.
Path 510 | total_timesteps 5550.
Path 511 | total_timesteps 5560.
Path 512 | total_timesteps 5569.
Path 513 | total_timesteps 5578.
Path 514 | total_timesteps 5592.
Path 515 | total_timesteps 5599.
Path 516 | total_timesteps 5608.
Path 517 | total_timesteps 5616.
Path 518 | total_timesteps 5624.
Path 519 | total_timesteps 5638.
Path 520 | total_timesteps 5648.
Path 521 | total_timesteps 5659.
Path 522 | total_timesteps 5667.
Path 523 | total_timesteps 5674.
Path 524 | total_timesteps 5686.
Path 525 | total_timesteps 5700.
Path 526 | total_timesteps 5707.
Path 527 | total_timesteps 5719.
Path 528 | total_timesteps 5736.
Path 529 | total_timesteps 5750.
Path 530 | total_timesteps 5756.
Path 531 | total_timesteps 5764.
Path 532 | total_timesteps 5773.
Path 533 | total_timesteps 5781.
Path 534 | total_timesteps 5793.
Path 535 | total_timesteps 5803.
Path 536 | total_timesteps 5811.
Path 537 | total_timesteps 5822.
Path 538 | total_timesteps 5833.
Path 539 | total_timesteps 5843.
Path 540 | total_timesteps 5852.
Path 541 | total_timesteps 5859.
Path 542 | total_timesteps 5868.
Path 543 | total_timesteps 5880.
Path 544 | total_timesteps 5888.
Path 545 | total_timesteps 5897.
Path 546 | total_timesteps 5905.
Path 547 | total_timesteps 5915.
Path 548 | total_timesteps 5926.
Path 549 | total_timesteps 5937.
Path 550 | total_timesteps 5943.
Path 551 | total_timesteps 5955.
Path 552 | total_timesteps 5964.
Path 553 | total_timesteps 5981.
Path 554 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.2     |
| Iteration     | 11       |
| MaximumReturn | 0.347    |
| MinimumReturn | -21.2    |
| TotalSamples  | 52042    |
----------------------------
itr #12 | 
Fitting dynamics.
Validation loss = 0.00829797051846981
Validation loss = 0.007890705019235611
Validation loss = 0.008406070061028004
Validation loss = 0.008251076564192772
Validation loss = 0.007762231398373842
Validation loss = 0.009189887903630733
Validation loss = 0.007460435852408409
Validation loss = 0.007217130623757839
Validation loss = 0.007696820423007011
Validation loss = 0.007729316130280495
Validation loss = 0.007928394712507725
Validation loss = 0.007167401723563671
Validation loss = 0.007404855918139219
Validation loss = 0.007607032544910908
Validation loss = 0.007427449803799391
Validation loss = 0.008351501077413559
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 24.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 45.
Path 6 | total_timesteps 55.
Path 7 | total_timesteps 64.
Path 8 | total_timesteps 75.
Path 9 | total_timesteps 87.
Path 10 | total_timesteps 95.
Path 11 | total_timesteps 106.
Path 12 | total_timesteps 122.
Path 13 | total_timesteps 132.
Path 14 | total_timesteps 141.
Path 15 | total_timesteps 150.
Path 16 | total_timesteps 162.
Path 17 | total_timesteps 170.
Path 18 | total_timesteps 178.
Path 19 | total_timesteps 186.
Path 20 | total_timesteps 193.
Path 21 | total_timesteps 201.
Path 22 | total_timesteps 213.
Path 23 | total_timesteps 225.
Path 24 | total_timesteps 232.
Path 25 | total_timesteps 243.
Path 26 | total_timesteps 264.
Path 27 | total_timesteps 271.
Path 28 | total_timesteps 280.
Path 29 | total_timesteps 295.
Path 30 | total_timesteps 301.
Path 31 | total_timesteps 308.
Path 32 | total_timesteps 319.
Path 33 | total_timesteps 329.
Path 34 | total_timesteps 337.
Path 35 | total_timesteps 347.
Path 36 | total_timesteps 357.
Path 37 | total_timesteps 367.
Path 38 | total_timesteps 377.
Path 39 | total_timesteps 387.
Path 40 | total_timesteps 397.
Path 41 | total_timesteps 405.
Path 42 | total_timesteps 411.
Path 43 | total_timesteps 418.
Path 44 | total_timesteps 426.
Path 45 | total_timesteps 433.
Path 46 | total_timesteps 446.
Path 47 | total_timesteps 458.
Path 48 | total_timesteps 470.
Path 49 | total_timesteps 479.
Path 50 | total_timesteps 497.
Path 51 | total_timesteps 511.
Path 52 | total_timesteps 520.
Path 53 | total_timesteps 526.
Path 54 | total_timesteps 540.
Path 55 | total_timesteps 551.
Path 56 | total_timesteps 558.
Path 57 | total_timesteps 567.
Path 58 | total_timesteps 574.
Path 59 | total_timesteps 585.
Path 60 | total_timesteps 595.
Path 61 | total_timesteps 611.
Path 62 | total_timesteps 623.
Path 63 | total_timesteps 634.
Path 64 | total_timesteps 642.
Path 65 | total_timesteps 650.
Path 66 | total_timesteps 659.
Path 67 | total_timesteps 665.
Path 68 | total_timesteps 677.
Path 69 | total_timesteps 687.
Path 70 | total_timesteps 697.
Path 71 | total_timesteps 707.
Path 72 | total_timesteps 719.
Path 73 | total_timesteps 727.
Path 74 | total_timesteps 738.
Path 75 | total_timesteps 748.
Path 76 | total_timesteps 758.
Path 77 | total_timesteps 774.
Path 78 | total_timesteps 782.
Path 79 | total_timesteps 797.
Path 80 | total_timesteps 806.
Path 81 | total_timesteps 818.
Path 82 | total_timesteps 830.
Path 83 | total_timesteps 842.
Path 84 | total_timesteps 850.
Path 85 | total_timesteps 857.
Path 86 | total_timesteps 867.
Path 87 | total_timesteps 876.
Path 88 | total_timesteps 885.
Path 89 | total_timesteps 897.
Path 90 | total_timesteps 905.
Path 91 | total_timesteps 915.
Path 92 | total_timesteps 926.
Path 93 | total_timesteps 935.
Path 94 | total_timesteps 945.
Path 95 | total_timesteps 955.
Path 96 | total_timesteps 964.
Path 97 | total_timesteps 975.
Path 98 | total_timesteps 985.
Path 99 | total_timesteps 998.
Path 100 | total_timesteps 1008.
Path 101 | total_timesteps 1015.
Path 102 | total_timesteps 1022.
Path 103 | total_timesteps 1030.
Path 104 | total_timesteps 1040.
Path 105 | total_timesteps 1050.
Path 106 | total_timesteps 1058.
Path 107 | total_timesteps 1070.
Path 108 | total_timesteps 1080.
Path 109 | total_timesteps 1092.
Path 110 | total_timesteps 1099.
Path 111 | total_timesteps 1110.
Path 112 | total_timesteps 1119.
Path 113 | total_timesteps 1130.
Path 114 | total_timesteps 1136.
Path 115 | total_timesteps 1143.
Path 116 | total_timesteps 1152.
Path 117 | total_timesteps 1165.
Path 118 | total_timesteps 1175.
Path 119 | total_timesteps 1184.
Path 120 | total_timesteps 1193.
Path 121 | total_timesteps 1203.
Path 122 | total_timesteps 1211.
Path 123 | total_timesteps 1220.
Path 124 | total_timesteps 1230.
Path 125 | total_timesteps 1240.
Path 126 | total_timesteps 1251.
Path 127 | total_timesteps 1265.
Path 128 | total_timesteps 1274.
Path 129 | total_timesteps 1282.
Path 130 | total_timesteps 1292.
Path 131 | total_timesteps 1301.
Path 132 | total_timesteps 1309.
Path 133 | total_timesteps 1318.
Path 134 | total_timesteps 1329.
Path 135 | total_timesteps 1339.
Path 136 | total_timesteps 1348.
Path 137 | total_timesteps 1358.
Path 138 | total_timesteps 1368.
Path 139 | total_timesteps 1378.
Path 140 | total_timesteps 1393.
Path 141 | total_timesteps 1403.
Path 142 | total_timesteps 1413.
Path 143 | total_timesteps 1422.
Path 144 | total_timesteps 1432.
Path 145 | total_timesteps 1443.
Path 146 | total_timesteps 1449.
Path 147 | total_timesteps 1456.
Path 148 | total_timesteps 1466.
Path 149 | total_timesteps 1479.
Path 150 | total_timesteps 1487.
Path 151 | total_timesteps 1497.
Path 152 | total_timesteps 1505.
Path 153 | total_timesteps 1523.
Path 154 | total_timesteps 1536.
Path 155 | total_timesteps 1543.
Path 156 | total_timesteps 1554.
Path 157 | total_timesteps 1562.
Path 158 | total_timesteps 1569.
Path 159 | total_timesteps 1575.
Path 160 | total_timesteps 1582.
Path 161 | total_timesteps 1590.
Path 162 | total_timesteps 1601.
Path 163 | total_timesteps 1623.
Path 164 | total_timesteps 1635.
Path 165 | total_timesteps 1644.
Path 166 | total_timesteps 1653.
Path 167 | total_timesteps 1663.
Path 168 | total_timesteps 1671.
Path 169 | total_timesteps 1682.
Path 170 | total_timesteps 1689.
Path 171 | total_timesteps 1697.
Path 172 | total_timesteps 1709.
Path 173 | total_timesteps 1724.
Path 174 | total_timesteps 1731.
Path 175 | total_timesteps 1745.
Path 176 | total_timesteps 1753.
Path 177 | total_timesteps 1771.
Path 178 | total_timesteps 1781.
Path 179 | total_timesteps 1789.
Path 180 | total_timesteps 1797.
Path 181 | total_timesteps 1804.
Path 182 | total_timesteps 1810.
Path 183 | total_timesteps 1822.
Path 184 | total_timesteps 1835.
Path 185 | total_timesteps 1845.
Path 186 | total_timesteps 1856.
Path 187 | total_timesteps 1871.
Path 188 | total_timesteps 1878.
Path 189 | total_timesteps 1889.
Path 190 | total_timesteps 1898.
Path 191 | total_timesteps 1910.
Path 192 | total_timesteps 1917.
Path 193 | total_timesteps 1927.
Path 194 | total_timesteps 1940.
Path 195 | total_timesteps 1947.
Path 196 | total_timesteps 1954.
Path 197 | total_timesteps 1977.
Path 198 | total_timesteps 1986.
Path 199 | total_timesteps 1997.
Path 200 | total_timesteps 2009.
Path 201 | total_timesteps 2018.
Path 202 | total_timesteps 2025.
Path 203 | total_timesteps 2035.
Path 204 | total_timesteps 2043.
Path 205 | total_timesteps 2051.
Path 206 | total_timesteps 2060.
Path 207 | total_timesteps 2069.
Path 208 | total_timesteps 2079.
Path 209 | total_timesteps 2087.
Path 210 | total_timesteps 2098.
Path 211 | total_timesteps 2111.
Path 212 | total_timesteps 2119.
Path 213 | total_timesteps 2128.
Path 214 | total_timesteps 2135.
Path 215 | total_timesteps 2146.
Path 216 | total_timesteps 2155.
Path 217 | total_timesteps 2164.
Path 218 | total_timesteps 2176.
Path 219 | total_timesteps 2185.
Path 220 | total_timesteps 2195.
Path 221 | total_timesteps 2203.
Path 222 | total_timesteps 2215.
Path 223 | total_timesteps 2225.
Path 224 | total_timesteps 2232.
Path 225 | total_timesteps 2241.
Path 226 | total_timesteps 2254.
Path 227 | total_timesteps 2262.
Path 228 | total_timesteps 2269.
Path 229 | total_timesteps 2280.
Path 230 | total_timesteps 2291.
Path 231 | total_timesteps 2300.
Path 232 | total_timesteps 2309.
Path 233 | total_timesteps 2325.
Path 234 | total_timesteps 2338.
Path 235 | total_timesteps 2345.
Path 236 | total_timesteps 2357.
Path 237 | total_timesteps 2364.
Path 238 | total_timesteps 2373.
Path 239 | total_timesteps 2384.
Path 240 | total_timesteps 2394.
Path 241 | total_timesteps 2407.
Path 242 | total_timesteps 2421.
Path 243 | total_timesteps 2430.
Path 244 | total_timesteps 2437.
Path 245 | total_timesteps 2453.
Path 246 | total_timesteps 2464.
Path 247 | total_timesteps 2473.
Path 248 | total_timesteps 2480.
Path 249 | total_timesteps 2490.
Path 250 | total_timesteps 2497.
Path 251 | total_timesteps 2504.
Path 252 | total_timesteps 2512.
Path 253 | total_timesteps 2519.
Path 254 | total_timesteps 2529.
Path 255 | total_timesteps 2536.
Path 256 | total_timesteps 2544.
Path 257 | total_timesteps 2556.
Path 258 | total_timesteps 2563.
Path 259 | total_timesteps 2575.
Path 260 | total_timesteps 2587.
Path 261 | total_timesteps 2595.
Path 262 | total_timesteps 2605.
Path 263 | total_timesteps 2613.
Path 264 | total_timesteps 2623.
Path 265 | total_timesteps 2634.
Path 266 | total_timesteps 2648.
Path 267 | total_timesteps 2661.
Path 268 | total_timesteps 2671.
Path 269 | total_timesteps 2683.
Path 270 | total_timesteps 2692.
Path 271 | total_timesteps 2699.
Path 272 | total_timesteps 2713.
Path 273 | total_timesteps 2724.
Path 274 | total_timesteps 2740.
Path 275 | total_timesteps 2757.
Path 276 | total_timesteps 2765.
Path 277 | total_timesteps 2775.
Path 278 | total_timesteps 2787.
Path 279 | total_timesteps 2796.
Path 280 | total_timesteps 2806.
Path 281 | total_timesteps 2816.
Path 282 | total_timesteps 2827.
Path 283 | total_timesteps 2836.
Path 284 | total_timesteps 2847.
Path 285 | total_timesteps 2855.
Path 286 | total_timesteps 2862.
Path 287 | total_timesteps 2871.
Path 288 | total_timesteps 2882.
Path 289 | total_timesteps 2898.
Path 290 | total_timesteps 2908.
Path 291 | total_timesteps 2921.
Path 292 | total_timesteps 2930.
Path 293 | total_timesteps 2937.
Path 294 | total_timesteps 2947.
Path 295 | total_timesteps 2955.
Path 296 | total_timesteps 2964.
Path 297 | total_timesteps 2972.
Path 298 | total_timesteps 2987.
Path 299 | total_timesteps 2999.
Path 300 | total_timesteps 3009.
Path 301 | total_timesteps 3023.
Path 302 | total_timesteps 3032.
Path 303 | total_timesteps 3044.
Path 304 | total_timesteps 3052.
Path 305 | total_timesteps 3063.
Path 306 | total_timesteps 3072.
Path 307 | total_timesteps 3083.
Path 308 | total_timesteps 3095.
Path 309 | total_timesteps 3106.
Path 310 | total_timesteps 3116.
Path 311 | total_timesteps 3125.
Path 312 | total_timesteps 3141.
Path 313 | total_timesteps 3155.
Path 314 | total_timesteps 3168.
Path 315 | total_timesteps 3179.
Path 316 | total_timesteps 3187.
Path 317 | total_timesteps 3196.
Path 318 | total_timesteps 3208.
Path 319 | total_timesteps 3217.
Path 320 | total_timesteps 3228.
Path 321 | total_timesteps 3243.
Path 322 | total_timesteps 3250.
Path 323 | total_timesteps 3260.
Path 324 | total_timesteps 3268.
Path 325 | total_timesteps 3275.
Path 326 | total_timesteps 3283.
Path 327 | total_timesteps 3289.
Path 328 | total_timesteps 3295.
Path 329 | total_timesteps 3307.
Path 330 | total_timesteps 3317.
Path 331 | total_timesteps 3327.
Path 332 | total_timesteps 3341.
Path 333 | total_timesteps 3350.
Path 334 | total_timesteps 3359.
Path 335 | total_timesteps 3366.
Path 336 | total_timesteps 3375.
Path 337 | total_timesteps 3384.
Path 338 | total_timesteps 3395.
Path 339 | total_timesteps 3404.
Path 340 | total_timesteps 3412.
Path 341 | total_timesteps 3426.
Path 342 | total_timesteps 3440.
Path 343 | total_timesteps 3448.
Path 344 | total_timesteps 3458.
Path 345 | total_timesteps 3465.
Path 346 | total_timesteps 3473.
Path 347 | total_timesteps 3482.
Path 348 | total_timesteps 3499.
Path 349 | total_timesteps 3507.
Path 350 | total_timesteps 3517.
Path 351 | total_timesteps 3528.
Path 352 | total_timesteps 3551.
Path 353 | total_timesteps 3560.
Path 354 | total_timesteps 3569.
Path 355 | total_timesteps 3580.
Path 356 | total_timesteps 3588.
Path 357 | total_timesteps 3601.
Path 358 | total_timesteps 3609.
Path 359 | total_timesteps 3622.
Path 360 | total_timesteps 3634.
Path 361 | total_timesteps 3648.
Path 362 | total_timesteps 3659.
Path 363 | total_timesteps 3670.
Path 364 | total_timesteps 3683.
Path 365 | total_timesteps 3692.
Path 366 | total_timesteps 3700.
Path 367 | total_timesteps 3709.
Path 368 | total_timesteps 3726.
Path 369 | total_timesteps 3736.
Path 370 | total_timesteps 3745.
Path 371 | total_timesteps 3764.
Path 372 | total_timesteps 3774.
Path 373 | total_timesteps 3783.
Path 374 | total_timesteps 3795.
Path 375 | total_timesteps 3806.
Path 376 | total_timesteps 3815.
Path 377 | total_timesteps 3827.
Path 378 | total_timesteps 3838.
Path 379 | total_timesteps 3845.
Path 380 | total_timesteps 3857.
Path 381 | total_timesteps 3878.
Path 382 | total_timesteps 3891.
Path 383 | total_timesteps 3899.
Path 384 | total_timesteps 3910.
Path 385 | total_timesteps 3919.
Path 386 | total_timesteps 3930.
Path 387 | total_timesteps 3945.
Path 388 | total_timesteps 3954.
Path 389 | total_timesteps 3963.
Path 390 | total_timesteps 3970.
Path 391 | total_timesteps 3979.
Path 392 | total_timesteps 3988.
Path 393 | total_timesteps 3996.
Path 394 | total_timesteps 4012.
Path 395 | total_timesteps 4022.
Path 396 | total_timesteps 4031.
Path 397 | total_timesteps 4040.
Path 398 | total_timesteps 4049.
Path 399 | total_timesteps 4057.
Path 400 | total_timesteps 4068.
Path 401 | total_timesteps 4088.
Path 402 | total_timesteps 4095.
Path 403 | total_timesteps 4107.
Path 404 | total_timesteps 4118.
Path 405 | total_timesteps 4131.
Path 406 | total_timesteps 4144.
Path 407 | total_timesteps 4156.
Path 408 | total_timesteps 4166.
Path 409 | total_timesteps 4174.
Path 410 | total_timesteps 4184.
Path 411 | total_timesteps 4194.
Path 412 | total_timesteps 4207.
Path 413 | total_timesteps 4217.
Path 414 | total_timesteps 4225.
Path 415 | total_timesteps 4232.
Path 416 | total_timesteps 4242.
Path 417 | total_timesteps 4249.
Path 418 | total_timesteps 4257.
Path 419 | total_timesteps 4266.
Path 420 | total_timesteps 4278.
Path 421 | total_timesteps 4285.
Path 422 | total_timesteps 4293.
Path 423 | total_timesteps 4306.
Path 424 | total_timesteps 4315.
Path 425 | total_timesteps 4323.
Path 426 | total_timesteps 4331.
Path 427 | total_timesteps 4341.
Path 428 | total_timesteps 4355.
Path 429 | total_timesteps 4364.
Path 430 | total_timesteps 4371.
Path 431 | total_timesteps 4383.
Path 432 | total_timesteps 4392.
Path 433 | total_timesteps 4405.
Path 434 | total_timesteps 4422.
Path 435 | total_timesteps 4430.
Path 436 | total_timesteps 4438.
Path 437 | total_timesteps 4444.
Path 438 | total_timesteps 4455.
Path 439 | total_timesteps 4464.
Path 440 | total_timesteps 4473.
Path 441 | total_timesteps 4480.
Path 442 | total_timesteps 4492.
Path 443 | total_timesteps 4501.
Path 444 | total_timesteps 4511.
Path 445 | total_timesteps 4524.
Path 446 | total_timesteps 4531.
Path 447 | total_timesteps 4538.
Path 448 | total_timesteps 4549.
Path 449 | total_timesteps 4557.
Path 450 | total_timesteps 4568.
Path 451 | total_timesteps 4581.
Path 452 | total_timesteps 4588.
Path 453 | total_timesteps 4600.
Path 454 | total_timesteps 4609.
Path 455 | total_timesteps 4617.
Path 456 | total_timesteps 4624.
Path 457 | total_timesteps 4634.
Path 458 | total_timesteps 4648.
Path 459 | total_timesteps 4656.
Path 460 | total_timesteps 4663.
Path 461 | total_timesteps 4678.
Path 462 | total_timesteps 4688.
Path 463 | total_timesteps 4699.
Path 464 | total_timesteps 4705.
Path 465 | total_timesteps 4717.
Path 466 | total_timesteps 4723.
Path 467 | total_timesteps 4735.
Path 468 | total_timesteps 4748.
Path 469 | total_timesteps 4758.
Path 470 | total_timesteps 4765.
Path 471 | total_timesteps 4774.
Path 472 | total_timesteps 4781.
Path 473 | total_timesteps 4793.
Path 474 | total_timesteps 4805.
Path 475 | total_timesteps 4814.
Path 476 | total_timesteps 4834.
Path 477 | total_timesteps 4842.
Path 478 | total_timesteps 4851.
Path 479 | total_timesteps 4860.
Path 480 | total_timesteps 4867.
Path 481 | total_timesteps 4876.
Path 482 | total_timesteps 4887.
Path 483 | total_timesteps 4896.
Path 484 | total_timesteps 4904.
Path 485 | total_timesteps 4918.
Path 486 | total_timesteps 4926.
Path 487 | total_timesteps 4935.
Path 488 | total_timesteps 4948.
Path 489 | total_timesteps 4958.
Path 490 | total_timesteps 4965.
Path 491 | total_timesteps 4981.
Path 492 | total_timesteps 4992.
Path 493 | total_timesteps 5001.
Path 494 | total_timesteps 5012.
Path 495 | total_timesteps 5020.
Path 496 | total_timesteps 5028.
Path 497 | total_timesteps 5039.
Path 498 | total_timesteps 5057.
Path 499 | total_timesteps 5069.
Path 500 | total_timesteps 5083.
Path 501 | total_timesteps 5090.
Path 502 | total_timesteps 5106.
Path 503 | total_timesteps 5118.
Path 504 | total_timesteps 5128.
Path 505 | total_timesteps 5139.
Path 506 | total_timesteps 5147.
Path 507 | total_timesteps 5155.
Path 508 | total_timesteps 5163.
Path 509 | total_timesteps 5171.
Path 510 | total_timesteps 5178.
Path 511 | total_timesteps 5189.
Path 512 | total_timesteps 5199.
Path 513 | total_timesteps 5207.
Path 514 | total_timesteps 5215.
Path 515 | total_timesteps 5223.
Path 516 | total_timesteps 5237.
Path 517 | total_timesteps 5244.
Path 518 | total_timesteps 5255.
Path 519 | total_timesteps 5262.
Path 520 | total_timesteps 5270.
Path 521 | total_timesteps 5281.
Path 522 | total_timesteps 5293.
Path 523 | total_timesteps 5301.
Path 524 | total_timesteps 5309.
Path 525 | total_timesteps 5321.
Path 526 | total_timesteps 5329.
Path 527 | total_timesteps 5338.
Path 528 | total_timesteps 5352.
Path 529 | total_timesteps 5364.
Path 530 | total_timesteps 5373.
Path 531 | total_timesteps 5381.
Path 532 | total_timesteps 5391.
Path 533 | total_timesteps 5401.
Path 534 | total_timesteps 5410.
Path 535 | total_timesteps 5416.
Path 536 | total_timesteps 5428.
Path 537 | total_timesteps 5436.
Path 538 | total_timesteps 5447.
Path 539 | total_timesteps 5459.
Path 540 | total_timesteps 5472.
Path 541 | total_timesteps 5479.
Path 542 | total_timesteps 5488.
Path 543 | total_timesteps 5496.
Path 544 | total_timesteps 5503.
Path 545 | total_timesteps 5514.
Path 546 | total_timesteps 5521.
Path 547 | total_timesteps 5530.
Path 548 | total_timesteps 5538.
Path 549 | total_timesteps 5545.
Path 550 | total_timesteps 5557.
Path 551 | total_timesteps 5568.
Path 552 | total_timesteps 5576.
Path 553 | total_timesteps 5588.
Path 554 | total_timesteps 5600.
Path 555 | total_timesteps 5608.
Path 556 | total_timesteps 5617.
Path 557 | total_timesteps 5626.
Path 558 | total_timesteps 5637.
Path 559 | total_timesteps 5645.
Path 560 | total_timesteps 5651.
Path 561 | total_timesteps 5658.
Path 562 | total_timesteps 5667.
Path 563 | total_timesteps 5682.
Path 564 | total_timesteps 5688.
Path 565 | total_timesteps 5696.
Path 566 | total_timesteps 5705.
Path 567 | total_timesteps 5719.
Path 568 | total_timesteps 5729.
Path 569 | total_timesteps 5740.
Path 570 | total_timesteps 5747.
Path 571 | total_timesteps 5756.
Path 572 | total_timesteps 5767.
Path 573 | total_timesteps 5775.
Path 574 | total_timesteps 5784.
Path 575 | total_timesteps 5793.
Path 576 | total_timesteps 5806.
Path 577 | total_timesteps 5821.
Path 578 | total_timesteps 5831.
Path 579 | total_timesteps 5838.
Path 580 | total_timesteps 5847.
Path 581 | total_timesteps 5855.
Path 582 | total_timesteps 5863.
Path 583 | total_timesteps 5869.
Path 584 | total_timesteps 5878.
Path 585 | total_timesteps 5889.
Path 586 | total_timesteps 5898.
Path 587 | total_timesteps 5905.
Path 588 | total_timesteps 5919.
Path 589 | total_timesteps 5927.
Path 590 | total_timesteps 5938.
Path 591 | total_timesteps 5951.
Path 592 | total_timesteps 5960.
Path 593 | total_timesteps 5969.
Path 594 | total_timesteps 5976.
Path 595 | total_timesteps 5984.
Path 596 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.92    |
| Iteration     | 12       |
| MaximumReturn | -0.946   |
| MinimumReturn | -19      |
| TotalSamples  | 56047    |
----------------------------
itr #13 | 
Fitting dynamics.
Validation loss = 0.007782944943755865
Validation loss = 0.007289370056241751
Validation loss = 0.007184911519289017
Validation loss = 0.007122603710740805
Validation loss = 0.006996522191911936
Validation loss = 0.00776993203908205
Validation loss = 0.007080498617142439
Validation loss = 0.007159375119954348
Validation loss = 0.007952249608933926
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 32.
Path 4 | total_timesteps 41.
Path 5 | total_timesteps 61.
Path 6 | total_timesteps 74.
Path 7 | total_timesteps 84.
Path 8 | total_timesteps 92.
Path 9 | total_timesteps 102.
Path 10 | total_timesteps 110.
Path 11 | total_timesteps 122.
Path 12 | total_timesteps 133.
Path 13 | total_timesteps 140.
Path 14 | total_timesteps 149.
Path 15 | total_timesteps 156.
Path 16 | total_timesteps 164.
Path 17 | total_timesteps 172.
Path 18 | total_timesteps 180.
Path 19 | total_timesteps 188.
Path 20 | total_timesteps 196.
Path 21 | total_timesteps 202.
Path 22 | total_timesteps 211.
Path 23 | total_timesteps 219.
Path 24 | total_timesteps 231.
Path 25 | total_timesteps 240.
Path 26 | total_timesteps 254.
Path 27 | total_timesteps 269.
Path 28 | total_timesteps 284.
Path 29 | total_timesteps 291.
Path 30 | total_timesteps 299.
Path 31 | total_timesteps 305.
Path 32 | total_timesteps 313.
Path 33 | total_timesteps 325.
Path 34 | total_timesteps 340.
Path 35 | total_timesteps 352.
Path 36 | total_timesteps 362.
Path 37 | total_timesteps 372.
Path 38 | total_timesteps 382.
Path 39 | total_timesteps 394.
Path 40 | total_timesteps 408.
Path 41 | total_timesteps 420.
Path 42 | total_timesteps 428.
Path 43 | total_timesteps 437.
Path 44 | total_timesteps 448.
Path 45 | total_timesteps 457.
Path 46 | total_timesteps 466.
Path 47 | total_timesteps 479.
Path 48 | total_timesteps 486.
Path 49 | total_timesteps 496.
Path 50 | total_timesteps 503.
Path 51 | total_timesteps 513.
Path 52 | total_timesteps 522.
Path 53 | total_timesteps 529.
Path 54 | total_timesteps 536.
Path 55 | total_timesteps 546.
Path 56 | total_timesteps 555.
Path 57 | total_timesteps 563.
Path 58 | total_timesteps 571.
Path 59 | total_timesteps 581.
Path 60 | total_timesteps 591.
Path 61 | total_timesteps 599.
Path 62 | total_timesteps 606.
Path 63 | total_timesteps 618.
Path 64 | total_timesteps 625.
Path 65 | total_timesteps 633.
Path 66 | total_timesteps 642.
Path 67 | total_timesteps 649.
Path 68 | total_timesteps 660.
Path 69 | total_timesteps 674.
Path 70 | total_timesteps 688.
Path 71 | total_timesteps 695.
Path 72 | total_timesteps 702.
Path 73 | total_timesteps 710.
Path 74 | total_timesteps 720.
Path 75 | total_timesteps 727.
Path 76 | total_timesteps 739.
Path 77 | total_timesteps 755.
Path 78 | total_timesteps 765.
Path 79 | total_timesteps 775.
Path 80 | total_timesteps 791.
Path 81 | total_timesteps 800.
Path 82 | total_timesteps 808.
Path 83 | total_timesteps 821.
Path 84 | total_timesteps 831.
Path 85 | total_timesteps 839.
Path 86 | total_timesteps 854.
Path 87 | total_timesteps 863.
Path 88 | total_timesteps 869.
Path 89 | total_timesteps 879.
Path 90 | total_timesteps 890.
Path 91 | total_timesteps 897.
Path 92 | total_timesteps 908.
Path 93 | total_timesteps 919.
Path 94 | total_timesteps 927.
Path 95 | total_timesteps 940.
Path 96 | total_timesteps 955.
Path 97 | total_timesteps 963.
Path 98 | total_timesteps 976.
Path 99 | total_timesteps 988.
Path 100 | total_timesteps 996.
Path 101 | total_timesteps 1004.
Path 102 | total_timesteps 1017.
Path 103 | total_timesteps 1028.
Path 104 | total_timesteps 1036.
Path 105 | total_timesteps 1044.
Path 106 | total_timesteps 1053.
Path 107 | total_timesteps 1066.
Path 108 | total_timesteps 1078.
Path 109 | total_timesteps 1086.
Path 110 | total_timesteps 1095.
Path 111 | total_timesteps 1102.
Path 112 | total_timesteps 1111.
Path 113 | total_timesteps 1117.
Path 114 | total_timesteps 1127.
Path 115 | total_timesteps 1135.
Path 116 | total_timesteps 1145.
Path 117 | total_timesteps 1156.
Path 118 | total_timesteps 1163.
Path 119 | total_timesteps 1173.
Path 120 | total_timesteps 1183.
Path 121 | total_timesteps 1191.
Path 122 | total_timesteps 1201.
Path 123 | total_timesteps 1209.
Path 124 | total_timesteps 1220.
Path 125 | total_timesteps 1230.
Path 126 | total_timesteps 1243.
Path 127 | total_timesteps 1256.
Path 128 | total_timesteps 1266.
Path 129 | total_timesteps 1278.
Path 130 | total_timesteps 1285.
Path 131 | total_timesteps 1295.
Path 132 | total_timesteps 1306.
Path 133 | total_timesteps 1316.
Path 134 | total_timesteps 1327.
Path 135 | total_timesteps 1336.
Path 136 | total_timesteps 1345.
Path 137 | total_timesteps 1357.
Path 138 | total_timesteps 1364.
Path 139 | total_timesteps 1375.
Path 140 | total_timesteps 1382.
Path 141 | total_timesteps 1391.
Path 142 | total_timesteps 1398.
Path 143 | total_timesteps 1405.
Path 144 | total_timesteps 1415.
Path 145 | total_timesteps 1425.
Path 146 | total_timesteps 1441.
Path 147 | total_timesteps 1448.
Path 148 | total_timesteps 1459.
Path 149 | total_timesteps 1467.
Path 150 | total_timesteps 1480.
Path 151 | total_timesteps 1487.
Path 152 | total_timesteps 1495.
Path 153 | total_timesteps 1504.
Path 154 | total_timesteps 1511.
Path 155 | total_timesteps 1527.
Path 156 | total_timesteps 1546.
Path 157 | total_timesteps 1553.
Path 158 | total_timesteps 1563.
Path 159 | total_timesteps 1572.
Path 160 | total_timesteps 1584.
Path 161 | total_timesteps 1602.
Path 162 | total_timesteps 1610.
Path 163 | total_timesteps 1624.
Path 164 | total_timesteps 1634.
Path 165 | total_timesteps 1643.
Path 166 | total_timesteps 1651.
Path 167 | total_timesteps 1664.
Path 168 | total_timesteps 1676.
Path 169 | total_timesteps 1684.
Path 170 | total_timesteps 1696.
Path 171 | total_timesteps 1703.
Path 172 | total_timesteps 1719.
Path 173 | total_timesteps 1738.
Path 174 | total_timesteps 1754.
Path 175 | total_timesteps 1762.
Path 176 | total_timesteps 1768.
Path 177 | total_timesteps 1781.
Path 178 | total_timesteps 1794.
Path 179 | total_timesteps 1811.
Path 180 | total_timesteps 1820.
Path 181 | total_timesteps 1834.
Path 182 | total_timesteps 1841.
Path 183 | total_timesteps 1851.
Path 184 | total_timesteps 1862.
Path 185 | total_timesteps 1876.
Path 186 | total_timesteps 1884.
Path 187 | total_timesteps 1895.
Path 188 | total_timesteps 1909.
Path 189 | total_timesteps 1918.
Path 190 | total_timesteps 1933.
Path 191 | total_timesteps 1943.
Path 192 | total_timesteps 1950.
Path 193 | total_timesteps 1962.
Path 194 | total_timesteps 1969.
Path 195 | total_timesteps 1978.
Path 196 | total_timesteps 1985.
Path 197 | total_timesteps 1994.
Path 198 | total_timesteps 2002.
Path 199 | total_timesteps 2012.
Path 200 | total_timesteps 2019.
Path 201 | total_timesteps 2029.
Path 202 | total_timesteps 2038.
Path 203 | total_timesteps 2049.
Path 204 | total_timesteps 2057.
Path 205 | total_timesteps 2065.
Path 206 | total_timesteps 2079.
Path 207 | total_timesteps 2089.
Path 208 | total_timesteps 2100.
Path 209 | total_timesteps 2107.
Path 210 | total_timesteps 2120.
Path 211 | total_timesteps 2127.
Path 212 | total_timesteps 2135.
Path 213 | total_timesteps 2145.
Path 214 | total_timesteps 2154.
Path 215 | total_timesteps 2167.
Path 216 | total_timesteps 2182.
Path 217 | total_timesteps 2189.
Path 218 | total_timesteps 2199.
Path 219 | total_timesteps 2208.
Path 220 | total_timesteps 2217.
Path 221 | total_timesteps 2226.
Path 222 | total_timesteps 2238.
Path 223 | total_timesteps 2251.
Path 224 | total_timesteps 2260.
Path 225 | total_timesteps 2269.
Path 226 | total_timesteps 2282.
Path 227 | total_timesteps 2289.
Path 228 | total_timesteps 2300.
Path 229 | total_timesteps 2309.
Path 230 | total_timesteps 2318.
Path 231 | total_timesteps 2327.
Path 232 | total_timesteps 2336.
Path 233 | total_timesteps 2344.
Path 234 | total_timesteps 2367.
Path 235 | total_timesteps 2375.
Path 236 | total_timesteps 2383.
Path 237 | total_timesteps 2393.
Path 238 | total_timesteps 2408.
Path 239 | total_timesteps 2428.
Path 240 | total_timesteps 2439.
Path 241 | total_timesteps 2447.
Path 242 | total_timesteps 2457.
Path 243 | total_timesteps 2467.
Path 244 | total_timesteps 2476.
Path 245 | total_timesteps 2487.
Path 246 | total_timesteps 2496.
Path 247 | total_timesteps 2503.
Path 248 | total_timesteps 2511.
Path 249 | total_timesteps 2522.
Path 250 | total_timesteps 2534.
Path 251 | total_timesteps 2546.
Path 252 | total_timesteps 2553.
Path 253 | total_timesteps 2562.
Path 254 | total_timesteps 2569.
Path 255 | total_timesteps 2580.
Path 256 | total_timesteps 2590.
Path 257 | total_timesteps 2603.
Path 258 | total_timesteps 2619.
Path 259 | total_timesteps 2630.
Path 260 | total_timesteps 2638.
Path 261 | total_timesteps 2648.
Path 262 | total_timesteps 2657.
Path 263 | total_timesteps 2665.
Path 264 | total_timesteps 2679.
Path 265 | total_timesteps 2691.
Path 266 | total_timesteps 2699.
Path 267 | total_timesteps 2709.
Path 268 | total_timesteps 2724.
Path 269 | total_timesteps 2733.
Path 270 | total_timesteps 2744.
Path 271 | total_timesteps 2751.
Path 272 | total_timesteps 2759.
Path 273 | total_timesteps 2769.
Path 274 | total_timesteps 2777.
Path 275 | total_timesteps 2789.
Path 276 | total_timesteps 2797.
Path 277 | total_timesteps 2807.
Path 278 | total_timesteps 2824.
Path 279 | total_timesteps 2835.
Path 280 | total_timesteps 2848.
Path 281 | total_timesteps 2855.
Path 282 | total_timesteps 2863.
Path 283 | total_timesteps 2870.
Path 284 | total_timesteps 2877.
Path 285 | total_timesteps 2888.
Path 286 | total_timesteps 2897.
Path 287 | total_timesteps 2906.
Path 288 | total_timesteps 2916.
Path 289 | total_timesteps 2923.
Path 290 | total_timesteps 2936.
Path 291 | total_timesteps 2947.
Path 292 | total_timesteps 2959.
Path 293 | total_timesteps 2967.
Path 294 | total_timesteps 2976.
Path 295 | total_timesteps 2984.
Path 296 | total_timesteps 2993.
Path 297 | total_timesteps 3003.
Path 298 | total_timesteps 3013.
Path 299 | total_timesteps 3020.
Path 300 | total_timesteps 3032.
Path 301 | total_timesteps 3040.
Path 302 | total_timesteps 3048.
Path 303 | total_timesteps 3055.
Path 304 | total_timesteps 3062.
Path 305 | total_timesteps 3071.
Path 306 | total_timesteps 3083.
Path 307 | total_timesteps 3093.
Path 308 | total_timesteps 3109.
Path 309 | total_timesteps 3117.
Path 310 | total_timesteps 3127.
Path 311 | total_timesteps 3135.
Path 312 | total_timesteps 3144.
Path 313 | total_timesteps 3167.
Path 314 | total_timesteps 3174.
Path 315 | total_timesteps 3183.
Path 316 | total_timesteps 3189.
Path 317 | total_timesteps 3198.
Path 318 | total_timesteps 3205.
Path 319 | total_timesteps 3214.
Path 320 | total_timesteps 3225.
Path 321 | total_timesteps 3231.
Path 322 | total_timesteps 3239.
Path 323 | total_timesteps 3252.
Path 324 | total_timesteps 3263.
Path 325 | total_timesteps 3276.
Path 326 | total_timesteps 3288.
Path 327 | total_timesteps 3296.
Path 328 | total_timesteps 3304.
Path 329 | total_timesteps 3313.
Path 330 | total_timesteps 3323.
Path 331 | total_timesteps 3351.
Path 332 | total_timesteps 3363.
Path 333 | total_timesteps 3375.
Path 334 | total_timesteps 3391.
Path 335 | total_timesteps 3400.
Path 336 | total_timesteps 3413.
Path 337 | total_timesteps 3424.
Path 338 | total_timesteps 3432.
Path 339 | total_timesteps 3444.
Path 340 | total_timesteps 3455.
Path 341 | total_timesteps 3464.
Path 342 | total_timesteps 3472.
Path 343 | total_timesteps 3484.
Path 344 | total_timesteps 3491.
Path 345 | total_timesteps 3501.
Path 346 | total_timesteps 3510.
Path 347 | total_timesteps 3521.
Path 348 | total_timesteps 3527.
Path 349 | total_timesteps 3535.
Path 350 | total_timesteps 3545.
Path 351 | total_timesteps 3556.
Path 352 | total_timesteps 3565.
Path 353 | total_timesteps 3572.
Path 354 | total_timesteps 3582.
Path 355 | total_timesteps 3598.
Path 356 | total_timesteps 3610.
Path 357 | total_timesteps 3617.
Path 358 | total_timesteps 3624.
Path 359 | total_timesteps 3632.
Path 360 | total_timesteps 3640.
Path 361 | total_timesteps 3655.
Path 362 | total_timesteps 3675.
Path 363 | total_timesteps 3686.
Path 364 | total_timesteps 3693.
Path 365 | total_timesteps 3702.
Path 366 | total_timesteps 3715.
Path 367 | total_timesteps 3722.
Path 368 | total_timesteps 3731.
Path 369 | total_timesteps 3740.
Path 370 | total_timesteps 3750.
Path 371 | total_timesteps 3768.
Path 372 | total_timesteps 3780.
Path 373 | total_timesteps 3790.
Path 374 | total_timesteps 3796.
Path 375 | total_timesteps 3809.
Path 376 | total_timesteps 3821.
Path 377 | total_timesteps 3831.
Path 378 | total_timesteps 3841.
Path 379 | total_timesteps 3848.
Path 380 | total_timesteps 3856.
Path 381 | total_timesteps 3865.
Path 382 | total_timesteps 3874.
Path 383 | total_timesteps 3883.
Path 384 | total_timesteps 3895.
Path 385 | total_timesteps 3905.
Path 386 | total_timesteps 3916.
Path 387 | total_timesteps 3927.
Path 388 | total_timesteps 3937.
Path 389 | total_timesteps 3947.
Path 390 | total_timesteps 3961.
Path 391 | total_timesteps 3970.
Path 392 | total_timesteps 3981.
Path 393 | total_timesteps 3988.
Path 394 | total_timesteps 3997.
Path 395 | total_timesteps 4003.
Path 396 | total_timesteps 4022.
Path 397 | total_timesteps 4032.
Path 398 | total_timesteps 4040.
Path 399 | total_timesteps 4052.
Path 400 | total_timesteps 4061.
Path 401 | total_timesteps 4069.
Path 402 | total_timesteps 4080.
Path 403 | total_timesteps 4089.
Path 404 | total_timesteps 4096.
Path 405 | total_timesteps 4102.
Path 406 | total_timesteps 4111.
Path 407 | total_timesteps 4128.
Path 408 | total_timesteps 4141.
Path 409 | total_timesteps 4168.
Path 410 | total_timesteps 4177.
Path 411 | total_timesteps 4188.
Path 412 | total_timesteps 4201.
Path 413 | total_timesteps 4210.
Path 414 | total_timesteps 4219.
Path 415 | total_timesteps 4233.
Path 416 | total_timesteps 4245.
Path 417 | total_timesteps 4254.
Path 418 | total_timesteps 4267.
Path 419 | total_timesteps 4278.
Path 420 | total_timesteps 4287.
Path 421 | total_timesteps 4295.
Path 422 | total_timesteps 4306.
Path 423 | total_timesteps 4322.
Path 424 | total_timesteps 4329.
Path 425 | total_timesteps 4342.
Path 426 | total_timesteps 4351.
Path 427 | total_timesteps 4363.
Path 428 | total_timesteps 4374.
Path 429 | total_timesteps 4382.
Path 430 | total_timesteps 4391.
Path 431 | total_timesteps 4401.
Path 432 | total_timesteps 4409.
Path 433 | total_timesteps 4421.
Path 434 | total_timesteps 4432.
Path 435 | total_timesteps 4441.
Path 436 | total_timesteps 4452.
Path 437 | total_timesteps 4464.
Path 438 | total_timesteps 4473.
Path 439 | total_timesteps 4480.
Path 440 | total_timesteps 4488.
Path 441 | total_timesteps 4499.
Path 442 | total_timesteps 4508.
Path 443 | total_timesteps 4518.
Path 444 | total_timesteps 4525.
Path 445 | total_timesteps 4534.
Path 446 | total_timesteps 4556.
Path 447 | total_timesteps 4565.
Path 448 | total_timesteps 4575.
Path 449 | total_timesteps 4582.
Path 450 | total_timesteps 4592.
Path 451 | total_timesteps 4603.
Path 452 | total_timesteps 4613.
Path 453 | total_timesteps 4627.
Path 454 | total_timesteps 4644.
Path 455 | total_timesteps 4655.
Path 456 | total_timesteps 4665.
Path 457 | total_timesteps 4675.
Path 458 | total_timesteps 4683.
Path 459 | total_timesteps 4696.
Path 460 | total_timesteps 4708.
Path 461 | total_timesteps 4717.
Path 462 | total_timesteps 4728.
Path 463 | total_timesteps 4745.
Path 464 | total_timesteps 4759.
Path 465 | total_timesteps 4767.
Path 466 | total_timesteps 4778.
Path 467 | total_timesteps 4787.
Path 468 | total_timesteps 4799.
Path 469 | total_timesteps 4812.
Path 470 | total_timesteps 4820.
Path 471 | total_timesteps 4828.
Path 472 | total_timesteps 4836.
Path 473 | total_timesteps 4848.
Path 474 | total_timesteps 4858.
Path 475 | total_timesteps 4869.
Path 476 | total_timesteps 4877.
Path 477 | total_timesteps 4903.
Path 478 | total_timesteps 4913.
Path 479 | total_timesteps 4921.
Path 480 | total_timesteps 4930.
Path 481 | total_timesteps 4947.
Path 482 | total_timesteps 4957.
Path 483 | total_timesteps 4966.
Path 484 | total_timesteps 4975.
Path 485 | total_timesteps 4983.
Path 486 | total_timesteps 4991.
Path 487 | total_timesteps 5001.
Path 488 | total_timesteps 5013.
Path 489 | total_timesteps 5021.
Path 490 | total_timesteps 5036.
Path 491 | total_timesteps 5044.
Path 492 | total_timesteps 5051.
Path 493 | total_timesteps 5060.
Path 494 | total_timesteps 5067.
Path 495 | total_timesteps 5075.
Path 496 | total_timesteps 5086.
Path 497 | total_timesteps 5095.
Path 498 | total_timesteps 5102.
Path 499 | total_timesteps 5109.
Path 500 | total_timesteps 5119.
Path 501 | total_timesteps 5128.
Path 502 | total_timesteps 5138.
Path 503 | total_timesteps 5146.
Path 504 | total_timesteps 5163.
Path 505 | total_timesteps 5170.
Path 506 | total_timesteps 5180.
Path 507 | total_timesteps 5192.
Path 508 | total_timesteps 5199.
Path 509 | total_timesteps 5207.
Path 510 | total_timesteps 5213.
Path 511 | total_timesteps 5223.
Path 512 | total_timesteps 5230.
Path 513 | total_timesteps 5241.
Path 514 | total_timesteps 5249.
Path 515 | total_timesteps 5260.
Path 516 | total_timesteps 5269.
Path 517 | total_timesteps 5281.
Path 518 | total_timesteps 5291.
Path 519 | total_timesteps 5304.
Path 520 | total_timesteps 5314.
Path 521 | total_timesteps 5322.
Path 522 | total_timesteps 5330.
Path 523 | total_timesteps 5345.
Path 524 | total_timesteps 5354.
Path 525 | total_timesteps 5364.
Path 526 | total_timesteps 5377.
Path 527 | total_timesteps 5387.
Path 528 | total_timesteps 5396.
Path 529 | total_timesteps 5404.
Path 530 | total_timesteps 5413.
Path 531 | total_timesteps 5422.
Path 532 | total_timesteps 5437.
Path 533 | total_timesteps 5446.
Path 534 | total_timesteps 5456.
Path 535 | total_timesteps 5469.
Path 536 | total_timesteps 5475.
Path 537 | total_timesteps 5483.
Path 538 | total_timesteps 5494.
Path 539 | total_timesteps 5505.
Path 540 | total_timesteps 5516.
Path 541 | total_timesteps 5526.
Path 542 | total_timesteps 5537.
Path 543 | total_timesteps 5550.
Path 544 | total_timesteps 5563.
Path 545 | total_timesteps 5571.
Path 546 | total_timesteps 5580.
Path 547 | total_timesteps 5592.
Path 548 | total_timesteps 5601.
Path 549 | total_timesteps 5612.
Path 550 | total_timesteps 5624.
Path 551 | total_timesteps 5635.
Path 552 | total_timesteps 5644.
Path 553 | total_timesteps 5658.
Path 554 | total_timesteps 5668.
Path 555 | total_timesteps 5677.
Path 556 | total_timesteps 5685.
Path 557 | total_timesteps 5694.
Path 558 | total_timesteps 5701.
Path 559 | total_timesteps 5710.
Path 560 | total_timesteps 5720.
Path 561 | total_timesteps 5727.
Path 562 | total_timesteps 5735.
Path 563 | total_timesteps 5745.
Path 564 | total_timesteps 5753.
Path 565 | total_timesteps 5762.
Path 566 | total_timesteps 5773.
Path 567 | total_timesteps 5784.
Path 568 | total_timesteps 5798.
Path 569 | total_timesteps 5808.
Path 570 | total_timesteps 5817.
Path 571 | total_timesteps 5828.
Path 572 | total_timesteps 5835.
Path 573 | total_timesteps 5848.
Path 574 | total_timesteps 5857.
Path 575 | total_timesteps 5867.
Path 576 | total_timesteps 5875.
Path 577 | total_timesteps 5884.
Path 578 | total_timesteps 5895.
Path 579 | total_timesteps 5910.
Path 580 | total_timesteps 5918.
Path 581 | total_timesteps 5927.
Path 582 | total_timesteps 5939.
Path 583 | total_timesteps 5947.
Path 584 | total_timesteps 5957.
Path 585 | total_timesteps 5967.
Path 586 | total_timesteps 5974.
Path 587 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.13    |
| Iteration     | 13       |
| MaximumReturn | -0.716   |
| MinimumReturn | -22.4    |
| TotalSamples  | 60049    |
----------------------------
itr #14 | 
Fitting dynamics.
Validation loss = 0.00752359488978982
Validation loss = 0.006984053645282984
Validation loss = 0.0070427157916128635
Validation loss = 0.007225989364087582
Validation loss = 0.007263318169862032
Validation loss = 0.006649403367191553
Validation loss = 0.006877652835100889
Validation loss = 0.007428652141243219
Validation loss = 0.0066760024055838585
Validation loss = 0.007394800893962383
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 24.
Path 2 | total_timesteps 33.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 53.
Path 5 | total_timesteps 71.
Path 6 | total_timesteps 79.
Path 7 | total_timesteps 91.
Path 8 | total_timesteps 109.
Path 9 | total_timesteps 120.
Path 10 | total_timesteps 139.
Path 11 | total_timesteps 148.
Path 12 | total_timesteps 155.
Path 13 | total_timesteps 164.
Path 14 | total_timesteps 172.
Path 15 | total_timesteps 180.
Path 16 | total_timesteps 187.
Path 17 | total_timesteps 200.
Path 18 | total_timesteps 209.
Path 19 | total_timesteps 215.
Path 20 | total_timesteps 235.
Path 21 | total_timesteps 242.
Path 22 | total_timesteps 263.
Path 23 | total_timesteps 273.
Path 24 | total_timesteps 284.
Path 25 | total_timesteps 295.
Path 26 | total_timesteps 305.
Path 27 | total_timesteps 311.
Path 28 | total_timesteps 319.
Path 29 | total_timesteps 326.
Path 30 | total_timesteps 336.
Path 31 | total_timesteps 344.
Path 32 | total_timesteps 352.
Path 33 | total_timesteps 363.
Path 34 | total_timesteps 373.
Path 35 | total_timesteps 384.
Path 36 | total_timesteps 392.
Path 37 | total_timesteps 402.
Path 38 | total_timesteps 412.
Path 39 | total_timesteps 421.
Path 40 | total_timesteps 431.
Path 41 | total_timesteps 441.
Path 42 | total_timesteps 450.
Path 43 | total_timesteps 461.
Path 44 | total_timesteps 472.
Path 45 | total_timesteps 480.
Path 46 | total_timesteps 489.
Path 47 | total_timesteps 503.
Path 48 | total_timesteps 512.
Path 49 | total_timesteps 524.
Path 50 | total_timesteps 535.
Path 51 | total_timesteps 543.
Path 52 | total_timesteps 553.
Path 53 | total_timesteps 560.
Path 54 | total_timesteps 571.
Path 55 | total_timesteps 578.
Path 56 | total_timesteps 588.
Path 57 | total_timesteps 599.
Path 58 | total_timesteps 610.
Path 59 | total_timesteps 617.
Path 60 | total_timesteps 627.
Path 61 | total_timesteps 640.
Path 62 | total_timesteps 649.
Path 63 | total_timesteps 656.
Path 64 | total_timesteps 671.
Path 65 | total_timesteps 680.
Path 66 | total_timesteps 691.
Path 67 | total_timesteps 706.
Path 68 | total_timesteps 717.
Path 69 | total_timesteps 732.
Path 70 | total_timesteps 742.
Path 71 | total_timesteps 750.
Path 72 | total_timesteps 763.
Path 73 | total_timesteps 778.
Path 74 | total_timesteps 791.
Path 75 | total_timesteps 804.
Path 76 | total_timesteps 814.
Path 77 | total_timesteps 823.
Path 78 | total_timesteps 832.
Path 79 | total_timesteps 843.
Path 80 | total_timesteps 853.
Path 81 | total_timesteps 862.
Path 82 | total_timesteps 871.
Path 83 | total_timesteps 885.
Path 84 | total_timesteps 896.
Path 85 | total_timesteps 909.
Path 86 | total_timesteps 925.
Path 87 | total_timesteps 934.
Path 88 | total_timesteps 942.
Path 89 | total_timesteps 950.
Path 90 | total_timesteps 959.
Path 91 | total_timesteps 966.
Path 92 | total_timesteps 974.
Path 93 | total_timesteps 982.
Path 94 | total_timesteps 988.
Path 95 | total_timesteps 996.
Path 96 | total_timesteps 1008.
Path 97 | total_timesteps 1020.
Path 98 | total_timesteps 1028.
Path 99 | total_timesteps 1037.
Path 100 | total_timesteps 1045.
Path 101 | total_timesteps 1055.
Path 102 | total_timesteps 1065.
Path 103 | total_timesteps 1075.
Path 104 | total_timesteps 1090.
Path 105 | total_timesteps 1099.
Path 106 | total_timesteps 1112.
Path 107 | total_timesteps 1122.
Path 108 | total_timesteps 1135.
Path 109 | total_timesteps 1144.
Path 110 | total_timesteps 1153.
Path 111 | total_timesteps 1163.
Path 112 | total_timesteps 1174.
Path 113 | total_timesteps 1184.
Path 114 | total_timesteps 1191.
Path 115 | total_timesteps 1199.
Path 116 | total_timesteps 1207.
Path 117 | total_timesteps 1216.
Path 118 | total_timesteps 1226.
Path 119 | total_timesteps 1237.
Path 120 | total_timesteps 1246.
Path 121 | total_timesteps 1253.
Path 122 | total_timesteps 1264.
Path 123 | total_timesteps 1277.
Path 124 | total_timesteps 1285.
Path 125 | total_timesteps 1305.
Path 126 | total_timesteps 1314.
Path 127 | total_timesteps 1321.
Path 128 | total_timesteps 1333.
Path 129 | total_timesteps 1343.
Path 130 | total_timesteps 1353.
Path 131 | total_timesteps 1362.
Path 132 | total_timesteps 1370.
Path 133 | total_timesteps 1378.
Path 134 | total_timesteps 1386.
Path 135 | total_timesteps 1396.
Path 136 | total_timesteps 1405.
Path 137 | total_timesteps 1414.
Path 138 | total_timesteps 1422.
Path 139 | total_timesteps 1432.
Path 140 | total_timesteps 1450.
Path 141 | total_timesteps 1458.
Path 142 | total_timesteps 1471.
Path 143 | total_timesteps 1481.
Path 144 | total_timesteps 1488.
Path 145 | total_timesteps 1497.
Path 146 | total_timesteps 1508.
Path 147 | total_timesteps 1521.
Path 148 | total_timesteps 1532.
Path 149 | total_timesteps 1543.
Path 150 | total_timesteps 1555.
Path 151 | total_timesteps 1566.
Path 152 | total_timesteps 1576.
Path 153 | total_timesteps 1583.
Path 154 | total_timesteps 1595.
Path 155 | total_timesteps 1607.
Path 156 | total_timesteps 1619.
Path 157 | total_timesteps 1638.
Path 158 | total_timesteps 1654.
Path 159 | total_timesteps 1667.
Path 160 | total_timesteps 1674.
Path 161 | total_timesteps 1686.
Path 162 | total_timesteps 1692.
Path 163 | total_timesteps 1701.
Path 164 | total_timesteps 1709.
Path 165 | total_timesteps 1721.
Path 166 | total_timesteps 1729.
Path 167 | total_timesteps 1738.
Path 168 | total_timesteps 1747.
Path 169 | total_timesteps 1755.
Path 170 | total_timesteps 1764.
Path 171 | total_timesteps 1773.
Path 172 | total_timesteps 1782.
Path 173 | total_timesteps 1788.
Path 174 | total_timesteps 1801.
Path 175 | total_timesteps 1810.
Path 176 | total_timesteps 1821.
Path 177 | total_timesteps 1827.
Path 178 | total_timesteps 1838.
Path 179 | total_timesteps 1850.
Path 180 | total_timesteps 1860.
Path 181 | total_timesteps 1870.
Path 182 | total_timesteps 1877.
Path 183 | total_timesteps 1886.
Path 184 | total_timesteps 1900.
Path 185 | total_timesteps 1906.
Path 186 | total_timesteps 1914.
Path 187 | total_timesteps 1921.
Path 188 | total_timesteps 1931.
Path 189 | total_timesteps 1938.
Path 190 | total_timesteps 1947.
Path 191 | total_timesteps 1953.
Path 192 | total_timesteps 1961.
Path 193 | total_timesteps 1971.
Path 194 | total_timesteps 1980.
Path 195 | total_timesteps 1988.
Path 196 | total_timesteps 1998.
Path 197 | total_timesteps 2008.
Path 198 | total_timesteps 2016.
Path 199 | total_timesteps 2024.
Path 200 | total_timesteps 2032.
Path 201 | total_timesteps 2046.
Path 202 | total_timesteps 2057.
Path 203 | total_timesteps 2065.
Path 204 | total_timesteps 2072.
Path 205 | total_timesteps 2080.
Path 206 | total_timesteps 2089.
Path 207 | total_timesteps 2109.
Path 208 | total_timesteps 2116.
Path 209 | total_timesteps 2126.
Path 210 | total_timesteps 2140.
Path 211 | total_timesteps 2151.
Path 212 | total_timesteps 2159.
Path 213 | total_timesteps 2166.
Path 214 | total_timesteps 2172.
Path 215 | total_timesteps 2183.
Path 216 | total_timesteps 2190.
Path 217 | total_timesteps 2203.
Path 218 | total_timesteps 2212.
Path 219 | total_timesteps 2224.
Path 220 | total_timesteps 2234.
Path 221 | total_timesteps 2243.
Path 222 | total_timesteps 2251.
Path 223 | total_timesteps 2263.
Path 224 | total_timesteps 2273.
Path 225 | total_timesteps 2281.
Path 226 | total_timesteps 2295.
Path 227 | total_timesteps 2302.
Path 228 | total_timesteps 2309.
Path 229 | total_timesteps 2319.
Path 230 | total_timesteps 2329.
Path 231 | total_timesteps 2336.
Path 232 | total_timesteps 2347.
Path 233 | total_timesteps 2362.
Path 234 | total_timesteps 2371.
Path 235 | total_timesteps 2382.
Path 236 | total_timesteps 2390.
Path 237 | total_timesteps 2402.
Path 238 | total_timesteps 2412.
Path 239 | total_timesteps 2425.
Path 240 | total_timesteps 2434.
Path 241 | total_timesteps 2443.
Path 242 | total_timesteps 2455.
Path 243 | total_timesteps 2472.
Path 244 | total_timesteps 2486.
Path 245 | total_timesteps 2493.
Path 246 | total_timesteps 2501.
Path 247 | total_timesteps 2508.
Path 248 | total_timesteps 2516.
Path 249 | total_timesteps 2525.
Path 250 | total_timesteps 2534.
Path 251 | total_timesteps 2543.
Path 252 | total_timesteps 2556.
Path 253 | total_timesteps 2565.
Path 254 | total_timesteps 2580.
Path 255 | total_timesteps 2586.
Path 256 | total_timesteps 2594.
Path 257 | total_timesteps 2600.
Path 258 | total_timesteps 2609.
Path 259 | total_timesteps 2620.
Path 260 | total_timesteps 2633.
Path 261 | total_timesteps 2653.
Path 262 | total_timesteps 2664.
Path 263 | total_timesteps 2674.
Path 264 | total_timesteps 2683.
Path 265 | total_timesteps 2692.
Path 266 | total_timesteps 2705.
Path 267 | total_timesteps 2729.
Path 268 | total_timesteps 2738.
Path 269 | total_timesteps 2746.
Path 270 | total_timesteps 2754.
Path 271 | total_timesteps 2763.
Path 272 | total_timesteps 2773.
Path 273 | total_timesteps 2785.
Path 274 | total_timesteps 2798.
Path 275 | total_timesteps 2807.
Path 276 | total_timesteps 2813.
Path 277 | total_timesteps 2823.
Path 278 | total_timesteps 2834.
Path 279 | total_timesteps 2844.
Path 280 | total_timesteps 2852.
Path 281 | total_timesteps 2860.
Path 282 | total_timesteps 2873.
Path 283 | total_timesteps 2886.
Path 284 | total_timesteps 2902.
Path 285 | total_timesteps 2913.
Path 286 | total_timesteps 2923.
Path 287 | total_timesteps 2933.
Path 288 | total_timesteps 2940.
Path 289 | total_timesteps 2951.
Path 290 | total_timesteps 2965.
Path 291 | total_timesteps 2975.
Path 292 | total_timesteps 2986.
Path 293 | total_timesteps 2997.
Path 294 | total_timesteps 3005.
Path 295 | total_timesteps 3020.
Path 296 | total_timesteps 3033.
Path 297 | total_timesteps 3058.
Path 298 | total_timesteps 3070.
Path 299 | total_timesteps 3082.
Path 300 | total_timesteps 3090.
Path 301 | total_timesteps 3099.
Path 302 | total_timesteps 3110.
Path 303 | total_timesteps 3117.
Path 304 | total_timesteps 3125.
Path 305 | total_timesteps 3138.
Path 306 | total_timesteps 3148.
Path 307 | total_timesteps 3160.
Path 308 | total_timesteps 3167.
Path 309 | total_timesteps 3177.
Path 310 | total_timesteps 3188.
Path 311 | total_timesteps 3199.
Path 312 | total_timesteps 3206.
Path 313 | total_timesteps 3227.
Path 314 | total_timesteps 3235.
Path 315 | total_timesteps 3251.
Path 316 | total_timesteps 3263.
Path 317 | total_timesteps 3273.
Path 318 | total_timesteps 3285.
Path 319 | total_timesteps 3295.
Path 320 | total_timesteps 3308.
Path 321 | total_timesteps 3316.
Path 322 | total_timesteps 3328.
Path 323 | total_timesteps 3338.
Path 324 | total_timesteps 3351.
Path 325 | total_timesteps 3360.
Path 326 | total_timesteps 3380.
Path 327 | total_timesteps 3391.
Path 328 | total_timesteps 3398.
Path 329 | total_timesteps 3409.
Path 330 | total_timesteps 3415.
Path 331 | total_timesteps 3422.
Path 332 | total_timesteps 3430.
Path 333 | total_timesteps 3440.
Path 334 | total_timesteps 3450.
Path 335 | total_timesteps 3458.
Path 336 | total_timesteps 3470.
Path 337 | total_timesteps 3480.
Path 338 | total_timesteps 3497.
Path 339 | total_timesteps 3505.
Path 340 | total_timesteps 3518.
Path 341 | total_timesteps 3529.
Path 342 | total_timesteps 3537.
Path 343 | total_timesteps 3545.
Path 344 | total_timesteps 3553.
Path 345 | total_timesteps 3562.
Path 346 | total_timesteps 3582.
Path 347 | total_timesteps 3589.
Path 348 | total_timesteps 3598.
Path 349 | total_timesteps 3607.
Path 350 | total_timesteps 3616.
Path 351 | total_timesteps 3626.
Path 352 | total_timesteps 3633.
Path 353 | total_timesteps 3645.
Path 354 | total_timesteps 3657.
Path 355 | total_timesteps 3664.
Path 356 | total_timesteps 3676.
Path 357 | total_timesteps 3686.
Path 358 | total_timesteps 3704.
Path 359 | total_timesteps 3715.
Path 360 | total_timesteps 3723.
Path 361 | total_timesteps 3732.
Path 362 | total_timesteps 3750.
Path 363 | total_timesteps 3759.
Path 364 | total_timesteps 3767.
Path 365 | total_timesteps 3774.
Path 366 | total_timesteps 3787.
Path 367 | total_timesteps 3796.
Path 368 | total_timesteps 3813.
Path 369 | total_timesteps 3825.
Path 370 | total_timesteps 3833.
Path 371 | total_timesteps 3848.
Path 372 | total_timesteps 3860.
Path 373 | total_timesteps 3870.
Path 374 | total_timesteps 3884.
Path 375 | total_timesteps 3895.
Path 376 | total_timesteps 3905.
Path 377 | total_timesteps 3912.
Path 378 | total_timesteps 3923.
Path 379 | total_timesteps 3932.
Path 380 | total_timesteps 3945.
Path 381 | total_timesteps 3956.
Path 382 | total_timesteps 3965.
Path 383 | total_timesteps 3973.
Path 384 | total_timesteps 3985.
Path 385 | total_timesteps 4000.
Path 386 | total_timesteps 4009.
Path 387 | total_timesteps 4017.
Path 388 | total_timesteps 4026.
Path 389 | total_timesteps 4041.
Path 390 | total_timesteps 4049.
Path 391 | total_timesteps 4058.
Path 392 | total_timesteps 4067.
Path 393 | total_timesteps 4078.
Path 394 | total_timesteps 4087.
Path 395 | total_timesteps 4096.
Path 396 | total_timesteps 4104.
Path 397 | total_timesteps 4116.
Path 398 | total_timesteps 4128.
Path 399 | total_timesteps 4140.
Path 400 | total_timesteps 4150.
Path 401 | total_timesteps 4162.
Path 402 | total_timesteps 4170.
Path 403 | total_timesteps 4179.
Path 404 | total_timesteps 4195.
Path 405 | total_timesteps 4202.
Path 406 | total_timesteps 4212.
Path 407 | total_timesteps 4222.
Path 408 | total_timesteps 4232.
Path 409 | total_timesteps 4242.
Path 410 | total_timesteps 4251.
Path 411 | total_timesteps 4263.
Path 412 | total_timesteps 4271.
Path 413 | total_timesteps 4279.
Path 414 | total_timesteps 4288.
Path 415 | total_timesteps 4298.
Path 416 | total_timesteps 4309.
Path 417 | total_timesteps 4324.
Path 418 | total_timesteps 4336.
Path 419 | total_timesteps 4343.
Path 420 | total_timesteps 4353.
Path 421 | total_timesteps 4366.
Path 422 | total_timesteps 4374.
Path 423 | total_timesteps 4386.
Path 424 | total_timesteps 4395.
Path 425 | total_timesteps 4404.
Path 426 | total_timesteps 4412.
Path 427 | total_timesteps 4421.
Path 428 | total_timesteps 4432.
Path 429 | total_timesteps 4443.
Path 430 | total_timesteps 4453.
Path 431 | total_timesteps 4462.
Path 432 | total_timesteps 4474.
Path 433 | total_timesteps 4486.
Path 434 | total_timesteps 4494.
Path 435 | total_timesteps 4502.
Path 436 | total_timesteps 4514.
Path 437 | total_timesteps 4522.
Path 438 | total_timesteps 4531.
Path 439 | total_timesteps 4543.
Path 440 | total_timesteps 4552.
Path 441 | total_timesteps 4560.
Path 442 | total_timesteps 4568.
Path 443 | total_timesteps 4580.
Path 444 | total_timesteps 4592.
Path 445 | total_timesteps 4605.
Path 446 | total_timesteps 4612.
Path 447 | total_timesteps 4626.
Path 448 | total_timesteps 4633.
Path 449 | total_timesteps 4641.
Path 450 | total_timesteps 4648.
Path 451 | total_timesteps 4656.
Path 452 | total_timesteps 4664.
Path 453 | total_timesteps 4672.
Path 454 | total_timesteps 4688.
Path 455 | total_timesteps 4696.
Path 456 | total_timesteps 4707.
Path 457 | total_timesteps 4720.
Path 458 | total_timesteps 4728.
Path 459 | total_timesteps 4735.
Path 460 | total_timesteps 4748.
Path 461 | total_timesteps 4756.
Path 462 | total_timesteps 4762.
Path 463 | total_timesteps 4774.
Path 464 | total_timesteps 4782.
Path 465 | total_timesteps 4791.
Path 466 | total_timesteps 4804.
Path 467 | total_timesteps 4819.
Path 468 | total_timesteps 4838.
Path 469 | total_timesteps 4847.
Path 470 | total_timesteps 4861.
Path 471 | total_timesteps 4870.
Path 472 | total_timesteps 4880.
Path 473 | total_timesteps 4892.
Path 474 | total_timesteps 4902.
Path 475 | total_timesteps 4912.
Path 476 | total_timesteps 4919.
Path 477 | total_timesteps 4933.
Path 478 | total_timesteps 4941.
Path 479 | total_timesteps 4953.
Path 480 | total_timesteps 4962.
Path 481 | total_timesteps 4971.
Path 482 | total_timesteps 4983.
Path 483 | total_timesteps 4993.
Path 484 | total_timesteps 5001.
Path 485 | total_timesteps 5010.
Path 486 | total_timesteps 5023.
Path 487 | total_timesteps 5030.
Path 488 | total_timesteps 5040.
Path 489 | total_timesteps 5048.
Path 490 | total_timesteps 5054.
Path 491 | total_timesteps 5063.
Path 492 | total_timesteps 5075.
Path 493 | total_timesteps 5083.
Path 494 | total_timesteps 5096.
Path 495 | total_timesteps 5104.
Path 496 | total_timesteps 5114.
Path 497 | total_timesteps 5123.
Path 498 | total_timesteps 5133.
Path 499 | total_timesteps 5145.
Path 500 | total_timesteps 5155.
Path 501 | total_timesteps 5167.
Path 502 | total_timesteps 5176.
Path 503 | total_timesteps 5182.
Path 504 | total_timesteps 5191.
Path 505 | total_timesteps 5200.
Path 506 | total_timesteps 5209.
Path 507 | total_timesteps 5219.
Path 508 | total_timesteps 5228.
Path 509 | total_timesteps 5244.
Path 510 | total_timesteps 5257.
Path 511 | total_timesteps 5268.
Path 512 | total_timesteps 5276.
Path 513 | total_timesteps 5287.
Path 514 | total_timesteps 5300.
Path 515 | total_timesteps 5310.
Path 516 | total_timesteps 5326.
Path 517 | total_timesteps 5337.
Path 518 | total_timesteps 5345.
Path 519 | total_timesteps 5353.
Path 520 | total_timesteps 5364.
Path 521 | total_timesteps 5375.
Path 522 | total_timesteps 5387.
Path 523 | total_timesteps 5398.
Path 524 | total_timesteps 5408.
Path 525 | total_timesteps 5415.
Path 526 | total_timesteps 5424.
Path 527 | total_timesteps 5433.
Path 528 | total_timesteps 5443.
Path 529 | total_timesteps 5451.
Path 530 | total_timesteps 5462.
Path 531 | total_timesteps 5473.
Path 532 | total_timesteps 5481.
Path 533 | total_timesteps 5490.
Path 534 | total_timesteps 5497.
Path 535 | total_timesteps 5505.
Path 536 | total_timesteps 5516.
Path 537 | total_timesteps 5526.
Path 538 | total_timesteps 5533.
Path 539 | total_timesteps 5542.
Path 540 | total_timesteps 5549.
Path 541 | total_timesteps 5566.
Path 542 | total_timesteps 5574.
Path 543 | total_timesteps 5587.
Path 544 | total_timesteps 5602.
Path 545 | total_timesteps 5611.
Path 546 | total_timesteps 5637.
Path 547 | total_timesteps 5650.
Path 548 | total_timesteps 5659.
Path 549 | total_timesteps 5673.
Path 550 | total_timesteps 5682.
Path 551 | total_timesteps 5692.
Path 552 | total_timesteps 5701.
Path 553 | total_timesteps 5708.
Path 554 | total_timesteps 5715.
Path 555 | total_timesteps 5725.
Path 556 | total_timesteps 5737.
Path 557 | total_timesteps 5745.
Path 558 | total_timesteps 5755.
Path 559 | total_timesteps 5773.
Path 560 | total_timesteps 5783.
Path 561 | total_timesteps 5792.
Path 562 | total_timesteps 5802.
Path 563 | total_timesteps 5812.
Path 564 | total_timesteps 5825.
Path 565 | total_timesteps 5838.
Path 566 | total_timesteps 5852.
Path 567 | total_timesteps 5861.
Path 568 | total_timesteps 5873.
Path 569 | total_timesteps 5882.
Path 570 | total_timesteps 5891.
Path 571 | total_timesteps 5902.
Path 572 | total_timesteps 5911.
Path 573 | total_timesteps 5918.
Path 574 | total_timesteps 5927.
Path 575 | total_timesteps 5936.
Path 576 | total_timesteps 5948.
Path 577 | total_timesteps 5955.
Path 578 | total_timesteps 5964.
Path 579 | total_timesteps 5975.
Path 580 | total_timesteps 5984.
Path 581 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7       |
| Iteration     | 14       |
| MaximumReturn | -0.9     |
| MinimumReturn | -20.2    |
| TotalSamples  | 64052    |
----------------------------
itr #15 | 
Fitting dynamics.
Validation loss = 0.006714193616062403
Validation loss = 0.006468439474701881
Validation loss = 0.0065851593390107155
Validation loss = 0.006403754465281963
Validation loss = 0.007754958234727383
Validation loss = 0.0076905665919184685
Validation loss = 0.006807924248278141
Validation loss = 0.006486415863037109
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 32.
Path 4 | total_timesteps 46.
Path 5 | total_timesteps 54.
Path 6 | total_timesteps 64.
Path 7 | total_timesteps 74.
Path 8 | total_timesteps 81.
Path 9 | total_timesteps 89.
Path 10 | total_timesteps 96.
Path 11 | total_timesteps 104.
Path 12 | total_timesteps 111.
Path 13 | total_timesteps 120.
Path 14 | total_timesteps 130.
Path 15 | total_timesteps 140.
Path 16 | total_timesteps 152.
Path 17 | total_timesteps 159.
Path 18 | total_timesteps 169.
Path 19 | total_timesteps 179.
Path 20 | total_timesteps 188.
Path 21 | total_timesteps 197.
Path 22 | total_timesteps 208.
Path 23 | total_timesteps 217.
Path 24 | total_timesteps 230.
Path 25 | total_timesteps 237.
Path 26 | total_timesteps 248.
Path 27 | total_timesteps 257.
Path 28 | total_timesteps 268.
Path 29 | total_timesteps 277.
Path 30 | total_timesteps 287.
Path 31 | total_timesteps 296.
Path 32 | total_timesteps 304.
Path 33 | total_timesteps 320.
Path 34 | total_timesteps 327.
Path 35 | total_timesteps 337.
Path 36 | total_timesteps 345.
Path 37 | total_timesteps 354.
Path 38 | total_timesteps 364.
Path 39 | total_timesteps 373.
Path 40 | total_timesteps 386.
Path 41 | total_timesteps 397.
Path 42 | total_timesteps 404.
Path 43 | total_timesteps 413.
Path 44 | total_timesteps 420.
Path 45 | total_timesteps 429.
Path 46 | total_timesteps 438.
Path 47 | total_timesteps 445.
Path 48 | total_timesteps 456.
Path 49 | total_timesteps 466.
Path 50 | total_timesteps 476.
Path 51 | total_timesteps 485.
Path 52 | total_timesteps 499.
Path 53 | total_timesteps 510.
Path 54 | total_timesteps 521.
Path 55 | total_timesteps 537.
Path 56 | total_timesteps 544.
Path 57 | total_timesteps 556.
Path 58 | total_timesteps 570.
Path 59 | total_timesteps 579.
Path 60 | total_timesteps 588.
Path 61 | total_timesteps 595.
Path 62 | total_timesteps 603.
Path 63 | total_timesteps 616.
Path 64 | total_timesteps 625.
Path 65 | total_timesteps 633.
Path 66 | total_timesteps 642.
Path 67 | total_timesteps 654.
Path 68 | total_timesteps 661.
Path 69 | total_timesteps 670.
Path 70 | total_timesteps 677.
Path 71 | total_timesteps 692.
Path 72 | total_timesteps 717.
Path 73 | total_timesteps 724.
Path 74 | total_timesteps 736.
Path 75 | total_timesteps 754.
Path 76 | total_timesteps 767.
Path 77 | total_timesteps 777.
Path 78 | total_timesteps 785.
Path 79 | total_timesteps 793.
Path 80 | total_timesteps 805.
Path 81 | total_timesteps 815.
Path 82 | total_timesteps 825.
Path 83 | total_timesteps 842.
Path 84 | total_timesteps 849.
Path 85 | total_timesteps 857.
Path 86 | total_timesteps 866.
Path 87 | total_timesteps 884.
Path 88 | total_timesteps 897.
Path 89 | total_timesteps 906.
Path 90 | total_timesteps 913.
Path 91 | total_timesteps 922.
Path 92 | total_timesteps 933.
Path 93 | total_timesteps 944.
Path 94 | total_timesteps 953.
Path 95 | total_timesteps 963.
Path 96 | total_timesteps 974.
Path 97 | total_timesteps 986.
Path 98 | total_timesteps 993.
Path 99 | total_timesteps 1000.
Path 100 | total_timesteps 1014.
Path 101 | total_timesteps 1022.
Path 102 | total_timesteps 1037.
Path 103 | total_timesteps 1050.
Path 104 | total_timesteps 1058.
Path 105 | total_timesteps 1070.
Path 106 | total_timesteps 1078.
Path 107 | total_timesteps 1088.
Path 108 | total_timesteps 1098.
Path 109 | total_timesteps 1107.
Path 110 | total_timesteps 1117.
Path 111 | total_timesteps 1125.
Path 112 | total_timesteps 1137.
Path 113 | total_timesteps 1147.
Path 114 | total_timesteps 1160.
Path 115 | total_timesteps 1168.
Path 116 | total_timesteps 1176.
Path 117 | total_timesteps 1189.
Path 118 | total_timesteps 1196.
Path 119 | total_timesteps 1206.
Path 120 | total_timesteps 1214.
Path 121 | total_timesteps 1221.
Path 122 | total_timesteps 1233.
Path 123 | total_timesteps 1242.
Path 124 | total_timesteps 1259.
Path 125 | total_timesteps 1272.
Path 126 | total_timesteps 1284.
Path 127 | total_timesteps 1295.
Path 128 | total_timesteps 1304.
Path 129 | total_timesteps 1312.
Path 130 | total_timesteps 1319.
Path 131 | total_timesteps 1326.
Path 132 | total_timesteps 1337.
Path 133 | total_timesteps 1346.
Path 134 | total_timesteps 1356.
Path 135 | total_timesteps 1364.
Path 136 | total_timesteps 1373.
Path 137 | total_timesteps 1395.
Path 138 | total_timesteps 1402.
Path 139 | total_timesteps 1412.
Path 140 | total_timesteps 1424.
Path 141 | total_timesteps 1434.
Path 142 | total_timesteps 1446.
Path 143 | total_timesteps 1453.
Path 144 | total_timesteps 1462.
Path 145 | total_timesteps 1474.
Path 146 | total_timesteps 1482.
Path 147 | total_timesteps 1491.
Path 148 | total_timesteps 1513.
Path 149 | total_timesteps 1522.
Path 150 | total_timesteps 1528.
Path 151 | total_timesteps 1540.
Path 152 | total_timesteps 1548.
Path 153 | total_timesteps 1559.
Path 154 | total_timesteps 1568.
Path 155 | total_timesteps 1574.
Path 156 | total_timesteps 1584.
Path 157 | total_timesteps 1594.
Path 158 | total_timesteps 1604.
Path 159 | total_timesteps 1617.
Path 160 | total_timesteps 1629.
Path 161 | total_timesteps 1638.
Path 162 | total_timesteps 1650.
Path 163 | total_timesteps 1664.
Path 164 | total_timesteps 1673.
Path 165 | total_timesteps 1681.
Path 166 | total_timesteps 1691.
Path 167 | total_timesteps 1701.
Path 168 | total_timesteps 1710.
Path 169 | total_timesteps 1718.
Path 170 | total_timesteps 1729.
Path 171 | total_timesteps 1737.
Path 172 | total_timesteps 1746.
Path 173 | total_timesteps 1754.
Path 174 | total_timesteps 1765.
Path 175 | total_timesteps 1773.
Path 176 | total_timesteps 1786.
Path 177 | total_timesteps 1794.
Path 178 | total_timesteps 1805.
Path 179 | total_timesteps 1816.
Path 180 | total_timesteps 1825.
Path 181 | total_timesteps 1837.
Path 182 | total_timesteps 1848.
Path 183 | total_timesteps 1856.
Path 184 | total_timesteps 1865.
Path 185 | total_timesteps 1874.
Path 186 | total_timesteps 1882.
Path 187 | total_timesteps 1891.
Path 188 | total_timesteps 1903.
Path 189 | total_timesteps 1912.
Path 190 | total_timesteps 1924.
Path 191 | total_timesteps 1936.
Path 192 | total_timesteps 1946.
Path 193 | total_timesteps 1956.
Path 194 | total_timesteps 1969.
Path 195 | total_timesteps 1976.
Path 196 | total_timesteps 1985.
Path 197 | total_timesteps 1993.
Path 198 | total_timesteps 2000.
Path 199 | total_timesteps 2013.
Path 200 | total_timesteps 2021.
Path 201 | total_timesteps 2034.
Path 202 | total_timesteps 2043.
Path 203 | total_timesteps 2049.
Path 204 | total_timesteps 2058.
Path 205 | total_timesteps 2071.
Path 206 | total_timesteps 2084.
Path 207 | total_timesteps 2091.
Path 208 | total_timesteps 2107.
Path 209 | total_timesteps 2118.
Path 210 | total_timesteps 2127.
Path 211 | total_timesteps 2139.
Path 212 | total_timesteps 2147.
Path 213 | total_timesteps 2156.
Path 214 | total_timesteps 2169.
Path 215 | total_timesteps 2190.
Path 216 | total_timesteps 2199.
Path 217 | total_timesteps 2213.
Path 218 | total_timesteps 2223.
Path 219 | total_timesteps 2231.
Path 220 | total_timesteps 2250.
Path 221 | total_timesteps 2259.
Path 222 | total_timesteps 2270.
Path 223 | total_timesteps 2277.
Path 224 | total_timesteps 2286.
Path 225 | total_timesteps 2293.
Path 226 | total_timesteps 2302.
Path 227 | total_timesteps 2309.
Path 228 | total_timesteps 2317.
Path 229 | total_timesteps 2324.
Path 230 | total_timesteps 2332.
Path 231 | total_timesteps 2342.
Path 232 | total_timesteps 2350.
Path 233 | total_timesteps 2360.
Path 234 | total_timesteps 2367.
Path 235 | total_timesteps 2378.
Path 236 | total_timesteps 2387.
Path 237 | total_timesteps 2395.
Path 238 | total_timesteps 2403.
Path 239 | total_timesteps 2413.
Path 240 | total_timesteps 2424.
Path 241 | total_timesteps 2432.
Path 242 | total_timesteps 2442.
Path 243 | total_timesteps 2451.
Path 244 | total_timesteps 2459.
Path 245 | total_timesteps 2472.
Path 246 | total_timesteps 2484.
Path 247 | total_timesteps 2493.
Path 248 | total_timesteps 2501.
Path 249 | total_timesteps 2514.
Path 250 | total_timesteps 2525.
Path 251 | total_timesteps 2535.
Path 252 | total_timesteps 2547.
Path 253 | total_timesteps 2558.
Path 254 | total_timesteps 2567.
Path 255 | total_timesteps 2574.
Path 256 | total_timesteps 2587.
Path 257 | total_timesteps 2595.
Path 258 | total_timesteps 2606.
Path 259 | total_timesteps 2618.
Path 260 | total_timesteps 2631.
Path 261 | total_timesteps 2639.
Path 262 | total_timesteps 2650.
Path 263 | total_timesteps 2662.
Path 264 | total_timesteps 2672.
Path 265 | total_timesteps 2689.
Path 266 | total_timesteps 2710.
Path 267 | total_timesteps 2721.
Path 268 | total_timesteps 2731.
Path 269 | total_timesteps 2738.
Path 270 | total_timesteps 2746.
Path 271 | total_timesteps 2756.
Path 272 | total_timesteps 2766.
Path 273 | total_timesteps 2773.
Path 274 | total_timesteps 2785.
Path 275 | total_timesteps 2795.
Path 276 | total_timesteps 2804.
Path 277 | total_timesteps 2811.
Path 278 | total_timesteps 2828.
Path 279 | total_timesteps 2844.
Path 280 | total_timesteps 2853.
Path 281 | total_timesteps 2863.
Path 282 | total_timesteps 2873.
Path 283 | total_timesteps 2882.
Path 284 | total_timesteps 2890.
Path 285 | total_timesteps 2898.
Path 286 | total_timesteps 2906.
Path 287 | total_timesteps 2914.
Path 288 | total_timesteps 2922.
Path 289 | total_timesteps 2931.
Path 290 | total_timesteps 2941.
Path 291 | total_timesteps 2959.
Path 292 | total_timesteps 2972.
Path 293 | total_timesteps 2986.
Path 294 | total_timesteps 2996.
Path 295 | total_timesteps 3005.
Path 296 | total_timesteps 3014.
Path 297 | total_timesteps 3025.
Path 298 | total_timesteps 3035.
Path 299 | total_timesteps 3044.
Path 300 | total_timesteps 3054.
Path 301 | total_timesteps 3072.
Path 302 | total_timesteps 3082.
Path 303 | total_timesteps 3098.
Path 304 | total_timesteps 3105.
Path 305 | total_timesteps 3116.
Path 306 | total_timesteps 3128.
Path 307 | total_timesteps 3135.
Path 308 | total_timesteps 3152.
Path 309 | total_timesteps 3161.
Path 310 | total_timesteps 3172.
Path 311 | total_timesteps 3186.
Path 312 | total_timesteps 3202.
Path 313 | total_timesteps 3212.
Path 314 | total_timesteps 3221.
Path 315 | total_timesteps 3230.
Path 316 | total_timesteps 3238.
Path 317 | total_timesteps 3248.
Path 318 | total_timesteps 3257.
Path 319 | total_timesteps 3271.
Path 320 | total_timesteps 3280.
Path 321 | total_timesteps 3290.
Path 322 | total_timesteps 3297.
Path 323 | total_timesteps 3307.
Path 324 | total_timesteps 3323.
Path 325 | total_timesteps 3335.
Path 326 | total_timesteps 3352.
Path 327 | total_timesteps 3358.
Path 328 | total_timesteps 3366.
Path 329 | total_timesteps 3373.
Path 330 | total_timesteps 3383.
Path 331 | total_timesteps 3392.
Path 332 | total_timesteps 3402.
Path 333 | total_timesteps 3409.
Path 334 | total_timesteps 3421.
Path 335 | total_timesteps 3430.
Path 336 | total_timesteps 3438.
Path 337 | total_timesteps 3447.
Path 338 | total_timesteps 3464.
Path 339 | total_timesteps 3483.
Path 340 | total_timesteps 3492.
Path 341 | total_timesteps 3501.
Path 342 | total_timesteps 3509.
Path 343 | total_timesteps 3524.
Path 344 | total_timesteps 3534.
Path 345 | total_timesteps 3544.
Path 346 | total_timesteps 3553.
Path 347 | total_timesteps 3562.
Path 348 | total_timesteps 3570.
Path 349 | total_timesteps 3580.
Path 350 | total_timesteps 3588.
Path 351 | total_timesteps 3596.
Path 352 | total_timesteps 3604.
Path 353 | total_timesteps 3612.
Path 354 | total_timesteps 3623.
Path 355 | total_timesteps 3635.
Path 356 | total_timesteps 3644.
Path 357 | total_timesteps 3656.
Path 358 | total_timesteps 3669.
Path 359 | total_timesteps 3677.
Path 360 | total_timesteps 3685.
Path 361 | total_timesteps 3695.
Path 362 | total_timesteps 3711.
Path 363 | total_timesteps 3723.
Path 364 | total_timesteps 3730.
Path 365 | total_timesteps 3740.
Path 366 | total_timesteps 3749.
Path 367 | total_timesteps 3757.
Path 368 | total_timesteps 3766.
Path 369 | total_timesteps 3776.
Path 370 | total_timesteps 3788.
Path 371 | total_timesteps 3796.
Path 372 | total_timesteps 3810.
Path 373 | total_timesteps 3818.
Path 374 | total_timesteps 3829.
Path 375 | total_timesteps 3840.
Path 376 | total_timesteps 3848.
Path 377 | total_timesteps 3856.
Path 378 | total_timesteps 3866.
Path 379 | total_timesteps 3879.
Path 380 | total_timesteps 3890.
Path 381 | total_timesteps 3902.
Path 382 | total_timesteps 3910.
Path 383 | total_timesteps 3922.
Path 384 | total_timesteps 3932.
Path 385 | total_timesteps 3940.
Path 386 | total_timesteps 3954.
Path 387 | total_timesteps 3964.
Path 388 | total_timesteps 3973.
Path 389 | total_timesteps 3980.
Path 390 | total_timesteps 3990.
Path 391 | total_timesteps 3999.
Path 392 | total_timesteps 4009.
Path 393 | total_timesteps 4018.
Path 394 | total_timesteps 4040.
Path 395 | total_timesteps 4051.
Path 396 | total_timesteps 4062.
Path 397 | total_timesteps 4071.
Path 398 | total_timesteps 4079.
Path 399 | total_timesteps 4088.
Path 400 | total_timesteps 4099.
Path 401 | total_timesteps 4110.
Path 402 | total_timesteps 4121.
Path 403 | total_timesteps 4134.
Path 404 | total_timesteps 4141.
Path 405 | total_timesteps 4155.
Path 406 | total_timesteps 4161.
Path 407 | total_timesteps 4172.
Path 408 | total_timesteps 4181.
Path 409 | total_timesteps 4190.
Path 410 | total_timesteps 4201.
Path 411 | total_timesteps 4209.
Path 412 | total_timesteps 4217.
Path 413 | total_timesteps 4225.
Path 414 | total_timesteps 4238.
Path 415 | total_timesteps 4248.
Path 416 | total_timesteps 4257.
Path 417 | total_timesteps 4264.
Path 418 | total_timesteps 4274.
Path 419 | total_timesteps 4284.
Path 420 | total_timesteps 4293.
Path 421 | total_timesteps 4303.
Path 422 | total_timesteps 4311.
Path 423 | total_timesteps 4325.
Path 424 | total_timesteps 4339.
Path 425 | total_timesteps 4347.
Path 426 | total_timesteps 4357.
Path 427 | total_timesteps 4366.
Path 428 | total_timesteps 4378.
Path 429 | total_timesteps 4388.
Path 430 | total_timesteps 4396.
Path 431 | total_timesteps 4405.
Path 432 | total_timesteps 4413.
Path 433 | total_timesteps 4424.
Path 434 | total_timesteps 4435.
Path 435 | total_timesteps 4444.
Path 436 | total_timesteps 4455.
Path 437 | total_timesteps 4463.
Path 438 | total_timesteps 4471.
Path 439 | total_timesteps 4478.
Path 440 | total_timesteps 4488.
Path 441 | total_timesteps 4496.
Path 442 | total_timesteps 4506.
Path 443 | total_timesteps 4521.
Path 444 | total_timesteps 4531.
Path 445 | total_timesteps 4540.
Path 446 | total_timesteps 4550.
Path 447 | total_timesteps 4558.
Path 448 | total_timesteps 4564.
Path 449 | total_timesteps 4570.
Path 450 | total_timesteps 4591.
Path 451 | total_timesteps 4602.
Path 452 | total_timesteps 4610.
Path 453 | total_timesteps 4618.
Path 454 | total_timesteps 4630.
Path 455 | total_timesteps 4640.
Path 456 | total_timesteps 4648.
Path 457 | total_timesteps 4663.
Path 458 | total_timesteps 4682.
Path 459 | total_timesteps 4692.
Path 460 | total_timesteps 4700.
Path 461 | total_timesteps 4708.
Path 462 | total_timesteps 4717.
Path 463 | total_timesteps 4727.
Path 464 | total_timesteps 4736.
Path 465 | total_timesteps 4746.
Path 466 | total_timesteps 4757.
Path 467 | total_timesteps 4767.
Path 468 | total_timesteps 4778.
Path 469 | total_timesteps 4796.
Path 470 | total_timesteps 4807.
Path 471 | total_timesteps 4816.
Path 472 | total_timesteps 4832.
Path 473 | total_timesteps 4839.
Path 474 | total_timesteps 4845.
Path 475 | total_timesteps 4855.
Path 476 | total_timesteps 4861.
Path 477 | total_timesteps 4871.
Path 478 | total_timesteps 4879.
Path 479 | total_timesteps 4891.
Path 480 | total_timesteps 4900.
Path 481 | total_timesteps 4914.
Path 482 | total_timesteps 4923.
Path 483 | total_timesteps 4935.
Path 484 | total_timesteps 4944.
Path 485 | total_timesteps 4962.
Path 486 | total_timesteps 4977.
Path 487 | total_timesteps 4986.
Path 488 | total_timesteps 4992.
Path 489 | total_timesteps 5005.
Path 490 | total_timesteps 5015.
Path 491 | total_timesteps 5023.
Path 492 | total_timesteps 5034.
Path 493 | total_timesteps 5040.
Path 494 | total_timesteps 5049.
Path 495 | total_timesteps 5057.
Path 496 | total_timesteps 5073.
Path 497 | total_timesteps 5080.
Path 498 | total_timesteps 5087.
Path 499 | total_timesteps 5094.
Path 500 | total_timesteps 5105.
Path 501 | total_timesteps 5123.
Path 502 | total_timesteps 5140.
Path 503 | total_timesteps 5153.
Path 504 | total_timesteps 5162.
Path 505 | total_timesteps 5171.
Path 506 | total_timesteps 5181.
Path 507 | total_timesteps 5196.
Path 508 | total_timesteps 5207.
Path 509 | total_timesteps 5219.
Path 510 | total_timesteps 5232.
Path 511 | total_timesteps 5240.
Path 512 | total_timesteps 5252.
Path 513 | total_timesteps 5262.
Path 514 | total_timesteps 5271.
Path 515 | total_timesteps 5284.
Path 516 | total_timesteps 5296.
Path 517 | total_timesteps 5307.
Path 518 | total_timesteps 5326.
Path 519 | total_timesteps 5333.
Path 520 | total_timesteps 5342.
Path 521 | total_timesteps 5370.
Path 522 | total_timesteps 5378.
Path 523 | total_timesteps 5393.
Path 524 | total_timesteps 5402.
Path 525 | total_timesteps 5409.
Path 526 | total_timesteps 5422.
Path 527 | total_timesteps 5431.
Path 528 | total_timesteps 5443.
Path 529 | total_timesteps 5453.
Path 530 | total_timesteps 5461.
Path 531 | total_timesteps 5474.
Path 532 | total_timesteps 5483.
Path 533 | total_timesteps 5495.
Path 534 | total_timesteps 5506.
Path 535 | total_timesteps 5518.
Path 536 | total_timesteps 5525.
Path 537 | total_timesteps 5536.
Path 538 | total_timesteps 5549.
Path 539 | total_timesteps 5562.
Path 540 | total_timesteps 5579.
Path 541 | total_timesteps 5588.
Path 542 | total_timesteps 5596.
Path 543 | total_timesteps 5604.
Path 544 | total_timesteps 5612.
Path 545 | total_timesteps 5624.
Path 546 | total_timesteps 5634.
Path 547 | total_timesteps 5641.
Path 548 | total_timesteps 5650.
Path 549 | total_timesteps 5659.
Path 550 | total_timesteps 5665.
Path 551 | total_timesteps 5676.
Path 552 | total_timesteps 5688.
Path 553 | total_timesteps 5703.
Path 554 | total_timesteps 5710.
Path 555 | total_timesteps 5724.
Path 556 | total_timesteps 5731.
Path 557 | total_timesteps 5741.
Path 558 | total_timesteps 5748.
Path 559 | total_timesteps 5759.
Path 560 | total_timesteps 5769.
Path 561 | total_timesteps 5779.
Path 562 | total_timesteps 5789.
Path 563 | total_timesteps 5802.
Path 564 | total_timesteps 5816.
Path 565 | total_timesteps 5840.
Path 566 | total_timesteps 5848.
Path 567 | total_timesteps 5860.
Path 568 | total_timesteps 5868.
Path 569 | total_timesteps 5880.
Path 570 | total_timesteps 5888.
Path 571 | total_timesteps 5903.
Path 572 | total_timesteps 5909.
Path 573 | total_timesteps 5920.
Path 574 | total_timesteps 5927.
Path 575 | total_timesteps 5939.
Path 576 | total_timesteps 5948.
Path 577 | total_timesteps 5959.
Path 578 | total_timesteps 5967.
Path 579 | total_timesteps 5979.
Path 580 | total_timesteps 5987.
Path 581 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.84    |
| Iteration     | 15       |
| MaximumReturn | 2.43     |
| MinimumReturn | -18.7    |
| TotalSamples  | 68058    |
----------------------------
itr #16 | 
Fitting dynamics.
Validation loss = 0.0068758768029510975
Validation loss = 0.006898039020597935
Validation loss = 0.006203435827046633
Validation loss = 0.006278974004089832
Validation loss = 0.0061667300760746
Validation loss = 0.006409874185919762
Validation loss = 0.005999411456286907
Validation loss = 0.006344430148601532
Validation loss = 0.005984895396977663
Validation loss = 0.0070856730453670025
Validation loss = 0.006191594526171684
Validation loss = 0.00606800289824605
Validation loss = 0.006082944571971893
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 24.
Path 4 | total_timesteps 34.
Path 5 | total_timesteps 43.
Path 6 | total_timesteps 53.
Path 7 | total_timesteps 65.
Path 8 | total_timesteps 74.
Path 9 | total_timesteps 83.
Path 10 | total_timesteps 92.
Path 11 | total_timesteps 101.
Path 12 | total_timesteps 110.
Path 13 | total_timesteps 117.
Path 14 | total_timesteps 125.
Path 15 | total_timesteps 136.
Path 16 | total_timesteps 149.
Path 17 | total_timesteps 162.
Path 18 | total_timesteps 171.
Path 19 | total_timesteps 183.
Path 20 | total_timesteps 196.
Path 21 | total_timesteps 207.
Path 22 | total_timesteps 214.
Path 23 | total_timesteps 223.
Path 24 | total_timesteps 232.
Path 25 | total_timesteps 239.
Path 26 | total_timesteps 248.
Path 27 | total_timesteps 256.
Path 28 | total_timesteps 269.
Path 29 | total_timesteps 277.
Path 30 | total_timesteps 287.
Path 31 | total_timesteps 295.
Path 32 | total_timesteps 305.
Path 33 | total_timesteps 312.
Path 34 | total_timesteps 319.
Path 35 | total_timesteps 328.
Path 36 | total_timesteps 339.
Path 37 | total_timesteps 352.
Path 38 | total_timesteps 362.
Path 39 | total_timesteps 369.
Path 40 | total_timesteps 377.
Path 41 | total_timesteps 393.
Path 42 | total_timesteps 402.
Path 43 | total_timesteps 415.
Path 44 | total_timesteps 424.
Path 45 | total_timesteps 438.
Path 46 | total_timesteps 451.
Path 47 | total_timesteps 459.
Path 48 | total_timesteps 473.
Path 49 | total_timesteps 482.
Path 50 | total_timesteps 491.
Path 51 | total_timesteps 499.
Path 52 | total_timesteps 511.
Path 53 | total_timesteps 520.
Path 54 | total_timesteps 529.
Path 55 | total_timesteps 537.
Path 56 | total_timesteps 549.
Path 57 | total_timesteps 558.
Path 58 | total_timesteps 571.
Path 59 | total_timesteps 583.
Path 60 | total_timesteps 593.
Path 61 | total_timesteps 602.
Path 62 | total_timesteps 613.
Path 63 | total_timesteps 622.
Path 64 | total_timesteps 632.
Path 65 | total_timesteps 645.
Path 66 | total_timesteps 653.
Path 67 | total_timesteps 665.
Path 68 | total_timesteps 675.
Path 69 | total_timesteps 683.
Path 70 | total_timesteps 689.
Path 71 | total_timesteps 697.
Path 72 | total_timesteps 704.
Path 73 | total_timesteps 712.
Path 74 | total_timesteps 728.
Path 75 | total_timesteps 740.
Path 76 | total_timesteps 751.
Path 77 | total_timesteps 759.
Path 78 | total_timesteps 771.
Path 79 | total_timesteps 781.
Path 80 | total_timesteps 792.
Path 81 | total_timesteps 806.
Path 82 | total_timesteps 816.
Path 83 | total_timesteps 826.
Path 84 | total_timesteps 837.
Path 85 | total_timesteps 852.
Path 86 | total_timesteps 866.
Path 87 | total_timesteps 881.
Path 88 | total_timesteps 894.
Path 89 | total_timesteps 903.
Path 90 | total_timesteps 914.
Path 91 | total_timesteps 923.
Path 92 | total_timesteps 933.
Path 93 | total_timesteps 942.
Path 94 | total_timesteps 950.
Path 95 | total_timesteps 957.
Path 96 | total_timesteps 970.
Path 97 | total_timesteps 979.
Path 98 | total_timesteps 986.
Path 99 | total_timesteps 994.
Path 100 | total_timesteps 1003.
Path 101 | total_timesteps 1011.
Path 102 | total_timesteps 1019.
Path 103 | total_timesteps 1036.
Path 104 | total_timesteps 1043.
Path 105 | total_timesteps 1053.
Path 106 | total_timesteps 1062.
Path 107 | total_timesteps 1073.
Path 108 | total_timesteps 1084.
Path 109 | total_timesteps 1095.
Path 110 | total_timesteps 1102.
Path 111 | total_timesteps 1108.
Path 112 | total_timesteps 1118.
Path 113 | total_timesteps 1129.
Path 114 | total_timesteps 1136.
Path 115 | total_timesteps 1147.
Path 116 | total_timesteps 1156.
Path 117 | total_timesteps 1165.
Path 118 | total_timesteps 1175.
Path 119 | total_timesteps 1185.
Path 120 | total_timesteps 1193.
Path 121 | total_timesteps 1205.
Path 122 | total_timesteps 1218.
Path 123 | total_timesteps 1228.
Path 124 | total_timesteps 1238.
Path 125 | total_timesteps 1248.
Path 126 | total_timesteps 1257.
Path 127 | total_timesteps 1269.
Path 128 | total_timesteps 1279.
Path 129 | total_timesteps 1288.
Path 130 | total_timesteps 1295.
Path 131 | total_timesteps 1306.
Path 132 | total_timesteps 1313.
Path 133 | total_timesteps 1321.
Path 134 | total_timesteps 1331.
Path 135 | total_timesteps 1342.
Path 136 | total_timesteps 1350.
Path 137 | total_timesteps 1361.
Path 138 | total_timesteps 1369.
Path 139 | total_timesteps 1376.
Path 140 | total_timesteps 1383.
Path 141 | total_timesteps 1392.
Path 142 | total_timesteps 1403.
Path 143 | total_timesteps 1410.
Path 144 | total_timesteps 1429.
Path 145 | total_timesteps 1437.
Path 146 | total_timesteps 1446.
Path 147 | total_timesteps 1457.
Path 148 | total_timesteps 1467.
Path 149 | total_timesteps 1478.
Path 150 | total_timesteps 1487.
Path 151 | total_timesteps 1501.
Path 152 | total_timesteps 1508.
Path 153 | total_timesteps 1518.
Path 154 | total_timesteps 1529.
Path 155 | total_timesteps 1539.
Path 156 | total_timesteps 1551.
Path 157 | total_timesteps 1560.
Path 158 | total_timesteps 1568.
Path 159 | total_timesteps 1578.
Path 160 | total_timesteps 1586.
Path 161 | total_timesteps 1595.
Path 162 | total_timesteps 1604.
Path 163 | total_timesteps 1614.
Path 164 | total_timesteps 1624.
Path 165 | total_timesteps 1633.
Path 166 | total_timesteps 1645.
Path 167 | total_timesteps 1659.
Path 168 | total_timesteps 1672.
Path 169 | total_timesteps 1679.
Path 170 | total_timesteps 1694.
Path 171 | total_timesteps 1704.
Path 172 | total_timesteps 1712.
Path 173 | total_timesteps 1726.
Path 174 | total_timesteps 1737.
Path 175 | total_timesteps 1746.
Path 176 | total_timesteps 1757.
Path 177 | total_timesteps 1766.
Path 178 | total_timesteps 1776.
Path 179 | total_timesteps 1782.
Path 180 | total_timesteps 1789.
Path 181 | total_timesteps 1804.
Path 182 | total_timesteps 1817.
Path 183 | total_timesteps 1830.
Path 184 | total_timesteps 1840.
Path 185 | total_timesteps 1847.
Path 186 | total_timesteps 1861.
Path 187 | total_timesteps 1873.
Path 188 | total_timesteps 1884.
Path 189 | total_timesteps 1895.
Path 190 | total_timesteps 1906.
Path 191 | total_timesteps 1922.
Path 192 | total_timesteps 1936.
Path 193 | total_timesteps 1947.
Path 194 | total_timesteps 1960.
Path 195 | total_timesteps 1969.
Path 196 | total_timesteps 1988.
Path 197 | total_timesteps 1997.
Path 198 | total_timesteps 2006.
Path 199 | total_timesteps 2015.
Path 200 | total_timesteps 2024.
Path 201 | total_timesteps 2032.
Path 202 | total_timesteps 2039.
Path 203 | total_timesteps 2047.
Path 204 | total_timesteps 2059.
Path 205 | total_timesteps 2072.
Path 206 | total_timesteps 2085.
Path 207 | total_timesteps 2094.
Path 208 | total_timesteps 2102.
Path 209 | total_timesteps 2114.
Path 210 | total_timesteps 2123.
Path 211 | total_timesteps 2131.
Path 212 | total_timesteps 2139.
Path 213 | total_timesteps 2153.
Path 214 | total_timesteps 2167.
Path 215 | total_timesteps 2178.
Path 216 | total_timesteps 2187.
Path 217 | total_timesteps 2200.
Path 218 | total_timesteps 2211.
Path 219 | total_timesteps 2218.
Path 220 | total_timesteps 2224.
Path 221 | total_timesteps 2235.
Path 222 | total_timesteps 2245.
Path 223 | total_timesteps 2255.
Path 224 | total_timesteps 2261.
Path 225 | total_timesteps 2271.
Path 226 | total_timesteps 2277.
Path 227 | total_timesteps 2286.
Path 228 | total_timesteps 2294.
Path 229 | total_timesteps 2302.
Path 230 | total_timesteps 2310.
Path 231 | total_timesteps 2321.
Path 232 | total_timesteps 2332.
Path 233 | total_timesteps 2343.
Path 234 | total_timesteps 2355.
Path 235 | total_timesteps 2365.
Path 236 | total_timesteps 2372.
Path 237 | total_timesteps 2380.
Path 238 | total_timesteps 2388.
Path 239 | total_timesteps 2397.
Path 240 | total_timesteps 2405.
Path 241 | total_timesteps 2412.
Path 242 | total_timesteps 2419.
Path 243 | total_timesteps 2428.
Path 244 | total_timesteps 2440.
Path 245 | total_timesteps 2447.
Path 246 | total_timesteps 2459.
Path 247 | total_timesteps 2466.
Path 248 | total_timesteps 2475.
Path 249 | total_timesteps 2484.
Path 250 | total_timesteps 2491.
Path 251 | total_timesteps 2504.
Path 252 | total_timesteps 2518.
Path 253 | total_timesteps 2528.
Path 254 | total_timesteps 2535.
Path 255 | total_timesteps 2542.
Path 256 | total_timesteps 2553.
Path 257 | total_timesteps 2561.
Path 258 | total_timesteps 2570.
Path 259 | total_timesteps 2576.
Path 260 | total_timesteps 2585.
Path 261 | total_timesteps 2592.
Path 262 | total_timesteps 2599.
Path 263 | total_timesteps 2609.
Path 264 | total_timesteps 2617.
Path 265 | total_timesteps 2626.
Path 266 | total_timesteps 2637.
Path 267 | total_timesteps 2648.
Path 268 | total_timesteps 2657.
Path 269 | total_timesteps 2665.
Path 270 | total_timesteps 2676.
Path 271 | total_timesteps 2687.
Path 272 | total_timesteps 2697.
Path 273 | total_timesteps 2704.
Path 274 | total_timesteps 2714.
Path 275 | total_timesteps 2723.
Path 276 | total_timesteps 2731.
Path 277 | total_timesteps 2745.
Path 278 | total_timesteps 2758.
Path 279 | total_timesteps 2769.
Path 280 | total_timesteps 2780.
Path 281 | total_timesteps 2790.
Path 282 | total_timesteps 2798.
Path 283 | total_timesteps 2806.
Path 284 | total_timesteps 2824.
Path 285 | total_timesteps 2832.
Path 286 | total_timesteps 2849.
Path 287 | total_timesteps 2859.
Path 288 | total_timesteps 2866.
Path 289 | total_timesteps 2873.
Path 290 | total_timesteps 2882.
Path 291 | total_timesteps 2889.
Path 292 | total_timesteps 2896.
Path 293 | total_timesteps 2907.
Path 294 | total_timesteps 2918.
Path 295 | total_timesteps 2932.
Path 296 | total_timesteps 2943.
Path 297 | total_timesteps 2953.
Path 298 | total_timesteps 2963.
Path 299 | total_timesteps 2976.
Path 300 | total_timesteps 2987.
Path 301 | total_timesteps 2994.
Path 302 | total_timesteps 3007.
Path 303 | total_timesteps 3015.
Path 304 | total_timesteps 3027.
Path 305 | total_timesteps 3033.
Path 306 | total_timesteps 3044.
Path 307 | total_timesteps 3054.
Path 308 | total_timesteps 3063.
Path 309 | total_timesteps 3072.
Path 310 | total_timesteps 3082.
Path 311 | total_timesteps 3092.
Path 312 | total_timesteps 3111.
Path 313 | total_timesteps 3121.
Path 314 | total_timesteps 3132.
Path 315 | total_timesteps 3140.
Path 316 | total_timesteps 3147.
Path 317 | total_timesteps 3157.
Path 318 | total_timesteps 3164.
Path 319 | total_timesteps 3172.
Path 320 | total_timesteps 3179.
Path 321 | total_timesteps 3194.
Path 322 | total_timesteps 3203.
Path 323 | total_timesteps 3212.
Path 324 | total_timesteps 3223.
Path 325 | total_timesteps 3233.
Path 326 | total_timesteps 3245.
Path 327 | total_timesteps 3253.
Path 328 | total_timesteps 3263.
Path 329 | total_timesteps 3273.
Path 330 | total_timesteps 3281.
Path 331 | total_timesteps 3289.
Path 332 | total_timesteps 3302.
Path 333 | total_timesteps 3311.
Path 334 | total_timesteps 3321.
Path 335 | total_timesteps 3329.
Path 336 | total_timesteps 3340.
Path 337 | total_timesteps 3362.
Path 338 | total_timesteps 3372.
Path 339 | total_timesteps 3379.
Path 340 | total_timesteps 3386.
Path 341 | total_timesteps 3397.
Path 342 | total_timesteps 3405.
Path 343 | total_timesteps 3416.
Path 344 | total_timesteps 3430.
Path 345 | total_timesteps 3437.
Path 346 | total_timesteps 3445.
Path 347 | total_timesteps 3456.
Path 348 | total_timesteps 3464.
Path 349 | total_timesteps 3472.
Path 350 | total_timesteps 3490.
Path 351 | total_timesteps 3499.
Path 352 | total_timesteps 3511.
Path 353 | total_timesteps 3519.
Path 354 | total_timesteps 3529.
Path 355 | total_timesteps 3540.
Path 356 | total_timesteps 3551.
Path 357 | total_timesteps 3559.
Path 358 | total_timesteps 3569.
Path 359 | total_timesteps 3577.
Path 360 | total_timesteps 3593.
Path 361 | total_timesteps 3601.
Path 362 | total_timesteps 3610.
Path 363 | total_timesteps 3622.
Path 364 | total_timesteps 3630.
Path 365 | total_timesteps 3637.
Path 366 | total_timesteps 3646.
Path 367 | total_timesteps 3664.
Path 368 | total_timesteps 3674.
Path 369 | total_timesteps 3682.
Path 370 | total_timesteps 3690.
Path 371 | total_timesteps 3698.
Path 372 | total_timesteps 3707.
Path 373 | total_timesteps 3721.
Path 374 | total_timesteps 3732.
Path 375 | total_timesteps 3740.
Path 376 | total_timesteps 3748.
Path 377 | total_timesteps 3761.
Path 378 | total_timesteps 3769.
Path 379 | total_timesteps 3775.
Path 380 | total_timesteps 3784.
Path 381 | total_timesteps 3792.
Path 382 | total_timesteps 3803.
Path 383 | total_timesteps 3811.
Path 384 | total_timesteps 3824.
Path 385 | total_timesteps 3832.
Path 386 | total_timesteps 3841.
Path 387 | total_timesteps 3848.
Path 388 | total_timesteps 3858.
Path 389 | total_timesteps 3866.
Path 390 | total_timesteps 3873.
Path 391 | total_timesteps 3881.
Path 392 | total_timesteps 3890.
Path 393 | total_timesteps 3898.
Path 394 | total_timesteps 3909.
Path 395 | total_timesteps 3917.
Path 396 | total_timesteps 3926.
Path 397 | total_timesteps 3934.
Path 398 | total_timesteps 3946.
Path 399 | total_timesteps 3954.
Path 400 | total_timesteps 3961.
Path 401 | total_timesteps 3971.
Path 402 | total_timesteps 3979.
Path 403 | total_timesteps 3992.
Path 404 | total_timesteps 4002.
Path 405 | total_timesteps 4012.
Path 406 | total_timesteps 4019.
Path 407 | total_timesteps 4029.
Path 408 | total_timesteps 4042.
Path 409 | total_timesteps 4052.
Path 410 | total_timesteps 4061.
Path 411 | total_timesteps 4069.
Path 412 | total_timesteps 4078.
Path 413 | total_timesteps 4086.
Path 414 | total_timesteps 4092.
Path 415 | total_timesteps 4101.
Path 416 | total_timesteps 4108.
Path 417 | total_timesteps 4116.
Path 418 | total_timesteps 4126.
Path 419 | total_timesteps 4133.
Path 420 | total_timesteps 4146.
Path 421 | total_timesteps 4154.
Path 422 | total_timesteps 4162.
Path 423 | total_timesteps 4172.
Path 424 | total_timesteps 4182.
Path 425 | total_timesteps 4193.
Path 426 | total_timesteps 4203.
Path 427 | total_timesteps 4212.
Path 428 | total_timesteps 4225.
Path 429 | total_timesteps 4237.
Path 430 | total_timesteps 4247.
Path 431 | total_timesteps 4254.
Path 432 | total_timesteps 4263.
Path 433 | total_timesteps 4271.
Path 434 | total_timesteps 4283.
Path 435 | total_timesteps 4297.
Path 436 | total_timesteps 4307.
Path 437 | total_timesteps 4316.
Path 438 | total_timesteps 4327.
Path 439 | total_timesteps 4339.
Path 440 | total_timesteps 4355.
Path 441 | total_timesteps 4370.
Path 442 | total_timesteps 4378.
Path 443 | total_timesteps 4387.
Path 444 | total_timesteps 4398.
Path 445 | total_timesteps 4410.
Path 446 | total_timesteps 4418.
Path 447 | total_timesteps 4427.
Path 448 | total_timesteps 4437.
Path 449 | total_timesteps 4448.
Path 450 | total_timesteps 4458.
Path 451 | total_timesteps 4465.
Path 452 | total_timesteps 4473.
Path 453 | total_timesteps 4487.
Path 454 | total_timesteps 4495.
Path 455 | total_timesteps 4502.
Path 456 | total_timesteps 4512.
Path 457 | total_timesteps 4521.
Path 458 | total_timesteps 4531.
Path 459 | total_timesteps 4540.
Path 460 | total_timesteps 4548.
Path 461 | total_timesteps 4555.
Path 462 | total_timesteps 4570.
Path 463 | total_timesteps 4577.
Path 464 | total_timesteps 4585.
Path 465 | total_timesteps 4601.
Path 466 | total_timesteps 4613.
Path 467 | total_timesteps 4624.
Path 468 | total_timesteps 4633.
Path 469 | total_timesteps 4647.
Path 470 | total_timesteps 4654.
Path 471 | total_timesteps 4663.
Path 472 | total_timesteps 4672.
Path 473 | total_timesteps 4681.
Path 474 | total_timesteps 4694.
Path 475 | total_timesteps 4701.
Path 476 | total_timesteps 4709.
Path 477 | total_timesteps 4720.
Path 478 | total_timesteps 4730.
Path 479 | total_timesteps 4739.
Path 480 | total_timesteps 4749.
Path 481 | total_timesteps 4757.
Path 482 | total_timesteps 4768.
Path 483 | total_timesteps 4774.
Path 484 | total_timesteps 4784.
Path 485 | total_timesteps 4792.
Path 486 | total_timesteps 4801.
Path 487 | total_timesteps 4812.
Path 488 | total_timesteps 4823.
Path 489 | total_timesteps 4832.
Path 490 | total_timesteps 4843.
Path 491 | total_timesteps 4855.
Path 492 | total_timesteps 4861.
Path 493 | total_timesteps 4868.
Path 494 | total_timesteps 4878.
Path 495 | total_timesteps 4886.
Path 496 | total_timesteps 4894.
Path 497 | total_timesteps 4901.
Path 498 | total_timesteps 4910.
Path 499 | total_timesteps 4918.
Path 500 | total_timesteps 4927.
Path 501 | total_timesteps 4942.
Path 502 | total_timesteps 4950.
Path 503 | total_timesteps 4958.
Path 504 | total_timesteps 4966.
Path 505 | total_timesteps 4974.
Path 506 | total_timesteps 4982.
Path 507 | total_timesteps 4990.
Path 508 | total_timesteps 4996.
Path 509 | total_timesteps 5007.
Path 510 | total_timesteps 5018.
Path 511 | total_timesteps 5026.
Path 512 | total_timesteps 5037.
Path 513 | total_timesteps 5046.
Path 514 | total_timesteps 5055.
Path 515 | total_timesteps 5063.
Path 516 | total_timesteps 5071.
Path 517 | total_timesteps 5079.
Path 518 | total_timesteps 5087.
Path 519 | total_timesteps 5101.
Path 520 | total_timesteps 5111.
Path 521 | total_timesteps 5122.
Path 522 | total_timesteps 5131.
Path 523 | total_timesteps 5141.
Path 524 | total_timesteps 5154.
Path 525 | total_timesteps 5165.
Path 526 | total_timesteps 5176.
Path 527 | total_timesteps 5186.
Path 528 | total_timesteps 5196.
Path 529 | total_timesteps 5206.
Path 530 | total_timesteps 5219.
Path 531 | total_timesteps 5238.
Path 532 | total_timesteps 5248.
Path 533 | total_timesteps 5257.
Path 534 | total_timesteps 5266.
Path 535 | total_timesteps 5275.
Path 536 | total_timesteps 5284.
Path 537 | total_timesteps 5294.
Path 538 | total_timesteps 5303.
Path 539 | total_timesteps 5312.
Path 540 | total_timesteps 5319.
Path 541 | total_timesteps 5328.
Path 542 | total_timesteps 5337.
Path 543 | total_timesteps 5345.
Path 544 | total_timesteps 5353.
Path 545 | total_timesteps 5364.
Path 546 | total_timesteps 5371.
Path 547 | total_timesteps 5381.
Path 548 | total_timesteps 5393.
Path 549 | total_timesteps 5402.
Path 550 | total_timesteps 5412.
Path 551 | total_timesteps 5424.
Path 552 | total_timesteps 5432.
Path 553 | total_timesteps 5439.
Path 554 | total_timesteps 5451.
Path 555 | total_timesteps 5466.
Path 556 | total_timesteps 5476.
Path 557 | total_timesteps 5489.
Path 558 | total_timesteps 5495.
Path 559 | total_timesteps 5510.
Path 560 | total_timesteps 5518.
Path 561 | total_timesteps 5527.
Path 562 | total_timesteps 5556.
Path 563 | total_timesteps 5566.
Path 564 | total_timesteps 5576.
Path 565 | total_timesteps 5584.
Path 566 | total_timesteps 5594.
Path 567 | total_timesteps 5604.
Path 568 | total_timesteps 5615.
Path 569 | total_timesteps 5626.
Path 570 | total_timesteps 5638.
Path 571 | total_timesteps 5652.
Path 572 | total_timesteps 5658.
Path 573 | total_timesteps 5664.
Path 574 | total_timesteps 5676.
Path 575 | total_timesteps 5683.
Path 576 | total_timesteps 5692.
Path 577 | total_timesteps 5702.
Path 578 | total_timesteps 5714.
Path 579 | total_timesteps 5722.
Path 580 | total_timesteps 5734.
Path 581 | total_timesteps 5742.
Path 582 | total_timesteps 5754.
Path 583 | total_timesteps 5774.
Path 584 | total_timesteps 5781.
Path 585 | total_timesteps 5791.
Path 586 | total_timesteps 5804.
Path 587 | total_timesteps 5814.
Path 588 | total_timesteps 5821.
Path 589 | total_timesteps 5829.
Path 590 | total_timesteps 5838.
Path 591 | total_timesteps 5855.
Path 592 | total_timesteps 5864.
Path 593 | total_timesteps 5875.
Path 594 | total_timesteps 5891.
Path 595 | total_timesteps 5900.
Path 596 | total_timesteps 5906.
Path 597 | total_timesteps 5915.
Path 598 | total_timesteps 5927.
Path 599 | total_timesteps 5934.
Path 600 | total_timesteps 5942.
Path 601 | total_timesteps 5953.
Path 602 | total_timesteps 5961.
Path 603 | total_timesteps 5969.
Path 604 | total_timesteps 5983.
Path 605 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.92    |
| Iteration     | 16       |
| MaximumReturn | -1.06    |
| MinimumReturn | -21.5    |
| TotalSamples  | 72060    |
----------------------------
itr #17 | 
Fitting dynamics.
Validation loss = 0.006694182753562927
Validation loss = 0.0060573481023311615
Validation loss = 0.006226187106221914
Validation loss = 0.0059724547900259495
Validation loss = 0.006220515817403793
Validation loss = 0.0068927351385355
Validation loss = 0.006271822843700647
Validation loss = 0.006273491308093071
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 25.
Path 2 | total_timesteps 38.
Path 3 | total_timesteps 51.
Path 4 | total_timesteps 59.
Path 5 | total_timesteps 66.
Path 6 | total_timesteps 77.
Path 7 | total_timesteps 85.
Path 8 | total_timesteps 95.
Path 9 | total_timesteps 115.
Path 10 | total_timesteps 129.
Path 11 | total_timesteps 138.
Path 12 | total_timesteps 147.
Path 13 | total_timesteps 160.
Path 14 | total_timesteps 167.
Path 15 | total_timesteps 176.
Path 16 | total_timesteps 184.
Path 17 | total_timesteps 193.
Path 18 | total_timesteps 210.
Path 19 | total_timesteps 221.
Path 20 | total_timesteps 231.
Path 21 | total_timesteps 249.
Path 22 | total_timesteps 257.
Path 23 | total_timesteps 269.
Path 24 | total_timesteps 281.
Path 25 | total_timesteps 288.
Path 26 | total_timesteps 296.
Path 27 | total_timesteps 310.
Path 28 | total_timesteps 321.
Path 29 | total_timesteps 333.
Path 30 | total_timesteps 341.
Path 31 | total_timesteps 357.
Path 32 | total_timesteps 365.
Path 33 | total_timesteps 376.
Path 34 | total_timesteps 389.
Path 35 | total_timesteps 397.
Path 36 | total_timesteps 408.
Path 37 | total_timesteps 419.
Path 38 | total_timesteps 431.
Path 39 | total_timesteps 439.
Path 40 | total_timesteps 449.
Path 41 | total_timesteps 455.
Path 42 | total_timesteps 464.
Path 43 | total_timesteps 471.
Path 44 | total_timesteps 485.
Path 45 | total_timesteps 497.
Path 46 | total_timesteps 508.
Path 47 | total_timesteps 520.
Path 48 | total_timesteps 528.
Path 49 | total_timesteps 536.
Path 50 | total_timesteps 548.
Path 51 | total_timesteps 556.
Path 52 | total_timesteps 569.
Path 53 | total_timesteps 578.
Path 54 | total_timesteps 588.
Path 55 | total_timesteps 597.
Path 56 | total_timesteps 607.
Path 57 | total_timesteps 615.
Path 58 | total_timesteps 623.
Path 59 | total_timesteps 634.
Path 60 | total_timesteps 641.
Path 61 | total_timesteps 659.
Path 62 | total_timesteps 668.
Path 63 | total_timesteps 682.
Path 64 | total_timesteps 693.
Path 65 | total_timesteps 700.
Path 66 | total_timesteps 711.
Path 67 | total_timesteps 723.
Path 68 | total_timesteps 736.
Path 69 | total_timesteps 744.
Path 70 | total_timesteps 756.
Path 71 | total_timesteps 771.
Path 72 | total_timesteps 786.
Path 73 | total_timesteps 794.
Path 74 | total_timesteps 805.
Path 75 | total_timesteps 813.
Path 76 | total_timesteps 824.
Path 77 | total_timesteps 836.
Path 78 | total_timesteps 844.
Path 79 | total_timesteps 853.
Path 80 | total_timesteps 860.
Path 81 | total_timesteps 871.
Path 82 | total_timesteps 883.
Path 83 | total_timesteps 892.
Path 84 | total_timesteps 903.
Path 85 | total_timesteps 919.
Path 86 | total_timesteps 931.
Path 87 | total_timesteps 948.
Path 88 | total_timesteps 957.
Path 89 | total_timesteps 969.
Path 90 | total_timesteps 980.
Path 91 | total_timesteps 993.
Path 92 | total_timesteps 1003.
Path 93 | total_timesteps 1012.
Path 94 | total_timesteps 1020.
Path 95 | total_timesteps 1032.
Path 96 | total_timesteps 1038.
Path 97 | total_timesteps 1048.
Path 98 | total_timesteps 1058.
Path 99 | total_timesteps 1077.
Path 100 | total_timesteps 1089.
Path 101 | total_timesteps 1098.
Path 102 | total_timesteps 1108.
Path 103 | total_timesteps 1119.
Path 104 | total_timesteps 1130.
Path 105 | total_timesteps 1139.
Path 106 | total_timesteps 1150.
Path 107 | total_timesteps 1159.
Path 108 | total_timesteps 1172.
Path 109 | total_timesteps 1187.
Path 110 | total_timesteps 1195.
Path 111 | total_timesteps 1207.
Path 112 | total_timesteps 1216.
Path 113 | total_timesteps 1227.
Path 114 | total_timesteps 1237.
Path 115 | total_timesteps 1249.
Path 116 | total_timesteps 1259.
Path 117 | total_timesteps 1269.
Path 118 | total_timesteps 1277.
Path 119 | total_timesteps 1284.
Path 120 | total_timesteps 1295.
Path 121 | total_timesteps 1305.
Path 122 | total_timesteps 1313.
Path 123 | total_timesteps 1324.
Path 124 | total_timesteps 1335.
Path 125 | total_timesteps 1344.
Path 126 | total_timesteps 1360.
Path 127 | total_timesteps 1367.
Path 128 | total_timesteps 1392.
Path 129 | total_timesteps 1406.
Path 130 | total_timesteps 1415.
Path 131 | total_timesteps 1425.
Path 132 | total_timesteps 1436.
Path 133 | total_timesteps 1446.
Path 134 | total_timesteps 1454.
Path 135 | total_timesteps 1471.
Path 136 | total_timesteps 1483.
Path 137 | total_timesteps 1500.
Path 138 | total_timesteps 1508.
Path 139 | total_timesteps 1519.
Path 140 | total_timesteps 1529.
Path 141 | total_timesteps 1540.
Path 142 | total_timesteps 1552.
Path 143 | total_timesteps 1563.
Path 144 | total_timesteps 1575.
Path 145 | total_timesteps 1590.
Path 146 | total_timesteps 1597.
Path 147 | total_timesteps 1609.
Path 148 | total_timesteps 1620.
Path 149 | total_timesteps 1635.
Path 150 | total_timesteps 1652.
Path 151 | total_timesteps 1664.
Path 152 | total_timesteps 1676.
Path 153 | total_timesteps 1684.
Path 154 | total_timesteps 1699.
Path 155 | total_timesteps 1709.
Path 156 | total_timesteps 1723.
Path 157 | total_timesteps 1731.
Path 158 | total_timesteps 1741.
Path 159 | total_timesteps 1749.
Path 160 | total_timesteps 1757.
Path 161 | total_timesteps 1773.
Path 162 | total_timesteps 1781.
Path 163 | total_timesteps 1788.
Path 164 | total_timesteps 1803.
Path 165 | total_timesteps 1811.
Path 166 | total_timesteps 1822.
Path 167 | total_timesteps 1832.
Path 168 | total_timesteps 1841.
Path 169 | total_timesteps 1852.
Path 170 | total_timesteps 1863.
Path 171 | total_timesteps 1871.
Path 172 | total_timesteps 1883.
Path 173 | total_timesteps 1892.
Path 174 | total_timesteps 1900.
Path 175 | total_timesteps 1909.
Path 176 | total_timesteps 1917.
Path 177 | total_timesteps 1932.
Path 178 | total_timesteps 1940.
Path 179 | total_timesteps 1953.
Path 180 | total_timesteps 1961.
Path 181 | total_timesteps 1972.
Path 182 | total_timesteps 1979.
Path 183 | total_timesteps 1989.
Path 184 | total_timesteps 1997.
Path 185 | total_timesteps 2006.
Path 186 | total_timesteps 2024.
Path 187 | total_timesteps 2033.
Path 188 | total_timesteps 2047.
Path 189 | total_timesteps 2059.
Path 190 | total_timesteps 2068.
Path 191 | total_timesteps 2081.
Path 192 | total_timesteps 2100.
Path 193 | total_timesteps 2109.
Path 194 | total_timesteps 2125.
Path 195 | total_timesteps 2133.
Path 196 | total_timesteps 2144.
Path 197 | total_timesteps 2153.
Path 198 | total_timesteps 2163.
Path 199 | total_timesteps 2171.
Path 200 | total_timesteps 2178.
Path 201 | total_timesteps 2185.
Path 202 | total_timesteps 2194.
Path 203 | total_timesteps 2203.
Path 204 | total_timesteps 2219.
Path 205 | total_timesteps 2229.
Path 206 | total_timesteps 2236.
Path 207 | total_timesteps 2243.
Path 208 | total_timesteps 2253.
Path 209 | total_timesteps 2269.
Path 210 | total_timesteps 2283.
Path 211 | total_timesteps 2291.
Path 212 | total_timesteps 2302.
Path 213 | total_timesteps 2308.
Path 214 | total_timesteps 2321.
Path 215 | total_timesteps 2328.
Path 216 | total_timesteps 2338.
Path 217 | total_timesteps 2357.
Path 218 | total_timesteps 2364.
Path 219 | total_timesteps 2371.
Path 220 | total_timesteps 2378.
Path 221 | total_timesteps 2390.
Path 222 | total_timesteps 2396.
Path 223 | total_timesteps 2404.
Path 224 | total_timesteps 2415.
Path 225 | total_timesteps 2429.
Path 226 | total_timesteps 2444.
Path 227 | total_timesteps 2453.
Path 228 | total_timesteps 2469.
Path 229 | total_timesteps 2478.
Path 230 | total_timesteps 2491.
Path 231 | total_timesteps 2501.
Path 232 | total_timesteps 2516.
Path 233 | total_timesteps 2525.
Path 234 | total_timesteps 2542.
Path 235 | total_timesteps 2556.
Path 236 | total_timesteps 2566.
Path 237 | total_timesteps 2579.
Path 238 | total_timesteps 2591.
Path 239 | total_timesteps 2598.
Path 240 | total_timesteps 2611.
Path 241 | total_timesteps 2626.
Path 242 | total_timesteps 2635.
Path 243 | total_timesteps 2643.
Path 244 | total_timesteps 2653.
Path 245 | total_timesteps 2659.
Path 246 | total_timesteps 2673.
Path 247 | total_timesteps 2680.
Path 248 | total_timesteps 2698.
Path 249 | total_timesteps 2713.
Path 250 | total_timesteps 2721.
Path 251 | total_timesteps 2728.
Path 252 | total_timesteps 2737.
Path 253 | total_timesteps 2748.
Path 254 | total_timesteps 2758.
Path 255 | total_timesteps 2769.
Path 256 | total_timesteps 2775.
Path 257 | total_timesteps 2790.
Path 258 | total_timesteps 2801.
Path 259 | total_timesteps 2814.
Path 260 | total_timesteps 2823.
Path 261 | total_timesteps 2834.
Path 262 | total_timesteps 2845.
Path 263 | total_timesteps 2856.
Path 264 | total_timesteps 2867.
Path 265 | total_timesteps 2877.
Path 266 | total_timesteps 2892.
Path 267 | total_timesteps 2902.
Path 268 | total_timesteps 2914.
Path 269 | total_timesteps 2924.
Path 270 | total_timesteps 2935.
Path 271 | total_timesteps 2946.
Path 272 | total_timesteps 2961.
Path 273 | total_timesteps 2972.
Path 274 | total_timesteps 2985.
Path 275 | total_timesteps 3002.
Path 276 | total_timesteps 3012.
Path 277 | total_timesteps 3026.
Path 278 | total_timesteps 3041.
Path 279 | total_timesteps 3052.
Path 280 | total_timesteps 3061.
Path 281 | total_timesteps 3070.
Path 282 | total_timesteps 3079.
Path 283 | total_timesteps 3094.
Path 284 | total_timesteps 3105.
Path 285 | total_timesteps 3129.
Path 286 | total_timesteps 3140.
Path 287 | total_timesteps 3149.
Path 288 | total_timesteps 3157.
Path 289 | total_timesteps 3167.
Path 290 | total_timesteps 3181.
Path 291 | total_timesteps 3190.
Path 292 | total_timesteps 3198.
Path 293 | total_timesteps 3208.
Path 294 | total_timesteps 3223.
Path 295 | total_timesteps 3236.
Path 296 | total_timesteps 3255.
Path 297 | total_timesteps 3264.
Path 298 | total_timesteps 3273.
Path 299 | total_timesteps 3282.
Path 300 | total_timesteps 3292.
Path 301 | total_timesteps 3306.
Path 302 | total_timesteps 3314.
Path 303 | total_timesteps 3323.
Path 304 | total_timesteps 3331.
Path 305 | total_timesteps 3341.
Path 306 | total_timesteps 3350.
Path 307 | total_timesteps 3358.
Path 308 | total_timesteps 3367.
Path 309 | total_timesteps 3378.
Path 310 | total_timesteps 3391.
Path 311 | total_timesteps 3401.
Path 312 | total_timesteps 3410.
Path 313 | total_timesteps 3423.
Path 314 | total_timesteps 3429.
Path 315 | total_timesteps 3441.
Path 316 | total_timesteps 3452.
Path 317 | total_timesteps 3464.
Path 318 | total_timesteps 3471.
Path 319 | total_timesteps 3480.
Path 320 | total_timesteps 3490.
Path 321 | total_timesteps 3499.
Path 322 | total_timesteps 3513.
Path 323 | total_timesteps 3522.
Path 324 | total_timesteps 3530.
Path 325 | total_timesteps 3541.
Path 326 | total_timesteps 3556.
Path 327 | total_timesteps 3565.
Path 328 | total_timesteps 3578.
Path 329 | total_timesteps 3589.
Path 330 | total_timesteps 3598.
Path 331 | total_timesteps 3608.
Path 332 | total_timesteps 3616.
Path 333 | total_timesteps 3626.
Path 334 | total_timesteps 3635.
Path 335 | total_timesteps 3646.
Path 336 | total_timesteps 3658.
Path 337 | total_timesteps 3669.
Path 338 | total_timesteps 3678.
Path 339 | total_timesteps 3686.
Path 340 | total_timesteps 3692.
Path 341 | total_timesteps 3702.
Path 342 | total_timesteps 3713.
Path 343 | total_timesteps 3720.
Path 344 | total_timesteps 3728.
Path 345 | total_timesteps 3740.
Path 346 | total_timesteps 3750.
Path 347 | total_timesteps 3771.
Path 348 | total_timesteps 3784.
Path 349 | total_timesteps 3793.
Path 350 | total_timesteps 3806.
Path 351 | total_timesteps 3816.
Path 352 | total_timesteps 3827.
Path 353 | total_timesteps 3839.
Path 354 | total_timesteps 3853.
Path 355 | total_timesteps 3861.
Path 356 | total_timesteps 3872.
Path 357 | total_timesteps 3881.
Path 358 | total_timesteps 3888.
Path 359 | total_timesteps 3896.
Path 360 | total_timesteps 3904.
Path 361 | total_timesteps 3921.
Path 362 | total_timesteps 3932.
Path 363 | total_timesteps 3954.
Path 364 | total_timesteps 3971.
Path 365 | total_timesteps 3979.
Path 366 | total_timesteps 3991.
Path 367 | total_timesteps 4003.
Path 368 | total_timesteps 4013.
Path 369 | total_timesteps 4025.
Path 370 | total_timesteps 4038.
Path 371 | total_timesteps 4046.
Path 372 | total_timesteps 4056.
Path 373 | total_timesteps 4065.
Path 374 | total_timesteps 4076.
Path 375 | total_timesteps 4086.
Path 376 | total_timesteps 4097.
Path 377 | total_timesteps 4107.
Path 378 | total_timesteps 4115.
Path 379 | total_timesteps 4131.
Path 380 | total_timesteps 4143.
Path 381 | total_timesteps 4154.
Path 382 | total_timesteps 4171.
Path 383 | total_timesteps 4188.
Path 384 | total_timesteps 4203.
Path 385 | total_timesteps 4213.
Path 386 | total_timesteps 4223.
Path 387 | total_timesteps 4236.
Path 388 | total_timesteps 4247.
Path 389 | total_timesteps 4257.
Path 390 | total_timesteps 4265.
Path 391 | total_timesteps 4276.
Path 392 | total_timesteps 4287.
Path 393 | total_timesteps 4301.
Path 394 | total_timesteps 4312.
Path 395 | total_timesteps 4321.
Path 396 | total_timesteps 4332.
Path 397 | total_timesteps 4348.
Path 398 | total_timesteps 4355.
Path 399 | total_timesteps 4371.
Path 400 | total_timesteps 4382.
Path 401 | total_timesteps 4392.
Path 402 | total_timesteps 4400.
Path 403 | total_timesteps 4411.
Path 404 | total_timesteps 4422.
Path 405 | total_timesteps 4432.
Path 406 | total_timesteps 4442.
Path 407 | total_timesteps 4452.
Path 408 | total_timesteps 4465.
Path 409 | total_timesteps 4473.
Path 410 | total_timesteps 4484.
Path 411 | total_timesteps 4495.
Path 412 | total_timesteps 4508.
Path 413 | total_timesteps 4515.
Path 414 | total_timesteps 4525.
Path 415 | total_timesteps 4532.
Path 416 | total_timesteps 4543.
Path 417 | total_timesteps 4550.
Path 418 | total_timesteps 4564.
Path 419 | total_timesteps 4579.
Path 420 | total_timesteps 4591.
Path 421 | total_timesteps 4605.
Path 422 | total_timesteps 4613.
Path 423 | total_timesteps 4624.
Path 424 | total_timesteps 4633.
Path 425 | total_timesteps 4643.
Path 426 | total_timesteps 4656.
Path 427 | total_timesteps 4664.
Path 428 | total_timesteps 4675.
Path 429 | total_timesteps 4687.
Path 430 | total_timesteps 4694.
Path 431 | total_timesteps 4703.
Path 432 | total_timesteps 4715.
Path 433 | total_timesteps 4726.
Path 434 | total_timesteps 4734.
Path 435 | total_timesteps 4743.
Path 436 | total_timesteps 4752.
Path 437 | total_timesteps 4761.
Path 438 | total_timesteps 4775.
Path 439 | total_timesteps 4785.
Path 440 | total_timesteps 4796.
Path 441 | total_timesteps 4808.
Path 442 | total_timesteps 4818.
Path 443 | total_timesteps 4831.
Path 444 | total_timesteps 4839.
Path 445 | total_timesteps 4850.
Path 446 | total_timesteps 4859.
Path 447 | total_timesteps 4870.
Path 448 | total_timesteps 4879.
Path 449 | total_timesteps 4892.
Path 450 | total_timesteps 4899.
Path 451 | total_timesteps 4908.
Path 452 | total_timesteps 4921.
Path 453 | total_timesteps 4930.
Path 454 | total_timesteps 4946.
Path 455 | total_timesteps 4956.
Path 456 | total_timesteps 4969.
Path 457 | total_timesteps 4977.
Path 458 | total_timesteps 4985.
Path 459 | total_timesteps 4998.
Path 460 | total_timesteps 5006.
Path 461 | total_timesteps 5019.
Path 462 | total_timesteps 5026.
Path 463 | total_timesteps 5037.
Path 464 | total_timesteps 5045.
Path 465 | total_timesteps 5054.
Path 466 | total_timesteps 5064.
Path 467 | total_timesteps 5075.
Path 468 | total_timesteps 5087.
Path 469 | total_timesteps 5098.
Path 470 | total_timesteps 5109.
Path 471 | total_timesteps 5120.
Path 472 | total_timesteps 5136.
Path 473 | total_timesteps 5151.
Path 474 | total_timesteps 5161.
Path 475 | total_timesteps 5170.
Path 476 | total_timesteps 5179.
Path 477 | total_timesteps 5189.
Path 478 | total_timesteps 5198.
Path 479 | total_timesteps 5213.
Path 480 | total_timesteps 5225.
Path 481 | total_timesteps 5233.
Path 482 | total_timesteps 5245.
Path 483 | total_timesteps 5251.
Path 484 | total_timesteps 5259.
Path 485 | total_timesteps 5274.
Path 486 | total_timesteps 5282.
Path 487 | total_timesteps 5294.
Path 488 | total_timesteps 5304.
Path 489 | total_timesteps 5326.
Path 490 | total_timesteps 5335.
Path 491 | total_timesteps 5345.
Path 492 | total_timesteps 5352.
Path 493 | total_timesteps 5361.
Path 494 | total_timesteps 5368.
Path 495 | total_timesteps 5377.
Path 496 | total_timesteps 5393.
Path 497 | total_timesteps 5403.
Path 498 | total_timesteps 5410.
Path 499 | total_timesteps 5423.
Path 500 | total_timesteps 5441.
Path 501 | total_timesteps 5450.
Path 502 | total_timesteps 5465.
Path 503 | total_timesteps 5476.
Path 504 | total_timesteps 5484.
Path 505 | total_timesteps 5492.
Path 506 | total_timesteps 5503.
Path 507 | total_timesteps 5514.
Path 508 | total_timesteps 5532.
Path 509 | total_timesteps 5545.
Path 510 | total_timesteps 5556.
Path 511 | total_timesteps 5581.
Path 512 | total_timesteps 5592.
Path 513 | total_timesteps 5602.
Path 514 | total_timesteps 5615.
Path 515 | total_timesteps 5623.
Path 516 | total_timesteps 5640.
Path 517 | total_timesteps 5649.
Path 518 | total_timesteps 5663.
Path 519 | total_timesteps 5672.
Path 520 | total_timesteps 5685.
Path 521 | total_timesteps 5696.
Path 522 | total_timesteps 5715.
Path 523 | total_timesteps 5727.
Path 524 | total_timesteps 5734.
Path 525 | total_timesteps 5751.
Path 526 | total_timesteps 5760.
Path 527 | total_timesteps 5772.
Path 528 | total_timesteps 5780.
Path 529 | total_timesteps 5794.
Path 530 | total_timesteps 5810.
Path 531 | total_timesteps 5817.
Path 532 | total_timesteps 5825.
Path 533 | total_timesteps 5843.
Path 534 | total_timesteps 5859.
Path 535 | total_timesteps 5872.
Path 536 | total_timesteps 5885.
Path 537 | total_timesteps 5904.
Path 538 | total_timesteps 5928.
Path 539 | total_timesteps 5937.
Path 540 | total_timesteps 5947.
Path 541 | total_timesteps 5957.
Path 542 | total_timesteps 5966.
Path 543 | total_timesteps 5978.
Path 544 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.95    |
| Iteration     | 17       |
| MaximumReturn | 5.24     |
| MinimumReturn | -21.4    |
| TotalSamples  | 76063    |
----------------------------
itr #18 | 
Fitting dynamics.
Validation loss = 0.006178694777190685
Validation loss = 0.006362464744597673
Validation loss = 0.005823749583214521
Validation loss = 0.005872040055692196
Validation loss = 0.0056954617612063885
Validation loss = 0.005805869121104479
Validation loss = 0.0058168694376945496
Validation loss = 0.006440976168960333
Validation loss = 0.005651124753057957
Validation loss = 0.005842147395014763
Validation loss = 0.005810909904539585
Validation loss = 0.005800975952297449
Validation loss = 0.005713750142604113
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 39.
Path 5 | total_timesteps 67.
Path 6 | total_timesteps 81.
Path 7 | total_timesteps 99.
Path 8 | total_timesteps 109.
Path 9 | total_timesteps 125.
Path 10 | total_timesteps 141.
Path 11 | total_timesteps 157.
Path 12 | total_timesteps 175.
Path 13 | total_timesteps 212.
Path 14 | total_timesteps 221.
Path 15 | total_timesteps 238.
Path 16 | total_timesteps 250.
Path 17 | total_timesteps 266.
Path 18 | total_timesteps 284.
Path 19 | total_timesteps 295.
Path 20 | total_timesteps 305.
Path 21 | total_timesteps 316.
Path 22 | total_timesteps 330.
Path 23 | total_timesteps 340.
Path 24 | total_timesteps 357.
Path 25 | total_timesteps 382.
Path 26 | total_timesteps 392.
Path 27 | total_timesteps 413.
Path 28 | total_timesteps 430.
Path 29 | total_timesteps 447.
Path 30 | total_timesteps 457.
Path 31 | total_timesteps 468.
Path 32 | total_timesteps 482.
Path 33 | total_timesteps 505.
Path 34 | total_timesteps 520.
Path 35 | total_timesteps 552.
Path 36 | total_timesteps 560.
Path 37 | total_timesteps 580.
Path 38 | total_timesteps 589.
Path 39 | total_timesteps 598.
Path 40 | total_timesteps 615.
Path 41 | total_timesteps 628.
Path 42 | total_timesteps 643.
Path 43 | total_timesteps 651.
Path 44 | total_timesteps 662.
Path 45 | total_timesteps 676.
Path 46 | total_timesteps 693.
Path 47 | total_timesteps 705.
Path 48 | total_timesteps 722.
Path 49 | total_timesteps 755.
Path 50 | total_timesteps 770.
Path 51 | total_timesteps 784.
Path 52 | total_timesteps 803.
Path 53 | total_timesteps 817.
Path 54 | total_timesteps 830.
Path 55 | total_timesteps 854.
Path 56 | total_timesteps 869.
Path 57 | total_timesteps 880.
Path 58 | total_timesteps 894.
Path 59 | total_timesteps 905.
Path 60 | total_timesteps 918.
Path 61 | total_timesteps 940.
Path 62 | total_timesteps 956.
Path 63 | total_timesteps 976.
Path 64 | total_timesteps 999.
Path 65 | total_timesteps 1012.
Path 66 | total_timesteps 1026.
Path 67 | total_timesteps 1037.
Path 68 | total_timesteps 1056.
Path 69 | total_timesteps 1068.
Path 70 | total_timesteps 1086.
Path 71 | total_timesteps 1095.
Path 72 | total_timesteps 1109.
Path 73 | total_timesteps 1127.
Path 74 | total_timesteps 1140.
Path 75 | total_timesteps 1156.
Path 76 | total_timesteps 1165.
Path 77 | total_timesteps 1181.
Path 78 | total_timesteps 1194.
Path 79 | total_timesteps 1218.
Path 80 | total_timesteps 1241.
Path 81 | total_timesteps 1260.
Path 82 | total_timesteps 1270.
Path 83 | total_timesteps 1288.
Path 84 | total_timesteps 1297.
Path 85 | total_timesteps 1324.
Path 86 | total_timesteps 1334.
Path 87 | total_timesteps 1353.
Path 88 | total_timesteps 1361.
Path 89 | total_timesteps 1389.
Path 90 | total_timesteps 1408.
Path 91 | total_timesteps 1421.
Path 92 | total_timesteps 1441.
Path 93 | total_timesteps 1457.
Path 94 | total_timesteps 1478.
Path 95 | total_timesteps 1505.
Path 96 | total_timesteps 1514.
Path 97 | total_timesteps 1527.
Path 98 | total_timesteps 1537.
Path 99 | total_timesteps 1560.
Path 100 | total_timesteps 1585.
Path 101 | total_timesteps 1610.
Path 102 | total_timesteps 1624.
Path 103 | total_timesteps 1637.
Path 104 | total_timesteps 1661.
Path 105 | total_timesteps 1671.
Path 106 | total_timesteps 1687.
Path 107 | total_timesteps 1701.
Path 108 | total_timesteps 1720.
Path 109 | total_timesteps 1731.
Path 110 | total_timesteps 1752.
Path 111 | total_timesteps 1780.
Path 112 | total_timesteps 1788.
Path 113 | total_timesteps 1803.
Path 114 | total_timesteps 1822.
Path 115 | total_timesteps 1839.
Path 116 | total_timesteps 1873.
Path 117 | total_timesteps 1887.
Path 118 | total_timesteps 1902.
Path 119 | total_timesteps 1924.
Path 120 | total_timesteps 1942.
Path 121 | total_timesteps 1961.
Path 122 | total_timesteps 1975.
Path 123 | total_timesteps 1982.
Path 124 | total_timesteps 1989.
Path 125 | total_timesteps 1996.
Path 126 | total_timesteps 2035.
Path 127 | total_timesteps 2046.
Path 128 | total_timesteps 2057.
Path 129 | total_timesteps 2068.
Path 130 | total_timesteps 2086.
Path 131 | total_timesteps 2103.
Path 132 | total_timesteps 2130.
Path 133 | total_timesteps 2148.
Path 134 | total_timesteps 2159.
Path 135 | total_timesteps 2180.
Path 136 | total_timesteps 2200.
Path 137 | total_timesteps 2212.
Path 138 | total_timesteps 2221.
Path 139 | total_timesteps 2232.
Path 140 | total_timesteps 2260.
Path 141 | total_timesteps 2269.
Path 142 | total_timesteps 2280.
Path 143 | total_timesteps 2300.
Path 144 | total_timesteps 2317.
Path 145 | total_timesteps 2339.
Path 146 | total_timesteps 2356.
Path 147 | total_timesteps 2371.
Path 148 | total_timesteps 2383.
Path 149 | total_timesteps 2392.
Path 150 | total_timesteps 2409.
Path 151 | total_timesteps 2422.
Path 152 | total_timesteps 2436.
Path 153 | total_timesteps 2452.
Path 154 | total_timesteps 2464.
Path 155 | total_timesteps 2488.
Path 156 | total_timesteps 2496.
Path 157 | total_timesteps 2518.
Path 158 | total_timesteps 2532.
Path 159 | total_timesteps 2550.
Path 160 | total_timesteps 2563.
Path 161 | total_timesteps 2585.
Path 162 | total_timesteps 2598.
Path 163 | total_timesteps 2617.
Path 164 | total_timesteps 2642.
Path 165 | total_timesteps 2650.
Path 166 | total_timesteps 2663.
Path 167 | total_timesteps 2674.
Path 168 | total_timesteps 2698.
Path 169 | total_timesteps 2721.
Path 170 | total_timesteps 2734.
Path 171 | total_timesteps 2755.
Path 172 | total_timesteps 2764.
Path 173 | total_timesteps 2784.
Path 174 | total_timesteps 2798.
Path 175 | total_timesteps 2812.
Path 176 | total_timesteps 2827.
Path 177 | total_timesteps 2849.
Path 178 | total_timesteps 2860.
Path 179 | total_timesteps 2892.
Path 180 | total_timesteps 2907.
Path 181 | total_timesteps 2921.
Path 182 | total_timesteps 2933.
Path 183 | total_timesteps 2953.
Path 184 | total_timesteps 2969.
Path 185 | total_timesteps 2984.
Path 186 | total_timesteps 3002.
Path 187 | total_timesteps 3020.
Path 188 | total_timesteps 3029.
Path 189 | total_timesteps 3042.
Path 190 | total_timesteps 3063.
Path 191 | total_timesteps 3078.
Path 192 | total_timesteps 3101.
Path 193 | total_timesteps 3115.
Path 194 | total_timesteps 3124.
Path 195 | total_timesteps 3140.
Path 196 | total_timesteps 3161.
Path 197 | total_timesteps 3181.
Path 198 | total_timesteps 3194.
Path 199 | total_timesteps 3214.
Path 200 | total_timesteps 3243.
Path 201 | total_timesteps 3261.
Path 202 | total_timesteps 3271.
Path 203 | total_timesteps 3291.
Path 204 | total_timesteps 3304.
Path 205 | total_timesteps 3316.
Path 206 | total_timesteps 3327.
Path 207 | total_timesteps 3337.
Path 208 | total_timesteps 3350.
Path 209 | total_timesteps 3369.
Path 210 | total_timesteps 3388.
Path 211 | total_timesteps 3409.
Path 212 | total_timesteps 3422.
Path 213 | total_timesteps 3445.
Path 214 | total_timesteps 3458.
Path 215 | total_timesteps 3477.
Path 216 | total_timesteps 3486.
Path 217 | total_timesteps 3502.
Path 218 | total_timesteps 3529.
Path 219 | total_timesteps 3542.
Path 220 | total_timesteps 3553.
Path 221 | total_timesteps 3564.
Path 222 | total_timesteps 3577.
Path 223 | total_timesteps 3596.
Path 224 | total_timesteps 3605.
Path 225 | total_timesteps 3621.
Path 226 | total_timesteps 3632.
Path 227 | total_timesteps 3645.
Path 228 | total_timesteps 3661.
Path 229 | total_timesteps 3673.
Path 230 | total_timesteps 3689.
Path 231 | total_timesteps 3704.
Path 232 | total_timesteps 3715.
Path 233 | total_timesteps 3734.
Path 234 | total_timesteps 3745.
Path 235 | total_timesteps 3793.
Path 236 | total_timesteps 3805.
Path 237 | total_timesteps 3818.
Path 238 | total_timesteps 3843.
Path 239 | total_timesteps 3861.
Path 240 | total_timesteps 3876.
Path 241 | total_timesteps 3897.
Path 242 | total_timesteps 3915.
Path 243 | total_timesteps 3928.
Path 244 | total_timesteps 3959.
Path 245 | total_timesteps 3969.
Path 246 | total_timesteps 3988.
Path 247 | total_timesteps 3998.
Path 248 | total_timesteps 4016.
Path 249 | total_timesteps 4033.
Path 250 | total_timesteps 4046.
Path 251 | total_timesteps 4056.
Path 252 | total_timesteps 4071.
Path 253 | total_timesteps 4084.
Path 254 | total_timesteps 4096.
Path 255 | total_timesteps 4110.
Path 256 | total_timesteps 4126.
Path 257 | total_timesteps 4137.
Path 258 | total_timesteps 4154.
Path 259 | total_timesteps 4164.
Path 260 | total_timesteps 4183.
Path 261 | total_timesteps 4196.
Path 262 | total_timesteps 4211.
Path 263 | total_timesteps 4224.
Path 264 | total_timesteps 4241.
Path 265 | total_timesteps 4256.
Path 266 | total_timesteps 4272.
Path 267 | total_timesteps 4285.
Path 268 | total_timesteps 4298.
Path 269 | total_timesteps 4310.
Path 270 | total_timesteps 4327.
Path 271 | total_timesteps 4345.
Path 272 | total_timesteps 4357.
Path 273 | total_timesteps 4385.
Path 274 | total_timesteps 4412.
Path 275 | total_timesteps 4446.
Path 276 | total_timesteps 4462.
Path 277 | total_timesteps 4472.
Path 278 | total_timesteps 4480.
Path 279 | total_timesteps 4489.
Path 280 | total_timesteps 4514.
Path 281 | total_timesteps 4522.
Path 282 | total_timesteps 4532.
Path 283 | total_timesteps 4546.
Path 284 | total_timesteps 4571.
Path 285 | total_timesteps 4583.
Path 286 | total_timesteps 4603.
Path 287 | total_timesteps 4615.
Path 288 | total_timesteps 4627.
Path 289 | total_timesteps 4639.
Path 290 | total_timesteps 4648.
Path 291 | total_timesteps 4664.
Path 292 | total_timesteps 4678.
Path 293 | total_timesteps 4690.
Path 294 | total_timesteps 4702.
Path 295 | total_timesteps 4726.
Path 296 | total_timesteps 4763.
Path 297 | total_timesteps 4785.
Path 298 | total_timesteps 4804.
Path 299 | total_timesteps 4817.
Path 300 | total_timesteps 4827.
Path 301 | total_timesteps 4842.
Path 302 | total_timesteps 4854.
Path 303 | total_timesteps 4870.
Path 304 | total_timesteps 4882.
Path 305 | total_timesteps 4898.
Path 306 | total_timesteps 4917.
Path 307 | total_timesteps 4929.
Path 308 | total_timesteps 4945.
Path 309 | total_timesteps 4964.
Path 310 | total_timesteps 4976.
Path 311 | total_timesteps 4994.
Path 312 | total_timesteps 5013.
Path 313 | total_timesteps 5035.
Path 314 | total_timesteps 5048.
Path 315 | total_timesteps 5062.
Path 316 | total_timesteps 5086.
Path 317 | total_timesteps 5099.
Path 318 | total_timesteps 5112.
Path 319 | total_timesteps 5120.
Path 320 | total_timesteps 5135.
Path 321 | total_timesteps 5147.
Path 322 | total_timesteps 5155.
Path 323 | total_timesteps 5166.
Path 324 | total_timesteps 5176.
Path 325 | total_timesteps 5198.
Path 326 | total_timesteps 5207.
Path 327 | total_timesteps 5220.
Path 328 | total_timesteps 5239.
Path 329 | total_timesteps 5251.
Path 330 | total_timesteps 5260.
Path 331 | total_timesteps 5271.
Path 332 | total_timesteps 5280.
Path 333 | total_timesteps 5294.
Path 334 | total_timesteps 5306.
Path 335 | total_timesteps 5314.
Path 336 | total_timesteps 5340.
Path 337 | total_timesteps 5359.
Path 338 | total_timesteps 5379.
Path 339 | total_timesteps 5391.
Path 340 | total_timesteps 5403.
Path 341 | total_timesteps 5431.
Path 342 | total_timesteps 5444.
Path 343 | total_timesteps 5467.
Path 344 | total_timesteps 5476.
Path 345 | total_timesteps 5492.
Path 346 | total_timesteps 5513.
Path 347 | total_timesteps 5531.
Path 348 | total_timesteps 5539.
Path 349 | total_timesteps 5562.
Path 350 | total_timesteps 5578.
Path 351 | total_timesteps 5600.
Path 352 | total_timesteps 5610.
Path 353 | total_timesteps 5624.
Path 354 | total_timesteps 5648.
Path 355 | total_timesteps 5662.
Path 356 | total_timesteps 5685.
Path 357 | total_timesteps 5714.
Path 358 | total_timesteps 5723.
Path 359 | total_timesteps 5736.
Path 360 | total_timesteps 5746.
Path 361 | total_timesteps 5756.
Path 362 | total_timesteps 5777.
Path 363 | total_timesteps 5790.
Path 364 | total_timesteps 5814.
Path 365 | total_timesteps 5822.
Path 366 | total_timesteps 5844.
Path 367 | total_timesteps 5863.
Path 368 | total_timesteps 5876.
Path 369 | total_timesteps 5886.
Path 370 | total_timesteps 5897.
Path 371 | total_timesteps 5905.
Path 372 | total_timesteps 5915.
Path 373 | total_timesteps 5925.
Path 374 | total_timesteps 5935.
Path 375 | total_timesteps 5948.
Path 376 | total_timesteps 5979.
Path 377 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -10.4    |
| Iteration     | 18       |
| MaximumReturn | 0.516    |
| MinimumReturn | -27.7    |
| TotalSamples  | 80067    |
----------------------------
itr #19 | 
Fitting dynamics.
Validation loss = 0.005999796092510223
Validation loss = 0.00567981880158186
Validation loss = 0.005818773992359638
Validation loss = 0.00559214735403657
Validation loss = 0.00579778291285038
Validation loss = 0.005789879709482193
Validation loss = 0.005626322235912085
Validation loss = 0.005330747924745083
Validation loss = 0.005588921718299389
Validation loss = 0.005465126130729914
Validation loss = 0.005859869997948408
Validation loss = 0.0060081472620368
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 55.
Path 4 | total_timesteps 66.
Path 5 | total_timesteps 78.
Path 6 | total_timesteps 86.
Path 7 | total_timesteps 95.
Path 8 | total_timesteps 104.
Path 9 | total_timesteps 111.
Path 10 | total_timesteps 121.
Path 11 | total_timesteps 131.
Path 12 | total_timesteps 140.
Path 13 | total_timesteps 150.
Path 14 | total_timesteps 160.
Path 15 | total_timesteps 174.
Path 16 | total_timesteps 181.
Path 17 | total_timesteps 191.
Path 18 | total_timesteps 200.
Path 19 | total_timesteps 210.
Path 20 | total_timesteps 219.
Path 21 | total_timesteps 231.
Path 22 | total_timesteps 243.
Path 23 | total_timesteps 251.
Path 24 | total_timesteps 258.
Path 25 | total_timesteps 265.
Path 26 | total_timesteps 283.
Path 27 | total_timesteps 292.
Path 28 | total_timesteps 299.
Path 29 | total_timesteps 311.
Path 30 | total_timesteps 319.
Path 31 | total_timesteps 332.
Path 32 | total_timesteps 341.
Path 33 | total_timesteps 353.
Path 34 | total_timesteps 381.
Path 35 | total_timesteps 389.
Path 36 | total_timesteps 403.
Path 37 | total_timesteps 417.
Path 38 | total_timesteps 430.
Path 39 | total_timesteps 438.
Path 40 | total_timesteps 449.
Path 41 | total_timesteps 462.
Path 42 | total_timesteps 474.
Path 43 | total_timesteps 487.
Path 44 | total_timesteps 497.
Path 45 | total_timesteps 506.
Path 46 | total_timesteps 524.
Path 47 | total_timesteps 537.
Path 48 | total_timesteps 545.
Path 49 | total_timesteps 554.
Path 50 | total_timesteps 567.
Path 51 | total_timesteps 578.
Path 52 | total_timesteps 589.
Path 53 | total_timesteps 596.
Path 54 | total_timesteps 605.
Path 55 | total_timesteps 612.
Path 56 | total_timesteps 621.
Path 57 | total_timesteps 628.
Path 58 | total_timesteps 638.
Path 59 | total_timesteps 650.
Path 60 | total_timesteps 656.
Path 61 | total_timesteps 666.
Path 62 | total_timesteps 686.
Path 63 | total_timesteps 702.
Path 64 | total_timesteps 709.
Path 65 | total_timesteps 718.
Path 66 | total_timesteps 726.
Path 67 | total_timesteps 745.
Path 68 | total_timesteps 758.
Path 69 | total_timesteps 768.
Path 70 | total_timesteps 778.
Path 71 | total_timesteps 788.
Path 72 | total_timesteps 796.
Path 73 | total_timesteps 807.
Path 74 | total_timesteps 816.
Path 75 | total_timesteps 825.
Path 76 | total_timesteps 835.
Path 77 | total_timesteps 846.
Path 78 | total_timesteps 854.
Path 79 | total_timesteps 862.
Path 80 | total_timesteps 875.
Path 81 | total_timesteps 883.
Path 82 | total_timesteps 897.
Path 83 | total_timesteps 906.
Path 84 | total_timesteps 912.
Path 85 | total_timesteps 921.
Path 86 | total_timesteps 935.
Path 87 | total_timesteps 956.
Path 88 | total_timesteps 964.
Path 89 | total_timesteps 972.
Path 90 | total_timesteps 981.
Path 91 | total_timesteps 996.
Path 92 | total_timesteps 1005.
Path 93 | total_timesteps 1021.
Path 94 | total_timesteps 1035.
Path 95 | total_timesteps 1049.
Path 96 | total_timesteps 1059.
Path 97 | total_timesteps 1067.
Path 98 | total_timesteps 1075.
Path 99 | total_timesteps 1092.
Path 100 | total_timesteps 1100.
Path 101 | total_timesteps 1113.
Path 102 | total_timesteps 1125.
Path 103 | total_timesteps 1133.
Path 104 | total_timesteps 1145.
Path 105 | total_timesteps 1160.
Path 106 | total_timesteps 1167.
Path 107 | total_timesteps 1180.
Path 108 | total_timesteps 1191.
Path 109 | total_timesteps 1201.
Path 110 | total_timesteps 1210.
Path 111 | total_timesteps 1217.
Path 112 | total_timesteps 1224.
Path 113 | total_timesteps 1232.
Path 114 | total_timesteps 1245.
Path 115 | total_timesteps 1253.
Path 116 | total_timesteps 1261.
Path 117 | total_timesteps 1270.
Path 118 | total_timesteps 1277.
Path 119 | total_timesteps 1286.
Path 120 | total_timesteps 1294.
Path 121 | total_timesteps 1306.
Path 122 | total_timesteps 1321.
Path 123 | total_timesteps 1331.
Path 124 | total_timesteps 1338.
Path 125 | total_timesteps 1349.
Path 126 | total_timesteps 1357.
Path 127 | total_timesteps 1363.
Path 128 | total_timesteps 1387.
Path 129 | total_timesteps 1395.
Path 130 | total_timesteps 1404.
Path 131 | total_timesteps 1414.
Path 132 | total_timesteps 1423.
Path 133 | total_timesteps 1433.
Path 134 | total_timesteps 1452.
Path 135 | total_timesteps 1463.
Path 136 | total_timesteps 1485.
Path 137 | total_timesteps 1496.
Path 138 | total_timesteps 1512.
Path 139 | total_timesteps 1521.
Path 140 | total_timesteps 1528.
Path 141 | total_timesteps 1542.
Path 142 | total_timesteps 1553.
Path 143 | total_timesteps 1563.
Path 144 | total_timesteps 1576.
Path 145 | total_timesteps 1587.
Path 146 | total_timesteps 1595.
Path 147 | total_timesteps 1607.
Path 148 | total_timesteps 1624.
Path 149 | total_timesteps 1636.
Path 150 | total_timesteps 1646.
Path 151 | total_timesteps 1660.
Path 152 | total_timesteps 1673.
Path 153 | total_timesteps 1684.
Path 154 | total_timesteps 1693.
Path 155 | total_timesteps 1705.
Path 156 | total_timesteps 1716.
Path 157 | total_timesteps 1727.
Path 158 | total_timesteps 1737.
Path 159 | total_timesteps 1745.
Path 160 | total_timesteps 1757.
Path 161 | total_timesteps 1765.
Path 162 | total_timesteps 1775.
Path 163 | total_timesteps 1785.
Path 164 | total_timesteps 1793.
Path 165 | total_timesteps 1800.
Path 166 | total_timesteps 1808.
Path 167 | total_timesteps 1818.
Path 168 | total_timesteps 1826.
Path 169 | total_timesteps 1834.
Path 170 | total_timesteps 1847.
Path 171 | total_timesteps 1859.
Path 172 | total_timesteps 1877.
Path 173 | total_timesteps 1896.
Path 174 | total_timesteps 1908.
Path 175 | total_timesteps 1926.
Path 176 | total_timesteps 1937.
Path 177 | total_timesteps 1944.
Path 178 | total_timesteps 1954.
Path 179 | total_timesteps 1970.
Path 180 | total_timesteps 1980.
Path 181 | total_timesteps 1994.
Path 182 | total_timesteps 2001.
Path 183 | total_timesteps 2010.
Path 184 | total_timesteps 2022.
Path 185 | total_timesteps 2030.
Path 186 | total_timesteps 2038.
Path 187 | total_timesteps 2047.
Path 188 | total_timesteps 2054.
Path 189 | total_timesteps 2062.
Path 190 | total_timesteps 2072.
Path 191 | total_timesteps 2081.
Path 192 | total_timesteps 2090.
Path 193 | total_timesteps 2112.
Path 194 | total_timesteps 2134.
Path 195 | total_timesteps 2150.
Path 196 | total_timesteps 2159.
Path 197 | total_timesteps 2169.
Path 198 | total_timesteps 2191.
Path 199 | total_timesteps 2210.
Path 200 | total_timesteps 2223.
Path 201 | total_timesteps 2236.
Path 202 | total_timesteps 2251.
Path 203 | total_timesteps 2260.
Path 204 | total_timesteps 2273.
Path 205 | total_timesteps 2285.
Path 206 | total_timesteps 2298.
Path 207 | total_timesteps 2310.
Path 208 | total_timesteps 2320.
Path 209 | total_timesteps 2336.
Path 210 | total_timesteps 2351.
Path 211 | total_timesteps 2365.
Path 212 | total_timesteps 2372.
Path 213 | total_timesteps 2385.
Path 214 | total_timesteps 2398.
Path 215 | total_timesteps 2410.
Path 216 | total_timesteps 2423.
Path 217 | total_timesteps 2435.
Path 218 | total_timesteps 2447.
Path 219 | total_timesteps 2460.
Path 220 | total_timesteps 2473.
Path 221 | total_timesteps 2488.
Path 222 | total_timesteps 2497.
Path 223 | total_timesteps 2508.
Path 224 | total_timesteps 2522.
Path 225 | total_timesteps 2533.
Path 226 | total_timesteps 2555.
Path 227 | total_timesteps 2565.
Path 228 | total_timesteps 2575.
Path 229 | total_timesteps 2587.
Path 230 | total_timesteps 2599.
Path 231 | total_timesteps 2612.
Path 232 | total_timesteps 2624.
Path 233 | total_timesteps 2632.
Path 234 | total_timesteps 2639.
Path 235 | total_timesteps 2660.
Path 236 | total_timesteps 2675.
Path 237 | total_timesteps 2691.
Path 238 | total_timesteps 2698.
Path 239 | total_timesteps 2710.
Path 240 | total_timesteps 2720.
Path 241 | total_timesteps 2729.
Path 242 | total_timesteps 2740.
Path 243 | total_timesteps 2750.
Path 244 | total_timesteps 2759.
Path 245 | total_timesteps 2771.
Path 246 | total_timesteps 2782.
Path 247 | total_timesteps 2789.
Path 248 | total_timesteps 2796.
Path 249 | total_timesteps 2804.
Path 250 | total_timesteps 2813.
Path 251 | total_timesteps 2822.
Path 252 | total_timesteps 2833.
Path 253 | total_timesteps 2849.
Path 254 | total_timesteps 2856.
Path 255 | total_timesteps 2874.
Path 256 | total_timesteps 2882.
Path 257 | total_timesteps 2890.
Path 258 | total_timesteps 2900.
Path 259 | total_timesteps 2906.
Path 260 | total_timesteps 2918.
Path 261 | total_timesteps 2926.
Path 262 | total_timesteps 2940.
Path 263 | total_timesteps 2947.
Path 264 | total_timesteps 2954.
Path 265 | total_timesteps 2966.
Path 266 | total_timesteps 2973.
Path 267 | total_timesteps 2981.
Path 268 | total_timesteps 2991.
Path 269 | total_timesteps 3002.
Path 270 | total_timesteps 3015.
Path 271 | total_timesteps 3023.
Path 272 | total_timesteps 3034.
Path 273 | total_timesteps 3042.
Path 274 | total_timesteps 3054.
Path 275 | total_timesteps 3062.
Path 276 | total_timesteps 3072.
Path 277 | total_timesteps 3083.
Path 278 | total_timesteps 3094.
Path 279 | total_timesteps 3103.
Path 280 | total_timesteps 3115.
Path 281 | total_timesteps 3126.
Path 282 | total_timesteps 3136.
Path 283 | total_timesteps 3144.
Path 284 | total_timesteps 3154.
Path 285 | total_timesteps 3163.
Path 286 | total_timesteps 3174.
Path 287 | total_timesteps 3183.
Path 288 | total_timesteps 3194.
Path 289 | total_timesteps 3204.
Path 290 | total_timesteps 3218.
Path 291 | total_timesteps 3227.
Path 292 | total_timesteps 3238.
Path 293 | total_timesteps 3254.
Path 294 | total_timesteps 3263.
Path 295 | total_timesteps 3277.
Path 296 | total_timesteps 3287.
Path 297 | total_timesteps 3300.
Path 298 | total_timesteps 3309.
Path 299 | total_timesteps 3323.
Path 300 | total_timesteps 3331.
Path 301 | total_timesteps 3338.
Path 302 | total_timesteps 3344.
Path 303 | total_timesteps 3357.
Path 304 | total_timesteps 3368.
Path 305 | total_timesteps 3375.
Path 306 | total_timesteps 3382.
Path 307 | total_timesteps 3396.
Path 308 | total_timesteps 3408.
Path 309 | total_timesteps 3420.
Path 310 | total_timesteps 3436.
Path 311 | total_timesteps 3454.
Path 312 | total_timesteps 3465.
Path 313 | total_timesteps 3473.
Path 314 | total_timesteps 3497.
Path 315 | total_timesteps 3507.
Path 316 | total_timesteps 3521.
Path 317 | total_timesteps 3534.
Path 318 | total_timesteps 3541.
Path 319 | total_timesteps 3551.
Path 320 | total_timesteps 3561.
Path 321 | total_timesteps 3572.
Path 322 | total_timesteps 3580.
Path 323 | total_timesteps 3590.
Path 324 | total_timesteps 3602.
Path 325 | total_timesteps 3611.
Path 326 | total_timesteps 3620.
Path 327 | total_timesteps 3631.
Path 328 | total_timesteps 3640.
Path 329 | total_timesteps 3654.
Path 330 | total_timesteps 3668.
Path 331 | total_timesteps 3680.
Path 332 | total_timesteps 3690.
Path 333 | total_timesteps 3698.
Path 334 | total_timesteps 3706.
Path 335 | total_timesteps 3714.
Path 336 | total_timesteps 3723.
Path 337 | total_timesteps 3730.
Path 338 | total_timesteps 3738.
Path 339 | total_timesteps 3745.
Path 340 | total_timesteps 3756.
Path 341 | total_timesteps 3771.
Path 342 | total_timesteps 3788.
Path 343 | total_timesteps 3799.
Path 344 | total_timesteps 3816.
Path 345 | total_timesteps 3824.
Path 346 | total_timesteps 3832.
Path 347 | total_timesteps 3841.
Path 348 | total_timesteps 3851.
Path 349 | total_timesteps 3864.
Path 350 | total_timesteps 3875.
Path 351 | total_timesteps 3888.
Path 352 | total_timesteps 3896.
Path 353 | total_timesteps 3905.
Path 354 | total_timesteps 3913.
Path 355 | total_timesteps 3923.
Path 356 | total_timesteps 3933.
Path 357 | total_timesteps 3943.
Path 358 | total_timesteps 3952.
Path 359 | total_timesteps 3964.
Path 360 | total_timesteps 3975.
Path 361 | total_timesteps 3983.
Path 362 | total_timesteps 3997.
Path 363 | total_timesteps 4011.
Path 364 | total_timesteps 4020.
Path 365 | total_timesteps 4038.
Path 366 | total_timesteps 4045.
Path 367 | total_timesteps 4053.
Path 368 | total_timesteps 4068.
Path 369 | total_timesteps 4084.
Path 370 | total_timesteps 4095.
Path 371 | total_timesteps 4105.
Path 372 | total_timesteps 4114.
Path 373 | total_timesteps 4124.
Path 374 | total_timesteps 4131.
Path 375 | total_timesteps 4145.
Path 376 | total_timesteps 4153.
Path 377 | total_timesteps 4170.
Path 378 | total_timesteps 4179.
Path 379 | total_timesteps 4188.
Path 380 | total_timesteps 4196.
Path 381 | total_timesteps 4214.
Path 382 | total_timesteps 4224.
Path 383 | total_timesteps 4236.
Path 384 | total_timesteps 4243.
Path 385 | total_timesteps 4250.
Path 386 | total_timesteps 4258.
Path 387 | total_timesteps 4266.
Path 388 | total_timesteps 4275.
Path 389 | total_timesteps 4283.
Path 390 | total_timesteps 4296.
Path 391 | total_timesteps 4304.
Path 392 | total_timesteps 4323.
Path 393 | total_timesteps 4331.
Path 394 | total_timesteps 4340.
Path 395 | total_timesteps 4348.
Path 396 | total_timesteps 4359.
Path 397 | total_timesteps 4368.
Path 398 | total_timesteps 4375.
Path 399 | total_timesteps 4382.
Path 400 | total_timesteps 4392.
Path 401 | total_timesteps 4407.
Path 402 | total_timesteps 4417.
Path 403 | total_timesteps 4425.
Path 404 | total_timesteps 4434.
Path 405 | total_timesteps 4447.
Path 406 | total_timesteps 4459.
Path 407 | total_timesteps 4470.
Path 408 | total_timesteps 4479.
Path 409 | total_timesteps 4497.
Path 410 | total_timesteps 4505.
Path 411 | total_timesteps 4524.
Path 412 | total_timesteps 4531.
Path 413 | total_timesteps 4547.
Path 414 | total_timesteps 4556.
Path 415 | total_timesteps 4576.
Path 416 | total_timesteps 4585.
Path 417 | total_timesteps 4597.
Path 418 | total_timesteps 4611.
Path 419 | total_timesteps 4619.
Path 420 | total_timesteps 4639.
Path 421 | total_timesteps 4651.
Path 422 | total_timesteps 4666.
Path 423 | total_timesteps 4674.
Path 424 | total_timesteps 4704.
Path 425 | total_timesteps 4712.
Path 426 | total_timesteps 4720.
Path 427 | total_timesteps 4730.
Path 428 | total_timesteps 4739.
Path 429 | total_timesteps 4748.
Path 430 | total_timesteps 4775.
Path 431 | total_timesteps 4791.
Path 432 | total_timesteps 4799.
Path 433 | total_timesteps 4807.
Path 434 | total_timesteps 4821.
Path 435 | total_timesteps 4837.
Path 436 | total_timesteps 4851.
Path 437 | total_timesteps 4862.
Path 438 | total_timesteps 4873.
Path 439 | total_timesteps 4881.
Path 440 | total_timesteps 4888.
Path 441 | total_timesteps 4900.
Path 442 | total_timesteps 4910.
Path 443 | total_timesteps 4920.
Path 444 | total_timesteps 4931.
Path 445 | total_timesteps 4942.
Path 446 | total_timesteps 4956.
Path 447 | total_timesteps 4965.
Path 448 | total_timesteps 4973.
Path 449 | total_timesteps 4982.
Path 450 | total_timesteps 4996.
Path 451 | total_timesteps 5013.
Path 452 | total_timesteps 5028.
Path 453 | total_timesteps 5039.
Path 454 | total_timesteps 5048.
Path 455 | total_timesteps 5065.
Path 456 | total_timesteps 5074.
Path 457 | total_timesteps 5099.
Path 458 | total_timesteps 5108.
Path 459 | total_timesteps 5115.
Path 460 | total_timesteps 5127.
Path 461 | total_timesteps 5144.
Path 462 | total_timesteps 5158.
Path 463 | total_timesteps 5168.
Path 464 | total_timesteps 5180.
Path 465 | total_timesteps 5198.
Path 466 | total_timesteps 5209.
Path 467 | total_timesteps 5223.
Path 468 | total_timesteps 5236.
Path 469 | total_timesteps 5246.
Path 470 | total_timesteps 5258.
Path 471 | total_timesteps 5268.
Path 472 | total_timesteps 5277.
Path 473 | total_timesteps 5286.
Path 474 | total_timesteps 5298.
Path 475 | total_timesteps 5309.
Path 476 | total_timesteps 5324.
Path 477 | total_timesteps 5330.
Path 478 | total_timesteps 5341.
Path 479 | total_timesteps 5359.
Path 480 | total_timesteps 5366.
Path 481 | total_timesteps 5378.
Path 482 | total_timesteps 5387.
Path 483 | total_timesteps 5396.
Path 484 | total_timesteps 5405.
Path 485 | total_timesteps 5417.
Path 486 | total_timesteps 5430.
Path 487 | total_timesteps 5444.
Path 488 | total_timesteps 5451.
Path 489 | total_timesteps 5458.
Path 490 | total_timesteps 5480.
Path 491 | total_timesteps 5488.
Path 492 | total_timesteps 5499.
Path 493 | total_timesteps 5507.
Path 494 | total_timesteps 5516.
Path 495 | total_timesteps 5523.
Path 496 | total_timesteps 5532.
Path 497 | total_timesteps 5542.
Path 498 | total_timesteps 5552.
Path 499 | total_timesteps 5564.
Path 500 | total_timesteps 5572.
Path 501 | total_timesteps 5582.
Path 502 | total_timesteps 5590.
Path 503 | total_timesteps 5600.
Path 504 | total_timesteps 5610.
Path 505 | total_timesteps 5621.
Path 506 | total_timesteps 5629.
Path 507 | total_timesteps 5642.
Path 508 | total_timesteps 5651.
Path 509 | total_timesteps 5660.
Path 510 | total_timesteps 5673.
Path 511 | total_timesteps 5681.
Path 512 | total_timesteps 5690.
Path 513 | total_timesteps 5700.
Path 514 | total_timesteps 5711.
Path 515 | total_timesteps 5727.
Path 516 | total_timesteps 5744.
Path 517 | total_timesteps 5753.
Path 518 | total_timesteps 5759.
Path 519 | total_timesteps 5771.
Path 520 | total_timesteps 5781.
Path 521 | total_timesteps 5789.
Path 522 | total_timesteps 5798.
Path 523 | total_timesteps 5805.
Path 524 | total_timesteps 5819.
Path 525 | total_timesteps 5838.
Path 526 | total_timesteps 5849.
Path 527 | total_timesteps 5860.
Path 528 | total_timesteps 5866.
Path 529 | total_timesteps 5886.
Path 530 | total_timesteps 5898.
Path 531 | total_timesteps 5909.
Path 532 | total_timesteps 5928.
Path 533 | total_timesteps 5936.
Path 534 | total_timesteps 5946.
Path 535 | total_timesteps 5954.
Path 536 | total_timesteps 5965.
Path 537 | total_timesteps 5975.
Path 538 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.07    |
| Iteration     | 19       |
| MaximumReturn | 1.72     |
| MinimumReturn | -21.7    |
| TotalSamples  | 84069    |
----------------------------
itr #20 | 
Fitting dynamics.
Validation loss = 0.006077378988265991
Validation loss = 0.006017601117491722
Validation loss = 0.00568957906216383
Validation loss = 0.005386087112128735
Validation loss = 0.005422232672572136
Validation loss = 0.005601059179753065
Validation loss = 0.00560852000489831
Validation loss = 0.005814650095999241
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 35.
Path 3 | total_timesteps 47.
Path 4 | total_timesteps 64.
Path 5 | total_timesteps 80.
Path 6 | total_timesteps 87.
Path 7 | total_timesteps 96.
Path 8 | total_timesteps 115.
Path 9 | total_timesteps 125.
Path 10 | total_timesteps 141.
Path 11 | total_timesteps 162.
Path 12 | total_timesteps 175.
Path 13 | total_timesteps 188.
Path 14 | total_timesteps 203.
Path 15 | total_timesteps 220.
Path 16 | total_timesteps 241.
Path 17 | total_timesteps 249.
Path 18 | total_timesteps 256.
Path 19 | total_timesteps 266.
Path 20 | total_timesteps 275.
Path 21 | total_timesteps 288.
Path 22 | total_timesteps 299.
Path 23 | total_timesteps 309.
Path 24 | total_timesteps 316.
Path 25 | total_timesteps 327.
Path 26 | total_timesteps 340.
Path 27 | total_timesteps 361.
Path 28 | total_timesteps 374.
Path 29 | total_timesteps 382.
Path 30 | total_timesteps 394.
Path 31 | total_timesteps 416.
Path 32 | total_timesteps 425.
Path 33 | total_timesteps 432.
Path 34 | total_timesteps 439.
Path 35 | total_timesteps 445.
Path 36 | total_timesteps 454.
Path 37 | total_timesteps 467.
Path 38 | total_timesteps 477.
Path 39 | total_timesteps 486.
Path 40 | total_timesteps 496.
Path 41 | total_timesteps 506.
Path 42 | total_timesteps 514.
Path 43 | total_timesteps 525.
Path 44 | total_timesteps 537.
Path 45 | total_timesteps 548.
Path 46 | total_timesteps 559.
Path 47 | total_timesteps 570.
Path 48 | total_timesteps 581.
Path 49 | total_timesteps 589.
Path 50 | total_timesteps 596.
Path 51 | total_timesteps 604.
Path 52 | total_timesteps 618.
Path 53 | total_timesteps 629.
Path 54 | total_timesteps 652.
Path 55 | total_timesteps 659.
Path 56 | total_timesteps 677.
Path 57 | total_timesteps 686.
Path 58 | total_timesteps 702.
Path 59 | total_timesteps 713.
Path 60 | total_timesteps 723.
Path 61 | total_timesteps 740.
Path 62 | total_timesteps 747.
Path 63 | total_timesteps 762.
Path 64 | total_timesteps 769.
Path 65 | total_timesteps 781.
Path 66 | total_timesteps 792.
Path 67 | total_timesteps 804.
Path 68 | total_timesteps 817.
Path 69 | total_timesteps 835.
Path 70 | total_timesteps 844.
Path 71 | total_timesteps 850.
Path 72 | total_timesteps 858.
Path 73 | total_timesteps 871.
Path 74 | total_timesteps 882.
Path 75 | total_timesteps 896.
Path 76 | total_timesteps 906.
Path 77 | total_timesteps 916.
Path 78 | total_timesteps 928.
Path 79 | total_timesteps 941.
Path 80 | total_timesteps 949.
Path 81 | total_timesteps 958.
Path 82 | total_timesteps 970.
Path 83 | total_timesteps 977.
Path 84 | total_timesteps 989.
Path 85 | total_timesteps 1000.
Path 86 | total_timesteps 1011.
Path 87 | total_timesteps 1019.
Path 88 | total_timesteps 1029.
Path 89 | total_timesteps 1037.
Path 90 | total_timesteps 1046.
Path 91 | total_timesteps 1055.
Path 92 | total_timesteps 1068.
Path 93 | total_timesteps 1078.
Path 94 | total_timesteps 1091.
Path 95 | total_timesteps 1099.
Path 96 | total_timesteps 1111.
Path 97 | total_timesteps 1117.
Path 98 | total_timesteps 1127.
Path 99 | total_timesteps 1140.
Path 100 | total_timesteps 1147.
Path 101 | total_timesteps 1155.
Path 102 | total_timesteps 1164.
Path 103 | total_timesteps 1191.
Path 104 | total_timesteps 1201.
Path 105 | total_timesteps 1211.
Path 106 | total_timesteps 1220.
Path 107 | total_timesteps 1227.
Path 108 | total_timesteps 1239.
Path 109 | total_timesteps 1247.
Path 110 | total_timesteps 1261.
Path 111 | total_timesteps 1274.
Path 112 | total_timesteps 1285.
Path 113 | total_timesteps 1296.
Path 114 | total_timesteps 1309.
Path 115 | total_timesteps 1317.
Path 116 | total_timesteps 1328.
Path 117 | total_timesteps 1346.
Path 118 | total_timesteps 1354.
Path 119 | total_timesteps 1363.
Path 120 | total_timesteps 1374.
Path 121 | total_timesteps 1385.
Path 122 | total_timesteps 1407.
Path 123 | total_timesteps 1414.
Path 124 | total_timesteps 1429.
Path 125 | total_timesteps 1451.
Path 126 | total_timesteps 1462.
Path 127 | total_timesteps 1470.
Path 128 | total_timesteps 1479.
Path 129 | total_timesteps 1496.
Path 130 | total_timesteps 1510.
Path 131 | total_timesteps 1518.
Path 132 | total_timesteps 1532.
Path 133 | total_timesteps 1544.
Path 134 | total_timesteps 1554.
Path 135 | total_timesteps 1563.
Path 136 | total_timesteps 1572.
Path 137 | total_timesteps 1584.
Path 138 | total_timesteps 1592.
Path 139 | total_timesteps 1603.
Path 140 | total_timesteps 1611.
Path 141 | total_timesteps 1626.
Path 142 | total_timesteps 1637.
Path 143 | total_timesteps 1647.
Path 144 | total_timesteps 1658.
Path 145 | total_timesteps 1672.
Path 146 | total_timesteps 1679.
Path 147 | total_timesteps 1689.
Path 148 | total_timesteps 1698.
Path 149 | total_timesteps 1712.
Path 150 | total_timesteps 1719.
Path 151 | total_timesteps 1729.
Path 152 | total_timesteps 1745.
Path 153 | total_timesteps 1754.
Path 154 | total_timesteps 1764.
Path 155 | total_timesteps 1775.
Path 156 | total_timesteps 1786.
Path 157 | total_timesteps 1796.
Path 158 | total_timesteps 1824.
Path 159 | total_timesteps 1832.
Path 160 | total_timesteps 1843.
Path 161 | total_timesteps 1850.
Path 162 | total_timesteps 1856.
Path 163 | total_timesteps 1865.
Path 164 | total_timesteps 1875.
Path 165 | total_timesteps 1884.
Path 166 | total_timesteps 1893.
Path 167 | total_timesteps 1902.
Path 168 | total_timesteps 1917.
Path 169 | total_timesteps 1924.
Path 170 | total_timesteps 1932.
Path 171 | total_timesteps 1947.
Path 172 | total_timesteps 1960.
Path 173 | total_timesteps 1971.
Path 174 | total_timesteps 1981.
Path 175 | total_timesteps 1995.
Path 176 | total_timesteps 2002.
Path 177 | total_timesteps 2010.
Path 178 | total_timesteps 2024.
Path 179 | total_timesteps 2039.
Path 180 | total_timesteps 2047.
Path 181 | total_timesteps 2055.
Path 182 | total_timesteps 2066.
Path 183 | total_timesteps 2075.
Path 184 | total_timesteps 2086.
Path 185 | total_timesteps 2095.
Path 186 | total_timesteps 2105.
Path 187 | total_timesteps 2120.
Path 188 | total_timesteps 2138.
Path 189 | total_timesteps 2163.
Path 190 | total_timesteps 2176.
Path 191 | total_timesteps 2193.
Path 192 | total_timesteps 2206.
Path 193 | total_timesteps 2219.
Path 194 | total_timesteps 2228.
Path 195 | total_timesteps 2235.
Path 196 | total_timesteps 2249.
Path 197 | total_timesteps 2261.
Path 198 | total_timesteps 2271.
Path 199 | total_timesteps 2279.
Path 200 | total_timesteps 2291.
Path 201 | total_timesteps 2301.
Path 202 | total_timesteps 2311.
Path 203 | total_timesteps 2319.
Path 204 | total_timesteps 2332.
Path 205 | total_timesteps 2341.
Path 206 | total_timesteps 2360.
Path 207 | total_timesteps 2369.
Path 208 | total_timesteps 2381.
Path 209 | total_timesteps 2399.
Path 210 | total_timesteps 2418.
Path 211 | total_timesteps 2425.
Path 212 | total_timesteps 2431.
Path 213 | total_timesteps 2445.
Path 214 | total_timesteps 2453.
Path 215 | total_timesteps 2468.
Path 216 | total_timesteps 2479.
Path 217 | total_timesteps 2487.
Path 218 | total_timesteps 2497.
Path 219 | total_timesteps 2506.
Path 220 | total_timesteps 2516.
Path 221 | total_timesteps 2527.
Path 222 | total_timesteps 2541.
Path 223 | total_timesteps 2550.
Path 224 | total_timesteps 2558.
Path 225 | total_timesteps 2568.
Path 226 | total_timesteps 2577.
Path 227 | total_timesteps 2589.
Path 228 | total_timesteps 2596.
Path 229 | total_timesteps 2608.
Path 230 | total_timesteps 2618.
Path 231 | total_timesteps 2625.
Path 232 | total_timesteps 2648.
Path 233 | total_timesteps 2670.
Path 234 | total_timesteps 2681.
Path 235 | total_timesteps 2690.
Path 236 | total_timesteps 2708.
Path 237 | total_timesteps 2719.
Path 238 | total_timesteps 2733.
Path 239 | total_timesteps 2745.
Path 240 | total_timesteps 2755.
Path 241 | total_timesteps 2782.
Path 242 | total_timesteps 2793.
Path 243 | total_timesteps 2806.
Path 244 | total_timesteps 2817.
Path 245 | total_timesteps 2831.
Path 246 | total_timesteps 2846.
Path 247 | total_timesteps 2858.
Path 248 | total_timesteps 2872.
Path 249 | total_timesteps 2882.
Path 250 | total_timesteps 2890.
Path 251 | total_timesteps 2898.
Path 252 | total_timesteps 2912.
Path 253 | total_timesteps 2925.
Path 254 | total_timesteps 2933.
Path 255 | total_timesteps 2943.
Path 256 | total_timesteps 2950.
Path 257 | total_timesteps 2959.
Path 258 | total_timesteps 2983.
Path 259 | total_timesteps 2996.
Path 260 | total_timesteps 3008.
Path 261 | total_timesteps 3019.
Path 262 | total_timesteps 3031.
Path 263 | total_timesteps 3050.
Path 264 | total_timesteps 3061.
Path 265 | total_timesteps 3071.
Path 266 | total_timesteps 3084.
Path 267 | total_timesteps 3094.
Path 268 | total_timesteps 3106.
Path 269 | total_timesteps 3118.
Path 270 | total_timesteps 3131.
Path 271 | total_timesteps 3139.
Path 272 | total_timesteps 3146.
Path 273 | total_timesteps 3153.
Path 274 | total_timesteps 3161.
Path 275 | total_timesteps 3173.
Path 276 | total_timesteps 3182.
Path 277 | total_timesteps 3189.
Path 278 | total_timesteps 3198.
Path 279 | total_timesteps 3209.
Path 280 | total_timesteps 3230.
Path 281 | total_timesteps 3240.
Path 282 | total_timesteps 3248.
Path 283 | total_timesteps 3257.
Path 284 | total_timesteps 3267.
Path 285 | total_timesteps 3279.
Path 286 | total_timesteps 3291.
Path 287 | total_timesteps 3302.
Path 288 | total_timesteps 3312.
Path 289 | total_timesteps 3322.
Path 290 | total_timesteps 3328.
Path 291 | total_timesteps 3342.
Path 292 | total_timesteps 3353.
Path 293 | total_timesteps 3361.
Path 294 | total_timesteps 3370.
Path 295 | total_timesteps 3381.
Path 296 | total_timesteps 3395.
Path 297 | total_timesteps 3404.
Path 298 | total_timesteps 3414.
Path 299 | total_timesteps 3422.
Path 300 | total_timesteps 3429.
Path 301 | total_timesteps 3436.
Path 302 | total_timesteps 3443.
Path 303 | total_timesteps 3450.
Path 304 | total_timesteps 3457.
Path 305 | total_timesteps 3475.
Path 306 | total_timesteps 3502.
Path 307 | total_timesteps 3511.
Path 308 | total_timesteps 3522.
Path 309 | total_timesteps 3532.
Path 310 | total_timesteps 3544.
Path 311 | total_timesteps 3552.
Path 312 | total_timesteps 3567.
Path 313 | total_timesteps 3576.
Path 314 | total_timesteps 3588.
Path 315 | total_timesteps 3597.
Path 316 | total_timesteps 3604.
Path 317 | total_timesteps 3619.
Path 318 | total_timesteps 3629.
Path 319 | total_timesteps 3646.
Path 320 | total_timesteps 3654.
Path 321 | total_timesteps 3664.
Path 322 | total_timesteps 3674.
Path 323 | total_timesteps 3682.
Path 324 | total_timesteps 3696.
Path 325 | total_timesteps 3703.
Path 326 | total_timesteps 3715.
Path 327 | total_timesteps 3724.
Path 328 | total_timesteps 3733.
Path 329 | total_timesteps 3740.
Path 330 | total_timesteps 3748.
Path 331 | total_timesteps 3758.
Path 332 | total_timesteps 3775.
Path 333 | total_timesteps 3786.
Path 334 | total_timesteps 3803.
Path 335 | total_timesteps 3814.
Path 336 | total_timesteps 3823.
Path 337 | total_timesteps 3833.
Path 338 | total_timesteps 3843.
Path 339 | total_timesteps 3858.
Path 340 | total_timesteps 3866.
Path 341 | total_timesteps 3875.
Path 342 | total_timesteps 3884.
Path 343 | total_timesteps 3895.
Path 344 | total_timesteps 3907.
Path 345 | total_timesteps 3917.
Path 346 | total_timesteps 3928.
Path 347 | total_timesteps 3948.
Path 348 | total_timesteps 3956.
Path 349 | total_timesteps 3964.
Path 350 | total_timesteps 3973.
Path 351 | total_timesteps 3983.
Path 352 | total_timesteps 4024.
Path 353 | total_timesteps 4032.
Path 354 | total_timesteps 4049.
Path 355 | total_timesteps 4058.
Path 356 | total_timesteps 4068.
Path 357 | total_timesteps 4088.
Path 358 | total_timesteps 4095.
Path 359 | total_timesteps 4103.
Path 360 | total_timesteps 4110.
Path 361 | total_timesteps 4127.
Path 362 | total_timesteps 4140.
Path 363 | total_timesteps 4150.
Path 364 | total_timesteps 4159.
Path 365 | total_timesteps 4168.
Path 366 | total_timesteps 4175.
Path 367 | total_timesteps 4186.
Path 368 | total_timesteps 4196.
Path 369 | total_timesteps 4207.
Path 370 | total_timesteps 4215.
Path 371 | total_timesteps 4230.
Path 372 | total_timesteps 4242.
Path 373 | total_timesteps 4252.
Path 374 | total_timesteps 4261.
Path 375 | total_timesteps 4273.
Path 376 | total_timesteps 4286.
Path 377 | total_timesteps 4302.
Path 378 | total_timesteps 4314.
Path 379 | total_timesteps 4329.
Path 380 | total_timesteps 4340.
Path 381 | total_timesteps 4353.
Path 382 | total_timesteps 4372.
Path 383 | total_timesteps 4385.
Path 384 | total_timesteps 4397.
Path 385 | total_timesteps 4409.
Path 386 | total_timesteps 4427.
Path 387 | total_timesteps 4437.
Path 388 | total_timesteps 4450.
Path 389 | total_timesteps 4459.
Path 390 | total_timesteps 4478.
Path 391 | total_timesteps 4488.
Path 392 | total_timesteps 4497.
Path 393 | total_timesteps 4506.
Path 394 | total_timesteps 4514.
Path 395 | total_timesteps 4522.
Path 396 | total_timesteps 4535.
Path 397 | total_timesteps 4548.
Path 398 | total_timesteps 4558.
Path 399 | total_timesteps 4571.
Path 400 | total_timesteps 4578.
Path 401 | total_timesteps 4596.
Path 402 | total_timesteps 4610.
Path 403 | total_timesteps 4618.
Path 404 | total_timesteps 4632.
Path 405 | total_timesteps 4645.
Path 406 | total_timesteps 4655.
Path 407 | total_timesteps 4666.
Path 408 | total_timesteps 4678.
Path 409 | total_timesteps 4687.
Path 410 | total_timesteps 4696.
Path 411 | total_timesteps 4704.
Path 412 | total_timesteps 4717.
Path 413 | total_timesteps 4725.
Path 414 | total_timesteps 4738.
Path 415 | total_timesteps 4749.
Path 416 | total_timesteps 4758.
Path 417 | total_timesteps 4766.
Path 418 | total_timesteps 4776.
Path 419 | total_timesteps 4783.
Path 420 | total_timesteps 4796.
Path 421 | total_timesteps 4807.
Path 422 | total_timesteps 4815.
Path 423 | total_timesteps 4834.
Path 424 | total_timesteps 4842.
Path 425 | total_timesteps 4860.
Path 426 | total_timesteps 4869.
Path 427 | total_timesteps 4876.
Path 428 | total_timesteps 4885.
Path 429 | total_timesteps 4896.
Path 430 | total_timesteps 4903.
Path 431 | total_timesteps 4910.
Path 432 | total_timesteps 4921.
Path 433 | total_timesteps 4936.
Path 434 | total_timesteps 4958.
Path 435 | total_timesteps 4967.
Path 436 | total_timesteps 4979.
Path 437 | total_timesteps 4998.
Path 438 | total_timesteps 5013.
Path 439 | total_timesteps 5026.
Path 440 | total_timesteps 5038.
Path 441 | total_timesteps 5044.
Path 442 | total_timesteps 5055.
Path 443 | total_timesteps 5066.
Path 444 | total_timesteps 5074.
Path 445 | total_timesteps 5084.
Path 446 | total_timesteps 5097.
Path 447 | total_timesteps 5111.
Path 448 | total_timesteps 5123.
Path 449 | total_timesteps 5132.
Path 450 | total_timesteps 5143.
Path 451 | total_timesteps 5151.
Path 452 | total_timesteps 5160.
Path 453 | total_timesteps 5167.
Path 454 | total_timesteps 5175.
Path 455 | total_timesteps 5189.
Path 456 | total_timesteps 5197.
Path 457 | total_timesteps 5211.
Path 458 | total_timesteps 5224.
Path 459 | total_timesteps 5233.
Path 460 | total_timesteps 5240.
Path 461 | total_timesteps 5256.
Path 462 | total_timesteps 5265.
Path 463 | total_timesteps 5276.
Path 464 | total_timesteps 5284.
Path 465 | total_timesteps 5295.
Path 466 | total_timesteps 5309.
Path 467 | total_timesteps 5327.
Path 468 | total_timesteps 5338.
Path 469 | total_timesteps 5347.
Path 470 | total_timesteps 5354.
Path 471 | total_timesteps 5362.
Path 472 | total_timesteps 5370.
Path 473 | total_timesteps 5385.
Path 474 | total_timesteps 5395.
Path 475 | total_timesteps 5412.
Path 476 | total_timesteps 5424.
Path 477 | total_timesteps 5437.
Path 478 | total_timesteps 5443.
Path 479 | total_timesteps 5450.
Path 480 | total_timesteps 5462.
Path 481 | total_timesteps 5469.
Path 482 | total_timesteps 5485.
Path 483 | total_timesteps 5497.
Path 484 | total_timesteps 5508.
Path 485 | total_timesteps 5515.
Path 486 | total_timesteps 5539.
Path 487 | total_timesteps 5549.
Path 488 | total_timesteps 5559.
Path 489 | total_timesteps 5569.
Path 490 | total_timesteps 5577.
Path 491 | total_timesteps 5585.
Path 492 | total_timesteps 5594.
Path 493 | total_timesteps 5608.
Path 494 | total_timesteps 5619.
Path 495 | total_timesteps 5627.
Path 496 | total_timesteps 5634.
Path 497 | total_timesteps 5656.
Path 498 | total_timesteps 5667.
Path 499 | total_timesteps 5690.
Path 500 | total_timesteps 5698.
Path 501 | total_timesteps 5706.
Path 502 | total_timesteps 5713.
Path 503 | total_timesteps 5720.
Path 504 | total_timesteps 5730.
Path 505 | total_timesteps 5739.
Path 506 | total_timesteps 5747.
Path 507 | total_timesteps 5754.
Path 508 | total_timesteps 5764.
Path 509 | total_timesteps 5777.
Path 510 | total_timesteps 5787.
Path 511 | total_timesteps 5799.
Path 512 | total_timesteps 5811.
Path 513 | total_timesteps 5821.
Path 514 | total_timesteps 5833.
Path 515 | total_timesteps 5840.
Path 516 | total_timesteps 5850.
Path 517 | total_timesteps 5861.
Path 518 | total_timesteps 5869.
Path 519 | total_timesteps 5879.
Path 520 | total_timesteps 5888.
Path 521 | total_timesteps 5900.
Path 522 | total_timesteps 5913.
Path 523 | total_timesteps 5924.
Path 524 | total_timesteps 5933.
Path 525 | total_timesteps 5953.
Path 526 | total_timesteps 5964.
Path 527 | total_timesteps 5975.
Path 528 | total_timesteps 5985.
Path 529 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.35    |
| Iteration     | 20       |
| MaximumReturn | 1.49     |
| MinimumReturn | -22.2    |
| TotalSamples  | 88072    |
----------------------------
itr #21 | 
Fitting dynamics.
Validation loss = 0.005648566409945488
Validation loss = 0.005652513820677996
Validation loss = 0.005305636208504438
Validation loss = 0.0058260695077478886
Validation loss = 0.00526143703609705
Validation loss = 0.005201111081987619
Validation loss = 0.0055989231914281845
Validation loss = 0.00505034439265728
Validation loss = 0.005646299105137587
Validation loss = 0.005229413043707609
Validation loss = 0.00530684320256114
Validation loss = 0.005305582191795111
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 36.
Path 4 | total_timesteps 48.
Path 5 | total_timesteps 61.
Path 6 | total_timesteps 77.
Path 7 | total_timesteps 88.
Path 8 | total_timesteps 101.
Path 9 | total_timesteps 109.
Path 10 | total_timesteps 121.
Path 11 | total_timesteps 133.
Path 12 | total_timesteps 151.
Path 13 | total_timesteps 161.
Path 14 | total_timesteps 179.
Path 15 | total_timesteps 191.
Path 16 | total_timesteps 206.
Path 17 | total_timesteps 224.
Path 18 | total_timesteps 238.
Path 19 | total_timesteps 245.
Path 20 | total_timesteps 257.
Path 21 | total_timesteps 268.
Path 22 | total_timesteps 280.
Path 23 | total_timesteps 291.
Path 24 | total_timesteps 300.
Path 25 | total_timesteps 317.
Path 26 | total_timesteps 325.
Path 27 | total_timesteps 337.
Path 28 | total_timesteps 346.
Path 29 | total_timesteps 356.
Path 30 | total_timesteps 367.
Path 31 | total_timesteps 379.
Path 32 | total_timesteps 394.
Path 33 | total_timesteps 402.
Path 34 | total_timesteps 416.
Path 35 | total_timesteps 428.
Path 36 | total_timesteps 437.
Path 37 | total_timesteps 454.
Path 38 | total_timesteps 462.
Path 39 | total_timesteps 474.
Path 40 | total_timesteps 492.
Path 41 | total_timesteps 502.
Path 42 | total_timesteps 510.
Path 43 | total_timesteps 523.
Path 44 | total_timesteps 533.
Path 45 | total_timesteps 543.
Path 46 | total_timesteps 567.
Path 47 | total_timesteps 575.
Path 48 | total_timesteps 584.
Path 49 | total_timesteps 595.
Path 50 | total_timesteps 604.
Path 51 | total_timesteps 613.
Path 52 | total_timesteps 621.
Path 53 | total_timesteps 629.
Path 54 | total_timesteps 644.
Path 55 | total_timesteps 661.
Path 56 | total_timesteps 677.
Path 57 | total_timesteps 691.
Path 58 | total_timesteps 699.
Path 59 | total_timesteps 706.
Path 60 | total_timesteps 716.
Path 61 | total_timesteps 729.
Path 62 | total_timesteps 738.
Path 63 | total_timesteps 748.
Path 64 | total_timesteps 765.
Path 65 | total_timesteps 773.
Path 66 | total_timesteps 781.
Path 67 | total_timesteps 801.
Path 68 | total_timesteps 822.
Path 69 | total_timesteps 832.
Path 70 | total_timesteps 840.
Path 71 | total_timesteps 854.
Path 72 | total_timesteps 863.
Path 73 | total_timesteps 873.
Path 74 | total_timesteps 885.
Path 75 | total_timesteps 893.
Path 76 | total_timesteps 906.
Path 77 | total_timesteps 928.
Path 78 | total_timesteps 935.
Path 79 | total_timesteps 943.
Path 80 | total_timesteps 956.
Path 81 | total_timesteps 967.
Path 82 | total_timesteps 977.
Path 83 | total_timesteps 987.
Path 84 | total_timesteps 1002.
Path 85 | total_timesteps 1022.
Path 86 | total_timesteps 1039.
Path 87 | total_timesteps 1048.
Path 88 | total_timesteps 1057.
Path 89 | total_timesteps 1066.
Path 90 | total_timesteps 1074.
Path 91 | total_timesteps 1084.
Path 92 | total_timesteps 1093.
Path 93 | total_timesteps 1105.
Path 94 | total_timesteps 1118.
Path 95 | total_timesteps 1125.
Path 96 | total_timesteps 1136.
Path 97 | total_timesteps 1150.
Path 98 | total_timesteps 1177.
Path 99 | total_timesteps 1187.
Path 100 | total_timesteps 1196.
Path 101 | total_timesteps 1211.
Path 102 | total_timesteps 1218.
Path 103 | total_timesteps 1226.
Path 104 | total_timesteps 1239.
Path 105 | total_timesteps 1249.
Path 106 | total_timesteps 1260.
Path 107 | total_timesteps 1267.
Path 108 | total_timesteps 1274.
Path 109 | total_timesteps 1287.
Path 110 | total_timesteps 1298.
Path 111 | total_timesteps 1315.
Path 112 | total_timesteps 1322.
Path 113 | total_timesteps 1331.
Path 114 | total_timesteps 1341.
Path 115 | total_timesteps 1350.
Path 116 | total_timesteps 1364.
Path 117 | total_timesteps 1375.
Path 118 | total_timesteps 1385.
Path 119 | total_timesteps 1403.
Path 120 | total_timesteps 1416.
Path 121 | total_timesteps 1425.
Path 122 | total_timesteps 1431.
Path 123 | total_timesteps 1438.
Path 124 | total_timesteps 1455.
Path 125 | total_timesteps 1464.
Path 126 | total_timesteps 1480.
Path 127 | total_timesteps 1490.
Path 128 | total_timesteps 1504.
Path 129 | total_timesteps 1514.
Path 130 | total_timesteps 1531.
Path 131 | total_timesteps 1542.
Path 132 | total_timesteps 1552.
Path 133 | total_timesteps 1576.
Path 134 | total_timesteps 1591.
Path 135 | total_timesteps 1602.
Path 136 | total_timesteps 1608.
Path 137 | total_timesteps 1615.
Path 138 | total_timesteps 1622.
Path 139 | total_timesteps 1630.
Path 140 | total_timesteps 1640.
Path 141 | total_timesteps 1659.
Path 142 | total_timesteps 1667.
Path 143 | total_timesteps 1683.
Path 144 | total_timesteps 1694.
Path 145 | total_timesteps 1702.
Path 146 | total_timesteps 1713.
Path 147 | total_timesteps 1727.
Path 148 | total_timesteps 1741.
Path 149 | total_timesteps 1754.
Path 150 | total_timesteps 1776.
Path 151 | total_timesteps 1786.
Path 152 | total_timesteps 1794.
Path 153 | total_timesteps 1801.
Path 154 | total_timesteps 1810.
Path 155 | total_timesteps 1828.
Path 156 | total_timesteps 1840.
Path 157 | total_timesteps 1848.
Path 158 | total_timesteps 1856.
Path 159 | total_timesteps 1866.
Path 160 | total_timesteps 1881.
Path 161 | total_timesteps 1888.
Path 162 | total_timesteps 1895.
Path 163 | total_timesteps 1912.
Path 164 | total_timesteps 1919.
Path 165 | total_timesteps 1927.
Path 166 | total_timesteps 1934.
Path 167 | total_timesteps 1944.
Path 168 | total_timesteps 1952.
Path 169 | total_timesteps 1969.
Path 170 | total_timesteps 1982.
Path 171 | total_timesteps 2000.
Path 172 | total_timesteps 2011.
Path 173 | total_timesteps 2020.
Path 174 | total_timesteps 2046.
Path 175 | total_timesteps 2054.
Path 176 | total_timesteps 2065.
Path 177 | total_timesteps 2075.
Path 178 | total_timesteps 2086.
Path 179 | total_timesteps 2096.
Path 180 | total_timesteps 2106.
Path 181 | total_timesteps 2116.
Path 182 | total_timesteps 2127.
Path 183 | total_timesteps 2137.
Path 184 | total_timesteps 2149.
Path 185 | total_timesteps 2159.
Path 186 | total_timesteps 2165.
Path 187 | total_timesteps 2173.
Path 188 | total_timesteps 2189.
Path 189 | total_timesteps 2196.
Path 190 | total_timesteps 2209.
Path 191 | total_timesteps 2218.
Path 192 | total_timesteps 2237.
Path 193 | total_timesteps 2250.
Path 194 | total_timesteps 2258.
Path 195 | total_timesteps 2267.
Path 196 | total_timesteps 2278.
Path 197 | total_timesteps 2289.
Path 198 | total_timesteps 2304.
Path 199 | total_timesteps 2313.
Path 200 | total_timesteps 2323.
Path 201 | total_timesteps 2331.
Path 202 | total_timesteps 2338.
Path 203 | total_timesteps 2351.
Path 204 | total_timesteps 2361.
Path 205 | total_timesteps 2370.
Path 206 | total_timesteps 2379.
Path 207 | total_timesteps 2386.
Path 208 | total_timesteps 2399.
Path 209 | total_timesteps 2406.
Path 210 | total_timesteps 2418.
Path 211 | total_timesteps 2426.
Path 212 | total_timesteps 2434.
Path 213 | total_timesteps 2453.
Path 214 | total_timesteps 2462.
Path 215 | total_timesteps 2471.
Path 216 | total_timesteps 2477.
Path 217 | total_timesteps 2485.
Path 218 | total_timesteps 2498.
Path 219 | total_timesteps 2509.
Path 220 | total_timesteps 2520.
Path 221 | total_timesteps 2529.
Path 222 | total_timesteps 2543.
Path 223 | total_timesteps 2554.
Path 224 | total_timesteps 2572.
Path 225 | total_timesteps 2584.
Path 226 | total_timesteps 2595.
Path 227 | total_timesteps 2603.
Path 228 | total_timesteps 2611.
Path 229 | total_timesteps 2619.
Path 230 | total_timesteps 2633.
Path 231 | total_timesteps 2642.
Path 232 | total_timesteps 2652.
Path 233 | total_timesteps 2660.
Path 234 | total_timesteps 2671.
Path 235 | total_timesteps 2678.
Path 236 | total_timesteps 2692.
Path 237 | total_timesteps 2706.
Path 238 | total_timesteps 2716.
Path 239 | total_timesteps 2727.
Path 240 | total_timesteps 2745.
Path 241 | total_timesteps 2754.
Path 242 | total_timesteps 2766.
Path 243 | total_timesteps 2774.
Path 244 | total_timesteps 2781.
Path 245 | total_timesteps 2789.
Path 246 | total_timesteps 2798.
Path 247 | total_timesteps 2809.
Path 248 | total_timesteps 2825.
Path 249 | total_timesteps 2837.
Path 250 | total_timesteps 2854.
Path 251 | total_timesteps 2864.
Path 252 | total_timesteps 2873.
Path 253 | total_timesteps 2887.
Path 254 | total_timesteps 2896.
Path 255 | total_timesteps 2903.
Path 256 | total_timesteps 2915.
Path 257 | total_timesteps 2924.
Path 258 | total_timesteps 2936.
Path 259 | total_timesteps 2947.
Path 260 | total_timesteps 2958.
Path 261 | total_timesteps 2969.
Path 262 | total_timesteps 2979.
Path 263 | total_timesteps 2989.
Path 264 | total_timesteps 3006.
Path 265 | total_timesteps 3016.
Path 266 | total_timesteps 3023.
Path 267 | total_timesteps 3030.
Path 268 | total_timesteps 3045.
Path 269 | total_timesteps 3059.
Path 270 | total_timesteps 3066.
Path 271 | total_timesteps 3077.
Path 272 | total_timesteps 3093.
Path 273 | total_timesteps 3116.
Path 274 | total_timesteps 3125.
Path 275 | total_timesteps 3137.
Path 276 | total_timesteps 3156.
Path 277 | total_timesteps 3169.
Path 278 | total_timesteps 3176.
Path 279 | total_timesteps 3188.
Path 280 | total_timesteps 3196.
Path 281 | total_timesteps 3205.
Path 282 | total_timesteps 3222.
Path 283 | total_timesteps 3232.
Path 284 | total_timesteps 3241.
Path 285 | total_timesteps 3260.
Path 286 | total_timesteps 3268.
Path 287 | total_timesteps 3278.
Path 288 | total_timesteps 3293.
Path 289 | total_timesteps 3310.
Path 290 | total_timesteps 3329.
Path 291 | total_timesteps 3340.
Path 292 | total_timesteps 3349.
Path 293 | total_timesteps 3356.
Path 294 | total_timesteps 3369.
Path 295 | total_timesteps 3376.
Path 296 | total_timesteps 3398.
Path 297 | total_timesteps 3406.
Path 298 | total_timesteps 3413.
Path 299 | total_timesteps 3424.
Path 300 | total_timesteps 3432.
Path 301 | total_timesteps 3449.
Path 302 | total_timesteps 3460.
Path 303 | total_timesteps 3486.
Path 304 | total_timesteps 3498.
Path 305 | total_timesteps 3511.
Path 306 | total_timesteps 3533.
Path 307 | total_timesteps 3553.
Path 308 | total_timesteps 3561.
Path 309 | total_timesteps 3572.
Path 310 | total_timesteps 3581.
Path 311 | total_timesteps 3602.
Path 312 | total_timesteps 3610.
Path 313 | total_timesteps 3625.
Path 314 | total_timesteps 3635.
Path 315 | total_timesteps 3644.
Path 316 | total_timesteps 3654.
Path 317 | total_timesteps 3663.
Path 318 | total_timesteps 3671.
Path 319 | total_timesteps 3680.
Path 320 | total_timesteps 3699.
Path 321 | total_timesteps 3709.
Path 322 | total_timesteps 3738.
Path 323 | total_timesteps 3749.
Path 324 | total_timesteps 3761.
Path 325 | total_timesteps 3770.
Path 326 | total_timesteps 3779.
Path 327 | total_timesteps 3798.
Path 328 | total_timesteps 3810.
Path 329 | total_timesteps 3818.
Path 330 | total_timesteps 3838.
Path 331 | total_timesteps 3845.
Path 332 | total_timesteps 3854.
Path 333 | total_timesteps 3863.
Path 334 | total_timesteps 3871.
Path 335 | total_timesteps 3883.
Path 336 | total_timesteps 3897.
Path 337 | total_timesteps 3908.
Path 338 | total_timesteps 3918.
Path 339 | total_timesteps 3930.
Path 340 | total_timesteps 3943.
Path 341 | total_timesteps 3956.
Path 342 | total_timesteps 3965.
Path 343 | total_timesteps 3974.
Path 344 | total_timesteps 3984.
Path 345 | total_timesteps 3995.
Path 346 | total_timesteps 4005.
Path 347 | total_timesteps 4014.
Path 348 | total_timesteps 4024.
Path 349 | total_timesteps 4034.
Path 350 | total_timesteps 4051.
Path 351 | total_timesteps 4063.
Path 352 | total_timesteps 4073.
Path 353 | total_timesteps 4080.
Path 354 | total_timesteps 4089.
Path 355 | total_timesteps 4099.
Path 356 | total_timesteps 4107.
Path 357 | total_timesteps 4119.
Path 358 | total_timesteps 4128.
Path 359 | total_timesteps 4144.
Path 360 | total_timesteps 4155.
Path 361 | total_timesteps 4163.
Path 362 | total_timesteps 4174.
Path 363 | total_timesteps 4188.
Path 364 | total_timesteps 4200.
Path 365 | total_timesteps 4212.
Path 366 | total_timesteps 4226.
Path 367 | total_timesteps 4235.
Path 368 | total_timesteps 4242.
Path 369 | total_timesteps 4254.
Path 370 | total_timesteps 4276.
Path 371 | total_timesteps 4285.
Path 372 | total_timesteps 4292.
Path 373 | total_timesteps 4304.
Path 374 | total_timesteps 4318.
Path 375 | total_timesteps 4330.
Path 376 | total_timesteps 4338.
Path 377 | total_timesteps 4348.
Path 378 | total_timesteps 4355.
Path 379 | total_timesteps 4364.
Path 380 | total_timesteps 4383.
Path 381 | total_timesteps 4390.
Path 382 | total_timesteps 4425.
Path 383 | total_timesteps 4439.
Path 384 | total_timesteps 4458.
Path 385 | total_timesteps 4467.
Path 386 | total_timesteps 4477.
Path 387 | total_timesteps 4495.
Path 388 | total_timesteps 4507.
Path 389 | total_timesteps 4519.
Path 390 | total_timesteps 4535.
Path 391 | total_timesteps 4551.
Path 392 | total_timesteps 4564.
Path 393 | total_timesteps 4571.
Path 394 | total_timesteps 4583.
Path 395 | total_timesteps 4595.
Path 396 | total_timesteps 4604.
Path 397 | total_timesteps 4613.
Path 398 | total_timesteps 4627.
Path 399 | total_timesteps 4640.
Path 400 | total_timesteps 4660.
Path 401 | total_timesteps 4672.
Path 402 | total_timesteps 4684.
Path 403 | total_timesteps 4693.
Path 404 | total_timesteps 4705.
Path 405 | total_timesteps 4716.
Path 406 | total_timesteps 4723.
Path 407 | total_timesteps 4731.
Path 408 | total_timesteps 4743.
Path 409 | total_timesteps 4751.
Path 410 | total_timesteps 4759.
Path 411 | total_timesteps 4768.
Path 412 | total_timesteps 4778.
Path 413 | total_timesteps 4785.
Path 414 | total_timesteps 4804.
Path 415 | total_timesteps 4818.
Path 416 | total_timesteps 4829.
Path 417 | total_timesteps 4846.
Path 418 | total_timesteps 4856.
Path 419 | total_timesteps 4867.
Path 420 | total_timesteps 4877.
Path 421 | total_timesteps 4886.
Path 422 | total_timesteps 4897.
Path 423 | total_timesteps 4913.
Path 424 | total_timesteps 4921.
Path 425 | total_timesteps 4932.
Path 426 | total_timesteps 4945.
Path 427 | total_timesteps 4960.
Path 428 | total_timesteps 4971.
Path 429 | total_timesteps 4979.
Path 430 | total_timesteps 4989.
Path 431 | total_timesteps 5001.
Path 432 | total_timesteps 5009.
Path 433 | total_timesteps 5017.
Path 434 | total_timesteps 5023.
Path 435 | total_timesteps 5029.
Path 436 | total_timesteps 5038.
Path 437 | total_timesteps 5051.
Path 438 | total_timesteps 5061.
Path 439 | total_timesteps 5073.
Path 440 | total_timesteps 5081.
Path 441 | total_timesteps 5094.
Path 442 | total_timesteps 5110.
Path 443 | total_timesteps 5128.
Path 444 | total_timesteps 5134.
Path 445 | total_timesteps 5142.
Path 446 | total_timesteps 5158.
Path 447 | total_timesteps 5169.
Path 448 | total_timesteps 5182.
Path 449 | total_timesteps 5188.
Path 450 | total_timesteps 5197.
Path 451 | total_timesteps 5203.
Path 452 | total_timesteps 5213.
Path 453 | total_timesteps 5227.
Path 454 | total_timesteps 5236.
Path 455 | total_timesteps 5247.
Path 456 | total_timesteps 5257.
Path 457 | total_timesteps 5264.
Path 458 | total_timesteps 5275.
Path 459 | total_timesteps 5282.
Path 460 | total_timesteps 5295.
Path 461 | total_timesteps 5305.
Path 462 | total_timesteps 5314.
Path 463 | total_timesteps 5321.
Path 464 | total_timesteps 5327.
Path 465 | total_timesteps 5336.
Path 466 | total_timesteps 5350.
Path 467 | total_timesteps 5365.
Path 468 | total_timesteps 5385.
Path 469 | total_timesteps 5404.
Path 470 | total_timesteps 5411.
Path 471 | total_timesteps 5419.
Path 472 | total_timesteps 5431.
Path 473 | total_timesteps 5445.
Path 474 | total_timesteps 5453.
Path 475 | total_timesteps 5465.
Path 476 | total_timesteps 5479.
Path 477 | total_timesteps 5490.
Path 478 | total_timesteps 5505.
Path 479 | total_timesteps 5512.
Path 480 | total_timesteps 5519.
Path 481 | total_timesteps 5530.
Path 482 | total_timesteps 5542.
Path 483 | total_timesteps 5552.
Path 484 | total_timesteps 5560.
Path 485 | total_timesteps 5586.
Path 486 | total_timesteps 5596.
Path 487 | total_timesteps 5608.
Path 488 | total_timesteps 5618.
Path 489 | total_timesteps 5631.
Path 490 | total_timesteps 5638.
Path 491 | total_timesteps 5648.
Path 492 | total_timesteps 5655.
Path 493 | total_timesteps 5672.
Path 494 | total_timesteps 5683.
Path 495 | total_timesteps 5692.
Path 496 | total_timesteps 5705.
Path 497 | total_timesteps 5720.
Path 498 | total_timesteps 5734.
Path 499 | total_timesteps 5744.
Path 500 | total_timesteps 5752.
Path 501 | total_timesteps 5764.
Path 502 | total_timesteps 5773.
Path 503 | total_timesteps 5780.
Path 504 | total_timesteps 5793.
Path 505 | total_timesteps 5811.
Path 506 | total_timesteps 5825.
Path 507 | total_timesteps 5846.
Path 508 | total_timesteps 5854.
Path 509 | total_timesteps 5863.
Path 510 | total_timesteps 5872.
Path 511 | total_timesteps 5879.
Path 512 | total_timesteps 5890.
Path 513 | total_timesteps 5903.
Path 514 | total_timesteps 5912.
Path 515 | total_timesteps 5921.
Path 516 | total_timesteps 5930.
Path 517 | total_timesteps 5938.
Path 518 | total_timesteps 5947.
Path 519 | total_timesteps 5961.
Path 520 | total_timesteps 5983.
Path 521 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.69    |
| Iteration     | 21       |
| MaximumReturn | -0.742   |
| MinimumReturn | -21.9    |
| TotalSamples  | 92073    |
----------------------------
itr #22 | 
Fitting dynamics.
Validation loss = 0.0052061923779547215
Validation loss = 0.0053067333064973354
Validation loss = 0.005896040704101324
Validation loss = 0.005193411838263273
Validation loss = 0.005353851243853569
Validation loss = 0.005285713821649551
Validation loss = 0.005422279704362154
Validation loss = 0.004925643093883991
Validation loss = 0.005331052001565695
Validation loss = 0.005010745022445917
Validation loss = 0.005057195667177439
Validation loss = 0.005157736595720053
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 37.
Path 5 | total_timesteps 51.
Path 6 | total_timesteps 61.
Path 7 | total_timesteps 68.
Path 8 | total_timesteps 81.
Path 9 | total_timesteps 94.
Path 10 | total_timesteps 102.
Path 11 | total_timesteps 111.
Path 12 | total_timesteps 119.
Path 13 | total_timesteps 129.
Path 14 | total_timesteps 150.
Path 15 | total_timesteps 163.
Path 16 | total_timesteps 173.
Path 17 | total_timesteps 183.
Path 18 | total_timesteps 193.
Path 19 | total_timesteps 205.
Path 20 | total_timesteps 212.
Path 21 | total_timesteps 222.
Path 22 | total_timesteps 237.
Path 23 | total_timesteps 253.
Path 24 | total_timesteps 262.
Path 25 | total_timesteps 271.
Path 26 | total_timesteps 286.
Path 27 | total_timesteps 299.
Path 28 | total_timesteps 306.
Path 29 | total_timesteps 313.
Path 30 | total_timesteps 322.
Path 31 | total_timesteps 328.
Path 32 | total_timesteps 339.
Path 33 | total_timesteps 351.
Path 34 | total_timesteps 372.
Path 35 | total_timesteps 379.
Path 36 | total_timesteps 386.
Path 37 | total_timesteps 398.
Path 38 | total_timesteps 409.
Path 39 | total_timesteps 424.
Path 40 | total_timesteps 437.
Path 41 | total_timesteps 450.
Path 42 | total_timesteps 458.
Path 43 | total_timesteps 466.
Path 44 | total_timesteps 477.
Path 45 | total_timesteps 486.
Path 46 | total_timesteps 495.
Path 47 | total_timesteps 505.
Path 48 | total_timesteps 511.
Path 49 | total_timesteps 520.
Path 50 | total_timesteps 529.
Path 51 | total_timesteps 541.
Path 52 | total_timesteps 553.
Path 53 | total_timesteps 561.
Path 54 | total_timesteps 570.
Path 55 | total_timesteps 577.
Path 56 | total_timesteps 586.
Path 57 | total_timesteps 599.
Path 58 | total_timesteps 611.
Path 59 | total_timesteps 618.
Path 60 | total_timesteps 625.
Path 61 | total_timesteps 633.
Path 62 | total_timesteps 646.
Path 63 | total_timesteps 661.
Path 64 | total_timesteps 683.
Path 65 | total_timesteps 689.
Path 66 | total_timesteps 698.
Path 67 | total_timesteps 711.
Path 68 | total_timesteps 719.
Path 69 | total_timesteps 728.
Path 70 | total_timesteps 737.
Path 71 | total_timesteps 746.
Path 72 | total_timesteps 757.
Path 73 | total_timesteps 774.
Path 74 | total_timesteps 784.
Path 75 | total_timesteps 795.
Path 76 | total_timesteps 805.
Path 77 | total_timesteps 814.
Path 78 | total_timesteps 821.
Path 79 | total_timesteps 827.
Path 80 | total_timesteps 837.
Path 81 | total_timesteps 845.
Path 82 | total_timesteps 858.
Path 83 | total_timesteps 866.
Path 84 | total_timesteps 882.
Path 85 | total_timesteps 901.
Path 86 | total_timesteps 914.
Path 87 | total_timesteps 923.
Path 88 | total_timesteps 932.
Path 89 | total_timesteps 953.
Path 90 | total_timesteps 965.
Path 91 | total_timesteps 974.
Path 92 | total_timesteps 991.
Path 93 | total_timesteps 1002.
Path 94 | total_timesteps 1012.
Path 95 | total_timesteps 1022.
Path 96 | total_timesteps 1030.
Path 97 | total_timesteps 1040.
Path 98 | total_timesteps 1048.
Path 99 | total_timesteps 1054.
Path 100 | total_timesteps 1064.
Path 101 | total_timesteps 1077.
Path 102 | total_timesteps 1091.
Path 103 | total_timesteps 1112.
Path 104 | total_timesteps 1134.
Path 105 | total_timesteps 1145.
Path 106 | total_timesteps 1153.
Path 107 | total_timesteps 1166.
Path 108 | total_timesteps 1182.
Path 109 | total_timesteps 1193.
Path 110 | total_timesteps 1200.
Path 111 | total_timesteps 1207.
Path 112 | total_timesteps 1216.
Path 113 | total_timesteps 1227.
Path 114 | total_timesteps 1240.
Path 115 | total_timesteps 1251.
Path 116 | total_timesteps 1262.
Path 117 | total_timesteps 1278.
Path 118 | total_timesteps 1292.
Path 119 | total_timesteps 1310.
Path 120 | total_timesteps 1325.
Path 121 | total_timesteps 1332.
Path 122 | total_timesteps 1343.
Path 123 | total_timesteps 1357.
Path 124 | total_timesteps 1366.
Path 125 | total_timesteps 1379.
Path 126 | total_timesteps 1390.
Path 127 | total_timesteps 1402.
Path 128 | total_timesteps 1410.
Path 129 | total_timesteps 1419.
Path 130 | total_timesteps 1427.
Path 131 | total_timesteps 1442.
Path 132 | total_timesteps 1454.
Path 133 | total_timesteps 1465.
Path 134 | total_timesteps 1474.
Path 135 | total_timesteps 1484.
Path 136 | total_timesteps 1492.
Path 137 | total_timesteps 1502.
Path 138 | total_timesteps 1515.
Path 139 | total_timesteps 1533.
Path 140 | total_timesteps 1540.
Path 141 | total_timesteps 1549.
Path 142 | total_timesteps 1562.
Path 143 | total_timesteps 1574.
Path 144 | total_timesteps 1593.
Path 145 | total_timesteps 1608.
Path 146 | total_timesteps 1617.
Path 147 | total_timesteps 1637.
Path 148 | total_timesteps 1648.
Path 149 | total_timesteps 1659.
Path 150 | total_timesteps 1668.
Path 151 | total_timesteps 1677.
Path 152 | total_timesteps 1686.
Path 153 | total_timesteps 1693.
Path 154 | total_timesteps 1700.
Path 155 | total_timesteps 1711.
Path 156 | total_timesteps 1720.
Path 157 | total_timesteps 1730.
Path 158 | total_timesteps 1741.
Path 159 | total_timesteps 1754.
Path 160 | total_timesteps 1763.
Path 161 | total_timesteps 1771.
Path 162 | total_timesteps 1778.
Path 163 | total_timesteps 1786.
Path 164 | total_timesteps 1793.
Path 165 | total_timesteps 1809.
Path 166 | total_timesteps 1817.
Path 167 | total_timesteps 1833.
Path 168 | total_timesteps 1844.
Path 169 | total_timesteps 1855.
Path 170 | total_timesteps 1872.
Path 171 | total_timesteps 1883.
Path 172 | total_timesteps 1890.
Path 173 | total_timesteps 1901.
Path 174 | total_timesteps 1914.
Path 175 | total_timesteps 1922.
Path 176 | total_timesteps 1929.
Path 177 | total_timesteps 1945.
Path 178 | total_timesteps 1954.
Path 179 | total_timesteps 1965.
Path 180 | total_timesteps 1983.
Path 181 | total_timesteps 1997.
Path 182 | total_timesteps 2005.
Path 183 | total_timesteps 2021.
Path 184 | total_timesteps 2037.
Path 185 | total_timesteps 2051.
Path 186 | total_timesteps 2062.
Path 187 | total_timesteps 2070.
Path 188 | total_timesteps 2077.
Path 189 | total_timesteps 2087.
Path 190 | total_timesteps 2093.
Path 191 | total_timesteps 2104.
Path 192 | total_timesteps 2114.
Path 193 | total_timesteps 2122.
Path 194 | total_timesteps 2133.
Path 195 | total_timesteps 2141.
Path 196 | total_timesteps 2150.
Path 197 | total_timesteps 2158.
Path 198 | total_timesteps 2184.
Path 199 | total_timesteps 2196.
Path 200 | total_timesteps 2206.
Path 201 | total_timesteps 2214.
Path 202 | total_timesteps 2223.
Path 203 | total_timesteps 2245.
Path 204 | total_timesteps 2260.
Path 205 | total_timesteps 2269.
Path 206 | total_timesteps 2282.
Path 207 | total_timesteps 2290.
Path 208 | total_timesteps 2303.
Path 209 | total_timesteps 2310.
Path 210 | total_timesteps 2319.
Path 211 | total_timesteps 2332.
Path 212 | total_timesteps 2339.
Path 213 | total_timesteps 2351.
Path 214 | total_timesteps 2364.
Path 215 | total_timesteps 2374.
Path 216 | total_timesteps 2390.
Path 217 | total_timesteps 2398.
Path 218 | total_timesteps 2406.
Path 219 | total_timesteps 2413.
Path 220 | total_timesteps 2422.
Path 221 | total_timesteps 2432.
Path 222 | total_timesteps 2443.
Path 223 | total_timesteps 2452.
Path 224 | total_timesteps 2461.
Path 225 | total_timesteps 2468.
Path 226 | total_timesteps 2478.
Path 227 | total_timesteps 2486.
Path 228 | total_timesteps 2498.
Path 229 | total_timesteps 2517.
Path 230 | total_timesteps 2526.
Path 231 | total_timesteps 2535.
Path 232 | total_timesteps 2543.
Path 233 | total_timesteps 2556.
Path 234 | total_timesteps 2565.
Path 235 | total_timesteps 2577.
Path 236 | total_timesteps 2587.
Path 237 | total_timesteps 2595.
Path 238 | total_timesteps 2601.
Path 239 | total_timesteps 2610.
Path 240 | total_timesteps 2616.
Path 241 | total_timesteps 2629.
Path 242 | total_timesteps 2636.
Path 243 | total_timesteps 2647.
Path 244 | total_timesteps 2660.
Path 245 | total_timesteps 2676.
Path 246 | total_timesteps 2687.
Path 247 | total_timesteps 2695.
Path 248 | total_timesteps 2706.
Path 249 | total_timesteps 2714.
Path 250 | total_timesteps 2722.
Path 251 | total_timesteps 2741.
Path 252 | total_timesteps 2750.
Path 253 | total_timesteps 2762.
Path 254 | total_timesteps 2770.
Path 255 | total_timesteps 2779.
Path 256 | total_timesteps 2787.
Path 257 | total_timesteps 2800.
Path 258 | total_timesteps 2811.
Path 259 | total_timesteps 2824.
Path 260 | total_timesteps 2832.
Path 261 | total_timesteps 2844.
Path 262 | total_timesteps 2855.
Path 263 | total_timesteps 2864.
Path 264 | total_timesteps 2873.
Path 265 | total_timesteps 2882.
Path 266 | total_timesteps 2894.
Path 267 | total_timesteps 2905.
Path 268 | total_timesteps 2916.
Path 269 | total_timesteps 2928.
Path 270 | total_timesteps 2935.
Path 271 | total_timesteps 2949.
Path 272 | total_timesteps 2960.
Path 273 | total_timesteps 2968.
Path 274 | total_timesteps 2975.
Path 275 | total_timesteps 2983.
Path 276 | total_timesteps 2990.
Path 277 | total_timesteps 3000.
Path 278 | total_timesteps 3011.
Path 279 | total_timesteps 3020.
Path 280 | total_timesteps 3028.
Path 281 | total_timesteps 3038.
Path 282 | total_timesteps 3048.
Path 283 | total_timesteps 3057.
Path 284 | total_timesteps 3073.
Path 285 | total_timesteps 3081.
Path 286 | total_timesteps 3092.
Path 287 | total_timesteps 3105.
Path 288 | total_timesteps 3115.
Path 289 | total_timesteps 3125.
Path 290 | total_timesteps 3138.
Path 291 | total_timesteps 3148.
Path 292 | total_timesteps 3160.
Path 293 | total_timesteps 3167.
Path 294 | total_timesteps 3184.
Path 295 | total_timesteps 3192.
Path 296 | total_timesteps 3205.
Path 297 | total_timesteps 3223.
Path 298 | total_timesteps 3230.
Path 299 | total_timesteps 3242.
Path 300 | total_timesteps 3249.
Path 301 | total_timesteps 3262.
Path 302 | total_timesteps 3270.
Path 303 | total_timesteps 3278.
Path 304 | total_timesteps 3291.
Path 305 | total_timesteps 3300.
Path 306 | total_timesteps 3308.
Path 307 | total_timesteps 3317.
Path 308 | total_timesteps 3335.
Path 309 | total_timesteps 3346.
Path 310 | total_timesteps 3356.
Path 311 | total_timesteps 3364.
Path 312 | total_timesteps 3372.
Path 313 | total_timesteps 3380.
Path 314 | total_timesteps 3391.
Path 315 | total_timesteps 3401.
Path 316 | total_timesteps 3411.
Path 317 | total_timesteps 3419.
Path 318 | total_timesteps 3428.
Path 319 | total_timesteps 3437.
Path 320 | total_timesteps 3443.
Path 321 | total_timesteps 3449.
Path 322 | total_timesteps 3457.
Path 323 | total_timesteps 3464.
Path 324 | total_timesteps 3474.
Path 325 | total_timesteps 3483.
Path 326 | total_timesteps 3496.
Path 327 | total_timesteps 3508.
Path 328 | total_timesteps 3515.
Path 329 | total_timesteps 3522.
Path 330 | total_timesteps 3530.
Path 331 | total_timesteps 3538.
Path 332 | total_timesteps 3544.
Path 333 | total_timesteps 3552.
Path 334 | total_timesteps 3562.
Path 335 | total_timesteps 3574.
Path 336 | total_timesteps 3594.
Path 337 | total_timesteps 3606.
Path 338 | total_timesteps 3617.
Path 339 | total_timesteps 3625.
Path 340 | total_timesteps 3636.
Path 341 | total_timesteps 3647.
Path 342 | total_timesteps 3658.
Path 343 | total_timesteps 3667.
Path 344 | total_timesteps 3679.
Path 345 | total_timesteps 3688.
Path 346 | total_timesteps 3702.
Path 347 | total_timesteps 3709.
Path 348 | total_timesteps 3717.
Path 349 | total_timesteps 3724.
Path 350 | total_timesteps 3735.
Path 351 | total_timesteps 3742.
Path 352 | total_timesteps 3756.
Path 353 | total_timesteps 3768.
Path 354 | total_timesteps 3775.
Path 355 | total_timesteps 3783.
Path 356 | total_timesteps 3797.
Path 357 | total_timesteps 3807.
Path 358 | total_timesteps 3815.
Path 359 | total_timesteps 3833.
Path 360 | total_timesteps 3845.
Path 361 | total_timesteps 3855.
Path 362 | total_timesteps 3866.
Path 363 | total_timesteps 3875.
Path 364 | total_timesteps 3888.
Path 365 | total_timesteps 3897.
Path 366 | total_timesteps 3903.
Path 367 | total_timesteps 3915.
Path 368 | total_timesteps 3927.
Path 369 | total_timesteps 3937.
Path 370 | total_timesteps 3944.
Path 371 | total_timesteps 3951.
Path 372 | total_timesteps 3961.
Path 373 | total_timesteps 3972.
Path 374 | total_timesteps 3982.
Path 375 | total_timesteps 3989.
Path 376 | total_timesteps 3999.
Path 377 | total_timesteps 4007.
Path 378 | total_timesteps 4032.
Path 379 | total_timesteps 4043.
Path 380 | total_timesteps 4053.
Path 381 | total_timesteps 4061.
Path 382 | total_timesteps 4070.
Path 383 | total_timesteps 4099.
Path 384 | total_timesteps 4112.
Path 385 | total_timesteps 4123.
Path 386 | total_timesteps 4130.
Path 387 | total_timesteps 4149.
Path 388 | total_timesteps 4157.
Path 389 | total_timesteps 4167.
Path 390 | total_timesteps 4180.
Path 391 | total_timesteps 4188.
Path 392 | total_timesteps 4196.
Path 393 | total_timesteps 4204.
Path 394 | total_timesteps 4212.
Path 395 | total_timesteps 4226.
Path 396 | total_timesteps 4235.
Path 397 | total_timesteps 4247.
Path 398 | total_timesteps 4262.
Path 399 | total_timesteps 4269.
Path 400 | total_timesteps 4277.
Path 401 | total_timesteps 4288.
Path 402 | total_timesteps 4298.
Path 403 | total_timesteps 4310.
Path 404 | total_timesteps 4319.
Path 405 | total_timesteps 4326.
Path 406 | total_timesteps 4334.
Path 407 | total_timesteps 4342.
Path 408 | total_timesteps 4355.
Path 409 | total_timesteps 4364.
Path 410 | total_timesteps 4386.
Path 411 | total_timesteps 4397.
Path 412 | total_timesteps 4407.
Path 413 | total_timesteps 4419.
Path 414 | total_timesteps 4428.
Path 415 | total_timesteps 4441.
Path 416 | total_timesteps 4454.
Path 417 | total_timesteps 4467.
Path 418 | total_timesteps 4475.
Path 419 | total_timesteps 4482.
Path 420 | total_timesteps 4490.
Path 421 | total_timesteps 4498.
Path 422 | total_timesteps 4510.
Path 423 | total_timesteps 4523.
Path 424 | total_timesteps 4532.
Path 425 | total_timesteps 4541.
Path 426 | total_timesteps 4552.
Path 427 | total_timesteps 4564.
Path 428 | total_timesteps 4576.
Path 429 | total_timesteps 4582.
Path 430 | total_timesteps 4601.
Path 431 | total_timesteps 4620.
Path 432 | total_timesteps 4628.
Path 433 | total_timesteps 4641.
Path 434 | total_timesteps 4653.
Path 435 | total_timesteps 4664.
Path 436 | total_timesteps 4676.
Path 437 | total_timesteps 4684.
Path 438 | total_timesteps 4697.
Path 439 | total_timesteps 4710.
Path 440 | total_timesteps 4721.
Path 441 | total_timesteps 4730.
Path 442 | total_timesteps 4749.
Path 443 | total_timesteps 4756.
Path 444 | total_timesteps 4775.
Path 445 | total_timesteps 4788.
Path 446 | total_timesteps 4798.
Path 447 | total_timesteps 4816.
Path 448 | total_timesteps 4824.
Path 449 | total_timesteps 4831.
Path 450 | total_timesteps 4843.
Path 451 | total_timesteps 4851.
Path 452 | total_timesteps 4865.
Path 453 | total_timesteps 4875.
Path 454 | total_timesteps 4883.
Path 455 | total_timesteps 4892.
Path 456 | total_timesteps 4900.
Path 457 | total_timesteps 4908.
Path 458 | total_timesteps 4917.
Path 459 | total_timesteps 4925.
Path 460 | total_timesteps 4931.
Path 461 | total_timesteps 4940.
Path 462 | total_timesteps 4949.
Path 463 | total_timesteps 4959.
Path 464 | total_timesteps 4968.
Path 465 | total_timesteps 4976.
Path 466 | total_timesteps 4987.
Path 467 | total_timesteps 4994.
Path 468 | total_timesteps 5008.
Path 469 | total_timesteps 5017.
Path 470 | total_timesteps 5024.
Path 471 | total_timesteps 5034.
Path 472 | total_timesteps 5043.
Path 473 | total_timesteps 5053.
Path 474 | total_timesteps 5061.
Path 475 | total_timesteps 5069.
Path 476 | total_timesteps 5078.
Path 477 | total_timesteps 5089.
Path 478 | total_timesteps 5099.
Path 479 | total_timesteps 5108.
Path 480 | total_timesteps 5119.
Path 481 | total_timesteps 5136.
Path 482 | total_timesteps 5146.
Path 483 | total_timesteps 5155.
Path 484 | total_timesteps 5170.
Path 485 | total_timesteps 5177.
Path 486 | total_timesteps 5186.
Path 487 | total_timesteps 5195.
Path 488 | total_timesteps 5204.
Path 489 | total_timesteps 5219.
Path 490 | total_timesteps 5232.
Path 491 | total_timesteps 5248.
Path 492 | total_timesteps 5257.
Path 493 | total_timesteps 5266.
Path 494 | total_timesteps 5276.
Path 495 | total_timesteps 5290.
Path 496 | total_timesteps 5297.
Path 497 | total_timesteps 5305.
Path 498 | total_timesteps 5314.
Path 499 | total_timesteps 5323.
Path 500 | total_timesteps 5331.
Path 501 | total_timesteps 5353.
Path 502 | total_timesteps 5364.
Path 503 | total_timesteps 5379.
Path 504 | total_timesteps 5391.
Path 505 | total_timesteps 5398.
Path 506 | total_timesteps 5408.
Path 507 | total_timesteps 5418.
Path 508 | total_timesteps 5424.
Path 509 | total_timesteps 5434.
Path 510 | total_timesteps 5444.
Path 511 | total_timesteps 5451.
Path 512 | total_timesteps 5463.
Path 513 | total_timesteps 5472.
Path 514 | total_timesteps 5485.
Path 515 | total_timesteps 5496.
Path 516 | total_timesteps 5509.
Path 517 | total_timesteps 5518.
Path 518 | total_timesteps 5527.
Path 519 | total_timesteps 5536.
Path 520 | total_timesteps 5545.
Path 521 | total_timesteps 5561.
Path 522 | total_timesteps 5570.
Path 523 | total_timesteps 5580.
Path 524 | total_timesteps 5588.
Path 525 | total_timesteps 5601.
Path 526 | total_timesteps 5612.
Path 527 | total_timesteps 5620.
Path 528 | total_timesteps 5630.
Path 529 | total_timesteps 5641.
Path 530 | total_timesteps 5648.
Path 531 | total_timesteps 5661.
Path 532 | total_timesteps 5675.
Path 533 | total_timesteps 5684.
Path 534 | total_timesteps 5693.
Path 535 | total_timesteps 5701.
Path 536 | total_timesteps 5707.
Path 537 | total_timesteps 5716.
Path 538 | total_timesteps 5730.
Path 539 | total_timesteps 5738.
Path 540 | total_timesteps 5755.
Path 541 | total_timesteps 5767.
Path 542 | total_timesteps 5777.
Path 543 | total_timesteps 5789.
Path 544 | total_timesteps 5796.
Path 545 | total_timesteps 5814.
Path 546 | total_timesteps 5826.
Path 547 | total_timesteps 5835.
Path 548 | total_timesteps 5845.
Path 549 | total_timesteps 5854.
Path 550 | total_timesteps 5864.
Path 551 | total_timesteps 5881.
Path 552 | total_timesteps 5894.
Path 553 | total_timesteps 5903.
Path 554 | total_timesteps 5911.
Path 555 | total_timesteps 5933.
Path 556 | total_timesteps 5940.
Path 557 | total_timesteps 5953.
Path 558 | total_timesteps 5960.
Path 559 | total_timesteps 5969.
Path 560 | total_timesteps 5983.
Path 561 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.31    |
| Iteration     | 22       |
| MaximumReturn | -1       |
| MinimumReturn | -22.6    |
| TotalSamples  | 96087    |
----------------------------
itr #23 | 
Fitting dynamics.
Validation loss = 0.005302028264850378
Validation loss = 0.004970640875399113
Validation loss = 0.005271269008517265
Validation loss = 0.005034109577536583
Validation loss = 0.004995555151253939
Validation loss = 0.005542801693081856
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 43.
Path 5 | total_timesteps 54.
Path 6 | total_timesteps 66.
Path 7 | total_timesteps 72.
Path 8 | total_timesteps 83.
Path 9 | total_timesteps 90.
Path 10 | total_timesteps 101.
Path 11 | total_timesteps 110.
Path 12 | total_timesteps 120.
Path 13 | total_timesteps 126.
Path 14 | total_timesteps 139.
Path 15 | total_timesteps 149.
Path 16 | total_timesteps 161.
Path 17 | total_timesteps 170.
Path 18 | total_timesteps 183.
Path 19 | total_timesteps 193.
Path 20 | total_timesteps 205.
Path 21 | total_timesteps 219.
Path 22 | total_timesteps 227.
Path 23 | total_timesteps 233.
Path 24 | total_timesteps 246.
Path 25 | total_timesteps 258.
Path 26 | total_timesteps 266.
Path 27 | total_timesteps 279.
Path 28 | total_timesteps 296.
Path 29 | total_timesteps 309.
Path 30 | total_timesteps 316.
Path 31 | total_timesteps 329.
Path 32 | total_timesteps 337.
Path 33 | total_timesteps 349.
Path 34 | total_timesteps 357.
Path 35 | total_timesteps 366.
Path 36 | total_timesteps 380.
Path 37 | total_timesteps 417.
Path 38 | total_timesteps 425.
Path 39 | total_timesteps 433.
Path 40 | total_timesteps 440.
Path 41 | total_timesteps 447.
Path 42 | total_timesteps 457.
Path 43 | total_timesteps 471.
Path 44 | total_timesteps 486.
Path 45 | total_timesteps 503.
Path 46 | total_timesteps 512.
Path 47 | total_timesteps 522.
Path 48 | total_timesteps 531.
Path 49 | total_timesteps 540.
Path 50 | total_timesteps 549.
Path 51 | total_timesteps 556.
Path 52 | total_timesteps 566.
Path 53 | total_timesteps 573.
Path 54 | total_timesteps 581.
Path 55 | total_timesteps 589.
Path 56 | total_timesteps 604.
Path 57 | total_timesteps 613.
Path 58 | total_timesteps 623.
Path 59 | total_timesteps 629.
Path 60 | total_timesteps 638.
Path 61 | total_timesteps 652.
Path 62 | total_timesteps 665.
Path 63 | total_timesteps 674.
Path 64 | total_timesteps 684.
Path 65 | total_timesteps 692.
Path 66 | total_timesteps 702.
Path 67 | total_timesteps 710.
Path 68 | total_timesteps 721.
Path 69 | total_timesteps 730.
Path 70 | total_timesteps 744.
Path 71 | total_timesteps 755.
Path 72 | total_timesteps 765.
Path 73 | total_timesteps 774.
Path 74 | total_timesteps 784.
Path 75 | total_timesteps 795.
Path 76 | total_timesteps 803.
Path 77 | total_timesteps 813.
Path 78 | total_timesteps 821.
Path 79 | total_timesteps 833.
Path 80 | total_timesteps 846.
Path 81 | total_timesteps 859.
Path 82 | total_timesteps 868.
Path 83 | total_timesteps 876.
Path 84 | total_timesteps 887.
Path 85 | total_timesteps 897.
Path 86 | total_timesteps 913.
Path 87 | total_timesteps 926.
Path 88 | total_timesteps 937.
Path 89 | total_timesteps 946.
Path 90 | total_timesteps 956.
Path 91 | total_timesteps 963.
Path 92 | total_timesteps 972.
Path 93 | total_timesteps 985.
Path 94 | total_timesteps 995.
Path 95 | total_timesteps 1005.
Path 96 | total_timesteps 1012.
Path 97 | total_timesteps 1023.
Path 98 | total_timesteps 1033.
Path 99 | total_timesteps 1044.
Path 100 | total_timesteps 1057.
Path 101 | total_timesteps 1071.
Path 102 | total_timesteps 1080.
Path 103 | total_timesteps 1087.
Path 104 | total_timesteps 1095.
Path 105 | total_timesteps 1106.
Path 106 | total_timesteps 1114.
Path 107 | total_timesteps 1124.
Path 108 | total_timesteps 1132.
Path 109 | total_timesteps 1143.
Path 110 | total_timesteps 1155.
Path 111 | total_timesteps 1165.
Path 112 | total_timesteps 1172.
Path 113 | total_timesteps 1190.
Path 114 | total_timesteps 1200.
Path 115 | total_timesteps 1209.
Path 116 | total_timesteps 1218.
Path 117 | total_timesteps 1227.
Path 118 | total_timesteps 1239.
Path 119 | total_timesteps 1258.
Path 120 | total_timesteps 1267.
Path 121 | total_timesteps 1279.
Path 122 | total_timesteps 1289.
Path 123 | total_timesteps 1298.
Path 124 | total_timesteps 1304.
Path 125 | total_timesteps 1322.
Path 126 | total_timesteps 1336.
Path 127 | total_timesteps 1346.
Path 128 | total_timesteps 1352.
Path 129 | total_timesteps 1371.
Path 130 | total_timesteps 1388.
Path 131 | total_timesteps 1397.
Path 132 | total_timesteps 1409.
Path 133 | total_timesteps 1416.
Path 134 | total_timesteps 1425.
Path 135 | total_timesteps 1436.
Path 136 | total_timesteps 1443.
Path 137 | total_timesteps 1451.
Path 138 | total_timesteps 1462.
Path 139 | total_timesteps 1471.
Path 140 | total_timesteps 1480.
Path 141 | total_timesteps 1487.
Path 142 | total_timesteps 1499.
Path 143 | total_timesteps 1514.
Path 144 | total_timesteps 1521.
Path 145 | total_timesteps 1529.
Path 146 | total_timesteps 1541.
Path 147 | total_timesteps 1549.
Path 148 | total_timesteps 1562.
Path 149 | total_timesteps 1578.
Path 150 | total_timesteps 1586.
Path 151 | total_timesteps 1597.
Path 152 | total_timesteps 1608.
Path 153 | total_timesteps 1617.
Path 154 | total_timesteps 1624.
Path 155 | total_timesteps 1635.
Path 156 | total_timesteps 1645.
Path 157 | total_timesteps 1655.
Path 158 | total_timesteps 1667.
Path 159 | total_timesteps 1674.
Path 160 | total_timesteps 1681.
Path 161 | total_timesteps 1691.
Path 162 | total_timesteps 1704.
Path 163 | total_timesteps 1715.
Path 164 | total_timesteps 1726.
Path 165 | total_timesteps 1735.
Path 166 | total_timesteps 1743.
Path 167 | total_timesteps 1754.
Path 168 | total_timesteps 1765.
Path 169 | total_timesteps 1777.
Path 170 | total_timesteps 1785.
Path 171 | total_timesteps 1794.
Path 172 | total_timesteps 1804.
Path 173 | total_timesteps 1814.
Path 174 | total_timesteps 1823.
Path 175 | total_timesteps 1829.
Path 176 | total_timesteps 1839.
Path 177 | total_timesteps 1851.
Path 178 | total_timesteps 1862.
Path 179 | total_timesteps 1872.
Path 180 | total_timesteps 1881.
Path 181 | total_timesteps 1890.
Path 182 | total_timesteps 1903.
Path 183 | total_timesteps 1915.
Path 184 | total_timesteps 1925.
Path 185 | total_timesteps 1936.
Path 186 | total_timesteps 1948.
Path 187 | total_timesteps 1959.
Path 188 | total_timesteps 1969.
Path 189 | total_timesteps 1979.
Path 190 | total_timesteps 1988.
Path 191 | total_timesteps 1997.
Path 192 | total_timesteps 2010.
Path 193 | total_timesteps 2018.
Path 194 | total_timesteps 2032.
Path 195 | total_timesteps 2046.
Path 196 | total_timesteps 2063.
Path 197 | total_timesteps 2072.
Path 198 | total_timesteps 2080.
Path 199 | total_timesteps 2090.
Path 200 | total_timesteps 2100.
Path 201 | total_timesteps 2112.
Path 202 | total_timesteps 2126.
Path 203 | total_timesteps 2137.
Path 204 | total_timesteps 2146.
Path 205 | total_timesteps 2153.
Path 206 | total_timesteps 2161.
Path 207 | total_timesteps 2169.
Path 208 | total_timesteps 2178.
Path 209 | total_timesteps 2188.
Path 210 | total_timesteps 2200.
Path 211 | total_timesteps 2208.
Path 212 | total_timesteps 2217.
Path 213 | total_timesteps 2231.
Path 214 | total_timesteps 2246.
Path 215 | total_timesteps 2252.
Path 216 | total_timesteps 2262.
Path 217 | total_timesteps 2273.
Path 218 | total_timesteps 2285.
Path 219 | total_timesteps 2294.
Path 220 | total_timesteps 2310.
Path 221 | total_timesteps 2319.
Path 222 | total_timesteps 2326.
Path 223 | total_timesteps 2333.
Path 224 | total_timesteps 2340.
Path 225 | total_timesteps 2354.
Path 226 | total_timesteps 2362.
Path 227 | total_timesteps 2373.
Path 228 | total_timesteps 2393.
Path 229 | total_timesteps 2403.
Path 230 | total_timesteps 2409.
Path 231 | total_timesteps 2416.
Path 232 | total_timesteps 2437.
Path 233 | total_timesteps 2444.
Path 234 | total_timesteps 2452.
Path 235 | total_timesteps 2464.
Path 236 | total_timesteps 2471.
Path 237 | total_timesteps 2479.
Path 238 | total_timesteps 2495.
Path 239 | total_timesteps 2514.
Path 240 | total_timesteps 2521.
Path 241 | total_timesteps 2534.
Path 242 | total_timesteps 2543.
Path 243 | total_timesteps 2563.
Path 244 | total_timesteps 2586.
Path 245 | total_timesteps 2597.
Path 246 | total_timesteps 2608.
Path 247 | total_timesteps 2615.
Path 248 | total_timesteps 2622.
Path 249 | total_timesteps 2631.
Path 250 | total_timesteps 2637.
Path 251 | total_timesteps 2650.
Path 252 | total_timesteps 2659.
Path 253 | total_timesteps 2669.
Path 254 | total_timesteps 2679.
Path 255 | total_timesteps 2687.
Path 256 | total_timesteps 2696.
Path 257 | total_timesteps 2703.
Path 258 | total_timesteps 2712.
Path 259 | total_timesteps 2720.
Path 260 | total_timesteps 2733.
Path 261 | total_timesteps 2750.
Path 262 | total_timesteps 2758.
Path 263 | total_timesteps 2766.
Path 264 | total_timesteps 2775.
Path 265 | total_timesteps 2787.
Path 266 | total_timesteps 2798.
Path 267 | total_timesteps 2811.
Path 268 | total_timesteps 2825.
Path 269 | total_timesteps 2843.
Path 270 | total_timesteps 2854.
Path 271 | total_timesteps 2864.
Path 272 | total_timesteps 2876.
Path 273 | total_timesteps 2884.
Path 274 | total_timesteps 2892.
Path 275 | total_timesteps 2901.
Path 276 | total_timesteps 2911.
Path 277 | total_timesteps 2918.
Path 278 | total_timesteps 2930.
Path 279 | total_timesteps 2945.
Path 280 | total_timesteps 2954.
Path 281 | total_timesteps 2965.
Path 282 | total_timesteps 2977.
Path 283 | total_timesteps 2984.
Path 284 | total_timesteps 2999.
Path 285 | total_timesteps 3015.
Path 286 | total_timesteps 3024.
Path 287 | total_timesteps 3032.
Path 288 | total_timesteps 3038.
Path 289 | total_timesteps 3052.
Path 290 | total_timesteps 3064.
Path 291 | total_timesteps 3074.
Path 292 | total_timesteps 3088.
Path 293 | total_timesteps 3097.
Path 294 | total_timesteps 3108.
Path 295 | total_timesteps 3118.
Path 296 | total_timesteps 3127.
Path 297 | total_timesteps 3152.
Path 298 | total_timesteps 3163.
Path 299 | total_timesteps 3175.
Path 300 | total_timesteps 3185.
Path 301 | total_timesteps 3198.
Path 302 | total_timesteps 3206.
Path 303 | total_timesteps 3212.
Path 304 | total_timesteps 3221.
Path 305 | total_timesteps 3233.
Path 306 | total_timesteps 3241.
Path 307 | total_timesteps 3257.
Path 308 | total_timesteps 3264.
Path 309 | total_timesteps 3277.
Path 310 | total_timesteps 3285.
Path 311 | total_timesteps 3297.
Path 312 | total_timesteps 3306.
Path 313 | total_timesteps 3314.
Path 314 | total_timesteps 3321.
Path 315 | total_timesteps 3337.
Path 316 | total_timesteps 3352.
Path 317 | total_timesteps 3366.
Path 318 | total_timesteps 3373.
Path 319 | total_timesteps 3384.
Path 320 | total_timesteps 3395.
Path 321 | total_timesteps 3404.
Path 322 | total_timesteps 3411.
Path 323 | total_timesteps 3420.
Path 324 | total_timesteps 3430.
Path 325 | total_timesteps 3437.
Path 326 | total_timesteps 3448.
Path 327 | total_timesteps 3456.
Path 328 | total_timesteps 3463.
Path 329 | total_timesteps 3474.
Path 330 | total_timesteps 3492.
Path 331 | total_timesteps 3500.
Path 332 | total_timesteps 3514.
Path 333 | total_timesteps 3528.
Path 334 | total_timesteps 3537.
Path 335 | total_timesteps 3547.
Path 336 | total_timesteps 3555.
Path 337 | total_timesteps 3564.
Path 338 | total_timesteps 3571.
Path 339 | total_timesteps 3579.
Path 340 | total_timesteps 3600.
Path 341 | total_timesteps 3611.
Path 342 | total_timesteps 3620.
Path 343 | total_timesteps 3629.
Path 344 | total_timesteps 3639.
Path 345 | total_timesteps 3649.
Path 346 | total_timesteps 3657.
Path 347 | total_timesteps 3665.
Path 348 | total_timesteps 3674.
Path 349 | total_timesteps 3687.
Path 350 | total_timesteps 3704.
Path 351 | total_timesteps 3716.
Path 352 | total_timesteps 3729.
Path 353 | total_timesteps 3738.
Path 354 | total_timesteps 3746.
Path 355 | total_timesteps 3762.
Path 356 | total_timesteps 3770.
Path 357 | total_timesteps 3779.
Path 358 | total_timesteps 3788.
Path 359 | total_timesteps 3796.
Path 360 | total_timesteps 3809.
Path 361 | total_timesteps 3815.
Path 362 | total_timesteps 3825.
Path 363 | total_timesteps 3838.
Path 364 | total_timesteps 3852.
Path 365 | total_timesteps 3866.
Path 366 | total_timesteps 3877.
Path 367 | total_timesteps 3888.
Path 368 | total_timesteps 3905.
Path 369 | total_timesteps 3912.
Path 370 | total_timesteps 3933.
Path 371 | total_timesteps 3942.
Path 372 | total_timesteps 3949.
Path 373 | total_timesteps 3957.
Path 374 | total_timesteps 3969.
Path 375 | total_timesteps 3980.
Path 376 | total_timesteps 3991.
Path 377 | total_timesteps 4008.
Path 378 | total_timesteps 4015.
Path 379 | total_timesteps 4026.
Path 380 | total_timesteps 4034.
Path 381 | total_timesteps 4043.
Path 382 | total_timesteps 4051.
Path 383 | total_timesteps 4064.
Path 384 | total_timesteps 4076.
Path 385 | total_timesteps 4085.
Path 386 | total_timesteps 4094.
Path 387 | total_timesteps 4105.
Path 388 | total_timesteps 4113.
Path 389 | total_timesteps 4123.
Path 390 | total_timesteps 4131.
Path 391 | total_timesteps 4140.
Path 392 | total_timesteps 4151.
Path 393 | total_timesteps 4159.
Path 394 | total_timesteps 4180.
Path 395 | total_timesteps 4187.
Path 396 | total_timesteps 4199.
Path 397 | total_timesteps 4209.
Path 398 | total_timesteps 4220.
Path 399 | total_timesteps 4228.
Path 400 | total_timesteps 4234.
Path 401 | total_timesteps 4243.
Path 402 | total_timesteps 4254.
Path 403 | total_timesteps 4267.
Path 404 | total_timesteps 4276.
Path 405 | total_timesteps 4285.
Path 406 | total_timesteps 4294.
Path 407 | total_timesteps 4304.
Path 408 | total_timesteps 4312.
Path 409 | total_timesteps 4319.
Path 410 | total_timesteps 4329.
Path 411 | total_timesteps 4340.
Path 412 | total_timesteps 4348.
Path 413 | total_timesteps 4356.
Path 414 | total_timesteps 4364.
Path 415 | total_timesteps 4377.
Path 416 | total_timesteps 4397.
Path 417 | total_timesteps 4409.
Path 418 | total_timesteps 4416.
Path 419 | total_timesteps 4424.
Path 420 | total_timesteps 4435.
Path 421 | total_timesteps 4444.
Path 422 | total_timesteps 4457.
Path 423 | total_timesteps 4467.
Path 424 | total_timesteps 4476.
Path 425 | total_timesteps 4489.
Path 426 | total_timesteps 4499.
Path 427 | total_timesteps 4507.
Path 428 | total_timesteps 4513.
Path 429 | total_timesteps 4520.
Path 430 | total_timesteps 4529.
Path 431 | total_timesteps 4537.
Path 432 | total_timesteps 4544.
Path 433 | total_timesteps 4552.
Path 434 | total_timesteps 4561.
Path 435 | total_timesteps 4571.
Path 436 | total_timesteps 4582.
Path 437 | total_timesteps 4595.
Path 438 | total_timesteps 4608.
Path 439 | total_timesteps 4617.
Path 440 | total_timesteps 4631.
Path 441 | total_timesteps 4639.
Path 442 | total_timesteps 4654.
Path 443 | total_timesteps 4665.
Path 444 | total_timesteps 4675.
Path 445 | total_timesteps 4683.
Path 446 | total_timesteps 4694.
Path 447 | total_timesteps 4703.
Path 448 | total_timesteps 4714.
Path 449 | total_timesteps 4722.
Path 450 | total_timesteps 4739.
Path 451 | total_timesteps 4749.
Path 452 | total_timesteps 4758.
Path 453 | total_timesteps 4767.
Path 454 | total_timesteps 4778.
Path 455 | total_timesteps 4788.
Path 456 | total_timesteps 4801.
Path 457 | total_timesteps 4815.
Path 458 | total_timesteps 4824.
Path 459 | total_timesteps 4836.
Path 460 | total_timesteps 4851.
Path 461 | total_timesteps 4863.
Path 462 | total_timesteps 4876.
Path 463 | total_timesteps 4883.
Path 464 | total_timesteps 4891.
Path 465 | total_timesteps 4907.
Path 466 | total_timesteps 4917.
Path 467 | total_timesteps 4938.
Path 468 | total_timesteps 4948.
Path 469 | total_timesteps 4960.
Path 470 | total_timesteps 4975.
Path 471 | total_timesteps 4987.
Path 472 | total_timesteps 5002.
Path 473 | total_timesteps 5012.
Path 474 | total_timesteps 5021.
Path 475 | total_timesteps 5033.
Path 476 | total_timesteps 5040.
Path 477 | total_timesteps 5054.
Path 478 | total_timesteps 5062.
Path 479 | total_timesteps 5079.
Path 480 | total_timesteps 5088.
Path 481 | total_timesteps 5103.
Path 482 | total_timesteps 5112.
Path 483 | total_timesteps 5125.
Path 484 | total_timesteps 5131.
Path 485 | total_timesteps 5144.
Path 486 | total_timesteps 5159.
Path 487 | total_timesteps 5167.
Path 488 | total_timesteps 5174.
Path 489 | total_timesteps 5192.
Path 490 | total_timesteps 5202.
Path 491 | total_timesteps 5223.
Path 492 | total_timesteps 5236.
Path 493 | total_timesteps 5248.
Path 494 | total_timesteps 5257.
Path 495 | total_timesteps 5269.
Path 496 | total_timesteps 5280.
Path 497 | total_timesteps 5293.
Path 498 | total_timesteps 5303.
Path 499 | total_timesteps 5315.
Path 500 | total_timesteps 5326.
Path 501 | total_timesteps 5335.
Path 502 | total_timesteps 5344.
Path 503 | total_timesteps 5351.
Path 504 | total_timesteps 5357.
Path 505 | total_timesteps 5367.
Path 506 | total_timesteps 5376.
Path 507 | total_timesteps 5384.
Path 508 | total_timesteps 5392.
Path 509 | total_timesteps 5402.
Path 510 | total_timesteps 5413.
Path 511 | total_timesteps 5421.
Path 512 | total_timesteps 5430.
Path 513 | total_timesteps 5446.
Path 514 | total_timesteps 5456.
Path 515 | total_timesteps 5468.
Path 516 | total_timesteps 5480.
Path 517 | total_timesteps 5489.
Path 518 | total_timesteps 5496.
Path 519 | total_timesteps 5505.
Path 520 | total_timesteps 5518.
Path 521 | total_timesteps 5527.
Path 522 | total_timesteps 5537.
Path 523 | total_timesteps 5547.
Path 524 | total_timesteps 5553.
Path 525 | total_timesteps 5570.
Path 526 | total_timesteps 5584.
Path 527 | total_timesteps 5592.
Path 528 | total_timesteps 5603.
Path 529 | total_timesteps 5617.
Path 530 | total_timesteps 5638.
Path 531 | total_timesteps 5647.
Path 532 | total_timesteps 5663.
Path 533 | total_timesteps 5672.
Path 534 | total_timesteps 5682.
Path 535 | total_timesteps 5691.
Path 536 | total_timesteps 5701.
Path 537 | total_timesteps 5711.
Path 538 | total_timesteps 5721.
Path 539 | total_timesteps 5728.
Path 540 | total_timesteps 5743.
Path 541 | total_timesteps 5753.
Path 542 | total_timesteps 5760.
Path 543 | total_timesteps 5772.
Path 544 | total_timesteps 5780.
Path 545 | total_timesteps 5789.
Path 546 | total_timesteps 5803.
Path 547 | total_timesteps 5812.
Path 548 | total_timesteps 5826.
Path 549 | total_timesteps 5835.
Path 550 | total_timesteps 5842.
Path 551 | total_timesteps 5854.
Path 552 | total_timesteps 5864.
Path 553 | total_timesteps 5873.
Path 554 | total_timesteps 5881.
Path 555 | total_timesteps 5890.
Path 556 | total_timesteps 5901.
Path 557 | total_timesteps 5911.
Path 558 | total_timesteps 5925.
Path 559 | total_timesteps 5932.
Path 560 | total_timesteps 5941.
Path 561 | total_timesteps 5953.
Path 562 | total_timesteps 5968.
Path 563 | total_timesteps 5983.
Path 564 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.5     |
| Iteration     | 23       |
| MaximumReturn | -1.07    |
| MinimumReturn | -22.7    |
| TotalSamples  | 100090   |
----------------------------
itr #24 | 
Fitting dynamics.
Validation loss = 0.005153783597052097
Validation loss = 0.005650092847645283
Validation loss = 0.00477569131180644
Validation loss = 0.005031985230743885
Validation loss = 0.00518101267516613
Validation loss = 0.0047579556703567505
Validation loss = 0.004800393711775541
Validation loss = 0.004802475217729807
Validation loss = 0.004885412752628326
Validation loss = 0.004717633593827486
Validation loss = 0.004841642454266548
Validation loss = 0.004876954946666956
Validation loss = 0.004895660560578108
Validation loss = 0.004752600099891424
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 34.
Path 5 | total_timesteps 45.
Path 6 | total_timesteps 65.
Path 7 | total_timesteps 75.
Path 8 | total_timesteps 86.
Path 9 | total_timesteps 96.
Path 10 | total_timesteps 108.
Path 11 | total_timesteps 118.
Path 12 | total_timesteps 134.
Path 13 | total_timesteps 150.
Path 14 | total_timesteps 161.
Path 15 | total_timesteps 180.
Path 16 | total_timesteps 191.
Path 17 | total_timesteps 197.
Path 18 | total_timesteps 207.
Path 19 | total_timesteps 214.
Path 20 | total_timesteps 221.
Path 21 | total_timesteps 233.
Path 22 | total_timesteps 244.
Path 23 | total_timesteps 255.
Path 24 | total_timesteps 271.
Path 25 | total_timesteps 280.
Path 26 | total_timesteps 288.
Path 27 | total_timesteps 299.
Path 28 | total_timesteps 308.
Path 29 | total_timesteps 320.
Path 30 | total_timesteps 332.
Path 31 | total_timesteps 341.
Path 32 | total_timesteps 352.
Path 33 | total_timesteps 364.
Path 34 | total_timesteps 376.
Path 35 | total_timesteps 388.
Path 36 | total_timesteps 396.
Path 37 | total_timesteps 406.
Path 38 | total_timesteps 414.
Path 39 | total_timesteps 422.
Path 40 | total_timesteps 428.
Path 41 | total_timesteps 438.
Path 42 | total_timesteps 446.
Path 43 | total_timesteps 453.
Path 44 | total_timesteps 461.
Path 45 | total_timesteps 474.
Path 46 | total_timesteps 482.
Path 47 | total_timesteps 490.
Path 48 | total_timesteps 497.
Path 49 | total_timesteps 514.
Path 50 | total_timesteps 526.
Path 51 | total_timesteps 533.
Path 52 | total_timesteps 544.
Path 53 | total_timesteps 552.
Path 54 | total_timesteps 561.
Path 55 | total_timesteps 570.
Path 56 | total_timesteps 578.
Path 57 | total_timesteps 586.
Path 58 | total_timesteps 594.
Path 59 | total_timesteps 607.
Path 60 | total_timesteps 615.
Path 61 | total_timesteps 630.
Path 62 | total_timesteps 637.
Path 63 | total_timesteps 644.
Path 64 | total_timesteps 653.
Path 65 | total_timesteps 662.
Path 66 | total_timesteps 674.
Path 67 | total_timesteps 683.
Path 68 | total_timesteps 695.
Path 69 | total_timesteps 701.
Path 70 | total_timesteps 716.
Path 71 | total_timesteps 724.
Path 72 | total_timesteps 740.
Path 73 | total_timesteps 747.
Path 74 | total_timesteps 759.
Path 75 | total_timesteps 767.
Path 76 | total_timesteps 779.
Path 77 | total_timesteps 787.
Path 78 | total_timesteps 794.
Path 79 | total_timesteps 811.
Path 80 | total_timesteps 824.
Path 81 | total_timesteps 833.
Path 82 | total_timesteps 845.
Path 83 | total_timesteps 855.
Path 84 | total_timesteps 868.
Path 85 | total_timesteps 876.
Path 86 | total_timesteps 883.
Path 87 | total_timesteps 903.
Path 88 | total_timesteps 911.
Path 89 | total_timesteps 924.
Path 90 | total_timesteps 931.
Path 91 | total_timesteps 942.
Path 92 | total_timesteps 958.
Path 93 | total_timesteps 969.
Path 94 | total_timesteps 980.
Path 95 | total_timesteps 988.
Path 96 | total_timesteps 1000.
Path 97 | total_timesteps 1011.
Path 98 | total_timesteps 1031.
Path 99 | total_timesteps 1040.
Path 100 | total_timesteps 1047.
Path 101 | total_timesteps 1059.
Path 102 | total_timesteps 1081.
Path 103 | total_timesteps 1089.
Path 104 | total_timesteps 1101.
Path 105 | total_timesteps 1113.
Path 106 | total_timesteps 1123.
Path 107 | total_timesteps 1140.
Path 108 | total_timesteps 1152.
Path 109 | total_timesteps 1163.
Path 110 | total_timesteps 1172.
Path 111 | total_timesteps 1185.
Path 112 | total_timesteps 1196.
Path 113 | total_timesteps 1206.
Path 114 | total_timesteps 1228.
Path 115 | total_timesteps 1237.
Path 116 | total_timesteps 1252.
Path 117 | total_timesteps 1260.
Path 118 | total_timesteps 1274.
Path 119 | total_timesteps 1282.
Path 120 | total_timesteps 1298.
Path 121 | total_timesteps 1311.
Path 122 | total_timesteps 1326.
Path 123 | total_timesteps 1333.
Path 124 | total_timesteps 1341.
Path 125 | total_timesteps 1351.
Path 126 | total_timesteps 1360.
Path 127 | total_timesteps 1375.
Path 128 | total_timesteps 1392.
Path 129 | total_timesteps 1408.
Path 130 | total_timesteps 1428.
Path 131 | total_timesteps 1435.
Path 132 | total_timesteps 1445.
Path 133 | total_timesteps 1455.
Path 134 | total_timesteps 1465.
Path 135 | total_timesteps 1474.
Path 136 | total_timesteps 1487.
Path 137 | total_timesteps 1497.
Path 138 | total_timesteps 1508.
Path 139 | total_timesteps 1516.
Path 140 | total_timesteps 1526.
Path 141 | total_timesteps 1543.
Path 142 | total_timesteps 1552.
Path 143 | total_timesteps 1564.
Path 144 | total_timesteps 1573.
Path 145 | total_timesteps 1580.
Path 146 | total_timesteps 1590.
Path 147 | total_timesteps 1598.
Path 148 | total_timesteps 1607.
Path 149 | total_timesteps 1620.
Path 150 | total_timesteps 1636.
Path 151 | total_timesteps 1650.
Path 152 | total_timesteps 1665.
Path 153 | total_timesteps 1674.
Path 154 | total_timesteps 1687.
Path 155 | total_timesteps 1699.
Path 156 | total_timesteps 1709.
Path 157 | total_timesteps 1720.
Path 158 | total_timesteps 1732.
Path 159 | total_timesteps 1741.
Path 160 | total_timesteps 1755.
Path 161 | total_timesteps 1761.
Path 162 | total_timesteps 1772.
Path 163 | total_timesteps 1780.
Path 164 | total_timesteps 1793.
Path 165 | total_timesteps 1802.
Path 166 | total_timesteps 1811.
Path 167 | total_timesteps 1820.
Path 168 | total_timesteps 1828.
Path 169 | total_timesteps 1837.
Path 170 | total_timesteps 1852.
Path 171 | total_timesteps 1863.
Path 172 | total_timesteps 1872.
Path 173 | total_timesteps 1885.
Path 174 | total_timesteps 1893.
Path 175 | total_timesteps 1906.
Path 176 | total_timesteps 1916.
Path 177 | total_timesteps 1927.
Path 178 | total_timesteps 1937.
Path 179 | total_timesteps 1945.
Path 180 | total_timesteps 1955.
Path 181 | total_timesteps 1963.
Path 182 | total_timesteps 1974.
Path 183 | total_timesteps 1984.
Path 184 | total_timesteps 1992.
Path 185 | total_timesteps 2000.
Path 186 | total_timesteps 2015.
Path 187 | total_timesteps 2025.
Path 188 | total_timesteps 2033.
Path 189 | total_timesteps 2047.
Path 190 | total_timesteps 2058.
Path 191 | total_timesteps 2070.
Path 192 | total_timesteps 2097.
Path 193 | total_timesteps 2114.
Path 194 | total_timesteps 2126.
Path 195 | total_timesteps 2136.
Path 196 | total_timesteps 2151.
Path 197 | total_timesteps 2162.
Path 198 | total_timesteps 2171.
Path 199 | total_timesteps 2181.
Path 200 | total_timesteps 2191.
Path 201 | total_timesteps 2200.
Path 202 | total_timesteps 2210.
Path 203 | total_timesteps 2221.
Path 204 | total_timesteps 2229.
Path 205 | total_timesteps 2241.
Path 206 | total_timesteps 2249.
Path 207 | total_timesteps 2256.
Path 208 | total_timesteps 2268.
Path 209 | total_timesteps 2289.
Path 210 | total_timesteps 2301.
Path 211 | total_timesteps 2325.
Path 212 | total_timesteps 2336.
Path 213 | total_timesteps 2354.
Path 214 | total_timesteps 2363.
Path 215 | total_timesteps 2371.
Path 216 | total_timesteps 2389.
Path 217 | total_timesteps 2398.
Path 218 | total_timesteps 2408.
Path 219 | total_timesteps 2425.
Path 220 | total_timesteps 2435.
Path 221 | total_timesteps 2442.
Path 222 | total_timesteps 2456.
Path 223 | total_timesteps 2470.
Path 224 | total_timesteps 2479.
Path 225 | total_timesteps 2495.
Path 226 | total_timesteps 2507.
Path 227 | total_timesteps 2518.
Path 228 | total_timesteps 2526.
Path 229 | total_timesteps 2537.
Path 230 | total_timesteps 2556.
Path 231 | total_timesteps 2566.
Path 232 | total_timesteps 2575.
Path 233 | total_timesteps 2585.
Path 234 | total_timesteps 2593.
Path 235 | total_timesteps 2603.
Path 236 | total_timesteps 2624.
Path 237 | total_timesteps 2638.
Path 238 | total_timesteps 2644.
Path 239 | total_timesteps 2656.
Path 240 | total_timesteps 2664.
Path 241 | total_timesteps 2676.
Path 242 | total_timesteps 2689.
Path 243 | total_timesteps 2712.
Path 244 | total_timesteps 2728.
Path 245 | total_timesteps 2742.
Path 246 | total_timesteps 2748.
Path 247 | total_timesteps 2757.
Path 248 | total_timesteps 2770.
Path 249 | total_timesteps 2777.
Path 250 | total_timesteps 2788.
Path 251 | total_timesteps 2795.
Path 252 | total_timesteps 2801.
Path 253 | total_timesteps 2813.
Path 254 | total_timesteps 2824.
Path 255 | total_timesteps 2833.
Path 256 | total_timesteps 2841.
Path 257 | total_timesteps 2849.
Path 258 | total_timesteps 2863.
Path 259 | total_timesteps 2872.
Path 260 | total_timesteps 2880.
Path 261 | total_timesteps 2887.
Path 262 | total_timesteps 2894.
Path 263 | total_timesteps 2904.
Path 264 | total_timesteps 2914.
Path 265 | total_timesteps 2926.
Path 266 | total_timesteps 2934.
Path 267 | total_timesteps 2943.
Path 268 | total_timesteps 2952.
Path 269 | total_timesteps 2959.
Path 270 | total_timesteps 2968.
Path 271 | total_timesteps 2978.
Path 272 | total_timesteps 2990.
Path 273 | total_timesteps 2999.
Path 274 | total_timesteps 3008.
Path 275 | total_timesteps 3023.
Path 276 | total_timesteps 3049.
Path 277 | total_timesteps 3056.
Path 278 | total_timesteps 3069.
Path 279 | total_timesteps 3081.
Path 280 | total_timesteps 3098.
Path 281 | total_timesteps 3107.
Path 282 | total_timesteps 3117.
Path 283 | total_timesteps 3126.
Path 284 | total_timesteps 3134.
Path 285 | total_timesteps 3150.
Path 286 | total_timesteps 3166.
Path 287 | total_timesteps 3174.
Path 288 | total_timesteps 3190.
Path 289 | total_timesteps 3197.
Path 290 | total_timesteps 3206.
Path 291 | total_timesteps 3213.
Path 292 | total_timesteps 3221.
Path 293 | total_timesteps 3230.
Path 294 | total_timesteps 3240.
Path 295 | total_timesteps 3249.
Path 296 | total_timesteps 3256.
Path 297 | total_timesteps 3265.
Path 298 | total_timesteps 3274.
Path 299 | total_timesteps 3288.
Path 300 | total_timesteps 3303.
Path 301 | total_timesteps 3311.
Path 302 | total_timesteps 3318.
Path 303 | total_timesteps 3339.
Path 304 | total_timesteps 3351.
Path 305 | total_timesteps 3358.
Path 306 | total_timesteps 3374.
Path 307 | total_timesteps 3382.
Path 308 | total_timesteps 3400.
Path 309 | total_timesteps 3410.
Path 310 | total_timesteps 3420.
Path 311 | total_timesteps 3428.
Path 312 | total_timesteps 3437.
Path 313 | total_timesteps 3445.
Path 314 | total_timesteps 3455.
Path 315 | total_timesteps 3467.
Path 316 | total_timesteps 3478.
Path 317 | total_timesteps 3490.
Path 318 | total_timesteps 3498.
Path 319 | total_timesteps 3512.
Path 320 | total_timesteps 3525.
Path 321 | total_timesteps 3532.
Path 322 | total_timesteps 3539.
Path 323 | total_timesteps 3550.
Path 324 | total_timesteps 3572.
Path 325 | total_timesteps 3581.
Path 326 | total_timesteps 3605.
Path 327 | total_timesteps 3616.
Path 328 | total_timesteps 3624.
Path 329 | total_timesteps 3631.
Path 330 | total_timesteps 3641.
Path 331 | total_timesteps 3651.
Path 332 | total_timesteps 3662.
Path 333 | total_timesteps 3677.
Path 334 | total_timesteps 3691.
Path 335 | total_timesteps 3702.
Path 336 | total_timesteps 3709.
Path 337 | total_timesteps 3718.
Path 338 | total_timesteps 3728.
Path 339 | total_timesteps 3742.
Path 340 | total_timesteps 3750.
Path 341 | total_timesteps 3771.
Path 342 | total_timesteps 3782.
Path 343 | total_timesteps 3792.
Path 344 | total_timesteps 3804.
Path 345 | total_timesteps 3815.
Path 346 | total_timesteps 3826.
Path 347 | total_timesteps 3841.
Path 348 | total_timesteps 3849.
Path 349 | total_timesteps 3856.
Path 350 | total_timesteps 3874.
Path 351 | total_timesteps 3883.
Path 352 | total_timesteps 3902.
Path 353 | total_timesteps 3913.
Path 354 | total_timesteps 3926.
Path 355 | total_timesteps 3933.
Path 356 | total_timesteps 3944.
Path 357 | total_timesteps 3953.
Path 358 | total_timesteps 3970.
Path 359 | total_timesteps 3978.
Path 360 | total_timesteps 3990.
Path 361 | total_timesteps 3999.
Path 362 | total_timesteps 4009.
Path 363 | total_timesteps 4024.
Path 364 | total_timesteps 4032.
Path 365 | total_timesteps 4039.
Path 366 | total_timesteps 4047.
Path 367 | total_timesteps 4057.
Path 368 | total_timesteps 4067.
Path 369 | total_timesteps 4076.
Path 370 | total_timesteps 4090.
Path 371 | total_timesteps 4097.
Path 372 | total_timesteps 4111.
Path 373 | total_timesteps 4122.
Path 374 | total_timesteps 4129.
Path 375 | total_timesteps 4137.
Path 376 | total_timesteps 4148.
Path 377 | total_timesteps 4158.
Path 378 | total_timesteps 4176.
Path 379 | total_timesteps 4186.
Path 380 | total_timesteps 4204.
Path 381 | total_timesteps 4211.
Path 382 | total_timesteps 4224.
Path 383 | total_timesteps 4238.
Path 384 | total_timesteps 4245.
Path 385 | total_timesteps 4256.
Path 386 | total_timesteps 4262.
Path 387 | total_timesteps 4270.
Path 388 | total_timesteps 4280.
Path 389 | total_timesteps 4296.
Path 390 | total_timesteps 4313.
Path 391 | total_timesteps 4321.
Path 392 | total_timesteps 4328.
Path 393 | total_timesteps 4341.
Path 394 | total_timesteps 4354.
Path 395 | total_timesteps 4362.
Path 396 | total_timesteps 4375.
Path 397 | total_timesteps 4385.
Path 398 | total_timesteps 4400.
Path 399 | total_timesteps 4407.
Path 400 | total_timesteps 4416.
Path 401 | total_timesteps 4425.
Path 402 | total_timesteps 4432.
Path 403 | total_timesteps 4456.
Path 404 | total_timesteps 4464.
Path 405 | total_timesteps 4471.
Path 406 | total_timesteps 4481.
Path 407 | total_timesteps 4495.
Path 408 | total_timesteps 4505.
Path 409 | total_timesteps 4515.
Path 410 | total_timesteps 4525.
Path 411 | total_timesteps 4535.
Path 412 | total_timesteps 4543.
Path 413 | total_timesteps 4551.
Path 414 | total_timesteps 4562.
Path 415 | total_timesteps 4569.
Path 416 | total_timesteps 4578.
Path 417 | total_timesteps 4587.
Path 418 | total_timesteps 4596.
Path 419 | total_timesteps 4617.
Path 420 | total_timesteps 4631.
Path 421 | total_timesteps 4648.
Path 422 | total_timesteps 4656.
Path 423 | total_timesteps 4672.
Path 424 | total_timesteps 4679.
Path 425 | total_timesteps 4689.
Path 426 | total_timesteps 4697.
Path 427 | total_timesteps 4704.
Path 428 | total_timesteps 4716.
Path 429 | total_timesteps 4726.
Path 430 | total_timesteps 4737.
Path 431 | total_timesteps 4757.
Path 432 | total_timesteps 4767.
Path 433 | total_timesteps 4778.
Path 434 | total_timesteps 4786.
Path 435 | total_timesteps 4797.
Path 436 | total_timesteps 4806.
Path 437 | total_timesteps 4814.
Path 438 | total_timesteps 4820.
Path 439 | total_timesteps 4828.
Path 440 | total_timesteps 4844.
Path 441 | total_timesteps 4857.
Path 442 | total_timesteps 4868.
Path 443 | total_timesteps 4883.
Path 444 | total_timesteps 4893.
Path 445 | total_timesteps 4901.
Path 446 | total_timesteps 4908.
Path 447 | total_timesteps 4917.
Path 448 | total_timesteps 4928.
Path 449 | total_timesteps 4941.
Path 450 | total_timesteps 4950.
Path 451 | total_timesteps 4957.
Path 452 | total_timesteps 4974.
Path 453 | total_timesteps 4985.
Path 454 | total_timesteps 4996.
Path 455 | total_timesteps 5006.
Path 456 | total_timesteps 5016.
Path 457 | total_timesteps 5027.
Path 458 | total_timesteps 5045.
Path 459 | total_timesteps 5056.
Path 460 | total_timesteps 5066.
Path 461 | total_timesteps 5074.
Path 462 | total_timesteps 5081.
Path 463 | total_timesteps 5093.
Path 464 | total_timesteps 5100.
Path 465 | total_timesteps 5109.
Path 466 | total_timesteps 5130.
Path 467 | total_timesteps 5139.
Path 468 | total_timesteps 5146.
Path 469 | total_timesteps 5157.
Path 470 | total_timesteps 5172.
Path 471 | total_timesteps 5183.
Path 472 | total_timesteps 5192.
Path 473 | total_timesteps 5202.
Path 474 | total_timesteps 5214.
Path 475 | total_timesteps 5222.
Path 476 | total_timesteps 5268.
Path 477 | total_timesteps 5278.
Path 478 | total_timesteps 5288.
Path 479 | total_timesteps 5303.
Path 480 | total_timesteps 5314.
Path 481 | total_timesteps 5329.
Path 482 | total_timesteps 5341.
Path 483 | total_timesteps 5353.
Path 484 | total_timesteps 5367.
Path 485 | total_timesteps 5373.
Path 486 | total_timesteps 5381.
Path 487 | total_timesteps 5396.
Path 488 | total_timesteps 5404.
Path 489 | total_timesteps 5414.
Path 490 | total_timesteps 5423.
Path 491 | total_timesteps 5429.
Path 492 | total_timesteps 5438.
Path 493 | total_timesteps 5445.
Path 494 | total_timesteps 5453.
Path 495 | total_timesteps 5471.
Path 496 | total_timesteps 5479.
Path 497 | total_timesteps 5500.
Path 498 | total_timesteps 5511.
Path 499 | total_timesteps 5523.
Path 500 | total_timesteps 5536.
Path 501 | total_timesteps 5549.
Path 502 | total_timesteps 5558.
Path 503 | total_timesteps 5570.
Path 504 | total_timesteps 5588.
Path 505 | total_timesteps 5599.
Path 506 | total_timesteps 5607.
Path 507 | total_timesteps 5614.
Path 508 | total_timesteps 5627.
Path 509 | total_timesteps 5638.
Path 510 | total_timesteps 5647.
Path 511 | total_timesteps 5662.
Path 512 | total_timesteps 5671.
Path 513 | total_timesteps 5681.
Path 514 | total_timesteps 5696.
Path 515 | total_timesteps 5704.
Path 516 | total_timesteps 5710.
Path 517 | total_timesteps 5729.
Path 518 | total_timesteps 5738.
Path 519 | total_timesteps 5747.
Path 520 | total_timesteps 5756.
Path 521 | total_timesteps 5763.
Path 522 | total_timesteps 5772.
Path 523 | total_timesteps 5781.
Path 524 | total_timesteps 5789.
Path 525 | total_timesteps 5800.
Path 526 | total_timesteps 5807.
Path 527 | total_timesteps 5815.
Path 528 | total_timesteps 5824.
Path 529 | total_timesteps 5845.
Path 530 | total_timesteps 5860.
Path 531 | total_timesteps 5870.
Path 532 | total_timesteps 5886.
Path 533 | total_timesteps 5899.
Path 534 | total_timesteps 5912.
Path 535 | total_timesteps 5928.
Path 536 | total_timesteps 5938.
Path 537 | total_timesteps 5949.
Path 538 | total_timesteps 5959.
Path 539 | total_timesteps 5973.
Path 540 | total_timesteps 5980.
Path 541 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.75    |
| Iteration     | 24       |
| MaximumReturn | -2.21    |
| MinimumReturn | -24.2    |
| TotalSamples  | 104091   |
----------------------------
itr #25 | 
Fitting dynamics.
Validation loss = 0.004846717696636915
Validation loss = 0.004985093139111996
Validation loss = 0.0046989331021904945
Validation loss = 0.00460388045758009
Validation loss = 0.00483302678912878
Validation loss = 0.00487159937620163
Validation loss = 0.004720277152955532
Validation loss = 0.005025588441640139
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 37.
Path 4 | total_timesteps 46.
Path 5 | total_timesteps 55.
Path 6 | total_timesteps 64.
Path 7 | total_timesteps 75.
Path 8 | total_timesteps 83.
Path 9 | total_timesteps 91.
Path 10 | total_timesteps 98.
Path 11 | total_timesteps 106.
Path 12 | total_timesteps 122.
Path 13 | total_timesteps 130.
Path 14 | total_timesteps 137.
Path 15 | total_timesteps 145.
Path 16 | total_timesteps 153.
Path 17 | total_timesteps 160.
Path 18 | total_timesteps 167.
Path 19 | total_timesteps 176.
Path 20 | total_timesteps 183.
Path 21 | total_timesteps 190.
Path 22 | total_timesteps 205.
Path 23 | total_timesteps 214.
Path 24 | total_timesteps 229.
Path 25 | total_timesteps 241.
Path 26 | total_timesteps 251.
Path 27 | total_timesteps 260.
Path 28 | total_timesteps 271.
Path 29 | total_timesteps 283.
Path 30 | total_timesteps 296.
Path 31 | total_timesteps 313.
Path 32 | total_timesteps 323.
Path 33 | total_timesteps 337.
Path 34 | total_timesteps 348.
Path 35 | total_timesteps 359.
Path 36 | total_timesteps 382.
Path 37 | total_timesteps 397.
Path 38 | total_timesteps 413.
Path 39 | total_timesteps 427.
Path 40 | total_timesteps 439.
Path 41 | total_timesteps 448.
Path 42 | total_timesteps 457.
Path 43 | total_timesteps 471.
Path 44 | total_timesteps 480.
Path 45 | total_timesteps 492.
Path 46 | total_timesteps 499.
Path 47 | total_timesteps 506.
Path 48 | total_timesteps 515.
Path 49 | total_timesteps 525.
Path 50 | total_timesteps 541.
Path 51 | total_timesteps 552.
Path 52 | total_timesteps 561.
Path 53 | total_timesteps 573.
Path 54 | total_timesteps 581.
Path 55 | total_timesteps 591.
Path 56 | total_timesteps 599.
Path 57 | total_timesteps 607.
Path 58 | total_timesteps 614.
Path 59 | total_timesteps 643.
Path 60 | total_timesteps 657.
Path 61 | total_timesteps 669.
Path 62 | total_timesteps 680.
Path 63 | total_timesteps 693.
Path 64 | total_timesteps 701.
Path 65 | total_timesteps 708.
Path 66 | total_timesteps 716.
Path 67 | total_timesteps 727.
Path 68 | total_timesteps 735.
Path 69 | total_timesteps 744.
Path 70 | total_timesteps 759.
Path 71 | total_timesteps 771.
Path 72 | total_timesteps 795.
Path 73 | total_timesteps 803.
Path 74 | total_timesteps 816.
Path 75 | total_timesteps 824.
Path 76 | total_timesteps 833.
Path 77 | total_timesteps 843.
Path 78 | total_timesteps 851.
Path 79 | total_timesteps 861.
Path 80 | total_timesteps 873.
Path 81 | total_timesteps 882.
Path 82 | total_timesteps 892.
Path 83 | total_timesteps 902.
Path 84 | total_timesteps 910.
Path 85 | total_timesteps 919.
Path 86 | total_timesteps 936.
Path 87 | total_timesteps 944.
Path 88 | total_timesteps 952.
Path 89 | total_timesteps 968.
Path 90 | total_timesteps 978.
Path 91 | total_timesteps 997.
Path 92 | total_timesteps 1010.
Path 93 | total_timesteps 1026.
Path 94 | total_timesteps 1037.
Path 95 | total_timesteps 1048.
Path 96 | total_timesteps 1056.
Path 97 | total_timesteps 1074.
Path 98 | total_timesteps 1087.
Path 99 | total_timesteps 1094.
Path 100 | total_timesteps 1114.
Path 101 | total_timesteps 1126.
Path 102 | total_timesteps 1136.
Path 103 | total_timesteps 1153.
Path 104 | total_timesteps 1181.
Path 105 | total_timesteps 1198.
Path 106 | total_timesteps 1205.
Path 107 | total_timesteps 1216.
Path 108 | total_timesteps 1227.
Path 109 | total_timesteps 1238.
Path 110 | total_timesteps 1253.
Path 111 | total_timesteps 1263.
Path 112 | total_timesteps 1274.
Path 113 | total_timesteps 1283.
Path 114 | total_timesteps 1290.
Path 115 | total_timesteps 1300.
Path 116 | total_timesteps 1309.
Path 117 | total_timesteps 1332.
Path 118 | total_timesteps 1341.
Path 119 | total_timesteps 1350.
Path 120 | total_timesteps 1362.
Path 121 | total_timesteps 1373.
Path 122 | total_timesteps 1383.
Path 123 | total_timesteps 1395.
Path 124 | total_timesteps 1407.
Path 125 | total_timesteps 1417.
Path 126 | total_timesteps 1431.
Path 127 | total_timesteps 1443.
Path 128 | total_timesteps 1454.
Path 129 | total_timesteps 1475.
Path 130 | total_timesteps 1483.
Path 131 | total_timesteps 1495.
Path 132 | total_timesteps 1504.
Path 133 | total_timesteps 1515.
Path 134 | total_timesteps 1523.
Path 135 | total_timesteps 1535.
Path 136 | total_timesteps 1543.
Path 137 | total_timesteps 1553.
Path 138 | total_timesteps 1565.
Path 139 | total_timesteps 1572.
Path 140 | total_timesteps 1584.
Path 141 | total_timesteps 1604.
Path 142 | total_timesteps 1613.
Path 143 | total_timesteps 1625.
Path 144 | total_timesteps 1636.
Path 145 | total_timesteps 1645.
Path 146 | total_timesteps 1655.
Path 147 | total_timesteps 1680.
Path 148 | total_timesteps 1696.
Path 149 | total_timesteps 1705.
Path 150 | total_timesteps 1716.
Path 151 | total_timesteps 1730.
Path 152 | total_timesteps 1740.
Path 153 | total_timesteps 1750.
Path 154 | total_timesteps 1758.
Path 155 | total_timesteps 1770.
Path 156 | total_timesteps 1784.
Path 157 | total_timesteps 1793.
Path 158 | total_timesteps 1804.
Path 159 | total_timesteps 1816.
Path 160 | total_timesteps 1840.
Path 161 | total_timesteps 1848.
Path 162 | total_timesteps 1859.
Path 163 | total_timesteps 1868.
Path 164 | total_timesteps 1876.
Path 165 | total_timesteps 1895.
Path 166 | total_timesteps 1905.
Path 167 | total_timesteps 1915.
Path 168 | total_timesteps 1926.
Path 169 | total_timesteps 1934.
Path 170 | total_timesteps 1945.
Path 171 | total_timesteps 1953.
Path 172 | total_timesteps 1959.
Path 173 | total_timesteps 1968.
Path 174 | total_timesteps 1978.
Path 175 | total_timesteps 1989.
Path 176 | total_timesteps 2007.
Path 177 | total_timesteps 2013.
Path 178 | total_timesteps 2023.
Path 179 | total_timesteps 2039.
Path 180 | total_timesteps 2046.
Path 181 | total_timesteps 2055.
Path 182 | total_timesteps 2064.
Path 183 | total_timesteps 2074.
Path 184 | total_timesteps 2081.
Path 185 | total_timesteps 2090.
Path 186 | total_timesteps 2097.
Path 187 | total_timesteps 2108.
Path 188 | total_timesteps 2122.
Path 189 | total_timesteps 2128.
Path 190 | total_timesteps 2136.
Path 191 | total_timesteps 2155.
Path 192 | total_timesteps 2169.
Path 193 | total_timesteps 2182.
Path 194 | total_timesteps 2189.
Path 195 | total_timesteps 2200.
Path 196 | total_timesteps 2216.
Path 197 | total_timesteps 2225.
Path 198 | total_timesteps 2237.
Path 199 | total_timesteps 2248.
Path 200 | total_timesteps 2256.
Path 201 | total_timesteps 2265.
Path 202 | total_timesteps 2272.
Path 203 | total_timesteps 2285.
Path 204 | total_timesteps 2299.
Path 205 | total_timesteps 2307.
Path 206 | total_timesteps 2332.
Path 207 | total_timesteps 2350.
Path 208 | total_timesteps 2361.
Path 209 | total_timesteps 2368.
Path 210 | total_timesteps 2384.
Path 211 | total_timesteps 2392.
Path 212 | total_timesteps 2403.
Path 213 | total_timesteps 2417.
Path 214 | total_timesteps 2433.
Path 215 | total_timesteps 2443.
Path 216 | total_timesteps 2454.
Path 217 | total_timesteps 2461.
Path 218 | total_timesteps 2472.
Path 219 | total_timesteps 2482.
Path 220 | total_timesteps 2490.
Path 221 | total_timesteps 2497.
Path 222 | total_timesteps 2504.
Path 223 | total_timesteps 2511.
Path 224 | total_timesteps 2519.
Path 225 | total_timesteps 2540.
Path 226 | total_timesteps 2552.
Path 227 | total_timesteps 2560.
Path 228 | total_timesteps 2569.
Path 229 | total_timesteps 2582.
Path 230 | total_timesteps 2595.
Path 231 | total_timesteps 2604.
Path 232 | total_timesteps 2613.
Path 233 | total_timesteps 2628.
Path 234 | total_timesteps 2637.
Path 235 | total_timesteps 2644.
Path 236 | total_timesteps 2653.
Path 237 | total_timesteps 2661.
Path 238 | total_timesteps 2670.
Path 239 | total_timesteps 2679.
Path 240 | total_timesteps 2688.
Path 241 | total_timesteps 2701.
Path 242 | total_timesteps 2719.
Path 243 | total_timesteps 2728.
Path 244 | total_timesteps 2738.
Path 245 | total_timesteps 2748.
Path 246 | total_timesteps 2758.
Path 247 | total_timesteps 2765.
Path 248 | total_timesteps 2773.
Path 249 | total_timesteps 2781.
Path 250 | total_timesteps 2790.
Path 251 | total_timesteps 2797.
Path 252 | total_timesteps 2806.
Path 253 | total_timesteps 2828.
Path 254 | total_timesteps 2837.
Path 255 | total_timesteps 2845.
Path 256 | total_timesteps 2859.
Path 257 | total_timesteps 2874.
Path 258 | total_timesteps 2881.
Path 259 | total_timesteps 2891.
Path 260 | total_timesteps 2897.
Path 261 | total_timesteps 2908.
Path 262 | total_timesteps 2918.
Path 263 | total_timesteps 2927.
Path 264 | total_timesteps 2935.
Path 265 | total_timesteps 2948.
Path 266 | total_timesteps 2963.
Path 267 | total_timesteps 2973.
Path 268 | total_timesteps 2981.
Path 269 | total_timesteps 2990.
Path 270 | total_timesteps 2998.
Path 271 | total_timesteps 3011.
Path 272 | total_timesteps 3022.
Path 273 | total_timesteps 3035.
Path 274 | total_timesteps 3044.
Path 275 | total_timesteps 3055.
Path 276 | total_timesteps 3066.
Path 277 | total_timesteps 3077.
Path 278 | total_timesteps 3084.
Path 279 | total_timesteps 3094.
Path 280 | total_timesteps 3104.
Path 281 | total_timesteps 3112.
Path 282 | total_timesteps 3126.
Path 283 | total_timesteps 3138.
Path 284 | total_timesteps 3148.
Path 285 | total_timesteps 3158.
Path 286 | total_timesteps 3174.
Path 287 | total_timesteps 3182.
Path 288 | total_timesteps 3196.
Path 289 | total_timesteps 3212.
Path 290 | total_timesteps 3223.
Path 291 | total_timesteps 3232.
Path 292 | total_timesteps 3245.
Path 293 | total_timesteps 3256.
Path 294 | total_timesteps 3265.
Path 295 | total_timesteps 3276.
Path 296 | total_timesteps 3283.
Path 297 | total_timesteps 3296.
Path 298 | total_timesteps 3304.
Path 299 | total_timesteps 3316.
Path 300 | total_timesteps 3331.
Path 301 | total_timesteps 3343.
Path 302 | total_timesteps 3356.
Path 303 | total_timesteps 3372.
Path 304 | total_timesteps 3389.
Path 305 | total_timesteps 3396.
Path 306 | total_timesteps 3423.
Path 307 | total_timesteps 3434.
Path 308 | total_timesteps 3444.
Path 309 | total_timesteps 3459.
Path 310 | total_timesteps 3467.
Path 311 | total_timesteps 3479.
Path 312 | total_timesteps 3488.
Path 313 | total_timesteps 3508.
Path 314 | total_timesteps 3524.
Path 315 | total_timesteps 3530.
Path 316 | total_timesteps 3544.
Path 317 | total_timesteps 3551.
Path 318 | total_timesteps 3566.
Path 319 | total_timesteps 3578.
Path 320 | total_timesteps 3591.
Path 321 | total_timesteps 3600.
Path 322 | total_timesteps 3611.
Path 323 | total_timesteps 3623.
Path 324 | total_timesteps 3638.
Path 325 | total_timesteps 3646.
Path 326 | total_timesteps 3654.
Path 327 | total_timesteps 3662.
Path 328 | total_timesteps 3676.
Path 329 | total_timesteps 3691.
Path 330 | total_timesteps 3705.
Path 331 | total_timesteps 3713.
Path 332 | total_timesteps 3733.
Path 333 | total_timesteps 3744.
Path 334 | total_timesteps 3753.
Path 335 | total_timesteps 3765.
Path 336 | total_timesteps 3775.
Path 337 | total_timesteps 3789.
Path 338 | total_timesteps 3796.
Path 339 | total_timesteps 3809.
Path 340 | total_timesteps 3818.
Path 341 | total_timesteps 3833.
Path 342 | total_timesteps 3842.
Path 343 | total_timesteps 3857.
Path 344 | total_timesteps 3865.
Path 345 | total_timesteps 3875.
Path 346 | total_timesteps 3885.
Path 347 | total_timesteps 3895.
Path 348 | total_timesteps 3903.
Path 349 | total_timesteps 3913.
Path 350 | total_timesteps 3924.
Path 351 | total_timesteps 3933.
Path 352 | total_timesteps 3949.
Path 353 | total_timesteps 3957.
Path 354 | total_timesteps 3967.
Path 355 | total_timesteps 3975.
Path 356 | total_timesteps 3987.
Path 357 | total_timesteps 3997.
Path 358 | total_timesteps 4008.
Path 359 | total_timesteps 4016.
Path 360 | total_timesteps 4032.
Path 361 | total_timesteps 4048.
Path 362 | total_timesteps 4061.
Path 363 | total_timesteps 4071.
Path 364 | total_timesteps 4079.
Path 365 | total_timesteps 4091.
Path 366 | total_timesteps 4103.
Path 367 | total_timesteps 4111.
Path 368 | total_timesteps 4122.
Path 369 | total_timesteps 4133.
Path 370 | total_timesteps 4142.
Path 371 | total_timesteps 4153.
Path 372 | total_timesteps 4160.
Path 373 | total_timesteps 4168.
Path 374 | total_timesteps 4182.
Path 375 | total_timesteps 4191.
Path 376 | total_timesteps 4200.
Path 377 | total_timesteps 4210.
Path 378 | total_timesteps 4221.
Path 379 | total_timesteps 4231.
Path 380 | total_timesteps 4239.
Path 381 | total_timesteps 4247.
Path 382 | total_timesteps 4262.
Path 383 | total_timesteps 4272.
Path 384 | total_timesteps 4281.
Path 385 | total_timesteps 4300.
Path 386 | total_timesteps 4308.
Path 387 | total_timesteps 4322.
Path 388 | total_timesteps 4337.
Path 389 | total_timesteps 4352.
Path 390 | total_timesteps 4363.
Path 391 | total_timesteps 4373.
Path 392 | total_timesteps 4386.
Path 393 | total_timesteps 4406.
Path 394 | total_timesteps 4414.
Path 395 | total_timesteps 4425.
Path 396 | total_timesteps 4434.
Path 397 | total_timesteps 4453.
Path 398 | total_timesteps 4461.
Path 399 | total_timesteps 4470.
Path 400 | total_timesteps 4480.
Path 401 | total_timesteps 4488.
Path 402 | total_timesteps 4495.
Path 403 | total_timesteps 4504.
Path 404 | total_timesteps 4522.
Path 405 | total_timesteps 4531.
Path 406 | total_timesteps 4541.
Path 407 | total_timesteps 4552.
Path 408 | total_timesteps 4564.
Path 409 | total_timesteps 4577.
Path 410 | total_timesteps 4588.
Path 411 | total_timesteps 4599.
Path 412 | total_timesteps 4609.
Path 413 | total_timesteps 4617.
Path 414 | total_timesteps 4626.
Path 415 | total_timesteps 4639.
Path 416 | total_timesteps 4646.
Path 417 | total_timesteps 4655.
Path 418 | total_timesteps 4665.
Path 419 | total_timesteps 4672.
Path 420 | total_timesteps 4682.
Path 421 | total_timesteps 4694.
Path 422 | total_timesteps 4703.
Path 423 | total_timesteps 4715.
Path 424 | total_timesteps 4726.
Path 425 | total_timesteps 4744.
Path 426 | total_timesteps 4752.
Path 427 | total_timesteps 4765.
Path 428 | total_timesteps 4777.
Path 429 | total_timesteps 4789.
Path 430 | total_timesteps 4800.
Path 431 | total_timesteps 4811.
Path 432 | total_timesteps 4826.
Path 433 | total_timesteps 4842.
Path 434 | total_timesteps 4850.
Path 435 | total_timesteps 4878.
Path 436 | total_timesteps 4889.
Path 437 | total_timesteps 4904.
Path 438 | total_timesteps 4916.
Path 439 | total_timesteps 4929.
Path 440 | total_timesteps 4940.
Path 441 | total_timesteps 4952.
Path 442 | total_timesteps 4962.
Path 443 | total_timesteps 4973.
Path 444 | total_timesteps 4994.
Path 445 | total_timesteps 5012.
Path 446 | total_timesteps 5026.
Path 447 | total_timesteps 5038.
Path 448 | total_timesteps 5050.
Path 449 | total_timesteps 5062.
Path 450 | total_timesteps 5071.
Path 451 | total_timesteps 5078.
Path 452 | total_timesteps 5097.
Path 453 | total_timesteps 5107.
Path 454 | total_timesteps 5118.
Path 455 | total_timesteps 5130.
Path 456 | total_timesteps 5152.
Path 457 | total_timesteps 5163.
Path 458 | total_timesteps 5175.
Path 459 | total_timesteps 5183.
Path 460 | total_timesteps 5190.
Path 461 | total_timesteps 5201.
Path 462 | total_timesteps 5213.
Path 463 | total_timesteps 5221.
Path 464 | total_timesteps 5238.
Path 465 | total_timesteps 5247.
Path 466 | total_timesteps 5253.
Path 467 | total_timesteps 5268.
Path 468 | total_timesteps 5280.
Path 469 | total_timesteps 5289.
Path 470 | total_timesteps 5302.
Path 471 | total_timesteps 5310.
Path 472 | total_timesteps 5323.
Path 473 | total_timesteps 5331.
Path 474 | total_timesteps 5338.
Path 475 | total_timesteps 5345.
Path 476 | total_timesteps 5360.
Path 477 | total_timesteps 5368.
Path 478 | total_timesteps 5380.
Path 479 | total_timesteps 5389.
Path 480 | total_timesteps 5397.
Path 481 | total_timesteps 5408.
Path 482 | total_timesteps 5419.
Path 483 | total_timesteps 5429.
Path 484 | total_timesteps 5438.
Path 485 | total_timesteps 5446.
Path 486 | total_timesteps 5454.
Path 487 | total_timesteps 5462.
Path 488 | total_timesteps 5474.
Path 489 | total_timesteps 5486.
Path 490 | total_timesteps 5495.
Path 491 | total_timesteps 5505.
Path 492 | total_timesteps 5517.
Path 493 | total_timesteps 5526.
Path 494 | total_timesteps 5534.
Path 495 | total_timesteps 5543.
Path 496 | total_timesteps 5552.
Path 497 | total_timesteps 5564.
Path 498 | total_timesteps 5580.
Path 499 | total_timesteps 5589.
Path 500 | total_timesteps 5599.
Path 501 | total_timesteps 5608.
Path 502 | total_timesteps 5626.
Path 503 | total_timesteps 5643.
Path 504 | total_timesteps 5653.
Path 505 | total_timesteps 5664.
Path 506 | total_timesteps 5671.
Path 507 | total_timesteps 5679.
Path 508 | total_timesteps 5693.
Path 509 | total_timesteps 5701.
Path 510 | total_timesteps 5707.
Path 511 | total_timesteps 5721.
Path 512 | total_timesteps 5729.
Path 513 | total_timesteps 5737.
Path 514 | total_timesteps 5746.
Path 515 | total_timesteps 5752.
Path 516 | total_timesteps 5766.
Path 517 | total_timesteps 5772.
Path 518 | total_timesteps 5783.
Path 519 | total_timesteps 5800.
Path 520 | total_timesteps 5808.
Path 521 | total_timesteps 5823.
Path 522 | total_timesteps 5833.
Path 523 | total_timesteps 5842.
Path 524 | total_timesteps 5849.
Path 525 | total_timesteps 5864.
Path 526 | total_timesteps 5878.
Path 527 | total_timesteps 5886.
Path 528 | total_timesteps 5897.
Path 529 | total_timesteps 5907.
Path 530 | total_timesteps 5920.
Path 531 | total_timesteps 5938.
Path 532 | total_timesteps 5946.
Path 533 | total_timesteps 5956.
Path 534 | total_timesteps 5964.
Path 535 | total_timesteps 5982.
Path 536 | total_timesteps 5989.
Path 537 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.02    |
| Iteration     | 25       |
| MaximumReturn | -2.3     |
| MinimumReturn | -23.4    |
| TotalSamples  | 108095   |
----------------------------
itr #26 | 
Fitting dynamics.
Validation loss = 0.005326357204467058
Validation loss = 0.004665132611989975
Validation loss = 0.004880834370851517
Validation loss = 0.004574494436383247
Validation loss = 0.004610932432115078
Validation loss = 0.004528671037405729
Validation loss = 0.004525193944573402
Validation loss = 0.0047689625062048435
Validation loss = 0.004823300521820784
Validation loss = 0.004743022844195366
Validation loss = 0.004708767868578434
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 25.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 49.
Path 6 | total_timesteps 58.
Path 7 | total_timesteps 70.
Path 8 | total_timesteps 86.
Path 9 | total_timesteps 96.
Path 10 | total_timesteps 105.
Path 11 | total_timesteps 115.
Path 12 | total_timesteps 134.
Path 13 | total_timesteps 145.
Path 14 | total_timesteps 153.
Path 15 | total_timesteps 160.
Path 16 | total_timesteps 172.
Path 17 | total_timesteps 180.
Path 18 | total_timesteps 189.
Path 19 | total_timesteps 204.
Path 20 | total_timesteps 213.
Path 21 | total_timesteps 229.
Path 22 | total_timesteps 239.
Path 23 | total_timesteps 249.
Path 24 | total_timesteps 260.
Path 25 | total_timesteps 270.
Path 26 | total_timesteps 278.
Path 27 | total_timesteps 285.
Path 28 | total_timesteps 291.
Path 29 | total_timesteps 318.
Path 30 | total_timesteps 326.
Path 31 | total_timesteps 343.
Path 32 | total_timesteps 354.
Path 33 | total_timesteps 361.
Path 34 | total_timesteps 370.
Path 35 | total_timesteps 379.
Path 36 | total_timesteps 387.
Path 37 | total_timesteps 395.
Path 38 | total_timesteps 411.
Path 39 | total_timesteps 422.
Path 40 | total_timesteps 435.
Path 41 | total_timesteps 447.
Path 42 | total_timesteps 464.
Path 43 | total_timesteps 477.
Path 44 | total_timesteps 486.
Path 45 | total_timesteps 495.
Path 46 | total_timesteps 502.
Path 47 | total_timesteps 512.
Path 48 | total_timesteps 521.
Path 49 | total_timesteps 528.
Path 50 | total_timesteps 539.
Path 51 | total_timesteps 546.
Path 52 | total_timesteps 560.
Path 53 | total_timesteps 570.
Path 54 | total_timesteps 585.
Path 55 | total_timesteps 595.
Path 56 | total_timesteps 605.
Path 57 | total_timesteps 620.
Path 58 | total_timesteps 629.
Path 59 | total_timesteps 641.
Path 60 | total_timesteps 649.
Path 61 | total_timesteps 660.
Path 62 | total_timesteps 676.
Path 63 | total_timesteps 698.
Path 64 | total_timesteps 708.
Path 65 | total_timesteps 717.
Path 66 | total_timesteps 724.
Path 67 | total_timesteps 734.
Path 68 | total_timesteps 740.
Path 69 | total_timesteps 750.
Path 70 | total_timesteps 760.
Path 71 | total_timesteps 770.
Path 72 | total_timesteps 778.
Path 73 | total_timesteps 786.
Path 74 | total_timesteps 794.
Path 75 | total_timesteps 804.
Path 76 | total_timesteps 811.
Path 77 | total_timesteps 822.
Path 78 | total_timesteps 831.
Path 79 | total_timesteps 839.
Path 80 | total_timesteps 847.
Path 81 | total_timesteps 855.
Path 82 | total_timesteps 862.
Path 83 | total_timesteps 873.
Path 84 | total_timesteps 879.
Path 85 | total_timesteps 888.
Path 86 | total_timesteps 899.
Path 87 | total_timesteps 908.
Path 88 | total_timesteps 918.
Path 89 | total_timesteps 926.
Path 90 | total_timesteps 933.
Path 91 | total_timesteps 944.
Path 92 | total_timesteps 956.
Path 93 | total_timesteps 970.
Path 94 | total_timesteps 978.
Path 95 | total_timesteps 996.
Path 96 | total_timesteps 1005.
Path 97 | total_timesteps 1012.
Path 98 | total_timesteps 1024.
Path 99 | total_timesteps 1031.
Path 100 | total_timesteps 1042.
Path 101 | total_timesteps 1054.
Path 102 | total_timesteps 1077.
Path 103 | total_timesteps 1088.
Path 104 | total_timesteps 1099.
Path 105 | total_timesteps 1126.
Path 106 | total_timesteps 1135.
Path 107 | total_timesteps 1146.
Path 108 | total_timesteps 1160.
Path 109 | total_timesteps 1172.
Path 110 | total_timesteps 1185.
Path 111 | total_timesteps 1193.
Path 112 | total_timesteps 1202.
Path 113 | total_timesteps 1215.
Path 114 | total_timesteps 1231.
Path 115 | total_timesteps 1241.
Path 116 | total_timesteps 1250.
Path 117 | total_timesteps 1258.
Path 118 | total_timesteps 1271.
Path 119 | total_timesteps 1283.
Path 120 | total_timesteps 1295.
Path 121 | total_timesteps 1305.
Path 122 | total_timesteps 1316.
Path 123 | total_timesteps 1325.
Path 124 | total_timesteps 1350.
Path 125 | total_timesteps 1368.
Path 126 | total_timesteps 1377.
Path 127 | total_timesteps 1388.
Path 128 | total_timesteps 1402.
Path 129 | total_timesteps 1415.
Path 130 | total_timesteps 1426.
Path 131 | total_timesteps 1438.
Path 132 | total_timesteps 1451.
Path 133 | total_timesteps 1458.
Path 134 | total_timesteps 1466.
Path 135 | total_timesteps 1478.
Path 136 | total_timesteps 1489.
Path 137 | total_timesteps 1500.
Path 138 | total_timesteps 1512.
Path 139 | total_timesteps 1520.
Path 140 | total_timesteps 1540.
Path 141 | total_timesteps 1548.
Path 142 | total_timesteps 1559.
Path 143 | total_timesteps 1572.
Path 144 | total_timesteps 1582.
Path 145 | total_timesteps 1591.
Path 146 | total_timesteps 1601.
Path 147 | total_timesteps 1609.
Path 148 | total_timesteps 1618.
Path 149 | total_timesteps 1625.
Path 150 | total_timesteps 1636.
Path 151 | total_timesteps 1645.
Path 152 | total_timesteps 1652.
Path 153 | total_timesteps 1666.
Path 154 | total_timesteps 1676.
Path 155 | total_timesteps 1691.
Path 156 | total_timesteps 1701.
Path 157 | total_timesteps 1708.
Path 158 | total_timesteps 1726.
Path 159 | total_timesteps 1735.
Path 160 | total_timesteps 1749.
Path 161 | total_timesteps 1756.
Path 162 | total_timesteps 1763.
Path 163 | total_timesteps 1774.
Path 164 | total_timesteps 1784.
Path 165 | total_timesteps 1794.
Path 166 | total_timesteps 1805.
Path 167 | total_timesteps 1817.
Path 168 | total_timesteps 1826.
Path 169 | total_timesteps 1843.
Path 170 | total_timesteps 1852.
Path 171 | total_timesteps 1859.
Path 172 | total_timesteps 1877.
Path 173 | total_timesteps 1891.
Path 174 | total_timesteps 1905.
Path 175 | total_timesteps 1914.
Path 176 | total_timesteps 1925.
Path 177 | total_timesteps 1932.
Path 178 | total_timesteps 1943.
Path 179 | total_timesteps 1954.
Path 180 | total_timesteps 1966.
Path 181 | total_timesteps 1980.
Path 182 | total_timesteps 1990.
Path 183 | total_timesteps 2001.
Path 184 | total_timesteps 2020.
Path 185 | total_timesteps 2029.
Path 186 | total_timesteps 2039.
Path 187 | total_timesteps 2051.
Path 188 | total_timesteps 2060.
Path 189 | total_timesteps 2077.
Path 190 | total_timesteps 2087.
Path 191 | total_timesteps 2101.
Path 192 | total_timesteps 2110.
Path 193 | total_timesteps 2120.
Path 194 | total_timesteps 2127.
Path 195 | total_timesteps 2136.
Path 196 | total_timesteps 2142.
Path 197 | total_timesteps 2154.
Path 198 | total_timesteps 2163.
Path 199 | total_timesteps 2172.
Path 200 | total_timesteps 2183.
Path 201 | total_timesteps 2194.
Path 202 | total_timesteps 2216.
Path 203 | total_timesteps 2224.
Path 204 | total_timesteps 2242.
Path 205 | total_timesteps 2257.
Path 206 | total_timesteps 2272.
Path 207 | total_timesteps 2281.
Path 208 | total_timesteps 2296.
Path 209 | total_timesteps 2304.
Path 210 | total_timesteps 2315.
Path 211 | total_timesteps 2331.
Path 212 | total_timesteps 2338.
Path 213 | total_timesteps 2358.
Path 214 | total_timesteps 2368.
Path 215 | total_timesteps 2382.
Path 216 | total_timesteps 2390.
Path 217 | total_timesteps 2399.
Path 218 | total_timesteps 2408.
Path 219 | total_timesteps 2423.
Path 220 | total_timesteps 2431.
Path 221 | total_timesteps 2438.
Path 222 | total_timesteps 2451.
Path 223 | total_timesteps 2464.
Path 224 | total_timesteps 2475.
Path 225 | total_timesteps 2484.
Path 226 | total_timesteps 2491.
Path 227 | total_timesteps 2503.
Path 228 | total_timesteps 2512.
Path 229 | total_timesteps 2522.
Path 230 | total_timesteps 2531.
Path 231 | total_timesteps 2544.
Path 232 | total_timesteps 2554.
Path 233 | total_timesteps 2564.
Path 234 | total_timesteps 2577.
Path 235 | total_timesteps 2589.
Path 236 | total_timesteps 2598.
Path 237 | total_timesteps 2605.
Path 238 | total_timesteps 2614.
Path 239 | total_timesteps 2620.
Path 240 | total_timesteps 2628.
Path 241 | total_timesteps 2640.
Path 242 | total_timesteps 2650.
Path 243 | total_timesteps 2658.
Path 244 | total_timesteps 2668.
Path 245 | total_timesteps 2679.
Path 246 | total_timesteps 2701.
Path 247 | total_timesteps 2707.
Path 248 | total_timesteps 2719.
Path 249 | total_timesteps 2729.
Path 250 | total_timesteps 2740.
Path 251 | total_timesteps 2746.
Path 252 | total_timesteps 2756.
Path 253 | total_timesteps 2769.
Path 254 | total_timesteps 2775.
Path 255 | total_timesteps 2783.
Path 256 | total_timesteps 2792.
Path 257 | total_timesteps 2803.
Path 258 | total_timesteps 2814.
Path 259 | total_timesteps 2831.
Path 260 | total_timesteps 2838.
Path 261 | total_timesteps 2847.
Path 262 | total_timesteps 2862.
Path 263 | total_timesteps 2871.
Path 264 | total_timesteps 2881.
Path 265 | total_timesteps 2899.
Path 266 | total_timesteps 2909.
Path 267 | total_timesteps 2920.
Path 268 | total_timesteps 2932.
Path 269 | total_timesteps 2942.
Path 270 | total_timesteps 2949.
Path 271 | total_timesteps 2955.
Path 272 | total_timesteps 2962.
Path 273 | total_timesteps 2980.
Path 274 | total_timesteps 2989.
Path 275 | total_timesteps 3002.
Path 276 | total_timesteps 3011.
Path 277 | total_timesteps 3021.
Path 278 | total_timesteps 3030.
Path 279 | total_timesteps 3045.
Path 280 | total_timesteps 3052.
Path 281 | total_timesteps 3060.
Path 282 | total_timesteps 3074.
Path 283 | total_timesteps 3081.
Path 284 | total_timesteps 3091.
Path 285 | total_timesteps 3098.
Path 286 | total_timesteps 3115.
Path 287 | total_timesteps 3125.
Path 288 | total_timesteps 3136.
Path 289 | total_timesteps 3144.
Path 290 | total_timesteps 3158.
Path 291 | total_timesteps 3168.
Path 292 | total_timesteps 3177.
Path 293 | total_timesteps 3187.
Path 294 | total_timesteps 3195.
Path 295 | total_timesteps 3206.
Path 296 | total_timesteps 3213.
Path 297 | total_timesteps 3224.
Path 298 | total_timesteps 3233.
Path 299 | total_timesteps 3243.
Path 300 | total_timesteps 3253.
Path 301 | total_timesteps 3260.
Path 302 | total_timesteps 3271.
Path 303 | total_timesteps 3282.
Path 304 | total_timesteps 3289.
Path 305 | total_timesteps 3299.
Path 306 | total_timesteps 3308.
Path 307 | total_timesteps 3325.
Path 308 | total_timesteps 3334.
Path 309 | total_timesteps 3344.
Path 310 | total_timesteps 3365.
Path 311 | total_timesteps 3373.
Path 312 | total_timesteps 3382.
Path 313 | total_timesteps 3401.
Path 314 | total_timesteps 3412.
Path 315 | total_timesteps 3420.
Path 316 | total_timesteps 3432.
Path 317 | total_timesteps 3441.
Path 318 | total_timesteps 3457.
Path 319 | total_timesteps 3475.
Path 320 | total_timesteps 3485.
Path 321 | total_timesteps 3493.
Path 322 | total_timesteps 3504.
Path 323 | total_timesteps 3520.
Path 324 | total_timesteps 3527.
Path 325 | total_timesteps 3536.
Path 326 | total_timesteps 3546.
Path 327 | total_timesteps 3555.
Path 328 | total_timesteps 3575.
Path 329 | total_timesteps 3588.
Path 330 | total_timesteps 3599.
Path 331 | total_timesteps 3617.
Path 332 | total_timesteps 3624.
Path 333 | total_timesteps 3631.
Path 334 | total_timesteps 3638.
Path 335 | total_timesteps 3648.
Path 336 | total_timesteps 3664.
Path 337 | total_timesteps 3671.
Path 338 | total_timesteps 3680.
Path 339 | total_timesteps 3689.
Path 340 | total_timesteps 3700.
Path 341 | total_timesteps 3712.
Path 342 | total_timesteps 3730.
Path 343 | total_timesteps 3740.
Path 344 | total_timesteps 3746.
Path 345 | total_timesteps 3757.
Path 346 | total_timesteps 3772.
Path 347 | total_timesteps 3789.
Path 348 | total_timesteps 3797.
Path 349 | total_timesteps 3805.
Path 350 | total_timesteps 3817.
Path 351 | total_timesteps 3840.
Path 352 | total_timesteps 3865.
Path 353 | total_timesteps 3875.
Path 354 | total_timesteps 3887.
Path 355 | total_timesteps 3894.
Path 356 | total_timesteps 3902.
Path 357 | total_timesteps 3910.
Path 358 | total_timesteps 3919.
Path 359 | total_timesteps 3930.
Path 360 | total_timesteps 3943.
Path 361 | total_timesteps 3954.
Path 362 | total_timesteps 3969.
Path 363 | total_timesteps 3977.
Path 364 | total_timesteps 3992.
Path 365 | total_timesteps 4009.
Path 366 | total_timesteps 4016.
Path 367 | total_timesteps 4023.
Path 368 | total_timesteps 4044.
Path 369 | total_timesteps 4052.
Path 370 | total_timesteps 4063.
Path 371 | total_timesteps 4073.
Path 372 | total_timesteps 4081.
Path 373 | total_timesteps 4092.
Path 374 | total_timesteps 4101.
Path 375 | total_timesteps 4112.
Path 376 | total_timesteps 4124.
Path 377 | total_timesteps 4135.
Path 378 | total_timesteps 4158.
Path 379 | total_timesteps 4167.
Path 380 | total_timesteps 4182.
Path 381 | total_timesteps 4189.
Path 382 | total_timesteps 4197.
Path 383 | total_timesteps 4204.
Path 384 | total_timesteps 4216.
Path 385 | total_timesteps 4225.
Path 386 | total_timesteps 4237.
Path 387 | total_timesteps 4245.
Path 388 | total_timesteps 4254.
Path 389 | total_timesteps 4260.
Path 390 | total_timesteps 4274.
Path 391 | total_timesteps 4284.
Path 392 | total_timesteps 4291.
Path 393 | total_timesteps 4312.
Path 394 | total_timesteps 4319.
Path 395 | total_timesteps 4328.
Path 396 | total_timesteps 4337.
Path 397 | total_timesteps 4346.
Path 398 | total_timesteps 4353.
Path 399 | total_timesteps 4361.
Path 400 | total_timesteps 4370.
Path 401 | total_timesteps 4382.
Path 402 | total_timesteps 4392.
Path 403 | total_timesteps 4409.
Path 404 | total_timesteps 4418.
Path 405 | total_timesteps 4435.
Path 406 | total_timesteps 4446.
Path 407 | total_timesteps 4467.
Path 408 | total_timesteps 4476.
Path 409 | total_timesteps 4488.
Path 410 | total_timesteps 4499.
Path 411 | total_timesteps 4509.
Path 412 | total_timesteps 4516.
Path 413 | total_timesteps 4524.
Path 414 | total_timesteps 4533.
Path 415 | total_timesteps 4540.
Path 416 | total_timesteps 4549.
Path 417 | total_timesteps 4556.
Path 418 | total_timesteps 4568.
Path 419 | total_timesteps 4586.
Path 420 | total_timesteps 4597.
Path 421 | total_timesteps 4604.
Path 422 | total_timesteps 4615.
Path 423 | total_timesteps 4624.
Path 424 | total_timesteps 4637.
Path 425 | total_timesteps 4645.
Path 426 | total_timesteps 4663.
Path 427 | total_timesteps 4672.
Path 428 | total_timesteps 4681.
Path 429 | total_timesteps 4692.
Path 430 | total_timesteps 4702.
Path 431 | total_timesteps 4720.
Path 432 | total_timesteps 4729.
Path 433 | total_timesteps 4739.
Path 434 | total_timesteps 4753.
Path 435 | total_timesteps 4762.
Path 436 | total_timesteps 4780.
Path 437 | total_timesteps 4789.
Path 438 | total_timesteps 4804.
Path 439 | total_timesteps 4811.
Path 440 | total_timesteps 4821.
Path 441 | total_timesteps 4827.
Path 442 | total_timesteps 4833.
Path 443 | total_timesteps 4843.
Path 444 | total_timesteps 4852.
Path 445 | total_timesteps 4865.
Path 446 | total_timesteps 4875.
Path 447 | total_timesteps 4883.
Path 448 | total_timesteps 4891.
Path 449 | total_timesteps 4899.
Path 450 | total_timesteps 4905.
Path 451 | total_timesteps 4914.
Path 452 | total_timesteps 4922.
Path 453 | total_timesteps 4933.
Path 454 | total_timesteps 4945.
Path 455 | total_timesteps 4959.
Path 456 | total_timesteps 4968.
Path 457 | total_timesteps 4979.
Path 458 | total_timesteps 5002.
Path 459 | total_timesteps 5018.
Path 460 | total_timesteps 5028.
Path 461 | total_timesteps 5037.
Path 462 | total_timesteps 5045.
Path 463 | total_timesteps 5053.
Path 464 | total_timesteps 5071.
Path 465 | total_timesteps 5081.
Path 466 | total_timesteps 5090.
Path 467 | total_timesteps 5107.
Path 468 | total_timesteps 5114.
Path 469 | total_timesteps 5123.
Path 470 | total_timesteps 5137.
Path 471 | total_timesteps 5143.
Path 472 | total_timesteps 5152.
Path 473 | total_timesteps 5163.
Path 474 | total_timesteps 5170.
Path 475 | total_timesteps 5178.
Path 476 | total_timesteps 5185.
Path 477 | total_timesteps 5197.
Path 478 | total_timesteps 5206.
Path 479 | total_timesteps 5213.
Path 480 | total_timesteps 5222.
Path 481 | total_timesteps 5232.
Path 482 | total_timesteps 5246.
Path 483 | total_timesteps 5256.
Path 484 | total_timesteps 5272.
Path 485 | total_timesteps 5281.
Path 486 | total_timesteps 5295.
Path 487 | total_timesteps 5304.
Path 488 | total_timesteps 5316.
Path 489 | total_timesteps 5326.
Path 490 | total_timesteps 5340.
Path 491 | total_timesteps 5353.
Path 492 | total_timesteps 5367.
Path 493 | total_timesteps 5381.
Path 494 | total_timesteps 5394.
Path 495 | total_timesteps 5403.
Path 496 | total_timesteps 5417.
Path 497 | total_timesteps 5424.
Path 498 | total_timesteps 5441.
Path 499 | total_timesteps 5450.
Path 500 | total_timesteps 5466.
Path 501 | total_timesteps 5475.
Path 502 | total_timesteps 5488.
Path 503 | total_timesteps 5496.
Path 504 | total_timesteps 5507.
Path 505 | total_timesteps 5516.
Path 506 | total_timesteps 5530.
Path 507 | total_timesteps 5545.
Path 508 | total_timesteps 5552.
Path 509 | total_timesteps 5563.
Path 510 | total_timesteps 5572.
Path 511 | total_timesteps 5584.
Path 512 | total_timesteps 5593.
Path 513 | total_timesteps 5604.
Path 514 | total_timesteps 5613.
Path 515 | total_timesteps 5622.
Path 516 | total_timesteps 5630.
Path 517 | total_timesteps 5640.
Path 518 | total_timesteps 5649.
Path 519 | total_timesteps 5664.
Path 520 | total_timesteps 5673.
Path 521 | total_timesteps 5688.
Path 522 | total_timesteps 5698.
Path 523 | total_timesteps 5709.
Path 524 | total_timesteps 5722.
Path 525 | total_timesteps 5732.
Path 526 | total_timesteps 5746.
Path 527 | total_timesteps 5757.
Path 528 | total_timesteps 5766.
Path 529 | total_timesteps 5773.
Path 530 | total_timesteps 5782.
Path 531 | total_timesteps 5791.
Path 532 | total_timesteps 5798.
Path 533 | total_timesteps 5810.
Path 534 | total_timesteps 5823.
Path 535 | total_timesteps 5834.
Path 536 | total_timesteps 5844.
Path 537 | total_timesteps 5853.
Path 538 | total_timesteps 5862.
Path 539 | total_timesteps 5872.
Path 540 | total_timesteps 5882.
Path 541 | total_timesteps 5896.
Path 542 | total_timesteps 5907.
Path 543 | total_timesteps 5917.
Path 544 | total_timesteps 5931.
Path 545 | total_timesteps 5939.
Path 546 | total_timesteps 5948.
Path 547 | total_timesteps 5956.
Path 548 | total_timesteps 5969.
Path 549 | total_timesteps 5979.
Path 550 | total_timesteps 5987.
Path 551 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.95    |
| Iteration     | 26       |
| MaximumReturn | 0.459    |
| MinimumReturn | -26.4    |
| TotalSamples  | 112108   |
----------------------------
itr #27 | 
Fitting dynamics.
Validation loss = 0.004758260678499937
Validation loss = 0.004623233340680599
Validation loss = 0.004479949828237295
Validation loss = 0.004521387163549662
Validation loss = 0.0045187692157924175
Validation loss = 0.004547496791929007
Validation loss = 0.004706482402980328
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 32.
Path 4 | total_timesteps 41.
Path 5 | total_timesteps 57.
Path 6 | total_timesteps 67.
Path 7 | total_timesteps 77.
Path 8 | total_timesteps 85.
Path 9 | total_timesteps 92.
Path 10 | total_timesteps 100.
Path 11 | total_timesteps 111.
Path 12 | total_timesteps 118.
Path 13 | total_timesteps 129.
Path 14 | total_timesteps 141.
Path 15 | total_timesteps 150.
Path 16 | total_timesteps 161.
Path 17 | total_timesteps 180.
Path 18 | total_timesteps 189.
Path 19 | total_timesteps 197.
Path 20 | total_timesteps 206.
Path 21 | total_timesteps 215.
Path 22 | total_timesteps 222.
Path 23 | total_timesteps 231.
Path 24 | total_timesteps 242.
Path 25 | total_timesteps 252.
Path 26 | total_timesteps 265.
Path 27 | total_timesteps 279.
Path 28 | total_timesteps 291.
Path 29 | total_timesteps 300.
Path 30 | total_timesteps 308.
Path 31 | total_timesteps 321.
Path 32 | total_timesteps 329.
Path 33 | total_timesteps 338.
Path 34 | total_timesteps 348.
Path 35 | total_timesteps 360.
Path 36 | total_timesteps 377.
Path 37 | total_timesteps 387.
Path 38 | total_timesteps 396.
Path 39 | total_timesteps 409.
Path 40 | total_timesteps 417.
Path 41 | total_timesteps 427.
Path 42 | total_timesteps 433.
Path 43 | total_timesteps 443.
Path 44 | total_timesteps 453.
Path 45 | total_timesteps 463.
Path 46 | total_timesteps 472.
Path 47 | total_timesteps 484.
Path 48 | total_timesteps 491.
Path 49 | total_timesteps 503.
Path 50 | total_timesteps 521.
Path 51 | total_timesteps 533.
Path 52 | total_timesteps 544.
Path 53 | total_timesteps 554.
Path 54 | total_timesteps 562.
Path 55 | total_timesteps 570.
Path 56 | total_timesteps 579.
Path 57 | total_timesteps 587.
Path 58 | total_timesteps 598.
Path 59 | total_timesteps 608.
Path 60 | total_timesteps 615.
Path 61 | total_timesteps 624.
Path 62 | total_timesteps 643.
Path 63 | total_timesteps 650.
Path 64 | total_timesteps 659.
Path 65 | total_timesteps 668.
Path 66 | total_timesteps 679.
Path 67 | total_timesteps 688.
Path 68 | total_timesteps 697.
Path 69 | total_timesteps 707.
Path 70 | total_timesteps 717.
Path 71 | total_timesteps 733.
Path 72 | total_timesteps 744.
Path 73 | total_timesteps 753.
Path 74 | total_timesteps 766.
Path 75 | total_timesteps 777.
Path 76 | total_timesteps 787.
Path 77 | total_timesteps 797.
Path 78 | total_timesteps 809.
Path 79 | total_timesteps 819.
Path 80 | total_timesteps 828.
Path 81 | total_timesteps 837.
Path 82 | total_timesteps 848.
Path 83 | total_timesteps 854.
Path 84 | total_timesteps 861.
Path 85 | total_timesteps 870.
Path 86 | total_timesteps 885.
Path 87 | total_timesteps 901.
Path 88 | total_timesteps 912.
Path 89 | total_timesteps 922.
Path 90 | total_timesteps 937.
Path 91 | total_timesteps 944.
Path 92 | total_timesteps 959.
Path 93 | total_timesteps 975.
Path 94 | total_timesteps 986.
Path 95 | total_timesteps 993.
Path 96 | total_timesteps 1001.
Path 97 | total_timesteps 1009.
Path 98 | total_timesteps 1016.
Path 99 | total_timesteps 1024.
Path 100 | total_timesteps 1034.
Path 101 | total_timesteps 1054.
Path 102 | total_timesteps 1065.
Path 103 | total_timesteps 1075.
Path 104 | total_timesteps 1082.
Path 105 | total_timesteps 1089.
Path 106 | total_timesteps 1096.
Path 107 | total_timesteps 1104.
Path 108 | total_timesteps 1113.
Path 109 | total_timesteps 1119.
Path 110 | total_timesteps 1130.
Path 111 | total_timesteps 1144.
Path 112 | total_timesteps 1156.
Path 113 | total_timesteps 1167.
Path 114 | total_timesteps 1174.
Path 115 | total_timesteps 1181.
Path 116 | total_timesteps 1189.
Path 117 | total_timesteps 1197.
Path 118 | total_timesteps 1214.
Path 119 | total_timesteps 1223.
Path 120 | total_timesteps 1231.
Path 121 | total_timesteps 1255.
Path 122 | total_timesteps 1262.
Path 123 | total_timesteps 1270.
Path 124 | total_timesteps 1280.
Path 125 | total_timesteps 1290.
Path 126 | total_timesteps 1297.
Path 127 | total_timesteps 1308.
Path 128 | total_timesteps 1315.
Path 129 | total_timesteps 1326.
Path 130 | total_timesteps 1343.
Path 131 | total_timesteps 1355.
Path 132 | total_timesteps 1367.
Path 133 | total_timesteps 1377.
Path 134 | total_timesteps 1387.
Path 135 | total_timesteps 1394.
Path 136 | total_timesteps 1409.
Path 137 | total_timesteps 1427.
Path 138 | total_timesteps 1436.
Path 139 | total_timesteps 1445.
Path 140 | total_timesteps 1454.
Path 141 | total_timesteps 1460.
Path 142 | total_timesteps 1468.
Path 143 | total_timesteps 1479.
Path 144 | total_timesteps 1492.
Path 145 | total_timesteps 1502.
Path 146 | total_timesteps 1508.
Path 147 | total_timesteps 1519.
Path 148 | total_timesteps 1529.
Path 149 | total_timesteps 1542.
Path 150 | total_timesteps 1554.
Path 151 | total_timesteps 1563.
Path 152 | total_timesteps 1572.
Path 153 | total_timesteps 1582.
Path 154 | total_timesteps 1590.
Path 155 | total_timesteps 1600.
Path 156 | total_timesteps 1607.
Path 157 | total_timesteps 1614.
Path 158 | total_timesteps 1621.
Path 159 | total_timesteps 1627.
Path 160 | total_timesteps 1640.
Path 161 | total_timesteps 1658.
Path 162 | total_timesteps 1670.
Path 163 | total_timesteps 1677.
Path 164 | total_timesteps 1685.
Path 165 | total_timesteps 1694.
Path 166 | total_timesteps 1701.
Path 167 | total_timesteps 1709.
Path 168 | total_timesteps 1717.
Path 169 | total_timesteps 1723.
Path 170 | total_timesteps 1731.
Path 171 | total_timesteps 1742.
Path 172 | total_timesteps 1752.
Path 173 | total_timesteps 1763.
Path 174 | total_timesteps 1770.
Path 175 | total_timesteps 1777.
Path 176 | total_timesteps 1792.
Path 177 | total_timesteps 1801.
Path 178 | total_timesteps 1808.
Path 179 | total_timesteps 1818.
Path 180 | total_timesteps 1825.
Path 181 | total_timesteps 1839.
Path 182 | total_timesteps 1848.
Path 183 | total_timesteps 1863.
Path 184 | total_timesteps 1875.
Path 185 | total_timesteps 1890.
Path 186 | total_timesteps 1903.
Path 187 | total_timesteps 1915.
Path 188 | total_timesteps 1927.
Path 189 | total_timesteps 1936.
Path 190 | total_timesteps 1949.
Path 191 | total_timesteps 1956.
Path 192 | total_timesteps 1963.
Path 193 | total_timesteps 1975.
Path 194 | total_timesteps 1992.
Path 195 | total_timesteps 2002.
Path 196 | total_timesteps 2008.
Path 197 | total_timesteps 2024.
Path 198 | total_timesteps 2030.
Path 199 | total_timesteps 2042.
Path 200 | total_timesteps 2050.
Path 201 | total_timesteps 2057.
Path 202 | total_timesteps 2068.
Path 203 | total_timesteps 2076.
Path 204 | total_timesteps 2084.
Path 205 | total_timesteps 2090.
Path 206 | total_timesteps 2098.
Path 207 | total_timesteps 2109.
Path 208 | total_timesteps 2118.
Path 209 | total_timesteps 2127.
Path 210 | total_timesteps 2135.
Path 211 | total_timesteps 2143.
Path 212 | total_timesteps 2151.
Path 213 | total_timesteps 2158.
Path 214 | total_timesteps 2165.
Path 215 | total_timesteps 2174.
Path 216 | total_timesteps 2183.
Path 217 | total_timesteps 2192.
Path 218 | total_timesteps 2200.
Path 219 | total_timesteps 2210.
Path 220 | total_timesteps 2219.
Path 221 | total_timesteps 2235.
Path 222 | total_timesteps 2248.
Path 223 | total_timesteps 2257.
Path 224 | total_timesteps 2269.
Path 225 | total_timesteps 2278.
Path 226 | total_timesteps 2287.
Path 227 | total_timesteps 2297.
Path 228 | total_timesteps 2312.
Path 229 | total_timesteps 2320.
Path 230 | total_timesteps 2340.
Path 231 | total_timesteps 2358.
Path 232 | total_timesteps 2371.
Path 233 | total_timesteps 2379.
Path 234 | total_timesteps 2386.
Path 235 | total_timesteps 2394.
Path 236 | total_timesteps 2405.
Path 237 | total_timesteps 2420.
Path 238 | total_timesteps 2427.
Path 239 | total_timesteps 2436.
Path 240 | total_timesteps 2444.
Path 241 | total_timesteps 2457.
Path 242 | total_timesteps 2466.
Path 243 | total_timesteps 2476.
Path 244 | total_timesteps 2484.
Path 245 | total_timesteps 2493.
Path 246 | total_timesteps 2506.
Path 247 | total_timesteps 2516.
Path 248 | total_timesteps 2524.
Path 249 | total_timesteps 2533.
Path 250 | total_timesteps 2540.
Path 251 | total_timesteps 2552.
Path 252 | total_timesteps 2564.
Path 253 | total_timesteps 2574.
Path 254 | total_timesteps 2581.
Path 255 | total_timesteps 2591.
Path 256 | total_timesteps 2599.
Path 257 | total_timesteps 2609.
Path 258 | total_timesteps 2621.
Path 259 | total_timesteps 2633.
Path 260 | total_timesteps 2646.
Path 261 | total_timesteps 2656.
Path 262 | total_timesteps 2665.
Path 263 | total_timesteps 2680.
Path 264 | total_timesteps 2688.
Path 265 | total_timesteps 2698.
Path 266 | total_timesteps 2706.
Path 267 | total_timesteps 2721.
Path 268 | total_timesteps 2733.
Path 269 | total_timesteps 2747.
Path 270 | total_timesteps 2758.
Path 271 | total_timesteps 2769.
Path 272 | total_timesteps 2776.
Path 273 | total_timesteps 2786.
Path 274 | total_timesteps 2796.
Path 275 | total_timesteps 2803.
Path 276 | total_timesteps 2815.
Path 277 | total_timesteps 2824.
Path 278 | total_timesteps 2831.
Path 279 | total_timesteps 2838.
Path 280 | total_timesteps 2849.
Path 281 | total_timesteps 2862.
Path 282 | total_timesteps 2879.
Path 283 | total_timesteps 2886.
Path 284 | total_timesteps 2897.
Path 285 | total_timesteps 2904.
Path 286 | total_timesteps 2917.
Path 287 | total_timesteps 2928.
Path 288 | total_timesteps 2938.
Path 289 | total_timesteps 2944.
Path 290 | total_timesteps 2961.
Path 291 | total_timesteps 2969.
Path 292 | total_timesteps 2979.
Path 293 | total_timesteps 2990.
Path 294 | total_timesteps 2996.
Path 295 | total_timesteps 3005.
Path 296 | total_timesteps 3013.
Path 297 | total_timesteps 3023.
Path 298 | total_timesteps 3051.
Path 299 | total_timesteps 3062.
Path 300 | total_timesteps 3073.
Path 301 | total_timesteps 3082.
Path 302 | total_timesteps 3092.
Path 303 | total_timesteps 3101.
Path 304 | total_timesteps 3109.
Path 305 | total_timesteps 3116.
Path 306 | total_timesteps 3127.
Path 307 | total_timesteps 3137.
Path 308 | total_timesteps 3143.
Path 309 | total_timesteps 3153.
Path 310 | total_timesteps 3160.
Path 311 | total_timesteps 3168.
Path 312 | total_timesteps 3181.
Path 313 | total_timesteps 3190.
Path 314 | total_timesteps 3197.
Path 315 | total_timesteps 3205.
Path 316 | total_timesteps 3211.
Path 317 | total_timesteps 3223.
Path 318 | total_timesteps 3237.
Path 319 | total_timesteps 3248.
Path 320 | total_timesteps 3257.
Path 321 | total_timesteps 3267.
Path 322 | total_timesteps 3274.
Path 323 | total_timesteps 3281.
Path 324 | total_timesteps 3290.
Path 325 | total_timesteps 3300.
Path 326 | total_timesteps 3306.
Path 327 | total_timesteps 3321.
Path 328 | total_timesteps 3329.
Path 329 | total_timesteps 3338.
Path 330 | total_timesteps 3349.
Path 331 | total_timesteps 3357.
Path 332 | total_timesteps 3363.
Path 333 | total_timesteps 3370.
Path 334 | total_timesteps 3381.
Path 335 | total_timesteps 3389.
Path 336 | total_timesteps 3399.
Path 337 | total_timesteps 3409.
Path 338 | total_timesteps 3417.
Path 339 | total_timesteps 3424.
Path 340 | total_timesteps 3435.
Path 341 | total_timesteps 3442.
Path 342 | total_timesteps 3452.
Path 343 | total_timesteps 3464.
Path 344 | total_timesteps 3474.
Path 345 | total_timesteps 3483.
Path 346 | total_timesteps 3491.
Path 347 | total_timesteps 3504.
Path 348 | total_timesteps 3523.
Path 349 | total_timesteps 3533.
Path 350 | total_timesteps 3546.
Path 351 | total_timesteps 3562.
Path 352 | total_timesteps 3574.
Path 353 | total_timesteps 3586.
Path 354 | total_timesteps 3594.
Path 355 | total_timesteps 3604.
Path 356 | total_timesteps 3621.
Path 357 | total_timesteps 3635.
Path 358 | total_timesteps 3647.
Path 359 | total_timesteps 3654.
Path 360 | total_timesteps 3662.
Path 361 | total_timesteps 3671.
Path 362 | total_timesteps 3682.
Path 363 | total_timesteps 3691.
Path 364 | total_timesteps 3701.
Path 365 | total_timesteps 3712.
Path 366 | total_timesteps 3721.
Path 367 | total_timesteps 3728.
Path 368 | total_timesteps 3742.
Path 369 | total_timesteps 3753.
Path 370 | total_timesteps 3763.
Path 371 | total_timesteps 3771.
Path 372 | total_timesteps 3779.
Path 373 | total_timesteps 3791.
Path 374 | total_timesteps 3802.
Path 375 | total_timesteps 3809.
Path 376 | total_timesteps 3818.
Path 377 | total_timesteps 3825.
Path 378 | total_timesteps 3834.
Path 379 | total_timesteps 3842.
Path 380 | total_timesteps 3853.
Path 381 | total_timesteps 3862.
Path 382 | total_timesteps 3870.
Path 383 | total_timesteps 3881.
Path 384 | total_timesteps 3890.
Path 385 | total_timesteps 3898.
Path 386 | total_timesteps 3910.
Path 387 | total_timesteps 3918.
Path 388 | total_timesteps 3925.
Path 389 | total_timesteps 3934.
Path 390 | total_timesteps 3943.
Path 391 | total_timesteps 3954.
Path 392 | total_timesteps 3965.
Path 393 | total_timesteps 3977.
Path 394 | total_timesteps 3984.
Path 395 | total_timesteps 3992.
Path 396 | total_timesteps 4001.
Path 397 | total_timesteps 4008.
Path 398 | total_timesteps 4016.
Path 399 | total_timesteps 4025.
Path 400 | total_timesteps 4033.
Path 401 | total_timesteps 4044.
Path 402 | total_timesteps 4052.
Path 403 | total_timesteps 4062.
Path 404 | total_timesteps 4069.
Path 405 | total_timesteps 4082.
Path 406 | total_timesteps 4090.
Path 407 | total_timesteps 4101.
Path 408 | total_timesteps 4115.
Path 409 | total_timesteps 4124.
Path 410 | total_timesteps 4134.
Path 411 | total_timesteps 4149.
Path 412 | total_timesteps 4156.
Path 413 | total_timesteps 4168.
Path 414 | total_timesteps 4176.
Path 415 | total_timesteps 4183.
Path 416 | total_timesteps 4197.
Path 417 | total_timesteps 4207.
Path 418 | total_timesteps 4216.
Path 419 | total_timesteps 4229.
Path 420 | total_timesteps 4236.
Path 421 | total_timesteps 4244.
Path 422 | total_timesteps 4258.
Path 423 | total_timesteps 4264.
Path 424 | total_timesteps 4271.
Path 425 | total_timesteps 4281.
Path 426 | total_timesteps 4293.
Path 427 | total_timesteps 4302.
Path 428 | total_timesteps 4309.
Path 429 | total_timesteps 4317.
Path 430 | total_timesteps 4324.
Path 431 | total_timesteps 4336.
Path 432 | total_timesteps 4354.
Path 433 | total_timesteps 4363.
Path 434 | total_timesteps 4371.
Path 435 | total_timesteps 4385.
Path 436 | total_timesteps 4396.
Path 437 | total_timesteps 4407.
Path 438 | total_timesteps 4421.
Path 439 | total_timesteps 4430.
Path 440 | total_timesteps 4442.
Path 441 | total_timesteps 4453.
Path 442 | total_timesteps 4460.
Path 443 | total_timesteps 4471.
Path 444 | total_timesteps 4480.
Path 445 | total_timesteps 4489.
Path 446 | total_timesteps 4502.
Path 447 | total_timesteps 4516.
Path 448 | total_timesteps 4528.
Path 449 | total_timesteps 4539.
Path 450 | total_timesteps 4549.
Path 451 | total_timesteps 4560.
Path 452 | total_timesteps 4569.
Path 453 | total_timesteps 4579.
Path 454 | total_timesteps 4591.
Path 455 | total_timesteps 4600.
Path 456 | total_timesteps 4607.
Path 457 | total_timesteps 4613.
Path 458 | total_timesteps 4620.
Path 459 | total_timesteps 4627.
Path 460 | total_timesteps 4639.
Path 461 | total_timesteps 4652.
Path 462 | total_timesteps 4658.
Path 463 | total_timesteps 4665.
Path 464 | total_timesteps 4673.
Path 465 | total_timesteps 4684.
Path 466 | total_timesteps 4695.
Path 467 | total_timesteps 4707.
Path 468 | total_timesteps 4722.
Path 469 | total_timesteps 4733.
Path 470 | total_timesteps 4741.
Path 471 | total_timesteps 4750.
Path 472 | total_timesteps 4758.
Path 473 | total_timesteps 4766.
Path 474 | total_timesteps 4775.
Path 475 | total_timesteps 4781.
Path 476 | total_timesteps 4793.
Path 477 | total_timesteps 4802.
Path 478 | total_timesteps 4817.
Path 479 | total_timesteps 4828.
Path 480 | total_timesteps 4835.
Path 481 | total_timesteps 4846.
Path 482 | total_timesteps 4863.
Path 483 | total_timesteps 4873.
Path 484 | total_timesteps 4882.
Path 485 | total_timesteps 4892.
Path 486 | total_timesteps 4904.
Path 487 | total_timesteps 4911.
Path 488 | total_timesteps 4919.
Path 489 | total_timesteps 4931.
Path 490 | total_timesteps 4939.
Path 491 | total_timesteps 4947.
Path 492 | total_timesteps 4957.
Path 493 | total_timesteps 4969.
Path 494 | total_timesteps 4980.
Path 495 | total_timesteps 4990.
Path 496 | total_timesteps 4997.
Path 497 | total_timesteps 5010.
Path 498 | total_timesteps 5019.
Path 499 | total_timesteps 5031.
Path 500 | total_timesteps 5040.
Path 501 | total_timesteps 5049.
Path 502 | total_timesteps 5056.
Path 503 | total_timesteps 5067.
Path 504 | total_timesteps 5076.
Path 505 | total_timesteps 5085.
Path 506 | total_timesteps 5095.
Path 507 | total_timesteps 5110.
Path 508 | total_timesteps 5128.
Path 509 | total_timesteps 5139.
Path 510 | total_timesteps 5150.
Path 511 | total_timesteps 5164.
Path 512 | total_timesteps 5172.
Path 513 | total_timesteps 5186.
Path 514 | total_timesteps 5194.
Path 515 | total_timesteps 5202.
Path 516 | total_timesteps 5211.
Path 517 | total_timesteps 5233.
Path 518 | total_timesteps 5241.
Path 519 | total_timesteps 5248.
Path 520 | total_timesteps 5255.
Path 521 | total_timesteps 5263.
Path 522 | total_timesteps 5272.
Path 523 | total_timesteps 5282.
Path 524 | total_timesteps 5289.
Path 525 | total_timesteps 5302.
Path 526 | total_timesteps 5320.
Path 527 | total_timesteps 5327.
Path 528 | total_timesteps 5338.
Path 529 | total_timesteps 5348.
Path 530 | total_timesteps 5361.
Path 531 | total_timesteps 5369.
Path 532 | total_timesteps 5380.
Path 533 | total_timesteps 5388.
Path 534 | total_timesteps 5396.
Path 535 | total_timesteps 5410.
Path 536 | total_timesteps 5420.
Path 537 | total_timesteps 5430.
Path 538 | total_timesteps 5437.
Path 539 | total_timesteps 5445.
Path 540 | total_timesteps 5457.
Path 541 | total_timesteps 5479.
Path 542 | total_timesteps 5488.
Path 543 | total_timesteps 5496.
Path 544 | total_timesteps 5503.
Path 545 | total_timesteps 5510.
Path 546 | total_timesteps 5524.
Path 547 | total_timesteps 5532.
Path 548 | total_timesteps 5550.
Path 549 | total_timesteps 5560.
Path 550 | total_timesteps 5571.
Path 551 | total_timesteps 5578.
Path 552 | total_timesteps 5591.
Path 553 | total_timesteps 5602.
Path 554 | total_timesteps 5611.
Path 555 | total_timesteps 5629.
Path 556 | total_timesteps 5644.
Path 557 | total_timesteps 5651.
Path 558 | total_timesteps 5661.
Path 559 | total_timesteps 5671.
Path 560 | total_timesteps 5683.
Path 561 | total_timesteps 5693.
Path 562 | total_timesteps 5701.
Path 563 | total_timesteps 5711.
Path 564 | total_timesteps 5722.
Path 565 | total_timesteps 5730.
Path 566 | total_timesteps 5738.
Path 567 | total_timesteps 5746.
Path 568 | total_timesteps 5754.
Path 569 | total_timesteps 5762.
Path 570 | total_timesteps 5778.
Path 571 | total_timesteps 5788.
Path 572 | total_timesteps 5795.
Path 573 | total_timesteps 5805.
Path 574 | total_timesteps 5815.
Path 575 | total_timesteps 5821.
Path 576 | total_timesteps 5829.
Path 577 | total_timesteps 5840.
Path 578 | total_timesteps 5857.
Path 579 | total_timesteps 5864.
Path 580 | total_timesteps 5875.
Path 581 | total_timesteps 5886.
Path 582 | total_timesteps 5900.
Path 583 | total_timesteps 5908.
Path 584 | total_timesteps 5920.
Path 585 | total_timesteps 5933.
Path 586 | total_timesteps 5941.
Path 587 | total_timesteps 5954.
Path 588 | total_timesteps 5965.
Path 589 | total_timesteps 5975.
Path 590 | total_timesteps 5985.
Path 591 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.55    |
| Iteration     | 27       |
| MaximumReturn | -0.436   |
| MinimumReturn | -23.2    |
| TotalSamples  | 116111   |
----------------------------
itr #28 | 
Fitting dynamics.
Validation loss = 0.0046152155846357346
Validation loss = 0.004531548358500004
Validation loss = 0.004870552569627762
Validation loss = 0.004677015822380781
Validation loss = 0.004671106580644846
Validation loss = 0.004441197030246258
Validation loss = 0.004483134485781193
Validation loss = 0.004596094135195017
Validation loss = 0.004371135961264372
Validation loss = 0.004520275164395571
Validation loss = 0.004693389870226383
Validation loss = 0.004344567656517029
Validation loss = 0.004778574220836163
Validation loss = 0.004331966862082481
Validation loss = 0.004520345013588667
Validation loss = 0.004364974331110716
Validation loss = 0.004391930066049099
Validation loss = 0.004206250421702862
Validation loss = 0.004535213112831116
Validation loss = 0.004407503176480532
Validation loss = 0.004329116083681583
Validation loss = 0.004278849810361862
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 35.
Path 4 | total_timesteps 44.
Path 5 | total_timesteps 52.
Path 6 | total_timesteps 62.
Path 7 | total_timesteps 71.
Path 8 | total_timesteps 85.
Path 9 | total_timesteps 94.
Path 10 | total_timesteps 106.
Path 11 | total_timesteps 120.
Path 12 | total_timesteps 127.
Path 13 | total_timesteps 135.
Path 14 | total_timesteps 152.
Path 15 | total_timesteps 160.
Path 16 | total_timesteps 167.
Path 17 | total_timesteps 174.
Path 18 | total_timesteps 187.
Path 19 | total_timesteps 196.
Path 20 | total_timesteps 208.
Path 21 | total_timesteps 217.
Path 22 | total_timesteps 224.
Path 23 | total_timesteps 231.
Path 24 | total_timesteps 242.
Path 25 | total_timesteps 254.
Path 26 | total_timesteps 263.
Path 27 | total_timesteps 271.
Path 28 | total_timesteps 278.
Path 29 | total_timesteps 290.
Path 30 | total_timesteps 297.
Path 31 | total_timesteps 304.
Path 32 | total_timesteps 319.
Path 33 | total_timesteps 329.
Path 34 | total_timesteps 347.
Path 35 | total_timesteps 361.
Path 36 | total_timesteps 371.
Path 37 | total_timesteps 378.
Path 38 | total_timesteps 390.
Path 39 | total_timesteps 405.
Path 40 | total_timesteps 414.
Path 41 | total_timesteps 423.
Path 42 | total_timesteps 430.
Path 43 | total_timesteps 445.
Path 44 | total_timesteps 453.
Path 45 | total_timesteps 462.
Path 46 | total_timesteps 471.
Path 47 | total_timesteps 483.
Path 48 | total_timesteps 495.
Path 49 | total_timesteps 505.
Path 50 | total_timesteps 515.
Path 51 | total_timesteps 525.
Path 52 | total_timesteps 534.
Path 53 | total_timesteps 551.
Path 54 | total_timesteps 567.
Path 55 | total_timesteps 574.
Path 56 | total_timesteps 581.
Path 57 | total_timesteps 589.
Path 58 | total_timesteps 600.
Path 59 | total_timesteps 609.
Path 60 | total_timesteps 619.
Path 61 | total_timesteps 633.
Path 62 | total_timesteps 668.
Path 63 | total_timesteps 674.
Path 64 | total_timesteps 684.
Path 65 | total_timesteps 697.
Path 66 | total_timesteps 707.
Path 67 | total_timesteps 714.
Path 68 | total_timesteps 722.
Path 69 | total_timesteps 734.
Path 70 | total_timesteps 744.
Path 71 | total_timesteps 751.
Path 72 | total_timesteps 760.
Path 73 | total_timesteps 767.
Path 74 | total_timesteps 773.
Path 75 | total_timesteps 782.
Path 76 | total_timesteps 790.
Path 77 | total_timesteps 798.
Path 78 | total_timesteps 806.
Path 79 | total_timesteps 814.
Path 80 | total_timesteps 825.
Path 81 | total_timesteps 840.
Path 82 | total_timesteps 853.
Path 83 | total_timesteps 860.
Path 84 | total_timesteps 874.
Path 85 | total_timesteps 886.
Path 86 | total_timesteps 898.
Path 87 | total_timesteps 905.
Path 88 | total_timesteps 912.
Path 89 | total_timesteps 919.
Path 90 | total_timesteps 930.
Path 91 | total_timesteps 938.
Path 92 | total_timesteps 950.
Path 93 | total_timesteps 963.
Path 94 | total_timesteps 971.
Path 95 | total_timesteps 978.
Path 96 | total_timesteps 989.
Path 97 | total_timesteps 996.
Path 98 | total_timesteps 1010.
Path 99 | total_timesteps 1019.
Path 100 | total_timesteps 1027.
Path 101 | total_timesteps 1035.
Path 102 | total_timesteps 1046.
Path 103 | total_timesteps 1058.
Path 104 | total_timesteps 1067.
Path 105 | total_timesteps 1074.
Path 106 | total_timesteps 1083.
Path 107 | total_timesteps 1095.
Path 108 | total_timesteps 1103.
Path 109 | total_timesteps 1113.
Path 110 | total_timesteps 1125.
Path 111 | total_timesteps 1134.
Path 112 | total_timesteps 1144.
Path 113 | total_timesteps 1152.
Path 114 | total_timesteps 1161.
Path 115 | total_timesteps 1176.
Path 116 | total_timesteps 1185.
Path 117 | total_timesteps 1195.
Path 118 | total_timesteps 1205.
Path 119 | total_timesteps 1219.
Path 120 | total_timesteps 1227.
Path 121 | total_timesteps 1236.
Path 122 | total_timesteps 1248.
Path 123 | total_timesteps 1256.
Path 124 | total_timesteps 1264.
Path 125 | total_timesteps 1280.
Path 126 | total_timesteps 1290.
Path 127 | total_timesteps 1305.
Path 128 | total_timesteps 1312.
Path 129 | total_timesteps 1320.
Path 130 | total_timesteps 1329.
Path 131 | total_timesteps 1340.
Path 132 | total_timesteps 1350.
Path 133 | total_timesteps 1357.
Path 134 | total_timesteps 1372.
Path 135 | total_timesteps 1380.
Path 136 | total_timesteps 1389.
Path 137 | total_timesteps 1398.
Path 138 | total_timesteps 1405.
Path 139 | total_timesteps 1412.
Path 140 | total_timesteps 1424.
Path 141 | total_timesteps 1435.
Path 142 | total_timesteps 1443.
Path 143 | total_timesteps 1451.
Path 144 | total_timesteps 1463.
Path 145 | total_timesteps 1470.
Path 146 | total_timesteps 1478.
Path 147 | total_timesteps 1488.
Path 148 | total_timesteps 1495.
Path 149 | total_timesteps 1510.
Path 150 | total_timesteps 1520.
Path 151 | total_timesteps 1528.
Path 152 | total_timesteps 1536.
Path 153 | total_timesteps 1544.
Path 154 | total_timesteps 1555.
Path 155 | total_timesteps 1572.
Path 156 | total_timesteps 1579.
Path 157 | total_timesteps 1589.
Path 158 | total_timesteps 1600.
Path 159 | total_timesteps 1608.
Path 160 | total_timesteps 1616.
Path 161 | total_timesteps 1625.
Path 162 | total_timesteps 1634.
Path 163 | total_timesteps 1642.
Path 164 | total_timesteps 1651.
Path 165 | total_timesteps 1667.
Path 166 | total_timesteps 1675.
Path 167 | total_timesteps 1682.
Path 168 | total_timesteps 1700.
Path 169 | total_timesteps 1712.
Path 170 | total_timesteps 1721.
Path 171 | total_timesteps 1730.
Path 172 | total_timesteps 1736.
Path 173 | total_timesteps 1755.
Path 174 | total_timesteps 1765.
Path 175 | total_timesteps 1772.
Path 176 | total_timesteps 1782.
Path 177 | total_timesteps 1795.
Path 178 | total_timesteps 1811.
Path 179 | total_timesteps 1818.
Path 180 | total_timesteps 1829.
Path 181 | total_timesteps 1840.
Path 182 | total_timesteps 1851.
Path 183 | total_timesteps 1860.
Path 184 | total_timesteps 1869.
Path 185 | total_timesteps 1884.
Path 186 | total_timesteps 1895.
Path 187 | total_timesteps 1904.
Path 188 | total_timesteps 1915.
Path 189 | total_timesteps 1923.
Path 190 | total_timesteps 1935.
Path 191 | total_timesteps 1942.
Path 192 | total_timesteps 1950.
Path 193 | total_timesteps 1963.
Path 194 | total_timesteps 1971.
Path 195 | total_timesteps 1983.
Path 196 | total_timesteps 1993.
Path 197 | total_timesteps 2003.
Path 198 | total_timesteps 2011.
Path 199 | total_timesteps 2020.
Path 200 | total_timesteps 2029.
Path 201 | total_timesteps 2038.
Path 202 | total_timesteps 2044.
Path 203 | total_timesteps 2053.
Path 204 | total_timesteps 2066.
Path 205 | total_timesteps 2075.
Path 206 | total_timesteps 2085.
Path 207 | total_timesteps 2093.
Path 208 | total_timesteps 2100.
Path 209 | total_timesteps 2106.
Path 210 | total_timesteps 2116.
Path 211 | total_timesteps 2125.
Path 212 | total_timesteps 2134.
Path 213 | total_timesteps 2145.
Path 214 | total_timesteps 2153.
Path 215 | total_timesteps 2164.
Path 216 | total_timesteps 2172.
Path 217 | total_timesteps 2179.
Path 218 | total_timesteps 2189.
Path 219 | total_timesteps 2195.
Path 220 | total_timesteps 2202.
Path 221 | total_timesteps 2215.
Path 222 | total_timesteps 2222.
Path 223 | total_timesteps 2232.
Path 224 | total_timesteps 2245.
Path 225 | total_timesteps 2256.
Path 226 | total_timesteps 2263.
Path 227 | total_timesteps 2277.
Path 228 | total_timesteps 2287.
Path 229 | total_timesteps 2297.
Path 230 | total_timesteps 2308.
Path 231 | total_timesteps 2316.
Path 232 | total_timesteps 2327.
Path 233 | total_timesteps 2342.
Path 234 | total_timesteps 2350.
Path 235 | total_timesteps 2358.
Path 236 | total_timesteps 2366.
Path 237 | total_timesteps 2377.
Path 238 | total_timesteps 2385.
Path 239 | total_timesteps 2396.
Path 240 | total_timesteps 2419.
Path 241 | total_timesteps 2431.
Path 242 | total_timesteps 2441.
Path 243 | total_timesteps 2449.
Path 244 | total_timesteps 2458.
Path 245 | total_timesteps 2467.
Path 246 | total_timesteps 2478.
Path 247 | total_timesteps 2484.
Path 248 | total_timesteps 2493.
Path 249 | total_timesteps 2507.
Path 250 | total_timesteps 2514.
Path 251 | total_timesteps 2521.
Path 252 | total_timesteps 2528.
Path 253 | total_timesteps 2538.
Path 254 | total_timesteps 2547.
Path 255 | total_timesteps 2556.
Path 256 | total_timesteps 2569.
Path 257 | total_timesteps 2577.
Path 258 | total_timesteps 2584.
Path 259 | total_timesteps 2598.
Path 260 | total_timesteps 2607.
Path 261 | total_timesteps 2617.
Path 262 | total_timesteps 2631.
Path 263 | total_timesteps 2645.
Path 264 | total_timesteps 2653.
Path 265 | total_timesteps 2660.
Path 266 | total_timesteps 2667.
Path 267 | total_timesteps 2675.
Path 268 | total_timesteps 2687.
Path 269 | total_timesteps 2695.
Path 270 | total_timesteps 2703.
Path 271 | total_timesteps 2714.
Path 272 | total_timesteps 2727.
Path 273 | total_timesteps 2735.
Path 274 | total_timesteps 2743.
Path 275 | total_timesteps 2752.
Path 276 | total_timesteps 2760.
Path 277 | total_timesteps 2773.
Path 278 | total_timesteps 2780.
Path 279 | total_timesteps 2791.
Path 280 | total_timesteps 2801.
Path 281 | total_timesteps 2810.
Path 282 | total_timesteps 2818.
Path 283 | total_timesteps 2833.
Path 284 | total_timesteps 2852.
Path 285 | total_timesteps 2860.
Path 286 | total_timesteps 2867.
Path 287 | total_timesteps 2876.
Path 288 | total_timesteps 2888.
Path 289 | total_timesteps 2900.
Path 290 | total_timesteps 2907.
Path 291 | total_timesteps 2913.
Path 292 | total_timesteps 2921.
Path 293 | total_timesteps 2930.
Path 294 | total_timesteps 2936.
Path 295 | total_timesteps 2945.
Path 296 | total_timesteps 2955.
Path 297 | total_timesteps 2963.
Path 298 | total_timesteps 2973.
Path 299 | total_timesteps 2984.
Path 300 | total_timesteps 2995.
Path 301 | total_timesteps 3011.
Path 302 | total_timesteps 3023.
Path 303 | total_timesteps 3034.
Path 304 | total_timesteps 3041.
Path 305 | total_timesteps 3049.
Path 306 | total_timesteps 3058.
Path 307 | total_timesteps 3066.
Path 308 | total_timesteps 3083.
Path 309 | total_timesteps 3090.
Path 310 | total_timesteps 3098.
Path 311 | total_timesteps 3106.
Path 312 | total_timesteps 3117.
Path 313 | total_timesteps 3126.
Path 314 | total_timesteps 3146.
Path 315 | total_timesteps 3154.
Path 316 | total_timesteps 3166.
Path 317 | total_timesteps 3173.
Path 318 | total_timesteps 3183.
Path 319 | total_timesteps 3193.
Path 320 | total_timesteps 3202.
Path 321 | total_timesteps 3209.
Path 322 | total_timesteps 3219.
Path 323 | total_timesteps 3239.
Path 324 | total_timesteps 3252.
Path 325 | total_timesteps 3260.
Path 326 | total_timesteps 3270.
Path 327 | total_timesteps 3279.
Path 328 | total_timesteps 3289.
Path 329 | total_timesteps 3298.
Path 330 | total_timesteps 3312.
Path 331 | total_timesteps 3321.
Path 332 | total_timesteps 3330.
Path 333 | total_timesteps 3336.
Path 334 | total_timesteps 3345.
Path 335 | total_timesteps 3353.
Path 336 | total_timesteps 3361.
Path 337 | total_timesteps 3371.
Path 338 | total_timesteps 3378.
Path 339 | total_timesteps 3386.
Path 340 | total_timesteps 3396.
Path 341 | total_timesteps 3409.
Path 342 | total_timesteps 3419.
Path 343 | total_timesteps 3433.
Path 344 | total_timesteps 3442.
Path 345 | total_timesteps 3453.
Path 346 | total_timesteps 3460.
Path 347 | total_timesteps 3468.
Path 348 | total_timesteps 3482.
Path 349 | total_timesteps 3502.
Path 350 | total_timesteps 3509.
Path 351 | total_timesteps 3519.
Path 352 | total_timesteps 3530.
Path 353 | total_timesteps 3539.
Path 354 | total_timesteps 3546.
Path 355 | total_timesteps 3555.
Path 356 | total_timesteps 3575.
Path 357 | total_timesteps 3584.
Path 358 | total_timesteps 3598.
Path 359 | total_timesteps 3609.
Path 360 | total_timesteps 3616.
Path 361 | total_timesteps 3625.
Path 362 | total_timesteps 3636.
Path 363 | total_timesteps 3646.
Path 364 | total_timesteps 3654.
Path 365 | total_timesteps 3661.
Path 366 | total_timesteps 3669.
Path 367 | total_timesteps 3675.
Path 368 | total_timesteps 3683.
Path 369 | total_timesteps 3694.
Path 370 | total_timesteps 3709.
Path 371 | total_timesteps 3720.
Path 372 | total_timesteps 3731.
Path 373 | total_timesteps 3739.
Path 374 | total_timesteps 3747.
Path 375 | total_timesteps 3759.
Path 376 | total_timesteps 3770.
Path 377 | total_timesteps 3777.
Path 378 | total_timesteps 3786.
Path 379 | total_timesteps 3796.
Path 380 | total_timesteps 3804.
Path 381 | total_timesteps 3812.
Path 382 | total_timesteps 3832.
Path 383 | total_timesteps 3841.
Path 384 | total_timesteps 3849.
Path 385 | total_timesteps 3859.
Path 386 | total_timesteps 3866.
Path 387 | total_timesteps 3873.
Path 388 | total_timesteps 3886.
Path 389 | total_timesteps 3897.
Path 390 | total_timesteps 3909.
Path 391 | total_timesteps 3918.
Path 392 | total_timesteps 3931.
Path 393 | total_timesteps 3939.
Path 394 | total_timesteps 3953.
Path 395 | total_timesteps 3962.
Path 396 | total_timesteps 3971.
Path 397 | total_timesteps 3980.
Path 398 | total_timesteps 3988.
Path 399 | total_timesteps 3996.
Path 400 | total_timesteps 4004.
Path 401 | total_timesteps 4011.
Path 402 | total_timesteps 4017.
Path 403 | total_timesteps 4024.
Path 404 | total_timesteps 4034.
Path 405 | total_timesteps 4042.
Path 406 | total_timesteps 4055.
Path 407 | total_timesteps 4062.
Path 408 | total_timesteps 4072.
Path 409 | total_timesteps 4079.
Path 410 | total_timesteps 4091.
Path 411 | total_timesteps 4099.
Path 412 | total_timesteps 4107.
Path 413 | total_timesteps 4114.
Path 414 | total_timesteps 4126.
Path 415 | total_timesteps 4135.
Path 416 | total_timesteps 4142.
Path 417 | total_timesteps 4158.
Path 418 | total_timesteps 4165.
Path 419 | total_timesteps 4177.
Path 420 | total_timesteps 4186.
Path 421 | total_timesteps 4196.
Path 422 | total_timesteps 4212.
Path 423 | total_timesteps 4222.
Path 424 | total_timesteps 4229.
Path 425 | total_timesteps 4237.
Path 426 | total_timesteps 4246.
Path 427 | total_timesteps 4255.
Path 428 | total_timesteps 4263.
Path 429 | total_timesteps 4271.
Path 430 | total_timesteps 4281.
Path 431 | total_timesteps 4289.
Path 432 | total_timesteps 4299.
Path 433 | total_timesteps 4310.
Path 434 | total_timesteps 4318.
Path 435 | total_timesteps 4329.
Path 436 | total_timesteps 4337.
Path 437 | total_timesteps 4347.
Path 438 | total_timesteps 4354.
Path 439 | total_timesteps 4365.
Path 440 | total_timesteps 4377.
Path 441 | total_timesteps 4388.
Path 442 | total_timesteps 4403.
Path 443 | total_timesteps 4412.
Path 444 | total_timesteps 4423.
Path 445 | total_timesteps 4431.
Path 446 | total_timesteps 4439.
Path 447 | total_timesteps 4452.
Path 448 | total_timesteps 4459.
Path 449 | total_timesteps 4468.
Path 450 | total_timesteps 4481.
Path 451 | total_timesteps 4490.
Path 452 | total_timesteps 4498.
Path 453 | total_timesteps 4511.
Path 454 | total_timesteps 4520.
Path 455 | total_timesteps 4531.
Path 456 | total_timesteps 4539.
Path 457 | total_timesteps 4556.
Path 458 | total_timesteps 4564.
Path 459 | total_timesteps 4571.
Path 460 | total_timesteps 4582.
Path 461 | total_timesteps 4591.
Path 462 | total_timesteps 4612.
Path 463 | total_timesteps 4618.
Path 464 | total_timesteps 4628.
Path 465 | total_timesteps 4639.
Path 466 | total_timesteps 4646.
Path 467 | total_timesteps 4664.
Path 468 | total_timesteps 4671.
Path 469 | total_timesteps 4686.
Path 470 | total_timesteps 4694.
Path 471 | total_timesteps 4702.
Path 472 | total_timesteps 4712.
Path 473 | total_timesteps 4720.
Path 474 | total_timesteps 4734.
Path 475 | total_timesteps 4742.
Path 476 | total_timesteps 4754.
Path 477 | total_timesteps 4772.
Path 478 | total_timesteps 4792.
Path 479 | total_timesteps 4800.
Path 480 | total_timesteps 4812.
Path 481 | total_timesteps 4823.
Path 482 | total_timesteps 4834.
Path 483 | total_timesteps 4843.
Path 484 | total_timesteps 4849.
Path 485 | total_timesteps 4858.
Path 486 | total_timesteps 4870.
Path 487 | total_timesteps 4881.
Path 488 | total_timesteps 4890.
Path 489 | total_timesteps 4900.
Path 490 | total_timesteps 4910.
Path 491 | total_timesteps 4920.
Path 492 | total_timesteps 4930.
Path 493 | total_timesteps 4938.
Path 494 | total_timesteps 4945.
Path 495 | total_timesteps 4957.
Path 496 | total_timesteps 4964.
Path 497 | total_timesteps 4971.
Path 498 | total_timesteps 4978.
Path 499 | total_timesteps 4987.
Path 500 | total_timesteps 4997.
Path 501 | total_timesteps 5007.
Path 502 | total_timesteps 5015.
Path 503 | total_timesteps 5024.
Path 504 | total_timesteps 5032.
Path 505 | total_timesteps 5042.
Path 506 | total_timesteps 5050.
Path 507 | total_timesteps 5057.
Path 508 | total_timesteps 5065.
Path 509 | total_timesteps 5080.
Path 510 | total_timesteps 5087.
Path 511 | total_timesteps 5098.
Path 512 | total_timesteps 5106.
Path 513 | total_timesteps 5113.
Path 514 | total_timesteps 5126.
Path 515 | total_timesteps 5136.
Path 516 | total_timesteps 5143.
Path 517 | total_timesteps 5149.
Path 518 | total_timesteps 5158.
Path 519 | total_timesteps 5171.
Path 520 | total_timesteps 5180.
Path 521 | total_timesteps 5206.
Path 522 | total_timesteps 5216.
Path 523 | total_timesteps 5230.
Path 524 | total_timesteps 5238.
Path 525 | total_timesteps 5248.
Path 526 | total_timesteps 5256.
Path 527 | total_timesteps 5267.
Path 528 | total_timesteps 5277.
Path 529 | total_timesteps 5287.
Path 530 | total_timesteps 5302.
Path 531 | total_timesteps 5315.
Path 532 | total_timesteps 5325.
Path 533 | total_timesteps 5336.
Path 534 | total_timesteps 5343.
Path 535 | total_timesteps 5357.
Path 536 | total_timesteps 5370.
Path 537 | total_timesteps 5382.
Path 538 | total_timesteps 5390.
Path 539 | total_timesteps 5401.
Path 540 | total_timesteps 5415.
Path 541 | total_timesteps 5425.
Path 542 | total_timesteps 5436.
Path 543 | total_timesteps 5445.
Path 544 | total_timesteps 5457.
Path 545 | total_timesteps 5465.
Path 546 | total_timesteps 5479.
Path 547 | total_timesteps 5491.
Path 548 | total_timesteps 5503.
Path 549 | total_timesteps 5511.
Path 550 | total_timesteps 5520.
Path 551 | total_timesteps 5528.
Path 552 | total_timesteps 5535.
Path 553 | total_timesteps 5543.
Path 554 | total_timesteps 5553.
Path 555 | total_timesteps 5566.
Path 556 | total_timesteps 5573.
Path 557 | total_timesteps 5583.
Path 558 | total_timesteps 5593.
Path 559 | total_timesteps 5608.
Path 560 | total_timesteps 5616.
Path 561 | total_timesteps 5626.
Path 562 | total_timesteps 5634.
Path 563 | total_timesteps 5643.
Path 564 | total_timesteps 5652.
Path 565 | total_timesteps 5665.
Path 566 | total_timesteps 5674.
Path 567 | total_timesteps 5686.
Path 568 | total_timesteps 5700.
Path 569 | total_timesteps 5714.
Path 570 | total_timesteps 5726.
Path 571 | total_timesteps 5736.
Path 572 | total_timesteps 5748.
Path 573 | total_timesteps 5761.
Path 574 | total_timesteps 5771.
Path 575 | total_timesteps 5783.
Path 576 | total_timesteps 5797.
Path 577 | total_timesteps 5809.
Path 578 | total_timesteps 5816.
Path 579 | total_timesteps 5824.
Path 580 | total_timesteps 5833.
Path 581 | total_timesteps 5843.
Path 582 | total_timesteps 5850.
Path 583 | total_timesteps 5860.
Path 584 | total_timesteps 5874.
Path 585 | total_timesteps 5881.
Path 586 | total_timesteps 5891.
Path 587 | total_timesteps 5900.
Path 588 | total_timesteps 5913.
Path 589 | total_timesteps 5921.
Path 590 | total_timesteps 5931.
Path 591 | total_timesteps 5942.
Path 592 | total_timesteps 5953.
Path 593 | total_timesteps 5960.
Path 594 | total_timesteps 5969.
Path 595 | total_timesteps 5985.
Path 596 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.16    |
| Iteration     | 28       |
| MaximumReturn | 1.02     |
| MinimumReturn | -22.5    |
| TotalSamples  | 120111   |
----------------------------
itr #29 | 
Fitting dynamics.
Validation loss = 0.004342560190707445
Validation loss = 0.004462534096091986
Validation loss = 0.004227264318615198
Validation loss = 0.004361884202808142
Validation loss = 0.004934912547469139
Validation loss = 0.004168420098721981
Validation loss = 0.004361767787486315
Validation loss = 0.00428216764703393
Validation loss = 0.004234103485941887
Validation loss = 0.004187507554888725
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 39.
Path 5 | total_timesteps 51.
Path 6 | total_timesteps 58.
Path 7 | total_timesteps 76.
Path 8 | total_timesteps 90.
Path 9 | total_timesteps 100.
Path 10 | total_timesteps 108.
Path 11 | total_timesteps 115.
Path 12 | total_timesteps 128.
Path 13 | total_timesteps 144.
Path 14 | total_timesteps 154.
Path 15 | total_timesteps 161.
Path 16 | total_timesteps 172.
Path 17 | total_timesteps 183.
Path 18 | total_timesteps 198.
Path 19 | total_timesteps 209.
Path 20 | total_timesteps 219.
Path 21 | total_timesteps 229.
Path 22 | total_timesteps 235.
Path 23 | total_timesteps 243.
Path 24 | total_timesteps 252.
Path 25 | total_timesteps 259.
Path 26 | total_timesteps 266.
Path 27 | total_timesteps 273.
Path 28 | total_timesteps 286.
Path 29 | total_timesteps 294.
Path 30 | total_timesteps 304.
Path 31 | total_timesteps 310.
Path 32 | total_timesteps 319.
Path 33 | total_timesteps 326.
Path 34 | total_timesteps 333.
Path 35 | total_timesteps 341.
Path 36 | total_timesteps 350.
Path 37 | total_timesteps 364.
Path 38 | total_timesteps 375.
Path 39 | total_timesteps 385.
Path 40 | total_timesteps 393.
Path 41 | total_timesteps 400.
Path 42 | total_timesteps 408.
Path 43 | total_timesteps 419.
Path 44 | total_timesteps 435.
Path 45 | total_timesteps 443.
Path 46 | total_timesteps 451.
Path 47 | total_timesteps 459.
Path 48 | total_timesteps 472.
Path 49 | total_timesteps 480.
Path 50 | total_timesteps 489.
Path 51 | total_timesteps 498.
Path 52 | total_timesteps 519.
Path 53 | total_timesteps 525.
Path 54 | total_timesteps 536.
Path 55 | total_timesteps 543.
Path 56 | total_timesteps 581.
Path 57 | total_timesteps 590.
Path 58 | total_timesteps 598.
Path 59 | total_timesteps 613.
Path 60 | total_timesteps 621.
Path 61 | total_timesteps 631.
Path 62 | total_timesteps 646.
Path 63 | total_timesteps 654.
Path 64 | total_timesteps 667.
Path 65 | total_timesteps 677.
Path 66 | total_timesteps 685.
Path 67 | total_timesteps 694.
Path 68 | total_timesteps 702.
Path 69 | total_timesteps 714.
Path 70 | total_timesteps 724.
Path 71 | total_timesteps 737.
Path 72 | total_timesteps 745.
Path 73 | total_timesteps 751.
Path 74 | total_timesteps 761.
Path 75 | total_timesteps 769.
Path 76 | total_timesteps 777.
Path 77 | total_timesteps 783.
Path 78 | total_timesteps 790.
Path 79 | total_timesteps 800.
Path 80 | total_timesteps 811.
Path 81 | total_timesteps 821.
Path 82 | total_timesteps 832.
Path 83 | total_timesteps 845.
Path 84 | total_timesteps 854.
Path 85 | total_timesteps 867.
Path 86 | total_timesteps 876.
Path 87 | total_timesteps 883.
Path 88 | total_timesteps 891.
Path 89 | total_timesteps 899.
Path 90 | total_timesteps 912.
Path 91 | total_timesteps 924.
Path 92 | total_timesteps 932.
Path 93 | total_timesteps 939.
Path 94 | total_timesteps 948.
Path 95 | total_timesteps 959.
Path 96 | total_timesteps 974.
Path 97 | total_timesteps 983.
Path 98 | total_timesteps 993.
Path 99 | total_timesteps 1003.
Path 100 | total_timesteps 1015.
Path 101 | total_timesteps 1024.
Path 102 | total_timesteps 1032.
Path 103 | total_timesteps 1040.
Path 104 | total_timesteps 1047.
Path 105 | total_timesteps 1055.
Path 106 | total_timesteps 1063.
Path 107 | total_timesteps 1073.
Path 108 | total_timesteps 1084.
Path 109 | total_timesteps 1093.
Path 110 | total_timesteps 1104.
Path 111 | total_timesteps 1112.
Path 112 | total_timesteps 1122.
Path 113 | total_timesteps 1131.
Path 114 | total_timesteps 1138.
Path 115 | total_timesteps 1152.
Path 116 | total_timesteps 1160.
Path 117 | total_timesteps 1171.
Path 118 | total_timesteps 1185.
Path 119 | total_timesteps 1194.
Path 120 | total_timesteps 1201.
Path 121 | total_timesteps 1209.
Path 122 | total_timesteps 1217.
Path 123 | total_timesteps 1225.
Path 124 | total_timesteps 1239.
Path 125 | total_timesteps 1250.
Path 126 | total_timesteps 1261.
Path 127 | total_timesteps 1273.
Path 128 | total_timesteps 1285.
Path 129 | total_timesteps 1292.
Path 130 | total_timesteps 1299.
Path 131 | total_timesteps 1306.
Path 132 | total_timesteps 1313.
Path 133 | total_timesteps 1323.
Path 134 | total_timesteps 1330.
Path 135 | total_timesteps 1348.
Path 136 | total_timesteps 1357.
Path 137 | total_timesteps 1366.
Path 138 | total_timesteps 1374.
Path 139 | total_timesteps 1381.
Path 140 | total_timesteps 1392.
Path 141 | total_timesteps 1404.
Path 142 | total_timesteps 1421.
Path 143 | total_timesteps 1430.
Path 144 | total_timesteps 1437.
Path 145 | total_timesteps 1447.
Path 146 | total_timesteps 1456.
Path 147 | total_timesteps 1470.
Path 148 | total_timesteps 1479.
Path 149 | total_timesteps 1487.
Path 150 | total_timesteps 1495.
Path 151 | total_timesteps 1507.
Path 152 | total_timesteps 1514.
Path 153 | total_timesteps 1525.
Path 154 | total_timesteps 1532.
Path 155 | total_timesteps 1541.
Path 156 | total_timesteps 1548.
Path 157 | total_timesteps 1556.
Path 158 | total_timesteps 1567.
Path 159 | total_timesteps 1576.
Path 160 | total_timesteps 1588.
Path 161 | total_timesteps 1595.
Path 162 | total_timesteps 1607.
Path 163 | total_timesteps 1615.
Path 164 | total_timesteps 1623.
Path 165 | total_timesteps 1630.
Path 166 | total_timesteps 1640.
Path 167 | total_timesteps 1649.
Path 168 | total_timesteps 1660.
Path 169 | total_timesteps 1671.
Path 170 | total_timesteps 1683.
Path 171 | total_timesteps 1694.
Path 172 | total_timesteps 1701.
Path 173 | total_timesteps 1711.
Path 174 | total_timesteps 1718.
Path 175 | total_timesteps 1727.
Path 176 | total_timesteps 1734.
Path 177 | total_timesteps 1742.
Path 178 | total_timesteps 1749.
Path 179 | total_timesteps 1759.
Path 180 | total_timesteps 1768.
Path 181 | total_timesteps 1779.
Path 182 | total_timesteps 1787.
Path 183 | total_timesteps 1807.
Path 184 | total_timesteps 1814.
Path 185 | total_timesteps 1822.
Path 186 | total_timesteps 1833.
Path 187 | total_timesteps 1843.
Path 188 | total_timesteps 1855.
Path 189 | total_timesteps 1864.
Path 190 | total_timesteps 1872.
Path 191 | total_timesteps 1881.
Path 192 | total_timesteps 1891.
Path 193 | total_timesteps 1897.
Path 194 | total_timesteps 1910.
Path 195 | total_timesteps 1916.
Path 196 | total_timesteps 1928.
Path 197 | total_timesteps 1936.
Path 198 | total_timesteps 1944.
Path 199 | total_timesteps 1951.
Path 200 | total_timesteps 1960.
Path 201 | total_timesteps 1969.
Path 202 | total_timesteps 1977.
Path 203 | total_timesteps 1989.
Path 204 | total_timesteps 1996.
Path 205 | total_timesteps 2004.
Path 206 | total_timesteps 2011.
Path 207 | total_timesteps 2020.
Path 208 | total_timesteps 2036.
Path 209 | total_timesteps 2044.
Path 210 | total_timesteps 2052.
Path 211 | total_timesteps 2061.
Path 212 | total_timesteps 2070.
Path 213 | total_timesteps 2081.
Path 214 | total_timesteps 2092.
Path 215 | total_timesteps 2100.
Path 216 | total_timesteps 2109.
Path 217 | total_timesteps 2119.
Path 218 | total_timesteps 2127.
Path 219 | total_timesteps 2135.
Path 220 | total_timesteps 2149.
Path 221 | total_timesteps 2161.
Path 222 | total_timesteps 2169.
Path 223 | total_timesteps 2182.
Path 224 | total_timesteps 2196.
Path 225 | total_timesteps 2206.
Path 226 | total_timesteps 2216.
Path 227 | total_timesteps 2230.
Path 228 | total_timesteps 2237.
Path 229 | total_timesteps 2249.
Path 230 | total_timesteps 2259.
Path 231 | total_timesteps 2267.
Path 232 | total_timesteps 2279.
Path 233 | total_timesteps 2301.
Path 234 | total_timesteps 2311.
Path 235 | total_timesteps 2317.
Path 236 | total_timesteps 2326.
Path 237 | total_timesteps 2333.
Path 238 | total_timesteps 2342.
Path 239 | total_timesteps 2353.
Path 240 | total_timesteps 2360.
Path 241 | total_timesteps 2372.
Path 242 | total_timesteps 2385.
Path 243 | total_timesteps 2392.
Path 244 | total_timesteps 2403.
Path 245 | total_timesteps 2411.
Path 246 | total_timesteps 2419.
Path 247 | total_timesteps 2426.
Path 248 | total_timesteps 2433.
Path 249 | total_timesteps 2441.
Path 250 | total_timesteps 2453.
Path 251 | total_timesteps 2463.
Path 252 | total_timesteps 2472.
Path 253 | total_timesteps 2482.
Path 254 | total_timesteps 2489.
Path 255 | total_timesteps 2497.
Path 256 | total_timesteps 2511.
Path 257 | total_timesteps 2530.
Path 258 | total_timesteps 2545.
Path 259 | total_timesteps 2561.
Path 260 | total_timesteps 2569.
Path 261 | total_timesteps 2582.
Path 262 | total_timesteps 2592.
Path 263 | total_timesteps 2603.
Path 264 | total_timesteps 2610.
Path 265 | total_timesteps 2624.
Path 266 | total_timesteps 2632.
Path 267 | total_timesteps 2642.
Path 268 | total_timesteps 2649.
Path 269 | total_timesteps 2659.
Path 270 | total_timesteps 2668.
Path 271 | total_timesteps 2675.
Path 272 | total_timesteps 2685.
Path 273 | total_timesteps 2693.
Path 274 | total_timesteps 2706.
Path 275 | total_timesteps 2715.
Path 276 | total_timesteps 2727.
Path 277 | total_timesteps 2735.
Path 278 | total_timesteps 2743.
Path 279 | total_timesteps 2751.
Path 280 | total_timesteps 2764.
Path 281 | total_timesteps 2775.
Path 282 | total_timesteps 2785.
Path 283 | total_timesteps 2793.
Path 284 | total_timesteps 2804.
Path 285 | total_timesteps 2815.
Path 286 | total_timesteps 2828.
Path 287 | total_timesteps 2837.
Path 288 | total_timesteps 2845.
Path 289 | total_timesteps 2852.
Path 290 | total_timesteps 2862.
Path 291 | total_timesteps 2876.
Path 292 | total_timesteps 2883.
Path 293 | total_timesteps 2892.
Path 294 | total_timesteps 2900.
Path 295 | total_timesteps 2907.
Path 296 | total_timesteps 2916.
Path 297 | total_timesteps 2924.
Path 298 | total_timesteps 2935.
Path 299 | total_timesteps 2942.
Path 300 | total_timesteps 2950.
Path 301 | total_timesteps 2957.
Path 302 | total_timesteps 2965.
Path 303 | total_timesteps 2976.
Path 304 | total_timesteps 2982.
Path 305 | total_timesteps 2989.
Path 306 | total_timesteps 2996.
Path 307 | total_timesteps 3005.
Path 308 | total_timesteps 3012.
Path 309 | total_timesteps 3025.
Path 310 | total_timesteps 3032.
Path 311 | total_timesteps 3039.
Path 312 | total_timesteps 3050.
Path 313 | total_timesteps 3061.
Path 314 | total_timesteps 3068.
Path 315 | total_timesteps 3075.
Path 316 | total_timesteps 3085.
Path 317 | total_timesteps 3092.
Path 318 | total_timesteps 3103.
Path 319 | total_timesteps 3110.
Path 320 | total_timesteps 3119.
Path 321 | total_timesteps 3128.
Path 322 | total_timesteps 3141.
Path 323 | total_timesteps 3151.
Path 324 | total_timesteps 3159.
Path 325 | total_timesteps 3170.
Path 326 | total_timesteps 3179.
Path 327 | total_timesteps 3185.
Path 328 | total_timesteps 3192.
Path 329 | total_timesteps 3202.
Path 330 | total_timesteps 3213.
Path 331 | total_timesteps 3221.
Path 332 | total_timesteps 3229.
Path 333 | total_timesteps 3240.
Path 334 | total_timesteps 3251.
Path 335 | total_timesteps 3259.
Path 336 | total_timesteps 3275.
Path 337 | total_timesteps 3286.
Path 338 | total_timesteps 3293.
Path 339 | total_timesteps 3303.
Path 340 | total_timesteps 3315.
Path 341 | total_timesteps 3324.
Path 342 | total_timesteps 3331.
Path 343 | total_timesteps 3339.
Path 344 | total_timesteps 3348.
Path 345 | total_timesteps 3357.
Path 346 | total_timesteps 3368.
Path 347 | total_timesteps 3377.
Path 348 | total_timesteps 3386.
Path 349 | total_timesteps 3393.
Path 350 | total_timesteps 3404.
Path 351 | total_timesteps 3411.
Path 352 | total_timesteps 3423.
Path 353 | total_timesteps 3431.
Path 354 | total_timesteps 3441.
Path 355 | total_timesteps 3452.
Path 356 | total_timesteps 3468.
Path 357 | total_timesteps 3480.
Path 358 | total_timesteps 3486.
Path 359 | total_timesteps 3495.
Path 360 | total_timesteps 3501.
Path 361 | total_timesteps 3518.
Path 362 | total_timesteps 3529.
Path 363 | total_timesteps 3541.
Path 364 | total_timesteps 3550.
Path 365 | total_timesteps 3558.
Path 366 | total_timesteps 3565.
Path 367 | total_timesteps 3572.
Path 368 | total_timesteps 3588.
Path 369 | total_timesteps 3601.
Path 370 | total_timesteps 3608.
Path 371 | total_timesteps 3614.
Path 372 | total_timesteps 3627.
Path 373 | total_timesteps 3636.
Path 374 | total_timesteps 3644.
Path 375 | total_timesteps 3654.
Path 376 | total_timesteps 3667.
Path 377 | total_timesteps 3679.
Path 378 | total_timesteps 3689.
Path 379 | total_timesteps 3696.
Path 380 | total_timesteps 3707.
Path 381 | total_timesteps 3717.
Path 382 | total_timesteps 3724.
Path 383 | total_timesteps 3734.
Path 384 | total_timesteps 3749.
Path 385 | total_timesteps 3769.
Path 386 | total_timesteps 3779.
Path 387 | total_timesteps 3789.
Path 388 | total_timesteps 3796.
Path 389 | total_timesteps 3802.
Path 390 | total_timesteps 3814.
Path 391 | total_timesteps 3823.
Path 392 | total_timesteps 3832.
Path 393 | total_timesteps 3839.
Path 394 | total_timesteps 3850.
Path 395 | total_timesteps 3859.
Path 396 | total_timesteps 3869.
Path 397 | total_timesteps 3880.
Path 398 | total_timesteps 3894.
Path 399 | total_timesteps 3904.
Path 400 | total_timesteps 3915.
Path 401 | total_timesteps 3930.
Path 402 | total_timesteps 3941.
Path 403 | total_timesteps 3953.
Path 404 | total_timesteps 3960.
Path 405 | total_timesteps 3968.
Path 406 | total_timesteps 3977.
Path 407 | total_timesteps 3984.
Path 408 | total_timesteps 3993.
Path 409 | total_timesteps 4000.
Path 410 | total_timesteps 4014.
Path 411 | total_timesteps 4023.
Path 412 | total_timesteps 4032.
Path 413 | total_timesteps 4044.
Path 414 | total_timesteps 4051.
Path 415 | total_timesteps 4060.
Path 416 | total_timesteps 4074.
Path 417 | total_timesteps 4082.
Path 418 | total_timesteps 4093.
Path 419 | total_timesteps 4103.
Path 420 | total_timesteps 4115.
Path 421 | total_timesteps 4122.
Path 422 | total_timesteps 4130.
Path 423 | total_timesteps 4144.
Path 424 | total_timesteps 4153.
Path 425 | total_timesteps 4163.
Path 426 | total_timesteps 4174.
Path 427 | total_timesteps 4182.
Path 428 | total_timesteps 4193.
Path 429 | total_timesteps 4205.
Path 430 | total_timesteps 4212.
Path 431 | total_timesteps 4219.
Path 432 | total_timesteps 4233.
Path 433 | total_timesteps 4243.
Path 434 | total_timesteps 4257.
Path 435 | total_timesteps 4267.
Path 436 | total_timesteps 4281.
Path 437 | total_timesteps 4288.
Path 438 | total_timesteps 4299.
Path 439 | total_timesteps 4306.
Path 440 | total_timesteps 4316.
Path 441 | total_timesteps 4323.
Path 442 | total_timesteps 4335.
Path 443 | total_timesteps 4346.
Path 444 | total_timesteps 4355.
Path 445 | total_timesteps 4365.
Path 446 | total_timesteps 4377.
Path 447 | total_timesteps 4384.
Path 448 | total_timesteps 4392.
Path 449 | total_timesteps 4402.
Path 450 | total_timesteps 4417.
Path 451 | total_timesteps 4424.
Path 452 | total_timesteps 4433.
Path 453 | total_timesteps 4446.
Path 454 | total_timesteps 4461.
Path 455 | total_timesteps 4470.
Path 456 | total_timesteps 4477.
Path 457 | total_timesteps 4492.
Path 458 | total_timesteps 4499.
Path 459 | total_timesteps 4513.
Path 460 | total_timesteps 4523.
Path 461 | total_timesteps 4534.
Path 462 | total_timesteps 4542.
Path 463 | total_timesteps 4553.
Path 464 | total_timesteps 4563.
Path 465 | total_timesteps 4570.
Path 466 | total_timesteps 4579.
Path 467 | total_timesteps 4586.
Path 468 | total_timesteps 4603.
Path 469 | total_timesteps 4619.
Path 470 | total_timesteps 4627.
Path 471 | total_timesteps 4637.
Path 472 | total_timesteps 4647.
Path 473 | total_timesteps 4655.
Path 474 | total_timesteps 4666.
Path 475 | total_timesteps 4678.
Path 476 | total_timesteps 4686.
Path 477 | total_timesteps 4694.
Path 478 | total_timesteps 4702.
Path 479 | total_timesteps 4718.
Path 480 | total_timesteps 4728.
Path 481 | total_timesteps 4742.
Path 482 | total_timesteps 4748.
Path 483 | total_timesteps 4759.
Path 484 | total_timesteps 4769.
Path 485 | total_timesteps 4779.
Path 486 | total_timesteps 4788.
Path 487 | total_timesteps 4797.
Path 488 | total_timesteps 4804.
Path 489 | total_timesteps 4811.
Path 490 | total_timesteps 4818.
Path 491 | total_timesteps 4824.
Path 492 | total_timesteps 4832.
Path 493 | total_timesteps 4841.
Path 494 | total_timesteps 4852.
Path 495 | total_timesteps 4861.
Path 496 | total_timesteps 4868.
Path 497 | total_timesteps 4876.
Path 498 | total_timesteps 4892.
Path 499 | total_timesteps 4903.
Path 500 | total_timesteps 4914.
Path 501 | total_timesteps 4925.
Path 502 | total_timesteps 4934.
Path 503 | total_timesteps 4943.
Path 504 | total_timesteps 4951.
Path 505 | total_timesteps 4962.
Path 506 | total_timesteps 4971.
Path 507 | total_timesteps 4979.
Path 508 | total_timesteps 4988.
Path 509 | total_timesteps 4998.
Path 510 | total_timesteps 5009.
Path 511 | total_timesteps 5017.
Path 512 | total_timesteps 5028.
Path 513 | total_timesteps 5037.
Path 514 | total_timesteps 5045.
Path 515 | total_timesteps 5054.
Path 516 | total_timesteps 5063.
Path 517 | total_timesteps 5072.
Path 518 | total_timesteps 5084.
Path 519 | total_timesteps 5090.
Path 520 | total_timesteps 5100.
Path 521 | total_timesteps 5109.
Path 522 | total_timesteps 5118.
Path 523 | total_timesteps 5131.
Path 524 | total_timesteps 5138.
Path 525 | total_timesteps 5145.
Path 526 | total_timesteps 5154.
Path 527 | total_timesteps 5166.
Path 528 | total_timesteps 5177.
Path 529 | total_timesteps 5188.
Path 530 | total_timesteps 5201.
Path 531 | total_timesteps 5213.
Path 532 | total_timesteps 5229.
Path 533 | total_timesteps 5236.
Path 534 | total_timesteps 5245.
Path 535 | total_timesteps 5254.
Path 536 | total_timesteps 5265.
Path 537 | total_timesteps 5273.
Path 538 | total_timesteps 5286.
Path 539 | total_timesteps 5296.
Path 540 | total_timesteps 5309.
Path 541 | total_timesteps 5319.
Path 542 | total_timesteps 5328.
Path 543 | total_timesteps 5335.
Path 544 | total_timesteps 5348.
Path 545 | total_timesteps 5355.
Path 546 | total_timesteps 5365.
Path 547 | total_timesteps 5374.
Path 548 | total_timesteps 5382.
Path 549 | total_timesteps 5392.
Path 550 | total_timesteps 5399.
Path 551 | total_timesteps 5407.
Path 552 | total_timesteps 5414.
Path 553 | total_timesteps 5422.
Path 554 | total_timesteps 5440.
Path 555 | total_timesteps 5450.
Path 556 | total_timesteps 5458.
Path 557 | total_timesteps 5466.
Path 558 | total_timesteps 5476.
Path 559 | total_timesteps 5488.
Path 560 | total_timesteps 5500.
Path 561 | total_timesteps 5508.
Path 562 | total_timesteps 5515.
Path 563 | total_timesteps 5526.
Path 564 | total_timesteps 5536.
Path 565 | total_timesteps 5543.
Path 566 | total_timesteps 5552.
Path 567 | total_timesteps 5569.
Path 568 | total_timesteps 5578.
Path 569 | total_timesteps 5588.
Path 570 | total_timesteps 5610.
Path 571 | total_timesteps 5617.
Path 572 | total_timesteps 5631.
Path 573 | total_timesteps 5639.
Path 574 | total_timesteps 5651.
Path 575 | total_timesteps 5660.
Path 576 | total_timesteps 5670.
Path 577 | total_timesteps 5680.
Path 578 | total_timesteps 5690.
Path 579 | total_timesteps 5700.
Path 580 | total_timesteps 5709.
Path 581 | total_timesteps 5715.
Path 582 | total_timesteps 5726.
Path 583 | total_timesteps 5736.
Path 584 | total_timesteps 5749.
Path 585 | total_timesteps 5759.
Path 586 | total_timesteps 5769.
Path 587 | total_timesteps 5779.
Path 588 | total_timesteps 5787.
Path 589 | total_timesteps 5801.
Path 590 | total_timesteps 5812.
Path 591 | total_timesteps 5825.
Path 592 | total_timesteps 5833.
Path 593 | total_timesteps 5841.
Path 594 | total_timesteps 5851.
Path 595 | total_timesteps 5864.
Path 596 | total_timesteps 5876.
Path 597 | total_timesteps 5884.
Path 598 | total_timesteps 5896.
Path 599 | total_timesteps 5911.
Path 600 | total_timesteps 5922.
Path 601 | total_timesteps 5930.
Path 602 | total_timesteps 5939.
Path 603 | total_timesteps 5946.
Path 604 | total_timesteps 5956.
Path 605 | total_timesteps 5969.
Path 606 | total_timesteps 5975.
Path 607 | total_timesteps 5982.
Path 608 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.59    |
| Iteration     | 29       |
| MaximumReturn | -2.75    |
| MinimumReturn | -27.7    |
| TotalSamples  | 124113   |
----------------------------
itr #30 | 
Fitting dynamics.
Validation loss = 0.004226100165396929
Validation loss = 0.0043600499629974365
Validation loss = 0.004663055296987295
Validation loss = 0.00416876096278429
Validation loss = 0.00440791854634881
Validation loss = 0.004231614526361227
Validation loss = 0.004356858320534229
Validation loss = 0.0044189454056322575
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 25.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 46.
Path 6 | total_timesteps 54.
Path 7 | total_timesteps 63.
Path 8 | total_timesteps 75.
Path 9 | total_timesteps 87.
Path 10 | total_timesteps 93.
Path 11 | total_timesteps 108.
Path 12 | total_timesteps 116.
Path 13 | total_timesteps 133.
Path 14 | total_timesteps 141.
Path 15 | total_timesteps 148.
Path 16 | total_timesteps 158.
Path 17 | total_timesteps 165.
Path 18 | total_timesteps 172.
Path 19 | total_timesteps 179.
Path 20 | total_timesteps 186.
Path 21 | total_timesteps 195.
Path 22 | total_timesteps 206.
Path 23 | total_timesteps 213.
Path 24 | total_timesteps 221.
Path 25 | total_timesteps 229.
Path 26 | total_timesteps 237.
Path 27 | total_timesteps 245.
Path 28 | total_timesteps 253.
Path 29 | total_timesteps 266.
Path 30 | total_timesteps 272.
Path 31 | total_timesteps 283.
Path 32 | total_timesteps 291.
Path 33 | total_timesteps 301.
Path 34 | total_timesteps 308.
Path 35 | total_timesteps 317.
Path 36 | total_timesteps 324.
Path 37 | total_timesteps 331.
Path 38 | total_timesteps 340.
Path 39 | total_timesteps 346.
Path 40 | total_timesteps 358.
Path 41 | total_timesteps 366.
Path 42 | total_timesteps 380.
Path 43 | total_timesteps 387.
Path 44 | total_timesteps 399.
Path 45 | total_timesteps 408.
Path 46 | total_timesteps 417.
Path 47 | total_timesteps 423.
Path 48 | total_timesteps 430.
Path 49 | total_timesteps 440.
Path 50 | total_timesteps 451.
Path 51 | total_timesteps 461.
Path 52 | total_timesteps 468.
Path 53 | total_timesteps 478.
Path 54 | total_timesteps 489.
Path 55 | total_timesteps 497.
Path 56 | total_timesteps 504.
Path 57 | total_timesteps 513.
Path 58 | total_timesteps 524.
Path 59 | total_timesteps 531.
Path 60 | total_timesteps 539.
Path 61 | total_timesteps 550.
Path 62 | total_timesteps 564.
Path 63 | total_timesteps 572.
Path 64 | total_timesteps 580.
Path 65 | total_timesteps 588.
Path 66 | total_timesteps 598.
Path 67 | total_timesteps 608.
Path 68 | total_timesteps 619.
Path 69 | total_timesteps 630.
Path 70 | total_timesteps 649.
Path 71 | total_timesteps 657.
Path 72 | total_timesteps 665.
Path 73 | total_timesteps 677.
Path 74 | total_timesteps 688.
Path 75 | total_timesteps 697.
Path 76 | total_timesteps 706.
Path 77 | total_timesteps 718.
Path 78 | total_timesteps 725.
Path 79 | total_timesteps 742.
Path 80 | total_timesteps 751.
Path 81 | total_timesteps 760.
Path 82 | total_timesteps 773.
Path 83 | total_timesteps 782.
Path 84 | total_timesteps 791.
Path 85 | total_timesteps 797.
Path 86 | total_timesteps 807.
Path 87 | total_timesteps 823.
Path 88 | total_timesteps 829.
Path 89 | total_timesteps 838.
Path 90 | total_timesteps 846.
Path 91 | total_timesteps 853.
Path 92 | total_timesteps 862.
Path 93 | total_timesteps 873.
Path 94 | total_timesteps 880.
Path 95 | total_timesteps 886.
Path 96 | total_timesteps 899.
Path 97 | total_timesteps 909.
Path 98 | total_timesteps 920.
Path 99 | total_timesteps 928.
Path 100 | total_timesteps 935.
Path 101 | total_timesteps 948.
Path 102 | total_timesteps 960.
Path 103 | total_timesteps 971.
Path 104 | total_timesteps 979.
Path 105 | total_timesteps 986.
Path 106 | total_timesteps 992.
Path 107 | total_timesteps 1000.
Path 108 | total_timesteps 1011.
Path 109 | total_timesteps 1019.
Path 110 | total_timesteps 1026.
Path 111 | total_timesteps 1036.
Path 112 | total_timesteps 1045.
Path 113 | total_timesteps 1054.
Path 114 | total_timesteps 1061.
Path 115 | total_timesteps 1068.
Path 116 | total_timesteps 1076.
Path 117 | total_timesteps 1083.
Path 118 | total_timesteps 1092.
Path 119 | total_timesteps 1103.
Path 120 | total_timesteps 1111.
Path 121 | total_timesteps 1119.
Path 122 | total_timesteps 1126.
Path 123 | total_timesteps 1135.
Path 124 | total_timesteps 1142.
Path 125 | total_timesteps 1154.
Path 126 | total_timesteps 1166.
Path 127 | total_timesteps 1173.
Path 128 | total_timesteps 1180.
Path 129 | total_timesteps 1192.
Path 130 | total_timesteps 1201.
Path 131 | total_timesteps 1212.
Path 132 | total_timesteps 1219.
Path 133 | total_timesteps 1231.
Path 134 | total_timesteps 1247.
Path 135 | total_timesteps 1262.
Path 136 | total_timesteps 1275.
Path 137 | total_timesteps 1287.
Path 138 | total_timesteps 1293.
Path 139 | total_timesteps 1303.
Path 140 | total_timesteps 1318.
Path 141 | total_timesteps 1328.
Path 142 | total_timesteps 1335.
Path 143 | total_timesteps 1348.
Path 144 | total_timesteps 1356.
Path 145 | total_timesteps 1364.
Path 146 | total_timesteps 1377.
Path 147 | total_timesteps 1386.
Path 148 | total_timesteps 1398.
Path 149 | total_timesteps 1418.
Path 150 | total_timesteps 1438.
Path 151 | total_timesteps 1445.
Path 152 | total_timesteps 1453.
Path 153 | total_timesteps 1464.
Path 154 | total_timesteps 1473.
Path 155 | total_timesteps 1482.
Path 156 | total_timesteps 1496.
Path 157 | total_timesteps 1512.
Path 158 | total_timesteps 1521.
Path 159 | total_timesteps 1534.
Path 160 | total_timesteps 1541.
Path 161 | total_timesteps 1547.
Path 162 | total_timesteps 1553.
Path 163 | total_timesteps 1561.
Path 164 | total_timesteps 1568.
Path 165 | total_timesteps 1577.
Path 166 | total_timesteps 1585.
Path 167 | total_timesteps 1596.
Path 168 | total_timesteps 1609.
Path 169 | total_timesteps 1617.
Path 170 | total_timesteps 1624.
Path 171 | total_timesteps 1630.
Path 172 | total_timesteps 1638.
Path 173 | total_timesteps 1649.
Path 174 | total_timesteps 1666.
Path 175 | total_timesteps 1676.
Path 176 | total_timesteps 1684.
Path 177 | total_timesteps 1691.
Path 178 | total_timesteps 1699.
Path 179 | total_timesteps 1706.
Path 180 | total_timesteps 1715.
Path 181 | total_timesteps 1722.
Path 182 | total_timesteps 1732.
Path 183 | total_timesteps 1740.
Path 184 | total_timesteps 1747.
Path 185 | total_timesteps 1755.
Path 186 | total_timesteps 1762.
Path 187 | total_timesteps 1773.
Path 188 | total_timesteps 1784.
Path 189 | total_timesteps 1791.
Path 190 | total_timesteps 1800.
Path 191 | total_timesteps 1807.
Path 192 | total_timesteps 1815.
Path 193 | total_timesteps 1824.
Path 194 | total_timesteps 1832.
Path 195 | total_timesteps 1844.
Path 196 | total_timesteps 1854.
Path 197 | total_timesteps 1862.
Path 198 | total_timesteps 1872.
Path 199 | total_timesteps 1880.
Path 200 | total_timesteps 1891.
Path 201 | total_timesteps 1899.
Path 202 | total_timesteps 1908.
Path 203 | total_timesteps 1919.
Path 204 | total_timesteps 1928.
Path 205 | total_timesteps 1940.
Path 206 | total_timesteps 1950.
Path 207 | total_timesteps 1960.
Path 208 | total_timesteps 1970.
Path 209 | total_timesteps 1977.
Path 210 | total_timesteps 1984.
Path 211 | total_timesteps 1992.
Path 212 | total_timesteps 1999.
Path 213 | total_timesteps 2006.
Path 214 | total_timesteps 2018.
Path 215 | total_timesteps 2027.
Path 216 | total_timesteps 2034.
Path 217 | total_timesteps 2041.
Path 218 | total_timesteps 2050.
Path 219 | total_timesteps 2062.
Path 220 | total_timesteps 2070.
Path 221 | total_timesteps 2095.
Path 222 | total_timesteps 2102.
Path 223 | total_timesteps 2109.
Path 224 | total_timesteps 2115.
Path 225 | total_timesteps 2123.
Path 226 | total_timesteps 2133.
Path 227 | total_timesteps 2144.
Path 228 | total_timesteps 2157.
Path 229 | total_timesteps 2170.
Path 230 | total_timesteps 2178.
Path 231 | total_timesteps 2190.
Path 232 | total_timesteps 2198.
Path 233 | total_timesteps 2207.
Path 234 | total_timesteps 2217.
Path 235 | total_timesteps 2227.
Path 236 | total_timesteps 2236.
Path 237 | total_timesteps 2248.
Path 238 | total_timesteps 2259.
Path 239 | total_timesteps 2271.
Path 240 | total_timesteps 2279.
Path 241 | total_timesteps 2293.
Path 242 | total_timesteps 2304.
Path 243 | total_timesteps 2317.
Path 244 | total_timesteps 2332.
Path 245 | total_timesteps 2341.
Path 246 | total_timesteps 2350.
Path 247 | total_timesteps 2357.
Path 248 | total_timesteps 2369.
Path 249 | total_timesteps 2377.
Path 250 | total_timesteps 2385.
Path 251 | total_timesteps 2393.
Path 252 | total_timesteps 2401.
Path 253 | total_timesteps 2412.
Path 254 | total_timesteps 2420.
Path 255 | total_timesteps 2426.
Path 256 | total_timesteps 2433.
Path 257 | total_timesteps 2442.
Path 258 | total_timesteps 2452.
Path 259 | total_timesteps 2460.
Path 260 | total_timesteps 2466.
Path 261 | total_timesteps 2473.
Path 262 | total_timesteps 2480.
Path 263 | total_timesteps 2489.
Path 264 | total_timesteps 2501.
Path 265 | total_timesteps 2509.
Path 266 | total_timesteps 2520.
Path 267 | total_timesteps 2526.
Path 268 | total_timesteps 2536.
Path 269 | total_timesteps 2542.
Path 270 | total_timesteps 2557.
Path 271 | total_timesteps 2567.
Path 272 | total_timesteps 2576.
Path 273 | total_timesteps 2589.
Path 274 | total_timesteps 2602.
Path 275 | total_timesteps 2608.
Path 276 | total_timesteps 2616.
Path 277 | total_timesteps 2628.
Path 278 | total_timesteps 2634.
Path 279 | total_timesteps 2644.
Path 280 | total_timesteps 2654.
Path 281 | total_timesteps 2661.
Path 282 | total_timesteps 2670.
Path 283 | total_timesteps 2678.
Path 284 | total_timesteps 2686.
Path 285 | total_timesteps 2695.
Path 286 | total_timesteps 2702.
Path 287 | total_timesteps 2710.
Path 288 | total_timesteps 2722.
Path 289 | total_timesteps 2731.
Path 290 | total_timesteps 2743.
Path 291 | total_timesteps 2751.
Path 292 | total_timesteps 2762.
Path 293 | total_timesteps 2774.
Path 294 | total_timesteps 2787.
Path 295 | total_timesteps 2795.
Path 296 | total_timesteps 2806.
Path 297 | total_timesteps 2815.
Path 298 | total_timesteps 2824.
Path 299 | total_timesteps 2833.
Path 300 | total_timesteps 2841.
Path 301 | total_timesteps 2849.
Path 302 | total_timesteps 2856.
Path 303 | total_timesteps 2863.
Path 304 | total_timesteps 2871.
Path 305 | total_timesteps 2880.
Path 306 | total_timesteps 2889.
Path 307 | total_timesteps 2896.
Path 308 | total_timesteps 2904.
Path 309 | total_timesteps 2912.
Path 310 | total_timesteps 2922.
Path 311 | total_timesteps 2929.
Path 312 | total_timesteps 2954.
Path 313 | total_timesteps 2967.
Path 314 | total_timesteps 2983.
Path 315 | total_timesteps 2992.
Path 316 | total_timesteps 3005.
Path 317 | total_timesteps 3014.
Path 318 | total_timesteps 3024.
Path 319 | total_timesteps 3032.
Path 320 | total_timesteps 3045.
Path 321 | total_timesteps 3055.
Path 322 | total_timesteps 3064.
Path 323 | total_timesteps 3077.
Path 324 | total_timesteps 3084.
Path 325 | total_timesteps 3094.
Path 326 | total_timesteps 3101.
Path 327 | total_timesteps 3113.
Path 328 | total_timesteps 3120.
Path 329 | total_timesteps 3129.
Path 330 | total_timesteps 3137.
Path 331 | total_timesteps 3148.
Path 332 | total_timesteps 3156.
Path 333 | total_timesteps 3165.
Path 334 | total_timesteps 3177.
Path 335 | total_timesteps 3183.
Path 336 | total_timesteps 3195.
Path 337 | total_timesteps 3210.
Path 338 | total_timesteps 3220.
Path 339 | total_timesteps 3230.
Path 340 | total_timesteps 3242.
Path 341 | total_timesteps 3253.
Path 342 | total_timesteps 3263.
Path 343 | total_timesteps 3276.
Path 344 | total_timesteps 3284.
Path 345 | total_timesteps 3302.
Path 346 | total_timesteps 3315.
Path 347 | total_timesteps 3325.
Path 348 | total_timesteps 3340.
Path 349 | total_timesteps 3348.
Path 350 | total_timesteps 3355.
Path 351 | total_timesteps 3363.
Path 352 | total_timesteps 3374.
Path 353 | total_timesteps 3385.
Path 354 | total_timesteps 3391.
Path 355 | total_timesteps 3400.
Path 356 | total_timesteps 3408.
Path 357 | total_timesteps 3418.
Path 358 | total_timesteps 3428.
Path 359 | total_timesteps 3441.
Path 360 | total_timesteps 3450.
Path 361 | total_timesteps 3457.
Path 362 | total_timesteps 3466.
Path 363 | total_timesteps 3474.
Path 364 | total_timesteps 3481.
Path 365 | total_timesteps 3493.
Path 366 | total_timesteps 3501.
Path 367 | total_timesteps 3509.
Path 368 | total_timesteps 3515.
Path 369 | total_timesteps 3524.
Path 370 | total_timesteps 3533.
Path 371 | total_timesteps 3541.
Path 372 | total_timesteps 3552.
Path 373 | total_timesteps 3560.
Path 374 | total_timesteps 3587.
Path 375 | total_timesteps 3595.
Path 376 | total_timesteps 3604.
Path 377 | total_timesteps 3622.
Path 378 | total_timesteps 3629.
Path 379 | total_timesteps 3637.
Path 380 | total_timesteps 3643.
Path 381 | total_timesteps 3650.
Path 382 | total_timesteps 3665.
Path 383 | total_timesteps 3674.
Path 384 | total_timesteps 3692.
Path 385 | total_timesteps 3699.
Path 386 | total_timesteps 3708.
Path 387 | total_timesteps 3716.
Path 388 | total_timesteps 3724.
Path 389 | total_timesteps 3731.
Path 390 | total_timesteps 3744.
Path 391 | total_timesteps 3754.
Path 392 | total_timesteps 3762.
Path 393 | total_timesteps 3771.
Path 394 | total_timesteps 3787.
Path 395 | total_timesteps 3797.
Path 396 | total_timesteps 3805.
Path 397 | total_timesteps 3813.
Path 398 | total_timesteps 3821.
Path 399 | total_timesteps 3829.
Path 400 | total_timesteps 3838.
Path 401 | total_timesteps 3845.
Path 402 | total_timesteps 3853.
Path 403 | total_timesteps 3861.
Path 404 | total_timesteps 3871.
Path 405 | total_timesteps 3878.
Path 406 | total_timesteps 3885.
Path 407 | total_timesteps 3893.
Path 408 | total_timesteps 3901.
Path 409 | total_timesteps 3907.
Path 410 | total_timesteps 3914.
Path 411 | total_timesteps 3921.
Path 412 | total_timesteps 3937.
Path 413 | total_timesteps 3946.
Path 414 | total_timesteps 3955.
Path 415 | total_timesteps 3967.
Path 416 | total_timesteps 3974.
Path 417 | total_timesteps 3982.
Path 418 | total_timesteps 3991.
Path 419 | total_timesteps 4003.
Path 420 | total_timesteps 4012.
Path 421 | total_timesteps 4026.
Path 422 | total_timesteps 4034.
Path 423 | total_timesteps 4042.
Path 424 | total_timesteps 4049.
Path 425 | total_timesteps 4059.
Path 426 | total_timesteps 4070.
Path 427 | total_timesteps 4084.
Path 428 | total_timesteps 4092.
Path 429 | total_timesteps 4101.
Path 430 | total_timesteps 4112.
Path 431 | total_timesteps 4123.
Path 432 | total_timesteps 4133.
Path 433 | total_timesteps 4140.
Path 434 | total_timesteps 4147.
Path 435 | total_timesteps 4160.
Path 436 | total_timesteps 4167.
Path 437 | total_timesteps 4176.
Path 438 | total_timesteps 4187.
Path 439 | total_timesteps 4195.
Path 440 | total_timesteps 4203.
Path 441 | total_timesteps 4212.
Path 442 | total_timesteps 4226.
Path 443 | total_timesteps 4233.
Path 444 | total_timesteps 4246.
Path 445 | total_timesteps 4253.
Path 446 | total_timesteps 4269.
Path 447 | total_timesteps 4276.
Path 448 | total_timesteps 4288.
Path 449 | total_timesteps 4299.
Path 450 | total_timesteps 4308.
Path 451 | total_timesteps 4315.
Path 452 | total_timesteps 4323.
Path 453 | total_timesteps 4333.
Path 454 | total_timesteps 4341.
Path 455 | total_timesteps 4353.
Path 456 | total_timesteps 4366.
Path 457 | total_timesteps 4373.
Path 458 | total_timesteps 4382.
Path 459 | total_timesteps 4391.
Path 460 | total_timesteps 4397.
Path 461 | total_timesteps 4405.
Path 462 | total_timesteps 4415.
Path 463 | total_timesteps 4426.
Path 464 | total_timesteps 4433.
Path 465 | total_timesteps 4440.
Path 466 | total_timesteps 4452.
Path 467 | total_timesteps 4458.
Path 468 | total_timesteps 4466.
Path 469 | total_timesteps 4476.
Path 470 | total_timesteps 4485.
Path 471 | total_timesteps 4493.
Path 472 | total_timesteps 4505.
Path 473 | total_timesteps 4513.
Path 474 | total_timesteps 4521.
Path 475 | total_timesteps 4532.
Path 476 | total_timesteps 4539.
Path 477 | total_timesteps 4564.
Path 478 | total_timesteps 4576.
Path 479 | total_timesteps 4588.
Path 480 | total_timesteps 4596.
Path 481 | total_timesteps 4606.
Path 482 | total_timesteps 4616.
Path 483 | total_timesteps 4625.
Path 484 | total_timesteps 4633.
Path 485 | total_timesteps 4639.
Path 486 | total_timesteps 4649.
Path 487 | total_timesteps 4657.
Path 488 | total_timesteps 4664.
Path 489 | total_timesteps 4673.
Path 490 | total_timesteps 4685.
Path 491 | total_timesteps 4696.
Path 492 | total_timesteps 4708.
Path 493 | total_timesteps 4717.
Path 494 | total_timesteps 4729.
Path 495 | total_timesteps 4737.
Path 496 | total_timesteps 4746.
Path 497 | total_timesteps 4753.
Path 498 | total_timesteps 4763.
Path 499 | total_timesteps 4770.
Path 500 | total_timesteps 4778.
Path 501 | total_timesteps 4785.
Path 502 | total_timesteps 4805.
Path 503 | total_timesteps 4813.
Path 504 | total_timesteps 4819.
Path 505 | total_timesteps 4826.
Path 506 | total_timesteps 4843.
Path 507 | total_timesteps 4850.
Path 508 | total_timesteps 4863.
Path 509 | total_timesteps 4872.
Path 510 | total_timesteps 4886.
Path 511 | total_timesteps 4898.
Path 512 | total_timesteps 4905.
Path 513 | total_timesteps 4914.
Path 514 | total_timesteps 4927.
Path 515 | total_timesteps 4936.
Path 516 | total_timesteps 4948.
Path 517 | total_timesteps 4956.
Path 518 | total_timesteps 4967.
Path 519 | total_timesteps 4975.
Path 520 | total_timesteps 4986.
Path 521 | total_timesteps 4992.
Path 522 | total_timesteps 5000.
Path 523 | total_timesteps 5009.
Path 524 | total_timesteps 5018.
Path 525 | total_timesteps 5029.
Path 526 | total_timesteps 5041.
Path 527 | total_timesteps 5051.
Path 528 | total_timesteps 5067.
Path 529 | total_timesteps 5078.
Path 530 | total_timesteps 5088.
Path 531 | total_timesteps 5095.
Path 532 | total_timesteps 5107.
Path 533 | total_timesteps 5115.
Path 534 | total_timesteps 5123.
Path 535 | total_timesteps 5131.
Path 536 | total_timesteps 5143.
Path 537 | total_timesteps 5152.
Path 538 | total_timesteps 5162.
Path 539 | total_timesteps 5171.
Path 540 | total_timesteps 5184.
Path 541 | total_timesteps 5191.
Path 542 | total_timesteps 5209.
Path 543 | total_timesteps 5220.
Path 544 | total_timesteps 5228.
Path 545 | total_timesteps 5235.
Path 546 | total_timesteps 5242.
Path 547 | total_timesteps 5250.
Path 548 | total_timesteps 5258.
Path 549 | total_timesteps 5265.
Path 550 | total_timesteps 5274.
Path 551 | total_timesteps 5281.
Path 552 | total_timesteps 5291.
Path 553 | total_timesteps 5310.
Path 554 | total_timesteps 5317.
Path 555 | total_timesteps 5330.
Path 556 | total_timesteps 5341.
Path 557 | total_timesteps 5349.
Path 558 | total_timesteps 5356.
Path 559 | total_timesteps 5362.
Path 560 | total_timesteps 5371.
Path 561 | total_timesteps 5379.
Path 562 | total_timesteps 5386.
Path 563 | total_timesteps 5393.
Path 564 | total_timesteps 5400.
Path 565 | total_timesteps 5408.
Path 566 | total_timesteps 5417.
Path 567 | total_timesteps 5425.
Path 568 | total_timesteps 5433.
Path 569 | total_timesteps 5442.
Path 570 | total_timesteps 5450.
Path 571 | total_timesteps 5459.
Path 572 | total_timesteps 5473.
Path 573 | total_timesteps 5480.
Path 574 | total_timesteps 5487.
Path 575 | total_timesteps 5496.
Path 576 | total_timesteps 5503.
Path 577 | total_timesteps 5513.
Path 578 | total_timesteps 5521.
Path 579 | total_timesteps 5528.
Path 580 | total_timesteps 5535.
Path 581 | total_timesteps 5545.
Path 582 | total_timesteps 5555.
Path 583 | total_timesteps 5562.
Path 584 | total_timesteps 5571.
Path 585 | total_timesteps 5583.
Path 586 | total_timesteps 5592.
Path 587 | total_timesteps 5608.
Path 588 | total_timesteps 5617.
Path 589 | total_timesteps 5628.
Path 590 | total_timesteps 5636.
Path 591 | total_timesteps 5642.
Path 592 | total_timesteps 5649.
Path 593 | total_timesteps 5656.
Path 594 | total_timesteps 5663.
Path 595 | total_timesteps 5673.
Path 596 | total_timesteps 5684.
Path 597 | total_timesteps 5692.
Path 598 | total_timesteps 5699.
Path 599 | total_timesteps 5706.
Path 600 | total_timesteps 5713.
Path 601 | total_timesteps 5721.
Path 602 | total_timesteps 5731.
Path 603 | total_timesteps 5739.
Path 604 | total_timesteps 5752.
Path 605 | total_timesteps 5762.
Path 606 | total_timesteps 5770.
Path 607 | total_timesteps 5783.
Path 608 | total_timesteps 5791.
Path 609 | total_timesteps 5799.
Path 610 | total_timesteps 5805.
Path 611 | total_timesteps 5814.
Path 612 | total_timesteps 5822.
Path 613 | total_timesteps 5830.
Path 614 | total_timesteps 5840.
Path 615 | total_timesteps 5849.
Path 616 | total_timesteps 5857.
Path 617 | total_timesteps 5864.
Path 618 | total_timesteps 5872.
Path 619 | total_timesteps 5880.
Path 620 | total_timesteps 5891.
Path 621 | total_timesteps 5903.
Path 622 | total_timesteps 5914.
Path 623 | total_timesteps 5924.
Path 624 | total_timesteps 5935.
Path 625 | total_timesteps 5944.
Path 626 | total_timesteps 5956.
Path 627 | total_timesteps 5972.
Path 628 | total_timesteps 5984.
Path 629 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.43    |
| Iteration     | 30       |
| MaximumReturn | 0.33     |
| MinimumReturn | -24      |
| TotalSamples  | 128118   |
----------------------------
itr #31 | 
Fitting dynamics.
Validation loss = 0.004247529432177544
Validation loss = 0.004258580040186644
Validation loss = 0.004236084874719381
Validation loss = 0.004231489263474941
Validation loss = 0.004309186711907387
Validation loss = 0.004183589480817318
Validation loss = 0.004230190068483353
Validation loss = 0.004249464254826307
Validation loss = 0.004226965829730034
Validation loss = 0.004559108056128025
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 46.
Path 6 | total_timesteps 56.
Path 7 | total_timesteps 67.
Path 8 | total_timesteps 81.
Path 9 | total_timesteps 89.
Path 10 | total_timesteps 97.
Path 11 | total_timesteps 111.
Path 12 | total_timesteps 121.
Path 13 | total_timesteps 135.
Path 14 | total_timesteps 145.
Path 15 | total_timesteps 153.
Path 16 | total_timesteps 174.
Path 17 | total_timesteps 190.
Path 18 | total_timesteps 197.
Path 19 | total_timesteps 209.
Path 20 | total_timesteps 217.
Path 21 | total_timesteps 228.
Path 22 | total_timesteps 241.
Path 23 | total_timesteps 250.
Path 24 | total_timesteps 260.
Path 25 | total_timesteps 267.
Path 26 | total_timesteps 277.
Path 27 | total_timesteps 283.
Path 28 | total_timesteps 291.
Path 29 | total_timesteps 301.
Path 30 | total_timesteps 309.
Path 31 | total_timesteps 316.
Path 32 | total_timesteps 329.
Path 33 | total_timesteps 336.
Path 34 | total_timesteps 343.
Path 35 | total_timesteps 352.
Path 36 | total_timesteps 360.
Path 37 | total_timesteps 367.
Path 38 | total_timesteps 373.
Path 39 | total_timesteps 385.
Path 40 | total_timesteps 395.
Path 41 | total_timesteps 403.
Path 42 | total_timesteps 409.
Path 43 | total_timesteps 416.
Path 44 | total_timesteps 430.
Path 45 | total_timesteps 439.
Path 46 | total_timesteps 447.
Path 47 | total_timesteps 457.
Path 48 | total_timesteps 472.
Path 49 | total_timesteps 479.
Path 50 | total_timesteps 486.
Path 51 | total_timesteps 493.
Path 52 | total_timesteps 501.
Path 53 | total_timesteps 510.
Path 54 | total_timesteps 521.
Path 55 | total_timesteps 528.
Path 56 | total_timesteps 536.
Path 57 | total_timesteps 544.
Path 58 | total_timesteps 557.
Path 59 | total_timesteps 566.
Path 60 | total_timesteps 574.
Path 61 | total_timesteps 581.
Path 62 | total_timesteps 589.
Path 63 | total_timesteps 598.
Path 64 | total_timesteps 606.
Path 65 | total_timesteps 613.
Path 66 | total_timesteps 624.
Path 67 | total_timesteps 633.
Path 68 | total_timesteps 641.
Path 69 | total_timesteps 649.
Path 70 | total_timesteps 657.
Path 71 | total_timesteps 667.
Path 72 | total_timesteps 674.
Path 73 | total_timesteps 683.
Path 74 | total_timesteps 691.
Path 75 | total_timesteps 704.
Path 76 | total_timesteps 715.
Path 77 | total_timesteps 729.
Path 78 | total_timesteps 740.
Path 79 | total_timesteps 748.
Path 80 | total_timesteps 758.
Path 81 | total_timesteps 764.
Path 82 | total_timesteps 774.
Path 83 | total_timesteps 782.
Path 84 | total_timesteps 791.
Path 85 | total_timesteps 798.
Path 86 | total_timesteps 809.
Path 87 | total_timesteps 816.
Path 88 | total_timesteps 822.
Path 89 | total_timesteps 830.
Path 90 | total_timesteps 838.
Path 91 | total_timesteps 847.
Path 92 | total_timesteps 857.
Path 93 | total_timesteps 868.
Path 94 | total_timesteps 874.
Path 95 | total_timesteps 885.
Path 96 | total_timesteps 899.
Path 97 | total_timesteps 907.
Path 98 | total_timesteps 915.
Path 99 | total_timesteps 924.
Path 100 | total_timesteps 932.
Path 101 | total_timesteps 941.
Path 102 | total_timesteps 953.
Path 103 | total_timesteps 961.
Path 104 | total_timesteps 968.
Path 105 | total_timesteps 981.
Path 106 | total_timesteps 988.
Path 107 | total_timesteps 998.
Path 108 | total_timesteps 1006.
Path 109 | total_timesteps 1016.
Path 110 | total_timesteps 1028.
Path 111 | total_timesteps 1038.
Path 112 | total_timesteps 1046.
Path 113 | total_timesteps 1058.
Path 114 | total_timesteps 1065.
Path 115 | total_timesteps 1073.
Path 116 | total_timesteps 1082.
Path 117 | total_timesteps 1101.
Path 118 | total_timesteps 1108.
Path 119 | total_timesteps 1115.
Path 120 | total_timesteps 1123.
Path 121 | total_timesteps 1129.
Path 122 | total_timesteps 1135.
Path 123 | total_timesteps 1142.
Path 124 | total_timesteps 1153.
Path 125 | total_timesteps 1161.
Path 126 | total_timesteps 1172.
Path 127 | total_timesteps 1179.
Path 128 | total_timesteps 1185.
Path 129 | total_timesteps 1195.
Path 130 | total_timesteps 1202.
Path 131 | total_timesteps 1211.
Path 132 | total_timesteps 1222.
Path 133 | total_timesteps 1228.
Path 134 | total_timesteps 1235.
Path 135 | total_timesteps 1241.
Path 136 | total_timesteps 1248.
Path 137 | total_timesteps 1262.
Path 138 | total_timesteps 1273.
Path 139 | total_timesteps 1280.
Path 140 | total_timesteps 1288.
Path 141 | total_timesteps 1296.
Path 142 | total_timesteps 1304.
Path 143 | total_timesteps 1312.
Path 144 | total_timesteps 1325.
Path 145 | total_timesteps 1336.
Path 146 | total_timesteps 1345.
Path 147 | total_timesteps 1353.
Path 148 | total_timesteps 1362.
Path 149 | total_timesteps 1369.
Path 150 | total_timesteps 1383.
Path 151 | total_timesteps 1390.
Path 152 | total_timesteps 1399.
Path 153 | total_timesteps 1407.
Path 154 | total_timesteps 1418.
Path 155 | total_timesteps 1431.
Path 156 | total_timesteps 1441.
Path 157 | total_timesteps 1450.
Path 158 | total_timesteps 1462.
Path 159 | total_timesteps 1475.
Path 160 | total_timesteps 1486.
Path 161 | total_timesteps 1496.
Path 162 | total_timesteps 1504.
Path 163 | total_timesteps 1513.
Path 164 | total_timesteps 1525.
Path 165 | total_timesteps 1533.
Path 166 | total_timesteps 1542.
Path 167 | total_timesteps 1550.
Path 168 | total_timesteps 1559.
Path 169 | total_timesteps 1568.
Path 170 | total_timesteps 1576.
Path 171 | total_timesteps 1583.
Path 172 | total_timesteps 1591.
Path 173 | total_timesteps 1599.
Path 174 | total_timesteps 1610.
Path 175 | total_timesteps 1617.
Path 176 | total_timesteps 1623.
Path 177 | total_timesteps 1641.
Path 178 | total_timesteps 1649.
Path 179 | total_timesteps 1660.
Path 180 | total_timesteps 1666.
Path 181 | total_timesteps 1674.
Path 182 | total_timesteps 1683.
Path 183 | total_timesteps 1690.
Path 184 | total_timesteps 1696.
Path 185 | total_timesteps 1706.
Path 186 | total_timesteps 1713.
Path 187 | total_timesteps 1720.
Path 188 | total_timesteps 1729.
Path 189 | total_timesteps 1737.
Path 190 | total_timesteps 1745.
Path 191 | total_timesteps 1752.
Path 192 | total_timesteps 1759.
Path 193 | total_timesteps 1770.
Path 194 | total_timesteps 1777.
Path 195 | total_timesteps 1785.
Path 196 | total_timesteps 1803.
Path 197 | total_timesteps 1811.
Path 198 | total_timesteps 1821.
Path 199 | total_timesteps 1829.
Path 200 | total_timesteps 1840.
Path 201 | total_timesteps 1849.
Path 202 | total_timesteps 1862.
Path 203 | total_timesteps 1871.
Path 204 | total_timesteps 1879.
Path 205 | total_timesteps 1891.
Path 206 | total_timesteps 1901.
Path 207 | total_timesteps 1909.
Path 208 | total_timesteps 1920.
Path 209 | total_timesteps 1935.
Path 210 | total_timesteps 1943.
Path 211 | total_timesteps 1950.
Path 212 | total_timesteps 1959.
Path 213 | total_timesteps 1966.
Path 214 | total_timesteps 1977.
Path 215 | total_timesteps 1989.
Path 216 | total_timesteps 1997.
Path 217 | total_timesteps 2005.
Path 218 | total_timesteps 2013.
Path 219 | total_timesteps 2021.
Path 220 | total_timesteps 2035.
Path 221 | total_timesteps 2046.
Path 222 | total_timesteps 2052.
Path 223 | total_timesteps 2063.
Path 224 | total_timesteps 2071.
Path 225 | total_timesteps 2087.
Path 226 | total_timesteps 2095.
Path 227 | total_timesteps 2103.
Path 228 | total_timesteps 2116.
Path 229 | total_timesteps 2125.
Path 230 | total_timesteps 2139.
Path 231 | total_timesteps 2149.
Path 232 | total_timesteps 2156.
Path 233 | total_timesteps 2167.
Path 234 | total_timesteps 2175.
Path 235 | total_timesteps 2183.
Path 236 | total_timesteps 2193.
Path 237 | total_timesteps 2200.
Path 238 | total_timesteps 2212.
Path 239 | total_timesteps 2219.
Path 240 | total_timesteps 2231.
Path 241 | total_timesteps 2242.
Path 242 | total_timesteps 2248.
Path 243 | total_timesteps 2259.
Path 244 | total_timesteps 2268.
Path 245 | total_timesteps 2276.
Path 246 | total_timesteps 2284.
Path 247 | total_timesteps 2297.
Path 248 | total_timesteps 2310.
Path 249 | total_timesteps 2325.
Path 250 | total_timesteps 2335.
Path 251 | total_timesteps 2344.
Path 252 | total_timesteps 2353.
Path 253 | total_timesteps 2362.
Path 254 | total_timesteps 2376.
Path 255 | total_timesteps 2386.
Path 256 | total_timesteps 2393.
Path 257 | total_timesteps 2405.
Path 258 | total_timesteps 2412.
Path 259 | total_timesteps 2423.
Path 260 | total_timesteps 2442.
Path 261 | total_timesteps 2456.
Path 262 | total_timesteps 2463.
Path 263 | total_timesteps 2473.
Path 264 | total_timesteps 2481.
Path 265 | total_timesteps 2490.
Path 266 | total_timesteps 2499.
Path 267 | total_timesteps 2507.
Path 268 | total_timesteps 2516.
Path 269 | total_timesteps 2523.
Path 270 | total_timesteps 2530.
Path 271 | total_timesteps 2541.
Path 272 | total_timesteps 2547.
Path 273 | total_timesteps 2556.
Path 274 | total_timesteps 2566.
Path 275 | total_timesteps 2573.
Path 276 | total_timesteps 2582.
Path 277 | total_timesteps 2595.
Path 278 | total_timesteps 2608.
Path 279 | total_timesteps 2615.
Path 280 | total_timesteps 2623.
Path 281 | total_timesteps 2631.
Path 282 | total_timesteps 2637.
Path 283 | total_timesteps 2646.
Path 284 | total_timesteps 2653.
Path 285 | total_timesteps 2662.
Path 286 | total_timesteps 2675.
Path 287 | total_timesteps 2682.
Path 288 | total_timesteps 2690.
Path 289 | total_timesteps 2701.
Path 290 | total_timesteps 2707.
Path 291 | total_timesteps 2718.
Path 292 | total_timesteps 2726.
Path 293 | total_timesteps 2740.
Path 294 | total_timesteps 2750.
Path 295 | total_timesteps 2757.
Path 296 | total_timesteps 2765.
Path 297 | total_timesteps 2771.
Path 298 | total_timesteps 2780.
Path 299 | total_timesteps 2792.
Path 300 | total_timesteps 2800.
Path 301 | total_timesteps 2811.
Path 302 | total_timesteps 2819.
Path 303 | total_timesteps 2827.
Path 304 | total_timesteps 2835.
Path 305 | total_timesteps 2844.
Path 306 | total_timesteps 2851.
Path 307 | total_timesteps 2860.
Path 308 | total_timesteps 2870.
Path 309 | total_timesteps 2883.
Path 310 | total_timesteps 2892.
Path 311 | total_timesteps 2905.
Path 312 | total_timesteps 2913.
Path 313 | total_timesteps 2923.
Path 314 | total_timesteps 2931.
Path 315 | total_timesteps 2937.
Path 316 | total_timesteps 2946.
Path 317 | total_timesteps 2953.
Path 318 | total_timesteps 2961.
Path 319 | total_timesteps 2969.
Path 320 | total_timesteps 2976.
Path 321 | total_timesteps 2984.
Path 322 | total_timesteps 3001.
Path 323 | total_timesteps 3010.
Path 324 | total_timesteps 3017.
Path 325 | total_timesteps 3025.
Path 326 | total_timesteps 3035.
Path 327 | total_timesteps 3049.
Path 328 | total_timesteps 3056.
Path 329 | total_timesteps 3065.
Path 330 | total_timesteps 3074.
Path 331 | total_timesteps 3082.
Path 332 | total_timesteps 3091.
Path 333 | total_timesteps 3100.
Path 334 | total_timesteps 3108.
Path 335 | total_timesteps 3126.
Path 336 | total_timesteps 3146.
Path 337 | total_timesteps 3155.
Path 338 | total_timesteps 3165.
Path 339 | total_timesteps 3176.
Path 340 | total_timesteps 3186.
Path 341 | total_timesteps 3194.
Path 342 | total_timesteps 3204.
Path 343 | total_timesteps 3218.
Path 344 | total_timesteps 3228.
Path 345 | total_timesteps 3236.
Path 346 | total_timesteps 3243.
Path 347 | total_timesteps 3251.
Path 348 | total_timesteps 3261.
Path 349 | total_timesteps 3273.
Path 350 | total_timesteps 3284.
Path 351 | total_timesteps 3291.
Path 352 | total_timesteps 3306.
Path 353 | total_timesteps 3314.
Path 354 | total_timesteps 3322.
Path 355 | total_timesteps 3330.
Path 356 | total_timesteps 3337.
Path 357 | total_timesteps 3345.
Path 358 | total_timesteps 3353.
Path 359 | total_timesteps 3360.
Path 360 | total_timesteps 3371.
Path 361 | total_timesteps 3380.
Path 362 | total_timesteps 3388.
Path 363 | total_timesteps 3394.
Path 364 | total_timesteps 3406.
Path 365 | total_timesteps 3415.
Path 366 | total_timesteps 3427.
Path 367 | total_timesteps 3434.
Path 368 | total_timesteps 3442.
Path 369 | total_timesteps 3454.
Path 370 | total_timesteps 3464.
Path 371 | total_timesteps 3472.
Path 372 | total_timesteps 3484.
Path 373 | total_timesteps 3492.
Path 374 | total_timesteps 3500.
Path 375 | total_timesteps 3510.
Path 376 | total_timesteps 3518.
Path 377 | total_timesteps 3525.
Path 378 | total_timesteps 3538.
Path 379 | total_timesteps 3556.
Path 380 | total_timesteps 3564.
Path 381 | total_timesteps 3573.
Path 382 | total_timesteps 3583.
Path 383 | total_timesteps 3593.
Path 384 | total_timesteps 3600.
Path 385 | total_timesteps 3610.
Path 386 | total_timesteps 3624.
Path 387 | total_timesteps 3633.
Path 388 | total_timesteps 3640.
Path 389 | total_timesteps 3652.
Path 390 | total_timesteps 3664.
Path 391 | total_timesteps 3673.
Path 392 | total_timesteps 3679.
Path 393 | total_timesteps 3688.
Path 394 | total_timesteps 3700.
Path 395 | total_timesteps 3706.
Path 396 | total_timesteps 3715.
Path 397 | total_timesteps 3724.
Path 398 | total_timesteps 3734.
Path 399 | total_timesteps 3742.
Path 400 | total_timesteps 3756.
Path 401 | total_timesteps 3764.
Path 402 | total_timesteps 3774.
Path 403 | total_timesteps 3785.
Path 404 | total_timesteps 3795.
Path 405 | total_timesteps 3805.
Path 406 | total_timesteps 3814.
Path 407 | total_timesteps 3822.
Path 408 | total_timesteps 3829.
Path 409 | total_timesteps 3836.
Path 410 | total_timesteps 3846.
Path 411 | total_timesteps 3858.
Path 412 | total_timesteps 3868.
Path 413 | total_timesteps 3880.
Path 414 | total_timesteps 3891.
Path 415 | total_timesteps 3906.
Path 416 | total_timesteps 3916.
Path 417 | total_timesteps 3926.
Path 418 | total_timesteps 3934.
Path 419 | total_timesteps 3941.
Path 420 | total_timesteps 3949.
Path 421 | total_timesteps 3955.
Path 422 | total_timesteps 3962.
Path 423 | total_timesteps 3972.
Path 424 | total_timesteps 3982.
Path 425 | total_timesteps 3993.
Path 426 | total_timesteps 4000.
Path 427 | total_timesteps 4007.
Path 428 | total_timesteps 4014.
Path 429 | total_timesteps 4022.
Path 430 | total_timesteps 4030.
Path 431 | total_timesteps 4037.
Path 432 | total_timesteps 4046.
Path 433 | total_timesteps 4055.
Path 434 | total_timesteps 4066.
Path 435 | total_timesteps 4077.
Path 436 | total_timesteps 4085.
Path 437 | total_timesteps 4094.
Path 438 | total_timesteps 4105.
Path 439 | total_timesteps 4114.
Path 440 | total_timesteps 4124.
Path 441 | total_timesteps 4143.
Path 442 | total_timesteps 4152.
Path 443 | total_timesteps 4161.
Path 444 | total_timesteps 4171.
Path 445 | total_timesteps 4186.
Path 446 | total_timesteps 4195.
Path 447 | total_timesteps 4205.
Path 448 | total_timesteps 4217.
Path 449 | total_timesteps 4225.
Path 450 | total_timesteps 4239.
Path 451 | total_timesteps 4249.
Path 452 | total_timesteps 4262.
Path 453 | total_timesteps 4269.
Path 454 | total_timesteps 4276.
Path 455 | total_timesteps 4284.
Path 456 | total_timesteps 4294.
Path 457 | total_timesteps 4307.
Path 458 | total_timesteps 4315.
Path 459 | total_timesteps 4323.
Path 460 | total_timesteps 4329.
Path 461 | total_timesteps 4337.
Path 462 | total_timesteps 4349.
Path 463 | total_timesteps 4360.
Path 464 | total_timesteps 4367.
Path 465 | total_timesteps 4374.
Path 466 | total_timesteps 4382.
Path 467 | total_timesteps 4393.
Path 468 | total_timesteps 4401.
Path 469 | total_timesteps 4408.
Path 470 | total_timesteps 4416.
Path 471 | total_timesteps 4430.
Path 472 | total_timesteps 4438.
Path 473 | total_timesteps 4445.
Path 474 | total_timesteps 4452.
Path 475 | total_timesteps 4461.
Path 476 | total_timesteps 4470.
Path 477 | total_timesteps 4477.
Path 478 | total_timesteps 4485.
Path 479 | total_timesteps 4496.
Path 480 | total_timesteps 4508.
Path 481 | total_timesteps 4515.
Path 482 | total_timesteps 4522.
Path 483 | total_timesteps 4530.
Path 484 | total_timesteps 4539.
Path 485 | total_timesteps 4546.
Path 486 | total_timesteps 4557.
Path 487 | total_timesteps 4567.
Path 488 | total_timesteps 4574.
Path 489 | total_timesteps 4582.
Path 490 | total_timesteps 4591.
Path 491 | total_timesteps 4599.
Path 492 | total_timesteps 4608.
Path 493 | total_timesteps 4616.
Path 494 | total_timesteps 4638.
Path 495 | total_timesteps 4653.
Path 496 | total_timesteps 4659.
Path 497 | total_timesteps 4668.
Path 498 | total_timesteps 4679.
Path 499 | total_timesteps 4691.
Path 500 | total_timesteps 4700.
Path 501 | total_timesteps 4709.
Path 502 | total_timesteps 4717.
Path 503 | total_timesteps 4725.
Path 504 | total_timesteps 4735.
Path 505 | total_timesteps 4743.
Path 506 | total_timesteps 4754.
Path 507 | total_timesteps 4766.
Path 508 | total_timesteps 4774.
Path 509 | total_timesteps 4784.
Path 510 | total_timesteps 4796.
Path 511 | total_timesteps 4806.
Path 512 | total_timesteps 4813.
Path 513 | total_timesteps 4824.
Path 514 | total_timesteps 4833.
Path 515 | total_timesteps 4843.
Path 516 | total_timesteps 4851.
Path 517 | total_timesteps 4859.
Path 518 | total_timesteps 4866.
Path 519 | total_timesteps 4875.
Path 520 | total_timesteps 4883.
Path 521 | total_timesteps 4892.
Path 522 | total_timesteps 4902.
Path 523 | total_timesteps 4908.
Path 524 | total_timesteps 4915.
Path 525 | total_timesteps 4923.
Path 526 | total_timesteps 4936.
Path 527 | total_timesteps 4945.
Path 528 | total_timesteps 4955.
Path 529 | total_timesteps 4965.
Path 530 | total_timesteps 4974.
Path 531 | total_timesteps 4982.
Path 532 | total_timesteps 4990.
Path 533 | total_timesteps 5002.
Path 534 | total_timesteps 5010.
Path 535 | total_timesteps 5022.
Path 536 | total_timesteps 5033.
Path 537 | total_timesteps 5048.
Path 538 | total_timesteps 5055.
Path 539 | total_timesteps 5062.
Path 540 | total_timesteps 5078.
Path 541 | total_timesteps 5085.
Path 542 | total_timesteps 5095.
Path 543 | total_timesteps 5107.
Path 544 | total_timesteps 5114.
Path 545 | total_timesteps 5123.
Path 546 | total_timesteps 5131.
Path 547 | total_timesteps 5141.
Path 548 | total_timesteps 5159.
Path 549 | total_timesteps 5166.
Path 550 | total_timesteps 5176.
Path 551 | total_timesteps 5186.
Path 552 | total_timesteps 5197.
Path 553 | total_timesteps 5204.
Path 554 | total_timesteps 5213.
Path 555 | total_timesteps 5220.
Path 556 | total_timesteps 5243.
Path 557 | total_timesteps 5253.
Path 558 | total_timesteps 5260.
Path 559 | total_timesteps 5268.
Path 560 | total_timesteps 5281.
Path 561 | total_timesteps 5288.
Path 562 | total_timesteps 5299.
Path 563 | total_timesteps 5310.
Path 564 | total_timesteps 5317.
Path 565 | total_timesteps 5323.
Path 566 | total_timesteps 5329.
Path 567 | total_timesteps 5335.
Path 568 | total_timesteps 5342.
Path 569 | total_timesteps 5355.
Path 570 | total_timesteps 5363.
Path 571 | total_timesteps 5372.
Path 572 | total_timesteps 5381.
Path 573 | total_timesteps 5388.
Path 574 | total_timesteps 5398.
Path 575 | total_timesteps 5409.
Path 576 | total_timesteps 5416.
Path 577 | total_timesteps 5425.
Path 578 | total_timesteps 5433.
Path 579 | total_timesteps 5441.
Path 580 | total_timesteps 5449.
Path 581 | total_timesteps 5459.
Path 582 | total_timesteps 5466.
Path 583 | total_timesteps 5473.
Path 584 | total_timesteps 5486.
Path 585 | total_timesteps 5492.
Path 586 | total_timesteps 5502.
Path 587 | total_timesteps 5511.
Path 588 | total_timesteps 5521.
Path 589 | total_timesteps 5531.
Path 590 | total_timesteps 5540.
Path 591 | total_timesteps 5547.
Path 592 | total_timesteps 5557.
Path 593 | total_timesteps 5565.
Path 594 | total_timesteps 5580.
Path 595 | total_timesteps 5587.
Path 596 | total_timesteps 5596.
Path 597 | total_timesteps 5608.
Path 598 | total_timesteps 5616.
Path 599 | total_timesteps 5626.
Path 600 | total_timesteps 5647.
Path 601 | total_timesteps 5655.
Path 602 | total_timesteps 5667.
Path 603 | total_timesteps 5674.
Path 604 | total_timesteps 5684.
Path 605 | total_timesteps 5694.
Path 606 | total_timesteps 5702.
Path 607 | total_timesteps 5712.
Path 608 | total_timesteps 5724.
Path 609 | total_timesteps 5732.
Path 610 | total_timesteps 5740.
Path 611 | total_timesteps 5752.
Path 612 | total_timesteps 5760.
Path 613 | total_timesteps 5766.
Path 614 | total_timesteps 5778.
Path 615 | total_timesteps 5786.
Path 616 | total_timesteps 5795.
Path 617 | total_timesteps 5804.
Path 618 | total_timesteps 5812.
Path 619 | total_timesteps 5822.
Path 620 | total_timesteps 5830.
Path 621 | total_timesteps 5839.
Path 622 | total_timesteps 5848.
Path 623 | total_timesteps 5862.
Path 624 | total_timesteps 5874.
Path 625 | total_timesteps 5883.
Path 626 | total_timesteps 5894.
Path 627 | total_timesteps 5904.
Path 628 | total_timesteps 5912.
Path 629 | total_timesteps 5921.
Path 630 | total_timesteps 5929.
Path 631 | total_timesteps 5938.
Path 632 | total_timesteps 5945.
Path 633 | total_timesteps 5952.
Path 634 | total_timesteps 5966.
Path 635 | total_timesteps 5978.
Path 636 | total_timesteps 5989.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.44    |
| Iteration     | 31       |
| MaximumReturn | -0.059   |
| MinimumReturn | -23.2    |
| TotalSamples  | 132122   |
----------------------------
itr #32 | 
Fitting dynamics.
Validation loss = 0.004121534526348114
Validation loss = 0.004133437294512987
Validation loss = 0.004080536775290966
Validation loss = 0.004204725846648216
Validation loss = 0.003993821330368519
Validation loss = 0.003969206009060144
Validation loss = 0.004000302869826555
Validation loss = 0.0041087307035923
Validation loss = 0.00402219220995903
Validation loss = 0.003942988812923431
Validation loss = 0.004008326213806868
Validation loss = 0.0040536606684327126
Validation loss = 0.0041645597666502
Validation loss = 0.003939399961382151
Validation loss = 0.003993649501353502
Validation loss = 0.004098216537386179
Validation loss = 0.003975247964262962
Validation loss = 0.0039736432954669
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 37.
Path 5 | total_timesteps 44.
Path 6 | total_timesteps 52.
Path 7 | total_timesteps 61.
Path 8 | total_timesteps 70.
Path 9 | total_timesteps 77.
Path 10 | total_timesteps 84.
Path 11 | total_timesteps 91.
Path 12 | total_timesteps 102.
Path 13 | total_timesteps 108.
Path 14 | total_timesteps 116.
Path 15 | total_timesteps 124.
Path 16 | total_timesteps 133.
Path 17 | total_timesteps 142.
Path 18 | total_timesteps 152.
Path 19 | total_timesteps 161.
Path 20 | total_timesteps 172.
Path 21 | total_timesteps 180.
Path 22 | total_timesteps 188.
Path 23 | total_timesteps 199.
Path 24 | total_timesteps 209.
Path 25 | total_timesteps 217.
Path 26 | total_timesteps 226.
Path 27 | total_timesteps 234.
Path 28 | total_timesteps 241.
Path 29 | total_timesteps 252.
Path 30 | total_timesteps 263.
Path 31 | total_timesteps 270.
Path 32 | total_timesteps 282.
Path 33 | total_timesteps 289.
Path 34 | total_timesteps 300.
Path 35 | total_timesteps 309.
Path 36 | total_timesteps 317.
Path 37 | total_timesteps 324.
Path 38 | total_timesteps 334.
Path 39 | total_timesteps 346.
Path 40 | total_timesteps 352.
Path 41 | total_timesteps 359.
Path 42 | total_timesteps 367.
Path 43 | total_timesteps 375.
Path 44 | total_timesteps 392.
Path 45 | total_timesteps 399.
Path 46 | total_timesteps 408.
Path 47 | total_timesteps 415.
Path 48 | total_timesteps 426.
Path 49 | total_timesteps 435.
Path 50 | total_timesteps 442.
Path 51 | total_timesteps 448.
Path 52 | total_timesteps 460.
Path 53 | total_timesteps 470.
Path 54 | total_timesteps 480.
Path 55 | total_timesteps 492.
Path 56 | total_timesteps 500.
Path 57 | total_timesteps 512.
Path 58 | total_timesteps 528.
Path 59 | total_timesteps 535.
Path 60 | total_timesteps 541.
Path 61 | total_timesteps 549.
Path 62 | total_timesteps 561.
Path 63 | total_timesteps 570.
Path 64 | total_timesteps 579.
Path 65 | total_timesteps 594.
Path 66 | total_timesteps 601.
Path 67 | total_timesteps 614.
Path 68 | total_timesteps 623.
Path 69 | total_timesteps 630.
Path 70 | total_timesteps 639.
Path 71 | total_timesteps 646.
Path 72 | total_timesteps 654.
Path 73 | total_timesteps 663.
Path 74 | total_timesteps 672.
Path 75 | total_timesteps 680.
Path 76 | total_timesteps 689.
Path 77 | total_timesteps 700.
Path 78 | total_timesteps 711.
Path 79 | total_timesteps 722.
Path 80 | total_timesteps 731.
Path 81 | total_timesteps 743.
Path 82 | total_timesteps 754.
Path 83 | total_timesteps 763.
Path 84 | total_timesteps 772.
Path 85 | total_timesteps 779.
Path 86 | total_timesteps 786.
Path 87 | total_timesteps 796.
Path 88 | total_timesteps 808.
Path 89 | total_timesteps 814.
Path 90 | total_timesteps 825.
Path 91 | total_timesteps 837.
Path 92 | total_timesteps 847.
Path 93 | total_timesteps 856.
Path 94 | total_timesteps 863.
Path 95 | total_timesteps 875.
Path 96 | total_timesteps 886.
Path 97 | total_timesteps 897.
Path 98 | total_timesteps 904.
Path 99 | total_timesteps 910.
Path 100 | total_timesteps 919.
Path 101 | total_timesteps 928.
Path 102 | total_timesteps 937.
Path 103 | total_timesteps 946.
Path 104 | total_timesteps 955.
Path 105 | total_timesteps 963.
Path 106 | total_timesteps 970.
Path 107 | total_timesteps 980.
Path 108 | total_timesteps 994.
Path 109 | total_timesteps 1003.
Path 110 | total_timesteps 1023.
Path 111 | total_timesteps 1030.
Path 112 | total_timesteps 1037.
Path 113 | total_timesteps 1044.
Path 114 | total_timesteps 1051.
Path 115 | total_timesteps 1059.
Path 116 | total_timesteps 1068.
Path 117 | total_timesteps 1086.
Path 118 | total_timesteps 1095.
Path 119 | total_timesteps 1104.
Path 120 | total_timesteps 1112.
Path 121 | total_timesteps 1120.
Path 122 | total_timesteps 1127.
Path 123 | total_timesteps 1137.
Path 124 | total_timesteps 1144.
Path 125 | total_timesteps 1154.
Path 126 | total_timesteps 1165.
Path 127 | total_timesteps 1172.
Path 128 | total_timesteps 1179.
Path 129 | total_timesteps 1186.
Path 130 | total_timesteps 1196.
Path 131 | total_timesteps 1206.
Path 132 | total_timesteps 1217.
Path 133 | total_timesteps 1227.
Path 134 | total_timesteps 1241.
Path 135 | total_timesteps 1251.
Path 136 | total_timesteps 1262.
Path 137 | total_timesteps 1269.
Path 138 | total_timesteps 1276.
Path 139 | total_timesteps 1283.
Path 140 | total_timesteps 1292.
Path 141 | total_timesteps 1300.
Path 142 | total_timesteps 1311.
Path 143 | total_timesteps 1323.
Path 144 | total_timesteps 1331.
Path 145 | total_timesteps 1342.
Path 146 | total_timesteps 1349.
Path 147 | total_timesteps 1357.
Path 148 | total_timesteps 1365.
Path 149 | total_timesteps 1378.
Path 150 | total_timesteps 1393.
Path 151 | total_timesteps 1402.
Path 152 | total_timesteps 1410.
Path 153 | total_timesteps 1419.
Path 154 | total_timesteps 1433.
Path 155 | total_timesteps 1440.
Path 156 | total_timesteps 1455.
Path 157 | total_timesteps 1464.
Path 158 | total_timesteps 1473.
Path 159 | total_timesteps 1483.
Path 160 | total_timesteps 1491.
Path 161 | total_timesteps 1498.
Path 162 | total_timesteps 1505.
Path 163 | total_timesteps 1514.
Path 164 | total_timesteps 1528.
Path 165 | total_timesteps 1541.
Path 166 | total_timesteps 1548.
Path 167 | total_timesteps 1555.
Path 168 | total_timesteps 1563.
Path 169 | total_timesteps 1572.
Path 170 | total_timesteps 1581.
Path 171 | total_timesteps 1588.
Path 172 | total_timesteps 1596.
Path 173 | total_timesteps 1603.
Path 174 | total_timesteps 1610.
Path 175 | total_timesteps 1617.
Path 176 | total_timesteps 1628.
Path 177 | total_timesteps 1639.
Path 178 | total_timesteps 1647.
Path 179 | total_timesteps 1657.
Path 180 | total_timesteps 1666.
Path 181 | total_timesteps 1673.
Path 182 | total_timesteps 1680.
Path 183 | total_timesteps 1695.
Path 184 | total_timesteps 1712.
Path 185 | total_timesteps 1721.
Path 186 | total_timesteps 1731.
Path 187 | total_timesteps 1742.
Path 188 | total_timesteps 1750.
Path 189 | total_timesteps 1768.
Path 190 | total_timesteps 1782.
Path 191 | total_timesteps 1794.
Path 192 | total_timesteps 1802.
Path 193 | total_timesteps 1812.
Path 194 | total_timesteps 1820.
Path 195 | total_timesteps 1826.
Path 196 | total_timesteps 1833.
Path 197 | total_timesteps 1839.
Path 198 | total_timesteps 1846.
Path 199 | total_timesteps 1854.
Path 200 | total_timesteps 1863.
Path 201 | total_timesteps 1871.
Path 202 | total_timesteps 1879.
Path 203 | total_timesteps 1891.
Path 204 | total_timesteps 1899.
Path 205 | total_timesteps 1907.
Path 206 | total_timesteps 1913.
Path 207 | total_timesteps 1920.
Path 208 | total_timesteps 1928.
Path 209 | total_timesteps 1942.
Path 210 | total_timesteps 1950.
Path 211 | total_timesteps 1962.
Path 212 | total_timesteps 1973.
Path 213 | total_timesteps 1982.
Path 214 | total_timesteps 1994.
Path 215 | total_timesteps 2001.
Path 216 | total_timesteps 2007.
Path 217 | total_timesteps 2018.
Path 218 | total_timesteps 2034.
Path 219 | total_timesteps 2045.
Path 220 | total_timesteps 2054.
Path 221 | total_timesteps 2063.
Path 222 | total_timesteps 2071.
Path 223 | total_timesteps 2081.
Path 224 | total_timesteps 2107.
Path 225 | total_timesteps 2115.
Path 226 | total_timesteps 2126.
Path 227 | total_timesteps 2142.
Path 228 | total_timesteps 2149.
Path 229 | total_timesteps 2160.
Path 230 | total_timesteps 2169.
Path 231 | total_timesteps 2176.
Path 232 | total_timesteps 2190.
Path 233 | total_timesteps 2199.
Path 234 | total_timesteps 2210.
Path 235 | total_timesteps 2223.
Path 236 | total_timesteps 2232.
Path 237 | total_timesteps 2244.
Path 238 | total_timesteps 2251.
Path 239 | total_timesteps 2258.
Path 240 | total_timesteps 2266.
Path 241 | total_timesteps 2280.
Path 242 | total_timesteps 2289.
Path 243 | total_timesteps 2298.
Path 244 | total_timesteps 2304.
Path 245 | total_timesteps 2312.
Path 246 | total_timesteps 2322.
Path 247 | total_timesteps 2331.
Path 248 | total_timesteps 2340.
Path 249 | total_timesteps 2347.
Path 250 | total_timesteps 2361.
Path 251 | total_timesteps 2368.
Path 252 | total_timesteps 2379.
Path 253 | total_timesteps 2388.
Path 254 | total_timesteps 2394.
Path 255 | total_timesteps 2404.
Path 256 | total_timesteps 2411.
Path 257 | total_timesteps 2421.
Path 258 | total_timesteps 2428.
Path 259 | total_timesteps 2436.
Path 260 | total_timesteps 2445.
Path 261 | total_timesteps 2452.
Path 262 | total_timesteps 2460.
Path 263 | total_timesteps 2470.
Path 264 | total_timesteps 2478.
Path 265 | total_timesteps 2488.
Path 266 | total_timesteps 2494.
Path 267 | total_timesteps 2500.
Path 268 | total_timesteps 2507.
Path 269 | total_timesteps 2520.
Path 270 | total_timesteps 2535.
Path 271 | total_timesteps 2544.
Path 272 | total_timesteps 2553.
Path 273 | total_timesteps 2562.
Path 274 | total_timesteps 2573.
Path 275 | total_timesteps 2580.
Path 276 | total_timesteps 2592.
Path 277 | total_timesteps 2602.
Path 278 | total_timesteps 2609.
Path 279 | total_timesteps 2622.
Path 280 | total_timesteps 2632.
Path 281 | total_timesteps 2642.
Path 282 | total_timesteps 2651.
Path 283 | total_timesteps 2660.
Path 284 | total_timesteps 2674.
Path 285 | total_timesteps 2685.
Path 286 | total_timesteps 2692.
Path 287 | total_timesteps 2699.
Path 288 | total_timesteps 2706.
Path 289 | total_timesteps 2717.
Path 290 | total_timesteps 2724.
Path 291 | total_timesteps 2734.
Path 292 | total_timesteps 2744.
Path 293 | total_timesteps 2751.
Path 294 | total_timesteps 2758.
Path 295 | total_timesteps 2766.
Path 296 | total_timesteps 2776.
Path 297 | total_timesteps 2784.
Path 298 | total_timesteps 2791.
Path 299 | total_timesteps 2798.
Path 300 | total_timesteps 2806.
Path 301 | total_timesteps 2819.
Path 302 | total_timesteps 2827.
Path 303 | total_timesteps 2836.
Path 304 | total_timesteps 2845.
Path 305 | total_timesteps 2854.
Path 306 | total_timesteps 2860.
Path 307 | total_timesteps 2871.
Path 308 | total_timesteps 2878.
Path 309 | total_timesteps 2884.
Path 310 | total_timesteps 2896.
Path 311 | total_timesteps 2907.
Path 312 | total_timesteps 2913.
Path 313 | total_timesteps 2921.
Path 314 | total_timesteps 2940.
Path 315 | total_timesteps 2954.
Path 316 | total_timesteps 2962.
Path 317 | total_timesteps 2973.
Path 318 | total_timesteps 2984.
Path 319 | total_timesteps 2991.
Path 320 | total_timesteps 2999.
Path 321 | total_timesteps 3007.
Path 322 | total_timesteps 3016.
Path 323 | total_timesteps 3024.
Path 324 | total_timesteps 3032.
Path 325 | total_timesteps 3039.
Path 326 | total_timesteps 3047.
Path 327 | total_timesteps 3054.
Path 328 | total_timesteps 3061.
Path 329 | total_timesteps 3068.
Path 330 | total_timesteps 3075.
Path 331 | total_timesteps 3083.
Path 332 | total_timesteps 3091.
Path 333 | total_timesteps 3102.
Path 334 | total_timesteps 3118.
Path 335 | total_timesteps 3125.
Path 336 | total_timesteps 3131.
Path 337 | total_timesteps 3143.
Path 338 | total_timesteps 3151.
Path 339 | total_timesteps 3161.
Path 340 | total_timesteps 3170.
Path 341 | total_timesteps 3180.
Path 342 | total_timesteps 3190.
Path 343 | total_timesteps 3201.
Path 344 | total_timesteps 3214.
Path 345 | total_timesteps 3221.
Path 346 | total_timesteps 3231.
Path 347 | total_timesteps 3241.
Path 348 | total_timesteps 3251.
Path 349 | total_timesteps 3258.
Path 350 | total_timesteps 3267.
Path 351 | total_timesteps 3275.
Path 352 | total_timesteps 3282.
Path 353 | total_timesteps 3293.
Path 354 | total_timesteps 3300.
Path 355 | total_timesteps 3312.
Path 356 | total_timesteps 3318.
Path 357 | total_timesteps 3329.
Path 358 | total_timesteps 3345.
Path 359 | total_timesteps 3354.
Path 360 | total_timesteps 3361.
Path 361 | total_timesteps 3372.
Path 362 | total_timesteps 3382.
Path 363 | total_timesteps 3390.
Path 364 | total_timesteps 3397.
Path 365 | total_timesteps 3406.
Path 366 | total_timesteps 3414.
Path 367 | total_timesteps 3422.
Path 368 | total_timesteps 3431.
Path 369 | total_timesteps 3444.
Path 370 | total_timesteps 3452.
Path 371 | total_timesteps 3460.
Path 372 | total_timesteps 3475.
Path 373 | total_timesteps 3484.
Path 374 | total_timesteps 3492.
Path 375 | total_timesteps 3500.
Path 376 | total_timesteps 3510.
Path 377 | total_timesteps 3516.
Path 378 | total_timesteps 3523.
Path 379 | total_timesteps 3533.
Path 380 | total_timesteps 3541.
Path 381 | total_timesteps 3552.
Path 382 | total_timesteps 3564.
Path 383 | total_timesteps 3574.
Path 384 | total_timesteps 3581.
Path 385 | total_timesteps 3590.
Path 386 | total_timesteps 3599.
Path 387 | total_timesteps 3610.
Path 388 | total_timesteps 3620.
Path 389 | total_timesteps 3629.
Path 390 | total_timesteps 3638.
Path 391 | total_timesteps 3647.
Path 392 | total_timesteps 3665.
Path 393 | total_timesteps 3678.
Path 394 | total_timesteps 3687.
Path 395 | total_timesteps 3695.
Path 396 | total_timesteps 3704.
Path 397 | total_timesteps 3712.
Path 398 | total_timesteps 3723.
Path 399 | total_timesteps 3731.
Path 400 | total_timesteps 3740.
Path 401 | total_timesteps 3749.
Path 402 | total_timesteps 3758.
Path 403 | total_timesteps 3768.
Path 404 | total_timesteps 3775.
Path 405 | total_timesteps 3788.
Path 406 | total_timesteps 3803.
Path 407 | total_timesteps 3813.
Path 408 | total_timesteps 3825.
Path 409 | total_timesteps 3833.
Path 410 | total_timesteps 3841.
Path 411 | total_timesteps 3847.
Path 412 | total_timesteps 3857.
Path 413 | total_timesteps 3867.
Path 414 | total_timesteps 3874.
Path 415 | total_timesteps 3881.
Path 416 | total_timesteps 3889.
Path 417 | total_timesteps 3897.
Path 418 | total_timesteps 3906.
Path 419 | total_timesteps 3913.
Path 420 | total_timesteps 3925.
Path 421 | total_timesteps 3931.
Path 422 | total_timesteps 3937.
Path 423 | total_timesteps 3950.
Path 424 | total_timesteps 3958.
Path 425 | total_timesteps 3965.
Path 426 | total_timesteps 3975.
Path 427 | total_timesteps 3984.
Path 428 | total_timesteps 3991.
Path 429 | total_timesteps 3999.
Path 430 | total_timesteps 4013.
Path 431 | total_timesteps 4025.
Path 432 | total_timesteps 4032.
Path 433 | total_timesteps 4042.
Path 434 | total_timesteps 4054.
Path 435 | total_timesteps 4065.
Path 436 | total_timesteps 4071.
Path 437 | total_timesteps 4078.
Path 438 | total_timesteps 4089.
Path 439 | total_timesteps 4097.
Path 440 | total_timesteps 4105.
Path 441 | total_timesteps 4113.
Path 442 | total_timesteps 4119.
Path 443 | total_timesteps 4127.
Path 444 | total_timesteps 4136.
Path 445 | total_timesteps 4144.
Path 446 | total_timesteps 4150.
Path 447 | total_timesteps 4157.
Path 448 | total_timesteps 4165.
Path 449 | total_timesteps 4173.
Path 450 | total_timesteps 4188.
Path 451 | total_timesteps 4197.
Path 452 | total_timesteps 4205.
Path 453 | total_timesteps 4214.
Path 454 | total_timesteps 4228.
Path 455 | total_timesteps 4235.
Path 456 | total_timesteps 4253.
Path 457 | total_timesteps 4263.
Path 458 | total_timesteps 4272.
Path 459 | total_timesteps 4281.
Path 460 | total_timesteps 4290.
Path 461 | total_timesteps 4300.
Path 462 | total_timesteps 4310.
Path 463 | total_timesteps 4318.
Path 464 | total_timesteps 4328.
Path 465 | total_timesteps 4343.
Path 466 | total_timesteps 4350.
Path 467 | total_timesteps 4362.
Path 468 | total_timesteps 4373.
Path 469 | total_timesteps 4382.
Path 470 | total_timesteps 4390.
Path 471 | total_timesteps 4399.
Path 472 | total_timesteps 4407.
Path 473 | total_timesteps 4421.
Path 474 | total_timesteps 4430.
Path 475 | total_timesteps 4440.
Path 476 | total_timesteps 4448.
Path 477 | total_timesteps 4454.
Path 478 | total_timesteps 4464.
Path 479 | total_timesteps 4471.
Path 480 | total_timesteps 4478.
Path 481 | total_timesteps 4489.
Path 482 | total_timesteps 4497.
Path 483 | total_timesteps 4504.
Path 484 | total_timesteps 4511.
Path 485 | total_timesteps 4522.
Path 486 | total_timesteps 4531.
Path 487 | total_timesteps 4541.
Path 488 | total_timesteps 4551.
Path 489 | total_timesteps 4561.
Path 490 | total_timesteps 4569.
Path 491 | total_timesteps 4578.
Path 492 | total_timesteps 4587.
Path 493 | total_timesteps 4595.
Path 494 | total_timesteps 4603.
Path 495 | total_timesteps 4609.
Path 496 | total_timesteps 4616.
Path 497 | total_timesteps 4624.
Path 498 | total_timesteps 4630.
Path 499 | total_timesteps 4645.
Path 500 | total_timesteps 4656.
Path 501 | total_timesteps 4665.
Path 502 | total_timesteps 4674.
Path 503 | total_timesteps 4683.
Path 504 | total_timesteps 4692.
Path 505 | total_timesteps 4702.
Path 506 | total_timesteps 4711.
Path 507 | total_timesteps 4719.
Path 508 | total_timesteps 4726.
Path 509 | total_timesteps 4734.
Path 510 | total_timesteps 4743.
Path 511 | total_timesteps 4752.
Path 512 | total_timesteps 4761.
Path 513 | total_timesteps 4768.
Path 514 | total_timesteps 4775.
Path 515 | total_timesteps 4783.
Path 516 | total_timesteps 4790.
Path 517 | total_timesteps 4797.
Path 518 | total_timesteps 4807.
Path 519 | total_timesteps 4815.
Path 520 | total_timesteps 4825.
Path 521 | total_timesteps 4833.
Path 522 | total_timesteps 4844.
Path 523 | total_timesteps 4851.
Path 524 | total_timesteps 4863.
Path 525 | total_timesteps 4870.
Path 526 | total_timesteps 4876.
Path 527 | total_timesteps 4885.
Path 528 | total_timesteps 4899.
Path 529 | total_timesteps 4909.
Path 530 | total_timesteps 4918.
Path 531 | total_timesteps 4928.
Path 532 | total_timesteps 4934.
Path 533 | total_timesteps 4941.
Path 534 | total_timesteps 4948.
Path 535 | total_timesteps 4956.
Path 536 | total_timesteps 4967.
Path 537 | total_timesteps 4982.
Path 538 | total_timesteps 4992.
Path 539 | total_timesteps 5001.
Path 540 | total_timesteps 5011.
Path 541 | total_timesteps 5020.
Path 542 | total_timesteps 5031.
Path 543 | total_timesteps 5039.
Path 544 | total_timesteps 5046.
Path 545 | total_timesteps 5053.
Path 546 | total_timesteps 5066.
Path 547 | total_timesteps 5074.
Path 548 | total_timesteps 5082.
Path 549 | total_timesteps 5091.
Path 550 | total_timesteps 5100.
Path 551 | total_timesteps 5108.
Path 552 | total_timesteps 5115.
Path 553 | total_timesteps 5122.
Path 554 | total_timesteps 5133.
Path 555 | total_timesteps 5141.
Path 556 | total_timesteps 5151.
Path 557 | total_timesteps 5165.
Path 558 | total_timesteps 5174.
Path 559 | total_timesteps 5182.
Path 560 | total_timesteps 5194.
Path 561 | total_timesteps 5201.
Path 562 | total_timesteps 5214.
Path 563 | total_timesteps 5223.
Path 564 | total_timesteps 5234.
Path 565 | total_timesteps 5244.
Path 566 | total_timesteps 5252.
Path 567 | total_timesteps 5259.
Path 568 | total_timesteps 5269.
Path 569 | total_timesteps 5279.
Path 570 | total_timesteps 5294.
Path 571 | total_timesteps 5303.
Path 572 | total_timesteps 5309.
Path 573 | total_timesteps 5316.
Path 574 | total_timesteps 5325.
Path 575 | total_timesteps 5333.
Path 576 | total_timesteps 5342.
Path 577 | total_timesteps 5352.
Path 578 | total_timesteps 5361.
Path 579 | total_timesteps 5377.
Path 580 | total_timesteps 5385.
Path 581 | total_timesteps 5393.
Path 582 | total_timesteps 5414.
Path 583 | total_timesteps 5423.
Path 584 | total_timesteps 5430.
Path 585 | total_timesteps 5440.
Path 586 | total_timesteps 5446.
Path 587 | total_timesteps 5459.
Path 588 | total_timesteps 5468.
Path 589 | total_timesteps 5476.
Path 590 | total_timesteps 5484.
Path 591 | total_timesteps 5490.
Path 592 | total_timesteps 5498.
Path 593 | total_timesteps 5508.
Path 594 | total_timesteps 5514.
Path 595 | total_timesteps 5524.
Path 596 | total_timesteps 5541.
Path 597 | total_timesteps 5548.
Path 598 | total_timesteps 5561.
Path 599 | total_timesteps 5569.
Path 600 | total_timesteps 5577.
Path 601 | total_timesteps 5583.
Path 602 | total_timesteps 5591.
Path 603 | total_timesteps 5601.
Path 604 | total_timesteps 5611.
Path 605 | total_timesteps 5619.
Path 606 | total_timesteps 5631.
Path 607 | total_timesteps 5638.
Path 608 | total_timesteps 5647.
Path 609 | total_timesteps 5658.
Path 610 | total_timesteps 5664.
Path 611 | total_timesteps 5671.
Path 612 | total_timesteps 5687.
Path 613 | total_timesteps 5695.
Path 614 | total_timesteps 5704.
Path 615 | total_timesteps 5712.
Path 616 | total_timesteps 5721.
Path 617 | total_timesteps 5727.
Path 618 | total_timesteps 5734.
Path 619 | total_timesteps 5744.
Path 620 | total_timesteps 5755.
Path 621 | total_timesteps 5763.
Path 622 | total_timesteps 5770.
Path 623 | total_timesteps 5783.
Path 624 | total_timesteps 5791.
Path 625 | total_timesteps 5802.
Path 626 | total_timesteps 5812.
Path 627 | total_timesteps 5821.
Path 628 | total_timesteps 5829.
Path 629 | total_timesteps 5840.
Path 630 | total_timesteps 5846.
Path 631 | total_timesteps 5857.
Path 632 | total_timesteps 5870.
Path 633 | total_timesteps 5883.
Path 634 | total_timesteps 5890.
Path 635 | total_timesteps 5897.
Path 636 | total_timesteps 5910.
Path 637 | total_timesteps 5921.
Path 638 | total_timesteps 5932.
Path 639 | total_timesteps 5942.
Path 640 | total_timesteps 5948.
Path 641 | total_timesteps 5955.
Path 642 | total_timesteps 5963.
Path 643 | total_timesteps 5974.
Path 644 | total_timesteps 5982.
Path 645 | total_timesteps 5990.
Path 646 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.4     |
| Iteration     | 32       |
| MaximumReturn | -1.16    |
| MinimumReturn | -20.3    |
| TotalSamples  | 136126   |
----------------------------
