Logging to experiments/gym_fwalker2d/Wa01/Mon-07-Nov-2022-10-29-40-AM-CST_gym_fwalker2d_trpo_iteration_20_seed2231
Print configuration .....
{'env_name': 'gym_fwalker2d', 'random_seeds': [3214, 2431, 2531, 2231], 'save_variables': False, 'model_save_dir': '/tmp/gym_fwalker2d_models/', 'restore_variables': False, 'start_onpol_iter': 0, 'onpol_iters': 33, 'num_path_random': 6, 'num_path_onpol': 6, 'env_horizon': 1000, 'max_train_data': 200000, 'max_val_data': 100000, 'discard_ratio': 0.0, 'dynamics': {'pre_training': {'mode': 'intrinsic_reward', 'itr': 0, 'policy_itr': 20}, 'model': 'nn', 'ensemble': False, 'ensemble_model_count': 5, 'enable_particle_ensemble': True, 'particles': 5, 'obs_var': 1.0, 'intrinsic_reward_coeff': 1.0, 'ita': 1.0, 'mode': 'random', 'val': True, 'n_layers': 4, 'hidden_size': 1000, 'activation': 'relu', 'batch_size': 1000, 'learning_rate': 0.001, 'reg_coeff': 0.0, 'epochs': 200, 'kfac_params': {'learning_rate': 0.1, 'damping': 0.001, 'momentum': 0.9, 'kl_clip': 0.0001, 'cov_ema_decay': 0.99}}, 'policy': {'network_shape': [64, 64], 'init_logstd': 0.0, 'activation': 'tanh', 'reinitialize_every_itr': False}, 'trpo': {'horizon': 1000, 'gamma': 0.99, 'step_size': 0.01, 'iterations': 20, 'batch_size': 50000, 'gae': 0.95, 'visualization': False, 'visualize_iterations': [0]}, 'algo': 'trpo'}
Generating random rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 22.
Path 2 | total_timesteps 38.
Path 3 | total_timesteps 57.
Path 4 | total_timesteps 80.
Path 5 | total_timesteps 96.
Path 6 | total_timesteps 112.
Path 7 | total_timesteps 125.
Path 8 | total_timesteps 155.
Path 9 | total_timesteps 175.
Path 10 | total_timesteps 188.
Path 11 | total_timesteps 203.
Path 12 | total_timesteps 226.
Path 13 | total_timesteps 251.
Path 14 | total_timesteps 279.
Path 15 | total_timesteps 304.
Path 16 | total_timesteps 321.
Path 17 | total_timesteps 340.
Path 18 | total_timesteps 378.
Path 19 | total_timesteps 403.
Path 20 | total_timesteps 414.
Path 21 | total_timesteps 436.
Path 22 | total_timesteps 467.
Path 23 | total_timesteps 482.
Path 24 | total_timesteps 511.
Path 25 | total_timesteps 553.
Path 26 | total_timesteps 575.
Path 27 | total_timesteps 595.
Path 28 | total_timesteps 608.
Path 29 | total_timesteps 621.
Path 30 | total_timesteps 653.
Path 31 | total_timesteps 705.
Path 32 | total_timesteps 732.
Path 33 | total_timesteps 762.
Path 34 | total_timesteps 788.
Path 35 | total_timesteps 808.
Path 36 | total_timesteps 823.
Path 37 | total_timesteps 840.
Path 38 | total_timesteps 863.
Path 39 | total_timesteps 883.
Path 40 | total_timesteps 907.
Path 41 | total_timesteps 921.
Path 42 | total_timesteps 940.
Path 43 | total_timesteps 970.
Path 44 | total_timesteps 993.
Path 45 | total_timesteps 1013.
Path 46 | total_timesteps 1029.
Path 47 | total_timesteps 1058.
Path 48 | total_timesteps 1072.
Path 49 | total_timesteps 1081.
Path 50 | total_timesteps 1109.
Path 51 | total_timesteps 1123.
Path 52 | total_timesteps 1165.
Path 53 | total_timesteps 1202.
Path 54 | total_timesteps 1234.
Path 55 | total_timesteps 1246.
Path 56 | total_timesteps 1277.
Path 57 | total_timesteps 1299.
Path 58 | total_timesteps 1313.
Path 59 | total_timesteps 1336.
Path 60 | total_timesteps 1369.
Path 61 | total_timesteps 1386.
Path 62 | total_timesteps 1398.
Path 63 | total_timesteps 1414.
Path 64 | total_timesteps 1430.
Path 65 | total_timesteps 1448.
Path 66 | total_timesteps 1482.
Path 67 | total_timesteps 1509.
Path 68 | total_timesteps 1533.
Path 69 | total_timesteps 1551.
Path 70 | total_timesteps 1605.
Path 71 | total_timesteps 1613.
Path 72 | total_timesteps 1630.
Path 73 | total_timesteps 1654.
Path 74 | total_timesteps 1670.
Path 75 | total_timesteps 1681.
Path 76 | total_timesteps 1706.
Path 77 | total_timesteps 1719.
Path 78 | total_timesteps 1750.
Path 79 | total_timesteps 1772.
Path 80 | total_timesteps 1787.
Path 81 | total_timesteps 1809.
Path 82 | total_timesteps 1835.
Path 83 | total_timesteps 1851.
Path 84 | total_timesteps 1866.
Path 85 | total_timesteps 1908.
Path 86 | total_timesteps 1918.
Path 87 | total_timesteps 1938.
Path 88 | total_timesteps 1959.
Path 89 | total_timesteps 1981.
Path 90 | total_timesteps 2014.
Path 91 | total_timesteps 2038.
Path 92 | total_timesteps 2062.
Path 93 | total_timesteps 2079.
Path 94 | total_timesteps 2106.
Path 95 | total_timesteps 2115.
Path 96 | total_timesteps 2131.
Path 97 | total_timesteps 2141.
Path 98 | total_timesteps 2151.
Path 99 | total_timesteps 2176.
Path 100 | total_timesteps 2201.
Path 101 | total_timesteps 2225.
Path 102 | total_timesteps 2241.
Path 103 | total_timesteps 2260.
Path 104 | total_timesteps 2299.
Path 105 | total_timesteps 2327.
Path 106 | total_timesteps 2346.
Path 107 | total_timesteps 2390.
Path 108 | total_timesteps 2411.
Path 109 | total_timesteps 2441.
Path 110 | total_timesteps 2458.
Path 111 | total_timesteps 2489.
Path 112 | total_timesteps 2507.
Path 113 | total_timesteps 2533.
Path 114 | total_timesteps 2554.
Path 115 | total_timesteps 2570.
Path 116 | total_timesteps 2603.
Path 117 | total_timesteps 2619.
Path 118 | total_timesteps 2631.
Path 119 | total_timesteps 2652.
Path 120 | total_timesteps 2668.
Path 121 | total_timesteps 2679.
Path 122 | total_timesteps 2702.
Path 123 | total_timesteps 2736.
Path 124 | total_timesteps 2751.
Path 125 | total_timesteps 2768.
Path 126 | total_timesteps 2779.
Path 127 | total_timesteps 2801.
Path 128 | total_timesteps 2812.
Path 129 | total_timesteps 2827.
Path 130 | total_timesteps 2837.
Path 131 | total_timesteps 2851.
Path 132 | total_timesteps 2888.
Path 133 | total_timesteps 2913.
Path 134 | total_timesteps 2948.
Path 135 | total_timesteps 2971.
Path 136 | total_timesteps 2984.
Path 137 | total_timesteps 3002.
Path 138 | total_timesteps 3011.
Path 139 | total_timesteps 3021.
Path 140 | total_timesteps 3044.
Path 141 | total_timesteps 3076.
Path 142 | total_timesteps 3109.
Path 143 | total_timesteps 3130.
Path 144 | total_timesteps 3147.
Path 145 | total_timesteps 3173.
Path 146 | total_timesteps 3191.
Path 147 | total_timesteps 3203.
Path 148 | total_timesteps 3228.
Path 149 | total_timesteps 3243.
Path 150 | total_timesteps 3273.
Path 151 | total_timesteps 3295.
Path 152 | total_timesteps 3318.
Path 153 | total_timesteps 3334.
Path 154 | total_timesteps 3367.
Path 155 | total_timesteps 3420.
Path 156 | total_timesteps 3446.
Path 157 | total_timesteps 3471.
Path 158 | total_timesteps 3483.
Path 159 | total_timesteps 3497.
Path 160 | total_timesteps 3512.
Path 161 | total_timesteps 3537.
Path 162 | total_timesteps 3553.
Path 163 | total_timesteps 3573.
Path 164 | total_timesteps 3600.
Path 165 | total_timesteps 3618.
Path 166 | total_timesteps 3647.
Path 167 | total_timesteps 3661.
Path 168 | total_timesteps 3688.
Path 169 | total_timesteps 3710.
Path 170 | total_timesteps 3725.
Path 171 | total_timesteps 3752.
Path 172 | total_timesteps 3769.
Path 173 | total_timesteps 3794.
Path 174 | total_timesteps 3822.
Path 175 | total_timesteps 3833.
Path 176 | total_timesteps 3864.
Path 177 | total_timesteps 3879.
Path 178 | total_timesteps 3898.
Path 179 | total_timesteps 3912.
Path 180 | total_timesteps 3942.
Path 181 | total_timesteps 3959.
Path 182 | total_timesteps 3971.
Path 183 | total_timesteps 4000.
Path 184 | total_timesteps 4036.
Path 185 | total_timesteps 4060.
Path 186 | total_timesteps 4073.
Path 187 | total_timesteps 4098.
Path 188 | total_timesteps 4114.
Path 189 | total_timesteps 4140.
Path 190 | total_timesteps 4165.
Path 191 | total_timesteps 4179.
Path 192 | total_timesteps 4201.
Path 193 | total_timesteps 4222.
Path 194 | total_timesteps 4242.
Path 195 | total_timesteps 4265.
Path 196 | total_timesteps 4284.
Path 197 | total_timesteps 4313.
Path 198 | total_timesteps 4331.
Path 199 | total_timesteps 4341.
Path 200 | total_timesteps 4369.
Path 201 | total_timesteps 4379.
Path 202 | total_timesteps 4399.
Path 203 | total_timesteps 4412.
Path 204 | total_timesteps 4437.
Path 205 | total_timesteps 4449.
Path 206 | total_timesteps 4472.
Path 207 | total_timesteps 4497.
Path 208 | total_timesteps 4521.
Path 209 | total_timesteps 4548.
Path 210 | total_timesteps 4561.
Path 211 | total_timesteps 4582.
Path 212 | total_timesteps 4610.
Path 213 | total_timesteps 4620.
Path 214 | total_timesteps 4640.
Path 215 | total_timesteps 4653.
Path 216 | total_timesteps 4665.
Path 217 | total_timesteps 4688.
Path 218 | total_timesteps 4714.
Path 219 | total_timesteps 4747.
Path 220 | total_timesteps 4772.
Path 221 | total_timesteps 4823.
Path 222 | total_timesteps 4859.
Path 223 | total_timesteps 4871.
Path 224 | total_timesteps 4891.
Path 225 | total_timesteps 4906.
Path 226 | total_timesteps 4927.
Path 227 | total_timesteps 4954.
Path 228 | total_timesteps 4973.
Path 229 | total_timesteps 5003.
Path 230 | total_timesteps 5022.
Path 231 | total_timesteps 5037.
Path 232 | total_timesteps 5071.
Path 233 | total_timesteps 5089.
Path 234 | total_timesteps 5115.
Path 235 | total_timesteps 5134.
Path 236 | total_timesteps 5165.
Path 237 | total_timesteps 5184.
Path 238 | total_timesteps 5198.
Path 239 | total_timesteps 5218.
Path 240 | total_timesteps 5253.
Path 241 | total_timesteps 5274.
Path 242 | total_timesteps 5291.
Path 243 | total_timesteps 5313.
Path 244 | total_timesteps 5326.
Path 245 | total_timesteps 5343.
Path 246 | total_timesteps 5358.
Path 247 | total_timesteps 5377.
Path 248 | total_timesteps 5407.
Path 249 | total_timesteps 5422.
Path 250 | total_timesteps 5435.
Path 251 | total_timesteps 5483.
Path 252 | total_timesteps 5499.
Path 253 | total_timesteps 5529.
Path 254 | total_timesteps 5545.
Path 255 | total_timesteps 5578.
Path 256 | total_timesteps 5605.
Path 257 | total_timesteps 5622.
Path 258 | total_timesteps 5635.
Path 259 | total_timesteps 5656.
Path 260 | total_timesteps 5673.
Path 261 | total_timesteps 5700.
Path 262 | total_timesteps 5707.
Path 263 | total_timesteps 5726.
Path 264 | total_timesteps 5770.
Path 265 | total_timesteps 5784.
Path 266 | total_timesteps 5797.
Path 267 | total_timesteps 5815.
Path 268 | total_timesteps 5834.
Path 269 | total_timesteps 5849.
Path 270 | total_timesteps 5878.
Path 271 | total_timesteps 5888.
Path 272 | total_timesteps 5902.
Path 273 | total_timesteps 5921.
Path 274 | total_timesteps 5943.
Path 275 | total_timesteps 5966.
Path 276 | total_timesteps 5976.
Done generating random rollouts.
Creating normalization for training data.
Done creating normalization for training data.
Train dynamics model with intrinsic reward only? False
Pre-training enabled. Using only intrinsic reward.
Pre-training dynamics model for 0 iterations...
Done pre-training dynamics model.
Using external reward only.
itr #0 | 
Fitting dynamics.
Validation loss = 0.3883135914802551
Validation loss = 0.13203635811805725
Validation loss = 0.09305402636528015
Validation loss = 0.07654979079961777
Validation loss = 0.07048432528972626
Validation loss = 0.06683618575334549
Validation loss = 0.062073905020952225
Validation loss = 0.05730670690536499
Validation loss = 0.06094241142272949
Validation loss = 0.05498387664556503
Validation loss = 0.05328724533319473
Validation loss = 0.04875441640615463
Validation loss = 0.05917699635028839
Validation loss = 0.0472913533449173
Validation loss = 0.048044923692941666
Validation loss = 0.04477783665060997
Validation loss = 0.04951504245400429
Validation loss = 0.04758576303720474
Validation loss = 0.05619291961193085
Validation loss = 0.04844515025615692
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 34.
Path 2 | total_timesteps 66.
Path 3 | total_timesteps 87.
Path 4 | total_timesteps 117.
Path 5 | total_timesteps 131.
Path 6 | total_timesteps 153.
Path 7 | total_timesteps 162.
Path 8 | total_timesteps 182.
Path 9 | total_timesteps 200.
Path 10 | total_timesteps 215.
Path 11 | total_timesteps 236.
Path 12 | total_timesteps 253.
Path 13 | total_timesteps 282.
Path 14 | total_timesteps 298.
Path 15 | total_timesteps 321.
Path 16 | total_timesteps 339.
Path 17 | total_timesteps 353.
Path 18 | total_timesteps 371.
Path 19 | total_timesteps 419.
Path 20 | total_timesteps 444.
Path 21 | total_timesteps 454.
Path 22 | total_timesteps 469.
Path 23 | total_timesteps 497.
Path 24 | total_timesteps 509.
Path 25 | total_timesteps 528.
Path 26 | total_timesteps 563.
Path 27 | total_timesteps 576.
Path 28 | total_timesteps 591.
Path 29 | total_timesteps 609.
Path 30 | total_timesteps 626.
Path 31 | total_timesteps 637.
Path 32 | total_timesteps 669.
Path 33 | total_timesteps 690.
Path 34 | total_timesteps 715.
Path 35 | total_timesteps 727.
Path 36 | total_timesteps 749.
Path 37 | total_timesteps 784.
Path 38 | total_timesteps 799.
Path 39 | total_timesteps 817.
Path 40 | total_timesteps 830.
Path 41 | total_timesteps 839.
Path 42 | total_timesteps 862.
Path 43 | total_timesteps 881.
Path 44 | total_timesteps 900.
Path 45 | total_timesteps 912.
Path 46 | total_timesteps 959.
Path 47 | total_timesteps 975.
Path 48 | total_timesteps 1013.
Path 49 | total_timesteps 1045.
Path 50 | total_timesteps 1068.
Path 51 | total_timesteps 1106.
Path 52 | total_timesteps 1118.
Path 53 | total_timesteps 1152.
Path 54 | total_timesteps 1167.
Path 55 | total_timesteps 1189.
Path 56 | total_timesteps 1218.
Path 57 | total_timesteps 1242.
Path 58 | total_timesteps 1274.
Path 59 | total_timesteps 1295.
Path 60 | total_timesteps 1326.
Path 61 | total_timesteps 1342.
Path 62 | total_timesteps 1376.
Path 63 | total_timesteps 1388.
Path 64 | total_timesteps 1413.
Path 65 | total_timesteps 1441.
Path 66 | total_timesteps 1506.
Path 67 | total_timesteps 1521.
Path 68 | total_timesteps 1539.
Path 69 | total_timesteps 1575.
Path 70 | total_timesteps 1596.
Path 71 | total_timesteps 1606.
Path 72 | total_timesteps 1625.
Path 73 | total_timesteps 1645.
Path 74 | total_timesteps 1666.
Path 75 | total_timesteps 1695.
Path 76 | total_timesteps 1714.
Path 77 | total_timesteps 1747.
Path 78 | total_timesteps 1762.
Path 79 | total_timesteps 1773.
Path 80 | total_timesteps 1786.
Path 81 | total_timesteps 1798.
Path 82 | total_timesteps 1814.
Path 83 | total_timesteps 1833.
Path 84 | total_timesteps 1851.
Path 85 | total_timesteps 1866.
Path 86 | total_timesteps 1889.
Path 87 | total_timesteps 1920.
Path 88 | total_timesteps 1941.
Path 89 | total_timesteps 1952.
Path 90 | total_timesteps 1970.
Path 91 | total_timesteps 1979.
Path 92 | total_timesteps 1995.
Path 93 | total_timesteps 2009.
Path 94 | total_timesteps 2026.
Path 95 | total_timesteps 2036.
Path 96 | total_timesteps 2097.
Path 97 | total_timesteps 2131.
Path 98 | total_timesteps 2150.
Path 99 | total_timesteps 2161.
Path 100 | total_timesteps 2174.
Path 101 | total_timesteps 2201.
Path 102 | total_timesteps 2213.
Path 103 | total_timesteps 2240.
Path 104 | total_timesteps 2251.
Path 105 | total_timesteps 2268.
Path 106 | total_timesteps 2294.
Path 107 | total_timesteps 2323.
Path 108 | total_timesteps 2331.
Path 109 | total_timesteps 2344.
Path 110 | total_timesteps 2360.
Path 111 | total_timesteps 2377.
Path 112 | total_timesteps 2391.
Path 113 | total_timesteps 2419.
Path 114 | total_timesteps 2440.
Path 115 | total_timesteps 2472.
Path 116 | total_timesteps 2493.
Path 117 | total_timesteps 2507.
Path 118 | total_timesteps 2535.
Path 119 | total_timesteps 2545.
Path 120 | total_timesteps 2563.
Path 121 | total_timesteps 2589.
Path 122 | total_timesteps 2607.
Path 123 | total_timesteps 2632.
Path 124 | total_timesteps 2684.
Path 125 | total_timesteps 2696.
Path 126 | total_timesteps 2732.
Path 127 | total_timesteps 2750.
Path 128 | total_timesteps 2769.
Path 129 | total_timesteps 2809.
Path 130 | total_timesteps 2820.
Path 131 | total_timesteps 2841.
Path 132 | total_timesteps 2862.
Path 133 | total_timesteps 2892.
Path 134 | total_timesteps 2928.
Path 135 | total_timesteps 2951.
Path 136 | total_timesteps 2965.
Path 137 | total_timesteps 2992.
Path 138 | total_timesteps 3009.
Path 139 | total_timesteps 3028.
Path 140 | total_timesteps 3042.
Path 141 | total_timesteps 3086.
Path 142 | total_timesteps 3103.
Path 143 | total_timesteps 3111.
Path 144 | total_timesteps 3127.
Path 145 | total_timesteps 3151.
Path 146 | total_timesteps 3159.
Path 147 | total_timesteps 3183.
Path 148 | total_timesteps 3217.
Path 149 | total_timesteps 3244.
Path 150 | total_timesteps 3259.
Path 151 | total_timesteps 3304.
Path 152 | total_timesteps 3313.
Path 153 | total_timesteps 3329.
Path 154 | total_timesteps 3349.
Path 155 | total_timesteps 3365.
Path 156 | total_timesteps 3375.
Path 157 | total_timesteps 3391.
Path 158 | total_timesteps 3416.
Path 159 | total_timesteps 3450.
Path 160 | total_timesteps 3475.
Path 161 | total_timesteps 3517.
Path 162 | total_timesteps 3547.
Path 163 | total_timesteps 3565.
Path 164 | total_timesteps 3576.
Path 165 | total_timesteps 3627.
Path 166 | total_timesteps 3640.
Path 167 | total_timesteps 3653.
Path 168 | total_timesteps 3669.
Path 169 | total_timesteps 3684.
Path 170 | total_timesteps 3702.
Path 171 | total_timesteps 3723.
Path 172 | total_timesteps 3740.
Path 173 | total_timesteps 3767.
Path 174 | total_timesteps 3782.
Path 175 | total_timesteps 3814.
Path 176 | total_timesteps 3835.
Path 177 | total_timesteps 3857.
Path 178 | total_timesteps 3869.
Path 179 | total_timesteps 3895.
Path 180 | total_timesteps 3937.
Path 181 | total_timesteps 3951.
Path 182 | total_timesteps 3963.
Path 183 | total_timesteps 3989.
Path 184 | total_timesteps 4012.
Path 185 | total_timesteps 4036.
Path 186 | total_timesteps 4069.
Path 187 | total_timesteps 4095.
Path 188 | total_timesteps 4110.
Path 189 | total_timesteps 4123.
Path 190 | total_timesteps 4142.
Path 191 | total_timesteps 4189.
Path 192 | total_timesteps 4209.
Path 193 | total_timesteps 4228.
Path 194 | total_timesteps 4252.
Path 195 | total_timesteps 4265.
Path 196 | total_timesteps 4308.
Path 197 | total_timesteps 4332.
Path 198 | total_timesteps 4347.
Path 199 | total_timesteps 4366.
Path 200 | total_timesteps 4381.
Path 201 | total_timesteps 4399.
Path 202 | total_timesteps 4411.
Path 203 | total_timesteps 4429.
Path 204 | total_timesteps 4463.
Path 205 | total_timesteps 4484.
Path 206 | total_timesteps 4516.
Path 207 | total_timesteps 4531.
Path 208 | total_timesteps 4544.
Path 209 | total_timesteps 4567.
Path 210 | total_timesteps 4585.
Path 211 | total_timesteps 4613.
Path 212 | total_timesteps 4637.
Path 213 | total_timesteps 4648.
Path 214 | total_timesteps 4661.
Path 215 | total_timesteps 4673.
Path 216 | total_timesteps 4689.
Path 217 | total_timesteps 4701.
Path 218 | total_timesteps 4721.
Path 219 | total_timesteps 4733.
Path 220 | total_timesteps 4758.
Path 221 | total_timesteps 4773.
Path 222 | total_timesteps 4793.
Path 223 | total_timesteps 4805.
Path 224 | total_timesteps 4823.
Path 225 | total_timesteps 4850.
Path 226 | total_timesteps 4887.
Path 227 | total_timesteps 4912.
Path 228 | total_timesteps 4947.
Path 229 | total_timesteps 4958.
Path 230 | total_timesteps 4973.
Path 231 | total_timesteps 4991.
Path 232 | total_timesteps 5004.
Path 233 | total_timesteps 5035.
Path 234 | total_timesteps 5062.
Path 235 | total_timesteps 5082.
Path 236 | total_timesteps 5104.
Path 237 | total_timesteps 5113.
Path 238 | total_timesteps 5149.
Path 239 | total_timesteps 5162.
Path 240 | total_timesteps 5172.
Path 241 | total_timesteps 5187.
Path 242 | total_timesteps 5199.
Path 243 | total_timesteps 5225.
Path 244 | total_timesteps 5238.
Path 245 | total_timesteps 5248.
Path 246 | total_timesteps 5263.
Path 247 | total_timesteps 5284.
Path 248 | total_timesteps 5309.
Path 249 | total_timesteps 5324.
Path 250 | total_timesteps 5341.
Path 251 | total_timesteps 5362.
Path 252 | total_timesteps 5376.
Path 253 | total_timesteps 5392.
Path 254 | total_timesteps 5401.
Path 255 | total_timesteps 5424.
Path 256 | total_timesteps 5437.
Path 257 | total_timesteps 5462.
Path 258 | total_timesteps 5478.
Path 259 | total_timesteps 5488.
Path 260 | total_timesteps 5508.
Path 261 | total_timesteps 5523.
Path 262 | total_timesteps 5533.
Path 263 | total_timesteps 5555.
Path 264 | total_timesteps 5568.
Path 265 | total_timesteps 5581.
Path 266 | total_timesteps 5594.
Path 267 | total_timesteps 5613.
Path 268 | total_timesteps 5644.
Path 269 | total_timesteps 5658.
Path 270 | total_timesteps 5679.
Path 271 | total_timesteps 5701.
Path 272 | total_timesteps 5714.
Path 273 | total_timesteps 5723.
Path 274 | total_timesteps 5740.
Path 275 | total_timesteps 5768.
Path 276 | total_timesteps 5806.
Path 277 | total_timesteps 5827.
Path 278 | total_timesteps 5837.
Path 279 | total_timesteps 5869.
Path 280 | total_timesteps 5889.
Path 281 | total_timesteps 5904.
Path 282 | total_timesteps 5920.
Path 283 | total_timesteps 5936.
Path 284 | total_timesteps 5955.
Path 285 | total_timesteps 5974.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.13    |
| Iteration     | 0        |
| MaximumReturn | 15.1     |
| MinimumReturn | -25.9    |
| TotalSamples  | 8015     |
----------------------------
itr #1 | 
Fitting dynamics.
Validation loss = 0.08695828169584274
Validation loss = 0.06430612504482269
Validation loss = 0.051649268716573715
Validation loss = 0.05458884686231613
Validation loss = 0.04461842030286789
Validation loss = 0.047097936272621155
Validation loss = 0.04602634161710739
Validation loss = 0.044165827333927155
Validation loss = 0.040948402136564255
Validation loss = 0.04213937371969223
Validation loss = 0.04053756594657898
Validation loss = 0.05700884014368057
Validation loss = 0.037435002624988556
Validation loss = 0.03905479609966278
Validation loss = 0.03614727780222893
Validation loss = 0.039424698799848557
Validation loss = 0.0376017689704895
Validation loss = 0.03629573807120323
Validation loss = 0.037225391715765
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 57.
Path 4 | total_timesteps 74.
Path 5 | total_timesteps 83.
Path 6 | total_timesteps 105.
Path 7 | total_timesteps 127.
Path 8 | total_timesteps 138.
Path 9 | total_timesteps 162.
Path 10 | total_timesteps 196.
Path 11 | total_timesteps 205.
Path 12 | total_timesteps 223.
Path 13 | total_timesteps 243.
Path 14 | total_timesteps 260.
Path 15 | total_timesteps 276.
Path 16 | total_timesteps 298.
Path 17 | total_timesteps 309.
Path 18 | total_timesteps 321.
Path 19 | total_timesteps 335.
Path 20 | total_timesteps 350.
Path 21 | total_timesteps 366.
Path 22 | total_timesteps 379.
Path 23 | total_timesteps 426.
Path 24 | total_timesteps 458.
Path 25 | total_timesteps 482.
Path 26 | total_timesteps 516.
Path 27 | total_timesteps 530.
Path 28 | total_timesteps 542.
Path 29 | total_timesteps 555.
Path 30 | total_timesteps 573.
Path 31 | total_timesteps 591.
Path 32 | total_timesteps 609.
Path 33 | total_timesteps 634.
Path 34 | total_timesteps 671.
Path 35 | total_timesteps 696.
Path 36 | total_timesteps 722.
Path 37 | total_timesteps 738.
Path 38 | total_timesteps 753.
Path 39 | total_timesteps 773.
Path 40 | total_timesteps 792.
Path 41 | total_timesteps 815.
Path 42 | total_timesteps 836.
Path 43 | total_timesteps 873.
Path 44 | total_timesteps 887.
Path 45 | total_timesteps 914.
Path 46 | total_timesteps 926.
Path 47 | total_timesteps 953.
Path 48 | total_timesteps 974.
Path 49 | total_timesteps 989.
Path 50 | total_timesteps 1000.
Path 51 | total_timesteps 1009.
Path 52 | total_timesteps 1040.
Path 53 | total_timesteps 1062.
Path 54 | total_timesteps 1074.
Path 55 | total_timesteps 1087.
Path 56 | total_timesteps 1102.
Path 57 | total_timesteps 1116.
Path 58 | total_timesteps 1129.
Path 59 | total_timesteps 1149.
Path 60 | total_timesteps 1166.
Path 61 | total_timesteps 1182.
Path 62 | total_timesteps 1200.
Path 63 | total_timesteps 1216.
Path 64 | total_timesteps 1225.
Path 65 | total_timesteps 1250.
Path 66 | total_timesteps 1282.
Path 67 | total_timesteps 1295.
Path 68 | total_timesteps 1321.
Path 69 | total_timesteps 1343.
Path 70 | total_timesteps 1351.
Path 71 | total_timesteps 1371.
Path 72 | total_timesteps 1408.
Path 73 | total_timesteps 1420.
Path 74 | total_timesteps 1434.
Path 75 | total_timesteps 1472.
Path 76 | total_timesteps 1483.
Path 77 | total_timesteps 1493.
Path 78 | total_timesteps 1508.
Path 79 | total_timesteps 1519.
Path 80 | total_timesteps 1533.
Path 81 | total_timesteps 1551.
Path 82 | total_timesteps 1575.
Path 83 | total_timesteps 1587.
Path 84 | total_timesteps 1608.
Path 85 | total_timesteps 1619.
Path 86 | total_timesteps 1647.
Path 87 | total_timesteps 1672.
Path 88 | total_timesteps 1720.
Path 89 | total_timesteps 1738.
Path 90 | total_timesteps 1759.
Path 91 | total_timesteps 1774.
Path 92 | total_timesteps 1789.
Path 93 | total_timesteps 1804.
Path 94 | total_timesteps 1825.
Path 95 | total_timesteps 1839.
Path 96 | total_timesteps 1853.
Path 97 | total_timesteps 1864.
Path 98 | total_timesteps 1875.
Path 99 | total_timesteps 1891.
Path 100 | total_timesteps 1907.
Path 101 | total_timesteps 1929.
Path 102 | total_timesteps 1956.
Path 103 | total_timesteps 1979.
Path 104 | total_timesteps 1988.
Path 105 | total_timesteps 2003.
Path 106 | total_timesteps 2022.
Path 107 | total_timesteps 2052.
Path 108 | total_timesteps 2068.
Path 109 | total_timesteps 2090.
Path 110 | total_timesteps 2101.
Path 111 | total_timesteps 2115.
Path 112 | total_timesteps 2129.
Path 113 | total_timesteps 2142.
Path 114 | total_timesteps 2156.
Path 115 | total_timesteps 2169.
Path 116 | total_timesteps 2184.
Path 117 | total_timesteps 2197.
Path 118 | total_timesteps 2219.
Path 119 | total_timesteps 2252.
Path 120 | total_timesteps 2271.
Path 121 | total_timesteps 2284.
Path 122 | total_timesteps 2318.
Path 123 | total_timesteps 2332.
Path 124 | total_timesteps 2360.
Path 125 | total_timesteps 2372.
Path 126 | total_timesteps 2388.
Path 127 | total_timesteps 2410.
Path 128 | total_timesteps 2431.
Path 129 | total_timesteps 2455.
Path 130 | total_timesteps 2477.
Path 131 | total_timesteps 2510.
Path 132 | total_timesteps 2535.
Path 133 | total_timesteps 2545.
Path 134 | total_timesteps 2558.
Path 135 | total_timesteps 2589.
Path 136 | total_timesteps 2615.
Path 137 | total_timesteps 2639.
Path 138 | total_timesteps 2672.
Path 139 | total_timesteps 2687.
Path 140 | total_timesteps 2709.
Path 141 | total_timesteps 2726.
Path 142 | total_timesteps 2754.
Path 143 | total_timesteps 2772.
Path 144 | total_timesteps 2798.
Path 145 | total_timesteps 2825.
Path 146 | total_timesteps 2840.
Path 147 | total_timesteps 2851.
Path 148 | total_timesteps 2870.
Path 149 | total_timesteps 2884.
Path 150 | total_timesteps 2902.
Path 151 | total_timesteps 2921.
Path 152 | total_timesteps 2946.
Path 153 | total_timesteps 2956.
Path 154 | total_timesteps 2971.
Path 155 | total_timesteps 2990.
Path 156 | total_timesteps 3005.
Path 157 | total_timesteps 3021.
Path 158 | total_timesteps 3031.
Path 159 | total_timesteps 3052.
Path 160 | total_timesteps 3064.
Path 161 | total_timesteps 3085.
Path 162 | total_timesteps 3105.
Path 163 | total_timesteps 3119.
Path 164 | total_timesteps 3134.
Path 165 | total_timesteps 3151.
Path 166 | total_timesteps 3179.
Path 167 | total_timesteps 3197.
Path 168 | total_timesteps 3210.
Path 169 | total_timesteps 3229.
Path 170 | total_timesteps 3249.
Path 171 | total_timesteps 3264.
Path 172 | total_timesteps 3286.
Path 173 | total_timesteps 3310.
Path 174 | total_timesteps 3324.
Path 175 | total_timesteps 3341.
Path 176 | total_timesteps 3364.
Path 177 | total_timesteps 3378.
Path 178 | total_timesteps 3388.
Path 179 | total_timesteps 3404.
Path 180 | total_timesteps 3417.
Path 181 | total_timesteps 3430.
Path 182 | total_timesteps 3468.
Path 183 | total_timesteps 3485.
Path 184 | total_timesteps 3505.
Path 185 | total_timesteps 3518.
Path 186 | total_timesteps 3531.
Path 187 | total_timesteps 3545.
Path 188 | total_timesteps 3564.
Path 189 | total_timesteps 3586.
Path 190 | total_timesteps 3610.
Path 191 | total_timesteps 3625.
Path 192 | total_timesteps 3647.
Path 193 | total_timesteps 3663.
Path 194 | total_timesteps 3683.
Path 195 | total_timesteps 3696.
Path 196 | total_timesteps 3716.
Path 197 | total_timesteps 3730.
Path 198 | total_timesteps 3742.
Path 199 | total_timesteps 3783.
Path 200 | total_timesteps 3798.
Path 201 | total_timesteps 3812.
Path 202 | total_timesteps 3831.
Path 203 | total_timesteps 3839.
Path 204 | total_timesteps 3847.
Path 205 | total_timesteps 3865.
Path 206 | total_timesteps 3893.
Path 207 | total_timesteps 3913.
Path 208 | total_timesteps 3926.
Path 209 | total_timesteps 3979.
Path 210 | total_timesteps 3986.
Path 211 | total_timesteps 3996.
Path 212 | total_timesteps 4009.
Path 213 | total_timesteps 4023.
Path 214 | total_timesteps 4051.
Path 215 | total_timesteps 4077.
Path 216 | total_timesteps 4094.
Path 217 | total_timesteps 4114.
Path 218 | total_timesteps 4131.
Path 219 | total_timesteps 4155.
Path 220 | total_timesteps 4168.
Path 221 | total_timesteps 4191.
Path 222 | total_timesteps 4198.
Path 223 | total_timesteps 4215.
Path 224 | total_timesteps 4224.
Path 225 | total_timesteps 4241.
Path 226 | total_timesteps 4259.
Path 227 | total_timesteps 4277.
Path 228 | total_timesteps 4304.
Path 229 | total_timesteps 4317.
Path 230 | total_timesteps 4330.
Path 231 | total_timesteps 4353.
Path 232 | total_timesteps 4369.
Path 233 | total_timesteps 4392.
Path 234 | total_timesteps 4416.
Path 235 | total_timesteps 4430.
Path 236 | total_timesteps 4453.
Path 237 | total_timesteps 4469.
Path 238 | total_timesteps 4484.
Path 239 | total_timesteps 4498.
Path 240 | total_timesteps 4512.
Path 241 | total_timesteps 4533.
Path 242 | total_timesteps 4546.
Path 243 | total_timesteps 4557.
Path 244 | total_timesteps 4571.
Path 245 | total_timesteps 4590.
Path 246 | total_timesteps 4614.
Path 247 | total_timesteps 4627.
Path 248 | total_timesteps 4635.
Path 249 | total_timesteps 4650.
Path 250 | total_timesteps 4664.
Path 251 | total_timesteps 4679.
Path 252 | total_timesteps 4696.
Path 253 | total_timesteps 4706.
Path 254 | total_timesteps 4728.
Path 255 | total_timesteps 4738.
Path 256 | total_timesteps 4751.
Path 257 | total_timesteps 4770.
Path 258 | total_timesteps 4785.
Path 259 | total_timesteps 4798.
Path 260 | total_timesteps 4829.
Path 261 | total_timesteps 4850.
Path 262 | total_timesteps 4904.
Path 263 | total_timesteps 4927.
Path 264 | total_timesteps 4943.
Path 265 | total_timesteps 4962.
Path 266 | total_timesteps 4976.
Path 267 | total_timesteps 4988.
Path 268 | total_timesteps 5013.
Path 269 | total_timesteps 5044.
Path 270 | total_timesteps 5077.
Path 271 | total_timesteps 5098.
Path 272 | total_timesteps 5122.
Path 273 | total_timesteps 5152.
Path 274 | total_timesteps 5162.
Path 275 | total_timesteps 5178.
Path 276 | total_timesteps 5195.
Path 277 | total_timesteps 5209.
Path 278 | total_timesteps 5229.
Path 279 | total_timesteps 5250.
Path 280 | total_timesteps 5262.
Path 281 | total_timesteps 5273.
Path 282 | total_timesteps 5286.
Path 283 | total_timesteps 5305.
Path 284 | total_timesteps 5323.
Path 285 | total_timesteps 5339.
Path 286 | total_timesteps 5351.
Path 287 | total_timesteps 5358.
Path 288 | total_timesteps 5382.
Path 289 | total_timesteps 5397.
Path 290 | total_timesteps 5442.
Path 291 | total_timesteps 5452.
Path 292 | total_timesteps 5464.
Path 293 | total_timesteps 5480.
Path 294 | total_timesteps 5488.
Path 295 | total_timesteps 5508.
Path 296 | total_timesteps 5529.
Path 297 | total_timesteps 5537.
Path 298 | total_timesteps 5550.
Path 299 | total_timesteps 5589.
Path 300 | total_timesteps 5612.
Path 301 | total_timesteps 5629.
Path 302 | total_timesteps 5646.
Path 303 | total_timesteps 5659.
Path 304 | total_timesteps 5675.
Path 305 | total_timesteps 5694.
Path 306 | total_timesteps 5711.
Path 307 | total_timesteps 5733.
Path 308 | total_timesteps 5763.
Path 309 | total_timesteps 5792.
Path 310 | total_timesteps 5805.
Path 311 | total_timesteps 5823.
Path 312 | total_timesteps 5852.
Path 313 | total_timesteps 5867.
Path 314 | total_timesteps 5892.
Path 315 | total_timesteps 5905.
Path 316 | total_timesteps 5930.
Path 317 | total_timesteps 5955.
Path 318 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.04    |
| Iteration     | 1        |
| MaximumReturn | 19.1     |
| MinimumReturn | -20.3    |
| TotalSamples  | 12025    |
----------------------------
itr #2 | 
Fitting dynamics.
Validation loss = 0.04480937123298645
Validation loss = 0.03467995673418045
Validation loss = 0.03299418091773987
Validation loss = 0.03655233606696129
Validation loss = 0.02855486236512661
Validation loss = 0.03079683519899845
Validation loss = 0.03350933641195297
Validation loss = 0.028774917125701904
Validation loss = 0.031266991049051285
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 37.
Path 4 | total_timesteps 51.
Path 5 | total_timesteps 71.
Path 6 | total_timesteps 91.
Path 7 | total_timesteps 104.
Path 8 | total_timesteps 126.
Path 9 | total_timesteps 136.
Path 10 | total_timesteps 147.
Path 11 | total_timesteps 162.
Path 12 | total_timesteps 173.
Path 13 | total_timesteps 184.
Path 14 | total_timesteps 199.
Path 15 | total_timesteps 210.
Path 16 | total_timesteps 226.
Path 17 | total_timesteps 238.
Path 18 | total_timesteps 247.
Path 19 | total_timesteps 269.
Path 20 | total_timesteps 289.
Path 21 | total_timesteps 299.
Path 22 | total_timesteps 310.
Path 23 | total_timesteps 324.
Path 24 | total_timesteps 341.
Path 25 | total_timesteps 351.
Path 26 | total_timesteps 365.
Path 27 | total_timesteps 375.
Path 28 | total_timesteps 385.
Path 29 | total_timesteps 406.
Path 30 | total_timesteps 418.
Path 31 | total_timesteps 431.
Path 32 | total_timesteps 451.
Path 33 | total_timesteps 473.
Path 34 | total_timesteps 487.
Path 35 | total_timesteps 520.
Path 36 | total_timesteps 531.
Path 37 | total_timesteps 549.
Path 38 | total_timesteps 566.
Path 39 | total_timesteps 579.
Path 40 | total_timesteps 589.
Path 41 | total_timesteps 599.
Path 42 | total_timesteps 614.
Path 43 | total_timesteps 632.
Path 44 | total_timesteps 639.
Path 45 | total_timesteps 650.
Path 46 | total_timesteps 659.
Path 47 | total_timesteps 688.
Path 48 | total_timesteps 706.
Path 49 | total_timesteps 725.
Path 50 | total_timesteps 735.
Path 51 | total_timesteps 754.
Path 52 | total_timesteps 764.
Path 53 | total_timesteps 775.
Path 54 | total_timesteps 791.
Path 55 | total_timesteps 801.
Path 56 | total_timesteps 830.
Path 57 | total_timesteps 843.
Path 58 | total_timesteps 856.
Path 59 | total_timesteps 870.
Path 60 | total_timesteps 884.
Path 61 | total_timesteps 898.
Path 62 | total_timesteps 918.
Path 63 | total_timesteps 927.
Path 64 | total_timesteps 940.
Path 65 | total_timesteps 953.
Path 66 | total_timesteps 970.
Path 67 | total_timesteps 989.
Path 68 | total_timesteps 1007.
Path 69 | total_timesteps 1028.
Path 70 | total_timesteps 1042.
Path 71 | total_timesteps 1062.
Path 72 | total_timesteps 1079.
Path 73 | total_timesteps 1092.
Path 74 | total_timesteps 1111.
Path 75 | total_timesteps 1120.
Path 76 | total_timesteps 1132.
Path 77 | total_timesteps 1146.
Path 78 | total_timesteps 1168.
Path 79 | total_timesteps 1185.
Path 80 | total_timesteps 1194.
Path 81 | total_timesteps 1207.
Path 82 | total_timesteps 1228.
Path 83 | total_timesteps 1237.
Path 84 | total_timesteps 1258.
Path 85 | total_timesteps 1268.
Path 86 | total_timesteps 1282.
Path 87 | total_timesteps 1298.
Path 88 | total_timesteps 1315.
Path 89 | total_timesteps 1327.
Path 90 | total_timesteps 1346.
Path 91 | total_timesteps 1359.
Path 92 | total_timesteps 1370.
Path 93 | total_timesteps 1383.
Path 94 | total_timesteps 1395.
Path 95 | total_timesteps 1410.
Path 96 | total_timesteps 1423.
Path 97 | total_timesteps 1435.
Path 98 | total_timesteps 1453.
Path 99 | total_timesteps 1469.
Path 100 | total_timesteps 1490.
Path 101 | total_timesteps 1509.
Path 102 | total_timesteps 1521.
Path 103 | total_timesteps 1534.
Path 104 | total_timesteps 1559.
Path 105 | total_timesteps 1571.
Path 106 | total_timesteps 1586.
Path 107 | total_timesteps 1601.
Path 108 | total_timesteps 1629.
Path 109 | total_timesteps 1670.
Path 110 | total_timesteps 1698.
Path 111 | total_timesteps 1714.
Path 112 | total_timesteps 1728.
Path 113 | total_timesteps 1743.
Path 114 | total_timesteps 1756.
Path 115 | total_timesteps 1768.
Path 116 | total_timesteps 1782.
Path 117 | total_timesteps 1793.
Path 118 | total_timesteps 1812.
Path 119 | total_timesteps 1823.
Path 120 | total_timesteps 1834.
Path 121 | total_timesteps 1847.
Path 122 | total_timesteps 1871.
Path 123 | total_timesteps 1894.
Path 124 | total_timesteps 1909.
Path 125 | total_timesteps 1920.
Path 126 | total_timesteps 1935.
Path 127 | total_timesteps 1948.
Path 128 | total_timesteps 1972.
Path 129 | total_timesteps 1991.
Path 130 | total_timesteps 2006.
Path 131 | total_timesteps 2036.
Path 132 | total_timesteps 2056.
Path 133 | total_timesteps 2078.
Path 134 | total_timesteps 2104.
Path 135 | total_timesteps 2119.
Path 136 | total_timesteps 2138.
Path 137 | total_timesteps 2159.
Path 138 | total_timesteps 2175.
Path 139 | total_timesteps 2186.
Path 140 | total_timesteps 2205.
Path 141 | total_timesteps 2218.
Path 142 | total_timesteps 2231.
Path 143 | total_timesteps 2261.
Path 144 | total_timesteps 2275.
Path 145 | total_timesteps 2292.
Path 146 | total_timesteps 2320.
Path 147 | total_timesteps 2344.
Path 148 | total_timesteps 2355.
Path 149 | total_timesteps 2366.
Path 150 | total_timesteps 2383.
Path 151 | total_timesteps 2395.
Path 152 | total_timesteps 2406.
Path 153 | total_timesteps 2419.
Path 154 | total_timesteps 2435.
Path 155 | total_timesteps 2459.
Path 156 | total_timesteps 2472.
Path 157 | total_timesteps 2491.
Path 158 | total_timesteps 2504.
Path 159 | total_timesteps 2516.
Path 160 | total_timesteps 2536.
Path 161 | total_timesteps 2548.
Path 162 | total_timesteps 2570.
Path 163 | total_timesteps 2585.
Path 164 | total_timesteps 2593.
Path 165 | total_timesteps 2611.
Path 166 | total_timesteps 2628.
Path 167 | total_timesteps 2645.
Path 168 | total_timesteps 2662.
Path 169 | total_timesteps 2676.
Path 170 | total_timesteps 2689.
Path 171 | total_timesteps 2705.
Path 172 | total_timesteps 2719.
Path 173 | total_timesteps 2735.
Path 174 | total_timesteps 2748.
Path 175 | total_timesteps 2766.
Path 176 | total_timesteps 2778.
Path 177 | total_timesteps 2804.
Path 178 | total_timesteps 2812.
Path 179 | total_timesteps 2833.
Path 180 | total_timesteps 2853.
Path 181 | total_timesteps 2865.
Path 182 | total_timesteps 2876.
Path 183 | total_timesteps 2903.
Path 184 | total_timesteps 2924.
Path 185 | total_timesteps 2934.
Path 186 | total_timesteps 2950.
Path 187 | total_timesteps 2959.
Path 188 | total_timesteps 2968.
Path 189 | total_timesteps 2980.
Path 190 | total_timesteps 2989.
Path 191 | total_timesteps 3004.
Path 192 | total_timesteps 3018.
Path 193 | total_timesteps 3027.
Path 194 | total_timesteps 3051.
Path 195 | total_timesteps 3065.
Path 196 | total_timesteps 3084.
Path 197 | total_timesteps 3103.
Path 198 | total_timesteps 3123.
Path 199 | total_timesteps 3140.
Path 200 | total_timesteps 3154.
Path 201 | total_timesteps 3172.
Path 202 | total_timesteps 3187.
Path 203 | total_timesteps 3199.
Path 204 | total_timesteps 3210.
Path 205 | total_timesteps 3228.
Path 206 | total_timesteps 3238.
Path 207 | total_timesteps 3257.
Path 208 | total_timesteps 3278.
Path 209 | total_timesteps 3289.
Path 210 | total_timesteps 3302.
Path 211 | total_timesteps 3312.
Path 212 | total_timesteps 3323.
Path 213 | total_timesteps 3332.
Path 214 | total_timesteps 3340.
Path 215 | total_timesteps 3355.
Path 216 | total_timesteps 3371.
Path 217 | total_timesteps 3390.
Path 218 | total_timesteps 3403.
Path 219 | total_timesteps 3428.
Path 220 | total_timesteps 3437.
Path 221 | total_timesteps 3460.
Path 222 | total_timesteps 3478.
Path 223 | total_timesteps 3491.
Path 224 | total_timesteps 3502.
Path 225 | total_timesteps 3515.
Path 226 | total_timesteps 3537.
Path 227 | total_timesteps 3556.
Path 228 | total_timesteps 3566.
Path 229 | total_timesteps 3586.
Path 230 | total_timesteps 3608.
Path 231 | total_timesteps 3623.
Path 232 | total_timesteps 3640.
Path 233 | total_timesteps 3661.
Path 234 | total_timesteps 3670.
Path 235 | total_timesteps 3684.
Path 236 | total_timesteps 3715.
Path 237 | total_timesteps 3726.
Path 238 | total_timesteps 3733.
Path 239 | total_timesteps 3742.
Path 240 | total_timesteps 3758.
Path 241 | total_timesteps 3792.
Path 242 | total_timesteps 3810.
Path 243 | total_timesteps 3823.
Path 244 | total_timesteps 3840.
Path 245 | total_timesteps 3848.
Path 246 | total_timesteps 3858.
Path 247 | total_timesteps 3875.
Path 248 | total_timesteps 3885.
Path 249 | total_timesteps 3909.
Path 250 | total_timesteps 3941.
Path 251 | total_timesteps 3951.
Path 252 | total_timesteps 3961.
Path 253 | total_timesteps 3975.
Path 254 | total_timesteps 4004.
Path 255 | total_timesteps 4022.
Path 256 | total_timesteps 4041.
Path 257 | total_timesteps 4073.
Path 258 | total_timesteps 4091.
Path 259 | total_timesteps 4105.
Path 260 | total_timesteps 4115.
Path 261 | total_timesteps 4145.
Path 262 | total_timesteps 4159.
Path 263 | total_timesteps 4171.
Path 264 | total_timesteps 4186.
Path 265 | total_timesteps 4197.
Path 266 | total_timesteps 4218.
Path 267 | total_timesteps 4229.
Path 268 | total_timesteps 4248.
Path 269 | total_timesteps 4259.
Path 270 | total_timesteps 4271.
Path 271 | total_timesteps 4289.
Path 272 | total_timesteps 4298.
Path 273 | total_timesteps 4313.
Path 274 | total_timesteps 4322.
Path 275 | total_timesteps 4335.
Path 276 | total_timesteps 4354.
Path 277 | total_timesteps 4372.
Path 278 | total_timesteps 4385.
Path 279 | total_timesteps 4401.
Path 280 | total_timesteps 4414.
Path 281 | total_timesteps 4438.
Path 282 | total_timesteps 4450.
Path 283 | total_timesteps 4464.
Path 284 | total_timesteps 4473.
Path 285 | total_timesteps 4488.
Path 286 | total_timesteps 4506.
Path 287 | total_timesteps 4519.
Path 288 | total_timesteps 4534.
Path 289 | total_timesteps 4548.
Path 290 | total_timesteps 4562.
Path 291 | total_timesteps 4575.
Path 292 | total_timesteps 4590.
Path 293 | total_timesteps 4600.
Path 294 | total_timesteps 4618.
Path 295 | total_timesteps 4635.
Path 296 | total_timesteps 4650.
Path 297 | total_timesteps 4659.
Path 298 | total_timesteps 4672.
Path 299 | total_timesteps 4686.
Path 300 | total_timesteps 4700.
Path 301 | total_timesteps 4712.
Path 302 | total_timesteps 4721.
Path 303 | total_timesteps 4755.
Path 304 | total_timesteps 4774.
Path 305 | total_timesteps 4798.
Path 306 | total_timesteps 4811.
Path 307 | total_timesteps 4828.
Path 308 | total_timesteps 4848.
Path 309 | total_timesteps 4861.
Path 310 | total_timesteps 4876.
Path 311 | total_timesteps 4888.
Path 312 | total_timesteps 4901.
Path 313 | total_timesteps 4913.
Path 314 | total_timesteps 4929.
Path 315 | total_timesteps 4958.
Path 316 | total_timesteps 4974.
Path 317 | total_timesteps 4993.
Path 318 | total_timesteps 5003.
Path 319 | total_timesteps 5028.
Path 320 | total_timesteps 5042.
Path 321 | total_timesteps 5053.
Path 322 | total_timesteps 5061.
Path 323 | total_timesteps 5087.
Path 324 | total_timesteps 5097.
Path 325 | total_timesteps 5108.
Path 326 | total_timesteps 5126.
Path 327 | total_timesteps 5136.
Path 328 | total_timesteps 5149.
Path 329 | total_timesteps 5170.
Path 330 | total_timesteps 5191.
Path 331 | total_timesteps 5202.
Path 332 | total_timesteps 5233.
Path 333 | total_timesteps 5248.
Path 334 | total_timesteps 5262.
Path 335 | total_timesteps 5280.
Path 336 | total_timesteps 5294.
Path 337 | total_timesteps 5304.
Path 338 | total_timesteps 5334.
Path 339 | total_timesteps 5350.
Path 340 | total_timesteps 5361.
Path 341 | total_timesteps 5373.
Path 342 | total_timesteps 5412.
Path 343 | total_timesteps 5425.
Path 344 | total_timesteps 5437.
Path 345 | total_timesteps 5450.
Path 346 | total_timesteps 5459.
Path 347 | total_timesteps 5467.
Path 348 | total_timesteps 5486.
Path 349 | total_timesteps 5508.
Path 350 | total_timesteps 5522.
Path 351 | total_timesteps 5542.
Path 352 | total_timesteps 5567.
Path 353 | total_timesteps 5582.
Path 354 | total_timesteps 5594.
Path 355 | total_timesteps 5603.
Path 356 | total_timesteps 5613.
Path 357 | total_timesteps 5634.
Path 358 | total_timesteps 5644.
Path 359 | total_timesteps 5658.
Path 360 | total_timesteps 5672.
Path 361 | total_timesteps 5682.
Path 362 | total_timesteps 5695.
Path 363 | total_timesteps 5715.
Path 364 | total_timesteps 5732.
Path 365 | total_timesteps 5761.
Path 366 | total_timesteps 5778.
Path 367 | total_timesteps 5787.
Path 368 | total_timesteps 5798.
Path 369 | total_timesteps 5808.
Path 370 | total_timesteps 5824.
Path 371 | total_timesteps 5834.
Path 372 | total_timesteps 5850.
Path 373 | total_timesteps 5871.
Path 374 | total_timesteps 5895.
Path 375 | total_timesteps 5916.
Path 376 | total_timesteps 5928.
Path 377 | total_timesteps 5954.
Path 378 | total_timesteps 5963.
Path 379 | total_timesteps 5980.
Path 380 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.21    |
| Iteration     | 2        |
| MaximumReturn | 5.02     |
| MinimumReturn | -20.1    |
| TotalSamples  | 16026    |
----------------------------
itr #3 | 
Fitting dynamics.
Validation loss = 0.030917931348085403
Validation loss = 0.02642384171485901
Validation loss = 0.030818620696663857
Validation loss = 0.02455729804933071
Validation loss = 0.024317452684044838
Validation loss = 0.023139728233218193
Validation loss = 0.02407354675233364
Validation loss = 0.024874353781342506
Validation loss = 0.024144131690263748
Validation loss = 0.022292722016572952
Validation loss = 0.02393145114183426
Validation loss = 0.025033358484506607
Validation loss = 0.027087297290563583
Validation loss = 0.021817419677972794
Validation loss = 0.02161203697323799
Validation loss = 0.02442987449467182
Validation loss = 0.02068307250738144
Validation loss = 0.02159574069082737
Validation loss = 0.021734673529863358
Validation loss = 0.02183564379811287
Validation loss = 0.025006718933582306
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 45.
Path 3 | total_timesteps 66.
Path 4 | total_timesteps 78.
Path 5 | total_timesteps 98.
Path 6 | total_timesteps 114.
Path 7 | total_timesteps 130.
Path 8 | total_timesteps 149.
Path 9 | total_timesteps 163.
Path 10 | total_timesteps 196.
Path 11 | total_timesteps 221.
Path 12 | total_timesteps 232.
Path 13 | total_timesteps 241.
Path 14 | total_timesteps 256.
Path 15 | total_timesteps 272.
Path 16 | total_timesteps 282.
Path 17 | total_timesteps 311.
Path 18 | total_timesteps 329.
Path 19 | total_timesteps 343.
Path 20 | total_timesteps 362.
Path 21 | total_timesteps 381.
Path 22 | total_timesteps 400.
Path 23 | total_timesteps 412.
Path 24 | total_timesteps 439.
Path 25 | total_timesteps 452.
Path 26 | total_timesteps 462.
Path 27 | total_timesteps 484.
Path 28 | total_timesteps 492.
Path 29 | total_timesteps 510.
Path 30 | total_timesteps 519.
Path 31 | total_timesteps 531.
Path 32 | total_timesteps 544.
Path 33 | total_timesteps 558.
Path 34 | total_timesteps 575.
Path 35 | total_timesteps 596.
Path 36 | total_timesteps 615.
Path 37 | total_timesteps 640.
Path 38 | total_timesteps 662.
Path 39 | total_timesteps 691.
Path 40 | total_timesteps 704.
Path 41 | total_timesteps 729.
Path 42 | total_timesteps 744.
Path 43 | total_timesteps 773.
Path 44 | total_timesteps 790.
Path 45 | total_timesteps 807.
Path 46 | total_timesteps 833.
Path 47 | total_timesteps 845.
Path 48 | total_timesteps 858.
Path 49 | total_timesteps 877.
Path 50 | total_timesteps 911.
Path 51 | total_timesteps 934.
Path 52 | total_timesteps 966.
Path 53 | total_timesteps 979.
Path 54 | total_timesteps 995.
Path 55 | total_timesteps 1034.
Path 56 | total_timesteps 1049.
Path 57 | total_timesteps 1065.
Path 58 | total_timesteps 1087.
Path 59 | total_timesteps 1150.
Path 60 | total_timesteps 1167.
Path 61 | total_timesteps 1189.
Path 62 | total_timesteps 1215.
Path 63 | total_timesteps 1227.
Path 64 | total_timesteps 1247.
Path 65 | total_timesteps 1263.
Path 66 | total_timesteps 1285.
Path 67 | total_timesteps 1308.
Path 68 | total_timesteps 1323.
Path 69 | total_timesteps 1341.
Path 70 | total_timesteps 1354.
Path 71 | total_timesteps 1374.
Path 72 | total_timesteps 1386.
Path 73 | total_timesteps 1395.
Path 74 | total_timesteps 1407.
Path 75 | total_timesteps 1428.
Path 76 | total_timesteps 1439.
Path 77 | total_timesteps 1459.
Path 78 | total_timesteps 1481.
Path 79 | total_timesteps 1496.
Path 80 | total_timesteps 1514.
Path 81 | total_timesteps 1524.
Path 82 | total_timesteps 1547.
Path 83 | total_timesteps 1576.
Path 84 | total_timesteps 1588.
Path 85 | total_timesteps 1657.
Path 86 | total_timesteps 1670.
Path 87 | total_timesteps 1691.
Path 88 | total_timesteps 1708.
Path 89 | total_timesteps 1732.
Path 90 | total_timesteps 1744.
Path 91 | total_timesteps 1770.
Path 92 | total_timesteps 1788.
Path 93 | total_timesteps 1807.
Path 94 | total_timesteps 1822.
Path 95 | total_timesteps 1842.
Path 96 | total_timesteps 1863.
Path 97 | total_timesteps 1876.
Path 98 | total_timesteps 1898.
Path 99 | total_timesteps 1918.
Path 100 | total_timesteps 1932.
Path 101 | total_timesteps 1947.
Path 102 | total_timesteps 1969.
Path 103 | total_timesteps 1982.
Path 104 | total_timesteps 2018.
Path 105 | total_timesteps 2033.
Path 106 | total_timesteps 2052.
Path 107 | total_timesteps 2069.
Path 108 | total_timesteps 2083.
Path 109 | total_timesteps 2099.
Path 110 | total_timesteps 2125.
Path 111 | total_timesteps 2145.
Path 112 | total_timesteps 2170.
Path 113 | total_timesteps 2185.
Path 114 | total_timesteps 2197.
Path 115 | total_timesteps 2208.
Path 116 | total_timesteps 2225.
Path 117 | total_timesteps 2243.
Path 118 | total_timesteps 2255.
Path 119 | total_timesteps 2278.
Path 120 | total_timesteps 2340.
Path 121 | total_timesteps 2360.
Path 122 | total_timesteps 2368.
Path 123 | total_timesteps 2401.
Path 124 | total_timesteps 2415.
Path 125 | total_timesteps 2429.
Path 126 | total_timesteps 2441.
Path 127 | total_timesteps 2457.
Path 128 | total_timesteps 2479.
Path 129 | total_timesteps 2503.
Path 130 | total_timesteps 2517.
Path 131 | total_timesteps 2542.
Path 132 | total_timesteps 2560.
Path 133 | total_timesteps 2598.
Path 134 | total_timesteps 2613.
Path 135 | total_timesteps 2651.
Path 136 | total_timesteps 2665.
Path 137 | total_timesteps 2680.
Path 138 | total_timesteps 2693.
Path 139 | total_timesteps 2717.
Path 140 | total_timesteps 2756.
Path 141 | total_timesteps 2777.
Path 142 | total_timesteps 2801.
Path 143 | total_timesteps 2812.
Path 144 | total_timesteps 2826.
Path 145 | total_timesteps 2837.
Path 146 | total_timesteps 2865.
Path 147 | total_timesteps 2889.
Path 148 | total_timesteps 2905.
Path 149 | total_timesteps 2953.
Path 150 | total_timesteps 2967.
Path 151 | total_timesteps 2991.
Path 152 | total_timesteps 3020.
Path 153 | total_timesteps 3046.
Path 154 | total_timesteps 3070.
Path 155 | total_timesteps 3084.
Path 156 | total_timesteps 3100.
Path 157 | total_timesteps 3111.
Path 158 | total_timesteps 3147.
Path 159 | total_timesteps 3167.
Path 160 | total_timesteps 3178.
Path 161 | total_timesteps 3194.
Path 162 | total_timesteps 3208.
Path 163 | total_timesteps 3245.
Path 164 | total_timesteps 3266.
Path 165 | total_timesteps 3275.
Path 166 | total_timesteps 3289.
Path 167 | total_timesteps 3305.
Path 168 | total_timesteps 3325.
Path 169 | total_timesteps 3359.
Path 170 | total_timesteps 3374.
Path 171 | total_timesteps 3396.
Path 172 | total_timesteps 3418.
Path 173 | total_timesteps 3430.
Path 174 | total_timesteps 3450.
Path 175 | total_timesteps 3473.
Path 176 | total_timesteps 3491.
Path 177 | total_timesteps 3509.
Path 178 | total_timesteps 3526.
Path 179 | total_timesteps 3540.
Path 180 | total_timesteps 3565.
Path 181 | total_timesteps 3581.
Path 182 | total_timesteps 3608.
Path 183 | total_timesteps 3626.
Path 184 | total_timesteps 3644.
Path 185 | total_timesteps 3664.
Path 186 | total_timesteps 3676.
Path 187 | total_timesteps 3705.
Path 188 | total_timesteps 3713.
Path 189 | total_timesteps 3727.
Path 190 | total_timesteps 3743.
Path 191 | total_timesteps 3759.
Path 192 | total_timesteps 3785.
Path 193 | total_timesteps 3800.
Path 194 | total_timesteps 3810.
Path 195 | total_timesteps 3820.
Path 196 | total_timesteps 3838.
Path 197 | total_timesteps 3860.
Path 198 | total_timesteps 3875.
Path 199 | total_timesteps 3902.
Path 200 | total_timesteps 3936.
Path 201 | total_timesteps 3955.
Path 202 | total_timesteps 3967.
Path 203 | total_timesteps 3980.
Path 204 | total_timesteps 3995.
Path 205 | total_timesteps 4010.
Path 206 | total_timesteps 4024.
Path 207 | total_timesteps 4035.
Path 208 | total_timesteps 4072.
Path 209 | total_timesteps 4082.
Path 210 | total_timesteps 4107.
Path 211 | total_timesteps 4132.
Path 212 | total_timesteps 4145.
Path 213 | total_timesteps 4154.
Path 214 | total_timesteps 4169.
Path 215 | total_timesteps 4183.
Path 216 | total_timesteps 4192.
Path 217 | total_timesteps 4206.
Path 218 | total_timesteps 4221.
Path 219 | total_timesteps 4240.
Path 220 | total_timesteps 4251.
Path 221 | total_timesteps 4263.
Path 222 | total_timesteps 4277.
Path 223 | total_timesteps 4286.
Path 224 | total_timesteps 4301.
Path 225 | total_timesteps 4350.
Path 226 | total_timesteps 4369.
Path 227 | total_timesteps 4379.
Path 228 | total_timesteps 4437.
Path 229 | total_timesteps 4471.
Path 230 | total_timesteps 4494.
Path 231 | total_timesteps 4519.
Path 232 | total_timesteps 4541.
Path 233 | total_timesteps 4567.
Path 234 | total_timesteps 4591.
Path 235 | total_timesteps 4602.
Path 236 | total_timesteps 4615.
Path 237 | total_timesteps 4626.
Path 238 | total_timesteps 4641.
Path 239 | total_timesteps 4654.
Path 240 | total_timesteps 4669.
Path 241 | total_timesteps 4687.
Path 242 | total_timesteps 4708.
Path 243 | total_timesteps 4732.
Path 244 | total_timesteps 4779.
Path 245 | total_timesteps 4790.
Path 246 | total_timesteps 4821.
Path 247 | total_timesteps 4855.
Path 248 | total_timesteps 4870.
Path 249 | total_timesteps 4893.
Path 250 | total_timesteps 4908.
Path 251 | total_timesteps 4923.
Path 252 | total_timesteps 4943.
Path 253 | total_timesteps 4957.
Path 254 | total_timesteps 4987.
Path 255 | total_timesteps 5018.
Path 256 | total_timesteps 5026.
Path 257 | total_timesteps 5042.
Path 258 | total_timesteps 5062.
Path 259 | total_timesteps 5073.
Path 260 | total_timesteps 5091.
Path 261 | total_timesteps 5112.
Path 262 | total_timesteps 5135.
Path 263 | total_timesteps 5144.
Path 264 | total_timesteps 5167.
Path 265 | total_timesteps 5178.
Path 266 | total_timesteps 5201.
Path 267 | total_timesteps 5225.
Path 268 | total_timesteps 5242.
Path 269 | total_timesteps 5262.
Path 270 | total_timesteps 5275.
Path 271 | total_timesteps 5294.
Path 272 | total_timesteps 5314.
Path 273 | total_timesteps 5326.
Path 274 | total_timesteps 5344.
Path 275 | total_timesteps 5360.
Path 276 | total_timesteps 5383.
Path 277 | total_timesteps 5392.
Path 278 | total_timesteps 5429.
Path 279 | total_timesteps 5467.
Path 280 | total_timesteps 5494.
Path 281 | total_timesteps 5543.
Path 282 | total_timesteps 5572.
Path 283 | total_timesteps 5585.
Path 284 | total_timesteps 5610.
Path 285 | total_timesteps 5639.
Path 286 | total_timesteps 5651.
Path 287 | total_timesteps 5676.
Path 288 | total_timesteps 5689.
Path 289 | total_timesteps 5699.
Path 290 | total_timesteps 5709.
Path 291 | total_timesteps 5718.
Path 292 | total_timesteps 5735.
Path 293 | total_timesteps 5753.
Path 294 | total_timesteps 5808.
Path 295 | total_timesteps 5825.
Path 296 | total_timesteps 5838.
Path 297 | total_timesteps 5908.
Path 298 | total_timesteps 5922.
Path 299 | total_timesteps 5938.
Path 300 | total_timesteps 5951.
Path 301 | total_timesteps 5960.
Path 302 | total_timesteps 5973.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.94    |
| Iteration     | 3        |
| MaximumReturn | 20.2     |
| MinimumReturn | -38.8    |
| TotalSamples  | 20032    |
----------------------------
itr #4 | 
Fitting dynamics.
Validation loss = 0.021864822134375572
Validation loss = 0.020615126937627792
Validation loss = 0.021201666444540024
Validation loss = 0.020351428538560867
Validation loss = 0.01823357678949833
Validation loss = 0.019157400354743004
Validation loss = 0.019955914467573166
Validation loss = 0.019439753144979477
Validation loss = 0.020143559202551842
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 36.
Path 2 | total_timesteps 49.
Path 3 | total_timesteps 65.
Path 4 | total_timesteps 105.
Path 5 | total_timesteps 125.
Path 6 | total_timesteps 159.
Path 7 | total_timesteps 203.
Path 8 | total_timesteps 242.
Path 9 | total_timesteps 265.
Path 10 | total_timesteps 280.
Path 11 | total_timesteps 295.
Path 12 | total_timesteps 308.
Path 13 | total_timesteps 325.
Path 14 | total_timesteps 345.
Path 15 | total_timesteps 363.
Path 16 | total_timesteps 382.
Path 17 | total_timesteps 398.
Path 18 | total_timesteps 413.
Path 19 | total_timesteps 436.
Path 20 | total_timesteps 455.
Path 21 | total_timesteps 464.
Path 22 | total_timesteps 488.
Path 23 | total_timesteps 498.
Path 24 | total_timesteps 506.
Path 25 | total_timesteps 522.
Path 26 | total_timesteps 543.
Path 27 | total_timesteps 556.
Path 28 | total_timesteps 572.
Path 29 | total_timesteps 597.
Path 30 | total_timesteps 627.
Path 31 | total_timesteps 661.
Path 32 | total_timesteps 681.
Path 33 | total_timesteps 705.
Path 34 | total_timesteps 719.
Path 35 | total_timesteps 742.
Path 36 | total_timesteps 754.
Path 37 | total_timesteps 778.
Path 38 | total_timesteps 826.
Path 39 | total_timesteps 841.
Path 40 | total_timesteps 863.
Path 41 | total_timesteps 906.
Path 42 | total_timesteps 930.
Path 43 | total_timesteps 949.
Path 44 | total_timesteps 963.
Path 45 | total_timesteps 999.
Path 46 | total_timesteps 1023.
Path 47 | total_timesteps 1039.
Path 48 | total_timesteps 1053.
Path 49 | total_timesteps 1076.
Path 50 | total_timesteps 1100.
Path 51 | total_timesteps 1115.
Path 52 | total_timesteps 1146.
Path 53 | total_timesteps 1160.
Path 54 | total_timesteps 1191.
Path 55 | total_timesteps 1219.
Path 56 | total_timesteps 1262.
Path 57 | total_timesteps 1276.
Path 58 | total_timesteps 1312.
Path 59 | total_timesteps 1327.
Path 60 | total_timesteps 1348.
Path 61 | total_timesteps 1363.
Path 62 | total_timesteps 1376.
Path 63 | total_timesteps 1404.
Path 64 | total_timesteps 1430.
Path 65 | total_timesteps 1448.
Path 66 | total_timesteps 1459.
Path 67 | total_timesteps 1474.
Path 68 | total_timesteps 1487.
Path 69 | total_timesteps 1527.
Path 70 | total_timesteps 1546.
Path 71 | total_timesteps 1559.
Path 72 | total_timesteps 1584.
Path 73 | total_timesteps 1595.
Path 74 | total_timesteps 1625.
Path 75 | total_timesteps 1640.
Path 76 | total_timesteps 1665.
Path 77 | total_timesteps 1687.
Path 78 | total_timesteps 1700.
Path 79 | total_timesteps 1719.
Path 80 | total_timesteps 1758.
Path 81 | total_timesteps 1776.
Path 82 | total_timesteps 1802.
Path 83 | total_timesteps 1813.
Path 84 | total_timesteps 1833.
Path 85 | total_timesteps 1847.
Path 86 | total_timesteps 1860.
Path 87 | total_timesteps 1871.
Path 88 | total_timesteps 1885.
Path 89 | total_timesteps 1908.
Path 90 | total_timesteps 1930.
Path 91 | total_timesteps 1959.
Path 92 | total_timesteps 1993.
Path 93 | total_timesteps 2016.
Path 94 | total_timesteps 2037.
Path 95 | total_timesteps 2062.
Path 96 | total_timesteps 2075.
Path 97 | total_timesteps 2086.
Path 98 | total_timesteps 2105.
Path 99 | total_timesteps 2116.
Path 100 | total_timesteps 2131.
Path 101 | total_timesteps 2152.
Path 102 | total_timesteps 2165.
Path 103 | total_timesteps 2185.
Path 104 | total_timesteps 2218.
Path 105 | total_timesteps 2238.
Path 106 | total_timesteps 2253.
Path 107 | total_timesteps 2269.
Path 108 | total_timesteps 2282.
Path 109 | total_timesteps 2294.
Path 110 | total_timesteps 2315.
Path 111 | total_timesteps 2337.
Path 112 | total_timesteps 2349.
Path 113 | total_timesteps 2378.
Path 114 | total_timesteps 2387.
Path 115 | total_timesteps 2398.
Path 116 | total_timesteps 2426.
Path 117 | total_timesteps 2470.
Path 118 | total_timesteps 2487.
Path 119 | total_timesteps 2517.
Path 120 | total_timesteps 2525.
Path 121 | total_timesteps 2545.
Path 122 | total_timesteps 2569.
Path 123 | total_timesteps 2578.
Path 124 | total_timesteps 2608.
Path 125 | total_timesteps 2623.
Path 126 | total_timesteps 2659.
Path 127 | total_timesteps 2676.
Path 128 | total_timesteps 2702.
Path 129 | total_timesteps 2714.
Path 130 | total_timesteps 2730.
Path 131 | total_timesteps 2766.
Path 132 | total_timesteps 2781.
Path 133 | total_timesteps 2792.
Path 134 | total_timesteps 2821.
Path 135 | total_timesteps 2850.
Path 136 | total_timesteps 2864.
Path 137 | total_timesteps 2898.
Path 138 | total_timesteps 2922.
Path 139 | total_timesteps 2932.
Path 140 | total_timesteps 2946.
Path 141 | total_timesteps 2963.
Path 142 | total_timesteps 3005.
Path 143 | total_timesteps 3015.
Path 144 | total_timesteps 3045.
Path 145 | total_timesteps 3065.
Path 146 | total_timesteps 3092.
Path 147 | total_timesteps 3108.
Path 148 | total_timesteps 3122.
Path 149 | total_timesteps 3143.
Path 150 | total_timesteps 3155.
Path 151 | total_timesteps 3171.
Path 152 | total_timesteps 3183.
Path 153 | total_timesteps 3197.
Path 154 | total_timesteps 3215.
Path 155 | total_timesteps 3233.
Path 156 | total_timesteps 3243.
Path 157 | total_timesteps 3276.
Path 158 | total_timesteps 3300.
Path 159 | total_timesteps 3316.
Path 160 | total_timesteps 3326.
Path 161 | total_timesteps 3345.
Path 162 | total_timesteps 3376.
Path 163 | total_timesteps 3412.
Path 164 | total_timesteps 3424.
Path 165 | total_timesteps 3434.
Path 166 | total_timesteps 3445.
Path 167 | total_timesteps 3456.
Path 168 | total_timesteps 3475.
Path 169 | total_timesteps 3484.
Path 170 | total_timesteps 3527.
Path 171 | total_timesteps 3544.
Path 172 | total_timesteps 3569.
Path 173 | total_timesteps 3587.
Path 174 | total_timesteps 3617.
Path 175 | total_timesteps 3645.
Path 176 | total_timesteps 3662.
Path 177 | total_timesteps 3678.
Path 178 | total_timesteps 3693.
Path 179 | total_timesteps 3747.
Path 180 | total_timesteps 3773.
Path 181 | total_timesteps 3792.
Path 182 | total_timesteps 3801.
Path 183 | total_timesteps 3819.
Path 184 | total_timesteps 3832.
Path 185 | total_timesteps 3852.
Path 186 | total_timesteps 3873.
Path 187 | total_timesteps 3904.
Path 188 | total_timesteps 3941.
Path 189 | total_timesteps 3956.
Path 190 | total_timesteps 3974.
Path 191 | total_timesteps 3988.
Path 192 | total_timesteps 3999.
Path 193 | total_timesteps 4020.
Path 194 | total_timesteps 4033.
Path 195 | total_timesteps 4043.
Path 196 | total_timesteps 4076.
Path 197 | total_timesteps 4091.
Path 198 | total_timesteps 4123.
Path 199 | total_timesteps 4138.
Path 200 | total_timesteps 4153.
Path 201 | total_timesteps 4187.
Path 202 | total_timesteps 4210.
Path 203 | total_timesteps 4228.
Path 204 | total_timesteps 4240.
Path 205 | total_timesteps 4277.
Path 206 | total_timesteps 4286.
Path 207 | total_timesteps 4302.
Path 208 | total_timesteps 4323.
Path 209 | total_timesteps 4340.
Path 210 | total_timesteps 4354.
Path 211 | total_timesteps 4380.
Path 212 | total_timesteps 4414.
Path 213 | total_timesteps 4438.
Path 214 | total_timesteps 4465.
Path 215 | total_timesteps 4489.
Path 216 | total_timesteps 4504.
Path 217 | total_timesteps 4533.
Path 218 | total_timesteps 4549.
Path 219 | total_timesteps 4564.
Path 220 | total_timesteps 4587.
Path 221 | total_timesteps 4609.
Path 222 | total_timesteps 4625.
Path 223 | total_timesteps 4648.
Path 224 | total_timesteps 4657.
Path 225 | total_timesteps 4673.
Path 226 | total_timesteps 4697.
Path 227 | total_timesteps 4710.
Path 228 | total_timesteps 4726.
Path 229 | total_timesteps 4740.
Path 230 | total_timesteps 4776.
Path 231 | total_timesteps 4798.
Path 232 | total_timesteps 4825.
Path 233 | total_timesteps 4854.
Path 234 | total_timesteps 4878.
Path 235 | total_timesteps 4897.
Path 236 | total_timesteps 4940.
Path 237 | total_timesteps 4952.
Path 238 | total_timesteps 4980.
Path 239 | total_timesteps 5017.
Path 240 | total_timesteps 5034.
Path 241 | total_timesteps 5063.
Path 242 | total_timesteps 5116.
Path 243 | total_timesteps 5140.
Path 244 | total_timesteps 5166.
Path 245 | total_timesteps 5204.
Path 246 | total_timesteps 5231.
Path 247 | total_timesteps 5241.
Path 248 | total_timesteps 5260.
Path 249 | total_timesteps 5280.
Path 250 | total_timesteps 5305.
Path 251 | total_timesteps 5328.
Path 252 | total_timesteps 5342.
Path 253 | total_timesteps 5368.
Path 254 | total_timesteps 5383.
Path 255 | total_timesteps 5400.
Path 256 | total_timesteps 5420.
Path 257 | total_timesteps 5429.
Path 258 | total_timesteps 5457.
Path 259 | total_timesteps 5468.
Path 260 | total_timesteps 5479.
Path 261 | total_timesteps 5497.
Path 262 | total_timesteps 5510.
Path 263 | total_timesteps 5525.
Path 264 | total_timesteps 5541.
Path 265 | total_timesteps 5570.
Path 266 | total_timesteps 5592.
Path 267 | total_timesteps 5608.
Path 268 | total_timesteps 5617.
Path 269 | total_timesteps 5626.
Path 270 | total_timesteps 5656.
Path 271 | total_timesteps 5671.
Path 272 | total_timesteps 5681.
Path 273 | total_timesteps 5691.
Path 274 | total_timesteps 5714.
Path 275 | total_timesteps 5725.
Path 276 | total_timesteps 5740.
Path 277 | total_timesteps 5758.
Path 278 | total_timesteps 5776.
Path 279 | total_timesteps 5797.
Path 280 | total_timesteps 5819.
Path 281 | total_timesteps 5832.
Path 282 | total_timesteps 5841.
Path 283 | total_timesteps 5858.
Path 284 | total_timesteps 5878.
Path 285 | total_timesteps 5892.
Path 286 | total_timesteps 5918.
Path 287 | total_timesteps 5929.
Path 288 | total_timesteps 5947.
Path 289 | total_timesteps 5962.
Path 290 | total_timesteps 5984.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.84    |
| Iteration     | 4        |
| MaximumReturn | 14.3     |
| MinimumReturn | -19.9    |
| TotalSamples  | 24043    |
----------------------------
itr #5 | 
Fitting dynamics.
Validation loss = 0.02281070314347744
Validation loss = 0.017183518037199974
Validation loss = 0.017286038026213646
Validation loss = 0.017409922555088997
Validation loss = 0.017365144565701485
Validation loss = 0.017643975093960762
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 27.
Path 2 | total_timesteps 63.
Path 3 | total_timesteps 117.
Path 4 | total_timesteps 136.
Path 5 | total_timesteps 192.
Path 6 | total_timesteps 210.
Path 7 | total_timesteps 252.
Path 8 | total_timesteps 261.
Path 9 | total_timesteps 319.
Path 10 | total_timesteps 351.
Path 11 | total_timesteps 377.
Path 12 | total_timesteps 390.
Path 13 | total_timesteps 413.
Path 14 | total_timesteps 427.
Path 15 | total_timesteps 462.
Path 16 | total_timesteps 494.
Path 17 | total_timesteps 528.
Path 18 | total_timesteps 540.
Path 19 | total_timesteps 564.
Path 20 | total_timesteps 581.
Path 21 | total_timesteps 596.
Path 22 | total_timesteps 611.
Path 23 | total_timesteps 636.
Path 24 | total_timesteps 660.
Path 25 | total_timesteps 691.
Path 26 | total_timesteps 723.
Path 27 | total_timesteps 743.
Path 28 | total_timesteps 775.
Path 29 | total_timesteps 806.
Path 30 | total_timesteps 833.
Path 31 | total_timesteps 854.
Path 32 | total_timesteps 876.
Path 33 | total_timesteps 926.
Path 34 | total_timesteps 947.
Path 35 | total_timesteps 964.
Path 36 | total_timesteps 994.
Path 37 | total_timesteps 1004.
Path 38 | total_timesteps 1018.
Path 39 | total_timesteps 1033.
Path 40 | total_timesteps 1054.
Path 41 | total_timesteps 1076.
Path 42 | total_timesteps 1100.
Path 43 | total_timesteps 1127.
Path 44 | total_timesteps 1150.
Path 45 | total_timesteps 1164.
Path 46 | total_timesteps 1184.
Path 47 | total_timesteps 1211.
Path 48 | total_timesteps 1245.
Path 49 | total_timesteps 1267.
Path 50 | total_timesteps 1303.
Path 51 | total_timesteps 1327.
Path 52 | total_timesteps 1344.
Path 53 | total_timesteps 1375.
Path 54 | total_timesteps 1392.
Path 55 | total_timesteps 1413.
Path 56 | total_timesteps 1444.
Path 57 | total_timesteps 1474.
Path 58 | total_timesteps 1500.
Path 59 | total_timesteps 1532.
Path 60 | total_timesteps 1562.
Path 61 | total_timesteps 1579.
Path 62 | total_timesteps 1607.
Path 63 | total_timesteps 1627.
Path 64 | total_timesteps 1654.
Path 65 | total_timesteps 1691.
Path 66 | total_timesteps 1716.
Path 67 | total_timesteps 1746.
Path 68 | total_timesteps 1774.
Path 69 | total_timesteps 1793.
Path 70 | total_timesteps 1826.
Path 71 | total_timesteps 1854.
Path 72 | total_timesteps 1872.
Path 73 | total_timesteps 1892.
Path 74 | total_timesteps 1926.
Path 75 | total_timesteps 1953.
Path 76 | total_timesteps 1977.
Path 77 | total_timesteps 1999.
Path 78 | total_timesteps 2044.
Path 79 | total_timesteps 2066.
Path 80 | total_timesteps 2083.
Path 81 | total_timesteps 2103.
Path 82 | total_timesteps 2123.
Path 83 | total_timesteps 2157.
Path 84 | total_timesteps 2183.
Path 85 | total_timesteps 2226.
Path 86 | total_timesteps 2256.
Path 87 | total_timesteps 2282.
Path 88 | total_timesteps 2293.
Path 89 | total_timesteps 2304.
Path 90 | total_timesteps 2341.
Path 91 | total_timesteps 2366.
Path 92 | total_timesteps 2385.
Path 93 | total_timesteps 2427.
Path 94 | total_timesteps 2474.
Path 95 | total_timesteps 2483.
Path 96 | total_timesteps 2505.
Path 97 | total_timesteps 2524.
Path 98 | total_timesteps 2551.
Path 99 | total_timesteps 2573.
Path 100 | total_timesteps 2610.
Path 101 | total_timesteps 2631.
Path 102 | total_timesteps 2661.
Path 103 | total_timesteps 2695.
Path 104 | total_timesteps 2724.
Path 105 | total_timesteps 2756.
Path 106 | total_timesteps 2783.
Path 107 | total_timesteps 2805.
Path 108 | total_timesteps 2829.
Path 109 | total_timesteps 2865.
Path 110 | total_timesteps 2893.
Path 111 | total_timesteps 2917.
Path 112 | total_timesteps 2936.
Path 113 | total_timesteps 2967.
Path 114 | total_timesteps 2990.
Path 115 | total_timesteps 3013.
Path 116 | total_timesteps 3031.
Path 117 | total_timesteps 3077.
Path 118 | total_timesteps 3096.
Path 119 | total_timesteps 3125.
Path 120 | total_timesteps 3142.
Path 121 | total_timesteps 3173.
Path 122 | total_timesteps 3209.
Path 123 | total_timesteps 3229.
Path 124 | total_timesteps 3240.
Path 125 | total_timesteps 3283.
Path 126 | total_timesteps 3311.
Path 127 | total_timesteps 3346.
Path 128 | total_timesteps 3401.
Path 129 | total_timesteps 3444.
Path 130 | total_timesteps 3467.
Path 131 | total_timesteps 3490.
Path 132 | total_timesteps 3542.
Path 133 | total_timesteps 3559.
Path 134 | total_timesteps 3608.
Path 135 | total_timesteps 3623.
Path 136 | total_timesteps 3668.
Path 137 | total_timesteps 3687.
Path 138 | total_timesteps 3713.
Path 139 | total_timesteps 3732.
Path 140 | total_timesteps 3761.
Path 141 | total_timesteps 3787.
Path 142 | total_timesteps 3821.
Path 143 | total_timesteps 3841.
Path 144 | total_timesteps 3881.
Path 145 | total_timesteps 3941.
Path 146 | total_timesteps 3965.
Path 147 | total_timesteps 3977.
Path 148 | total_timesteps 4019.
Path 149 | total_timesteps 4040.
Path 150 | total_timesteps 4050.
Path 151 | total_timesteps 4115.
Path 152 | total_timesteps 4127.
Path 153 | total_timesteps 4154.
Path 154 | total_timesteps 4168.
Path 155 | total_timesteps 4195.
Path 156 | total_timesteps 4217.
Path 157 | total_timesteps 4256.
Path 158 | total_timesteps 4276.
Path 159 | total_timesteps 4291.
Path 160 | total_timesteps 4352.
Path 161 | total_timesteps 4405.
Path 162 | total_timesteps 4440.
Path 163 | total_timesteps 4464.
Path 164 | total_timesteps 4475.
Path 165 | total_timesteps 4489.
Path 166 | total_timesteps 4525.
Path 167 | total_timesteps 4563.
Path 168 | total_timesteps 4592.
Path 169 | total_timesteps 4625.
Path 170 | total_timesteps 4657.
Path 171 | total_timesteps 4687.
Path 172 | total_timesteps 4703.
Path 173 | total_timesteps 4722.
Path 174 | total_timesteps 4752.
Path 175 | total_timesteps 4780.
Path 176 | total_timesteps 4817.
Path 177 | total_timesteps 4850.
Path 178 | total_timesteps 4878.
Path 179 | total_timesteps 4909.
Path 180 | total_timesteps 4926.
Path 181 | total_timesteps 4985.
Path 182 | total_timesteps 5005.
Path 183 | total_timesteps 5031.
Path 184 | total_timesteps 5057.
Path 185 | total_timesteps 5115.
Path 186 | total_timesteps 5142.
Path 187 | total_timesteps 5178.
Path 188 | total_timesteps 5236.
Path 189 | total_timesteps 5256.
Path 190 | total_timesteps 5286.
Path 191 | total_timesteps 5304.
Path 192 | total_timesteps 5325.
Path 193 | total_timesteps 5348.
Path 194 | total_timesteps 5377.
Path 195 | total_timesteps 5415.
Path 196 | total_timesteps 5446.
Path 197 | total_timesteps 5456.
Path 198 | total_timesteps 5490.
Path 199 | total_timesteps 5506.
Path 200 | total_timesteps 5516.
Path 201 | total_timesteps 5542.
Path 202 | total_timesteps 5558.
Path 203 | total_timesteps 5586.
Path 204 | total_timesteps 5614.
Path 205 | total_timesteps 5645.
Path 206 | total_timesteps 5681.
Path 207 | total_timesteps 5696.
Path 208 | total_timesteps 5726.
Path 209 | total_timesteps 5754.
Path 210 | total_timesteps 5779.
Path 211 | total_timesteps 5807.
Path 212 | total_timesteps 5822.
Path 213 | total_timesteps 5839.
Path 214 | total_timesteps 5867.
Path 215 | total_timesteps 5882.
Path 216 | total_timesteps 5921.
Path 217 | total_timesteps 5940.
Path 218 | total_timesteps 5961.
Path 219 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.67    |
| Iteration     | 5        |
| MaximumReturn | 14.1     |
| MinimumReturn | -28.2    |
| TotalSamples  | 28043    |
----------------------------
itr #6 | 
Fitting dynamics.
Validation loss = 0.020941555500030518
Validation loss = 0.018768923357129097
Validation loss = 0.01757238246500492
Validation loss = 0.01596563123166561
Validation loss = 0.016519414260983467
Validation loss = 0.01496774610131979
Validation loss = 0.018857240676879883
Validation loss = 0.018318278715014458
Validation loss = 0.015351925976574421
Validation loss = 0.015192239545285702
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 23.
Path 2 | total_timesteps 65.
Path 3 | total_timesteps 87.
Path 4 | total_timesteps 109.
Path 5 | total_timesteps 119.
Path 6 | total_timesteps 140.
Path 7 | total_timesteps 163.
Path 8 | total_timesteps 178.
Path 9 | total_timesteps 197.
Path 10 | total_timesteps 219.
Path 11 | total_timesteps 239.
Path 12 | total_timesteps 250.
Path 13 | total_timesteps 280.
Path 14 | total_timesteps 298.
Path 15 | total_timesteps 327.
Path 16 | total_timesteps 366.
Path 17 | total_timesteps 399.
Path 18 | total_timesteps 410.
Path 19 | total_timesteps 425.
Path 20 | total_timesteps 447.
Path 21 | total_timesteps 477.
Path 22 | total_timesteps 501.
Path 23 | total_timesteps 526.
Path 24 | total_timesteps 564.
Path 25 | total_timesteps 589.
Path 26 | total_timesteps 618.
Path 27 | total_timesteps 636.
Path 28 | total_timesteps 674.
Path 29 | total_timesteps 711.
Path 30 | total_timesteps 740.
Path 31 | total_timesteps 757.
Path 32 | total_timesteps 778.
Path 33 | total_timesteps 792.
Path 34 | total_timesteps 824.
Path 35 | total_timesteps 842.
Path 36 | total_timesteps 878.
Path 37 | total_timesteps 916.
Path 38 | total_timesteps 941.
Path 39 | total_timesteps 972.
Path 40 | total_timesteps 1023.
Path 41 | total_timesteps 1052.
Path 42 | total_timesteps 1109.
Path 43 | total_timesteps 1128.
Path 44 | total_timesteps 1157.
Path 45 | total_timesteps 1178.
Path 46 | total_timesteps 1222.
Path 47 | total_timesteps 1247.
Path 48 | total_timesteps 1272.
Path 49 | total_timesteps 1294.
Path 50 | total_timesteps 1322.
Path 51 | total_timesteps 1340.
Path 52 | total_timesteps 1368.
Path 53 | total_timesteps 1407.
Path 54 | total_timesteps 1437.
Path 55 | total_timesteps 1452.
Path 56 | total_timesteps 1478.
Path 57 | total_timesteps 1499.
Path 58 | total_timesteps 1540.
Path 59 | total_timesteps 1569.
Path 60 | total_timesteps 1591.
Path 61 | total_timesteps 1602.
Path 62 | total_timesteps 1622.
Path 63 | total_timesteps 1646.
Path 64 | total_timesteps 1671.
Path 65 | total_timesteps 1695.
Path 66 | total_timesteps 1714.
Path 67 | total_timesteps 1738.
Path 68 | total_timesteps 1754.
Path 69 | total_timesteps 1765.
Path 70 | total_timesteps 1786.
Path 71 | total_timesteps 1807.
Path 72 | total_timesteps 1827.
Path 73 | total_timesteps 1855.
Path 74 | total_timesteps 1878.
Path 75 | total_timesteps 1893.
Path 76 | total_timesteps 1921.
Path 77 | total_timesteps 1938.
Path 78 | total_timesteps 1966.
Path 79 | total_timesteps 1981.
Path 80 | total_timesteps 2011.
Path 81 | total_timesteps 2029.
Path 82 | total_timesteps 2051.
Path 83 | total_timesteps 2075.
Path 84 | total_timesteps 2098.
Path 85 | total_timesteps 2138.
Path 86 | total_timesteps 2178.
Path 87 | total_timesteps 2189.
Path 88 | total_timesteps 2214.
Path 89 | total_timesteps 2231.
Path 90 | total_timesteps 2251.
Path 91 | total_timesteps 2285.
Path 92 | total_timesteps 2310.
Path 93 | total_timesteps 2331.
Path 94 | total_timesteps 2358.
Path 95 | total_timesteps 2367.
Path 96 | total_timesteps 2399.
Path 97 | total_timesteps 2429.
Path 98 | total_timesteps 2457.
Path 99 | total_timesteps 2480.
Path 100 | total_timesteps 2494.
Path 101 | total_timesteps 2514.
Path 102 | total_timesteps 2544.
Path 103 | total_timesteps 2557.
Path 104 | total_timesteps 2575.
Path 105 | total_timesteps 2610.
Path 106 | total_timesteps 2626.
Path 107 | total_timesteps 2666.
Path 108 | total_timesteps 2686.
Path 109 | total_timesteps 2719.
Path 110 | total_timesteps 2734.
Path 111 | total_timesteps 2758.
Path 112 | total_timesteps 2780.
Path 113 | total_timesteps 2812.
Path 114 | total_timesteps 2850.
Path 115 | total_timesteps 2886.
Path 116 | total_timesteps 2921.
Path 117 | total_timesteps 2953.
Path 118 | total_timesteps 2981.
Path 119 | total_timesteps 3000.
Path 120 | total_timesteps 3024.
Path 121 | total_timesteps 3053.
Path 122 | total_timesteps 3097.
Path 123 | total_timesteps 3136.
Path 124 | total_timesteps 3152.
Path 125 | total_timesteps 3168.
Path 126 | total_timesteps 3187.
Path 127 | total_timesteps 3207.
Path 128 | total_timesteps 3234.
Path 129 | total_timesteps 3248.
Path 130 | total_timesteps 3272.
Path 131 | total_timesteps 3285.
Path 132 | total_timesteps 3303.
Path 133 | total_timesteps 3336.
Path 134 | total_timesteps 3366.
Path 135 | total_timesteps 3392.
Path 136 | total_timesteps 3412.
Path 137 | total_timesteps 3438.
Path 138 | total_timesteps 3452.
Path 139 | total_timesteps 3484.
Path 140 | total_timesteps 3509.
Path 141 | total_timesteps 3554.
Path 142 | total_timesteps 3580.
Path 143 | total_timesteps 3623.
Path 144 | total_timesteps 3640.
Path 145 | total_timesteps 3657.
Path 146 | total_timesteps 3668.
Path 147 | total_timesteps 3688.
Path 148 | total_timesteps 3704.
Path 149 | total_timesteps 3716.
Path 150 | total_timesteps 3741.
Path 151 | total_timesteps 3785.
Path 152 | total_timesteps 3798.
Path 153 | total_timesteps 3809.
Path 154 | total_timesteps 3824.
Path 155 | total_timesteps 3853.
Path 156 | total_timesteps 3869.
Path 157 | total_timesteps 3937.
Path 158 | total_timesteps 3954.
Path 159 | total_timesteps 3971.
Path 160 | total_timesteps 3987.
Path 161 | total_timesteps 4015.
Path 162 | total_timesteps 4038.
Path 163 | total_timesteps 4062.
Path 164 | total_timesteps 4087.
Path 165 | total_timesteps 4103.
Path 166 | total_timesteps 4136.
Path 167 | total_timesteps 4175.
Path 168 | total_timesteps 4204.
Path 169 | total_timesteps 4233.
Path 170 | total_timesteps 4256.
Path 171 | total_timesteps 4295.
Path 172 | total_timesteps 4317.
Path 173 | total_timesteps 4331.
Path 174 | total_timesteps 4349.
Path 175 | total_timesteps 4397.
Path 176 | total_timesteps 4407.
Path 177 | total_timesteps 4422.
Path 178 | total_timesteps 4437.
Path 179 | total_timesteps 4449.
Path 180 | total_timesteps 4475.
Path 181 | total_timesteps 4494.
Path 182 | total_timesteps 4515.
Path 183 | total_timesteps 4545.
Path 184 | total_timesteps 4563.
Path 185 | total_timesteps 4598.
Path 186 | total_timesteps 4620.
Path 187 | total_timesteps 4643.
Path 188 | total_timesteps 4677.
Path 189 | total_timesteps 4729.
Path 190 | total_timesteps 4764.
Path 191 | total_timesteps 4783.
Path 192 | total_timesteps 4800.
Path 193 | total_timesteps 4812.
Path 194 | total_timesteps 4830.
Path 195 | total_timesteps 4854.
Path 196 | total_timesteps 4871.
Path 197 | total_timesteps 4894.
Path 198 | total_timesteps 4919.
Path 199 | total_timesteps 4936.
Path 200 | total_timesteps 4970.
Path 201 | total_timesteps 4982.
Path 202 | total_timesteps 4994.
Path 203 | total_timesteps 5012.
Path 204 | total_timesteps 5026.
Path 205 | total_timesteps 5051.
Path 206 | total_timesteps 5071.
Path 207 | total_timesteps 5094.
Path 208 | total_timesteps 5105.
Path 209 | total_timesteps 5125.
Path 210 | total_timesteps 5135.
Path 211 | total_timesteps 5147.
Path 212 | total_timesteps 5172.
Path 213 | total_timesteps 5186.
Path 214 | total_timesteps 5205.
Path 215 | total_timesteps 5245.
Path 216 | total_timesteps 5266.
Path 217 | total_timesteps 5279.
Path 218 | total_timesteps 5294.
Path 219 | total_timesteps 5321.
Path 220 | total_timesteps 5346.
Path 221 | total_timesteps 5377.
Path 222 | total_timesteps 5403.
Path 223 | total_timesteps 5423.
Path 224 | total_timesteps 5450.
Path 225 | total_timesteps 5476.
Path 226 | total_timesteps 5504.
Path 227 | total_timesteps 5527.
Path 228 | total_timesteps 5551.
Path 229 | total_timesteps 5582.
Path 230 | total_timesteps 5602.
Path 231 | total_timesteps 5616.
Path 232 | total_timesteps 5632.
Path 233 | total_timesteps 5665.
Path 234 | total_timesteps 5678.
Path 235 | total_timesteps 5692.
Path 236 | total_timesteps 5720.
Path 237 | total_timesteps 5736.
Path 238 | total_timesteps 5760.
Path 239 | total_timesteps 5788.
Path 240 | total_timesteps 5817.
Path 241 | total_timesteps 5849.
Path 242 | total_timesteps 5866.
Path 243 | total_timesteps 5887.
Path 244 | total_timesteps 5908.
Path 245 | total_timesteps 5939.
Path 246 | total_timesteps 5962.
Path 247 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.5     |
| Iteration     | 6        |
| MaximumReturn | 7.03     |
| MinimumReturn | -26.3    |
| TotalSamples  | 32046    |
----------------------------
itr #7 | 
Fitting dynamics.
Validation loss = 0.018054135143756866
Validation loss = 0.017135798931121826
Validation loss = 0.015082720667123795
Validation loss = 0.016053784638643265
Validation loss = 0.016251981258392334
Validation loss = 0.01579105481505394
Validation loss = 0.015495425090193748
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 26.
Path 2 | total_timesteps 39.
Path 3 | total_timesteps 57.
Path 4 | total_timesteps 84.
Path 5 | total_timesteps 100.
Path 6 | total_timesteps 113.
Path 7 | total_timesteps 139.
Path 8 | total_timesteps 155.
Path 9 | total_timesteps 174.
Path 10 | total_timesteps 208.
Path 11 | total_timesteps 230.
Path 12 | total_timesteps 245.
Path 13 | total_timesteps 270.
Path 14 | total_timesteps 304.
Path 15 | total_timesteps 322.
Path 16 | total_timesteps 339.
Path 17 | total_timesteps 381.
Path 18 | total_timesteps 417.
Path 19 | total_timesteps 442.
Path 20 | total_timesteps 458.
Path 21 | total_timesteps 480.
Path 22 | total_timesteps 503.
Path 23 | total_timesteps 520.
Path 24 | total_timesteps 535.
Path 25 | total_timesteps 562.
Path 26 | total_timesteps 589.
Path 27 | total_timesteps 608.
Path 28 | total_timesteps 621.
Path 29 | total_timesteps 637.
Path 30 | total_timesteps 678.
Path 31 | total_timesteps 692.
Path 32 | total_timesteps 705.
Path 33 | total_timesteps 727.
Path 34 | total_timesteps 759.
Path 35 | total_timesteps 775.
Path 36 | total_timesteps 797.
Path 37 | total_timesteps 811.
Path 38 | total_timesteps 826.
Path 39 | total_timesteps 844.
Path 40 | total_timesteps 869.
Path 41 | total_timesteps 891.
Path 42 | total_timesteps 919.
Path 43 | total_timesteps 985.
Path 44 | total_timesteps 1017.
Path 45 | total_timesteps 1036.
Path 46 | total_timesteps 1057.
Path 47 | total_timesteps 1079.
Path 48 | total_timesteps 1120.
Path 49 | total_timesteps 1139.
Path 50 | total_timesteps 1153.
Path 51 | total_timesteps 1167.
Path 52 | total_timesteps 1178.
Path 53 | total_timesteps 1202.
Path 54 | total_timesteps 1221.
Path 55 | total_timesteps 1245.
Path 56 | total_timesteps 1265.
Path 57 | total_timesteps 1290.
Path 58 | total_timesteps 1316.
Path 59 | total_timesteps 1333.
Path 60 | total_timesteps 1369.
Path 61 | total_timesteps 1383.
Path 62 | total_timesteps 1396.
Path 63 | total_timesteps 1444.
Path 64 | total_timesteps 1473.
Path 65 | total_timesteps 1494.
Path 66 | total_timesteps 1514.
Path 67 | total_timesteps 1538.
Path 68 | total_timesteps 1558.
Path 69 | total_timesteps 1591.
Path 70 | total_timesteps 1609.
Path 71 | total_timesteps 1629.
Path 72 | total_timesteps 1646.
Path 73 | total_timesteps 1660.
Path 74 | total_timesteps 1683.
Path 75 | total_timesteps 1694.
Path 76 | total_timesteps 1716.
Path 77 | total_timesteps 1725.
Path 78 | total_timesteps 1746.
Path 79 | total_timesteps 1775.
Path 80 | total_timesteps 1796.
Path 81 | total_timesteps 1814.
Path 82 | total_timesteps 1829.
Path 83 | total_timesteps 1853.
Path 84 | total_timesteps 1868.
Path 85 | total_timesteps 1890.
Path 86 | total_timesteps 1917.
Path 87 | total_timesteps 1939.
Path 88 | total_timesteps 1960.
Path 89 | total_timesteps 1994.
Path 90 | total_timesteps 2011.
Path 91 | total_timesteps 2021.
Path 92 | total_timesteps 2035.
Path 93 | total_timesteps 2063.
Path 94 | total_timesteps 2083.
Path 95 | total_timesteps 2097.
Path 96 | total_timesteps 2124.
Path 97 | total_timesteps 2139.
Path 98 | total_timesteps 2153.
Path 99 | total_timesteps 2168.
Path 100 | total_timesteps 2191.
Path 101 | total_timesteps 2216.
Path 102 | total_timesteps 2238.
Path 103 | total_timesteps 2296.
Path 104 | total_timesteps 2315.
Path 105 | total_timesteps 2339.
Path 106 | total_timesteps 2354.
Path 107 | total_timesteps 2367.
Path 108 | total_timesteps 2384.
Path 109 | total_timesteps 2402.
Path 110 | total_timesteps 2424.
Path 111 | total_timesteps 2460.
Path 112 | total_timesteps 2503.
Path 113 | total_timesteps 2519.
Path 114 | total_timesteps 2546.
Path 115 | total_timesteps 2565.
Path 116 | total_timesteps 2576.
Path 117 | total_timesteps 2598.
Path 118 | total_timesteps 2616.
Path 119 | total_timesteps 2629.
Path 120 | total_timesteps 2648.
Path 121 | total_timesteps 2671.
Path 122 | total_timesteps 2694.
Path 123 | total_timesteps 2718.
Path 124 | total_timesteps 2734.
Path 125 | total_timesteps 2757.
Path 126 | total_timesteps 2801.
Path 127 | total_timesteps 2818.
Path 128 | total_timesteps 2828.
Path 129 | total_timesteps 2840.
Path 130 | total_timesteps 2864.
Path 131 | total_timesteps 2902.
Path 132 | total_timesteps 2928.
Path 133 | total_timesteps 2948.
Path 134 | total_timesteps 2961.
Path 135 | total_timesteps 2976.
Path 136 | total_timesteps 2997.
Path 137 | total_timesteps 3011.
Path 138 | total_timesteps 3048.
Path 139 | total_timesteps 3061.
Path 140 | total_timesteps 3094.
Path 141 | total_timesteps 3124.
Path 142 | total_timesteps 3137.
Path 143 | total_timesteps 3148.
Path 144 | total_timesteps 3168.
Path 145 | total_timesteps 3180.
Path 146 | total_timesteps 3203.
Path 147 | total_timesteps 3225.
Path 148 | total_timesteps 3278.
Path 149 | total_timesteps 3306.
Path 150 | total_timesteps 3318.
Path 151 | total_timesteps 3341.
Path 152 | total_timesteps 3359.
Path 153 | total_timesteps 3375.
Path 154 | total_timesteps 3413.
Path 155 | total_timesteps 3426.
Path 156 | total_timesteps 3454.
Path 157 | total_timesteps 3495.
Path 158 | total_timesteps 3508.
Path 159 | total_timesteps 3519.
Path 160 | total_timesteps 3532.
Path 161 | total_timesteps 3589.
Path 162 | total_timesteps 3603.
Path 163 | total_timesteps 3660.
Path 164 | total_timesteps 3669.
Path 165 | total_timesteps 3684.
Path 166 | total_timesteps 3703.
Path 167 | total_timesteps 3735.
Path 168 | total_timesteps 3772.
Path 169 | total_timesteps 3788.
Path 170 | total_timesteps 3808.
Path 171 | total_timesteps 3837.
Path 172 | total_timesteps 3856.
Path 173 | total_timesteps 3872.
Path 174 | total_timesteps 3897.
Path 175 | total_timesteps 3924.
Path 176 | total_timesteps 3948.
Path 177 | total_timesteps 3967.
Path 178 | total_timesteps 3994.
Path 179 | total_timesteps 4017.
Path 180 | total_timesteps 4030.
Path 181 | total_timesteps 4050.
Path 182 | total_timesteps 4073.
Path 183 | total_timesteps 4089.
Path 184 | total_timesteps 4112.
Path 185 | total_timesteps 4133.
Path 186 | total_timesteps 4158.
Path 187 | total_timesteps 4187.
Path 188 | total_timesteps 4199.
Path 189 | total_timesteps 4218.
Path 190 | total_timesteps 4234.
Path 191 | total_timesteps 4256.
Path 192 | total_timesteps 4287.
Path 193 | total_timesteps 4319.
Path 194 | total_timesteps 4338.
Path 195 | total_timesteps 4379.
Path 196 | total_timesteps 4402.
Path 197 | total_timesteps 4445.
Path 198 | total_timesteps 4466.
Path 199 | total_timesteps 4485.
Path 200 | total_timesteps 4495.
Path 201 | total_timesteps 4525.
Path 202 | total_timesteps 4544.
Path 203 | total_timesteps 4612.
Path 204 | total_timesteps 4638.
Path 205 | total_timesteps 4650.
Path 206 | total_timesteps 4679.
Path 207 | total_timesteps 4702.
Path 208 | total_timesteps 4723.
Path 209 | total_timesteps 4754.
Path 210 | total_timesteps 4772.
Path 211 | total_timesteps 4835.
Path 212 | total_timesteps 4858.
Path 213 | total_timesteps 4882.
Path 214 | total_timesteps 4900.
Path 215 | total_timesteps 4934.
Path 216 | total_timesteps 4947.
Path 217 | total_timesteps 4968.
Path 218 | total_timesteps 4991.
Path 219 | total_timesteps 5010.
Path 220 | total_timesteps 5030.
Path 221 | total_timesteps 5041.
Path 222 | total_timesteps 5074.
Path 223 | total_timesteps 5100.
Path 224 | total_timesteps 5134.
Path 225 | total_timesteps 5151.
Path 226 | total_timesteps 5174.
Path 227 | total_timesteps 5200.
Path 228 | total_timesteps 5219.
Path 229 | total_timesteps 5236.
Path 230 | total_timesteps 5261.
Path 231 | total_timesteps 5302.
Path 232 | total_timesteps 5326.
Path 233 | total_timesteps 5351.
Path 234 | total_timesteps 5363.
Path 235 | total_timesteps 5380.
Path 236 | total_timesteps 5403.
Path 237 | total_timesteps 5431.
Path 238 | total_timesteps 5456.
Path 239 | total_timesteps 5494.
Path 240 | total_timesteps 5521.
Path 241 | total_timesteps 5542.
Path 242 | total_timesteps 5562.
Path 243 | total_timesteps 5590.
Path 244 | total_timesteps 5619.
Path 245 | total_timesteps 5635.
Path 246 | total_timesteps 5676.
Path 247 | total_timesteps 5701.
Path 248 | total_timesteps 5715.
Path 249 | total_timesteps 5731.
Path 250 | total_timesteps 5756.
Path 251 | total_timesteps 5775.
Path 252 | total_timesteps 5798.
Path 253 | total_timesteps 5815.
Path 254 | total_timesteps 5839.
Path 255 | total_timesteps 5885.
Path 256 | total_timesteps 5915.
Path 257 | total_timesteps 5955.
Path 258 | total_timesteps 5974.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.31    |
| Iteration     | 7        |
| MaximumReturn | 18.5     |
| MinimumReturn | -22.3    |
| TotalSamples  | 36060    |
----------------------------
itr #8 | 
Fitting dynamics.
Validation loss = 0.014779546298086643
Validation loss = 0.014533436857163906
Validation loss = 0.014097422361373901
Validation loss = 0.01350422203540802
Validation loss = 0.01292339526116848
Validation loss = 0.014199580997228622
Validation loss = 0.013125147670507431
Validation loss = 0.013107995502650738
Validation loss = 0.012686692178249359
Validation loss = 0.01492932066321373
Validation loss = 0.012945453636348248
Validation loss = 0.014019131660461426
Validation loss = 0.01310866791754961
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 35.
Path 3 | total_timesteps 62.
Path 4 | total_timesteps 104.
Path 5 | total_timesteps 148.
Path 6 | total_timesteps 179.
Path 7 | total_timesteps 226.
Path 8 | total_timesteps 271.
Path 9 | total_timesteps 307.
Path 10 | total_timesteps 329.
Path 11 | total_timesteps 359.
Path 12 | total_timesteps 403.
Path 13 | total_timesteps 413.
Path 14 | total_timesteps 430.
Path 15 | total_timesteps 454.
Path 16 | total_timesteps 481.
Path 17 | total_timesteps 505.
Path 18 | total_timesteps 526.
Path 19 | total_timesteps 542.
Path 20 | total_timesteps 557.
Path 21 | total_timesteps 574.
Path 22 | total_timesteps 594.
Path 23 | total_timesteps 627.
Path 24 | total_timesteps 642.
Path 25 | total_timesteps 664.
Path 26 | total_timesteps 679.
Path 27 | total_timesteps 717.
Path 28 | total_timesteps 734.
Path 29 | total_timesteps 746.
Path 30 | total_timesteps 766.
Path 31 | total_timesteps 784.
Path 32 | total_timesteps 804.
Path 33 | total_timesteps 836.
Path 34 | total_timesteps 860.
Path 35 | total_timesteps 873.
Path 36 | total_timesteps 898.
Path 37 | total_timesteps 914.
Path 38 | total_timesteps 946.
Path 39 | total_timesteps 958.
Path 40 | total_timesteps 973.
Path 41 | total_timesteps 999.
Path 42 | total_timesteps 1011.
Path 43 | total_timesteps 1027.
Path 44 | total_timesteps 1046.
Path 45 | total_timesteps 1096.
Path 46 | total_timesteps 1126.
Path 47 | total_timesteps 1148.
Path 48 | total_timesteps 1183.
Path 49 | total_timesteps 1227.
Path 50 | total_timesteps 1259.
Path 51 | total_timesteps 1285.
Path 52 | total_timesteps 1311.
Path 53 | total_timesteps 1331.
Path 54 | total_timesteps 1358.
Path 55 | total_timesteps 1378.
Path 56 | total_timesteps 1407.
Path 57 | total_timesteps 1430.
Path 58 | total_timesteps 1444.
Path 59 | total_timesteps 1473.
Path 60 | total_timesteps 1501.
Path 61 | total_timesteps 1545.
Path 62 | total_timesteps 1585.
Path 63 | total_timesteps 1617.
Path 64 | total_timesteps 1629.
Path 65 | total_timesteps 1656.
Path 66 | total_timesteps 1675.
Path 67 | total_timesteps 1721.
Path 68 | total_timesteps 1737.
Path 69 | total_timesteps 1763.
Path 70 | total_timesteps 1791.
Path 71 | total_timesteps 1807.
Path 72 | total_timesteps 1846.
Path 73 | total_timesteps 1871.
Path 74 | total_timesteps 1900.
Path 75 | total_timesteps 1925.
Path 76 | total_timesteps 1966.
Path 77 | total_timesteps 1981.
Path 78 | total_timesteps 2010.
Path 79 | total_timesteps 2024.
Path 80 | total_timesteps 2045.
Path 81 | total_timesteps 2070.
Path 82 | total_timesteps 2097.
Path 83 | total_timesteps 2126.
Path 84 | total_timesteps 2159.
Path 85 | total_timesteps 2189.
Path 86 | total_timesteps 2221.
Path 87 | total_timesteps 2242.
Path 88 | total_timesteps 2264.
Path 89 | total_timesteps 2302.
Path 90 | total_timesteps 2324.
Path 91 | total_timesteps 2349.
Path 92 | total_timesteps 2379.
Path 93 | total_timesteps 2390.
Path 94 | total_timesteps 2409.
Path 95 | total_timesteps 2442.
Path 96 | total_timesteps 2454.
Path 97 | total_timesteps 2477.
Path 98 | total_timesteps 2499.
Path 99 | total_timesteps 2549.
Path 100 | total_timesteps 2563.
Path 101 | total_timesteps 2599.
Path 102 | total_timesteps 2618.
Path 103 | total_timesteps 2632.
Path 104 | total_timesteps 2657.
Path 105 | total_timesteps 2704.
Path 106 | total_timesteps 2714.
Path 107 | total_timesteps 2748.
Path 108 | total_timesteps 2780.
Path 109 | total_timesteps 2818.
Path 110 | total_timesteps 2842.
Path 111 | total_timesteps 2861.
Path 112 | total_timesteps 2883.
Path 113 | total_timesteps 2904.
Path 114 | total_timesteps 2926.
Path 115 | total_timesteps 2940.
Path 116 | total_timesteps 2968.
Path 117 | total_timesteps 2989.
Path 118 | total_timesteps 3002.
Path 119 | total_timesteps 3035.
Path 120 | total_timesteps 3053.
Path 121 | total_timesteps 3082.
Path 122 | total_timesteps 3113.
Path 123 | total_timesteps 3136.
Path 124 | total_timesteps 3157.
Path 125 | total_timesteps 3198.
Path 126 | total_timesteps 3215.
Path 127 | total_timesteps 3243.
Path 128 | total_timesteps 3257.
Path 129 | total_timesteps 3277.
Path 130 | total_timesteps 3296.
Path 131 | total_timesteps 3316.
Path 132 | total_timesteps 3337.
Path 133 | total_timesteps 3360.
Path 134 | total_timesteps 3374.
Path 135 | total_timesteps 3403.
Path 136 | total_timesteps 3432.
Path 137 | total_timesteps 3461.
Path 138 | total_timesteps 3482.
Path 139 | total_timesteps 3502.
Path 140 | total_timesteps 3521.
Path 141 | total_timesteps 3551.
Path 142 | total_timesteps 3583.
Path 143 | total_timesteps 3601.
Path 144 | total_timesteps 3645.
Path 145 | total_timesteps 3661.
Path 146 | total_timesteps 3675.
Path 147 | total_timesteps 3699.
Path 148 | total_timesteps 3731.
Path 149 | total_timesteps 3767.
Path 150 | total_timesteps 3784.
Path 151 | total_timesteps 3803.
Path 152 | total_timesteps 3831.
Path 153 | total_timesteps 3840.
Path 154 | total_timesteps 3851.
Path 155 | total_timesteps 3877.
Path 156 | total_timesteps 3900.
Path 157 | total_timesteps 3926.
Path 158 | total_timesteps 3987.
Path 159 | total_timesteps 4006.
Path 160 | total_timesteps 4031.
Path 161 | total_timesteps 4064.
Path 162 | total_timesteps 4090.
Path 163 | total_timesteps 4109.
Path 164 | total_timesteps 4124.
Path 165 | total_timesteps 4140.
Path 166 | total_timesteps 4166.
Path 167 | total_timesteps 4192.
Path 168 | total_timesteps 4228.
Path 169 | total_timesteps 4260.
Path 170 | total_timesteps 4275.
Path 171 | total_timesteps 4297.
Path 172 | total_timesteps 4315.
Path 173 | total_timesteps 4335.
Path 174 | total_timesteps 4377.
Path 175 | total_timesteps 4405.
Path 176 | total_timesteps 4427.
Path 177 | total_timesteps 4441.
Path 178 | total_timesteps 4457.
Path 179 | total_timesteps 4484.
Path 180 | total_timesteps 4499.
Path 181 | total_timesteps 4517.
Path 182 | total_timesteps 4528.
Path 183 | total_timesteps 4558.
Path 184 | total_timesteps 4591.
Path 185 | total_timesteps 4613.
Path 186 | total_timesteps 4627.
Path 187 | total_timesteps 4641.
Path 188 | total_timesteps 4666.
Path 189 | total_timesteps 4696.
Path 190 | total_timesteps 4731.
Path 191 | total_timesteps 4750.
Path 192 | total_timesteps 4780.
Path 193 | total_timesteps 4811.
Path 194 | total_timesteps 4828.
Path 195 | total_timesteps 4853.
Path 196 | total_timesteps 4886.
Path 197 | total_timesteps 4932.
Path 198 | total_timesteps 4945.
Path 199 | total_timesteps 4963.
Path 200 | total_timesteps 4987.
Path 201 | total_timesteps 5009.
Path 202 | total_timesteps 5044.
Path 203 | total_timesteps 5070.
Path 204 | total_timesteps 5089.
Path 205 | total_timesteps 5119.
Path 206 | total_timesteps 5131.
Path 207 | total_timesteps 5158.
Path 208 | total_timesteps 5174.
Path 209 | total_timesteps 5194.
Path 210 | total_timesteps 5214.
Path 211 | total_timesteps 5249.
Path 212 | total_timesteps 5280.
Path 213 | total_timesteps 5326.
Path 214 | total_timesteps 5357.
Path 215 | total_timesteps 5439.
Path 216 | total_timesteps 5466.
Path 217 | total_timesteps 5487.
Path 218 | total_timesteps 5513.
Path 219 | total_timesteps 5526.
Path 220 | total_timesteps 5545.
Path 221 | total_timesteps 5588.
Path 222 | total_timesteps 5616.
Path 223 | total_timesteps 5636.
Path 224 | total_timesteps 5657.
Path 225 | total_timesteps 5690.
Path 226 | total_timesteps 5709.
Path 227 | total_timesteps 5744.
Path 228 | total_timesteps 5759.
Path 229 | total_timesteps 5782.
Path 230 | total_timesteps 5851.
Path 231 | total_timesteps 5871.
Path 232 | total_timesteps 5902.
Path 233 | total_timesteps 5939.
Path 234 | total_timesteps 5948.
Path 235 | total_timesteps 5979.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.47    |
| Iteration     | 8        |
| MaximumReturn | 16.1     |
| MinimumReturn | -23.8    |
| TotalSamples  | 40078    |
----------------------------
itr #9 | 
Fitting dynamics.
Validation loss = 0.014764850027859211
Validation loss = 0.013600869104266167
Validation loss = 0.012557655572891235
Validation loss = 0.012915308587253094
Validation loss = 0.012404991313815117
Validation loss = 0.012622837908565998
Validation loss = 0.012315708212554455
Validation loss = 0.012619015760719776
Validation loss = 0.012973779812455177
Validation loss = 0.012636777944862843
Validation loss = 0.012471434660255909
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 24.
Path 2 | total_timesteps 37.
Path 3 | total_timesteps 58.
Path 4 | total_timesteps 92.
Path 5 | total_timesteps 108.
Path 6 | total_timesteps 116.
Path 7 | total_timesteps 156.
Path 8 | total_timesteps 173.
Path 9 | total_timesteps 195.
Path 10 | total_timesteps 223.
Path 11 | total_timesteps 235.
Path 12 | total_timesteps 258.
Path 13 | total_timesteps 280.
Path 14 | total_timesteps 303.
Path 15 | total_timesteps 318.
Path 16 | total_timesteps 353.
Path 17 | total_timesteps 377.
Path 18 | total_timesteps 406.
Path 19 | total_timesteps 435.
Path 20 | total_timesteps 476.
Path 21 | total_timesteps 510.
Path 22 | total_timesteps 527.
Path 23 | total_timesteps 543.
Path 24 | total_timesteps 558.
Path 25 | total_timesteps 583.
Path 26 | total_timesteps 610.
Path 27 | total_timesteps 628.
Path 28 | total_timesteps 645.
Path 29 | total_timesteps 665.
Path 30 | total_timesteps 677.
Path 31 | total_timesteps 704.
Path 32 | total_timesteps 721.
Path 33 | total_timesteps 735.
Path 34 | total_timesteps 759.
Path 35 | total_timesteps 778.
Path 36 | total_timesteps 804.
Path 37 | total_timesteps 841.
Path 38 | total_timesteps 881.
Path 39 | total_timesteps 902.
Path 40 | total_timesteps 925.
Path 41 | total_timesteps 936.
Path 42 | total_timesteps 953.
Path 43 | total_timesteps 994.
Path 44 | total_timesteps 1027.
Path 45 | total_timesteps 1047.
Path 46 | total_timesteps 1083.
Path 47 | total_timesteps 1109.
Path 48 | total_timesteps 1126.
Path 49 | total_timesteps 1153.
Path 50 | total_timesteps 1184.
Path 51 | total_timesteps 1219.
Path 52 | total_timesteps 1239.
Path 53 | total_timesteps 1258.
Path 54 | total_timesteps 1287.
Path 55 | total_timesteps 1302.
Path 56 | total_timesteps 1319.
Path 57 | total_timesteps 1332.
Path 58 | total_timesteps 1357.
Path 59 | total_timesteps 1368.
Path 60 | total_timesteps 1420.
Path 61 | total_timesteps 1450.
Path 62 | total_timesteps 1472.
Path 63 | total_timesteps 1493.
Path 64 | total_timesteps 1520.
Path 65 | total_timesteps 1544.
Path 66 | total_timesteps 1568.
Path 67 | total_timesteps 1594.
Path 68 | total_timesteps 1610.
Path 69 | total_timesteps 1637.
Path 70 | total_timesteps 1664.
Path 71 | total_timesteps 1706.
Path 72 | total_timesteps 1722.
Path 73 | total_timesteps 1744.
Path 74 | total_timesteps 1780.
Path 75 | total_timesteps 1794.
Path 76 | total_timesteps 1813.
Path 77 | total_timesteps 1852.
Path 78 | total_timesteps 1869.
Path 79 | total_timesteps 1890.
Path 80 | total_timesteps 1911.
Path 81 | total_timesteps 1931.
Path 82 | total_timesteps 1948.
Path 83 | total_timesteps 1962.
Path 84 | total_timesteps 1974.
Path 85 | total_timesteps 1990.
Path 86 | total_timesteps 2008.
Path 87 | total_timesteps 2035.
Path 88 | total_timesteps 2068.
Path 89 | total_timesteps 2115.
Path 90 | total_timesteps 2135.
Path 91 | total_timesteps 2148.
Path 92 | total_timesteps 2156.
Path 93 | total_timesteps 2177.
Path 94 | total_timesteps 2202.
Path 95 | total_timesteps 2224.
Path 96 | total_timesteps 2252.
Path 97 | total_timesteps 2279.
Path 98 | total_timesteps 2328.
Path 99 | total_timesteps 2347.
Path 100 | total_timesteps 2376.
Path 101 | total_timesteps 2396.
Path 102 | total_timesteps 2415.
Path 103 | total_timesteps 2446.
Path 104 | total_timesteps 2499.
Path 105 | total_timesteps 2517.
Path 106 | total_timesteps 2538.
Path 107 | total_timesteps 2555.
Path 108 | total_timesteps 2577.
Path 109 | total_timesteps 2605.
Path 110 | total_timesteps 2624.
Path 111 | total_timesteps 2643.
Path 112 | total_timesteps 2664.
Path 113 | total_timesteps 2683.
Path 114 | total_timesteps 2694.
Path 115 | total_timesteps 2706.
Path 116 | total_timesteps 2731.
Path 117 | total_timesteps 2753.
Path 118 | total_timesteps 2789.
Path 119 | total_timesteps 2801.
Path 120 | total_timesteps 2827.
Path 121 | total_timesteps 2852.
Path 122 | total_timesteps 2877.
Path 123 | total_timesteps 2905.
Path 124 | total_timesteps 2934.
Path 125 | total_timesteps 2964.
Path 126 | total_timesteps 2980.
Path 127 | total_timesteps 3015.
Path 128 | total_timesteps 3044.
Path 129 | total_timesteps 3059.
Path 130 | total_timesteps 3071.
Path 131 | total_timesteps 3088.
Path 132 | total_timesteps 3105.
Path 133 | total_timesteps 3124.
Path 134 | total_timesteps 3143.
Path 135 | total_timesteps 3176.
Path 136 | total_timesteps 3201.
Path 137 | total_timesteps 3236.
Path 138 | total_timesteps 3257.
Path 139 | total_timesteps 3275.
Path 140 | total_timesteps 3327.
Path 141 | total_timesteps 3351.
Path 142 | total_timesteps 3365.
Path 143 | total_timesteps 3407.
Path 144 | total_timesteps 3421.
Path 145 | total_timesteps 3446.
Path 146 | total_timesteps 3477.
Path 147 | total_timesteps 3495.
Path 148 | total_timesteps 3509.
Path 149 | total_timesteps 3545.
Path 150 | total_timesteps 3558.
Path 151 | total_timesteps 3569.
Path 152 | total_timesteps 3583.
Path 153 | total_timesteps 3597.
Path 154 | total_timesteps 3626.
Path 155 | total_timesteps 3638.
Path 156 | total_timesteps 3659.
Path 157 | total_timesteps 3674.
Path 158 | total_timesteps 3689.
Path 159 | total_timesteps 3709.
Path 160 | total_timesteps 3720.
Path 161 | total_timesteps 3762.
Path 162 | total_timesteps 3786.
Path 163 | total_timesteps 3806.
Path 164 | total_timesteps 3836.
Path 165 | total_timesteps 3865.
Path 166 | total_timesteps 3890.
Path 167 | total_timesteps 3908.
Path 168 | total_timesteps 3924.
Path 169 | total_timesteps 3936.
Path 170 | total_timesteps 3948.
Path 171 | total_timesteps 3979.
Path 172 | total_timesteps 4008.
Path 173 | total_timesteps 4038.
Path 174 | total_timesteps 4049.
Path 175 | total_timesteps 4084.
Path 176 | total_timesteps 4105.
Path 177 | total_timesteps 4127.
Path 178 | total_timesteps 4151.
Path 179 | total_timesteps 4188.
Path 180 | total_timesteps 4217.
Path 181 | total_timesteps 4229.
Path 182 | total_timesteps 4243.
Path 183 | total_timesteps 4275.
Path 184 | total_timesteps 4320.
Path 185 | total_timesteps 4346.
Path 186 | total_timesteps 4369.
Path 187 | total_timesteps 4384.
Path 188 | total_timesteps 4412.
Path 189 | total_timesteps 4442.
Path 190 | total_timesteps 4464.
Path 191 | total_timesteps 4481.
Path 192 | total_timesteps 4510.
Path 193 | total_timesteps 4565.
Path 194 | total_timesteps 4587.
Path 195 | total_timesteps 4599.
Path 196 | total_timesteps 4612.
Path 197 | total_timesteps 4642.
Path 198 | total_timesteps 4666.
Path 199 | total_timesteps 4689.
Path 200 | total_timesteps 4706.
Path 201 | total_timesteps 4733.
Path 202 | total_timesteps 4765.
Path 203 | total_timesteps 4793.
Path 204 | total_timesteps 4819.
Path 205 | total_timesteps 4835.
Path 206 | total_timesteps 4851.
Path 207 | total_timesteps 4876.
Path 208 | total_timesteps 4891.
Path 209 | total_timesteps 4915.
Path 210 | total_timesteps 4925.
Path 211 | total_timesteps 4957.
Path 212 | total_timesteps 4979.
Path 213 | total_timesteps 4999.
Path 214 | total_timesteps 5031.
Path 215 | total_timesteps 5042.
Path 216 | total_timesteps 5069.
Path 217 | total_timesteps 5079.
Path 218 | total_timesteps 5095.
Path 219 | total_timesteps 5110.
Path 220 | total_timesteps 5143.
Path 221 | total_timesteps 5173.
Path 222 | total_timesteps 5190.
Path 223 | total_timesteps 5208.
Path 224 | total_timesteps 5225.
Path 225 | total_timesteps 5264.
Path 226 | total_timesteps 5293.
Path 227 | total_timesteps 5324.
Path 228 | total_timesteps 5351.
Path 229 | total_timesteps 5364.
Path 230 | total_timesteps 5392.
Path 231 | total_timesteps 5418.
Path 232 | total_timesteps 5452.
Path 233 | total_timesteps 5476.
Path 234 | total_timesteps 5510.
Path 235 | total_timesteps 5536.
Path 236 | total_timesteps 5563.
Path 237 | total_timesteps 5581.
Path 238 | total_timesteps 5607.
Path 239 | total_timesteps 5618.
Path 240 | total_timesteps 5630.
Path 241 | total_timesteps 5661.
Path 242 | total_timesteps 5682.
Path 243 | total_timesteps 5696.
Path 244 | total_timesteps 5712.
Path 245 | total_timesteps 5734.
Path 246 | total_timesteps 5758.
Path 247 | total_timesteps 5768.
Path 248 | total_timesteps 5784.
Path 249 | total_timesteps 5818.
Path 250 | total_timesteps 5841.
Path 251 | total_timesteps 5869.
Path 252 | total_timesteps 5885.
Path 253 | total_timesteps 5898.
Path 254 | total_timesteps 5912.
Path 255 | total_timesteps 5947.
Path 256 | total_timesteps 5964.
Path 257 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.27    |
| Iteration     | 9        |
| MaximumReturn | 10.8     |
| MinimumReturn | -24.1    |
| TotalSamples  | 44078    |
----------------------------
itr #10 | 
Fitting dynamics.
Validation loss = 0.01322197075933218
Validation loss = 0.012285834178328514
Validation loss = 0.011745858006179333
Validation loss = 0.012510762549936771
Validation loss = 0.012511714361608028
Validation loss = 0.011568734422326088
Validation loss = 0.011532933451235294
Validation loss = 0.01309731975197792
Validation loss = 0.011278985068202019
Validation loss = 0.011618525721132755
Validation loss = 0.012424331158399582
Validation loss = 0.011273539625108242
Validation loss = 0.011041545309126377
Validation loss = 0.01354144886136055
Validation loss = 0.011236313730478287
Validation loss = 0.011774274520576
Validation loss = 0.011206277646124363
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 34.
Path 3 | total_timesteps 48.
Path 4 | total_timesteps 73.
Path 5 | total_timesteps 96.
Path 6 | total_timesteps 138.
Path 7 | total_timesteps 153.
Path 8 | total_timesteps 177.
Path 9 | total_timesteps 196.
Path 10 | total_timesteps 225.
Path 11 | total_timesteps 251.
Path 12 | total_timesteps 289.
Path 13 | total_timesteps 308.
Path 14 | total_timesteps 336.
Path 15 | total_timesteps 377.
Path 16 | total_timesteps 409.
Path 17 | total_timesteps 442.
Path 18 | total_timesteps 468.
Path 19 | total_timesteps 497.
Path 20 | total_timesteps 520.
Path 21 | total_timesteps 545.
Path 22 | total_timesteps 569.
Path 23 | total_timesteps 586.
Path 24 | total_timesteps 616.
Path 25 | total_timesteps 659.
Path 26 | total_timesteps 679.
Path 27 | total_timesteps 714.
Path 28 | total_timesteps 736.
Path 29 | total_timesteps 761.
Path 30 | total_timesteps 788.
Path 31 | total_timesteps 836.
Path 32 | total_timesteps 866.
Path 33 | total_timesteps 886.
Path 34 | total_timesteps 898.
Path 35 | total_timesteps 935.
Path 36 | total_timesteps 968.
Path 37 | total_timesteps 1037.
Path 38 | total_timesteps 1064.
Path 39 | total_timesteps 1106.
Path 40 | total_timesteps 1126.
Path 41 | total_timesteps 1161.
Path 42 | total_timesteps 1185.
Path 43 | total_timesteps 1204.
Path 44 | total_timesteps 1240.
Path 45 | total_timesteps 1261.
Path 46 | total_timesteps 1271.
Path 47 | total_timesteps 1291.
Path 48 | total_timesteps 1325.
Path 49 | total_timesteps 1349.
Path 50 | total_timesteps 1412.
Path 51 | total_timesteps 1433.
Path 52 | total_timesteps 1460.
Path 53 | total_timesteps 1513.
Path 54 | total_timesteps 1541.
Path 55 | total_timesteps 1587.
Path 56 | total_timesteps 1615.
Path 57 | total_timesteps 1646.
Path 58 | total_timesteps 1656.
Path 59 | total_timesteps 1686.
Path 60 | total_timesteps 1721.
Path 61 | total_timesteps 1743.
Path 62 | total_timesteps 1758.
Path 63 | total_timesteps 1783.
Path 64 | total_timesteps 1809.
Path 65 | total_timesteps 1836.
Path 66 | total_timesteps 1852.
Path 67 | total_timesteps 1902.
Path 68 | total_timesteps 1925.
Path 69 | total_timesteps 1951.
Path 70 | total_timesteps 1971.
Path 71 | total_timesteps 1989.
Path 72 | total_timesteps 2017.
Path 73 | total_timesteps 2045.
Path 74 | total_timesteps 2088.
Path 75 | total_timesteps 2113.
Path 76 | total_timesteps 2124.
Path 77 | total_timesteps 2151.
Path 78 | total_timesteps 2169.
Path 79 | total_timesteps 2187.
Path 80 | total_timesteps 2211.
Path 81 | total_timesteps 2266.
Path 82 | total_timesteps 2296.
Path 83 | total_timesteps 2324.
Path 84 | total_timesteps 2343.
Path 85 | total_timesteps 2363.
Path 86 | total_timesteps 2389.
Path 87 | total_timesteps 2405.
Path 88 | total_timesteps 2417.
Path 89 | total_timesteps 2441.
Path 90 | total_timesteps 2461.
Path 91 | total_timesteps 2495.
Path 92 | total_timesteps 2508.
Path 93 | total_timesteps 2546.
Path 94 | total_timesteps 2596.
Path 95 | total_timesteps 2612.
Path 96 | total_timesteps 2643.
Path 97 | total_timesteps 2663.
Path 98 | total_timesteps 2691.
Path 99 | total_timesteps 2715.
Path 100 | total_timesteps 2746.
Path 101 | total_timesteps 2773.
Path 102 | total_timesteps 2804.
Path 103 | total_timesteps 2821.
Path 104 | total_timesteps 2855.
Path 105 | total_timesteps 2906.
Path 106 | total_timesteps 2933.
Path 107 | total_timesteps 2954.
Path 108 | total_timesteps 2974.
Path 109 | total_timesteps 3001.
Path 110 | total_timesteps 3019.
Path 111 | total_timesteps 3051.
Path 112 | total_timesteps 3090.
Path 113 | total_timesteps 3128.
Path 114 | total_timesteps 3146.
Path 115 | total_timesteps 3163.
Path 116 | total_timesteps 3186.
Path 117 | total_timesteps 3219.
Path 118 | total_timesteps 3234.
Path 119 | total_timesteps 3259.
Path 120 | total_timesteps 3285.
Path 121 | total_timesteps 3308.
Path 122 | total_timesteps 3339.
Path 123 | total_timesteps 3371.
Path 124 | total_timesteps 3400.
Path 125 | total_timesteps 3432.
Path 126 | total_timesteps 3468.
Path 127 | total_timesteps 3499.
Path 128 | total_timesteps 3535.
Path 129 | total_timesteps 3562.
Path 130 | total_timesteps 3598.
Path 131 | total_timesteps 3635.
Path 132 | total_timesteps 3655.
Path 133 | total_timesteps 3678.
Path 134 | total_timesteps 3696.
Path 135 | total_timesteps 3724.
Path 136 | total_timesteps 3750.
Path 137 | total_timesteps 3802.
Path 138 | total_timesteps 3839.
Path 139 | total_timesteps 3884.
Path 140 | total_timesteps 3894.
Path 141 | total_timesteps 3913.
Path 142 | total_timesteps 3939.
Path 143 | total_timesteps 3962.
Path 144 | total_timesteps 3978.
Path 145 | total_timesteps 4003.
Path 146 | total_timesteps 4029.
Path 147 | total_timesteps 4061.
Path 148 | total_timesteps 4101.
Path 149 | total_timesteps 4120.
Path 150 | total_timesteps 4194.
Path 151 | total_timesteps 4213.
Path 152 | total_timesteps 4229.
Path 153 | total_timesteps 4270.
Path 154 | total_timesteps 4291.
Path 155 | total_timesteps 4324.
Path 156 | total_timesteps 4346.
Path 157 | total_timesteps 4359.
Path 158 | total_timesteps 4382.
Path 159 | total_timesteps 4395.
Path 160 | total_timesteps 4423.
Path 161 | total_timesteps 4446.
Path 162 | total_timesteps 4460.
Path 163 | total_timesteps 4490.
Path 164 | total_timesteps 4522.
Path 165 | total_timesteps 4551.
Path 166 | total_timesteps 4597.
Path 167 | total_timesteps 4621.
Path 168 | total_timesteps 4640.
Path 169 | total_timesteps 4653.
Path 170 | total_timesteps 4690.
Path 171 | total_timesteps 4720.
Path 172 | total_timesteps 4750.
Path 173 | total_timesteps 4780.
Path 174 | total_timesteps 4800.
Path 175 | total_timesteps 4829.
Path 176 | total_timesteps 4854.
Path 177 | total_timesteps 4889.
Path 178 | total_timesteps 4901.
Path 179 | total_timesteps 4942.
Path 180 | total_timesteps 4972.
Path 181 | total_timesteps 5003.
Path 182 | total_timesteps 5042.
Path 183 | total_timesteps 5092.
Path 184 | total_timesteps 5130.
Path 185 | total_timesteps 5168.
Path 186 | total_timesteps 5182.
Path 187 | total_timesteps 5223.
Path 188 | total_timesteps 5241.
Path 189 | total_timesteps 5277.
Path 190 | total_timesteps 5313.
Path 191 | total_timesteps 5348.
Path 192 | total_timesteps 5362.
Path 193 | total_timesteps 5388.
Path 194 | total_timesteps 5408.
Path 195 | total_timesteps 5454.
Path 196 | total_timesteps 5482.
Path 197 | total_timesteps 5533.
Path 198 | total_timesteps 5604.
Path 199 | total_timesteps 5632.
Path 200 | total_timesteps 5678.
Path 201 | total_timesteps 5725.
Path 202 | total_timesteps 5738.
Path 203 | total_timesteps 5769.
Path 204 | total_timesteps 5785.
Path 205 | total_timesteps 5823.
Path 206 | total_timesteps 5836.
Path 207 | total_timesteps 5849.
Path 208 | total_timesteps 5876.
Path 209 | total_timesteps 5907.
Path 210 | total_timesteps 5923.
Path 211 | total_timesteps 5944.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.17    |
| Iteration     | 10       |
| MaximumReturn | 17.2     |
| MinimumReturn | -25.9    |
| TotalSamples  | 48081    |
----------------------------
itr #11 | 
Fitting dynamics.
Validation loss = 0.011562126688659191
Validation loss = 0.012158740311861038
Validation loss = 0.011037911288440228
Validation loss = 0.010441120713949203
Validation loss = 0.011106159538030624
Validation loss = 0.01131448894739151
Validation loss = 0.011976723559200764
Validation loss = 0.010550681501626968
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 27.
Path 2 | total_timesteps 56.
Path 3 | total_timesteps 70.
Path 4 | total_timesteps 109.
Path 5 | total_timesteps 127.
Path 6 | total_timesteps 149.
Path 7 | total_timesteps 169.
Path 8 | total_timesteps 180.
Path 9 | total_timesteps 211.
Path 10 | total_timesteps 242.
Path 11 | total_timesteps 261.
Path 12 | total_timesteps 374.
Path 13 | total_timesteps 393.
Path 14 | total_timesteps 426.
Path 15 | total_timesteps 452.
Path 16 | total_timesteps 486.
Path 17 | total_timesteps 509.
Path 18 | total_timesteps 542.
Path 19 | total_timesteps 555.
Path 20 | total_timesteps 596.
Path 21 | total_timesteps 612.
Path 22 | total_timesteps 628.
Path 23 | total_timesteps 641.
Path 24 | total_timesteps 665.
Path 25 | total_timesteps 682.
Path 26 | total_timesteps 712.
Path 27 | total_timesteps 738.
Path 28 | total_timesteps 793.
Path 29 | total_timesteps 823.
Path 30 | total_timesteps 858.
Path 31 | total_timesteps 890.
Path 32 | total_timesteps 905.
Path 33 | total_timesteps 930.
Path 34 | total_timesteps 950.
Path 35 | total_timesteps 962.
Path 36 | total_timesteps 998.
Path 37 | total_timesteps 1017.
Path 38 | total_timesteps 1033.
Path 39 | total_timesteps 1061.
Path 40 | total_timesteps 1077.
Path 41 | total_timesteps 1092.
Path 42 | total_timesteps 1117.
Path 43 | total_timesteps 1137.
Path 44 | total_timesteps 1169.
Path 45 | total_timesteps 1191.
Path 46 | total_timesteps 1219.
Path 47 | total_timesteps 1250.
Path 48 | total_timesteps 1269.
Path 49 | total_timesteps 1296.
Path 50 | total_timesteps 1324.
Path 51 | total_timesteps 1340.
Path 52 | total_timesteps 1360.
Path 53 | total_timesteps 1395.
Path 54 | total_timesteps 1408.
Path 55 | total_timesteps 1436.
Path 56 | total_timesteps 1446.
Path 57 | total_timesteps 1481.
Path 58 | total_timesteps 1507.
Path 59 | total_timesteps 1522.
Path 60 | total_timesteps 1552.
Path 61 | total_timesteps 1573.
Path 62 | total_timesteps 1592.
Path 63 | total_timesteps 1609.
Path 64 | total_timesteps 1635.
Path 65 | total_timesteps 1644.
Path 66 | total_timesteps 1654.
Path 67 | total_timesteps 1681.
Path 68 | total_timesteps 1691.
Path 69 | total_timesteps 1720.
Path 70 | total_timesteps 1747.
Path 71 | total_timesteps 1762.
Path 72 | total_timesteps 1784.
Path 73 | total_timesteps 1806.
Path 74 | total_timesteps 1860.
Path 75 | total_timesteps 1885.
Path 76 | total_timesteps 1906.
Path 77 | total_timesteps 1923.
Path 78 | total_timesteps 1936.
Path 79 | total_timesteps 1950.
Path 80 | total_timesteps 1976.
Path 81 | total_timesteps 1986.
Path 82 | total_timesteps 2004.
Path 83 | total_timesteps 2022.
Path 84 | total_timesteps 2049.
Path 85 | total_timesteps 2076.
Path 86 | total_timesteps 2104.
Path 87 | total_timesteps 2123.
Path 88 | total_timesteps 2146.
Path 89 | total_timesteps 2184.
Path 90 | total_timesteps 2199.
Path 91 | total_timesteps 2243.
Path 92 | total_timesteps 2274.
Path 93 | total_timesteps 2296.
Path 94 | total_timesteps 2320.
Path 95 | total_timesteps 2337.
Path 96 | total_timesteps 2355.
Path 97 | total_timesteps 2371.
Path 98 | total_timesteps 2398.
Path 99 | total_timesteps 2429.
Path 100 | total_timesteps 2443.
Path 101 | total_timesteps 2467.
Path 102 | total_timesteps 2494.
Path 103 | total_timesteps 2516.
Path 104 | total_timesteps 2552.
Path 105 | total_timesteps 2592.
Path 106 | total_timesteps 2603.
Path 107 | total_timesteps 2621.
Path 108 | total_timesteps 2646.
Path 109 | total_timesteps 2670.
Path 110 | total_timesteps 2697.
Path 111 | total_timesteps 2717.
Path 112 | total_timesteps 2736.
Path 113 | total_timesteps 2751.
Path 114 | total_timesteps 2793.
Path 115 | total_timesteps 2826.
Path 116 | total_timesteps 2836.
Path 117 | total_timesteps 2872.
Path 118 | total_timesteps 2886.
Path 119 | total_timesteps 2903.
Path 120 | total_timesteps 2933.
Path 121 | total_timesteps 2945.
Path 122 | total_timesteps 2965.
Path 123 | total_timesteps 2996.
Path 124 | total_timesteps 3011.
Path 125 | total_timesteps 3045.
Path 126 | total_timesteps 3064.
Path 127 | total_timesteps 3081.
Path 128 | total_timesteps 3092.
Path 129 | total_timesteps 3107.
Path 130 | total_timesteps 3141.
Path 131 | total_timesteps 3159.
Path 132 | total_timesteps 3196.
Path 133 | total_timesteps 3219.
Path 134 | total_timesteps 3240.
Path 135 | total_timesteps 3263.
Path 136 | total_timesteps 3295.
Path 137 | total_timesteps 3318.
Path 138 | total_timesteps 3354.
Path 139 | total_timesteps 3376.
Path 140 | total_timesteps 3401.
Path 141 | total_timesteps 3428.
Path 142 | total_timesteps 3449.
Path 143 | total_timesteps 3462.
Path 144 | total_timesteps 3483.
Path 145 | total_timesteps 3505.
Path 146 | total_timesteps 3519.
Path 147 | total_timesteps 3547.
Path 148 | total_timesteps 3584.
Path 149 | total_timesteps 3598.
Path 150 | total_timesteps 3621.
Path 151 | total_timesteps 3642.
Path 152 | total_timesteps 3664.
Path 153 | total_timesteps 3687.
Path 154 | total_timesteps 3748.
Path 155 | total_timesteps 3775.
Path 156 | total_timesteps 3816.
Path 157 | total_timesteps 3837.
Path 158 | total_timesteps 3865.
Path 159 | total_timesteps 3878.
Path 160 | total_timesteps 3890.
Path 161 | total_timesteps 3901.
Path 162 | total_timesteps 3934.
Path 163 | total_timesteps 3958.
Path 164 | total_timesteps 3974.
Path 165 | total_timesteps 4002.
Path 166 | total_timesteps 4014.
Path 167 | total_timesteps 4044.
Path 168 | total_timesteps 4064.
Path 169 | total_timesteps 4085.
Path 170 | total_timesteps 4099.
Path 171 | total_timesteps 4137.
Path 172 | total_timesteps 4173.
Path 173 | total_timesteps 4182.
Path 174 | total_timesteps 4213.
Path 175 | total_timesteps 4236.
Path 176 | total_timesteps 4259.
Path 177 | total_timesteps 4290.
Path 178 | total_timesteps 4324.
Path 179 | total_timesteps 4348.
Path 180 | total_timesteps 4362.
Path 181 | total_timesteps 4391.
Path 182 | total_timesteps 4405.
Path 183 | total_timesteps 4419.
Path 184 | total_timesteps 4443.
Path 185 | total_timesteps 4476.
Path 186 | total_timesteps 4485.
Path 187 | total_timesteps 4510.
Path 188 | total_timesteps 4581.
Path 189 | total_timesteps 4598.
Path 190 | total_timesteps 4607.
Path 191 | total_timesteps 4628.
Path 192 | total_timesteps 4647.
Path 193 | total_timesteps 4672.
Path 194 | total_timesteps 4699.
Path 195 | total_timesteps 4717.
Path 196 | total_timesteps 4737.
Path 197 | total_timesteps 4746.
Path 198 | total_timesteps 4775.
Path 199 | total_timesteps 4791.
Path 200 | total_timesteps 4812.
Path 201 | total_timesteps 4858.
Path 202 | total_timesteps 4880.
Path 203 | total_timesteps 4912.
Path 204 | total_timesteps 4931.
Path 205 | total_timesteps 4955.
Path 206 | total_timesteps 4970.
Path 207 | total_timesteps 4982.
Path 208 | total_timesteps 5008.
Path 209 | total_timesteps 5036.
Path 210 | total_timesteps 5067.
Path 211 | total_timesteps 5076.
Path 212 | total_timesteps 5102.
Path 213 | total_timesteps 5137.
Path 214 | total_timesteps 5166.
Path 215 | total_timesteps 5192.
Path 216 | total_timesteps 5209.
Path 217 | total_timesteps 5235.
Path 218 | total_timesteps 5274.
Path 219 | total_timesteps 5302.
Path 220 | total_timesteps 5328.
Path 221 | total_timesteps 5351.
Path 222 | total_timesteps 5366.
Path 223 | total_timesteps 5385.
Path 224 | total_timesteps 5409.
Path 225 | total_timesteps 5433.
Path 226 | total_timesteps 5452.
Path 227 | total_timesteps 5483.
Path 228 | total_timesteps 5501.
Path 229 | total_timesteps 5525.
Path 230 | total_timesteps 5566.
Path 231 | total_timesteps 5590.
Path 232 | total_timesteps 5603.
Path 233 | total_timesteps 5634.
Path 234 | total_timesteps 5664.
Path 235 | total_timesteps 5692.
Path 236 | total_timesteps 5722.
Path 237 | total_timesteps 5738.
Path 238 | total_timesteps 5769.
Path 239 | total_timesteps 5784.
Path 240 | total_timesteps 5810.
Path 241 | total_timesteps 5831.
Path 242 | total_timesteps 5857.
Path 243 | total_timesteps 5873.
Path 244 | total_timesteps 5897.
Path 245 | total_timesteps 5917.
Path 246 | total_timesteps 5933.
Path 247 | total_timesteps 5942.
Path 248 | total_timesteps 5974.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.24    |
| Iteration     | 11       |
| MaximumReturn | 80.6     |
| MinimumReturn | -24      |
| TotalSamples  | 52086    |
----------------------------
itr #12 | 
Fitting dynamics.
Validation loss = 0.012004312127828598
Validation loss = 0.011092016473412514
Validation loss = 0.011512481607496738
Validation loss = 0.010967200621962547
Validation loss = 0.011083574034273624
Validation loss = 0.010540032759308815
Validation loss = 0.0108517250046134
Validation loss = 0.010926355607807636
Validation loss = 0.009784381836652756
Validation loss = 0.011471203528344631
Validation loss = 0.009760994464159012
Validation loss = 0.010249137878417969
Validation loss = 0.010591923259198666
Validation loss = 0.011066082864999771
Validation loss = 0.01000906527042389
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 27.
Path 2 | total_timesteps 47.
Path 3 | total_timesteps 72.
Path 4 | total_timesteps 87.
Path 5 | total_timesteps 113.
Path 6 | total_timesteps 163.
Path 7 | total_timesteps 186.
Path 8 | total_timesteps 205.
Path 9 | total_timesteps 237.
Path 10 | total_timesteps 261.
Path 11 | total_timesteps 280.
Path 12 | total_timesteps 301.
Path 13 | total_timesteps 314.
Path 14 | total_timesteps 325.
Path 15 | total_timesteps 349.
Path 16 | total_timesteps 368.
Path 17 | total_timesteps 397.
Path 18 | total_timesteps 435.
Path 19 | total_timesteps 468.
Path 20 | total_timesteps 507.
Path 21 | total_timesteps 537.
Path 22 | total_timesteps 564.
Path 23 | total_timesteps 582.
Path 24 | total_timesteps 605.
Path 25 | total_timesteps 624.
Path 26 | total_timesteps 646.
Path 27 | total_timesteps 669.
Path 28 | total_timesteps 690.
Path 29 | total_timesteps 720.
Path 30 | total_timesteps 738.
Path 31 | total_timesteps 771.
Path 32 | total_timesteps 788.
Path 33 | total_timesteps 818.
Path 34 | total_timesteps 831.
Path 35 | total_timesteps 851.
Path 36 | total_timesteps 868.
Path 37 | total_timesteps 879.
Path 38 | total_timesteps 906.
Path 39 | total_timesteps 920.
Path 40 | total_timesteps 927.
Path 41 | total_timesteps 946.
Path 42 | total_timesteps 1000.
Path 43 | total_timesteps 1031.
Path 44 | total_timesteps 1052.
Path 45 | total_timesteps 1064.
Path 46 | total_timesteps 1096.
Path 47 | total_timesteps 1149.
Path 48 | total_timesteps 1162.
Path 49 | total_timesteps 1181.
Path 50 | total_timesteps 1220.
Path 51 | total_timesteps 1242.
Path 52 | total_timesteps 1262.
Path 53 | total_timesteps 1290.
Path 54 | total_timesteps 1304.
Path 55 | total_timesteps 1332.
Path 56 | total_timesteps 1341.
Path 57 | total_timesteps 1377.
Path 58 | total_timesteps 1398.
Path 59 | total_timesteps 1413.
Path 60 | total_timesteps 1423.
Path 61 | total_timesteps 1454.
Path 62 | total_timesteps 1465.
Path 63 | total_timesteps 1484.
Path 64 | total_timesteps 1512.
Path 65 | total_timesteps 1539.
Path 66 | total_timesteps 1568.
Path 67 | total_timesteps 1591.
Path 68 | total_timesteps 1612.
Path 69 | total_timesteps 1667.
Path 70 | total_timesteps 1688.
Path 71 | total_timesteps 1725.
Path 72 | total_timesteps 1747.
Path 73 | total_timesteps 1768.
Path 74 | total_timesteps 1796.
Path 75 | total_timesteps 1821.
Path 76 | total_timesteps 1840.
Path 77 | total_timesteps 1851.
Path 78 | total_timesteps 1875.
Path 79 | total_timesteps 1908.
Path 80 | total_timesteps 1943.
Path 81 | total_timesteps 1982.
Path 82 | total_timesteps 2016.
Path 83 | total_timesteps 2038.
Path 84 | total_timesteps 2058.
Path 85 | total_timesteps 2078.
Path 86 | total_timesteps 2095.
Path 87 | total_timesteps 2110.
Path 88 | total_timesteps 2136.
Path 89 | total_timesteps 2164.
Path 90 | total_timesteps 2187.
Path 91 | total_timesteps 2206.
Path 92 | total_timesteps 2230.
Path 93 | total_timesteps 2251.
Path 94 | total_timesteps 2275.
Path 95 | total_timesteps 2286.
Path 96 | total_timesteps 2304.
Path 97 | total_timesteps 2344.
Path 98 | total_timesteps 2372.
Path 99 | total_timesteps 2384.
Path 100 | total_timesteps 2414.
Path 101 | total_timesteps 2428.
Path 102 | total_timesteps 2473.
Path 103 | total_timesteps 2503.
Path 104 | total_timesteps 2515.
Path 105 | total_timesteps 2536.
Path 106 | total_timesteps 2566.
Path 107 | total_timesteps 2582.
Path 108 | total_timesteps 2610.
Path 109 | total_timesteps 2644.
Path 110 | total_timesteps 2671.
Path 111 | total_timesteps 2685.
Path 112 | total_timesteps 2702.
Path 113 | total_timesteps 2726.
Path 114 | total_timesteps 2767.
Path 115 | total_timesteps 2795.
Path 116 | total_timesteps 2811.
Path 117 | total_timesteps 2834.
Path 118 | total_timesteps 2847.
Path 119 | total_timesteps 2870.
Path 120 | total_timesteps 2919.
Path 121 | total_timesteps 2953.
Path 122 | total_timesteps 2977.
Path 123 | total_timesteps 3011.
Path 124 | total_timesteps 3032.
Path 125 | total_timesteps 3055.
Path 126 | total_timesteps 3074.
Path 127 | total_timesteps 3122.
Path 128 | total_timesteps 3146.
Path 129 | total_timesteps 3170.
Path 130 | total_timesteps 3185.
Path 131 | total_timesteps 3209.
Path 132 | total_timesteps 3235.
Path 133 | total_timesteps 3256.
Path 134 | total_timesteps 3283.
Path 135 | total_timesteps 3321.
Path 136 | total_timesteps 3340.
Path 137 | total_timesteps 3356.
Path 138 | total_timesteps 3402.
Path 139 | total_timesteps 3417.
Path 140 | total_timesteps 3464.
Path 141 | total_timesteps 3500.
Path 142 | total_timesteps 3516.
Path 143 | total_timesteps 3545.
Path 144 | total_timesteps 3557.
Path 145 | total_timesteps 3583.
Path 146 | total_timesteps 3605.
Path 147 | total_timesteps 3650.
Path 148 | total_timesteps 3664.
Path 149 | total_timesteps 3684.
Path 150 | total_timesteps 3716.
Path 151 | total_timesteps 3752.
Path 152 | total_timesteps 3785.
Path 153 | total_timesteps 3810.
Path 154 | total_timesteps 3822.
Path 155 | total_timesteps 3853.
Path 156 | total_timesteps 3886.
Path 157 | total_timesteps 3899.
Path 158 | total_timesteps 3922.
Path 159 | total_timesteps 3953.
Path 160 | total_timesteps 3973.
Path 161 | total_timesteps 4000.
Path 162 | total_timesteps 4034.
Path 163 | total_timesteps 4046.
Path 164 | total_timesteps 4070.
Path 165 | total_timesteps 4091.
Path 166 | total_timesteps 4125.
Path 167 | total_timesteps 4165.
Path 168 | total_timesteps 4202.
Path 169 | total_timesteps 4219.
Path 170 | total_timesteps 4239.
Path 171 | total_timesteps 4265.
Path 172 | total_timesteps 4282.
Path 173 | total_timesteps 4298.
Path 174 | total_timesteps 4312.
Path 175 | total_timesteps 4331.
Path 176 | total_timesteps 4352.
Path 177 | total_timesteps 4406.
Path 178 | total_timesteps 4432.
Path 179 | total_timesteps 4460.
Path 180 | total_timesteps 4473.
Path 181 | total_timesteps 4496.
Path 182 | total_timesteps 4513.
Path 183 | total_timesteps 4541.
Path 184 | total_timesteps 4564.
Path 185 | total_timesteps 4595.
Path 186 | total_timesteps 4624.
Path 187 | total_timesteps 4641.
Path 188 | total_timesteps 4664.
Path 189 | total_timesteps 4681.
Path 190 | total_timesteps 4723.
Path 191 | total_timesteps 4752.
Path 192 | total_timesteps 4767.
Path 193 | total_timesteps 4786.
Path 194 | total_timesteps 4823.
Path 195 | total_timesteps 4866.
Path 196 | total_timesteps 4898.
Path 197 | total_timesteps 4920.
Path 198 | total_timesteps 4955.
Path 199 | total_timesteps 4971.
Path 200 | total_timesteps 5018.
Path 201 | total_timesteps 5036.
Path 202 | total_timesteps 5064.
Path 203 | total_timesteps 5106.
Path 204 | total_timesteps 5130.
Path 205 | total_timesteps 5151.
Path 206 | total_timesteps 5172.
Path 207 | total_timesteps 5188.
Path 208 | total_timesteps 5226.
Path 209 | total_timesteps 5243.
Path 210 | total_timesteps 5264.
Path 211 | total_timesteps 5284.
Path 212 | total_timesteps 5299.
Path 213 | total_timesteps 5318.
Path 214 | total_timesteps 5338.
Path 215 | total_timesteps 5361.
Path 216 | total_timesteps 5391.
Path 217 | total_timesteps 5414.
Path 218 | total_timesteps 5449.
Path 219 | total_timesteps 5466.
Path 220 | total_timesteps 5493.
Path 221 | total_timesteps 5522.
Path 222 | total_timesteps 5533.
Path 223 | total_timesteps 5562.
Path 224 | total_timesteps 5575.
Path 225 | total_timesteps 5601.
Path 226 | total_timesteps 5633.
Path 227 | total_timesteps 5672.
Path 228 | total_timesteps 5697.
Path 229 | total_timesteps 5726.
Path 230 | total_timesteps 5763.
Path 231 | total_timesteps 5788.
Path 232 | total_timesteps 5811.
Path 233 | total_timesteps 5832.
Path 234 | total_timesteps 5862.
Path 235 | total_timesteps 5873.
Path 236 | total_timesteps 5894.
Path 237 | total_timesteps 5910.
Path 238 | total_timesteps 5924.
Path 239 | total_timesteps 5944.
Path 240 | total_timesteps 5968.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.95    |
| Iteration     | 12       |
| MaximumReturn | 14.5     |
| MinimumReturn | -20.2    |
| TotalSamples  | 56093    |
----------------------------
itr #13 | 
Fitting dynamics.
Validation loss = 0.01215954590588808
Validation loss = 0.01047827210277319
Validation loss = 0.010000839829444885
Validation loss = 0.010691615752875805
Validation loss = 0.01058475486934185
Validation loss = 0.009320678189396858
Validation loss = 0.00971191842108965
Validation loss = 0.00966558326035738
Validation loss = 0.009594887495040894
Validation loss = 0.010135327465832233
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 50.
Path 2 | total_timesteps 79.
Path 3 | total_timesteps 97.
Path 4 | total_timesteps 114.
Path 5 | total_timesteps 138.
Path 6 | total_timesteps 167.
Path 7 | total_timesteps 188.
Path 8 | total_timesteps 217.
Path 9 | total_timesteps 244.
Path 10 | total_timesteps 272.
Path 11 | total_timesteps 302.
Path 12 | total_timesteps 325.
Path 13 | total_timesteps 351.
Path 14 | total_timesteps 366.
Path 15 | total_timesteps 394.
Path 16 | total_timesteps 416.
Path 17 | total_timesteps 438.
Path 18 | total_timesteps 464.
Path 19 | total_timesteps 489.
Path 20 | total_timesteps 533.
Path 21 | total_timesteps 560.
Path 22 | total_timesteps 581.
Path 23 | total_timesteps 612.
Path 24 | total_timesteps 652.
Path 25 | total_timesteps 669.
Path 26 | total_timesteps 697.
Path 27 | total_timesteps 731.
Path 28 | total_timesteps 760.
Path 29 | total_timesteps 791.
Path 30 | total_timesteps 832.
Path 31 | total_timesteps 851.
Path 32 | total_timesteps 875.
Path 33 | total_timesteps 902.
Path 34 | total_timesteps 920.
Path 35 | total_timesteps 957.
Path 36 | total_timesteps 979.
Path 37 | total_timesteps 1004.
Path 38 | total_timesteps 1019.
Path 39 | total_timesteps 1042.
Path 40 | total_timesteps 1072.
Path 41 | total_timesteps 1085.
Path 42 | total_timesteps 1129.
Path 43 | total_timesteps 1145.
Path 44 | total_timesteps 1163.
Path 45 | total_timesteps 1183.
Path 46 | total_timesteps 1207.
Path 47 | total_timesteps 1240.
Path 48 | total_timesteps 1271.
Path 49 | total_timesteps 1290.
Path 50 | total_timesteps 1317.
Path 51 | total_timesteps 1344.
Path 52 | total_timesteps 1370.
Path 53 | total_timesteps 1397.
Path 54 | total_timesteps 1414.
Path 55 | total_timesteps 1437.
Path 56 | total_timesteps 1461.
Path 57 | total_timesteps 1492.
Path 58 | total_timesteps 1531.
Path 59 | total_timesteps 1556.
Path 60 | total_timesteps 1580.
Path 61 | total_timesteps 1610.
Path 62 | total_timesteps 1630.
Path 63 | total_timesteps 1653.
Path 64 | total_timesteps 1675.
Path 65 | total_timesteps 1693.
Path 66 | total_timesteps 1711.
Path 67 | total_timesteps 1730.
Path 68 | total_timesteps 1752.
Path 69 | total_timesteps 1766.
Path 70 | total_timesteps 1807.
Path 71 | total_timesteps 1844.
Path 72 | total_timesteps 1863.
Path 73 | total_timesteps 1879.
Path 74 | total_timesteps 1923.
Path 75 | total_timesteps 1941.
Path 76 | total_timesteps 1977.
Path 77 | total_timesteps 1994.
Path 78 | total_timesteps 2015.
Path 79 | total_timesteps 2071.
Path 80 | total_timesteps 2098.
Path 81 | total_timesteps 2141.
Path 82 | total_timesteps 2163.
Path 83 | total_timesteps 2195.
Path 84 | total_timesteps 2217.
Path 85 | total_timesteps 2241.
Path 86 | total_timesteps 2263.
Path 87 | total_timesteps 2288.
Path 88 | total_timesteps 2321.
Path 89 | total_timesteps 2350.
Path 90 | total_timesteps 2369.
Path 91 | total_timesteps 2404.
Path 92 | total_timesteps 2418.
Path 93 | total_timesteps 2477.
Path 94 | total_timesteps 2502.
Path 95 | total_timesteps 2524.
Path 96 | total_timesteps 2586.
Path 97 | total_timesteps 2620.
Path 98 | total_timesteps 2637.
Path 99 | total_timesteps 2664.
Path 100 | total_timesteps 2683.
Path 101 | total_timesteps 2708.
Path 102 | total_timesteps 2727.
Path 103 | total_timesteps 2755.
Path 104 | total_timesteps 2821.
Path 105 | total_timesteps 2862.
Path 106 | total_timesteps 2881.
Path 107 | total_timesteps 2901.
Path 108 | total_timesteps 2925.
Path 109 | total_timesteps 2936.
Path 110 | total_timesteps 2964.
Path 111 | total_timesteps 2996.
Path 112 | total_timesteps 3081.
Path 113 | total_timesteps 3110.
Path 114 | total_timesteps 3117.
Path 115 | total_timesteps 3149.
Path 116 | total_timesteps 3165.
Path 117 | total_timesteps 3277.
Path 118 | total_timesteps 3327.
Path 119 | total_timesteps 3341.
Path 120 | total_timesteps 3353.
Path 121 | total_timesteps 3382.
Path 122 | total_timesteps 3393.
Path 123 | total_timesteps 3414.
Path 124 | total_timesteps 3476.
Path 125 | total_timesteps 3506.
Path 126 | total_timesteps 3529.
Path 127 | total_timesteps 3548.
Path 128 | total_timesteps 3585.
Path 129 | total_timesteps 3608.
Path 130 | total_timesteps 3626.
Path 131 | total_timesteps 3654.
Path 132 | total_timesteps 3690.
Path 133 | total_timesteps 3708.
Path 134 | total_timesteps 3755.
Path 135 | total_timesteps 3778.
Path 136 | total_timesteps 3834.
Path 137 | total_timesteps 3849.
Path 138 | total_timesteps 3924.
Path 139 | total_timesteps 3946.
Path 140 | total_timesteps 3985.
Path 141 | total_timesteps 4011.
Path 142 | total_timesteps 4033.
Path 143 | total_timesteps 4057.
Path 144 | total_timesteps 4076.
Path 145 | total_timesteps 4101.
Path 146 | total_timesteps 4197.
Path 147 | total_timesteps 4214.
Path 148 | total_timesteps 4242.
Path 149 | total_timesteps 4307.
Path 150 | total_timesteps 4330.
Path 151 | total_timesteps 4361.
Path 152 | total_timesteps 4419.
Path 153 | total_timesteps 4442.
Path 154 | total_timesteps 4479.
Path 155 | total_timesteps 4501.
Path 156 | total_timesteps 4537.
Path 157 | total_timesteps 4551.
Path 158 | total_timesteps 4578.
Path 159 | total_timesteps 4619.
Path 160 | total_timesteps 4652.
Path 161 | total_timesteps 4686.
Path 162 | total_timesteps 4729.
Path 163 | total_timesteps 4755.
Path 164 | total_timesteps 4789.
Path 165 | total_timesteps 4829.
Path 166 | total_timesteps 4870.
Path 167 | total_timesteps 4885.
Path 168 | total_timesteps 4914.
Path 169 | total_timesteps 4939.
Path 170 | total_timesteps 4982.
Path 171 | total_timesteps 5008.
Path 172 | total_timesteps 5030.
Path 173 | total_timesteps 5054.
Path 174 | total_timesteps 5070.
Path 175 | total_timesteps 5081.
Path 176 | total_timesteps 5122.
Path 177 | total_timesteps 5154.
Path 178 | total_timesteps 5181.
Path 179 | total_timesteps 5216.
Path 180 | total_timesteps 5260.
Path 181 | total_timesteps 5281.
Path 182 | total_timesteps 5312.
Path 183 | total_timesteps 5335.
Path 184 | total_timesteps 5371.
Path 185 | total_timesteps 5399.
Path 186 | total_timesteps 5432.
Path 187 | total_timesteps 5448.
Path 188 | total_timesteps 5481.
Path 189 | total_timesteps 5511.
Path 190 | total_timesteps 5540.
Path 191 | total_timesteps 5571.
Path 192 | total_timesteps 5600.
Path 193 | total_timesteps 5642.
Path 194 | total_timesteps 5653.
Path 195 | total_timesteps 5695.
Path 196 | total_timesteps 5758.
Path 197 | total_timesteps 5778.
Path 198 | total_timesteps 5808.
Path 199 | total_timesteps 5829.
Path 200 | total_timesteps 5888.
Path 201 | total_timesteps 5916.
Path 202 | total_timesteps 5931.
Path 203 | total_timesteps 5960.
Path 204 | total_timesteps 5984.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.02    |
| Iteration     | 13       |
| MaximumReturn | 36.8     |
| MinimumReturn | -41.1    |
| TotalSamples  | 60100    |
----------------------------
itr #14 | 
Fitting dynamics.
Validation loss = 0.009801579639315605
Validation loss = 0.009356051683425903
Validation loss = 0.009902938269078732
Validation loss = 0.009271357208490372
Validation loss = 0.009505173191428185
Validation loss = 0.00937100313603878
Validation loss = 0.00915218424052
Validation loss = 0.009434479288756847
Validation loss = 0.009161476977169514
Validation loss = 0.009257483296096325
Validation loss = 0.00946321152150631
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 22.
Path 2 | total_timesteps 78.
Path 3 | total_timesteps 116.
Path 4 | total_timesteps 147.
Path 5 | total_timesteps 171.
Path 6 | total_timesteps 202.
Path 7 | total_timesteps 219.
Path 8 | total_timesteps 243.
Path 9 | total_timesteps 259.
Path 10 | total_timesteps 277.
Path 11 | total_timesteps 304.
Path 12 | total_timesteps 326.
Path 13 | total_timesteps 336.
Path 14 | total_timesteps 352.
Path 15 | total_timesteps 375.
Path 16 | total_timesteps 394.
Path 17 | total_timesteps 424.
Path 18 | total_timesteps 448.
Path 19 | total_timesteps 468.
Path 20 | total_timesteps 542.
Path 21 | total_timesteps 572.
Path 22 | total_timesteps 597.
Path 23 | total_timesteps 633.
Path 24 | total_timesteps 661.
Path 25 | total_timesteps 676.
Path 26 | total_timesteps 697.
Path 27 | total_timesteps 727.
Path 28 | total_timesteps 743.
Path 29 | total_timesteps 761.
Path 30 | total_timesteps 820.
Path 31 | total_timesteps 847.
Path 32 | total_timesteps 870.
Path 33 | total_timesteps 896.
Path 34 | total_timesteps 918.
Path 35 | total_timesteps 962.
Path 36 | total_timesteps 986.
Path 37 | total_timesteps 1008.
Path 38 | total_timesteps 1029.
Path 39 | total_timesteps 1050.
Path 40 | total_timesteps 1091.
Path 41 | total_timesteps 1126.
Path 42 | total_timesteps 1146.
Path 43 | total_timesteps 1169.
Path 44 | total_timesteps 1192.
Path 45 | total_timesteps 1210.
Path 46 | total_timesteps 1223.
Path 47 | total_timesteps 1255.
Path 48 | total_timesteps 1291.
Path 49 | total_timesteps 1305.
Path 50 | total_timesteps 1322.
Path 51 | total_timesteps 1352.
Path 52 | total_timesteps 1381.
Path 53 | total_timesteps 1410.
Path 54 | total_timesteps 1439.
Path 55 | total_timesteps 1466.
Path 56 | total_timesteps 1512.
Path 57 | total_timesteps 1535.
Path 58 | total_timesteps 1561.
Path 59 | total_timesteps 1583.
Path 60 | total_timesteps 1630.
Path 61 | total_timesteps 1685.
Path 62 | total_timesteps 1722.
Path 63 | total_timesteps 1741.
Path 64 | total_timesteps 1757.
Path 65 | total_timesteps 1779.
Path 66 | total_timesteps 1812.
Path 67 | total_timesteps 1844.
Path 68 | total_timesteps 1865.
Path 69 | total_timesteps 1889.
Path 70 | total_timesteps 1899.
Path 71 | total_timesteps 1930.
Path 72 | total_timesteps 1962.
Path 73 | total_timesteps 1982.
Path 74 | total_timesteps 2001.
Path 75 | total_timesteps 2030.
Path 76 | total_timesteps 2042.
Path 77 | total_timesteps 2084.
Path 78 | total_timesteps 2123.
Path 79 | total_timesteps 2148.
Path 80 | total_timesteps 2156.
Path 81 | total_timesteps 2170.
Path 82 | total_timesteps 2183.
Path 83 | total_timesteps 2213.
Path 84 | total_timesteps 2240.
Path 85 | total_timesteps 2253.
Path 86 | total_timesteps 2286.
Path 87 | total_timesteps 2315.
Path 88 | total_timesteps 2332.
Path 89 | total_timesteps 2392.
Path 90 | total_timesteps 2412.
Path 91 | total_timesteps 2445.
Path 92 | total_timesteps 2462.
Path 93 | total_timesteps 2499.
Path 94 | total_timesteps 2517.
Path 95 | total_timesteps 2533.
Path 96 | total_timesteps 2596.
Path 97 | total_timesteps 2614.
Path 98 | total_timesteps 2626.
Path 99 | total_timesteps 2638.
Path 100 | total_timesteps 2653.
Path 101 | total_timesteps 2678.
Path 102 | total_timesteps 2700.
Path 103 | total_timesteps 2740.
Path 104 | total_timesteps 2775.
Path 105 | total_timesteps 2795.
Path 106 | total_timesteps 2820.
Path 107 | total_timesteps 2841.
Path 108 | total_timesteps 2864.
Path 109 | total_timesteps 2889.
Path 110 | total_timesteps 2911.
Path 111 | total_timesteps 2931.
Path 112 | total_timesteps 2947.
Path 113 | total_timesteps 2961.
Path 114 | total_timesteps 2993.
Path 115 | total_timesteps 3032.
Path 116 | total_timesteps 3052.
Path 117 | total_timesteps 3077.
Path 118 | total_timesteps 3098.
Path 119 | total_timesteps 3119.
Path 120 | total_timesteps 3132.
Path 121 | total_timesteps 3196.
Path 122 | total_timesteps 3218.
Path 123 | total_timesteps 3241.
Path 124 | total_timesteps 3262.
Path 125 | total_timesteps 3292.
Path 126 | total_timesteps 3314.
Path 127 | total_timesteps 3341.
Path 128 | total_timesteps 3361.
Path 129 | total_timesteps 3381.
Path 130 | total_timesteps 3407.
Path 131 | total_timesteps 3436.
Path 132 | total_timesteps 3468.
Path 133 | total_timesteps 3486.
Path 134 | total_timesteps 3511.
Path 135 | total_timesteps 3538.
Path 136 | total_timesteps 3551.
Path 137 | total_timesteps 3574.
Path 138 | total_timesteps 3624.
Path 139 | total_timesteps 3646.
Path 140 | total_timesteps 3675.
Path 141 | total_timesteps 3691.
Path 142 | total_timesteps 3735.
Path 143 | total_timesteps 3781.
Path 144 | total_timesteps 3805.
Path 145 | total_timesteps 3824.
Path 146 | total_timesteps 3883.
Path 147 | total_timesteps 3901.
Path 148 | total_timesteps 3921.
Path 149 | total_timesteps 3943.
Path 150 | total_timesteps 3969.
Path 151 | total_timesteps 3992.
Path 152 | total_timesteps 4021.
Path 153 | total_timesteps 4035.
Path 154 | total_timesteps 4052.
Path 155 | total_timesteps 4071.
Path 156 | total_timesteps 4110.
Path 157 | total_timesteps 4129.
Path 158 | total_timesteps 4149.
Path 159 | total_timesteps 4182.
Path 160 | total_timesteps 4211.
Path 161 | total_timesteps 4238.
Path 162 | total_timesteps 4254.
Path 163 | total_timesteps 4270.
Path 164 | total_timesteps 4296.
Path 165 | total_timesteps 4324.
Path 166 | total_timesteps 4346.
Path 167 | total_timesteps 4364.
Path 168 | total_timesteps 4386.
Path 169 | total_timesteps 4445.
Path 170 | total_timesteps 4465.
Path 171 | total_timesteps 4486.
Path 172 | total_timesteps 4518.
Path 173 | total_timesteps 4538.
Path 174 | total_timesteps 4570.
Path 175 | total_timesteps 4609.
Path 176 | total_timesteps 4626.
Path 177 | total_timesteps 4653.
Path 178 | total_timesteps 4699.
Path 179 | total_timesteps 4721.
Path 180 | total_timesteps 4750.
Path 181 | total_timesteps 4784.
Path 182 | total_timesteps 4813.
Path 183 | total_timesteps 4832.
Path 184 | total_timesteps 4870.
Path 185 | total_timesteps 4882.
Path 186 | total_timesteps 4907.
Path 187 | total_timesteps 4929.
Path 188 | total_timesteps 4942.
Path 189 | total_timesteps 4957.
Path 190 | total_timesteps 4980.
Path 191 | total_timesteps 5012.
Path 192 | total_timesteps 5052.
Path 193 | total_timesteps 5073.
Path 194 | total_timesteps 5104.
Path 195 | total_timesteps 5141.
Path 196 | total_timesteps 5170.
Path 197 | total_timesteps 5198.
Path 198 | total_timesteps 5214.
Path 199 | total_timesteps 5241.
Path 200 | total_timesteps 5260.
Path 201 | total_timesteps 5300.
Path 202 | total_timesteps 5326.
Path 203 | total_timesteps 5349.
Path 204 | total_timesteps 5371.
Path 205 | total_timesteps 5395.
Path 206 | total_timesteps 5409.
Path 207 | total_timesteps 5444.
Path 208 | total_timesteps 5474.
Path 209 | total_timesteps 5538.
Path 210 | total_timesteps 5555.
Path 211 | total_timesteps 5583.
Path 212 | total_timesteps 5591.
Path 213 | total_timesteps 5622.
Path 214 | total_timesteps 5643.
Path 215 | total_timesteps 5661.
Path 216 | total_timesteps 5672.
Path 217 | total_timesteps 5706.
Path 218 | total_timesteps 5734.
Path 219 | total_timesteps 5758.
Path 220 | total_timesteps 5776.
Path 221 | total_timesteps 5805.
Path 222 | total_timesteps 5836.
Path 223 | total_timesteps 5854.
Path 224 | total_timesteps 5876.
Path 225 | total_timesteps 5934.
Path 226 | total_timesteps 5954.
Path 227 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.26    |
| Iteration     | 14       |
| MaximumReturn | 18       |
| MinimumReturn | -31.2    |
| TotalSamples  | 64113    |
----------------------------
itr #15 | 
Fitting dynamics.
Validation loss = 0.010302181355655193
Validation loss = 0.0090592997148633
Validation loss = 0.009817018173635006
Validation loss = 0.010138969868421555
Validation loss = 0.009226626716554165
Validation loss = 0.00854199007153511
Validation loss = 0.00848911888897419
Validation loss = 0.009094228968024254
Validation loss = 0.008769199252128601
Validation loss = 0.008516853675246239
Validation loss = 0.00898681115359068
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 20.
Path 2 | total_timesteps 47.
Path 3 | total_timesteps 78.
Path 4 | total_timesteps 90.
Path 5 | total_timesteps 112.
Path 6 | total_timesteps 135.
Path 7 | total_timesteps 159.
Path 8 | total_timesteps 172.
Path 9 | total_timesteps 205.
Path 10 | total_timesteps 227.
Path 11 | total_timesteps 263.
Path 12 | total_timesteps 287.
Path 13 | total_timesteps 307.
Path 14 | total_timesteps 340.
Path 15 | total_timesteps 367.
Path 16 | total_timesteps 386.
Path 17 | total_timesteps 428.
Path 18 | total_timesteps 461.
Path 19 | total_timesteps 489.
Path 20 | total_timesteps 512.
Path 21 | total_timesteps 536.
Path 22 | total_timesteps 551.
Path 23 | total_timesteps 580.
Path 24 | total_timesteps 601.
Path 25 | total_timesteps 620.
Path 26 | total_timesteps 645.
Path 27 | total_timesteps 732.
Path 28 | total_timesteps 746.
Path 29 | total_timesteps 761.
Path 30 | total_timesteps 783.
Path 31 | total_timesteps 813.
Path 32 | total_timesteps 835.
Path 33 | total_timesteps 887.
Path 34 | total_timesteps 904.
Path 35 | total_timesteps 918.
Path 36 | total_timesteps 947.
Path 37 | total_timesteps 974.
Path 38 | total_timesteps 1009.
Path 39 | total_timesteps 1019.
Path 40 | total_timesteps 1037.
Path 41 | total_timesteps 1069.
Path 42 | total_timesteps 1100.
Path 43 | total_timesteps 1120.
Path 44 | total_timesteps 1139.
Path 45 | total_timesteps 1193.
Path 46 | total_timesteps 1224.
Path 47 | total_timesteps 1257.
Path 48 | total_timesteps 1287.
Path 49 | total_timesteps 1309.
Path 50 | total_timesteps 1336.
Path 51 | total_timesteps 1361.
Path 52 | total_timesteps 1370.
Path 53 | total_timesteps 1387.
Path 54 | total_timesteps 1428.
Path 55 | total_timesteps 1445.
Path 56 | total_timesteps 1468.
Path 57 | total_timesteps 1518.
Path 58 | total_timesteps 1535.
Path 59 | total_timesteps 1558.
Path 60 | total_timesteps 1605.
Path 61 | total_timesteps 1624.
Path 62 | total_timesteps 1658.
Path 63 | total_timesteps 1673.
Path 64 | total_timesteps 1693.
Path 65 | total_timesteps 1723.
Path 66 | total_timesteps 1753.
Path 67 | total_timesteps 1771.
Path 68 | total_timesteps 1780.
Path 69 | total_timesteps 1799.
Path 70 | total_timesteps 1816.
Path 71 | total_timesteps 1835.
Path 72 | total_timesteps 1869.
Path 73 | total_timesteps 1907.
Path 74 | total_timesteps 1939.
Path 75 | total_timesteps 1959.
Path 76 | total_timesteps 1976.
Path 77 | total_timesteps 1996.
Path 78 | total_timesteps 2028.
Path 79 | total_timesteps 2045.
Path 80 | total_timesteps 2064.
Path 81 | total_timesteps 2078.
Path 82 | total_timesteps 2101.
Path 83 | total_timesteps 2126.
Path 84 | total_timesteps 2148.
Path 85 | total_timesteps 2182.
Path 86 | total_timesteps 2232.
Path 87 | total_timesteps 2248.
Path 88 | total_timesteps 2268.
Path 89 | total_timesteps 2295.
Path 90 | total_timesteps 2335.
Path 91 | total_timesteps 2358.
Path 92 | total_timesteps 2381.
Path 93 | total_timesteps 2399.
Path 94 | total_timesteps 2412.
Path 95 | total_timesteps 2438.
Path 96 | total_timesteps 2462.
Path 97 | total_timesteps 2481.
Path 98 | total_timesteps 2503.
Path 99 | total_timesteps 2520.
Path 100 | total_timesteps 2544.
Path 101 | total_timesteps 2563.
Path 102 | total_timesteps 2598.
Path 103 | total_timesteps 2613.
Path 104 | total_timesteps 2646.
Path 105 | total_timesteps 2665.
Path 106 | total_timesteps 2684.
Path 107 | total_timesteps 2700.
Path 108 | total_timesteps 2739.
Path 109 | total_timesteps 2776.
Path 110 | total_timesteps 2847.
Path 111 | total_timesteps 2869.
Path 112 | total_timesteps 2924.
Path 113 | total_timesteps 2985.
Path 114 | total_timesteps 3002.
Path 115 | total_timesteps 3029.
Path 116 | total_timesteps 3045.
Path 117 | total_timesteps 3062.
Path 118 | total_timesteps 3088.
Path 119 | total_timesteps 3109.
Path 120 | total_timesteps 3146.
Path 121 | total_timesteps 3180.
Path 122 | total_timesteps 3202.
Path 123 | total_timesteps 3222.
Path 124 | total_timesteps 3250.
Path 125 | total_timesteps 3267.
Path 126 | total_timesteps 3283.
Path 127 | total_timesteps 3311.
Path 128 | total_timesteps 3333.
Path 129 | total_timesteps 3390.
Path 130 | total_timesteps 3509.
Path 131 | total_timesteps 3531.
Path 132 | total_timesteps 3546.
Path 133 | total_timesteps 3592.
Path 134 | total_timesteps 3614.
Path 135 | total_timesteps 3639.
Path 136 | total_timesteps 3665.
Path 137 | total_timesteps 3684.
Path 138 | total_timesteps 3722.
Path 139 | total_timesteps 3761.
Path 140 | total_timesteps 3776.
Path 141 | total_timesteps 3812.
Path 142 | total_timesteps 3850.
Path 143 | total_timesteps 3869.
Path 144 | total_timesteps 3900.
Path 145 | total_timesteps 3919.
Path 146 | total_timesteps 3954.
Path 147 | total_timesteps 3979.
Path 148 | total_timesteps 3994.
Path 149 | total_timesteps 4019.
Path 150 | total_timesteps 4032.
Path 151 | total_timesteps 4044.
Path 152 | total_timesteps 4064.
Path 153 | total_timesteps 4106.
Path 154 | total_timesteps 4144.
Path 155 | total_timesteps 4162.
Path 156 | total_timesteps 4186.
Path 157 | total_timesteps 4263.
Path 158 | total_timesteps 4325.
Path 159 | total_timesteps 4343.
Path 160 | total_timesteps 4356.
Path 161 | total_timesteps 4379.
Path 162 | total_timesteps 4399.
Path 163 | total_timesteps 4416.
Path 164 | total_timesteps 4442.
Path 165 | total_timesteps 4472.
Path 166 | total_timesteps 4498.
Path 167 | total_timesteps 4511.
Path 168 | total_timesteps 4534.
Path 169 | total_timesteps 4566.
Path 170 | total_timesteps 4586.
Path 171 | total_timesteps 4610.
Path 172 | total_timesteps 4658.
Path 173 | total_timesteps 4688.
Path 174 | total_timesteps 4711.
Path 175 | total_timesteps 4736.
Path 176 | total_timesteps 4762.
Path 177 | total_timesteps 4798.
Path 178 | total_timesteps 4825.
Path 179 | total_timesteps 4850.
Path 180 | total_timesteps 4864.
Path 181 | total_timesteps 4883.
Path 182 | total_timesteps 4933.
Path 183 | total_timesteps 5006.
Path 184 | total_timesteps 5018.
Path 185 | total_timesteps 5029.
Path 186 | total_timesteps 5043.
Path 187 | total_timesteps 5082.
Path 188 | total_timesteps 5106.
Path 189 | total_timesteps 5136.
Path 190 | total_timesteps 5156.
Path 191 | total_timesteps 5191.
Path 192 | total_timesteps 5242.
Path 193 | total_timesteps 5267.
Path 194 | total_timesteps 5294.
Path 195 | total_timesteps 5317.
Path 196 | total_timesteps 5336.
Path 197 | total_timesteps 5361.
Path 198 | total_timesteps 5398.
Path 199 | total_timesteps 5417.
Path 200 | total_timesteps 5437.
Path 201 | total_timesteps 5465.
Path 202 | total_timesteps 5495.
Path 203 | total_timesteps 5526.
Path 204 | total_timesteps 5561.
Path 205 | total_timesteps 5584.
Path 206 | total_timesteps 5610.
Path 207 | total_timesteps 5637.
Path 208 | total_timesteps 5668.
Path 209 | total_timesteps 5708.
Path 210 | total_timesteps 5734.
Path 211 | total_timesteps 5762.
Path 212 | total_timesteps 5772.
Path 213 | total_timesteps 5788.
Path 214 | total_timesteps 5803.
Path 215 | total_timesteps 5826.
Path 216 | total_timesteps 5886.
Path 217 | total_timesteps 5923.
Path 218 | total_timesteps 5946.
Path 219 | total_timesteps 5959.
Path 220 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.34    |
| Iteration     | 15       |
| MaximumReturn | 97.3     |
| MinimumReturn | -26.3    |
| TotalSamples  | 68117    |
----------------------------
itr #16 | 
Fitting dynamics.
Validation loss = 0.008627564646303654
Validation loss = 0.008239011280238628
Validation loss = 0.008799965493381023
Validation loss = 0.008679361082613468
Validation loss = 0.00833526998758316
Validation loss = 0.00820036418735981
Validation loss = 0.00884097721427679
Validation loss = 0.008672840893268585
Validation loss = 0.00818206463009119
Validation loss = 0.008388321846723557
Validation loss = 0.007852603681385517
Validation loss = 0.008197140879929066
Validation loss = 0.008461722172796726
Validation loss = 0.007809824775904417
Validation loss = 0.007830888032913208
Validation loss = 0.007605079561471939
Validation loss = 0.00805685669183731
Validation loss = 0.008006229996681213
Validation loss = 0.008796172216534615
Validation loss = 0.00814421009272337
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 19.
Path 2 | total_timesteps 45.
Path 3 | total_timesteps 80.
Path 4 | total_timesteps 105.
Path 5 | total_timesteps 119.
Path 6 | total_timesteps 145.
Path 7 | total_timesteps 175.
Path 8 | total_timesteps 194.
Path 9 | total_timesteps 205.
Path 10 | total_timesteps 227.
Path 11 | total_timesteps 265.
Path 12 | total_timesteps 279.
Path 13 | total_timesteps 291.
Path 14 | total_timesteps 328.
Path 15 | total_timesteps 350.
Path 16 | total_timesteps 369.
Path 17 | total_timesteps 452.
Path 18 | total_timesteps 468.
Path 19 | total_timesteps 515.
Path 20 | total_timesteps 533.
Path 21 | total_timesteps 559.
Path 22 | total_timesteps 580.
Path 23 | total_timesteps 592.
Path 24 | total_timesteps 602.
Path 25 | total_timesteps 625.
Path 26 | total_timesteps 658.
Path 27 | total_timesteps 669.
Path 28 | total_timesteps 700.
Path 29 | total_timesteps 715.
Path 30 | total_timesteps 743.
Path 31 | total_timesteps 766.
Path 32 | total_timesteps 790.
Path 33 | total_timesteps 823.
Path 34 | total_timesteps 836.
Path 35 | total_timesteps 849.
Path 36 | total_timesteps 870.
Path 37 | total_timesteps 889.
Path 38 | total_timesteps 912.
Path 39 | total_timesteps 953.
Path 40 | total_timesteps 973.
Path 41 | total_timesteps 993.
Path 42 | total_timesteps 1018.
Path 43 | total_timesteps 1044.
Path 44 | total_timesteps 1059.
Path 45 | total_timesteps 1097.
Path 46 | total_timesteps 1126.
Path 47 | total_timesteps 1147.
Path 48 | total_timesteps 1172.
Path 49 | total_timesteps 1196.
Path 50 | total_timesteps 1210.
Path 51 | total_timesteps 1245.
Path 52 | total_timesteps 1273.
Path 53 | total_timesteps 1301.
Path 54 | total_timesteps 1317.
Path 55 | total_timesteps 1339.
Path 56 | total_timesteps 1359.
Path 57 | total_timesteps 1394.
Path 58 | total_timesteps 1413.
Path 59 | total_timesteps 1429.
Path 60 | total_timesteps 1452.
Path 61 | total_timesteps 1473.
Path 62 | total_timesteps 1503.
Path 63 | total_timesteps 1522.
Path 64 | total_timesteps 1533.
Path 65 | total_timesteps 1568.
Path 66 | total_timesteps 1594.
Path 67 | total_timesteps 1605.
Path 68 | total_timesteps 1620.
Path 69 | total_timesteps 1639.
Path 70 | total_timesteps 1659.
Path 71 | total_timesteps 1678.
Path 72 | total_timesteps 1723.
Path 73 | total_timesteps 1749.
Path 74 | total_timesteps 1759.
Path 75 | total_timesteps 1787.
Path 76 | total_timesteps 1812.
Path 77 | total_timesteps 1847.
Path 78 | total_timesteps 1891.
Path 79 | total_timesteps 1913.
Path 80 | total_timesteps 1922.
Path 81 | total_timesteps 1939.
Path 82 | total_timesteps 1962.
Path 83 | total_timesteps 1984.
Path 84 | total_timesteps 2003.
Path 85 | total_timesteps 2048.
Path 86 | total_timesteps 2061.
Path 87 | total_timesteps 2102.
Path 88 | total_timesteps 2117.
Path 89 | total_timesteps 2149.
Path 90 | total_timesteps 2176.
Path 91 | total_timesteps 2198.
Path 92 | total_timesteps 2210.
Path 93 | total_timesteps 2254.
Path 94 | total_timesteps 2280.
Path 95 | total_timesteps 2300.
Path 96 | total_timesteps 2329.
Path 97 | total_timesteps 2343.
Path 98 | total_timesteps 2368.
Path 99 | total_timesteps 2385.
Path 100 | total_timesteps 2408.
Path 101 | total_timesteps 2428.
Path 102 | total_timesteps 2455.
Path 103 | total_timesteps 2467.
Path 104 | total_timesteps 2493.
Path 105 | total_timesteps 2525.
Path 106 | total_timesteps 2544.
Path 107 | total_timesteps 2570.
Path 108 | total_timesteps 2587.
Path 109 | total_timesteps 2600.
Path 110 | total_timesteps 2626.
Path 111 | total_timesteps 2658.
Path 112 | total_timesteps 2693.
Path 113 | total_timesteps 2732.
Path 114 | total_timesteps 2747.
Path 115 | total_timesteps 2776.
Path 116 | total_timesteps 2805.
Path 117 | total_timesteps 2855.
Path 118 | total_timesteps 2881.
Path 119 | total_timesteps 2918.
Path 120 | total_timesteps 2944.
Path 121 | total_timesteps 2958.
Path 122 | total_timesteps 2974.
Path 123 | total_timesteps 2993.
Path 124 | total_timesteps 3010.
Path 125 | total_timesteps 3033.
Path 126 | total_timesteps 3051.
Path 127 | total_timesteps 3073.
Path 128 | total_timesteps 3088.
Path 129 | total_timesteps 3106.
Path 130 | total_timesteps 3127.
Path 131 | total_timesteps 3148.
Path 132 | total_timesteps 3180.
Path 133 | total_timesteps 3198.
Path 134 | total_timesteps 3216.
Path 135 | total_timesteps 3235.
Path 136 | total_timesteps 3259.
Path 137 | total_timesteps 3289.
Path 138 | total_timesteps 3305.
Path 139 | total_timesteps 3331.
Path 140 | total_timesteps 3343.
Path 141 | total_timesteps 3360.
Path 142 | total_timesteps 3376.
Path 143 | total_timesteps 3402.
Path 144 | total_timesteps 3433.
Path 145 | total_timesteps 3457.
Path 146 | total_timesteps 3482.
Path 147 | total_timesteps 3510.
Path 148 | total_timesteps 3565.
Path 149 | total_timesteps 3591.
Path 150 | total_timesteps 3610.
Path 151 | total_timesteps 3639.
Path 152 | total_timesteps 3660.
Path 153 | total_timesteps 3671.
Path 154 | total_timesteps 3682.
Path 155 | total_timesteps 3706.
Path 156 | total_timesteps 3725.
Path 157 | total_timesteps 3735.
Path 158 | total_timesteps 3753.
Path 159 | total_timesteps 3778.
Path 160 | total_timesteps 3790.
Path 161 | total_timesteps 3809.
Path 162 | total_timesteps 3828.
Path 163 | total_timesteps 3846.
Path 164 | total_timesteps 3884.
Path 165 | total_timesteps 3925.
Path 166 | total_timesteps 3948.
Path 167 | total_timesteps 3979.
Path 168 | total_timesteps 4030.
Path 169 | total_timesteps 4040.
Path 170 | total_timesteps 4066.
Path 171 | total_timesteps 4083.
Path 172 | total_timesteps 4105.
Path 173 | total_timesteps 4142.
Path 174 | total_timesteps 4170.
Path 175 | total_timesteps 4207.
Path 176 | total_timesteps 4248.
Path 177 | total_timesteps 4268.
Path 178 | total_timesteps 4313.
Path 179 | total_timesteps 4342.
Path 180 | total_timesteps 4355.
Path 181 | total_timesteps 4388.
Path 182 | total_timesteps 4437.
Path 183 | total_timesteps 4465.
Path 184 | total_timesteps 4489.
Path 185 | total_timesteps 4517.
Path 186 | total_timesteps 4542.
Path 187 | total_timesteps 4561.
Path 188 | total_timesteps 4610.
Path 189 | total_timesteps 4629.
Path 190 | total_timesteps 4641.
Path 191 | total_timesteps 4659.
Path 192 | total_timesteps 4686.
Path 193 | total_timesteps 4701.
Path 194 | total_timesteps 4712.
Path 195 | total_timesteps 4734.
Path 196 | total_timesteps 4752.
Path 197 | total_timesteps 4763.
Path 198 | total_timesteps 4778.
Path 199 | total_timesteps 4812.
Path 200 | total_timesteps 4824.
Path 201 | total_timesteps 4864.
Path 202 | total_timesteps 4884.
Path 203 | total_timesteps 4899.
Path 204 | total_timesteps 4939.
Path 205 | total_timesteps 4958.
Path 206 | total_timesteps 4985.
Path 207 | total_timesteps 5020.
Path 208 | total_timesteps 5046.
Path 209 | total_timesteps 5072.
Path 210 | total_timesteps 5087.
Path 211 | total_timesteps 5104.
Path 212 | total_timesteps 5122.
Path 213 | total_timesteps 5167.
Path 214 | total_timesteps 5203.
Path 215 | total_timesteps 5241.
Path 216 | total_timesteps 5284.
Path 217 | total_timesteps 5315.
Path 218 | total_timesteps 5331.
Path 219 | total_timesteps 5352.
Path 220 | total_timesteps 5392.
Path 221 | total_timesteps 5413.
Path 222 | total_timesteps 5438.
Path 223 | total_timesteps 5450.
Path 224 | total_timesteps 5466.
Path 225 | total_timesteps 5492.
Path 226 | total_timesteps 5526.
Path 227 | total_timesteps 5559.
Path 228 | total_timesteps 5574.
Path 229 | total_timesteps 5593.
Path 230 | total_timesteps 5615.
Path 231 | total_timesteps 5640.
Path 232 | total_timesteps 5656.
Path 233 | total_timesteps 5684.
Path 234 | total_timesteps 5706.
Path 235 | total_timesteps 5725.
Path 236 | total_timesteps 5751.
Path 237 | total_timesteps 5772.
Path 238 | total_timesteps 5792.
Path 239 | total_timesteps 5819.
Path 240 | total_timesteps 5844.
Path 241 | total_timesteps 5861.
Path 242 | total_timesteps 5879.
Path 243 | total_timesteps 5897.
Path 244 | total_timesteps 5915.
Path 245 | total_timesteps 5941.
Path 246 | total_timesteps 5979.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.39    |
| Iteration     | 16       |
| MaximumReturn | 26.9     |
| MinimumReturn | -22.6    |
| TotalSamples  | 72124    |
----------------------------
itr #17 | 
Fitting dynamics.
Validation loss = 0.008178438991308212
Validation loss = 0.008038019761443138
Validation loss = 0.008216558024287224
Validation loss = 0.00823197141289711
Validation loss = 0.007872401736676693
Validation loss = 0.0076600853353738785
Validation loss = 0.007851291447877884
Validation loss = 0.00815833080559969
Validation loss = 0.007686689496040344
Validation loss = 0.007841194979846478
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 29.
Path 2 | total_timesteps 58.
Path 3 | total_timesteps 91.
Path 4 | total_timesteps 113.
Path 5 | total_timesteps 124.
Path 6 | total_timesteps 149.
Path 7 | total_timesteps 171.
Path 8 | total_timesteps 193.
Path 9 | total_timesteps 222.
Path 10 | total_timesteps 243.
Path 11 | total_timesteps 261.
Path 12 | total_timesteps 304.
Path 13 | total_timesteps 318.
Path 14 | total_timesteps 340.
Path 15 | total_timesteps 364.
Path 16 | total_timesteps 385.
Path 17 | total_timesteps 404.
Path 18 | total_timesteps 443.
Path 19 | total_timesteps 470.
Path 20 | total_timesteps 488.
Path 21 | total_timesteps 512.
Path 22 | total_timesteps 531.
Path 23 | total_timesteps 575.
Path 24 | total_timesteps 594.
Path 25 | total_timesteps 621.
Path 26 | total_timesteps 648.
Path 27 | total_timesteps 665.
Path 28 | total_timesteps 693.
Path 29 | total_timesteps 715.
Path 30 | total_timesteps 741.
Path 31 | total_timesteps 763.
Path 32 | total_timesteps 773.
Path 33 | total_timesteps 785.
Path 34 | total_timesteps 821.
Path 35 | total_timesteps 841.
Path 36 | total_timesteps 863.
Path 37 | total_timesteps 938.
Path 38 | total_timesteps 995.
Path 39 | total_timesteps 1019.
Path 40 | total_timesteps 1033.
Path 41 | total_timesteps 1055.
Path 42 | total_timesteps 1066.
Path 43 | total_timesteps 1103.
Path 44 | total_timesteps 1128.
Path 45 | total_timesteps 1139.
Path 46 | total_timesteps 1154.
Path 47 | total_timesteps 1169.
Path 48 | total_timesteps 1193.
Path 49 | total_timesteps 1227.
Path 50 | total_timesteps 1246.
Path 51 | total_timesteps 1259.
Path 52 | total_timesteps 1274.
Path 53 | total_timesteps 1290.
Path 54 | total_timesteps 1317.
Path 55 | total_timesteps 1355.
Path 56 | total_timesteps 1423.
Path 57 | total_timesteps 1444.
Path 58 | total_timesteps 1469.
Path 59 | total_timesteps 1510.
Path 60 | total_timesteps 1530.
Path 61 | total_timesteps 1574.
Path 62 | total_timesteps 1583.
Path 63 | total_timesteps 1606.
Path 64 | total_timesteps 1629.
Path 65 | total_timesteps 1652.
Path 66 | total_timesteps 1670.
Path 67 | total_timesteps 1689.
Path 68 | total_timesteps 1701.
Path 69 | total_timesteps 1739.
Path 70 | total_timesteps 1756.
Path 71 | total_timesteps 1779.
Path 72 | total_timesteps 1798.
Path 73 | total_timesteps 1820.
Path 74 | total_timesteps 1877.
Path 75 | total_timesteps 1895.
Path 76 | total_timesteps 1926.
Path 77 | total_timesteps 1947.
Path 78 | total_timesteps 1967.
Path 79 | total_timesteps 1993.
Path 80 | total_timesteps 2016.
Path 81 | total_timesteps 2052.
Path 82 | total_timesteps 2074.
Path 83 | total_timesteps 2091.
Path 84 | total_timesteps 2108.
Path 85 | total_timesteps 2118.
Path 86 | total_timesteps 2150.
Path 87 | total_timesteps 2169.
Path 88 | total_timesteps 2180.
Path 89 | total_timesteps 2221.
Path 90 | total_timesteps 2234.
Path 91 | total_timesteps 2264.
Path 92 | total_timesteps 2292.
Path 93 | total_timesteps 2308.
Path 94 | total_timesteps 2330.
Path 95 | total_timesteps 2351.
Path 96 | total_timesteps 2371.
Path 97 | total_timesteps 2398.
Path 98 | total_timesteps 2425.
Path 99 | total_timesteps 2443.
Path 100 | total_timesteps 2457.
Path 101 | total_timesteps 2480.
Path 102 | total_timesteps 2507.
Path 103 | total_timesteps 2541.
Path 104 | total_timesteps 2557.
Path 105 | total_timesteps 2586.
Path 106 | total_timesteps 2598.
Path 107 | total_timesteps 2619.
Path 108 | total_timesteps 2634.
Path 109 | total_timesteps 2669.
Path 110 | total_timesteps 2693.
Path 111 | total_timesteps 2721.
Path 112 | total_timesteps 2745.
Path 113 | total_timesteps 2774.
Path 114 | total_timesteps 2814.
Path 115 | total_timesteps 2841.
Path 116 | total_timesteps 2873.
Path 117 | total_timesteps 2893.
Path 118 | total_timesteps 2924.
Path 119 | total_timesteps 2946.
Path 120 | total_timesteps 2955.
Path 121 | total_timesteps 2980.
Path 122 | total_timesteps 3003.
Path 123 | total_timesteps 3023.
Path 124 | total_timesteps 3048.
Path 125 | total_timesteps 3077.
Path 126 | total_timesteps 3102.
Path 127 | total_timesteps 3121.
Path 128 | total_timesteps 3142.
Path 129 | total_timesteps 3161.
Path 130 | total_timesteps 3168.
Path 131 | total_timesteps 3187.
Path 132 | total_timesteps 3206.
Path 133 | total_timesteps 3229.
Path 134 | total_timesteps 3256.
Path 135 | total_timesteps 3285.
Path 136 | total_timesteps 3309.
Path 137 | total_timesteps 3334.
Path 138 | total_timesteps 3369.
Path 139 | total_timesteps 3400.
Path 140 | total_timesteps 3428.
Path 141 | total_timesteps 3461.
Path 142 | total_timesteps 3492.
Path 143 | total_timesteps 3505.
Path 144 | total_timesteps 3524.
Path 145 | total_timesteps 3555.
Path 146 | total_timesteps 3574.
Path 147 | total_timesteps 3595.
Path 148 | total_timesteps 3622.
Path 149 | total_timesteps 3638.
Path 150 | total_timesteps 3662.
Path 151 | total_timesteps 3677.
Path 152 | total_timesteps 3697.
Path 153 | total_timesteps 3717.
Path 154 | total_timesteps 3731.
Path 155 | total_timesteps 3751.
Path 156 | total_timesteps 3773.
Path 157 | total_timesteps 3793.
Path 158 | total_timesteps 3824.
Path 159 | total_timesteps 3850.
Path 160 | total_timesteps 3864.
Path 161 | total_timesteps 3881.
Path 162 | total_timesteps 3900.
Path 163 | total_timesteps 3911.
Path 164 | total_timesteps 3925.
Path 165 | total_timesteps 3945.
Path 166 | total_timesteps 3982.
Path 167 | total_timesteps 3992.
Path 168 | total_timesteps 4012.
Path 169 | total_timesteps 4043.
Path 170 | total_timesteps 4059.
Path 171 | total_timesteps 4078.
Path 172 | total_timesteps 4097.
Path 173 | total_timesteps 4111.
Path 174 | total_timesteps 4125.
Path 175 | total_timesteps 4139.
Path 176 | total_timesteps 4152.
Path 177 | total_timesteps 4196.
Path 178 | total_timesteps 4230.
Path 179 | total_timesteps 4268.
Path 180 | total_timesteps 4285.
Path 181 | total_timesteps 4299.
Path 182 | total_timesteps 4326.
Path 183 | total_timesteps 4342.
Path 184 | total_timesteps 4360.
Path 185 | total_timesteps 4392.
Path 186 | total_timesteps 4415.
Path 187 | total_timesteps 4426.
Path 188 | total_timesteps 4458.
Path 189 | total_timesteps 4491.
Path 190 | total_timesteps 4513.
Path 191 | total_timesteps 4522.
Path 192 | total_timesteps 4543.
Path 193 | total_timesteps 4558.
Path 194 | total_timesteps 4584.
Path 195 | total_timesteps 4606.
Path 196 | total_timesteps 4625.
Path 197 | total_timesteps 4636.
Path 198 | total_timesteps 4649.
Path 199 | total_timesteps 4703.
Path 200 | total_timesteps 4718.
Path 201 | total_timesteps 4728.
Path 202 | total_timesteps 4748.
Path 203 | total_timesteps 4769.
Path 204 | total_timesteps 4794.
Path 205 | total_timesteps 4818.
Path 206 | total_timesteps 4843.
Path 207 | total_timesteps 4860.
Path 208 | total_timesteps 4887.
Path 209 | total_timesteps 4906.
Path 210 | total_timesteps 4929.
Path 211 | total_timesteps 4941.
Path 212 | total_timesteps 4967.
Path 213 | total_timesteps 4992.
Path 214 | total_timesteps 5009.
Path 215 | total_timesteps 5038.
Path 216 | total_timesteps 5054.
Path 217 | total_timesteps 5074.
Path 218 | total_timesteps 5084.
Path 219 | total_timesteps 5104.
Path 220 | total_timesteps 5114.
Path 221 | total_timesteps 5141.
Path 222 | total_timesteps 5169.
Path 223 | total_timesteps 5197.
Path 224 | total_timesteps 5236.
Path 225 | total_timesteps 5258.
Path 226 | total_timesteps 5291.
Path 227 | total_timesteps 5309.
Path 228 | total_timesteps 5362.
Path 229 | total_timesteps 5381.
Path 230 | total_timesteps 5400.
Path 231 | total_timesteps 5430.
Path 232 | total_timesteps 5471.
Path 233 | total_timesteps 5490.
Path 234 | total_timesteps 5521.
Path 235 | total_timesteps 5534.
Path 236 | total_timesteps 5566.
Path 237 | total_timesteps 5588.
Path 238 | total_timesteps 5611.
Path 239 | total_timesteps 5656.
Path 240 | total_timesteps 5680.
Path 241 | total_timesteps 5690.
Path 242 | total_timesteps 5705.
Path 243 | total_timesteps 5715.
Path 244 | total_timesteps 5776.
Path 245 | total_timesteps 5805.
Path 246 | total_timesteps 5815.
Path 247 | total_timesteps 5834.
Path 248 | total_timesteps 5854.
Path 249 | total_timesteps 5883.
Path 250 | total_timesteps 5909.
Path 251 | total_timesteps 5952.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.38    |
| Iteration     | 17       |
| MaximumReturn | 21.5     |
| MinimumReturn | -26.4    |
| TotalSamples  | 76138    |
----------------------------
itr #18 | 
Fitting dynamics.
Validation loss = 0.007436701562255621
Validation loss = 0.007868025451898575
Validation loss = 0.007499577011913061
Validation loss = 0.007663864176720381
Validation loss = 0.0069931307807564735
Validation loss = 0.007102330215275288
Validation loss = 0.007373627740889788
Validation loss = 0.007527361623942852
Validation loss = 0.007941290736198425
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 32.
Path 2 | total_timesteps 41.
Path 3 | total_timesteps 59.
Path 4 | total_timesteps 81.
Path 5 | total_timesteps 111.
Path 6 | total_timesteps 134.
Path 7 | total_timesteps 150.
Path 8 | total_timesteps 168.
Path 9 | total_timesteps 215.
Path 10 | total_timesteps 266.
Path 11 | total_timesteps 295.
Path 12 | total_timesteps 318.
Path 13 | total_timesteps 342.
Path 14 | total_timesteps 408.
Path 15 | total_timesteps 437.
Path 16 | total_timesteps 455.
Path 17 | total_timesteps 478.
Path 18 | total_timesteps 495.
Path 19 | total_timesteps 523.
Path 20 | total_timesteps 543.
Path 21 | total_timesteps 603.
Path 22 | total_timesteps 615.
Path 23 | total_timesteps 632.
Path 24 | total_timesteps 655.
Path 25 | total_timesteps 684.
Path 26 | total_timesteps 707.
Path 27 | total_timesteps 722.
Path 28 | total_timesteps 732.
Path 29 | total_timesteps 806.
Path 30 | total_timesteps 835.
Path 31 | total_timesteps 845.
Path 32 | total_timesteps 871.
Path 33 | total_timesteps 891.
Path 34 | total_timesteps 919.
Path 35 | total_timesteps 968.
Path 36 | total_timesteps 988.
Path 37 | total_timesteps 1024.
Path 38 | total_timesteps 1046.
Path 39 | total_timesteps 1076.
Path 40 | total_timesteps 1098.
Path 41 | total_timesteps 1118.
Path 42 | total_timesteps 1167.
Path 43 | total_timesteps 1191.
Path 44 | total_timesteps 1219.
Path 45 | total_timesteps 1239.
Path 46 | total_timesteps 1296.
Path 47 | total_timesteps 1330.
Path 48 | total_timesteps 1377.
Path 49 | total_timesteps 1397.
Path 50 | total_timesteps 1424.
Path 51 | total_timesteps 1451.
Path 52 | total_timesteps 1466.
Path 53 | total_timesteps 1494.
Path 54 | total_timesteps 1510.
Path 55 | total_timesteps 1530.
Path 56 | total_timesteps 1547.
Path 57 | total_timesteps 1570.
Path 58 | total_timesteps 1600.
Path 59 | total_timesteps 1629.
Path 60 | total_timesteps 1654.
Path 61 | total_timesteps 1671.
Path 62 | total_timesteps 1682.
Path 63 | total_timesteps 1703.
Path 64 | total_timesteps 1716.
Path 65 | total_timesteps 1731.
Path 66 | total_timesteps 1746.
Path 67 | total_timesteps 1765.
Path 68 | total_timesteps 1788.
Path 69 | total_timesteps 1806.
Path 70 | total_timesteps 1826.
Path 71 | total_timesteps 1845.
Path 72 | total_timesteps 1862.
Path 73 | total_timesteps 1904.
Path 74 | total_timesteps 1931.
Path 75 | total_timesteps 1959.
Path 76 | total_timesteps 1972.
Path 77 | total_timesteps 2003.
Path 78 | total_timesteps 2030.
Path 79 | total_timesteps 2044.
Path 80 | total_timesteps 2057.
Path 81 | total_timesteps 2084.
Path 82 | total_timesteps 2099.
Path 83 | total_timesteps 2110.
Path 84 | total_timesteps 2130.
Path 85 | total_timesteps 2154.
Path 86 | total_timesteps 2176.
Path 87 | total_timesteps 2206.
Path 88 | total_timesteps 2237.
Path 89 | total_timesteps 2256.
Path 90 | total_timesteps 2278.
Path 91 | total_timesteps 2312.
Path 92 | total_timesteps 2340.
Path 93 | total_timesteps 2359.
Path 94 | total_timesteps 2381.
Path 95 | total_timesteps 2400.
Path 96 | total_timesteps 2426.
Path 97 | total_timesteps 2470.
Path 98 | total_timesteps 2484.
Path 99 | total_timesteps 2517.
Path 100 | total_timesteps 2537.
Path 101 | total_timesteps 2557.
Path 102 | total_timesteps 2583.
Path 103 | total_timesteps 2603.
Path 104 | total_timesteps 2627.
Path 105 | total_timesteps 2655.
Path 106 | total_timesteps 2672.
Path 107 | total_timesteps 2686.
Path 108 | total_timesteps 2701.
Path 109 | total_timesteps 2724.
Path 110 | total_timesteps 2751.
Path 111 | total_timesteps 2765.
Path 112 | total_timesteps 2791.
Path 113 | total_timesteps 2826.
Path 114 | total_timesteps 2846.
Path 115 | total_timesteps 2862.
Path 116 | total_timesteps 2889.
Path 117 | total_timesteps 2918.
Path 118 | total_timesteps 2932.
Path 119 | total_timesteps 2956.
Path 120 | total_timesteps 2988.
Path 121 | total_timesteps 3008.
Path 122 | total_timesteps 3030.
Path 123 | total_timesteps 3061.
Path 124 | total_timesteps 3069.
Path 125 | total_timesteps 3092.
Path 126 | total_timesteps 3103.
Path 127 | total_timesteps 3128.
Path 128 | total_timesteps 3166.
Path 129 | total_timesteps 3199.
Path 130 | total_timesteps 3226.
Path 131 | total_timesteps 3268.
Path 132 | total_timesteps 3289.
Path 133 | total_timesteps 3310.
Path 134 | total_timesteps 3329.
Path 135 | total_timesteps 3353.
Path 136 | total_timesteps 3385.
Path 137 | total_timesteps 3407.
Path 138 | total_timesteps 3463.
Path 139 | total_timesteps 3483.
Path 140 | total_timesteps 3510.
Path 141 | total_timesteps 3527.
Path 142 | total_timesteps 3549.
Path 143 | total_timesteps 3561.
Path 144 | total_timesteps 3586.
Path 145 | total_timesteps 3609.
Path 146 | total_timesteps 3648.
Path 147 | total_timesteps 3669.
Path 148 | total_timesteps 3698.
Path 149 | total_timesteps 3718.
Path 150 | total_timesteps 3739.
Path 151 | total_timesteps 3779.
Path 152 | total_timesteps 3806.
Path 153 | total_timesteps 3849.
Path 154 | total_timesteps 3873.
Path 155 | total_timesteps 3889.
Path 156 | total_timesteps 3911.
Path 157 | total_timesteps 3939.
Path 158 | total_timesteps 3950.
Path 159 | total_timesteps 3971.
Path 160 | total_timesteps 3986.
Path 161 | total_timesteps 4016.
Path 162 | total_timesteps 4045.
Path 163 | total_timesteps 4055.
Path 164 | total_timesteps 4078.
Path 165 | total_timesteps 4090.
Path 166 | total_timesteps 4111.
Path 167 | total_timesteps 4154.
Path 168 | total_timesteps 4176.
Path 169 | total_timesteps 4190.
Path 170 | total_timesteps 4216.
Path 171 | total_timesteps 4233.
Path 172 | total_timesteps 4251.
Path 173 | total_timesteps 4275.
Path 174 | total_timesteps 4305.
Path 175 | total_timesteps 4320.
Path 176 | total_timesteps 4341.
Path 177 | total_timesteps 4375.
Path 178 | total_timesteps 4404.
Path 179 | total_timesteps 4413.
Path 180 | total_timesteps 4432.
Path 181 | total_timesteps 4452.
Path 182 | total_timesteps 4472.
Path 183 | total_timesteps 4493.
Path 184 | total_timesteps 4522.
Path 185 | total_timesteps 4544.
Path 186 | total_timesteps 4562.
Path 187 | total_timesteps 4587.
Path 188 | total_timesteps 4602.
Path 189 | total_timesteps 4627.
Path 190 | total_timesteps 4663.
Path 191 | total_timesteps 4692.
Path 192 | total_timesteps 4721.
Path 193 | total_timesteps 4774.
Path 194 | total_timesteps 4824.
Path 195 | total_timesteps 4846.
Path 196 | total_timesteps 4868.
Path 197 | total_timesteps 4887.
Path 198 | total_timesteps 4905.
Path 199 | total_timesteps 4925.
Path 200 | total_timesteps 4951.
Path 201 | total_timesteps 4974.
Path 202 | total_timesteps 4985.
Path 203 | total_timesteps 5010.
Path 204 | total_timesteps 5046.
Path 205 | total_timesteps 5062.
Path 206 | total_timesteps 5121.
Path 207 | total_timesteps 5150.
Path 208 | total_timesteps 5164.
Path 209 | total_timesteps 5186.
Path 210 | total_timesteps 5213.
Path 211 | total_timesteps 5235.
Path 212 | total_timesteps 5291.
Path 213 | total_timesteps 5303.
Path 214 | total_timesteps 5330.
Path 215 | total_timesteps 5357.
Path 216 | total_timesteps 5380.
Path 217 | total_timesteps 5409.
Path 218 | total_timesteps 5417.
Path 219 | total_timesteps 5435.
Path 220 | total_timesteps 5467.
Path 221 | total_timesteps 5487.
Path 222 | total_timesteps 5532.
Path 223 | total_timesteps 5572.
Path 224 | total_timesteps 5603.
Path 225 | total_timesteps 5626.
Path 226 | total_timesteps 5638.
Path 227 | total_timesteps 5655.
Path 228 | total_timesteps 5682.
Path 229 | total_timesteps 5696.
Path 230 | total_timesteps 5731.
Path 231 | total_timesteps 5754.
Path 232 | total_timesteps 5775.
Path 233 | total_timesteps 5805.
Path 234 | total_timesteps 5828.
Path 235 | total_timesteps 5847.
Path 236 | total_timesteps 5868.
Path 237 | total_timesteps 5882.
Path 238 | total_timesteps 5916.
Path 239 | total_timesteps 5938.
Path 240 | total_timesteps 5960.
Path 241 | total_timesteps 5989.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.02    |
| Iteration     | 18       |
| MaximumReturn | 38.3     |
| MinimumReturn | -25.5    |
| TotalSamples  | 80142    |
----------------------------
itr #19 | 
Fitting dynamics.
Validation loss = 0.0075680045410990715
Validation loss = 0.008069473318755627
Validation loss = 0.007897001691162586
Validation loss = 0.007885238155722618
Validation loss = 0.007080456707626581
Validation loss = 0.006959105841815472
Validation loss = 0.007027234882116318
Validation loss = 0.007261280901730061
Validation loss = 0.007099858485162258
Validation loss = 0.006949490401893854
Validation loss = 0.006841915659606457
Validation loss = 0.008370034396648407
Validation loss = 0.0074086980894207954
Validation loss = 0.007287120912224054
Validation loss = 0.007356621325016022
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 19.
Path 2 | total_timesteps 36.
Path 3 | total_timesteps 49.
Path 4 | total_timesteps 60.
Path 5 | total_timesteps 69.
Path 6 | total_timesteps 89.
Path 7 | total_timesteps 111.
Path 8 | total_timesteps 135.
Path 9 | total_timesteps 155.
Path 10 | total_timesteps 166.
Path 11 | total_timesteps 180.
Path 12 | total_timesteps 195.
Path 13 | total_timesteps 218.
Path 14 | total_timesteps 226.
Path 15 | total_timesteps 239.
Path 16 | total_timesteps 261.
Path 17 | total_timesteps 283.
Path 18 | total_timesteps 328.
Path 19 | total_timesteps 347.
Path 20 | total_timesteps 366.
Path 21 | total_timesteps 395.
Path 22 | total_timesteps 407.
Path 23 | total_timesteps 414.
Path 24 | total_timesteps 432.
Path 25 | total_timesteps 449.
Path 26 | total_timesteps 468.
Path 27 | total_timesteps 487.
Path 28 | total_timesteps 509.
Path 29 | total_timesteps 538.
Path 30 | total_timesteps 555.
Path 31 | total_timesteps 583.
Path 32 | total_timesteps 612.
Path 33 | total_timesteps 628.
Path 34 | total_timesteps 650.
Path 35 | total_timesteps 667.
Path 36 | total_timesteps 678.
Path 37 | total_timesteps 694.
Path 38 | total_timesteps 713.
Path 39 | total_timesteps 743.
Path 40 | total_timesteps 767.
Path 41 | total_timesteps 785.
Path 42 | total_timesteps 801.
Path 43 | total_timesteps 823.
Path 44 | total_timesteps 831.
Path 45 | total_timesteps 847.
Path 46 | total_timesteps 859.
Path 47 | total_timesteps 873.
Path 48 | total_timesteps 891.
Path 49 | total_timesteps 904.
Path 50 | total_timesteps 924.
Path 51 | total_timesteps 937.
Path 52 | total_timesteps 952.
Path 53 | total_timesteps 969.
Path 54 | total_timesteps 983.
Path 55 | total_timesteps 1027.
Path 56 | total_timesteps 1039.
Path 57 | total_timesteps 1062.
Path 58 | total_timesteps 1107.
Path 59 | total_timesteps 1121.
Path 60 | total_timesteps 1142.
Path 61 | total_timesteps 1156.
Path 62 | total_timesteps 1170.
Path 63 | total_timesteps 1183.
Path 64 | total_timesteps 1197.
Path 65 | total_timesteps 1214.
Path 66 | total_timesteps 1227.
Path 67 | total_timesteps 1241.
Path 68 | total_timesteps 1261.
Path 69 | total_timesteps 1280.
Path 70 | total_timesteps 1291.
Path 71 | total_timesteps 1320.
Path 72 | total_timesteps 1334.
Path 73 | total_timesteps 1349.
Path 74 | total_timesteps 1370.
Path 75 | total_timesteps 1390.
Path 76 | total_timesteps 1399.
Path 77 | total_timesteps 1412.
Path 78 | total_timesteps 1430.
Path 79 | total_timesteps 1440.
Path 80 | total_timesteps 1455.
Path 81 | total_timesteps 1474.
Path 82 | total_timesteps 1490.
Path 83 | total_timesteps 1504.
Path 84 | total_timesteps 1522.
Path 85 | total_timesteps 1534.
Path 86 | total_timesteps 1547.
Path 87 | total_timesteps 1561.
Path 88 | total_timesteps 1574.
Path 89 | total_timesteps 1597.
Path 90 | total_timesteps 1628.
Path 91 | total_timesteps 1648.
Path 92 | total_timesteps 1657.
Path 93 | total_timesteps 1671.
Path 94 | total_timesteps 1688.
Path 95 | total_timesteps 1718.
Path 96 | total_timesteps 1731.
Path 97 | total_timesteps 1742.
Path 98 | total_timesteps 1758.
Path 99 | total_timesteps 1770.
Path 100 | total_timesteps 1791.
Path 101 | total_timesteps 1805.
Path 102 | total_timesteps 1826.
Path 103 | total_timesteps 1837.
Path 104 | total_timesteps 1858.
Path 105 | total_timesteps 1872.
Path 106 | total_timesteps 1886.
Path 107 | total_timesteps 1911.
Path 108 | total_timesteps 1942.
Path 109 | total_timesteps 1951.
Path 110 | total_timesteps 1964.
Path 111 | total_timesteps 1984.
Path 112 | total_timesteps 2004.
Path 113 | total_timesteps 2015.
Path 114 | total_timesteps 2026.
Path 115 | total_timesteps 2042.
Path 116 | total_timesteps 2061.
Path 117 | total_timesteps 2085.
Path 118 | total_timesteps 2095.
Path 119 | total_timesteps 2120.
Path 120 | total_timesteps 2150.
Path 121 | total_timesteps 2170.
Path 122 | total_timesteps 2187.
Path 123 | total_timesteps 2198.
Path 124 | total_timesteps 2212.
Path 125 | total_timesteps 2248.
Path 126 | total_timesteps 2269.
Path 127 | total_timesteps 2280.
Path 128 | total_timesteps 2296.
Path 129 | total_timesteps 2334.
Path 130 | total_timesteps 2348.
Path 131 | total_timesteps 2361.
Path 132 | total_timesteps 2388.
Path 133 | total_timesteps 2408.
Path 134 | total_timesteps 2427.
Path 135 | total_timesteps 2467.
Path 136 | total_timesteps 2488.
Path 137 | total_timesteps 2507.
Path 138 | total_timesteps 2530.
Path 139 | total_timesteps 2540.
Path 140 | total_timesteps 2561.
Path 141 | total_timesteps 2575.
Path 142 | total_timesteps 2593.
Path 143 | total_timesteps 2607.
Path 144 | total_timesteps 2625.
Path 145 | total_timesteps 2633.
Path 146 | total_timesteps 2641.
Path 147 | total_timesteps 2653.
Path 148 | total_timesteps 2672.
Path 149 | total_timesteps 2685.
Path 150 | total_timesteps 2696.
Path 151 | total_timesteps 2714.
Path 152 | total_timesteps 2725.
Path 153 | total_timesteps 2734.
Path 154 | total_timesteps 2754.
Path 155 | total_timesteps 2772.
Path 156 | total_timesteps 2786.
Path 157 | total_timesteps 2804.
Path 158 | total_timesteps 2833.
Path 159 | total_timesteps 2849.
Path 160 | total_timesteps 2874.
Path 161 | total_timesteps 2894.
Path 162 | total_timesteps 2912.
Path 163 | total_timesteps 2932.
Path 164 | total_timesteps 2961.
Path 165 | total_timesteps 2977.
Path 166 | total_timesteps 2991.
Path 167 | total_timesteps 3009.
Path 168 | total_timesteps 3025.
Path 169 | total_timesteps 3050.
Path 170 | total_timesteps 3063.
Path 171 | total_timesteps 3095.
Path 172 | total_timesteps 3107.
Path 173 | total_timesteps 3120.
Path 174 | total_timesteps 3128.
Path 175 | total_timesteps 3141.
Path 176 | total_timesteps 3149.
Path 177 | total_timesteps 3174.
Path 178 | total_timesteps 3206.
Path 179 | total_timesteps 3232.
Path 180 | total_timesteps 3247.
Path 181 | total_timesteps 3267.
Path 182 | total_timesteps 3286.
Path 183 | total_timesteps 3295.
Path 184 | total_timesteps 3320.
Path 185 | total_timesteps 3330.
Path 186 | total_timesteps 3361.
Path 187 | total_timesteps 3376.
Path 188 | total_timesteps 3395.
Path 189 | total_timesteps 3419.
Path 190 | total_timesteps 3432.
Path 191 | total_timesteps 3453.
Path 192 | total_timesteps 3472.
Path 193 | total_timesteps 3487.
Path 194 | total_timesteps 3498.
Path 195 | total_timesteps 3508.
Path 196 | total_timesteps 3532.
Path 197 | total_timesteps 3555.
Path 198 | total_timesteps 3573.
Path 199 | total_timesteps 3589.
Path 200 | total_timesteps 3616.
Path 201 | total_timesteps 3625.
Path 202 | total_timesteps 3636.
Path 203 | total_timesteps 3653.
Path 204 | total_timesteps 3673.
Path 205 | total_timesteps 3684.
Path 206 | total_timesteps 3703.
Path 207 | total_timesteps 3723.
Path 208 | total_timesteps 3764.
Path 209 | total_timesteps 3783.
Path 210 | total_timesteps 3792.
Path 211 | total_timesteps 3810.
Path 212 | total_timesteps 3833.
Path 213 | total_timesteps 3850.
Path 214 | total_timesteps 3875.
Path 215 | total_timesteps 3890.
Path 216 | total_timesteps 3916.
Path 217 | total_timesteps 3930.
Path 218 | total_timesteps 3944.
Path 219 | total_timesteps 3970.
Path 220 | total_timesteps 3997.
Path 221 | total_timesteps 4012.
Path 222 | total_timesteps 4027.
Path 223 | total_timesteps 4045.
Path 224 | total_timesteps 4076.
Path 225 | total_timesteps 4095.
Path 226 | total_timesteps 4105.
Path 227 | total_timesteps 4120.
Path 228 | total_timesteps 4136.
Path 229 | total_timesteps 4158.
Path 230 | total_timesteps 4186.
Path 231 | total_timesteps 4201.
Path 232 | total_timesteps 4215.
Path 233 | total_timesteps 4231.
Path 234 | total_timesteps 4250.
Path 235 | total_timesteps 4265.
Path 236 | total_timesteps 4284.
Path 237 | total_timesteps 4294.
Path 238 | total_timesteps 4307.
Path 239 | total_timesteps 4338.
Path 240 | total_timesteps 4347.
Path 241 | total_timesteps 4362.
Path 242 | total_timesteps 4375.
Path 243 | total_timesteps 4388.
Path 244 | total_timesteps 4400.
Path 245 | total_timesteps 4420.
Path 246 | total_timesteps 4431.
Path 247 | total_timesteps 4446.
Path 248 | total_timesteps 4464.
Path 249 | total_timesteps 4476.
Path 250 | total_timesteps 4490.
Path 251 | total_timesteps 4497.
Path 252 | total_timesteps 4507.
Path 253 | total_timesteps 4516.
Path 254 | total_timesteps 4537.
Path 255 | total_timesteps 4564.
Path 256 | total_timesteps 4583.
Path 257 | total_timesteps 4591.
Path 258 | total_timesteps 4601.
Path 259 | total_timesteps 4623.
Path 260 | total_timesteps 4636.
Path 261 | total_timesteps 4645.
Path 262 | total_timesteps 4661.
Path 263 | total_timesteps 4670.
Path 264 | total_timesteps 4698.
Path 265 | total_timesteps 4709.
Path 266 | total_timesteps 4721.
Path 267 | total_timesteps 4739.
Path 268 | total_timesteps 4757.
Path 269 | total_timesteps 4781.
Path 270 | total_timesteps 4794.
Path 271 | total_timesteps 4812.
Path 272 | total_timesteps 4827.
Path 273 | total_timesteps 4850.
Path 274 | total_timesteps 4864.
Path 275 | total_timesteps 4873.
Path 276 | total_timesteps 4892.
Path 277 | total_timesteps 4909.
Path 278 | total_timesteps 4928.
Path 279 | total_timesteps 4941.
Path 280 | total_timesteps 4955.
Path 281 | total_timesteps 4979.
Path 282 | total_timesteps 4990.
Path 283 | total_timesteps 5010.
Path 284 | total_timesteps 5022.
Path 285 | total_timesteps 5034.
Path 286 | total_timesteps 5079.
Path 287 | total_timesteps 5094.
Path 288 | total_timesteps 5114.
Path 289 | total_timesteps 5130.
Path 290 | total_timesteps 5142.
Path 291 | total_timesteps 5156.
Path 292 | total_timesteps 5199.
Path 293 | total_timesteps 5217.
Path 294 | total_timesteps 5232.
Path 295 | total_timesteps 5253.
Path 296 | total_timesteps 5266.
Path 297 | total_timesteps 5284.
Path 298 | total_timesteps 5294.
Path 299 | total_timesteps 5304.
Path 300 | total_timesteps 5326.
Path 301 | total_timesteps 5349.
Path 302 | total_timesteps 5362.
Path 303 | total_timesteps 5375.
Path 304 | total_timesteps 5385.
Path 305 | total_timesteps 5407.
Path 306 | total_timesteps 5417.
Path 307 | total_timesteps 5438.
Path 308 | total_timesteps 5451.
Path 309 | total_timesteps 5473.
Path 310 | total_timesteps 5490.
Path 311 | total_timesteps 5498.
Path 312 | total_timesteps 5521.
Path 313 | total_timesteps 5538.
Path 314 | total_timesteps 5558.
Path 315 | total_timesteps 5569.
Path 316 | total_timesteps 5577.
Path 317 | total_timesteps 5586.
Path 318 | total_timesteps 5614.
Path 319 | total_timesteps 5629.
Path 320 | total_timesteps 5642.
Path 321 | total_timesteps 5660.
Path 322 | total_timesteps 5688.
Path 323 | total_timesteps 5708.
Path 324 | total_timesteps 5736.
Path 325 | total_timesteps 5762.
Path 326 | total_timesteps 5772.
Path 327 | total_timesteps 5787.
Path 328 | total_timesteps 5799.
Path 329 | total_timesteps 5808.
Path 330 | total_timesteps 5820.
Path 331 | total_timesteps 5841.
Path 332 | total_timesteps 5856.
Path 333 | total_timesteps 5874.
Path 334 | total_timesteps 5892.
Path 335 | total_timesteps 5902.
Path 336 | total_timesteps 5911.
Path 337 | total_timesteps 5921.
Path 338 | total_timesteps 5939.
Path 339 | total_timesteps 5960.
Path 340 | total_timesteps 5979.
Path 341 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.25    |
| Iteration     | 19       |
| MaximumReturn | 5.61     |
| MinimumReturn | -20.2    |
| TotalSamples  | 84146    |
----------------------------
itr #20 | 
Fitting dynamics.
Validation loss = 0.007316073402762413
Validation loss = 0.007033564615994692
Validation loss = 0.006598432082682848
Validation loss = 0.006695359013974667
Validation loss = 0.006676115561276674
Validation loss = 0.006979244761168957
Validation loss = 0.006664442829787731
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 28.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 57.
Path 5 | total_timesteps 73.
Path 6 | total_timesteps 86.
Path 7 | total_timesteps 102.
Path 8 | total_timesteps 117.
Path 9 | total_timesteps 129.
Path 10 | total_timesteps 138.
Path 11 | total_timesteps 153.
Path 12 | total_timesteps 166.
Path 13 | total_timesteps 174.
Path 14 | total_timesteps 188.
Path 15 | total_timesteps 209.
Path 16 | total_timesteps 224.
Path 17 | total_timesteps 241.
Path 18 | total_timesteps 255.
Path 19 | total_timesteps 263.
Path 20 | total_timesteps 273.
Path 21 | total_timesteps 286.
Path 22 | total_timesteps 296.
Path 23 | total_timesteps 317.
Path 24 | total_timesteps 327.
Path 25 | total_timesteps 347.
Path 26 | total_timesteps 378.
Path 27 | total_timesteps 386.
Path 28 | total_timesteps 401.
Path 29 | total_timesteps 410.
Path 30 | total_timesteps 420.
Path 31 | total_timesteps 441.
Path 32 | total_timesteps 452.
Path 33 | total_timesteps 469.
Path 34 | total_timesteps 490.
Path 35 | total_timesteps 499.
Path 36 | total_timesteps 508.
Path 37 | total_timesteps 520.
Path 38 | total_timesteps 528.
Path 39 | total_timesteps 539.
Path 40 | total_timesteps 557.
Path 41 | total_timesteps 564.
Path 42 | total_timesteps 581.
Path 43 | total_timesteps 601.
Path 44 | total_timesteps 615.
Path 45 | total_timesteps 624.
Path 46 | total_timesteps 634.
Path 47 | total_timesteps 649.
Path 48 | total_timesteps 676.
Path 49 | total_timesteps 689.
Path 50 | total_timesteps 711.
Path 51 | total_timesteps 723.
Path 52 | total_timesteps 730.
Path 53 | total_timesteps 746.
Path 54 | total_timesteps 762.
Path 55 | total_timesteps 773.
Path 56 | total_timesteps 788.
Path 57 | total_timesteps 805.
Path 58 | total_timesteps 832.
Path 59 | total_timesteps 847.
Path 60 | total_timesteps 884.
Path 61 | total_timesteps 897.
Path 62 | total_timesteps 913.
Path 63 | total_timesteps 928.
Path 64 | total_timesteps 940.
Path 65 | total_timesteps 966.
Path 66 | total_timesteps 978.
Path 67 | total_timesteps 994.
Path 68 | total_timesteps 1005.
Path 69 | total_timesteps 1014.
Path 70 | total_timesteps 1021.
Path 71 | total_timesteps 1036.
Path 72 | total_timesteps 1048.
Path 73 | total_timesteps 1068.
Path 74 | total_timesteps 1078.
Path 75 | total_timesteps 1086.
Path 76 | total_timesteps 1107.
Path 77 | total_timesteps 1118.
Path 78 | total_timesteps 1142.
Path 79 | total_timesteps 1160.
Path 80 | total_timesteps 1172.
Path 81 | total_timesteps 1184.
Path 82 | total_timesteps 1199.
Path 83 | total_timesteps 1209.
Path 84 | total_timesteps 1218.
Path 85 | total_timesteps 1225.
Path 86 | total_timesteps 1239.
Path 87 | total_timesteps 1251.
Path 88 | total_timesteps 1264.
Path 89 | total_timesteps 1274.
Path 90 | total_timesteps 1311.
Path 91 | total_timesteps 1317.
Path 92 | total_timesteps 1333.
Path 93 | total_timesteps 1344.
Path 94 | total_timesteps 1353.
Path 95 | total_timesteps 1367.
Path 96 | total_timesteps 1391.
Path 97 | total_timesteps 1403.
Path 98 | total_timesteps 1416.
Path 99 | total_timesteps 1435.
Path 100 | total_timesteps 1446.
Path 101 | total_timesteps 1458.
Path 102 | total_timesteps 1472.
Path 103 | total_timesteps 1482.
Path 104 | total_timesteps 1508.
Path 105 | total_timesteps 1525.
Path 106 | total_timesteps 1544.
Path 107 | total_timesteps 1567.
Path 108 | total_timesteps 1577.
Path 109 | total_timesteps 1592.
Path 110 | total_timesteps 1613.
Path 111 | total_timesteps 1622.
Path 112 | total_timesteps 1633.
Path 113 | total_timesteps 1649.
Path 114 | total_timesteps 1668.
Path 115 | total_timesteps 1682.
Path 116 | total_timesteps 1697.
Path 117 | total_timesteps 1711.
Path 118 | total_timesteps 1720.
Path 119 | total_timesteps 1740.
Path 120 | total_timesteps 1760.
Path 121 | total_timesteps 1769.
Path 122 | total_timesteps 1781.
Path 123 | total_timesteps 1799.
Path 124 | total_timesteps 1814.
Path 125 | total_timesteps 1824.
Path 126 | total_timesteps 1833.
Path 127 | total_timesteps 1856.
Path 128 | total_timesteps 1870.
Path 129 | total_timesteps 1885.
Path 130 | total_timesteps 1894.
Path 131 | total_timesteps 1912.
Path 132 | total_timesteps 1927.
Path 133 | total_timesteps 1945.
Path 134 | total_timesteps 1959.
Path 135 | total_timesteps 1969.
Path 136 | total_timesteps 1981.
Path 137 | total_timesteps 2012.
Path 138 | total_timesteps 2023.
Path 139 | total_timesteps 2035.
Path 140 | total_timesteps 2052.
Path 141 | total_timesteps 2070.
Path 142 | total_timesteps 2080.
Path 143 | total_timesteps 2098.
Path 144 | total_timesteps 2114.
Path 145 | total_timesteps 2139.
Path 146 | total_timesteps 2154.
Path 147 | total_timesteps 2172.
Path 148 | total_timesteps 2186.
Path 149 | total_timesteps 2194.
Path 150 | total_timesteps 2219.
Path 151 | total_timesteps 2229.
Path 152 | total_timesteps 2245.
Path 153 | total_timesteps 2260.
Path 154 | total_timesteps 2276.
Path 155 | total_timesteps 2288.
Path 156 | total_timesteps 2311.
Path 157 | total_timesteps 2322.
Path 158 | total_timesteps 2331.
Path 159 | total_timesteps 2340.
Path 160 | total_timesteps 2353.
Path 161 | total_timesteps 2364.
Path 162 | total_timesteps 2381.
Path 163 | total_timesteps 2396.
Path 164 | total_timesteps 2406.
Path 165 | total_timesteps 2426.
Path 166 | total_timesteps 2451.
Path 167 | total_timesteps 2464.
Path 168 | total_timesteps 2475.
Path 169 | total_timesteps 2497.
Path 170 | total_timesteps 2530.
Path 171 | total_timesteps 2542.
Path 172 | total_timesteps 2552.
Path 173 | total_timesteps 2564.
Path 174 | total_timesteps 2573.
Path 175 | total_timesteps 2588.
Path 176 | total_timesteps 2615.
Path 177 | total_timesteps 2633.
Path 178 | total_timesteps 2648.
Path 179 | total_timesteps 2667.
Path 180 | total_timesteps 2675.
Path 181 | total_timesteps 2688.
Path 182 | total_timesteps 2701.
Path 183 | total_timesteps 2721.
Path 184 | total_timesteps 2731.
Path 185 | total_timesteps 2745.
Path 186 | total_timesteps 2759.
Path 187 | total_timesteps 2769.
Path 188 | total_timesteps 2778.
Path 189 | total_timesteps 2793.
Path 190 | total_timesteps 2805.
Path 191 | total_timesteps 2818.
Path 192 | total_timesteps 2825.
Path 193 | total_timesteps 2838.
Path 194 | total_timesteps 2847.
Path 195 | total_timesteps 2859.
Path 196 | total_timesteps 2879.
Path 197 | total_timesteps 2894.
Path 198 | total_timesteps 2911.
Path 199 | total_timesteps 2920.
Path 200 | total_timesteps 2932.
Path 201 | total_timesteps 2941.
Path 202 | total_timesteps 2952.
Path 203 | total_timesteps 2969.
Path 204 | total_timesteps 2991.
Path 205 | total_timesteps 3004.
Path 206 | total_timesteps 3014.
Path 207 | total_timesteps 3027.
Path 208 | total_timesteps 3040.
Path 209 | total_timesteps 3053.
Path 210 | total_timesteps 3069.
Path 211 | total_timesteps 3079.
Path 212 | total_timesteps 3095.
Path 213 | total_timesteps 3104.
Path 214 | total_timesteps 3121.
Path 215 | total_timesteps 3131.
Path 216 | total_timesteps 3145.
Path 217 | total_timesteps 3152.
Path 218 | total_timesteps 3160.
Path 219 | total_timesteps 3180.
Path 220 | total_timesteps 3196.
Path 221 | total_timesteps 3206.
Path 222 | total_timesteps 3230.
Path 223 | total_timesteps 3245.
Path 224 | total_timesteps 3257.
Path 225 | total_timesteps 3277.
Path 226 | total_timesteps 3286.
Path 227 | total_timesteps 3299.
Path 228 | total_timesteps 3312.
Path 229 | total_timesteps 3330.
Path 230 | total_timesteps 3344.
Path 231 | total_timesteps 3368.
Path 232 | total_timesteps 3379.
Path 233 | total_timesteps 3388.
Path 234 | total_timesteps 3397.
Path 235 | total_timesteps 3409.
Path 236 | total_timesteps 3416.
Path 237 | total_timesteps 3429.
Path 238 | total_timesteps 3435.
Path 239 | total_timesteps 3451.
Path 240 | total_timesteps 3462.
Path 241 | total_timesteps 3474.
Path 242 | total_timesteps 3482.
Path 243 | total_timesteps 3497.
Path 244 | total_timesteps 3508.
Path 245 | total_timesteps 3535.
Path 246 | total_timesteps 3547.
Path 247 | total_timesteps 3558.
Path 248 | total_timesteps 3575.
Path 249 | total_timesteps 3588.
Path 250 | total_timesteps 3596.
Path 251 | total_timesteps 3607.
Path 252 | total_timesteps 3622.
Path 253 | total_timesteps 3633.
Path 254 | total_timesteps 3645.
Path 255 | total_timesteps 3658.
Path 256 | total_timesteps 3674.
Path 257 | total_timesteps 3686.
Path 258 | total_timesteps 3697.
Path 259 | total_timesteps 3709.
Path 260 | total_timesteps 3723.
Path 261 | total_timesteps 3732.
Path 262 | total_timesteps 3743.
Path 263 | total_timesteps 3753.
Path 264 | total_timesteps 3766.
Path 265 | total_timesteps 3782.
Path 266 | total_timesteps 3793.
Path 267 | total_timesteps 3807.
Path 268 | total_timesteps 3823.
Path 269 | total_timesteps 3839.
Path 270 | total_timesteps 3849.
Path 271 | total_timesteps 3858.
Path 272 | total_timesteps 3871.
Path 273 | total_timesteps 3884.
Path 274 | total_timesteps 3892.
Path 275 | total_timesteps 3904.
Path 276 | total_timesteps 3921.
Path 277 | total_timesteps 3934.
Path 278 | total_timesteps 3942.
Path 279 | total_timesteps 3952.
Path 280 | total_timesteps 3967.
Path 281 | total_timesteps 3975.
Path 282 | total_timesteps 3987.
Path 283 | total_timesteps 3997.
Path 284 | total_timesteps 4004.
Path 285 | total_timesteps 4025.
Path 286 | total_timesteps 4033.
Path 287 | total_timesteps 4061.
Path 288 | total_timesteps 4080.
Path 289 | total_timesteps 4089.
Path 290 | total_timesteps 4099.
Path 291 | total_timesteps 4108.
Path 292 | total_timesteps 4126.
Path 293 | total_timesteps 4139.
Path 294 | total_timesteps 4154.
Path 295 | total_timesteps 4166.
Path 296 | total_timesteps 4181.
Path 297 | total_timesteps 4194.
Path 298 | total_timesteps 4207.
Path 299 | total_timesteps 4216.
Path 300 | total_timesteps 4226.
Path 301 | total_timesteps 4235.
Path 302 | total_timesteps 4244.
Path 303 | total_timesteps 4251.
Path 304 | total_timesteps 4264.
Path 305 | total_timesteps 4274.
Path 306 | total_timesteps 4285.
Path 307 | total_timesteps 4295.
Path 308 | total_timesteps 4308.
Path 309 | total_timesteps 4322.
Path 310 | total_timesteps 4340.
Path 311 | total_timesteps 4352.
Path 312 | total_timesteps 4374.
Path 313 | total_timesteps 4396.
Path 314 | total_timesteps 4417.
Path 315 | total_timesteps 4424.
Path 316 | total_timesteps 4432.
Path 317 | total_timesteps 4452.
Path 318 | total_timesteps 4466.
Path 319 | total_timesteps 4488.
Path 320 | total_timesteps 4504.
Path 321 | total_timesteps 4519.
Path 322 | total_timesteps 4530.
Path 323 | total_timesteps 4543.
Path 324 | total_timesteps 4555.
Path 325 | total_timesteps 4573.
Path 326 | total_timesteps 4582.
Path 327 | total_timesteps 4607.
Path 328 | total_timesteps 4630.
Path 329 | total_timesteps 4639.
Path 330 | total_timesteps 4647.
Path 331 | total_timesteps 4662.
Path 332 | total_timesteps 4681.
Path 333 | total_timesteps 4689.
Path 334 | total_timesteps 4701.
Path 335 | total_timesteps 4736.
Path 336 | total_timesteps 4752.
Path 337 | total_timesteps 4765.
Path 338 | total_timesteps 4773.
Path 339 | total_timesteps 4782.
Path 340 | total_timesteps 4790.
Path 341 | total_timesteps 4799.
Path 342 | total_timesteps 4817.
Path 343 | total_timesteps 4828.
Path 344 | total_timesteps 4841.
Path 345 | total_timesteps 4849.
Path 346 | total_timesteps 4860.
Path 347 | total_timesteps 4873.
Path 348 | total_timesteps 4888.
Path 349 | total_timesteps 4909.
Path 350 | total_timesteps 4919.
Path 351 | total_timesteps 4936.
Path 352 | total_timesteps 4949.
Path 353 | total_timesteps 4963.
Path 354 | total_timesteps 4982.
Path 355 | total_timesteps 4991.
Path 356 | total_timesteps 5007.
Path 357 | total_timesteps 5019.
Path 358 | total_timesteps 5026.
Path 359 | total_timesteps 5040.
Path 360 | total_timesteps 5052.
Path 361 | total_timesteps 5060.
Path 362 | total_timesteps 5068.
Path 363 | total_timesteps 5087.
Path 364 | total_timesteps 5101.
Path 365 | total_timesteps 5121.
Path 366 | total_timesteps 5129.
Path 367 | total_timesteps 5137.
Path 368 | total_timesteps 5151.
Path 369 | total_timesteps 5167.
Path 370 | total_timesteps 5178.
Path 371 | total_timesteps 5189.
Path 372 | total_timesteps 5211.
Path 373 | total_timesteps 5227.
Path 374 | total_timesteps 5235.
Path 375 | total_timesteps 5249.
Path 376 | total_timesteps 5264.
Path 377 | total_timesteps 5273.
Path 378 | total_timesteps 5285.
Path 379 | total_timesteps 5298.
Path 380 | total_timesteps 5307.
Path 381 | total_timesteps 5325.
Path 382 | total_timesteps 5339.
Path 383 | total_timesteps 5355.
Path 384 | total_timesteps 5369.
Path 385 | total_timesteps 5376.
Path 386 | total_timesteps 5393.
Path 387 | total_timesteps 5406.
Path 388 | total_timesteps 5415.
Path 389 | total_timesteps 5423.
Path 390 | total_timesteps 5436.
Path 391 | total_timesteps 5445.
Path 392 | total_timesteps 5458.
Path 393 | total_timesteps 5472.
Path 394 | total_timesteps 5482.
Path 395 | total_timesteps 5498.
Path 396 | total_timesteps 5517.
Path 397 | total_timesteps 5526.
Path 398 | total_timesteps 5537.
Path 399 | total_timesteps 5551.
Path 400 | total_timesteps 5568.
Path 401 | total_timesteps 5583.
Path 402 | total_timesteps 5599.
Path 403 | total_timesteps 5608.
Path 404 | total_timesteps 5621.
Path 405 | total_timesteps 5637.
Path 406 | total_timesteps 5660.
Path 407 | total_timesteps 5689.
Path 408 | total_timesteps 5705.
Path 409 | total_timesteps 5717.
Path 410 | total_timesteps 5727.
Path 411 | total_timesteps 5743.
Path 412 | total_timesteps 5757.
Path 413 | total_timesteps 5770.
Path 414 | total_timesteps 5781.
Path 415 | total_timesteps 5799.
Path 416 | total_timesteps 5827.
Path 417 | total_timesteps 5841.
Path 418 | total_timesteps 5860.
Path 419 | total_timesteps 5896.
Path 420 | total_timesteps 5914.
Path 421 | total_timesteps 5929.
Path 422 | total_timesteps 5941.
Path 423 | total_timesteps 5949.
Path 424 | total_timesteps 5969.
Path 425 | total_timesteps 5984.
Path 426 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.29    |
| Iteration     | 20       |
| MaximumReturn | 5.2      |
| MinimumReturn | -19.5    |
| TotalSamples  | 88151    |
----------------------------
itr #21 | 
Fitting dynamics.
Validation loss = 0.006835354492068291
Validation loss = 0.00653883395716548
Validation loss = 0.006891107186675072
Validation loss = 0.006883286405354738
Validation loss = 0.006835097447037697
Validation loss = 0.006707276217639446
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 32.
Path 4 | total_timesteps 55.
Path 5 | total_timesteps 68.
Path 6 | total_timesteps 75.
Path 7 | total_timesteps 84.
Path 8 | total_timesteps 94.
Path 9 | total_timesteps 112.
Path 10 | total_timesteps 119.
Path 11 | total_timesteps 129.
Path 12 | total_timesteps 137.
Path 13 | total_timesteps 149.
Path 14 | total_timesteps 167.
Path 15 | total_timesteps 180.
Path 16 | total_timesteps 211.
Path 17 | total_timesteps 224.
Path 18 | total_timesteps 232.
Path 19 | total_timesteps 242.
Path 20 | total_timesteps 252.
Path 21 | total_timesteps 264.
Path 22 | total_timesteps 275.
Path 23 | total_timesteps 286.
Path 24 | total_timesteps 308.
Path 25 | total_timesteps 326.
Path 26 | total_timesteps 340.
Path 27 | total_timesteps 353.
Path 28 | total_timesteps 364.
Path 29 | total_timesteps 372.
Path 30 | total_timesteps 388.
Path 31 | total_timesteps 404.
Path 32 | total_timesteps 412.
Path 33 | total_timesteps 424.
Path 34 | total_timesteps 441.
Path 35 | total_timesteps 458.
Path 36 | total_timesteps 476.
Path 37 | total_timesteps 488.
Path 38 | total_timesteps 502.
Path 39 | total_timesteps 511.
Path 40 | total_timesteps 526.
Path 41 | total_timesteps 554.
Path 42 | total_timesteps 572.
Path 43 | total_timesteps 584.
Path 44 | total_timesteps 606.
Path 45 | total_timesteps 629.
Path 46 | total_timesteps 640.
Path 47 | total_timesteps 649.
Path 48 | total_timesteps 656.
Path 49 | total_timesteps 665.
Path 50 | total_timesteps 675.
Path 51 | total_timesteps 689.
Path 52 | total_timesteps 704.
Path 53 | total_timesteps 722.
Path 54 | total_timesteps 738.
Path 55 | total_timesteps 757.
Path 56 | total_timesteps 783.
Path 57 | total_timesteps 804.
Path 58 | total_timesteps 814.
Path 59 | total_timesteps 826.
Path 60 | total_timesteps 836.
Path 61 | total_timesteps 854.
Path 62 | total_timesteps 876.
Path 63 | total_timesteps 890.
Path 64 | total_timesteps 906.
Path 65 | total_timesteps 913.
Path 66 | total_timesteps 926.
Path 67 | total_timesteps 948.
Path 68 | total_timesteps 965.
Path 69 | total_timesteps 974.
Path 70 | total_timesteps 997.
Path 71 | total_timesteps 1005.
Path 72 | total_timesteps 1013.
Path 73 | total_timesteps 1024.
Path 74 | total_timesteps 1044.
Path 75 | total_timesteps 1054.
Path 76 | total_timesteps 1064.
Path 77 | total_timesteps 1083.
Path 78 | total_timesteps 1097.
Path 79 | total_timesteps 1113.
Path 80 | total_timesteps 1131.
Path 81 | total_timesteps 1142.
Path 82 | total_timesteps 1158.
Path 83 | total_timesteps 1166.
Path 84 | total_timesteps 1175.
Path 85 | total_timesteps 1195.
Path 86 | total_timesteps 1203.
Path 87 | total_timesteps 1215.
Path 88 | total_timesteps 1237.
Path 89 | total_timesteps 1252.
Path 90 | total_timesteps 1263.
Path 91 | total_timesteps 1271.
Path 92 | total_timesteps 1287.
Path 93 | total_timesteps 1297.
Path 94 | total_timesteps 1319.
Path 95 | total_timesteps 1337.
Path 96 | total_timesteps 1346.
Path 97 | total_timesteps 1356.
Path 98 | total_timesteps 1365.
Path 99 | total_timesteps 1376.
Path 100 | total_timesteps 1385.
Path 101 | total_timesteps 1396.
Path 102 | total_timesteps 1426.
Path 103 | total_timesteps 1442.
Path 104 | total_timesteps 1457.
Path 105 | total_timesteps 1470.
Path 106 | total_timesteps 1478.
Path 107 | total_timesteps 1498.
Path 108 | total_timesteps 1507.
Path 109 | total_timesteps 1519.
Path 110 | total_timesteps 1525.
Path 111 | total_timesteps 1533.
Path 112 | total_timesteps 1550.
Path 113 | total_timesteps 1564.
Path 114 | total_timesteps 1590.
Path 115 | total_timesteps 1608.
Path 116 | total_timesteps 1628.
Path 117 | total_timesteps 1636.
Path 118 | total_timesteps 1651.
Path 119 | total_timesteps 1658.
Path 120 | total_timesteps 1671.
Path 121 | total_timesteps 1680.
Path 122 | total_timesteps 1693.
Path 123 | total_timesteps 1703.
Path 124 | total_timesteps 1718.
Path 125 | total_timesteps 1731.
Path 126 | total_timesteps 1740.
Path 127 | total_timesteps 1764.
Path 128 | total_timesteps 1776.
Path 129 | total_timesteps 1792.
Path 130 | total_timesteps 1802.
Path 131 | total_timesteps 1812.
Path 132 | total_timesteps 1834.
Path 133 | total_timesteps 1843.
Path 134 | total_timesteps 1853.
Path 135 | total_timesteps 1876.
Path 136 | total_timesteps 1890.
Path 137 | total_timesteps 1904.
Path 138 | total_timesteps 1917.
Path 139 | total_timesteps 1931.
Path 140 | total_timesteps 1944.
Path 141 | total_timesteps 1957.
Path 142 | total_timesteps 1971.
Path 143 | total_timesteps 1990.
Path 144 | total_timesteps 2001.
Path 145 | total_timesteps 2010.
Path 146 | total_timesteps 2018.
Path 147 | total_timesteps 2035.
Path 148 | total_timesteps 2044.
Path 149 | total_timesteps 2057.
Path 150 | total_timesteps 2067.
Path 151 | total_timesteps 2077.
Path 152 | total_timesteps 2096.
Path 153 | total_timesteps 2118.
Path 154 | total_timesteps 2133.
Path 155 | total_timesteps 2152.
Path 156 | total_timesteps 2161.
Path 157 | total_timesteps 2174.
Path 158 | total_timesteps 2181.
Path 159 | total_timesteps 2199.
Path 160 | total_timesteps 2214.
Path 161 | total_timesteps 2226.
Path 162 | total_timesteps 2246.
Path 163 | total_timesteps 2254.
Path 164 | total_timesteps 2269.
Path 165 | total_timesteps 2285.
Path 166 | total_timesteps 2296.
Path 167 | total_timesteps 2309.
Path 168 | total_timesteps 2345.
Path 169 | total_timesteps 2368.
Path 170 | total_timesteps 2377.
Path 171 | total_timesteps 2386.
Path 172 | total_timesteps 2399.
Path 173 | total_timesteps 2417.
Path 174 | total_timesteps 2443.
Path 175 | total_timesteps 2467.
Path 176 | total_timesteps 2478.
Path 177 | total_timesteps 2485.
Path 178 | total_timesteps 2502.
Path 179 | total_timesteps 2512.
Path 180 | total_timesteps 2523.
Path 181 | total_timesteps 2530.
Path 182 | total_timesteps 2553.
Path 183 | total_timesteps 2565.
Path 184 | total_timesteps 2572.
Path 185 | total_timesteps 2581.
Path 186 | total_timesteps 2592.
Path 187 | total_timesteps 2606.
Path 188 | total_timesteps 2619.
Path 189 | total_timesteps 2644.
Path 190 | total_timesteps 2654.
Path 191 | total_timesteps 2669.
Path 192 | total_timesteps 2679.
Path 193 | total_timesteps 2687.
Path 194 | total_timesteps 2697.
Path 195 | total_timesteps 2707.
Path 196 | total_timesteps 2718.
Path 197 | total_timesteps 2735.
Path 198 | total_timesteps 2754.
Path 199 | total_timesteps 2764.
Path 200 | total_timesteps 2780.
Path 201 | total_timesteps 2797.
Path 202 | total_timesteps 2811.
Path 203 | total_timesteps 2832.
Path 204 | total_timesteps 2846.
Path 205 | total_timesteps 2862.
Path 206 | total_timesteps 2872.
Path 207 | total_timesteps 2885.
Path 208 | total_timesteps 2894.
Path 209 | total_timesteps 2907.
Path 210 | total_timesteps 2916.
Path 211 | total_timesteps 2925.
Path 212 | total_timesteps 2944.
Path 213 | total_timesteps 2955.
Path 214 | total_timesteps 2971.
Path 215 | total_timesteps 2990.
Path 216 | total_timesteps 3001.
Path 217 | total_timesteps 3019.
Path 218 | total_timesteps 3027.
Path 219 | total_timesteps 3036.
Path 220 | total_timesteps 3051.
Path 221 | total_timesteps 3066.
Path 222 | total_timesteps 3075.
Path 223 | total_timesteps 3086.
Path 224 | total_timesteps 3096.
Path 225 | total_timesteps 3111.
Path 226 | total_timesteps 3124.
Path 227 | total_timesteps 3132.
Path 228 | total_timesteps 3141.
Path 229 | total_timesteps 3151.
Path 230 | total_timesteps 3163.
Path 231 | total_timesteps 3180.
Path 232 | total_timesteps 3198.
Path 233 | total_timesteps 3209.
Path 234 | total_timesteps 3217.
Path 235 | total_timesteps 3228.
Path 236 | total_timesteps 3251.
Path 237 | total_timesteps 3261.
Path 238 | total_timesteps 3268.
Path 239 | total_timesteps 3294.
Path 240 | total_timesteps 3309.
Path 241 | total_timesteps 3319.
Path 242 | total_timesteps 3334.
Path 243 | total_timesteps 3350.
Path 244 | total_timesteps 3359.
Path 245 | total_timesteps 3366.
Path 246 | total_timesteps 3385.
Path 247 | total_timesteps 3407.
Path 248 | total_timesteps 3417.
Path 249 | total_timesteps 3429.
Path 250 | total_timesteps 3438.
Path 251 | total_timesteps 3461.
Path 252 | total_timesteps 3472.
Path 253 | total_timesteps 3493.
Path 254 | total_timesteps 3508.
Path 255 | total_timesteps 3523.
Path 256 | total_timesteps 3533.
Path 257 | total_timesteps 3548.
Path 258 | total_timesteps 3556.
Path 259 | total_timesteps 3572.
Path 260 | total_timesteps 3590.
Path 261 | total_timesteps 3607.
Path 262 | total_timesteps 3617.
Path 263 | total_timesteps 3635.
Path 264 | total_timesteps 3650.
Path 265 | total_timesteps 3661.
Path 266 | total_timesteps 3669.
Path 267 | total_timesteps 3686.
Path 268 | total_timesteps 3702.
Path 269 | total_timesteps 3717.
Path 270 | total_timesteps 3728.
Path 271 | total_timesteps 3738.
Path 272 | total_timesteps 3749.
Path 273 | total_timesteps 3756.
Path 274 | total_timesteps 3777.
Path 275 | total_timesteps 3787.
Path 276 | total_timesteps 3799.
Path 277 | total_timesteps 3807.
Path 278 | total_timesteps 3819.
Path 279 | total_timesteps 3851.
Path 280 | total_timesteps 3860.
Path 281 | total_timesteps 3873.
Path 282 | total_timesteps 3884.
Path 283 | total_timesteps 3891.
Path 284 | total_timesteps 3913.
Path 285 | total_timesteps 3925.
Path 286 | total_timesteps 3939.
Path 287 | total_timesteps 3948.
Path 288 | total_timesteps 3962.
Path 289 | total_timesteps 3975.
Path 290 | total_timesteps 3995.
Path 291 | total_timesteps 4004.
Path 292 | total_timesteps 4017.
Path 293 | total_timesteps 4031.
Path 294 | total_timesteps 4041.
Path 295 | total_timesteps 4049.
Path 296 | total_timesteps 4061.
Path 297 | total_timesteps 4070.
Path 298 | total_timesteps 4083.
Path 299 | total_timesteps 4101.
Path 300 | total_timesteps 4113.
Path 301 | total_timesteps 4123.
Path 302 | total_timesteps 4138.
Path 303 | total_timesteps 4150.
Path 304 | total_timesteps 4160.
Path 305 | total_timesteps 4169.
Path 306 | total_timesteps 4190.
Path 307 | total_timesteps 4209.
Path 308 | total_timesteps 4217.
Path 309 | total_timesteps 4245.
Path 310 | total_timesteps 4264.
Path 311 | total_timesteps 4278.
Path 312 | total_timesteps 4299.
Path 313 | total_timesteps 4309.
Path 314 | total_timesteps 4321.
Path 315 | total_timesteps 4337.
Path 316 | total_timesteps 4352.
Path 317 | total_timesteps 4364.
Path 318 | total_timesteps 4378.
Path 319 | total_timesteps 4391.
Path 320 | total_timesteps 4410.
Path 321 | total_timesteps 4423.
Path 322 | total_timesteps 4436.
Path 323 | total_timesteps 4445.
Path 324 | total_timesteps 4459.
Path 325 | total_timesteps 4467.
Path 326 | total_timesteps 4484.
Path 327 | total_timesteps 4492.
Path 328 | total_timesteps 4499.
Path 329 | total_timesteps 4512.
Path 330 | total_timesteps 4522.
Path 331 | total_timesteps 4541.
Path 332 | total_timesteps 4552.
Path 333 | total_timesteps 4562.
Path 334 | total_timesteps 4582.
Path 335 | total_timesteps 4590.
Path 336 | total_timesteps 4600.
Path 337 | total_timesteps 4617.
Path 338 | total_timesteps 4628.
Path 339 | total_timesteps 4637.
Path 340 | total_timesteps 4652.
Path 341 | total_timesteps 4663.
Path 342 | total_timesteps 4675.
Path 343 | total_timesteps 4683.
Path 344 | total_timesteps 4697.
Path 345 | total_timesteps 4712.
Path 346 | total_timesteps 4720.
Path 347 | total_timesteps 4730.
Path 348 | total_timesteps 4745.
Path 349 | total_timesteps 4759.
Path 350 | total_timesteps 4768.
Path 351 | total_timesteps 4777.
Path 352 | total_timesteps 4786.
Path 353 | total_timesteps 4796.
Path 354 | total_timesteps 4804.
Path 355 | total_timesteps 4820.
Path 356 | total_timesteps 4830.
Path 357 | total_timesteps 4844.
Path 358 | total_timesteps 4859.
Path 359 | total_timesteps 4867.
Path 360 | total_timesteps 4883.
Path 361 | total_timesteps 4897.
Path 362 | total_timesteps 4911.
Path 363 | total_timesteps 4926.
Path 364 | total_timesteps 4940.
Path 365 | total_timesteps 4955.
Path 366 | total_timesteps 4964.
Path 367 | total_timesteps 4983.
Path 368 | total_timesteps 4997.
Path 369 | total_timesteps 5015.
Path 370 | total_timesteps 5032.
Path 371 | total_timesteps 5042.
Path 372 | total_timesteps 5061.
Path 373 | total_timesteps 5068.
Path 374 | total_timesteps 5081.
Path 375 | total_timesteps 5089.
Path 376 | total_timesteps 5103.
Path 377 | total_timesteps 5113.
Path 378 | total_timesteps 5128.
Path 379 | total_timesteps 5151.
Path 380 | total_timesteps 5170.
Path 381 | total_timesteps 5189.
Path 382 | total_timesteps 5203.
Path 383 | total_timesteps 5216.
Path 384 | total_timesteps 5224.
Path 385 | total_timesteps 5231.
Path 386 | total_timesteps 5245.
Path 387 | total_timesteps 5260.
Path 388 | total_timesteps 5270.
Path 389 | total_timesteps 5280.
Path 390 | total_timesteps 5288.
Path 391 | total_timesteps 5304.
Path 392 | total_timesteps 5323.
Path 393 | total_timesteps 5335.
Path 394 | total_timesteps 5349.
Path 395 | total_timesteps 5366.
Path 396 | total_timesteps 5384.
Path 397 | total_timesteps 5392.
Path 398 | total_timesteps 5403.
Path 399 | total_timesteps 5420.
Path 400 | total_timesteps 5439.
Path 401 | total_timesteps 5449.
Path 402 | total_timesteps 5460.
Path 403 | total_timesteps 5468.
Path 404 | total_timesteps 5478.
Path 405 | total_timesteps 5497.
Path 406 | total_timesteps 5508.
Path 407 | total_timesteps 5522.
Path 408 | total_timesteps 5535.
Path 409 | total_timesteps 5546.
Path 410 | total_timesteps 5557.
Path 411 | total_timesteps 5576.
Path 412 | total_timesteps 5596.
Path 413 | total_timesteps 5605.
Path 414 | total_timesteps 5621.
Path 415 | total_timesteps 5632.
Path 416 | total_timesteps 5649.
Path 417 | total_timesteps 5661.
Path 418 | total_timesteps 5674.
Path 419 | total_timesteps 5689.
Path 420 | total_timesteps 5705.
Path 421 | total_timesteps 5716.
Path 422 | total_timesteps 5723.
Path 423 | total_timesteps 5731.
Path 424 | total_timesteps 5741.
Path 425 | total_timesteps 5752.
Path 426 | total_timesteps 5769.
Path 427 | total_timesteps 5789.
Path 428 | total_timesteps 5808.
Path 429 | total_timesteps 5817.
Path 430 | total_timesteps 5832.
Path 431 | total_timesteps 5849.
Path 432 | total_timesteps 5858.
Path 433 | total_timesteps 5878.
Path 434 | total_timesteps 5890.
Path 435 | total_timesteps 5899.
Path 436 | total_timesteps 5911.
Path 437 | total_timesteps 5927.
Path 438 | total_timesteps 5947.
Path 439 | total_timesteps 5956.
Path 440 | total_timesteps 5967.
Path 441 | total_timesteps 5976.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.66    |
| Iteration     | 21       |
| MaximumReturn | 8.03     |
| MinimumReturn | -19.2    |
| TotalSamples  | 92157    |
----------------------------
itr #22 | 
Fitting dynamics.
Validation loss = 0.006716408766806126
Validation loss = 0.006331156007945538
Validation loss = 0.0065523358061909676
Validation loss = 0.006524946540594101
Validation loss = 0.006909251678735018
Validation loss = 0.006765817757695913
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 59.
Path 6 | total_timesteps 68.
Path 7 | total_timesteps 78.
Path 8 | total_timesteps 90.
Path 9 | total_timesteps 107.
Path 10 | total_timesteps 120.
Path 11 | total_timesteps 131.
Path 12 | total_timesteps 148.
Path 13 | total_timesteps 158.
Path 14 | total_timesteps 175.
Path 15 | total_timesteps 192.
Path 16 | total_timesteps 201.
Path 17 | total_timesteps 220.
Path 18 | total_timesteps 228.
Path 19 | total_timesteps 250.
Path 20 | total_timesteps 281.
Path 21 | total_timesteps 291.
Path 22 | total_timesteps 299.
Path 23 | total_timesteps 306.
Path 24 | total_timesteps 322.
Path 25 | total_timesteps 335.
Path 26 | total_timesteps 352.
Path 27 | total_timesteps 367.
Path 28 | total_timesteps 386.
Path 29 | total_timesteps 404.
Path 30 | total_timesteps 425.
Path 31 | total_timesteps 440.
Path 32 | total_timesteps 455.
Path 33 | total_timesteps 482.
Path 34 | total_timesteps 498.
Path 35 | total_timesteps 514.
Path 36 | total_timesteps 533.
Path 37 | total_timesteps 553.
Path 38 | total_timesteps 577.
Path 39 | total_timesteps 601.
Path 40 | total_timesteps 625.
Path 41 | total_timesteps 639.
Path 42 | total_timesteps 652.
Path 43 | total_timesteps 665.
Path 44 | total_timesteps 674.
Path 45 | total_timesteps 688.
Path 46 | total_timesteps 701.
Path 47 | total_timesteps 724.
Path 48 | total_timesteps 740.
Path 49 | total_timesteps 754.
Path 50 | total_timesteps 763.
Path 51 | total_timesteps 775.
Path 52 | total_timesteps 790.
Path 53 | total_timesteps 801.
Path 54 | total_timesteps 812.
Path 55 | total_timesteps 829.
Path 56 | total_timesteps 842.
Path 57 | total_timesteps 862.
Path 58 | total_timesteps 871.
Path 59 | total_timesteps 880.
Path 60 | total_timesteps 908.
Path 61 | total_timesteps 916.
Path 62 | total_timesteps 930.
Path 63 | total_timesteps 948.
Path 64 | total_timesteps 967.
Path 65 | total_timesteps 989.
Path 66 | total_timesteps 1003.
Path 67 | total_timesteps 1019.
Path 68 | total_timesteps 1035.
Path 69 | total_timesteps 1055.
Path 70 | total_timesteps 1068.
Path 71 | total_timesteps 1080.
Path 72 | total_timesteps 1105.
Path 73 | total_timesteps 1113.
Path 74 | total_timesteps 1135.
Path 75 | total_timesteps 1145.
Path 76 | total_timesteps 1156.
Path 77 | total_timesteps 1163.
Path 78 | total_timesteps 1173.
Path 79 | total_timesteps 1186.
Path 80 | total_timesteps 1202.
Path 81 | total_timesteps 1217.
Path 82 | total_timesteps 1234.
Path 83 | total_timesteps 1245.
Path 84 | total_timesteps 1257.
Path 85 | total_timesteps 1271.
Path 86 | total_timesteps 1282.
Path 87 | total_timesteps 1310.
Path 88 | total_timesteps 1323.
Path 89 | total_timesteps 1338.
Path 90 | total_timesteps 1356.
Path 91 | total_timesteps 1371.
Path 92 | total_timesteps 1388.
Path 93 | total_timesteps 1399.
Path 94 | total_timesteps 1409.
Path 95 | total_timesteps 1427.
Path 96 | total_timesteps 1436.
Path 97 | total_timesteps 1458.
Path 98 | total_timesteps 1475.
Path 99 | total_timesteps 1483.
Path 100 | total_timesteps 1502.
Path 101 | total_timesteps 1512.
Path 102 | total_timesteps 1542.
Path 103 | total_timesteps 1553.
Path 104 | total_timesteps 1572.
Path 105 | total_timesteps 1597.
Path 106 | total_timesteps 1610.
Path 107 | total_timesteps 1620.
Path 108 | total_timesteps 1636.
Path 109 | total_timesteps 1652.
Path 110 | total_timesteps 1668.
Path 111 | total_timesteps 1696.
Path 112 | total_timesteps 1725.
Path 113 | total_timesteps 1735.
Path 114 | total_timesteps 1754.
Path 115 | total_timesteps 1782.
Path 116 | total_timesteps 1794.
Path 117 | total_timesteps 1800.
Path 118 | total_timesteps 1820.
Path 119 | total_timesteps 1837.
Path 120 | total_timesteps 1853.
Path 121 | total_timesteps 1864.
Path 122 | total_timesteps 1884.
Path 123 | total_timesteps 1896.
Path 124 | total_timesteps 1908.
Path 125 | total_timesteps 1938.
Path 126 | total_timesteps 1947.
Path 127 | total_timesteps 1961.
Path 128 | total_timesteps 1969.
Path 129 | total_timesteps 1983.
Path 130 | total_timesteps 1991.
Path 131 | total_timesteps 2003.
Path 132 | total_timesteps 2014.
Path 133 | total_timesteps 2037.
Path 134 | total_timesteps 2051.
Path 135 | total_timesteps 2065.
Path 136 | total_timesteps 2084.
Path 137 | total_timesteps 2100.
Path 138 | total_timesteps 2118.
Path 139 | total_timesteps 2135.
Path 140 | total_timesteps 2147.
Path 141 | total_timesteps 2156.
Path 142 | total_timesteps 2170.
Path 143 | total_timesteps 2177.
Path 144 | total_timesteps 2190.
Path 145 | total_timesteps 2200.
Path 146 | total_timesteps 2214.
Path 147 | total_timesteps 2222.
Path 148 | total_timesteps 2236.
Path 149 | total_timesteps 2251.
Path 150 | total_timesteps 2265.
Path 151 | total_timesteps 2276.
Path 152 | total_timesteps 2292.
Path 153 | total_timesteps 2299.
Path 154 | total_timesteps 2320.
Path 155 | total_timesteps 2331.
Path 156 | total_timesteps 2347.
Path 157 | total_timesteps 2374.
Path 158 | total_timesteps 2390.
Path 159 | total_timesteps 2401.
Path 160 | total_timesteps 2420.
Path 161 | total_timesteps 2438.
Path 162 | total_timesteps 2448.
Path 163 | total_timesteps 2468.
Path 164 | total_timesteps 2477.
Path 165 | total_timesteps 2491.
Path 166 | total_timesteps 2509.
Path 167 | total_timesteps 2518.
Path 168 | total_timesteps 2536.
Path 169 | total_timesteps 2554.
Path 170 | total_timesteps 2571.
Path 171 | total_timesteps 2581.
Path 172 | total_timesteps 2587.
Path 173 | total_timesteps 2596.
Path 174 | total_timesteps 2606.
Path 175 | total_timesteps 2620.
Path 176 | total_timesteps 2640.
Path 177 | total_timesteps 2656.
Path 178 | total_timesteps 2669.
Path 179 | total_timesteps 2691.
Path 180 | total_timesteps 2708.
Path 181 | total_timesteps 2716.
Path 182 | total_timesteps 2726.
Path 183 | total_timesteps 2742.
Path 184 | total_timesteps 2753.
Path 185 | total_timesteps 2769.
Path 186 | total_timesteps 2785.
Path 187 | total_timesteps 2800.
Path 188 | total_timesteps 2810.
Path 189 | total_timesteps 2818.
Path 190 | total_timesteps 2844.
Path 191 | total_timesteps 2858.
Path 192 | total_timesteps 2868.
Path 193 | total_timesteps 2894.
Path 194 | total_timesteps 2903.
Path 195 | total_timesteps 2924.
Path 196 | total_timesteps 2939.
Path 197 | total_timesteps 2951.
Path 198 | total_timesteps 2960.
Path 199 | total_timesteps 2973.
Path 200 | total_timesteps 2995.
Path 201 | total_timesteps 3002.
Path 202 | total_timesteps 3020.
Path 203 | total_timesteps 3039.
Path 204 | total_timesteps 3049.
Path 205 | total_timesteps 3072.
Path 206 | total_timesteps 3095.
Path 207 | total_timesteps 3112.
Path 208 | total_timesteps 3127.
Path 209 | total_timesteps 3137.
Path 210 | total_timesteps 3151.
Path 211 | total_timesteps 3167.
Path 212 | total_timesteps 3174.
Path 213 | total_timesteps 3196.
Path 214 | total_timesteps 3212.
Path 215 | total_timesteps 3223.
Path 216 | total_timesteps 3244.
Path 217 | total_timesteps 3255.
Path 218 | total_timesteps 3270.
Path 219 | total_timesteps 3278.
Path 220 | total_timesteps 3289.
Path 221 | total_timesteps 3300.
Path 222 | total_timesteps 3315.
Path 223 | total_timesteps 3334.
Path 224 | total_timesteps 3352.
Path 225 | total_timesteps 3369.
Path 226 | total_timesteps 3386.
Path 227 | total_timesteps 3401.
Path 228 | total_timesteps 3414.
Path 229 | total_timesteps 3427.
Path 230 | total_timesteps 3447.
Path 231 | total_timesteps 3456.
Path 232 | total_timesteps 3473.
Path 233 | total_timesteps 3481.
Path 234 | total_timesteps 3493.
Path 235 | total_timesteps 3507.
Path 236 | total_timesteps 3519.
Path 237 | total_timesteps 3536.
Path 238 | total_timesteps 3550.
Path 239 | total_timesteps 3566.
Path 240 | total_timesteps 3575.
Path 241 | total_timesteps 3592.
Path 242 | total_timesteps 3611.
Path 243 | total_timesteps 3618.
Path 244 | total_timesteps 3627.
Path 245 | total_timesteps 3646.
Path 246 | total_timesteps 3657.
Path 247 | total_timesteps 3673.
Path 248 | total_timesteps 3691.
Path 249 | total_timesteps 3705.
Path 250 | total_timesteps 3716.
Path 251 | total_timesteps 3726.
Path 252 | total_timesteps 3742.
Path 253 | total_timesteps 3761.
Path 254 | total_timesteps 3770.
Path 255 | total_timesteps 3777.
Path 256 | total_timesteps 3791.
Path 257 | total_timesteps 3799.
Path 258 | total_timesteps 3814.
Path 259 | total_timesteps 3833.
Path 260 | total_timesteps 3865.
Path 261 | total_timesteps 3888.
Path 262 | total_timesteps 3901.
Path 263 | total_timesteps 3915.
Path 264 | total_timesteps 3933.
Path 265 | total_timesteps 3947.
Path 266 | total_timesteps 3972.
Path 267 | total_timesteps 3985.
Path 268 | total_timesteps 3997.
Path 269 | total_timesteps 4021.
Path 270 | total_timesteps 4039.
Path 271 | total_timesteps 4047.
Path 272 | total_timesteps 4077.
Path 273 | total_timesteps 4090.
Path 274 | total_timesteps 4102.
Path 275 | total_timesteps 4109.
Path 276 | total_timesteps 4123.
Path 277 | total_timesteps 4138.
Path 278 | total_timesteps 4148.
Path 279 | total_timesteps 4173.
Path 280 | total_timesteps 4187.
Path 281 | total_timesteps 4203.
Path 282 | total_timesteps 4227.
Path 283 | total_timesteps 4234.
Path 284 | total_timesteps 4254.
Path 285 | total_timesteps 4260.
Path 286 | total_timesteps 4277.
Path 287 | total_timesteps 4291.
Path 288 | total_timesteps 4301.
Path 289 | total_timesteps 4314.
Path 290 | total_timesteps 4328.
Path 291 | total_timesteps 4350.
Path 292 | total_timesteps 4361.
Path 293 | total_timesteps 4376.
Path 294 | total_timesteps 4386.
Path 295 | total_timesteps 4409.
Path 296 | total_timesteps 4431.
Path 297 | total_timesteps 4449.
Path 298 | total_timesteps 4470.
Path 299 | total_timesteps 4484.
Path 300 | total_timesteps 4492.
Path 301 | total_timesteps 4509.
Path 302 | total_timesteps 4526.
Path 303 | total_timesteps 4540.
Path 304 | total_timesteps 4553.
Path 305 | total_timesteps 4571.
Path 306 | total_timesteps 4584.
Path 307 | total_timesteps 4591.
Path 308 | total_timesteps 4606.
Path 309 | total_timesteps 4629.
Path 310 | total_timesteps 4641.
Path 311 | total_timesteps 4653.
Path 312 | total_timesteps 4668.
Path 313 | total_timesteps 4682.
Path 314 | total_timesteps 4699.
Path 315 | total_timesteps 4716.
Path 316 | total_timesteps 4733.
Path 317 | total_timesteps 4742.
Path 318 | total_timesteps 4760.
Path 319 | total_timesteps 4776.
Path 320 | total_timesteps 4791.
Path 321 | total_timesteps 4801.
Path 322 | total_timesteps 4808.
Path 323 | total_timesteps 4817.
Path 324 | total_timesteps 4826.
Path 325 | total_timesteps 4840.
Path 326 | total_timesteps 4860.
Path 327 | total_timesteps 4878.
Path 328 | total_timesteps 4891.
Path 329 | total_timesteps 4903.
Path 330 | total_timesteps 4913.
Path 331 | total_timesteps 4926.
Path 332 | total_timesteps 4941.
Path 333 | total_timesteps 4954.
Path 334 | total_timesteps 4964.
Path 335 | total_timesteps 4977.
Path 336 | total_timesteps 4996.
Path 337 | total_timesteps 5008.
Path 338 | total_timesteps 5025.
Path 339 | total_timesteps 5038.
Path 340 | total_timesteps 5058.
Path 341 | total_timesteps 5066.
Path 342 | total_timesteps 5082.
Path 343 | total_timesteps 5093.
Path 344 | total_timesteps 5103.
Path 345 | total_timesteps 5121.
Path 346 | total_timesteps 5131.
Path 347 | total_timesteps 5139.
Path 348 | total_timesteps 5154.
Path 349 | total_timesteps 5169.
Path 350 | total_timesteps 5180.
Path 351 | total_timesteps 5191.
Path 352 | total_timesteps 5210.
Path 353 | total_timesteps 5226.
Path 354 | total_timesteps 5240.
Path 355 | total_timesteps 5253.
Path 356 | total_timesteps 5272.
Path 357 | total_timesteps 5286.
Path 358 | total_timesteps 5298.
Path 359 | total_timesteps 5306.
Path 360 | total_timesteps 5316.
Path 361 | total_timesteps 5334.
Path 362 | total_timesteps 5347.
Path 363 | total_timesteps 5361.
Path 364 | total_timesteps 5379.
Path 365 | total_timesteps 5395.
Path 366 | total_timesteps 5410.
Path 367 | total_timesteps 5422.
Path 368 | total_timesteps 5431.
Path 369 | total_timesteps 5441.
Path 370 | total_timesteps 5459.
Path 371 | total_timesteps 5480.
Path 372 | total_timesteps 5491.
Path 373 | total_timesteps 5505.
Path 374 | total_timesteps 5519.
Path 375 | total_timesteps 5538.
Path 376 | total_timesteps 5550.
Path 377 | total_timesteps 5566.
Path 378 | total_timesteps 5584.
Path 379 | total_timesteps 5599.
Path 380 | total_timesteps 5618.
Path 381 | total_timesteps 5635.
Path 382 | total_timesteps 5649.
Path 383 | total_timesteps 5661.
Path 384 | total_timesteps 5683.
Path 385 | total_timesteps 5695.
Path 386 | total_timesteps 5712.
Path 387 | total_timesteps 5723.
Path 388 | total_timesteps 5732.
Path 389 | total_timesteps 5744.
Path 390 | total_timesteps 5769.
Path 391 | total_timesteps 5784.
Path 392 | total_timesteps 5794.
Path 393 | total_timesteps 5810.
Path 394 | total_timesteps 5833.
Path 395 | total_timesteps 5846.
Path 396 | total_timesteps 5858.
Path 397 | total_timesteps 5870.
Path 398 | total_timesteps 5895.
Path 399 | total_timesteps 5907.
Path 400 | total_timesteps 5915.
Path 401 | total_timesteps 5934.
Path 402 | total_timesteps 5941.
Path 403 | total_timesteps 5964.
Path 404 | total_timesteps 5976.
Path 405 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.9     |
| Iteration     | 22       |
| MaximumReturn | 9.57     |
| MinimumReturn | -22.8    |
| TotalSamples  | 96171    |
----------------------------
itr #23 | 
Fitting dynamics.
Validation loss = 0.006382302846759558
Validation loss = 0.006147348787635565
Validation loss = 0.006955871358513832
Validation loss = 0.006200980395078659
Validation loss = 0.006270041223615408
Validation loss = 0.0068440623581409454
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 20.
Path 2 | total_timesteps 33.
Path 3 | total_timesteps 46.
Path 4 | total_timesteps 69.
Path 5 | total_timesteps 83.
Path 6 | total_timesteps 121.
Path 7 | total_timesteps 141.
Path 8 | total_timesteps 150.
Path 9 | total_timesteps 161.
Path 10 | total_timesteps 178.
Path 11 | total_timesteps 197.
Path 12 | total_timesteps 206.
Path 13 | total_timesteps 220.
Path 14 | total_timesteps 234.
Path 15 | total_timesteps 244.
Path 16 | total_timesteps 258.
Path 17 | total_timesteps 268.
Path 18 | total_timesteps 279.
Path 19 | total_timesteps 294.
Path 20 | total_timesteps 303.
Path 21 | total_timesteps 314.
Path 22 | total_timesteps 332.
Path 23 | total_timesteps 351.
Path 24 | total_timesteps 370.
Path 25 | total_timesteps 379.
Path 26 | total_timesteps 392.
Path 27 | total_timesteps 404.
Path 28 | total_timesteps 417.
Path 29 | total_timesteps 427.
Path 30 | total_timesteps 453.
Path 31 | total_timesteps 469.
Path 32 | total_timesteps 490.
Path 33 | total_timesteps 505.
Path 34 | total_timesteps 512.
Path 35 | total_timesteps 523.
Path 36 | total_timesteps 537.
Path 37 | total_timesteps 554.
Path 38 | total_timesteps 564.
Path 39 | total_timesteps 588.
Path 40 | total_timesteps 604.
Path 41 | total_timesteps 619.
Path 42 | total_timesteps 632.
Path 43 | total_timesteps 644.
Path 44 | total_timesteps 661.
Path 45 | total_timesteps 691.
Path 46 | total_timesteps 701.
Path 47 | total_timesteps 714.
Path 48 | total_timesteps 729.
Path 49 | total_timesteps 748.
Path 50 | total_timesteps 757.
Path 51 | total_timesteps 770.
Path 52 | total_timesteps 787.
Path 53 | total_timesteps 803.
Path 54 | total_timesteps 814.
Path 55 | total_timesteps 835.
Path 56 | total_timesteps 843.
Path 57 | total_timesteps 853.
Path 58 | total_timesteps 871.
Path 59 | total_timesteps 883.
Path 60 | total_timesteps 890.
Path 61 | total_timesteps 902.
Path 62 | total_timesteps 913.
Path 63 | total_timesteps 930.
Path 64 | total_timesteps 942.
Path 65 | total_timesteps 963.
Path 66 | total_timesteps 986.
Path 67 | total_timesteps 1003.
Path 68 | total_timesteps 1015.
Path 69 | total_timesteps 1028.
Path 70 | total_timesteps 1050.
Path 71 | total_timesteps 1070.
Path 72 | total_timesteps 1090.
Path 73 | total_timesteps 1101.
Path 74 | total_timesteps 1112.
Path 75 | total_timesteps 1119.
Path 76 | total_timesteps 1130.
Path 77 | total_timesteps 1147.
Path 78 | total_timesteps 1165.
Path 79 | total_timesteps 1184.
Path 80 | total_timesteps 1199.
Path 81 | total_timesteps 1212.
Path 82 | total_timesteps 1222.
Path 83 | total_timesteps 1232.
Path 84 | total_timesteps 1252.
Path 85 | total_timesteps 1273.
Path 86 | total_timesteps 1284.
Path 87 | total_timesteps 1303.
Path 88 | total_timesteps 1319.
Path 89 | total_timesteps 1328.
Path 90 | total_timesteps 1365.
Path 91 | total_timesteps 1378.
Path 92 | total_timesteps 1394.
Path 93 | total_timesteps 1408.
Path 94 | total_timesteps 1427.
Path 95 | total_timesteps 1439.
Path 96 | total_timesteps 1455.
Path 97 | total_timesteps 1481.
Path 98 | total_timesteps 1492.
Path 99 | total_timesteps 1506.
Path 100 | total_timesteps 1519.
Path 101 | total_timesteps 1534.
Path 102 | total_timesteps 1559.
Path 103 | total_timesteps 1573.
Path 104 | total_timesteps 1588.
Path 105 | total_timesteps 1596.
Path 106 | total_timesteps 1613.
Path 107 | total_timesteps 1623.
Path 108 | total_timesteps 1634.
Path 109 | total_timesteps 1652.
Path 110 | total_timesteps 1671.
Path 111 | total_timesteps 1690.
Path 112 | total_timesteps 1700.
Path 113 | total_timesteps 1727.
Path 114 | total_timesteps 1743.
Path 115 | total_timesteps 1754.
Path 116 | total_timesteps 1772.
Path 117 | total_timesteps 1789.
Path 118 | total_timesteps 1805.
Path 119 | total_timesteps 1812.
Path 120 | total_timesteps 1841.
Path 121 | total_timesteps 1852.
Path 122 | total_timesteps 1865.
Path 123 | total_timesteps 1876.
Path 124 | total_timesteps 1890.
Path 125 | total_timesteps 1906.
Path 126 | total_timesteps 1923.
Path 127 | total_timesteps 1931.
Path 128 | total_timesteps 1946.
Path 129 | total_timesteps 1963.
Path 130 | total_timesteps 1986.
Path 131 | total_timesteps 1996.
Path 132 | total_timesteps 2015.
Path 133 | total_timesteps 2027.
Path 134 | total_timesteps 2046.
Path 135 | total_timesteps 2059.
Path 136 | total_timesteps 2070.
Path 137 | total_timesteps 2082.
Path 138 | total_timesteps 2108.
Path 139 | total_timesteps 2124.
Path 140 | total_timesteps 2138.
Path 141 | total_timesteps 2165.
Path 142 | total_timesteps 2177.
Path 143 | total_timesteps 2199.
Path 144 | total_timesteps 2210.
Path 145 | total_timesteps 2227.
Path 146 | total_timesteps 2236.
Path 147 | total_timesteps 2245.
Path 148 | total_timesteps 2264.
Path 149 | total_timesteps 2274.
Path 150 | total_timesteps 2290.
Path 151 | total_timesteps 2305.
Path 152 | total_timesteps 2313.
Path 153 | total_timesteps 2323.
Path 154 | total_timesteps 2336.
Path 155 | total_timesteps 2353.
Path 156 | total_timesteps 2362.
Path 157 | total_timesteps 2374.
Path 158 | total_timesteps 2400.
Path 159 | total_timesteps 2408.
Path 160 | total_timesteps 2419.
Path 161 | total_timesteps 2445.
Path 162 | total_timesteps 2459.
Path 163 | total_timesteps 2471.
Path 164 | total_timesteps 2489.
Path 165 | total_timesteps 2505.
Path 166 | total_timesteps 2519.
Path 167 | total_timesteps 2536.
Path 168 | total_timesteps 2547.
Path 169 | total_timesteps 2554.
Path 170 | total_timesteps 2564.
Path 171 | total_timesteps 2580.
Path 172 | total_timesteps 2596.
Path 173 | total_timesteps 2612.
Path 174 | total_timesteps 2627.
Path 175 | total_timesteps 2635.
Path 176 | total_timesteps 2649.
Path 177 | total_timesteps 2658.
Path 178 | total_timesteps 2671.
Path 179 | total_timesteps 2689.
Path 180 | total_timesteps 2704.
Path 181 | total_timesteps 2715.
Path 182 | total_timesteps 2727.
Path 183 | total_timesteps 2738.
Path 184 | total_timesteps 2749.
Path 185 | total_timesteps 2759.
Path 186 | total_timesteps 2774.
Path 187 | total_timesteps 2788.
Path 188 | total_timesteps 2813.
Path 189 | total_timesteps 2823.
Path 190 | total_timesteps 2838.
Path 191 | total_timesteps 2849.
Path 192 | total_timesteps 2871.
Path 193 | total_timesteps 2884.
Path 194 | total_timesteps 2891.
Path 195 | total_timesteps 2904.
Path 196 | total_timesteps 2911.
Path 197 | total_timesteps 2924.
Path 198 | total_timesteps 2937.
Path 199 | total_timesteps 2956.
Path 200 | total_timesteps 2967.
Path 201 | total_timesteps 2989.
Path 202 | total_timesteps 3004.
Path 203 | total_timesteps 3021.
Path 204 | total_timesteps 3033.
Path 205 | total_timesteps 3056.
Path 206 | total_timesteps 3075.
Path 207 | total_timesteps 3087.
Path 208 | total_timesteps 3097.
Path 209 | total_timesteps 3106.
Path 210 | total_timesteps 3116.
Path 211 | total_timesteps 3133.
Path 212 | total_timesteps 3143.
Path 213 | total_timesteps 3158.
Path 214 | total_timesteps 3173.
Path 215 | total_timesteps 3185.
Path 216 | total_timesteps 3216.
Path 217 | total_timesteps 3224.
Path 218 | total_timesteps 3232.
Path 219 | total_timesteps 3259.
Path 220 | total_timesteps 3273.
Path 221 | total_timesteps 3296.
Path 222 | total_timesteps 3311.
Path 223 | total_timesteps 3323.
Path 224 | total_timesteps 3362.
Path 225 | total_timesteps 3375.
Path 226 | total_timesteps 3387.
Path 227 | total_timesteps 3395.
Path 228 | total_timesteps 3411.
Path 229 | total_timesteps 3424.
Path 230 | total_timesteps 3434.
Path 231 | total_timesteps 3452.
Path 232 | total_timesteps 3466.
Path 233 | total_timesteps 3480.
Path 234 | total_timesteps 3494.
Path 235 | total_timesteps 3506.
Path 236 | total_timesteps 3534.
Path 237 | total_timesteps 3548.
Path 238 | total_timesteps 3578.
Path 239 | total_timesteps 3596.
Path 240 | total_timesteps 3610.
Path 241 | total_timesteps 3630.
Path 242 | total_timesteps 3644.
Path 243 | total_timesteps 3656.
Path 244 | total_timesteps 3670.
Path 245 | total_timesteps 3682.
Path 246 | total_timesteps 3693.
Path 247 | total_timesteps 3700.
Path 248 | total_timesteps 3719.
Path 249 | total_timesteps 3742.
Path 250 | total_timesteps 3758.
Path 251 | total_timesteps 3765.
Path 252 | total_timesteps 3775.
Path 253 | total_timesteps 3786.
Path 254 | total_timesteps 3803.
Path 255 | total_timesteps 3813.
Path 256 | total_timesteps 3837.
Path 257 | total_timesteps 3855.
Path 258 | total_timesteps 3875.
Path 259 | total_timesteps 3888.
Path 260 | total_timesteps 3897.
Path 261 | total_timesteps 3907.
Path 262 | total_timesteps 3923.
Path 263 | total_timesteps 3933.
Path 264 | total_timesteps 3946.
Path 265 | total_timesteps 3958.
Path 266 | total_timesteps 3965.
Path 267 | total_timesteps 3981.
Path 268 | total_timesteps 4017.
Path 269 | total_timesteps 4027.
Path 270 | total_timesteps 4036.
Path 271 | total_timesteps 4053.
Path 272 | total_timesteps 4063.
Path 273 | total_timesteps 4079.
Path 274 | total_timesteps 4090.
Path 275 | total_timesteps 4101.
Path 276 | total_timesteps 4114.
Path 277 | total_timesteps 4136.
Path 278 | total_timesteps 4156.
Path 279 | total_timesteps 4167.
Path 280 | total_timesteps 4182.
Path 281 | total_timesteps 4195.
Path 282 | total_timesteps 4212.
Path 283 | total_timesteps 4222.
Path 284 | total_timesteps 4237.
Path 285 | total_timesteps 4248.
Path 286 | total_timesteps 4266.
Path 287 | total_timesteps 4283.
Path 288 | total_timesteps 4297.
Path 289 | total_timesteps 4312.
Path 290 | total_timesteps 4331.
Path 291 | total_timesteps 4344.
Path 292 | total_timesteps 4358.
Path 293 | total_timesteps 4367.
Path 294 | total_timesteps 4377.
Path 295 | total_timesteps 4391.
Path 296 | total_timesteps 4413.
Path 297 | total_timesteps 4432.
Path 298 | total_timesteps 4451.
Path 299 | total_timesteps 4465.
Path 300 | total_timesteps 4476.
Path 301 | total_timesteps 4487.
Path 302 | total_timesteps 4504.
Path 303 | total_timesteps 4518.
Path 304 | total_timesteps 4532.
Path 305 | total_timesteps 4543.
Path 306 | total_timesteps 4555.
Path 307 | total_timesteps 4575.
Path 308 | total_timesteps 4585.
Path 309 | total_timesteps 4604.
Path 310 | total_timesteps 4621.
Path 311 | total_timesteps 4633.
Path 312 | total_timesteps 4646.
Path 313 | total_timesteps 4659.
Path 314 | total_timesteps 4678.
Path 315 | total_timesteps 4698.
Path 316 | total_timesteps 4710.
Path 317 | total_timesteps 4733.
Path 318 | total_timesteps 4750.
Path 319 | total_timesteps 4761.
Path 320 | total_timesteps 4775.
Path 321 | total_timesteps 4784.
Path 322 | total_timesteps 4801.
Path 323 | total_timesteps 4812.
Path 324 | total_timesteps 4836.
Path 325 | total_timesteps 4856.
Path 326 | total_timesteps 4878.
Path 327 | total_timesteps 4893.
Path 328 | total_timesteps 4907.
Path 329 | total_timesteps 4915.
Path 330 | total_timesteps 4935.
Path 331 | total_timesteps 4944.
Path 332 | total_timesteps 4960.
Path 333 | total_timesteps 4971.
Path 334 | total_timesteps 4981.
Path 335 | total_timesteps 4990.
Path 336 | total_timesteps 5003.
Path 337 | total_timesteps 5015.
Path 338 | total_timesteps 5031.
Path 339 | total_timesteps 5045.
Path 340 | total_timesteps 5056.
Path 341 | total_timesteps 5065.
Path 342 | total_timesteps 5081.
Path 343 | total_timesteps 5103.
Path 344 | total_timesteps 5114.
Path 345 | total_timesteps 5123.
Path 346 | total_timesteps 5133.
Path 347 | total_timesteps 5142.
Path 348 | total_timesteps 5156.
Path 349 | total_timesteps 5170.
Path 350 | total_timesteps 5178.
Path 351 | total_timesteps 5188.
Path 352 | total_timesteps 5211.
Path 353 | total_timesteps 5221.
Path 354 | total_timesteps 5228.
Path 355 | total_timesteps 5248.
Path 356 | total_timesteps 5258.
Path 357 | total_timesteps 5271.
Path 358 | total_timesteps 5286.
Path 359 | total_timesteps 5300.
Path 360 | total_timesteps 5309.
Path 361 | total_timesteps 5318.
Path 362 | total_timesteps 5333.
Path 363 | total_timesteps 5348.
Path 364 | total_timesteps 5357.
Path 365 | total_timesteps 5374.
Path 366 | total_timesteps 5390.
Path 367 | total_timesteps 5397.
Path 368 | total_timesteps 5408.
Path 369 | total_timesteps 5430.
Path 370 | total_timesteps 5444.
Path 371 | total_timesteps 5457.
Path 372 | total_timesteps 5468.
Path 373 | total_timesteps 5486.
Path 374 | total_timesteps 5507.
Path 375 | total_timesteps 5518.
Path 376 | total_timesteps 5531.
Path 377 | total_timesteps 5550.
Path 378 | total_timesteps 5565.
Path 379 | total_timesteps 5573.
Path 380 | total_timesteps 5584.
Path 381 | total_timesteps 5593.
Path 382 | total_timesteps 5606.
Path 383 | total_timesteps 5619.
Path 384 | total_timesteps 5635.
Path 385 | total_timesteps 5643.
Path 386 | total_timesteps 5656.
Path 387 | total_timesteps 5686.
Path 388 | total_timesteps 5702.
Path 389 | total_timesteps 5730.
Path 390 | total_timesteps 5742.
Path 391 | total_timesteps 5750.
Path 392 | total_timesteps 5762.
Path 393 | total_timesteps 5772.
Path 394 | total_timesteps 5788.
Path 395 | total_timesteps 5801.
Path 396 | total_timesteps 5818.
Path 397 | total_timesteps 5831.
Path 398 | total_timesteps 5873.
Path 399 | total_timesteps 5883.
Path 400 | total_timesteps 5892.
Path 401 | total_timesteps 5900.
Path 402 | total_timesteps 5917.
Path 403 | total_timesteps 5928.
Path 404 | total_timesteps 5951.
Path 405 | total_timesteps 5962.
Path 406 | total_timesteps 5972.
Path 407 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.4     |
| Iteration     | 23       |
| MaximumReturn | 9.67     |
| MinimumReturn | -21.5    |
| TotalSamples  | 100175   |
----------------------------
itr #24 | 
Fitting dynamics.
Validation loss = 0.006724764127284288
Validation loss = 0.006414057686924934
Validation loss = 0.005938778631389141
Validation loss = 0.0060804360546171665
Validation loss = 0.005832909140735865
Validation loss = 0.006069319788366556
Validation loss = 0.006274182815104723
Validation loss = 0.006290740333497524
Validation loss = 0.006094762589782476
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 38.
Path 3 | total_timesteps 62.
Path 4 | total_timesteps 75.
Path 5 | total_timesteps 90.
Path 6 | total_timesteps 120.
Path 7 | total_timesteps 140.
Path 8 | total_timesteps 154.
Path 9 | total_timesteps 168.
Path 10 | total_timesteps 180.
Path 11 | total_timesteps 206.
Path 12 | total_timesteps 241.
Path 13 | total_timesteps 257.
Path 14 | total_timesteps 266.
Path 15 | total_timesteps 289.
Path 16 | total_timesteps 301.
Path 17 | total_timesteps 322.
Path 18 | total_timesteps 331.
Path 19 | total_timesteps 344.
Path 20 | total_timesteps 356.
Path 21 | total_timesteps 373.
Path 22 | total_timesteps 382.
Path 23 | total_timesteps 390.
Path 24 | total_timesteps 411.
Path 25 | total_timesteps 423.
Path 26 | total_timesteps 439.
Path 27 | total_timesteps 460.
Path 28 | total_timesteps 471.
Path 29 | total_timesteps 492.
Path 30 | total_timesteps 506.
Path 31 | total_timesteps 513.
Path 32 | total_timesteps 524.
Path 33 | total_timesteps 542.
Path 34 | total_timesteps 551.
Path 35 | total_timesteps 570.
Path 36 | total_timesteps 585.
Path 37 | total_timesteps 599.
Path 38 | total_timesteps 611.
Path 39 | total_timesteps 624.
Path 40 | total_timesteps 638.
Path 41 | total_timesteps 657.
Path 42 | total_timesteps 672.
Path 43 | total_timesteps 683.
Path 44 | total_timesteps 702.
Path 45 | total_timesteps 711.
Path 46 | total_timesteps 725.
Path 47 | total_timesteps 744.
Path 48 | total_timesteps 752.
Path 49 | total_timesteps 773.
Path 50 | total_timesteps 784.
Path 51 | total_timesteps 797.
Path 52 | total_timesteps 815.
Path 53 | total_timesteps 828.
Path 54 | total_timesteps 837.
Path 55 | total_timesteps 845.
Path 56 | total_timesteps 872.
Path 57 | total_timesteps 881.
Path 58 | total_timesteps 894.
Path 59 | total_timesteps 908.
Path 60 | total_timesteps 917.
Path 61 | total_timesteps 931.
Path 62 | total_timesteps 943.
Path 63 | total_timesteps 961.
Path 64 | total_timesteps 969.
Path 65 | total_timesteps 986.
Path 66 | total_timesteps 1003.
Path 67 | total_timesteps 1020.
Path 68 | total_timesteps 1047.
Path 69 | total_timesteps 1055.
Path 70 | total_timesteps 1071.
Path 71 | total_timesteps 1086.
Path 72 | total_timesteps 1098.
Path 73 | total_timesteps 1114.
Path 74 | total_timesteps 1132.
Path 75 | total_timesteps 1139.
Path 76 | total_timesteps 1154.
Path 77 | total_timesteps 1165.
Path 78 | total_timesteps 1175.
Path 79 | total_timesteps 1191.
Path 80 | total_timesteps 1209.
Path 81 | total_timesteps 1218.
Path 82 | total_timesteps 1228.
Path 83 | total_timesteps 1244.
Path 84 | total_timesteps 1253.
Path 85 | total_timesteps 1269.
Path 86 | total_timesteps 1291.
Path 87 | total_timesteps 1306.
Path 88 | total_timesteps 1320.
Path 89 | total_timesteps 1341.
Path 90 | total_timesteps 1351.
Path 91 | total_timesteps 1378.
Path 92 | total_timesteps 1388.
Path 93 | total_timesteps 1406.
Path 94 | total_timesteps 1413.
Path 95 | total_timesteps 1426.
Path 96 | total_timesteps 1447.
Path 97 | total_timesteps 1461.
Path 98 | total_timesteps 1473.
Path 99 | total_timesteps 1487.
Path 100 | total_timesteps 1503.
Path 101 | total_timesteps 1523.
Path 102 | total_timesteps 1540.
Path 103 | total_timesteps 1551.
Path 104 | total_timesteps 1566.
Path 105 | total_timesteps 1577.
Path 106 | total_timesteps 1595.
Path 107 | total_timesteps 1619.
Path 108 | total_timesteps 1644.
Path 109 | total_timesteps 1658.
Path 110 | total_timesteps 1670.
Path 111 | total_timesteps 1681.
Path 112 | total_timesteps 1698.
Path 113 | total_timesteps 1711.
Path 114 | total_timesteps 1724.
Path 115 | total_timesteps 1744.
Path 116 | total_timesteps 1754.
Path 117 | total_timesteps 1775.
Path 118 | total_timesteps 1785.
Path 119 | total_timesteps 1805.
Path 120 | total_timesteps 1831.
Path 121 | total_timesteps 1842.
Path 122 | total_timesteps 1860.
Path 123 | total_timesteps 1873.
Path 124 | total_timesteps 1891.
Path 125 | total_timesteps 1909.
Path 126 | total_timesteps 1923.
Path 127 | total_timesteps 1946.
Path 128 | total_timesteps 1955.
Path 129 | total_timesteps 1966.
Path 130 | total_timesteps 1980.
Path 131 | total_timesteps 2001.
Path 132 | total_timesteps 2010.
Path 133 | total_timesteps 2021.
Path 134 | total_timesteps 2032.
Path 135 | total_timesteps 2053.
Path 136 | total_timesteps 2060.
Path 137 | total_timesteps 2076.
Path 138 | total_timesteps 2091.
Path 139 | total_timesteps 2105.
Path 140 | total_timesteps 2137.
Path 141 | total_timesteps 2146.
Path 142 | total_timesteps 2166.
Path 143 | total_timesteps 2182.
Path 144 | total_timesteps 2192.
Path 145 | total_timesteps 2202.
Path 146 | total_timesteps 2215.
Path 147 | total_timesteps 2234.
Path 148 | total_timesteps 2250.
Path 149 | total_timesteps 2262.
Path 150 | total_timesteps 2271.
Path 151 | total_timesteps 2286.
Path 152 | total_timesteps 2297.
Path 153 | total_timesteps 2314.
Path 154 | total_timesteps 2322.
Path 155 | total_timesteps 2344.
Path 156 | total_timesteps 2356.
Path 157 | total_timesteps 2364.
Path 158 | total_timesteps 2380.
Path 159 | total_timesteps 2393.
Path 160 | total_timesteps 2406.
Path 161 | total_timesteps 2426.
Path 162 | total_timesteps 2438.
Path 163 | total_timesteps 2454.
Path 164 | total_timesteps 2468.
Path 165 | total_timesteps 2482.
Path 166 | total_timesteps 2493.
Path 167 | total_timesteps 2506.
Path 168 | total_timesteps 2524.
Path 169 | total_timesteps 2539.
Path 170 | total_timesteps 2560.
Path 171 | total_timesteps 2569.
Path 172 | total_timesteps 2593.
Path 173 | total_timesteps 2604.
Path 174 | total_timesteps 2617.
Path 175 | total_timesteps 2648.
Path 176 | total_timesteps 2663.
Path 177 | total_timesteps 2676.
Path 178 | total_timesteps 2685.
Path 179 | total_timesteps 2698.
Path 180 | total_timesteps 2713.
Path 181 | total_timesteps 2728.
Path 182 | total_timesteps 2744.
Path 183 | total_timesteps 2753.
Path 184 | total_timesteps 2767.
Path 185 | total_timesteps 2778.
Path 186 | total_timesteps 2793.
Path 187 | total_timesteps 2809.
Path 188 | total_timesteps 2817.
Path 189 | total_timesteps 2830.
Path 190 | total_timesteps 2847.
Path 191 | total_timesteps 2857.
Path 192 | total_timesteps 2867.
Path 193 | total_timesteps 2889.
Path 194 | total_timesteps 2907.
Path 195 | total_timesteps 2914.
Path 196 | total_timesteps 2927.
Path 197 | total_timesteps 2937.
Path 198 | total_timesteps 2944.
Path 199 | total_timesteps 2955.
Path 200 | total_timesteps 2975.
Path 201 | total_timesteps 2992.
Path 202 | total_timesteps 3002.
Path 203 | total_timesteps 3010.
Path 204 | total_timesteps 3023.
Path 205 | total_timesteps 3044.
Path 206 | total_timesteps 3052.
Path 207 | total_timesteps 3061.
Path 208 | total_timesteps 3073.
Path 209 | total_timesteps 3094.
Path 210 | total_timesteps 3103.
Path 211 | total_timesteps 3123.
Path 212 | total_timesteps 3142.
Path 213 | total_timesteps 3153.
Path 214 | total_timesteps 3164.
Path 215 | total_timesteps 3176.
Path 216 | total_timesteps 3188.
Path 217 | total_timesteps 3197.
Path 218 | total_timesteps 3208.
Path 219 | total_timesteps 3229.
Path 220 | total_timesteps 3242.
Path 221 | total_timesteps 3259.
Path 222 | total_timesteps 3267.
Path 223 | total_timesteps 3286.
Path 224 | total_timesteps 3300.
Path 225 | total_timesteps 3312.
Path 226 | total_timesteps 3327.
Path 227 | total_timesteps 3350.
Path 228 | total_timesteps 3368.
Path 229 | total_timesteps 3389.
Path 230 | total_timesteps 3401.
Path 231 | total_timesteps 3413.
Path 232 | total_timesteps 3428.
Path 233 | total_timesteps 3439.
Path 234 | total_timesteps 3448.
Path 235 | total_timesteps 3461.
Path 236 | total_timesteps 3470.
Path 237 | total_timesteps 3486.
Path 238 | total_timesteps 3498.
Path 239 | total_timesteps 3506.
Path 240 | total_timesteps 3526.
Path 241 | total_timesteps 3540.
Path 242 | total_timesteps 3561.
Path 243 | total_timesteps 3588.
Path 244 | total_timesteps 3600.
Path 245 | total_timesteps 3610.
Path 246 | total_timesteps 3623.
Path 247 | total_timesteps 3650.
Path 248 | total_timesteps 3667.
Path 249 | total_timesteps 3688.
Path 250 | total_timesteps 3698.
Path 251 | total_timesteps 3733.
Path 252 | total_timesteps 3747.
Path 253 | total_timesteps 3775.
Path 254 | total_timesteps 3793.
Path 255 | total_timesteps 3803.
Path 256 | total_timesteps 3815.
Path 257 | total_timesteps 3830.
Path 258 | total_timesteps 3847.
Path 259 | total_timesteps 3854.
Path 260 | total_timesteps 3863.
Path 261 | total_timesteps 3872.
Path 262 | total_timesteps 3880.
Path 263 | total_timesteps 3895.
Path 264 | total_timesteps 3915.
Path 265 | total_timesteps 3928.
Path 266 | total_timesteps 3936.
Path 267 | total_timesteps 3947.
Path 268 | total_timesteps 3970.
Path 269 | total_timesteps 3980.
Path 270 | total_timesteps 3990.
Path 271 | total_timesteps 3998.
Path 272 | total_timesteps 4028.
Path 273 | total_timesteps 4039.
Path 274 | total_timesteps 4061.
Path 275 | total_timesteps 4070.
Path 276 | total_timesteps 4086.
Path 277 | total_timesteps 4094.
Path 278 | total_timesteps 4117.
Path 279 | total_timesteps 4130.
Path 280 | total_timesteps 4145.
Path 281 | total_timesteps 4161.
Path 282 | total_timesteps 4175.
Path 283 | total_timesteps 4186.
Path 284 | total_timesteps 4193.
Path 285 | total_timesteps 4212.
Path 286 | total_timesteps 4224.
Path 287 | total_timesteps 4233.
Path 288 | total_timesteps 4254.
Path 289 | total_timesteps 4263.
Path 290 | total_timesteps 4278.
Path 291 | total_timesteps 4290.
Path 292 | total_timesteps 4310.
Path 293 | total_timesteps 4325.
Path 294 | total_timesteps 4339.
Path 295 | total_timesteps 4354.
Path 296 | total_timesteps 4369.
Path 297 | total_timesteps 4380.
Path 298 | total_timesteps 4395.
Path 299 | total_timesteps 4428.
Path 300 | total_timesteps 4457.
Path 301 | total_timesteps 4477.
Path 302 | total_timesteps 4489.
Path 303 | total_timesteps 4505.
Path 304 | total_timesteps 4515.
Path 305 | total_timesteps 4541.
Path 306 | total_timesteps 4557.
Path 307 | total_timesteps 4570.
Path 308 | total_timesteps 4586.
Path 309 | total_timesteps 4595.
Path 310 | total_timesteps 4608.
Path 311 | total_timesteps 4631.
Path 312 | total_timesteps 4649.
Path 313 | total_timesteps 4658.
Path 314 | total_timesteps 4673.
Path 315 | total_timesteps 4690.
Path 316 | total_timesteps 4700.
Path 317 | total_timesteps 4708.
Path 318 | total_timesteps 4716.
Path 319 | total_timesteps 4737.
Path 320 | total_timesteps 4758.
Path 321 | total_timesteps 4772.
Path 322 | total_timesteps 4786.
Path 323 | total_timesteps 4795.
Path 324 | total_timesteps 4812.
Path 325 | total_timesteps 4828.
Path 326 | total_timesteps 4843.
Path 327 | total_timesteps 4858.
Path 328 | total_timesteps 4876.
Path 329 | total_timesteps 4894.
Path 330 | total_timesteps 4919.
Path 331 | total_timesteps 4932.
Path 332 | total_timesteps 4948.
Path 333 | total_timesteps 4962.
Path 334 | total_timesteps 4973.
Path 335 | total_timesteps 4987.
Path 336 | total_timesteps 4997.
Path 337 | total_timesteps 5011.
Path 338 | total_timesteps 5024.
Path 339 | total_timesteps 5041.
Path 340 | total_timesteps 5050.
Path 341 | total_timesteps 5062.
Path 342 | total_timesteps 5077.
Path 343 | total_timesteps 5098.
Path 344 | total_timesteps 5106.
Path 345 | total_timesteps 5116.
Path 346 | total_timesteps 5127.
Path 347 | total_timesteps 5142.
Path 348 | total_timesteps 5164.
Path 349 | total_timesteps 5172.
Path 350 | total_timesteps 5180.
Path 351 | total_timesteps 5197.
Path 352 | total_timesteps 5211.
Path 353 | total_timesteps 5233.
Path 354 | total_timesteps 5241.
Path 355 | total_timesteps 5251.
Path 356 | total_timesteps 5279.
Path 357 | total_timesteps 5296.
Path 358 | total_timesteps 5305.
Path 359 | total_timesteps 5314.
Path 360 | total_timesteps 5323.
Path 361 | total_timesteps 5341.
Path 362 | total_timesteps 5353.
Path 363 | total_timesteps 5367.
Path 364 | total_timesteps 5375.
Path 365 | total_timesteps 5385.
Path 366 | total_timesteps 5397.
Path 367 | total_timesteps 5412.
Path 368 | total_timesteps 5421.
Path 369 | total_timesteps 5434.
Path 370 | total_timesteps 5453.
Path 371 | total_timesteps 5475.
Path 372 | total_timesteps 5490.
Path 373 | total_timesteps 5529.
Path 374 | total_timesteps 5538.
Path 375 | total_timesteps 5551.
Path 376 | total_timesteps 5560.
Path 377 | total_timesteps 5568.
Path 378 | total_timesteps 5580.
Path 379 | total_timesteps 5594.
Path 380 | total_timesteps 5606.
Path 381 | total_timesteps 5625.
Path 382 | total_timesteps 5656.
Path 383 | total_timesteps 5677.
Path 384 | total_timesteps 5686.
Path 385 | total_timesteps 5701.
Path 386 | total_timesteps 5717.
Path 387 | total_timesteps 5727.
Path 388 | total_timesteps 5744.
Path 389 | total_timesteps 5762.
Path 390 | total_timesteps 5775.
Path 391 | total_timesteps 5788.
Path 392 | total_timesteps 5813.
Path 393 | total_timesteps 5830.
Path 394 | total_timesteps 5847.
Path 395 | total_timesteps 5861.
Path 396 | total_timesteps 5877.
Path 397 | total_timesteps 5892.
Path 398 | total_timesteps 5906.
Path 399 | total_timesteps 5917.
Path 400 | total_timesteps 5930.
Path 401 | total_timesteps 5944.
Path 402 | total_timesteps 5953.
Path 403 | total_timesteps 5976.
Path 404 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.52    |
| Iteration     | 24       |
| MaximumReturn | 11.7     |
| MinimumReturn | -20.3    |
| TotalSamples  | 104181   |
----------------------------
itr #25 | 
Fitting dynamics.
Validation loss = 0.005678889807313681
Validation loss = 0.006537482142448425
Validation loss = 0.0057601663284003735
Validation loss = 0.006058935541659594
Validation loss = 0.005781126674264669
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 29.
Path 2 | total_timesteps 43.
Path 3 | total_timesteps 52.
Path 4 | total_timesteps 62.
Path 5 | total_timesteps 77.
Path 6 | total_timesteps 96.
Path 7 | total_timesteps 110.
Path 8 | total_timesteps 123.
Path 9 | total_timesteps 136.
Path 10 | total_timesteps 150.
Path 11 | total_timesteps 159.
Path 12 | total_timesteps 182.
Path 13 | total_timesteps 207.
Path 14 | total_timesteps 215.
Path 15 | total_timesteps 228.
Path 16 | total_timesteps 239.
Path 17 | total_timesteps 257.
Path 18 | total_timesteps 268.
Path 19 | total_timesteps 282.
Path 20 | total_timesteps 291.
Path 21 | total_timesteps 302.
Path 22 | total_timesteps 320.
Path 23 | total_timesteps 328.
Path 24 | total_timesteps 347.
Path 25 | total_timesteps 360.
Path 26 | total_timesteps 379.
Path 27 | total_timesteps 396.
Path 28 | total_timesteps 406.
Path 29 | total_timesteps 426.
Path 30 | total_timesteps 444.
Path 31 | total_timesteps 461.
Path 32 | total_timesteps 474.
Path 33 | total_timesteps 488.
Path 34 | total_timesteps 497.
Path 35 | total_timesteps 515.
Path 36 | total_timesteps 526.
Path 37 | total_timesteps 541.
Path 38 | total_timesteps 551.
Path 39 | total_timesteps 562.
Path 40 | total_timesteps 574.
Path 41 | total_timesteps 583.
Path 42 | total_timesteps 600.
Path 43 | total_timesteps 615.
Path 44 | total_timesteps 626.
Path 45 | total_timesteps 639.
Path 46 | total_timesteps 653.
Path 47 | total_timesteps 673.
Path 48 | total_timesteps 687.
Path 49 | total_timesteps 707.
Path 50 | total_timesteps 715.
Path 51 | total_timesteps 728.
Path 52 | total_timesteps 741.
Path 53 | total_timesteps 755.
Path 54 | total_timesteps 771.
Path 55 | total_timesteps 782.
Path 56 | total_timesteps 819.
Path 57 | total_timesteps 834.
Path 58 | total_timesteps 851.
Path 59 | total_timesteps 859.
Path 60 | total_timesteps 877.
Path 61 | total_timesteps 892.
Path 62 | total_timesteps 913.
Path 63 | total_timesteps 926.
Path 64 | total_timesteps 941.
Path 65 | total_timesteps 956.
Path 66 | total_timesteps 972.
Path 67 | total_timesteps 983.
Path 68 | total_timesteps 1000.
Path 69 | total_timesteps 1022.
Path 70 | total_timesteps 1036.
Path 71 | total_timesteps 1045.
Path 72 | total_timesteps 1054.
Path 73 | total_timesteps 1065.
Path 74 | total_timesteps 1078.
Path 75 | total_timesteps 1092.
Path 76 | total_timesteps 1106.
Path 77 | total_timesteps 1118.
Path 78 | total_timesteps 1126.
Path 79 | total_timesteps 1142.
Path 80 | total_timesteps 1154.
Path 81 | total_timesteps 1173.
Path 82 | total_timesteps 1185.
Path 83 | total_timesteps 1198.
Path 84 | total_timesteps 1206.
Path 85 | total_timesteps 1220.
Path 86 | total_timesteps 1231.
Path 87 | total_timesteps 1246.
Path 88 | total_timesteps 1257.
Path 89 | total_timesteps 1269.
Path 90 | total_timesteps 1279.
Path 91 | total_timesteps 1286.
Path 92 | total_timesteps 1316.
Path 93 | total_timesteps 1330.
Path 94 | total_timesteps 1343.
Path 95 | total_timesteps 1359.
Path 96 | total_timesteps 1376.
Path 97 | total_timesteps 1393.
Path 98 | total_timesteps 1413.
Path 99 | total_timesteps 1427.
Path 100 | total_timesteps 1443.
Path 101 | total_timesteps 1466.
Path 102 | total_timesteps 1476.
Path 103 | total_timesteps 1493.
Path 104 | total_timesteps 1501.
Path 105 | total_timesteps 1519.
Path 106 | total_timesteps 1534.
Path 107 | total_timesteps 1544.
Path 108 | total_timesteps 1557.
Path 109 | total_timesteps 1569.
Path 110 | total_timesteps 1583.
Path 111 | total_timesteps 1600.
Path 112 | total_timesteps 1608.
Path 113 | total_timesteps 1621.
Path 114 | total_timesteps 1628.
Path 115 | total_timesteps 1642.
Path 116 | total_timesteps 1659.
Path 117 | total_timesteps 1672.
Path 118 | total_timesteps 1681.
Path 119 | total_timesteps 1694.
Path 120 | total_timesteps 1709.
Path 121 | total_timesteps 1722.
Path 122 | total_timesteps 1734.
Path 123 | total_timesteps 1754.
Path 124 | total_timesteps 1767.
Path 125 | total_timesteps 1775.
Path 126 | total_timesteps 1789.
Path 127 | total_timesteps 1802.
Path 128 | total_timesteps 1817.
Path 129 | total_timesteps 1830.
Path 130 | total_timesteps 1844.
Path 131 | total_timesteps 1854.
Path 132 | total_timesteps 1877.
Path 133 | total_timesteps 1889.
Path 134 | total_timesteps 1909.
Path 135 | total_timesteps 1919.
Path 136 | total_timesteps 1930.
Path 137 | total_timesteps 1940.
Path 138 | total_timesteps 1952.
Path 139 | total_timesteps 1964.
Path 140 | total_timesteps 1975.
Path 141 | total_timesteps 1984.
Path 142 | total_timesteps 1994.
Path 143 | total_timesteps 2007.
Path 144 | total_timesteps 2017.
Path 145 | total_timesteps 2026.
Path 146 | total_timesteps 2041.
Path 147 | total_timesteps 2055.
Path 148 | total_timesteps 2062.
Path 149 | total_timesteps 2073.
Path 150 | total_timesteps 2085.
Path 151 | total_timesteps 2108.
Path 152 | total_timesteps 2119.
Path 153 | total_timesteps 2136.
Path 154 | total_timesteps 2163.
Path 155 | total_timesteps 2174.
Path 156 | total_timesteps 2188.
Path 157 | total_timesteps 2208.
Path 158 | total_timesteps 2225.
Path 159 | total_timesteps 2246.
Path 160 | total_timesteps 2259.
Path 161 | total_timesteps 2275.
Path 162 | total_timesteps 2294.
Path 163 | total_timesteps 2305.
Path 164 | total_timesteps 2320.
Path 165 | total_timesteps 2331.
Path 166 | total_timesteps 2348.
Path 167 | total_timesteps 2361.
Path 168 | total_timesteps 2371.
Path 169 | total_timesteps 2386.
Path 170 | total_timesteps 2406.
Path 171 | total_timesteps 2420.
Path 172 | total_timesteps 2430.
Path 173 | total_timesteps 2443.
Path 174 | total_timesteps 2456.
Path 175 | total_timesteps 2468.
Path 176 | total_timesteps 2480.
Path 177 | total_timesteps 2497.
Path 178 | total_timesteps 2506.
Path 179 | total_timesteps 2522.
Path 180 | total_timesteps 2541.
Path 181 | total_timesteps 2553.
Path 182 | total_timesteps 2568.
Path 183 | total_timesteps 2579.
Path 184 | total_timesteps 2592.
Path 185 | total_timesteps 2605.
Path 186 | total_timesteps 2617.
Path 187 | total_timesteps 2633.
Path 188 | total_timesteps 2648.
Path 189 | total_timesteps 2667.
Path 190 | total_timesteps 2689.
Path 191 | total_timesteps 2702.
Path 192 | total_timesteps 2718.
Path 193 | total_timesteps 2741.
Path 194 | total_timesteps 2756.
Path 195 | total_timesteps 2770.
Path 196 | total_timesteps 2778.
Path 197 | total_timesteps 2789.
Path 198 | total_timesteps 2812.
Path 199 | total_timesteps 2827.
Path 200 | total_timesteps 2844.
Path 201 | total_timesteps 2855.
Path 202 | total_timesteps 2876.
Path 203 | total_timesteps 2892.
Path 204 | total_timesteps 2908.
Path 205 | total_timesteps 2922.
Path 206 | total_timesteps 2931.
Path 207 | total_timesteps 2938.
Path 208 | total_timesteps 2948.
Path 209 | total_timesteps 2963.
Path 210 | total_timesteps 2976.
Path 211 | total_timesteps 2996.
Path 212 | total_timesteps 3009.
Path 213 | total_timesteps 3027.
Path 214 | total_timesteps 3046.
Path 215 | total_timesteps 3067.
Path 216 | total_timesteps 3082.
Path 217 | total_timesteps 3102.
Path 218 | total_timesteps 3116.
Path 219 | total_timesteps 3131.
Path 220 | total_timesteps 3147.
Path 221 | total_timesteps 3163.
Path 222 | total_timesteps 3183.
Path 223 | total_timesteps 3200.
Path 224 | total_timesteps 3214.
Path 225 | total_timesteps 3232.
Path 226 | total_timesteps 3245.
Path 227 | total_timesteps 3260.
Path 228 | total_timesteps 3272.
Path 229 | total_timesteps 3283.
Path 230 | total_timesteps 3291.
Path 231 | total_timesteps 3316.
Path 232 | total_timesteps 3332.
Path 233 | total_timesteps 3350.
Path 234 | total_timesteps 3358.
Path 235 | total_timesteps 3371.
Path 236 | total_timesteps 3382.
Path 237 | total_timesteps 3394.
Path 238 | total_timesteps 3419.
Path 239 | total_timesteps 3428.
Path 240 | total_timesteps 3441.
Path 241 | total_timesteps 3455.
Path 242 | total_timesteps 3470.
Path 243 | total_timesteps 3490.
Path 244 | total_timesteps 3503.
Path 245 | total_timesteps 3520.
Path 246 | total_timesteps 3535.
Path 247 | total_timesteps 3549.
Path 248 | total_timesteps 3556.
Path 249 | total_timesteps 3571.
Path 250 | total_timesteps 3582.
Path 251 | total_timesteps 3596.
Path 252 | total_timesteps 3611.
Path 253 | total_timesteps 3622.
Path 254 | total_timesteps 3638.
Path 255 | total_timesteps 3655.
Path 256 | total_timesteps 3672.
Path 257 | total_timesteps 3681.
Path 258 | total_timesteps 3692.
Path 259 | total_timesteps 3704.
Path 260 | total_timesteps 3711.
Path 261 | total_timesteps 3722.
Path 262 | total_timesteps 3735.
Path 263 | total_timesteps 3747.
Path 264 | total_timesteps 3766.
Path 265 | total_timesteps 3783.
Path 266 | total_timesteps 3796.
Path 267 | total_timesteps 3812.
Path 268 | total_timesteps 3823.
Path 269 | total_timesteps 3836.
Path 270 | total_timesteps 3855.
Path 271 | total_timesteps 3872.
Path 272 | total_timesteps 3894.
Path 273 | total_timesteps 3910.
Path 274 | total_timesteps 3924.
Path 275 | total_timesteps 3934.
Path 276 | total_timesteps 3945.
Path 277 | total_timesteps 3954.
Path 278 | total_timesteps 3978.
Path 279 | total_timesteps 3991.
Path 280 | total_timesteps 3998.
Path 281 | total_timesteps 4013.
Path 282 | total_timesteps 4038.
Path 283 | total_timesteps 4060.
Path 284 | total_timesteps 4079.
Path 285 | total_timesteps 4090.
Path 286 | total_timesteps 4111.
Path 287 | total_timesteps 4123.
Path 288 | total_timesteps 4139.
Path 289 | total_timesteps 4155.
Path 290 | total_timesteps 4175.
Path 291 | total_timesteps 4189.
Path 292 | total_timesteps 4204.
Path 293 | total_timesteps 4215.
Path 294 | total_timesteps 4240.
Path 295 | total_timesteps 4253.
Path 296 | total_timesteps 4269.
Path 297 | total_timesteps 4287.
Path 298 | total_timesteps 4303.
Path 299 | total_timesteps 4312.
Path 300 | total_timesteps 4323.
Path 301 | total_timesteps 4340.
Path 302 | total_timesteps 4358.
Path 303 | total_timesteps 4370.
Path 304 | total_timesteps 4380.
Path 305 | total_timesteps 4399.
Path 306 | total_timesteps 4412.
Path 307 | total_timesteps 4431.
Path 308 | total_timesteps 4438.
Path 309 | total_timesteps 4449.
Path 310 | total_timesteps 4463.
Path 311 | total_timesteps 4485.
Path 312 | total_timesteps 4504.
Path 313 | total_timesteps 4522.
Path 314 | total_timesteps 4532.
Path 315 | total_timesteps 4550.
Path 316 | total_timesteps 4563.
Path 317 | total_timesteps 4572.
Path 318 | total_timesteps 4584.
Path 319 | total_timesteps 4595.
Path 320 | total_timesteps 4611.
Path 321 | total_timesteps 4627.
Path 322 | total_timesteps 4637.
Path 323 | total_timesteps 4668.
Path 324 | total_timesteps 4682.
Path 325 | total_timesteps 4695.
Path 326 | total_timesteps 4703.
Path 327 | total_timesteps 4714.
Path 328 | total_timesteps 4746.
Path 329 | total_timesteps 4756.
Path 330 | total_timesteps 4766.
Path 331 | total_timesteps 4782.
Path 332 | total_timesteps 4797.
Path 333 | total_timesteps 4807.
Path 334 | total_timesteps 4821.
Path 335 | total_timesteps 4835.
Path 336 | total_timesteps 4854.
Path 337 | total_timesteps 4873.
Path 338 | total_timesteps 4883.
Path 339 | total_timesteps 4901.
Path 340 | total_timesteps 4911.
Path 341 | total_timesteps 4924.
Path 342 | total_timesteps 4946.
Path 343 | total_timesteps 4957.
Path 344 | total_timesteps 4972.
Path 345 | total_timesteps 4988.
Path 346 | total_timesteps 4998.
Path 347 | total_timesteps 5014.
Path 348 | total_timesteps 5025.
Path 349 | total_timesteps 5046.
Path 350 | total_timesteps 5056.
Path 351 | total_timesteps 5072.
Path 352 | total_timesteps 5085.
Path 353 | total_timesteps 5100.
Path 354 | total_timesteps 5110.
Path 355 | total_timesteps 5135.
Path 356 | total_timesteps 5148.
Path 357 | total_timesteps 5155.
Path 358 | total_timesteps 5166.
Path 359 | total_timesteps 5182.
Path 360 | total_timesteps 5198.
Path 361 | total_timesteps 5213.
Path 362 | total_timesteps 5226.
Path 363 | total_timesteps 5246.
Path 364 | total_timesteps 5262.
Path 365 | total_timesteps 5281.
Path 366 | total_timesteps 5301.
Path 367 | total_timesteps 5318.
Path 368 | total_timesteps 5335.
Path 369 | total_timesteps 5347.
Path 370 | total_timesteps 5356.
Path 371 | total_timesteps 5367.
Path 372 | total_timesteps 5374.
Path 373 | total_timesteps 5393.
Path 374 | total_timesteps 5409.
Path 375 | total_timesteps 5427.
Path 376 | total_timesteps 5441.
Path 377 | total_timesteps 5449.
Path 378 | total_timesteps 5463.
Path 379 | total_timesteps 5481.
Path 380 | total_timesteps 5495.
Path 381 | total_timesteps 5521.
Path 382 | total_timesteps 5534.
Path 383 | total_timesteps 5543.
Path 384 | total_timesteps 5554.
Path 385 | total_timesteps 5572.
Path 386 | total_timesteps 5586.
Path 387 | total_timesteps 5596.
Path 388 | total_timesteps 5607.
Path 389 | total_timesteps 5633.
Path 390 | total_timesteps 5657.
Path 391 | total_timesteps 5668.
Path 392 | total_timesteps 5677.
Path 393 | total_timesteps 5688.
Path 394 | total_timesteps 5701.
Path 395 | total_timesteps 5710.
Path 396 | total_timesteps 5725.
Path 397 | total_timesteps 5740.
Path 398 | total_timesteps 5755.
Path 399 | total_timesteps 5774.
Path 400 | total_timesteps 5786.
Path 401 | total_timesteps 5801.
Path 402 | total_timesteps 5813.
Path 403 | total_timesteps 5827.
Path 404 | total_timesteps 5837.
Path 405 | total_timesteps 5853.
Path 406 | total_timesteps 5878.
Path 407 | total_timesteps 5887.
Path 408 | total_timesteps 5898.
Path 409 | total_timesteps 5908.
Path 410 | total_timesteps 5922.
Path 411 | total_timesteps 5931.
Path 412 | total_timesteps 5945.
Path 413 | total_timesteps 5955.
Path 414 | total_timesteps 5967.
Path 415 | total_timesteps 5980.
Path 416 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.55    |
| Iteration     | 25       |
| MaximumReturn | 1.34     |
| MinimumReturn | -22.8    |
| TotalSamples  | 108190   |
----------------------------
itr #26 | 
Fitting dynamics.
Validation loss = 0.005708097945898771
Validation loss = 0.005799903534352779
Validation loss = 0.006006104405969381
Validation loss = 0.006086310371756554
Validation loss = 0.005958860740065575
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 20.
Path 2 | total_timesteps 35.
Path 3 | total_timesteps 54.
Path 4 | total_timesteps 74.
Path 5 | total_timesteps 88.
Path 6 | total_timesteps 98.
Path 7 | total_timesteps 106.
Path 8 | total_timesteps 116.
Path 9 | total_timesteps 130.
Path 10 | total_timesteps 153.
Path 11 | total_timesteps 162.
Path 12 | total_timesteps 170.
Path 13 | total_timesteps 185.
Path 14 | total_timesteps 198.
Path 15 | total_timesteps 205.
Path 16 | total_timesteps 216.
Path 17 | total_timesteps 230.
Path 18 | total_timesteps 240.
Path 19 | total_timesteps 249.
Path 20 | total_timesteps 262.
Path 21 | total_timesteps 272.
Path 22 | total_timesteps 285.
Path 23 | total_timesteps 300.
Path 24 | total_timesteps 318.
Path 25 | total_timesteps 338.
Path 26 | total_timesteps 345.
Path 27 | total_timesteps 357.
Path 28 | total_timesteps 364.
Path 29 | total_timesteps 384.
Path 30 | total_timesteps 402.
Path 31 | total_timesteps 414.
Path 32 | total_timesteps 428.
Path 33 | total_timesteps 451.
Path 34 | total_timesteps 463.
Path 35 | total_timesteps 474.
Path 36 | total_timesteps 487.
Path 37 | total_timesteps 496.
Path 38 | total_timesteps 510.
Path 39 | total_timesteps 520.
Path 40 | total_timesteps 542.
Path 41 | total_timesteps 554.
Path 42 | total_timesteps 567.
Path 43 | total_timesteps 585.
Path 44 | total_timesteps 599.
Path 45 | total_timesteps 609.
Path 46 | total_timesteps 619.
Path 47 | total_timesteps 634.
Path 48 | total_timesteps 650.
Path 49 | total_timesteps 662.
Path 50 | total_timesteps 680.
Path 51 | total_timesteps 688.
Path 52 | total_timesteps 695.
Path 53 | total_timesteps 710.
Path 54 | total_timesteps 733.
Path 55 | total_timesteps 746.
Path 56 | total_timesteps 767.
Path 57 | total_timesteps 786.
Path 58 | total_timesteps 796.
Path 59 | total_timesteps 804.
Path 60 | total_timesteps 814.
Path 61 | total_timesteps 831.
Path 62 | total_timesteps 847.
Path 63 | total_timesteps 864.
Path 64 | total_timesteps 877.
Path 65 | total_timesteps 892.
Path 66 | total_timesteps 900.
Path 67 | total_timesteps 913.
Path 68 | total_timesteps 925.
Path 69 | total_timesteps 933.
Path 70 | total_timesteps 944.
Path 71 | total_timesteps 962.
Path 72 | total_timesteps 974.
Path 73 | total_timesteps 1002.
Path 74 | total_timesteps 1022.
Path 75 | total_timesteps 1040.
Path 76 | total_timesteps 1060.
Path 77 | total_timesteps 1078.
Path 78 | total_timesteps 1087.
Path 79 | total_timesteps 1102.
Path 80 | total_timesteps 1110.
Path 81 | total_timesteps 1120.
Path 82 | total_timesteps 1129.
Path 83 | total_timesteps 1143.
Path 84 | total_timesteps 1158.
Path 85 | total_timesteps 1167.
Path 86 | total_timesteps 1192.
Path 87 | total_timesteps 1208.
Path 88 | total_timesteps 1234.
Path 89 | total_timesteps 1243.
Path 90 | total_timesteps 1254.
Path 91 | total_timesteps 1261.
Path 92 | total_timesteps 1280.
Path 93 | total_timesteps 1288.
Path 94 | total_timesteps 1303.
Path 95 | total_timesteps 1311.
Path 96 | total_timesteps 1327.
Path 97 | total_timesteps 1343.
Path 98 | total_timesteps 1355.
Path 99 | total_timesteps 1361.
Path 100 | total_timesteps 1367.
Path 101 | total_timesteps 1395.
Path 102 | total_timesteps 1404.
Path 103 | total_timesteps 1414.
Path 104 | total_timesteps 1427.
Path 105 | total_timesteps 1445.
Path 106 | total_timesteps 1466.
Path 107 | total_timesteps 1478.
Path 108 | total_timesteps 1493.
Path 109 | total_timesteps 1516.
Path 110 | total_timesteps 1527.
Path 111 | total_timesteps 1540.
Path 112 | total_timesteps 1554.
Path 113 | total_timesteps 1568.
Path 114 | total_timesteps 1581.
Path 115 | total_timesteps 1592.
Path 116 | total_timesteps 1600.
Path 117 | total_timesteps 1611.
Path 118 | total_timesteps 1627.
Path 119 | total_timesteps 1640.
Path 120 | total_timesteps 1648.
Path 121 | total_timesteps 1662.
Path 122 | total_timesteps 1680.
Path 123 | total_timesteps 1703.
Path 124 | total_timesteps 1717.
Path 125 | total_timesteps 1738.
Path 126 | total_timesteps 1754.
Path 127 | total_timesteps 1771.
Path 128 | total_timesteps 1780.
Path 129 | total_timesteps 1787.
Path 130 | total_timesteps 1816.
Path 131 | total_timesteps 1829.
Path 132 | total_timesteps 1839.
Path 133 | total_timesteps 1854.
Path 134 | total_timesteps 1866.
Path 135 | total_timesteps 1875.
Path 136 | total_timesteps 1888.
Path 137 | total_timesteps 1904.
Path 138 | total_timesteps 1915.
Path 139 | total_timesteps 1935.
Path 140 | total_timesteps 1947.
Path 141 | total_timesteps 1960.
Path 142 | total_timesteps 1975.
Path 143 | total_timesteps 1994.
Path 144 | total_timesteps 2004.
Path 145 | total_timesteps 2031.
Path 146 | total_timesteps 2044.
Path 147 | total_timesteps 2054.
Path 148 | total_timesteps 2069.
Path 149 | total_timesteps 2086.
Path 150 | total_timesteps 2106.
Path 151 | total_timesteps 2117.
Path 152 | total_timesteps 2129.
Path 153 | total_timesteps 2139.
Path 154 | total_timesteps 2147.
Path 155 | total_timesteps 2173.
Path 156 | total_timesteps 2186.
Path 157 | total_timesteps 2200.
Path 158 | total_timesteps 2212.
Path 159 | total_timesteps 2224.
Path 160 | total_timesteps 2236.
Path 161 | total_timesteps 2248.
Path 162 | total_timesteps 2257.
Path 163 | total_timesteps 2269.
Path 164 | total_timesteps 2281.
Path 165 | total_timesteps 2294.
Path 166 | total_timesteps 2311.
Path 167 | total_timesteps 2325.
Path 168 | total_timesteps 2334.
Path 169 | total_timesteps 2347.
Path 170 | total_timesteps 2358.
Path 171 | total_timesteps 2373.
Path 172 | total_timesteps 2384.
Path 173 | total_timesteps 2397.
Path 174 | total_timesteps 2417.
Path 175 | total_timesteps 2432.
Path 176 | total_timesteps 2449.
Path 177 | total_timesteps 2461.
Path 178 | total_timesteps 2478.
Path 179 | total_timesteps 2493.
Path 180 | total_timesteps 2501.
Path 181 | total_timesteps 2526.
Path 182 | total_timesteps 2538.
Path 183 | total_timesteps 2552.
Path 184 | total_timesteps 2561.
Path 185 | total_timesteps 2570.
Path 186 | total_timesteps 2582.
Path 187 | total_timesteps 2598.
Path 188 | total_timesteps 2613.
Path 189 | total_timesteps 2630.
Path 190 | total_timesteps 2639.
Path 191 | total_timesteps 2650.
Path 192 | total_timesteps 2671.
Path 193 | total_timesteps 2680.
Path 194 | total_timesteps 2690.
Path 195 | total_timesteps 2710.
Path 196 | total_timesteps 2730.
Path 197 | total_timesteps 2742.
Path 198 | total_timesteps 2760.
Path 199 | total_timesteps 2774.
Path 200 | total_timesteps 2789.
Path 201 | total_timesteps 2800.
Path 202 | total_timesteps 2808.
Path 203 | total_timesteps 2819.
Path 204 | total_timesteps 2827.
Path 205 | total_timesteps 2839.
Path 206 | total_timesteps 2849.
Path 207 | total_timesteps 2859.
Path 208 | total_timesteps 2867.
Path 209 | total_timesteps 2877.
Path 210 | total_timesteps 2898.
Path 211 | total_timesteps 2905.
Path 212 | total_timesteps 2919.
Path 213 | total_timesteps 2934.
Path 214 | total_timesteps 2943.
Path 215 | total_timesteps 2955.
Path 216 | total_timesteps 2971.
Path 217 | total_timesteps 3000.
Path 218 | total_timesteps 3015.
Path 219 | total_timesteps 3027.
Path 220 | total_timesteps 3043.
Path 221 | total_timesteps 3051.
Path 222 | total_timesteps 3061.
Path 223 | total_timesteps 3071.
Path 224 | total_timesteps 3081.
Path 225 | total_timesteps 3101.
Path 226 | total_timesteps 3121.
Path 227 | total_timesteps 3137.
Path 228 | total_timesteps 3150.
Path 229 | total_timesteps 3163.
Path 230 | total_timesteps 3170.
Path 231 | total_timesteps 3182.
Path 232 | total_timesteps 3197.
Path 233 | total_timesteps 3206.
Path 234 | total_timesteps 3226.
Path 235 | total_timesteps 3239.
Path 236 | total_timesteps 3257.
Path 237 | total_timesteps 3274.
Path 238 | total_timesteps 3288.
Path 239 | total_timesteps 3303.
Path 240 | total_timesteps 3311.
Path 241 | total_timesteps 3323.
Path 242 | total_timesteps 3332.
Path 243 | total_timesteps 3352.
Path 244 | total_timesteps 3367.
Path 245 | total_timesteps 3384.
Path 246 | total_timesteps 3394.
Path 247 | total_timesteps 3408.
Path 248 | total_timesteps 3420.
Path 249 | total_timesteps 3438.
Path 250 | total_timesteps 3460.
Path 251 | total_timesteps 3483.
Path 252 | total_timesteps 3499.
Path 253 | total_timesteps 3510.
Path 254 | total_timesteps 3520.
Path 255 | total_timesteps 3533.
Path 256 | total_timesteps 3549.
Path 257 | total_timesteps 3563.
Path 258 | total_timesteps 3572.
Path 259 | total_timesteps 3583.
Path 260 | total_timesteps 3600.
Path 261 | total_timesteps 3618.
Path 262 | total_timesteps 3638.
Path 263 | total_timesteps 3650.
Path 264 | total_timesteps 3666.
Path 265 | total_timesteps 3677.
Path 266 | total_timesteps 3689.
Path 267 | total_timesteps 3697.
Path 268 | total_timesteps 3715.
Path 269 | total_timesteps 3731.
Path 270 | total_timesteps 3752.
Path 271 | total_timesteps 3760.
Path 272 | total_timesteps 3775.
Path 273 | total_timesteps 3793.
Path 274 | total_timesteps 3805.
Path 275 | total_timesteps 3825.
Path 276 | total_timesteps 3840.
Path 277 | total_timesteps 3848.
Path 278 | total_timesteps 3859.
Path 279 | total_timesteps 3878.
Path 280 | total_timesteps 3890.
Path 281 | total_timesteps 3907.
Path 282 | total_timesteps 3917.
Path 283 | total_timesteps 3931.
Path 284 | total_timesteps 3940.
Path 285 | total_timesteps 3951.
Path 286 | total_timesteps 3963.
Path 287 | total_timesteps 3976.
Path 288 | total_timesteps 3989.
Path 289 | total_timesteps 4001.
Path 290 | total_timesteps 4012.
Path 291 | total_timesteps 4025.
Path 292 | total_timesteps 4043.
Path 293 | total_timesteps 4056.
Path 294 | total_timesteps 4066.
Path 295 | total_timesteps 4077.
Path 296 | total_timesteps 4090.
Path 297 | total_timesteps 4100.
Path 298 | total_timesteps 4119.
Path 299 | total_timesteps 4131.
Path 300 | total_timesteps 4138.
Path 301 | total_timesteps 4147.
Path 302 | total_timesteps 4165.
Path 303 | total_timesteps 4175.
Path 304 | total_timesteps 4192.
Path 305 | total_timesteps 4207.
Path 306 | total_timesteps 4221.
Path 307 | total_timesteps 4234.
Path 308 | total_timesteps 4247.
Path 309 | total_timesteps 4256.
Path 310 | total_timesteps 4266.
Path 311 | total_timesteps 4287.
Path 312 | total_timesteps 4308.
Path 313 | total_timesteps 4319.
Path 314 | total_timesteps 4334.
Path 315 | total_timesteps 4350.
Path 316 | total_timesteps 4366.
Path 317 | total_timesteps 4377.
Path 318 | total_timesteps 4391.
Path 319 | total_timesteps 4410.
Path 320 | total_timesteps 4427.
Path 321 | total_timesteps 4444.
Path 322 | total_timesteps 4456.
Path 323 | total_timesteps 4473.
Path 324 | total_timesteps 4487.
Path 325 | total_timesteps 4494.
Path 326 | total_timesteps 4502.
Path 327 | total_timesteps 4510.
Path 328 | total_timesteps 4535.
Path 329 | total_timesteps 4553.
Path 330 | total_timesteps 4566.
Path 331 | total_timesteps 4582.
Path 332 | total_timesteps 4597.
Path 333 | total_timesteps 4608.
Path 334 | total_timesteps 4619.
Path 335 | total_timesteps 4633.
Path 336 | total_timesteps 4643.
Path 337 | total_timesteps 4653.
Path 338 | total_timesteps 4661.
Path 339 | total_timesteps 4681.
Path 340 | total_timesteps 4700.
Path 341 | total_timesteps 4713.
Path 342 | total_timesteps 4727.
Path 343 | total_timesteps 4745.
Path 344 | total_timesteps 4754.
Path 345 | total_timesteps 4766.
Path 346 | total_timesteps 4778.
Path 347 | total_timesteps 4786.
Path 348 | total_timesteps 4795.
Path 349 | total_timesteps 4810.
Path 350 | total_timesteps 4821.
Path 351 | total_timesteps 4829.
Path 352 | total_timesteps 4836.
Path 353 | total_timesteps 4852.
Path 354 | total_timesteps 4865.
Path 355 | total_timesteps 4875.
Path 356 | total_timesteps 4888.
Path 357 | total_timesteps 4897.
Path 358 | total_timesteps 4909.
Path 359 | total_timesteps 4916.
Path 360 | total_timesteps 4927.
Path 361 | total_timesteps 4936.
Path 362 | total_timesteps 4952.
Path 363 | total_timesteps 4979.
Path 364 | total_timesteps 4998.
Path 365 | total_timesteps 5015.
Path 366 | total_timesteps 5029.
Path 367 | total_timesteps 5043.
Path 368 | total_timesteps 5052.
Path 369 | total_timesteps 5061.
Path 370 | total_timesteps 5073.
Path 371 | total_timesteps 5093.
Path 372 | total_timesteps 5103.
Path 373 | total_timesteps 5125.
Path 374 | total_timesteps 5137.
Path 375 | total_timesteps 5147.
Path 376 | total_timesteps 5159.
Path 377 | total_timesteps 5177.
Path 378 | total_timesteps 5187.
Path 379 | total_timesteps 5202.
Path 380 | total_timesteps 5218.
Path 381 | total_timesteps 5230.
Path 382 | total_timesteps 5251.
Path 383 | total_timesteps 5264.
Path 384 | total_timesteps 5275.
Path 385 | total_timesteps 5286.
Path 386 | total_timesteps 5298.
Path 387 | total_timesteps 5310.
Path 388 | total_timesteps 5344.
Path 389 | total_timesteps 5353.
Path 390 | total_timesteps 5371.
Path 391 | total_timesteps 5389.
Path 392 | total_timesteps 5398.
Path 393 | total_timesteps 5411.
Path 394 | total_timesteps 5424.
Path 395 | total_timesteps 5443.
Path 396 | total_timesteps 5456.
Path 397 | total_timesteps 5465.
Path 398 | total_timesteps 5475.
Path 399 | total_timesteps 5484.
Path 400 | total_timesteps 5497.
Path 401 | total_timesteps 5508.
Path 402 | total_timesteps 5526.
Path 403 | total_timesteps 5534.
Path 404 | total_timesteps 5545.
Path 405 | total_timesteps 5557.
Path 406 | total_timesteps 5568.
Path 407 | total_timesteps 5581.
Path 408 | total_timesteps 5593.
Path 409 | total_timesteps 5605.
Path 410 | total_timesteps 5616.
Path 411 | total_timesteps 5634.
Path 412 | total_timesteps 5651.
Path 413 | total_timesteps 5659.
Path 414 | total_timesteps 5673.
Path 415 | total_timesteps 5683.
Path 416 | total_timesteps 5690.
Path 417 | total_timesteps 5708.
Path 418 | total_timesteps 5723.
Path 419 | total_timesteps 5739.
Path 420 | total_timesteps 5754.
Path 421 | total_timesteps 5769.
Path 422 | total_timesteps 5783.
Path 423 | total_timesteps 5793.
Path 424 | total_timesteps 5803.
Path 425 | total_timesteps 5812.
Path 426 | total_timesteps 5822.
Path 427 | total_timesteps 5843.
Path 428 | total_timesteps 5854.
Path 429 | total_timesteps 5863.
Path 430 | total_timesteps 5874.
Path 431 | total_timesteps 5887.
Path 432 | total_timesteps 5905.
Path 433 | total_timesteps 5917.
Path 434 | total_timesteps 5926.
Path 435 | total_timesteps 5933.
Path 436 | total_timesteps 5944.
Path 437 | total_timesteps 5953.
Path 438 | total_timesteps 5964.
Path 439 | total_timesteps 5985.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.69    |
| Iteration     | 26       |
| MaximumReturn | 6.82     |
| MinimumReturn | -20.3    |
| TotalSamples  | 112190   |
----------------------------
itr #27 | 
Fitting dynamics.
Validation loss = 0.005864961538463831
Validation loss = 0.005583793856203556
Validation loss = 0.00570665393024683
Validation loss = 0.005655755288898945
Validation loss = 0.005575584713369608
Validation loss = 0.006296050269156694
Validation loss = 0.00556534668430686
Validation loss = 0.005784181412309408
Validation loss = 0.0055833011865615845
Validation loss = 0.005756788421422243
Validation loss = 0.005469758063554764
Validation loss = 0.005276133306324482
Validation loss = 0.005633133929222822
Validation loss = 0.005450795404613018
Validation loss = 0.006044137757271528
Validation loss = 0.005555914249271154
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 27.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 49.
Path 5 | total_timesteps 65.
Path 6 | total_timesteps 79.
Path 7 | total_timesteps 92.
Path 8 | total_timesteps 104.
Path 9 | total_timesteps 116.
Path 10 | total_timesteps 146.
Path 11 | total_timesteps 160.
Path 12 | total_timesteps 169.
Path 13 | total_timesteps 181.
Path 14 | total_timesteps 198.
Path 15 | total_timesteps 214.
Path 16 | total_timesteps 230.
Path 17 | total_timesteps 240.
Path 18 | total_timesteps 265.
Path 19 | total_timesteps 272.
Path 20 | total_timesteps 282.
Path 21 | total_timesteps 304.
Path 22 | total_timesteps 312.
Path 23 | total_timesteps 321.
Path 24 | total_timesteps 334.
Path 25 | total_timesteps 351.
Path 26 | total_timesteps 377.
Path 27 | total_timesteps 386.
Path 28 | total_timesteps 398.
Path 29 | total_timesteps 418.
Path 30 | total_timesteps 431.
Path 31 | total_timesteps 441.
Path 32 | total_timesteps 457.
Path 33 | total_timesteps 467.
Path 34 | total_timesteps 476.
Path 35 | total_timesteps 495.
Path 36 | total_timesteps 519.
Path 37 | total_timesteps 531.
Path 38 | total_timesteps 549.
Path 39 | total_timesteps 563.
Path 40 | total_timesteps 576.
Path 41 | total_timesteps 594.
Path 42 | total_timesteps 602.
Path 43 | total_timesteps 612.
Path 44 | total_timesteps 628.
Path 45 | total_timesteps 638.
Path 46 | total_timesteps 645.
Path 47 | total_timesteps 662.
Path 48 | total_timesteps 685.
Path 49 | total_timesteps 701.
Path 50 | total_timesteps 726.
Path 51 | total_timesteps 733.
Path 52 | total_timesteps 741.
Path 53 | total_timesteps 755.
Path 54 | total_timesteps 764.
Path 55 | total_timesteps 774.
Path 56 | total_timesteps 784.
Path 57 | total_timesteps 805.
Path 58 | total_timesteps 814.
Path 59 | total_timesteps 822.
Path 60 | total_timesteps 839.
Path 61 | total_timesteps 852.
Path 62 | total_timesteps 870.
Path 63 | total_timesteps 883.
Path 64 | total_timesteps 892.
Path 65 | total_timesteps 908.
Path 66 | total_timesteps 921.
Path 67 | total_timesteps 936.
Path 68 | total_timesteps 950.
Path 69 | total_timesteps 966.
Path 70 | total_timesteps 981.
Path 71 | total_timesteps 992.
Path 72 | total_timesteps 1004.
Path 73 | total_timesteps 1020.
Path 74 | total_timesteps 1044.
Path 75 | total_timesteps 1064.
Path 76 | total_timesteps 1074.
Path 77 | total_timesteps 1092.
Path 78 | total_timesteps 1100.
Path 79 | total_timesteps 1115.
Path 80 | total_timesteps 1125.
Path 81 | total_timesteps 1143.
Path 82 | total_timesteps 1156.
Path 83 | total_timesteps 1169.
Path 84 | total_timesteps 1178.
Path 85 | total_timesteps 1187.
Path 86 | total_timesteps 1200.
Path 87 | total_timesteps 1223.
Path 88 | total_timesteps 1235.
Path 89 | total_timesteps 1253.
Path 90 | total_timesteps 1275.
Path 91 | total_timesteps 1288.
Path 92 | total_timesteps 1298.
Path 93 | total_timesteps 1308.
Path 94 | total_timesteps 1330.
Path 95 | total_timesteps 1342.
Path 96 | total_timesteps 1358.
Path 97 | total_timesteps 1367.
Path 98 | total_timesteps 1382.
Path 99 | total_timesteps 1394.
Path 100 | total_timesteps 1407.
Path 101 | total_timesteps 1414.
Path 102 | total_timesteps 1431.
Path 103 | total_timesteps 1442.
Path 104 | total_timesteps 1465.
Path 105 | total_timesteps 1478.
Path 106 | total_timesteps 1487.
Path 107 | total_timesteps 1497.
Path 108 | total_timesteps 1513.
Path 109 | total_timesteps 1526.
Path 110 | total_timesteps 1544.
Path 111 | total_timesteps 1555.
Path 112 | total_timesteps 1563.
Path 113 | total_timesteps 1570.
Path 114 | total_timesteps 1579.
Path 115 | total_timesteps 1589.
Path 116 | total_timesteps 1603.
Path 117 | total_timesteps 1615.
Path 118 | total_timesteps 1630.
Path 119 | total_timesteps 1639.
Path 120 | total_timesteps 1654.
Path 121 | total_timesteps 1667.
Path 122 | total_timesteps 1693.
Path 123 | total_timesteps 1701.
Path 124 | total_timesteps 1713.
Path 125 | total_timesteps 1731.
Path 126 | total_timesteps 1740.
Path 127 | total_timesteps 1774.
Path 128 | total_timesteps 1791.
Path 129 | total_timesteps 1801.
Path 130 | total_timesteps 1819.
Path 131 | total_timesteps 1853.
Path 132 | total_timesteps 1861.
Path 133 | total_timesteps 1870.
Path 134 | total_timesteps 1879.
Path 135 | total_timesteps 1893.
Path 136 | total_timesteps 1906.
Path 137 | total_timesteps 1915.
Path 138 | total_timesteps 1929.
Path 139 | total_timesteps 1938.
Path 140 | total_timesteps 1949.
Path 141 | total_timesteps 1963.
Path 142 | total_timesteps 1977.
Path 143 | total_timesteps 1993.
Path 144 | total_timesteps 2003.
Path 145 | total_timesteps 2024.
Path 146 | total_timesteps 2041.
Path 147 | total_timesteps 2050.
Path 148 | total_timesteps 2067.
Path 149 | total_timesteps 2078.
Path 150 | total_timesteps 2093.
Path 151 | total_timesteps 2118.
Path 152 | total_timesteps 2126.
Path 153 | total_timesteps 2136.
Path 154 | total_timesteps 2157.
Path 155 | total_timesteps 2171.
Path 156 | total_timesteps 2181.
Path 157 | total_timesteps 2195.
Path 158 | total_timesteps 2205.
Path 159 | total_timesteps 2219.
Path 160 | total_timesteps 2229.
Path 161 | total_timesteps 2244.
Path 162 | total_timesteps 2254.
Path 163 | total_timesteps 2280.
Path 164 | total_timesteps 2292.
Path 165 | total_timesteps 2305.
Path 166 | total_timesteps 2313.
Path 167 | total_timesteps 2333.
Path 168 | total_timesteps 2349.
Path 169 | total_timesteps 2360.
Path 170 | total_timesteps 2380.
Path 171 | total_timesteps 2392.
Path 172 | total_timesteps 2401.
Path 173 | total_timesteps 2414.
Path 174 | total_timesteps 2422.
Path 175 | total_timesteps 2443.
Path 176 | total_timesteps 2470.
Path 177 | total_timesteps 2478.
Path 178 | total_timesteps 2500.
Path 179 | total_timesteps 2514.
Path 180 | total_timesteps 2523.
Path 181 | total_timesteps 2537.
Path 182 | total_timesteps 2546.
Path 183 | total_timesteps 2556.
Path 184 | total_timesteps 2570.
Path 185 | total_timesteps 2577.
Path 186 | total_timesteps 2595.
Path 187 | total_timesteps 2606.
Path 188 | total_timesteps 2618.
Path 189 | total_timesteps 2630.
Path 190 | total_timesteps 2643.
Path 191 | total_timesteps 2655.
Path 192 | total_timesteps 2665.
Path 193 | total_timesteps 2676.
Path 194 | total_timesteps 2692.
Path 195 | total_timesteps 2701.
Path 196 | total_timesteps 2720.
Path 197 | total_timesteps 2740.
Path 198 | total_timesteps 2754.
Path 199 | total_timesteps 2767.
Path 200 | total_timesteps 2782.
Path 201 | total_timesteps 2792.
Path 202 | total_timesteps 2804.
Path 203 | total_timesteps 2814.
Path 204 | total_timesteps 2830.
Path 205 | total_timesteps 2849.
Path 206 | total_timesteps 2861.
Path 207 | total_timesteps 2871.
Path 208 | total_timesteps 2882.
Path 209 | total_timesteps 2898.
Path 210 | total_timesteps 2912.
Path 211 | total_timesteps 2926.
Path 212 | total_timesteps 2933.
Path 213 | total_timesteps 2947.
Path 214 | total_timesteps 2962.
Path 215 | total_timesteps 2976.
Path 216 | total_timesteps 2990.
Path 217 | total_timesteps 2999.
Path 218 | total_timesteps 3008.
Path 219 | total_timesteps 3021.
Path 220 | total_timesteps 3031.
Path 221 | total_timesteps 3044.
Path 222 | total_timesteps 3056.
Path 223 | total_timesteps 3073.
Path 224 | total_timesteps 3085.
Path 225 | total_timesteps 3104.
Path 226 | total_timesteps 3115.
Path 227 | total_timesteps 3129.
Path 228 | total_timesteps 3146.
Path 229 | total_timesteps 3157.
Path 230 | total_timesteps 3171.
Path 231 | total_timesteps 3186.
Path 232 | total_timesteps 3205.
Path 233 | total_timesteps 3217.
Path 234 | total_timesteps 3229.
Path 235 | total_timesteps 3241.
Path 236 | total_timesteps 3266.
Path 237 | total_timesteps 3275.
Path 238 | total_timesteps 3296.
Path 239 | total_timesteps 3313.
Path 240 | total_timesteps 3326.
Path 241 | total_timesteps 3337.
Path 242 | total_timesteps 3347.
Path 243 | total_timesteps 3356.
Path 244 | total_timesteps 3369.
Path 245 | total_timesteps 3381.
Path 246 | total_timesteps 3397.
Path 247 | total_timesteps 3406.
Path 248 | total_timesteps 3422.
Path 249 | total_timesteps 3444.
Path 250 | total_timesteps 3461.
Path 251 | total_timesteps 3473.
Path 252 | total_timesteps 3489.
Path 253 | total_timesteps 3510.
Path 254 | total_timesteps 3525.
Path 255 | total_timesteps 3536.
Path 256 | total_timesteps 3549.
Path 257 | total_timesteps 3562.
Path 258 | total_timesteps 3570.
Path 259 | total_timesteps 3586.
Path 260 | total_timesteps 3597.
Path 261 | total_timesteps 3609.
Path 262 | total_timesteps 3621.
Path 263 | total_timesteps 3636.
Path 264 | total_timesteps 3649.
Path 265 | total_timesteps 3657.
Path 266 | total_timesteps 3666.
Path 267 | total_timesteps 3683.
Path 268 | total_timesteps 3721.
Path 269 | total_timesteps 3731.
Path 270 | total_timesteps 3743.
Path 271 | total_timesteps 3759.
Path 272 | total_timesteps 3770.
Path 273 | total_timesteps 3779.
Path 274 | total_timesteps 3790.
Path 275 | total_timesteps 3798.
Path 276 | total_timesteps 3808.
Path 277 | total_timesteps 3828.
Path 278 | total_timesteps 3840.
Path 279 | total_timesteps 3847.
Path 280 | total_timesteps 3856.
Path 281 | total_timesteps 3873.
Path 282 | total_timesteps 3885.
Path 283 | total_timesteps 3894.
Path 284 | total_timesteps 3904.
Path 285 | total_timesteps 3919.
Path 286 | total_timesteps 3934.
Path 287 | total_timesteps 3949.
Path 288 | total_timesteps 3962.
Path 289 | total_timesteps 3979.
Path 290 | total_timesteps 3996.
Path 291 | total_timesteps 4009.
Path 292 | total_timesteps 4019.
Path 293 | total_timesteps 4043.
Path 294 | total_timesteps 4055.
Path 295 | total_timesteps 4066.
Path 296 | total_timesteps 4077.
Path 297 | total_timesteps 4095.
Path 298 | total_timesteps 4114.
Path 299 | total_timesteps 4128.
Path 300 | total_timesteps 4149.
Path 301 | total_timesteps 4169.
Path 302 | total_timesteps 4184.
Path 303 | total_timesteps 4196.
Path 304 | total_timesteps 4212.
Path 305 | total_timesteps 4236.
Path 306 | total_timesteps 4248.
Path 307 | total_timesteps 4264.
Path 308 | total_timesteps 4278.
Path 309 | total_timesteps 4299.
Path 310 | total_timesteps 4310.
Path 311 | total_timesteps 4317.
Path 312 | total_timesteps 4332.
Path 313 | total_timesteps 4356.
Path 314 | total_timesteps 4365.
Path 315 | total_timesteps 4379.
Path 316 | total_timesteps 4392.
Path 317 | total_timesteps 4407.
Path 318 | total_timesteps 4421.
Path 319 | total_timesteps 4432.
Path 320 | total_timesteps 4447.
Path 321 | total_timesteps 4455.
Path 322 | total_timesteps 4463.
Path 323 | total_timesteps 4480.
Path 324 | total_timesteps 4498.
Path 325 | total_timesteps 4512.
Path 326 | total_timesteps 4523.
Path 327 | total_timesteps 4538.
Path 328 | total_timesteps 4568.
Path 329 | total_timesteps 4579.
Path 330 | total_timesteps 4589.
Path 331 | total_timesteps 4609.
Path 332 | total_timesteps 4619.
Path 333 | total_timesteps 4637.
Path 334 | total_timesteps 4646.
Path 335 | total_timesteps 4658.
Path 336 | total_timesteps 4667.
Path 337 | total_timesteps 4684.
Path 338 | total_timesteps 4697.
Path 339 | total_timesteps 4706.
Path 340 | total_timesteps 4714.
Path 341 | total_timesteps 4721.
Path 342 | total_timesteps 4730.
Path 343 | total_timesteps 4740.
Path 344 | total_timesteps 4763.
Path 345 | total_timesteps 4780.
Path 346 | total_timesteps 4792.
Path 347 | total_timesteps 4813.
Path 348 | total_timesteps 4824.
Path 349 | total_timesteps 4839.
Path 350 | total_timesteps 4850.
Path 351 | total_timesteps 4861.
Path 352 | total_timesteps 4882.
Path 353 | total_timesteps 4911.
Path 354 | total_timesteps 4923.
Path 355 | total_timesteps 4937.
Path 356 | total_timesteps 4952.
Path 357 | total_timesteps 4967.
Path 358 | total_timesteps 4979.
Path 359 | total_timesteps 5000.
Path 360 | total_timesteps 5010.
Path 361 | total_timesteps 5020.
Path 362 | total_timesteps 5032.
Path 363 | total_timesteps 5046.
Path 364 | total_timesteps 5064.
Path 365 | total_timesteps 5080.
Path 366 | total_timesteps 5105.
Path 367 | total_timesteps 5114.
Path 368 | total_timesteps 5124.
Path 369 | total_timesteps 5136.
Path 370 | total_timesteps 5146.
Path 371 | total_timesteps 5164.
Path 372 | total_timesteps 5173.
Path 373 | total_timesteps 5186.
Path 374 | total_timesteps 5193.
Path 375 | total_timesteps 5205.
Path 376 | total_timesteps 5214.
Path 377 | total_timesteps 5225.
Path 378 | total_timesteps 5242.
Path 379 | total_timesteps 5254.
Path 380 | total_timesteps 5271.
Path 381 | total_timesteps 5292.
Path 382 | total_timesteps 5302.
Path 383 | total_timesteps 5317.
Path 384 | total_timesteps 5333.
Path 385 | total_timesteps 5344.
Path 386 | total_timesteps 5357.
Path 387 | total_timesteps 5366.
Path 388 | total_timesteps 5379.
Path 389 | total_timesteps 5395.
Path 390 | total_timesteps 5414.
Path 391 | total_timesteps 5425.
Path 392 | total_timesteps 5435.
Path 393 | total_timesteps 5445.
Path 394 | total_timesteps 5460.
Path 395 | total_timesteps 5470.
Path 396 | total_timesteps 5480.
Path 397 | total_timesteps 5516.
Path 398 | total_timesteps 5534.
Path 399 | total_timesteps 5544.
Path 400 | total_timesteps 5559.
Path 401 | total_timesteps 5570.
Path 402 | total_timesteps 5578.
Path 403 | total_timesteps 5593.
Path 404 | total_timesteps 5607.
Path 405 | total_timesteps 5623.
Path 406 | total_timesteps 5640.
Path 407 | total_timesteps 5652.
Path 408 | total_timesteps 5663.
Path 409 | total_timesteps 5670.
Path 410 | total_timesteps 5689.
Path 411 | total_timesteps 5703.
Path 412 | total_timesteps 5711.
Path 413 | total_timesteps 5728.
Path 414 | total_timesteps 5736.
Path 415 | total_timesteps 5747.
Path 416 | total_timesteps 5768.
Path 417 | total_timesteps 5788.
Path 418 | total_timesteps 5804.
Path 419 | total_timesteps 5816.
Path 420 | total_timesteps 5834.
Path 421 | total_timesteps 5857.
Path 422 | total_timesteps 5872.
Path 423 | total_timesteps 5886.
Path 424 | total_timesteps 5907.
Path 425 | total_timesteps 5923.
Path 426 | total_timesteps 5933.
Path 427 | total_timesteps 5947.
Path 428 | total_timesteps 5964.
Path 429 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.41    |
| Iteration     | 27       |
| MaximumReturn | 8.74     |
| MinimumReturn | -20.1    |
| TotalSamples  | 116194   |
----------------------------
itr #28 | 
Fitting dynamics.
Validation loss = 0.005503978114575148
Validation loss = 0.005688387434929609
Validation loss = 0.005743722431361675
Validation loss = 0.005272276233881712
Validation loss = 0.005878131370991468
Validation loss = 0.005239721853286028
Validation loss = 0.005437055137008429
Validation loss = 0.00537283206358552
Validation loss = 0.005409862380474806
Validation loss = 0.005472084973007441
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 27.
Path 3 | total_timesteps 41.
Path 4 | total_timesteps 55.
Path 5 | total_timesteps 68.
Path 6 | total_timesteps 81.
Path 7 | total_timesteps 91.
Path 8 | total_timesteps 101.
Path 9 | total_timesteps 115.
Path 10 | total_timesteps 125.
Path 11 | total_timesteps 138.
Path 12 | total_timesteps 155.
Path 13 | total_timesteps 173.
Path 14 | total_timesteps 195.
Path 15 | total_timesteps 209.
Path 16 | total_timesteps 229.
Path 17 | total_timesteps 241.
Path 18 | total_timesteps 253.
Path 19 | total_timesteps 263.
Path 20 | total_timesteps 276.
Path 21 | total_timesteps 291.
Path 22 | total_timesteps 300.
Path 23 | total_timesteps 319.
Path 24 | total_timesteps 341.
Path 25 | total_timesteps 355.
Path 26 | total_timesteps 365.
Path 27 | total_timesteps 377.
Path 28 | total_timesteps 398.
Path 29 | total_timesteps 409.
Path 30 | total_timesteps 423.
Path 31 | total_timesteps 434.
Path 32 | total_timesteps 448.
Path 33 | total_timesteps 456.
Path 34 | total_timesteps 486.
Path 35 | total_timesteps 494.
Path 36 | total_timesteps 501.
Path 37 | total_timesteps 512.
Path 38 | total_timesteps 530.
Path 39 | total_timesteps 548.
Path 40 | total_timesteps 559.
Path 41 | total_timesteps 575.
Path 42 | total_timesteps 589.
Path 43 | total_timesteps 603.
Path 44 | total_timesteps 619.
Path 45 | total_timesteps 633.
Path 46 | total_timesteps 645.
Path 47 | total_timesteps 660.
Path 48 | total_timesteps 669.
Path 49 | total_timesteps 687.
Path 50 | total_timesteps 697.
Path 51 | total_timesteps 719.
Path 52 | total_timesteps 728.
Path 53 | total_timesteps 740.
Path 54 | total_timesteps 757.
Path 55 | total_timesteps 765.
Path 56 | total_timesteps 785.
Path 57 | total_timesteps 794.
Path 58 | total_timesteps 809.
Path 59 | total_timesteps 823.
Path 60 | total_timesteps 844.
Path 61 | total_timesteps 854.
Path 62 | total_timesteps 870.
Path 63 | total_timesteps 898.
Path 64 | total_timesteps 912.
Path 65 | total_timesteps 924.
Path 66 | total_timesteps 936.
Path 67 | total_timesteps 945.
Path 68 | total_timesteps 963.
Path 69 | total_timesteps 974.
Path 70 | total_timesteps 987.
Path 71 | total_timesteps 1011.
Path 72 | total_timesteps 1018.
Path 73 | total_timesteps 1029.
Path 74 | total_timesteps 1040.
Path 75 | total_timesteps 1055.
Path 76 | total_timesteps 1068.
Path 77 | total_timesteps 1077.
Path 78 | total_timesteps 1084.
Path 79 | total_timesteps 1096.
Path 80 | total_timesteps 1133.
Path 81 | total_timesteps 1143.
Path 82 | total_timesteps 1157.
Path 83 | total_timesteps 1174.
Path 84 | total_timesteps 1196.
Path 85 | total_timesteps 1206.
Path 86 | total_timesteps 1216.
Path 87 | total_timesteps 1224.
Path 88 | total_timesteps 1233.
Path 89 | total_timesteps 1252.
Path 90 | total_timesteps 1261.
Path 91 | total_timesteps 1273.
Path 92 | total_timesteps 1287.
Path 93 | total_timesteps 1303.
Path 94 | total_timesteps 1313.
Path 95 | total_timesteps 1329.
Path 96 | total_timesteps 1350.
Path 97 | total_timesteps 1363.
Path 98 | total_timesteps 1379.
Path 99 | total_timesteps 1388.
Path 100 | total_timesteps 1406.
Path 101 | total_timesteps 1414.
Path 102 | total_timesteps 1424.
Path 103 | total_timesteps 1436.
Path 104 | total_timesteps 1448.
Path 105 | total_timesteps 1459.
Path 106 | total_timesteps 1471.
Path 107 | total_timesteps 1496.
Path 108 | total_timesteps 1510.
Path 109 | total_timesteps 1522.
Path 110 | total_timesteps 1532.
Path 111 | total_timesteps 1551.
Path 112 | total_timesteps 1566.
Path 113 | total_timesteps 1589.
Path 114 | total_timesteps 1598.
Path 115 | total_timesteps 1624.
Path 116 | total_timesteps 1634.
Path 117 | total_timesteps 1657.
Path 118 | total_timesteps 1668.
Path 119 | total_timesteps 1677.
Path 120 | total_timesteps 1700.
Path 121 | total_timesteps 1709.
Path 122 | total_timesteps 1729.
Path 123 | total_timesteps 1738.
Path 124 | total_timesteps 1753.
Path 125 | total_timesteps 1763.
Path 126 | total_timesteps 1786.
Path 127 | total_timesteps 1792.
Path 128 | total_timesteps 1805.
Path 129 | total_timesteps 1814.
Path 130 | total_timesteps 1823.
Path 131 | total_timesteps 1830.
Path 132 | total_timesteps 1840.
Path 133 | total_timesteps 1848.
Path 134 | total_timesteps 1870.
Path 135 | total_timesteps 1879.
Path 136 | total_timesteps 1888.
Path 137 | total_timesteps 1901.
Path 138 | total_timesteps 1913.
Path 139 | total_timesteps 1936.
Path 140 | total_timesteps 1945.
Path 141 | total_timesteps 1957.
Path 142 | total_timesteps 1970.
Path 143 | total_timesteps 1985.
Path 144 | total_timesteps 1996.
Path 145 | total_timesteps 2013.
Path 146 | total_timesteps 2024.
Path 147 | total_timesteps 2042.
Path 148 | total_timesteps 2058.
Path 149 | total_timesteps 2078.
Path 150 | total_timesteps 2089.
Path 151 | total_timesteps 2103.
Path 152 | total_timesteps 2113.
Path 153 | total_timesteps 2122.
Path 154 | total_timesteps 2129.
Path 155 | total_timesteps 2150.
Path 156 | total_timesteps 2166.
Path 157 | total_timesteps 2179.
Path 158 | total_timesteps 2192.
Path 159 | total_timesteps 2206.
Path 160 | total_timesteps 2215.
Path 161 | total_timesteps 2226.
Path 162 | total_timesteps 2245.
Path 163 | total_timesteps 2262.
Path 164 | total_timesteps 2285.
Path 165 | total_timesteps 2299.
Path 166 | total_timesteps 2318.
Path 167 | total_timesteps 2345.
Path 168 | total_timesteps 2357.
Path 169 | total_timesteps 2382.
Path 170 | total_timesteps 2391.
Path 171 | total_timesteps 2401.
Path 172 | total_timesteps 2412.
Path 173 | total_timesteps 2419.
Path 174 | total_timesteps 2432.
Path 175 | total_timesteps 2442.
Path 176 | total_timesteps 2455.
Path 177 | total_timesteps 2469.
Path 178 | total_timesteps 2487.
Path 179 | total_timesteps 2500.
Path 180 | total_timesteps 2523.
Path 181 | total_timesteps 2534.
Path 182 | total_timesteps 2554.
Path 183 | total_timesteps 2565.
Path 184 | total_timesteps 2589.
Path 185 | total_timesteps 2605.
Path 186 | total_timesteps 2618.
Path 187 | total_timesteps 2632.
Path 188 | total_timesteps 2645.
Path 189 | total_timesteps 2658.
Path 190 | total_timesteps 2672.
Path 191 | total_timesteps 2684.
Path 192 | total_timesteps 2695.
Path 193 | total_timesteps 2706.
Path 194 | total_timesteps 2716.
Path 195 | total_timesteps 2729.
Path 196 | total_timesteps 2747.
Path 197 | total_timesteps 2754.
Path 198 | total_timesteps 2770.
Path 199 | total_timesteps 2785.
Path 200 | total_timesteps 2798.
Path 201 | total_timesteps 2807.
Path 202 | total_timesteps 2826.
Path 203 | total_timesteps 2837.
Path 204 | total_timesteps 2850.
Path 205 | total_timesteps 2857.
Path 206 | total_timesteps 2869.
Path 207 | total_timesteps 2884.
Path 208 | total_timesteps 2895.
Path 209 | total_timesteps 2911.
Path 210 | total_timesteps 2925.
Path 211 | total_timesteps 2933.
Path 212 | total_timesteps 2941.
Path 213 | total_timesteps 2961.
Path 214 | total_timesteps 2980.
Path 215 | total_timesteps 2996.
Path 216 | total_timesteps 3012.
Path 217 | total_timesteps 3025.
Path 218 | total_timesteps 3038.
Path 219 | total_timesteps 3053.
Path 220 | total_timesteps 3062.
Path 221 | total_timesteps 3076.
Path 222 | total_timesteps 3087.
Path 223 | total_timesteps 3104.
Path 224 | total_timesteps 3117.
Path 225 | total_timesteps 3127.
Path 226 | total_timesteps 3142.
Path 227 | total_timesteps 3152.
Path 228 | total_timesteps 3164.
Path 229 | total_timesteps 3181.
Path 230 | total_timesteps 3203.
Path 231 | total_timesteps 3216.
Path 232 | total_timesteps 3236.
Path 233 | total_timesteps 3244.
Path 234 | total_timesteps 3251.
Path 235 | total_timesteps 3277.
Path 236 | total_timesteps 3289.
Path 237 | total_timesteps 3299.
Path 238 | total_timesteps 3307.
Path 239 | total_timesteps 3331.
Path 240 | total_timesteps 3342.
Path 241 | total_timesteps 3360.
Path 242 | total_timesteps 3374.
Path 243 | total_timesteps 3382.
Path 244 | total_timesteps 3392.
Path 245 | total_timesteps 3406.
Path 246 | total_timesteps 3420.
Path 247 | total_timesteps 3434.
Path 248 | total_timesteps 3454.
Path 249 | total_timesteps 3468.
Path 250 | total_timesteps 3486.
Path 251 | total_timesteps 3498.
Path 252 | total_timesteps 3519.
Path 253 | total_timesteps 3529.
Path 254 | total_timesteps 3549.
Path 255 | total_timesteps 3560.
Path 256 | total_timesteps 3574.
Path 257 | total_timesteps 3585.
Path 258 | total_timesteps 3604.
Path 259 | total_timesteps 3621.
Path 260 | total_timesteps 3629.
Path 261 | total_timesteps 3640.
Path 262 | total_timesteps 3650.
Path 263 | total_timesteps 3664.
Path 264 | total_timesteps 3677.
Path 265 | total_timesteps 3683.
Path 266 | total_timesteps 3699.
Path 267 | total_timesteps 3723.
Path 268 | total_timesteps 3733.
Path 269 | total_timesteps 3743.
Path 270 | total_timesteps 3756.
Path 271 | total_timesteps 3770.
Path 272 | total_timesteps 3783.
Path 273 | total_timesteps 3793.
Path 274 | total_timesteps 3816.
Path 275 | total_timesteps 3839.
Path 276 | total_timesteps 3852.
Path 277 | total_timesteps 3864.
Path 278 | total_timesteps 3880.
Path 279 | total_timesteps 3899.
Path 280 | total_timesteps 3914.
Path 281 | total_timesteps 3941.
Path 282 | total_timesteps 3956.
Path 283 | total_timesteps 3968.
Path 284 | total_timesteps 3987.
Path 285 | total_timesteps 4002.
Path 286 | total_timesteps 4015.
Path 287 | total_timesteps 4027.
Path 288 | total_timesteps 4047.
Path 289 | total_timesteps 4059.
Path 290 | total_timesteps 4076.
Path 291 | total_timesteps 4089.
Path 292 | total_timesteps 4097.
Path 293 | total_timesteps 4108.
Path 294 | total_timesteps 4129.
Path 295 | total_timesteps 4146.
Path 296 | total_timesteps 4156.
Path 297 | total_timesteps 4163.
Path 298 | total_timesteps 4180.
Path 299 | total_timesteps 4189.
Path 300 | total_timesteps 4208.
Path 301 | total_timesteps 4218.
Path 302 | total_timesteps 4227.
Path 303 | total_timesteps 4237.
Path 304 | total_timesteps 4249.
Path 305 | total_timesteps 4263.
Path 306 | total_timesteps 4272.
Path 307 | total_timesteps 4287.
Path 308 | total_timesteps 4307.
Path 309 | total_timesteps 4319.
Path 310 | total_timesteps 4328.
Path 311 | total_timesteps 4336.
Path 312 | total_timesteps 4348.
Path 313 | total_timesteps 4359.
Path 314 | total_timesteps 4389.
Path 315 | total_timesteps 4400.
Path 316 | total_timesteps 4414.
Path 317 | total_timesteps 4427.
Path 318 | total_timesteps 4444.
Path 319 | total_timesteps 4482.
Path 320 | total_timesteps 4490.
Path 321 | total_timesteps 4500.
Path 322 | total_timesteps 4513.
Path 323 | total_timesteps 4529.
Path 324 | total_timesteps 4538.
Path 325 | total_timesteps 4554.
Path 326 | total_timesteps 4562.
Path 327 | total_timesteps 4575.
Path 328 | total_timesteps 4583.
Path 329 | total_timesteps 4599.
Path 330 | total_timesteps 4610.
Path 331 | total_timesteps 4617.
Path 332 | total_timesteps 4631.
Path 333 | total_timesteps 4648.
Path 334 | total_timesteps 4660.
Path 335 | total_timesteps 4668.
Path 336 | total_timesteps 4678.
Path 337 | total_timesteps 4694.
Path 338 | total_timesteps 4703.
Path 339 | total_timesteps 4713.
Path 340 | total_timesteps 4726.
Path 341 | total_timesteps 4736.
Path 342 | total_timesteps 4747.
Path 343 | total_timesteps 4762.
Path 344 | total_timesteps 4774.
Path 345 | total_timesteps 4787.
Path 346 | total_timesteps 4796.
Path 347 | total_timesteps 4811.
Path 348 | total_timesteps 4822.
Path 349 | total_timesteps 4831.
Path 350 | total_timesteps 4844.
Path 351 | total_timesteps 4861.
Path 352 | total_timesteps 4871.
Path 353 | total_timesteps 4886.
Path 354 | total_timesteps 4906.
Path 355 | total_timesteps 4918.
Path 356 | total_timesteps 4931.
Path 357 | total_timesteps 4942.
Path 358 | total_timesteps 4951.
Path 359 | total_timesteps 4964.
Path 360 | total_timesteps 4976.
Path 361 | total_timesteps 4986.
Path 362 | total_timesteps 5012.
Path 363 | total_timesteps 5022.
Path 364 | total_timesteps 5030.
Path 365 | total_timesteps 5042.
Path 366 | total_timesteps 5053.
Path 367 | total_timesteps 5061.
Path 368 | total_timesteps 5069.
Path 369 | total_timesteps 5077.
Path 370 | total_timesteps 5094.
Path 371 | total_timesteps 5103.
Path 372 | total_timesteps 5115.
Path 373 | total_timesteps 5127.
Path 374 | total_timesteps 5136.
Path 375 | total_timesteps 5143.
Path 376 | total_timesteps 5151.
Path 377 | total_timesteps 5173.
Path 378 | total_timesteps 5193.
Path 379 | total_timesteps 5205.
Path 380 | total_timesteps 5217.
Path 381 | total_timesteps 5228.
Path 382 | total_timesteps 5235.
Path 383 | total_timesteps 5246.
Path 384 | total_timesteps 5256.
Path 385 | total_timesteps 5267.
Path 386 | total_timesteps 5281.
Path 387 | total_timesteps 5295.
Path 388 | total_timesteps 5303.
Path 389 | total_timesteps 5322.
Path 390 | total_timesteps 5336.
Path 391 | total_timesteps 5346.
Path 392 | total_timesteps 5360.
Path 393 | total_timesteps 5371.
Path 394 | total_timesteps 5379.
Path 395 | total_timesteps 5388.
Path 396 | total_timesteps 5399.
Path 397 | total_timesteps 5411.
Path 398 | total_timesteps 5422.
Path 399 | total_timesteps 5434.
Path 400 | total_timesteps 5452.
Path 401 | total_timesteps 5473.
Path 402 | total_timesteps 5495.
Path 403 | total_timesteps 5507.
Path 404 | total_timesteps 5520.
Path 405 | total_timesteps 5536.
Path 406 | total_timesteps 5546.
Path 407 | total_timesteps 5558.
Path 408 | total_timesteps 5573.
Path 409 | total_timesteps 5584.
Path 410 | total_timesteps 5595.
Path 411 | total_timesteps 5604.
Path 412 | total_timesteps 5626.
Path 413 | total_timesteps 5635.
Path 414 | total_timesteps 5645.
Path 415 | total_timesteps 5657.
Path 416 | total_timesteps 5670.
Path 417 | total_timesteps 5686.
Path 418 | total_timesteps 5702.
Path 419 | total_timesteps 5716.
Path 420 | total_timesteps 5742.
Path 421 | total_timesteps 5753.
Path 422 | total_timesteps 5763.
Path 423 | total_timesteps 5777.
Path 424 | total_timesteps 5794.
Path 425 | total_timesteps 5819.
Path 426 | total_timesteps 5831.
Path 427 | total_timesteps 5845.
Path 428 | total_timesteps 5855.
Path 429 | total_timesteps 5869.
Path 430 | total_timesteps 5879.
Path 431 | total_timesteps 5893.
Path 432 | total_timesteps 5902.
Path 433 | total_timesteps 5915.
Path 434 | total_timesteps 5930.
Path 435 | total_timesteps 5948.
Path 436 | total_timesteps 5957.
Path 437 | total_timesteps 5975.
Path 438 | total_timesteps 5984.
Path 439 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.41    |
| Iteration     | 28       |
| MaximumReturn | 6.51     |
| MinimumReturn | -19.5    |
| TotalSamples  | 120202   |
----------------------------
itr #29 | 
Fitting dynamics.
Validation loss = 0.005479395855218172
Validation loss = 0.005564373917877674
Validation loss = 0.005170919932425022
Validation loss = 0.005433772224932909
Validation loss = 0.005233942996710539
Validation loss = 0.005128806456923485
Validation loss = 0.00518093490973115
Validation loss = 0.00508793443441391
Validation loss = 0.005293427500873804
Validation loss = 0.005241661332547665
Validation loss = 0.005216368939727545
Validation loss = 0.005332509521394968
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 36.
Path 4 | total_timesteps 49.
Path 5 | total_timesteps 59.
Path 6 | total_timesteps 78.
Path 7 | total_timesteps 96.
Path 8 | total_timesteps 106.
Path 9 | total_timesteps 122.
Path 10 | total_timesteps 136.
Path 11 | total_timesteps 144.
Path 12 | total_timesteps 158.
Path 13 | total_timesteps 175.
Path 14 | total_timesteps 188.
Path 15 | total_timesteps 200.
Path 16 | total_timesteps 222.
Path 17 | total_timesteps 242.
Path 18 | total_timesteps 252.
Path 19 | total_timesteps 262.
Path 20 | total_timesteps 277.
Path 21 | total_timesteps 297.
Path 22 | total_timesteps 307.
Path 23 | total_timesteps 318.
Path 24 | total_timesteps 331.
Path 25 | total_timesteps 341.
Path 26 | total_timesteps 348.
Path 27 | total_timesteps 380.
Path 28 | total_timesteps 395.
Path 29 | total_timesteps 412.
Path 30 | total_timesteps 421.
Path 31 | total_timesteps 433.
Path 32 | total_timesteps 442.
Path 33 | total_timesteps 454.
Path 34 | total_timesteps 464.
Path 35 | total_timesteps 478.
Path 36 | total_timesteps 490.
Path 37 | total_timesteps 506.
Path 38 | total_timesteps 516.
Path 39 | total_timesteps 541.
Path 40 | total_timesteps 557.
Path 41 | total_timesteps 579.
Path 42 | total_timesteps 586.
Path 43 | total_timesteps 597.
Path 44 | total_timesteps 608.
Path 45 | total_timesteps 619.
Path 46 | total_timesteps 632.
Path 47 | total_timesteps 642.
Path 48 | total_timesteps 656.
Path 49 | total_timesteps 665.
Path 50 | total_timesteps 677.
Path 51 | total_timesteps 690.
Path 52 | total_timesteps 699.
Path 53 | total_timesteps 714.
Path 54 | total_timesteps 726.
Path 55 | total_timesteps 735.
Path 56 | total_timesteps 750.
Path 57 | total_timesteps 757.
Path 58 | total_timesteps 776.
Path 59 | total_timesteps 787.
Path 60 | total_timesteps 802.
Path 61 | total_timesteps 815.
Path 62 | total_timesteps 827.
Path 63 | total_timesteps 839.
Path 64 | total_timesteps 855.
Path 65 | total_timesteps 866.
Path 66 | total_timesteps 876.
Path 67 | total_timesteps 891.
Path 68 | total_timesteps 907.
Path 69 | total_timesteps 922.
Path 70 | total_timesteps 933.
Path 71 | total_timesteps 943.
Path 72 | total_timesteps 953.
Path 73 | total_timesteps 967.
Path 74 | total_timesteps 978.
Path 75 | total_timesteps 989.
Path 76 | total_timesteps 998.
Path 77 | total_timesteps 1007.
Path 78 | total_timesteps 1019.
Path 79 | total_timesteps 1034.
Path 80 | total_timesteps 1045.
Path 81 | total_timesteps 1066.
Path 82 | total_timesteps 1082.
Path 83 | total_timesteps 1099.
Path 84 | total_timesteps 1116.
Path 85 | total_timesteps 1131.
Path 86 | total_timesteps 1144.
Path 87 | total_timesteps 1160.
Path 88 | total_timesteps 1174.
Path 89 | total_timesteps 1193.
Path 90 | total_timesteps 1207.
Path 91 | total_timesteps 1229.
Path 92 | total_timesteps 1245.
Path 93 | total_timesteps 1260.
Path 94 | total_timesteps 1270.
Path 95 | total_timesteps 1287.
Path 96 | total_timesteps 1298.
Path 97 | total_timesteps 1314.
Path 98 | total_timesteps 1326.
Path 99 | total_timesteps 1335.
Path 100 | total_timesteps 1343.
Path 101 | total_timesteps 1353.
Path 102 | total_timesteps 1361.
Path 103 | total_timesteps 1373.
Path 104 | total_timesteps 1389.
Path 105 | total_timesteps 1406.
Path 106 | total_timesteps 1419.
Path 107 | total_timesteps 1433.
Path 108 | total_timesteps 1449.
Path 109 | total_timesteps 1474.
Path 110 | total_timesteps 1492.
Path 111 | total_timesteps 1523.
Path 112 | total_timesteps 1543.
Path 113 | total_timesteps 1559.
Path 114 | total_timesteps 1581.
Path 115 | total_timesteps 1591.
Path 116 | total_timesteps 1607.
Path 117 | total_timesteps 1622.
Path 118 | total_timesteps 1639.
Path 119 | total_timesteps 1652.
Path 120 | total_timesteps 1669.
Path 121 | total_timesteps 1697.
Path 122 | total_timesteps 1715.
Path 123 | total_timesteps 1727.
Path 124 | total_timesteps 1743.
Path 125 | total_timesteps 1751.
Path 126 | total_timesteps 1762.
Path 127 | total_timesteps 1776.
Path 128 | total_timesteps 1790.
Path 129 | total_timesteps 1799.
Path 130 | total_timesteps 1814.
Path 131 | total_timesteps 1822.
Path 132 | total_timesteps 1845.
Path 133 | total_timesteps 1858.
Path 134 | total_timesteps 1871.
Path 135 | total_timesteps 1885.
Path 136 | total_timesteps 1895.
Path 137 | total_timesteps 1906.
Path 138 | total_timesteps 1939.
Path 139 | total_timesteps 1948.
Path 140 | total_timesteps 1955.
Path 141 | total_timesteps 1969.
Path 142 | total_timesteps 1978.
Path 143 | total_timesteps 1993.
Path 144 | total_timesteps 2003.
Path 145 | total_timesteps 2024.
Path 146 | total_timesteps 2037.
Path 147 | total_timesteps 2046.
Path 148 | total_timesteps 2059.
Path 149 | total_timesteps 2075.
Path 150 | total_timesteps 2082.
Path 151 | total_timesteps 2097.
Path 152 | total_timesteps 2122.
Path 153 | total_timesteps 2142.
Path 154 | total_timesteps 2155.
Path 155 | total_timesteps 2171.
Path 156 | total_timesteps 2178.
Path 157 | total_timesteps 2193.
Path 158 | total_timesteps 2202.
Path 159 | total_timesteps 2215.
Path 160 | total_timesteps 2227.
Path 161 | total_timesteps 2245.
Path 162 | total_timesteps 2262.
Path 163 | total_timesteps 2272.
Path 164 | total_timesteps 2283.
Path 165 | total_timesteps 2293.
Path 166 | total_timesteps 2312.
Path 167 | total_timesteps 2324.
Path 168 | total_timesteps 2337.
Path 169 | total_timesteps 2353.
Path 170 | total_timesteps 2379.
Path 171 | total_timesteps 2388.
Path 172 | total_timesteps 2396.
Path 173 | total_timesteps 2411.
Path 174 | total_timesteps 2421.
Path 175 | total_timesteps 2434.
Path 176 | total_timesteps 2446.
Path 177 | total_timesteps 2459.
Path 178 | total_timesteps 2480.
Path 179 | total_timesteps 2498.
Path 180 | total_timesteps 2512.
Path 181 | total_timesteps 2521.
Path 182 | total_timesteps 2532.
Path 183 | total_timesteps 2547.
Path 184 | total_timesteps 2561.
Path 185 | total_timesteps 2572.
Path 186 | total_timesteps 2586.
Path 187 | total_timesteps 2594.
Path 188 | total_timesteps 2602.
Path 189 | total_timesteps 2621.
Path 190 | total_timesteps 2639.
Path 191 | total_timesteps 2658.
Path 192 | total_timesteps 2667.
Path 193 | total_timesteps 2687.
Path 194 | total_timesteps 2697.
Path 195 | total_timesteps 2711.
Path 196 | total_timesteps 2722.
Path 197 | total_timesteps 2734.
Path 198 | total_timesteps 2745.
Path 199 | total_timesteps 2760.
Path 200 | total_timesteps 2771.
Path 201 | total_timesteps 2785.
Path 202 | total_timesteps 2796.
Path 203 | total_timesteps 2808.
Path 204 | total_timesteps 2817.
Path 205 | total_timesteps 2826.
Path 206 | total_timesteps 2836.
Path 207 | total_timesteps 2853.
Path 208 | total_timesteps 2864.
Path 209 | total_timesteps 2871.
Path 210 | total_timesteps 2885.
Path 211 | total_timesteps 2895.
Path 212 | total_timesteps 2905.
Path 213 | total_timesteps 2918.
Path 214 | total_timesteps 2934.
Path 215 | total_timesteps 2942.
Path 216 | total_timesteps 2957.
Path 217 | total_timesteps 2966.
Path 218 | total_timesteps 2982.
Path 219 | total_timesteps 3004.
Path 220 | total_timesteps 3027.
Path 221 | total_timesteps 3046.
Path 222 | total_timesteps 3056.
Path 223 | total_timesteps 3066.
Path 224 | total_timesteps 3082.
Path 225 | total_timesteps 3099.
Path 226 | total_timesteps 3113.
Path 227 | total_timesteps 3129.
Path 228 | total_timesteps 3140.
Path 229 | total_timesteps 3154.
Path 230 | total_timesteps 3165.
Path 231 | total_timesteps 3176.
Path 232 | total_timesteps 3188.
Path 233 | total_timesteps 3205.
Path 234 | total_timesteps 3219.
Path 235 | total_timesteps 3228.
Path 236 | total_timesteps 3239.
Path 237 | total_timesteps 3254.
Path 238 | total_timesteps 3276.
Path 239 | total_timesteps 3289.
Path 240 | total_timesteps 3300.
Path 241 | total_timesteps 3314.
Path 242 | total_timesteps 3339.
Path 243 | total_timesteps 3353.
Path 244 | total_timesteps 3370.
Path 245 | total_timesteps 3381.
Path 246 | total_timesteps 3395.
Path 247 | total_timesteps 3408.
Path 248 | total_timesteps 3418.
Path 249 | total_timesteps 3428.
Path 250 | total_timesteps 3443.
Path 251 | total_timesteps 3452.
Path 252 | total_timesteps 3464.
Path 253 | total_timesteps 3479.
Path 254 | total_timesteps 3489.
Path 255 | total_timesteps 3498.
Path 256 | total_timesteps 3517.
Path 257 | total_timesteps 3529.
Path 258 | total_timesteps 3540.
Path 259 | total_timesteps 3548.
Path 260 | total_timesteps 3555.
Path 261 | total_timesteps 3572.
Path 262 | total_timesteps 3595.
Path 263 | total_timesteps 3607.
Path 264 | total_timesteps 3621.
Path 265 | total_timesteps 3628.
Path 266 | total_timesteps 3645.
Path 267 | total_timesteps 3651.
Path 268 | total_timesteps 3663.
Path 269 | total_timesteps 3674.
Path 270 | total_timesteps 3689.
Path 271 | total_timesteps 3699.
Path 272 | total_timesteps 3709.
Path 273 | total_timesteps 3721.
Path 274 | total_timesteps 3730.
Path 275 | total_timesteps 3740.
Path 276 | total_timesteps 3756.
Path 277 | total_timesteps 3771.
Path 278 | total_timesteps 3796.
Path 279 | total_timesteps 3805.
Path 280 | total_timesteps 3825.
Path 281 | total_timesteps 3832.
Path 282 | total_timesteps 3843.
Path 283 | total_timesteps 3851.
Path 284 | total_timesteps 3860.
Path 285 | total_timesteps 3873.
Path 286 | total_timesteps 3883.
Path 287 | total_timesteps 3895.
Path 288 | total_timesteps 3909.
Path 289 | total_timesteps 3920.
Path 290 | total_timesteps 3935.
Path 291 | total_timesteps 3947.
Path 292 | total_timesteps 3959.
Path 293 | total_timesteps 3972.
Path 294 | total_timesteps 3986.
Path 295 | total_timesteps 3995.
Path 296 | total_timesteps 4008.
Path 297 | total_timesteps 4021.
Path 298 | total_timesteps 4055.
Path 299 | total_timesteps 4071.
Path 300 | total_timesteps 4081.
Path 301 | total_timesteps 4092.
Path 302 | total_timesteps 4108.
Path 303 | total_timesteps 4126.
Path 304 | total_timesteps 4141.
Path 305 | total_timesteps 4159.
Path 306 | total_timesteps 4177.
Path 307 | total_timesteps 4185.
Path 308 | total_timesteps 4206.
Path 309 | total_timesteps 4220.
Path 310 | total_timesteps 4229.
Path 311 | total_timesteps 4249.
Path 312 | total_timesteps 4261.
Path 313 | total_timesteps 4269.
Path 314 | total_timesteps 4288.
Path 315 | total_timesteps 4303.
Path 316 | total_timesteps 4320.
Path 317 | total_timesteps 4329.
Path 318 | total_timesteps 4338.
Path 319 | total_timesteps 4344.
Path 320 | total_timesteps 4367.
Path 321 | total_timesteps 4380.
Path 322 | total_timesteps 4396.
Path 323 | total_timesteps 4410.
Path 324 | total_timesteps 4427.
Path 325 | total_timesteps 4437.
Path 326 | total_timesteps 4457.
Path 327 | total_timesteps 4480.
Path 328 | total_timesteps 4495.
Path 329 | total_timesteps 4510.
Path 330 | total_timesteps 4530.
Path 331 | total_timesteps 4544.
Path 332 | total_timesteps 4562.
Path 333 | total_timesteps 4587.
Path 334 | total_timesteps 4594.
Path 335 | total_timesteps 4614.
Path 336 | total_timesteps 4623.
Path 337 | total_timesteps 4645.
Path 338 | total_timesteps 4664.
Path 339 | total_timesteps 4672.
Path 340 | total_timesteps 4680.
Path 341 | total_timesteps 4695.
Path 342 | total_timesteps 4706.
Path 343 | total_timesteps 4717.
Path 344 | total_timesteps 4736.
Path 345 | total_timesteps 4754.
Path 346 | total_timesteps 4765.
Path 347 | total_timesteps 4773.
Path 348 | total_timesteps 4785.
Path 349 | total_timesteps 4795.
Path 350 | total_timesteps 4807.
Path 351 | total_timesteps 4815.
Path 352 | total_timesteps 4825.
Path 353 | total_timesteps 4834.
Path 354 | total_timesteps 4855.
Path 355 | total_timesteps 4864.
Path 356 | total_timesteps 4876.
Path 357 | total_timesteps 4889.
Path 358 | total_timesteps 4899.
Path 359 | total_timesteps 4911.
Path 360 | total_timesteps 4922.
Path 361 | total_timesteps 4933.
Path 362 | total_timesteps 4953.
Path 363 | total_timesteps 4964.
Path 364 | total_timesteps 4991.
Path 365 | total_timesteps 5002.
Path 366 | total_timesteps 5020.
Path 367 | total_timesteps 5031.
Path 368 | total_timesteps 5044.
Path 369 | total_timesteps 5054.
Path 370 | total_timesteps 5066.
Path 371 | total_timesteps 5084.
Path 372 | total_timesteps 5094.
Path 373 | total_timesteps 5113.
Path 374 | total_timesteps 5121.
Path 375 | total_timesteps 5133.
Path 376 | total_timesteps 5150.
Path 377 | total_timesteps 5168.
Path 378 | total_timesteps 5188.
Path 379 | total_timesteps 5198.
Path 380 | total_timesteps 5211.
Path 381 | total_timesteps 5229.
Path 382 | total_timesteps 5247.
Path 383 | total_timesteps 5255.
Path 384 | total_timesteps 5266.
Path 385 | total_timesteps 5284.
Path 386 | total_timesteps 5302.
Path 387 | total_timesteps 5315.
Path 388 | total_timesteps 5324.
Path 389 | total_timesteps 5345.
Path 390 | total_timesteps 5353.
Path 391 | total_timesteps 5368.
Path 392 | total_timesteps 5385.
Path 393 | total_timesteps 5397.
Path 394 | total_timesteps 5410.
Path 395 | total_timesteps 5418.
Path 396 | total_timesteps 5429.
Path 397 | total_timesteps 5442.
Path 398 | total_timesteps 5464.
Path 399 | total_timesteps 5473.
Path 400 | total_timesteps 5483.
Path 401 | total_timesteps 5492.
Path 402 | total_timesteps 5501.
Path 403 | total_timesteps 5516.
Path 404 | total_timesteps 5535.
Path 405 | total_timesteps 5545.
Path 406 | total_timesteps 5553.
Path 407 | total_timesteps 5571.
Path 408 | total_timesteps 5583.
Path 409 | total_timesteps 5592.
Path 410 | total_timesteps 5599.
Path 411 | total_timesteps 5616.
Path 412 | total_timesteps 5632.
Path 413 | total_timesteps 5651.
Path 414 | total_timesteps 5668.
Path 415 | total_timesteps 5681.
Path 416 | total_timesteps 5697.
Path 417 | total_timesteps 5713.
Path 418 | total_timesteps 5721.
Path 419 | total_timesteps 5734.
Path 420 | total_timesteps 5746.
Path 421 | total_timesteps 5757.
Path 422 | total_timesteps 5766.
Path 423 | total_timesteps 5781.
Path 424 | total_timesteps 5793.
Path 425 | total_timesteps 5810.
Path 426 | total_timesteps 5818.
Path 427 | total_timesteps 5834.
Path 428 | total_timesteps 5847.
Path 429 | total_timesteps 5862.
Path 430 | total_timesteps 5875.
Path 431 | total_timesteps 5884.
Path 432 | total_timesteps 5893.
Path 433 | total_timesteps 5901.
Path 434 | total_timesteps 5911.
Path 435 | total_timesteps 5945.
Path 436 | total_timesteps 5958.
Path 437 | total_timesteps 5967.
Path 438 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.98    |
| Iteration     | 29       |
| MaximumReturn | 7.23     |
| MinimumReturn | -19.8    |
| TotalSamples  | 124215   |
----------------------------
itr #30 | 
Fitting dynamics.
Validation loss = 0.00509496359154582
Validation loss = 0.004927812609821558
Validation loss = 0.005146203562617302
Validation loss = 0.005350114777684212
Validation loss = 0.005000821780413389
Validation loss = 0.00528895715251565
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 36.
Path 4 | total_timesteps 52.
Path 5 | total_timesteps 67.
Path 6 | total_timesteps 82.
Path 7 | total_timesteps 108.
Path 8 | total_timesteps 124.
Path 9 | total_timesteps 135.
Path 10 | total_timesteps 146.
Path 11 | total_timesteps 158.
Path 12 | total_timesteps 168.
Path 13 | total_timesteps 178.
Path 14 | total_timesteps 185.
Path 15 | total_timesteps 200.
Path 16 | total_timesteps 213.
Path 17 | total_timesteps 223.
Path 18 | total_timesteps 231.
Path 19 | total_timesteps 238.
Path 20 | total_timesteps 255.
Path 21 | total_timesteps 266.
Path 22 | total_timesteps 280.
Path 23 | total_timesteps 298.
Path 24 | total_timesteps 324.
Path 25 | total_timesteps 337.
Path 26 | total_timesteps 355.
Path 27 | total_timesteps 368.
Path 28 | total_timesteps 386.
Path 29 | total_timesteps 394.
Path 30 | total_timesteps 403.
Path 31 | total_timesteps 440.
Path 32 | total_timesteps 449.
Path 33 | total_timesteps 464.
Path 34 | total_timesteps 474.
Path 35 | total_timesteps 490.
Path 36 | total_timesteps 504.
Path 37 | total_timesteps 525.
Path 38 | total_timesteps 536.
Path 39 | total_timesteps 563.
Path 40 | total_timesteps 577.
Path 41 | total_timesteps 596.
Path 42 | total_timesteps 614.
Path 43 | total_timesteps 629.
Path 44 | total_timesteps 641.
Path 45 | total_timesteps 658.
Path 46 | total_timesteps 676.
Path 47 | total_timesteps 689.
Path 48 | total_timesteps 699.
Path 49 | total_timesteps 712.
Path 50 | total_timesteps 737.
Path 51 | total_timesteps 752.
Path 52 | total_timesteps 775.
Path 53 | total_timesteps 786.
Path 54 | total_timesteps 808.
Path 55 | total_timesteps 821.
Path 56 | total_timesteps 835.
Path 57 | total_timesteps 854.
Path 58 | total_timesteps 867.
Path 59 | total_timesteps 874.
Path 60 | total_timesteps 889.
Path 61 | total_timesteps 900.
Path 62 | total_timesteps 910.
Path 63 | total_timesteps 916.
Path 64 | total_timesteps 927.
Path 65 | total_timesteps 937.
Path 66 | total_timesteps 950.
Path 67 | total_timesteps 959.
Path 68 | total_timesteps 969.
Path 69 | total_timesteps 980.
Path 70 | total_timesteps 994.
Path 71 | total_timesteps 1012.
Path 72 | total_timesteps 1026.
Path 73 | total_timesteps 1038.
Path 74 | total_timesteps 1053.
Path 75 | total_timesteps 1060.
Path 76 | total_timesteps 1069.
Path 77 | total_timesteps 1087.
Path 78 | total_timesteps 1096.
Path 79 | total_timesteps 1105.
Path 80 | total_timesteps 1113.
Path 81 | total_timesteps 1121.
Path 82 | total_timesteps 1144.
Path 83 | total_timesteps 1156.
Path 84 | total_timesteps 1167.
Path 85 | total_timesteps 1190.
Path 86 | total_timesteps 1200.
Path 87 | total_timesteps 1212.
Path 88 | total_timesteps 1222.
Path 89 | total_timesteps 1235.
Path 90 | total_timesteps 1248.
Path 91 | total_timesteps 1263.
Path 92 | total_timesteps 1275.
Path 93 | total_timesteps 1284.
Path 94 | total_timesteps 1302.
Path 95 | total_timesteps 1317.
Path 96 | total_timesteps 1331.
Path 97 | total_timesteps 1346.
Path 98 | total_timesteps 1363.
Path 99 | total_timesteps 1379.
Path 100 | total_timesteps 1391.
Path 101 | total_timesteps 1407.
Path 102 | total_timesteps 1424.
Path 103 | total_timesteps 1440.
Path 104 | total_timesteps 1454.
Path 105 | total_timesteps 1462.
Path 106 | total_timesteps 1476.
Path 107 | total_timesteps 1495.
Path 108 | total_timesteps 1507.
Path 109 | total_timesteps 1518.
Path 110 | total_timesteps 1527.
Path 111 | total_timesteps 1539.
Path 112 | total_timesteps 1552.
Path 113 | total_timesteps 1563.
Path 114 | total_timesteps 1571.
Path 115 | total_timesteps 1590.
Path 116 | total_timesteps 1604.
Path 117 | total_timesteps 1615.
Path 118 | total_timesteps 1630.
Path 119 | total_timesteps 1644.
Path 120 | total_timesteps 1650.
Path 121 | total_timesteps 1666.
Path 122 | total_timesteps 1683.
Path 123 | total_timesteps 1693.
Path 124 | total_timesteps 1708.
Path 125 | total_timesteps 1717.
Path 126 | total_timesteps 1727.
Path 127 | total_timesteps 1738.
Path 128 | total_timesteps 1747.
Path 129 | total_timesteps 1760.
Path 130 | total_timesteps 1774.
Path 131 | total_timesteps 1790.
Path 132 | total_timesteps 1810.
Path 133 | total_timesteps 1827.
Path 134 | total_timesteps 1852.
Path 135 | total_timesteps 1862.
Path 136 | total_timesteps 1881.
Path 137 | total_timesteps 1892.
Path 138 | total_timesteps 1901.
Path 139 | total_timesteps 1914.
Path 140 | total_timesteps 1924.
Path 141 | total_timesteps 1940.
Path 142 | total_timesteps 1956.
Path 143 | total_timesteps 1989.
Path 144 | total_timesteps 2006.
Path 145 | total_timesteps 2021.
Path 146 | total_timesteps 2030.
Path 147 | total_timesteps 2042.
Path 148 | total_timesteps 2057.
Path 149 | total_timesteps 2066.
Path 150 | total_timesteps 2090.
Path 151 | total_timesteps 2099.
Path 152 | total_timesteps 2107.
Path 153 | total_timesteps 2116.
Path 154 | total_timesteps 2138.
Path 155 | total_timesteps 2157.
Path 156 | total_timesteps 2164.
Path 157 | total_timesteps 2174.
Path 158 | total_timesteps 2183.
Path 159 | total_timesteps 2205.
Path 160 | total_timesteps 2215.
Path 161 | total_timesteps 2227.
Path 162 | total_timesteps 2240.
Path 163 | total_timesteps 2254.
Path 164 | total_timesteps 2266.
Path 165 | total_timesteps 2274.
Path 166 | total_timesteps 2287.
Path 167 | total_timesteps 2299.
Path 168 | total_timesteps 2314.
Path 169 | total_timesteps 2331.
Path 170 | total_timesteps 2345.
Path 171 | total_timesteps 2361.
Path 172 | total_timesteps 2368.
Path 173 | total_timesteps 2377.
Path 174 | total_timesteps 2393.
Path 175 | total_timesteps 2414.
Path 176 | total_timesteps 2429.
Path 177 | total_timesteps 2443.
Path 178 | total_timesteps 2450.
Path 179 | total_timesteps 2467.
Path 180 | total_timesteps 2478.
Path 181 | total_timesteps 2486.
Path 182 | total_timesteps 2495.
Path 183 | total_timesteps 2507.
Path 184 | total_timesteps 2520.
Path 185 | total_timesteps 2533.
Path 186 | total_timesteps 2556.
Path 187 | total_timesteps 2566.
Path 188 | total_timesteps 2581.
Path 189 | total_timesteps 2591.
Path 190 | total_timesteps 2605.
Path 191 | total_timesteps 2618.
Path 192 | total_timesteps 2640.
Path 193 | total_timesteps 2655.
Path 194 | total_timesteps 2669.
Path 195 | total_timesteps 2680.
Path 196 | total_timesteps 2691.
Path 197 | total_timesteps 2703.
Path 198 | total_timesteps 2721.
Path 199 | total_timesteps 2733.
Path 200 | total_timesteps 2747.
Path 201 | total_timesteps 2754.
Path 202 | total_timesteps 2765.
Path 203 | total_timesteps 2777.
Path 204 | total_timesteps 2792.
Path 205 | total_timesteps 2811.
Path 206 | total_timesteps 2829.
Path 207 | total_timesteps 2847.
Path 208 | total_timesteps 2860.
Path 209 | total_timesteps 2876.
Path 210 | total_timesteps 2888.
Path 211 | total_timesteps 2895.
Path 212 | total_timesteps 2905.
Path 213 | total_timesteps 2925.
Path 214 | total_timesteps 2934.
Path 215 | total_timesteps 2945.
Path 216 | total_timesteps 2957.
Path 217 | total_timesteps 2978.
Path 218 | total_timesteps 2987.
Path 219 | total_timesteps 3005.
Path 220 | total_timesteps 3020.
Path 221 | total_timesteps 3030.
Path 222 | total_timesteps 3046.
Path 223 | total_timesteps 3059.
Path 224 | total_timesteps 3078.
Path 225 | total_timesteps 3089.
Path 226 | total_timesteps 3100.
Path 227 | total_timesteps 3109.
Path 228 | total_timesteps 3126.
Path 229 | total_timesteps 3142.
Path 230 | total_timesteps 3155.
Path 231 | total_timesteps 3165.
Path 232 | total_timesteps 3178.
Path 233 | total_timesteps 3200.
Path 234 | total_timesteps 3212.
Path 235 | total_timesteps 3227.
Path 236 | total_timesteps 3245.
Path 237 | total_timesteps 3258.
Path 238 | total_timesteps 3267.
Path 239 | total_timesteps 3288.
Path 240 | total_timesteps 3302.
Path 241 | total_timesteps 3316.
Path 242 | total_timesteps 3335.
Path 243 | total_timesteps 3344.
Path 244 | total_timesteps 3351.
Path 245 | total_timesteps 3379.
Path 246 | total_timesteps 3392.
Path 247 | total_timesteps 3401.
Path 248 | total_timesteps 3413.
Path 249 | total_timesteps 3427.
Path 250 | total_timesteps 3441.
Path 251 | total_timesteps 3448.
Path 252 | total_timesteps 3456.
Path 253 | total_timesteps 3468.
Path 254 | total_timesteps 3485.
Path 255 | total_timesteps 3497.
Path 256 | total_timesteps 3508.
Path 257 | total_timesteps 3516.
Path 258 | total_timesteps 3533.
Path 259 | total_timesteps 3539.
Path 260 | total_timesteps 3550.
Path 261 | total_timesteps 3560.
Path 262 | total_timesteps 3570.
Path 263 | total_timesteps 3577.
Path 264 | total_timesteps 3593.
Path 265 | total_timesteps 3604.
Path 266 | total_timesteps 3614.
Path 267 | total_timesteps 3624.
Path 268 | total_timesteps 3644.
Path 269 | total_timesteps 3656.
Path 270 | total_timesteps 3669.
Path 271 | total_timesteps 3679.
Path 272 | total_timesteps 3693.
Path 273 | total_timesteps 3708.
Path 274 | total_timesteps 3723.
Path 275 | total_timesteps 3735.
Path 276 | total_timesteps 3748.
Path 277 | total_timesteps 3756.
Path 278 | total_timesteps 3766.
Path 279 | total_timesteps 3791.
Path 280 | total_timesteps 3799.
Path 281 | total_timesteps 3821.
Path 282 | total_timesteps 3839.
Path 283 | total_timesteps 3850.
Path 284 | total_timesteps 3862.
Path 285 | total_timesteps 3879.
Path 286 | total_timesteps 3895.
Path 287 | total_timesteps 3913.
Path 288 | total_timesteps 3925.
Path 289 | total_timesteps 3938.
Path 290 | total_timesteps 3946.
Path 291 | total_timesteps 3959.
Path 292 | total_timesteps 3973.
Path 293 | total_timesteps 3988.
Path 294 | total_timesteps 4003.
Path 295 | total_timesteps 4016.
Path 296 | total_timesteps 4023.
Path 297 | total_timesteps 4038.
Path 298 | total_timesteps 4051.
Path 299 | total_timesteps 4059.
Path 300 | total_timesteps 4068.
Path 301 | total_timesteps 4077.
Path 302 | total_timesteps 4104.
Path 303 | total_timesteps 4118.
Path 304 | total_timesteps 4144.
Path 305 | total_timesteps 4159.
Path 306 | total_timesteps 4173.
Path 307 | total_timesteps 4180.
Path 308 | total_timesteps 4191.
Path 309 | total_timesteps 4213.
Path 310 | total_timesteps 4223.
Path 311 | total_timesteps 4236.
Path 312 | total_timesteps 4254.
Path 313 | total_timesteps 4267.
Path 314 | total_timesteps 4277.
Path 315 | total_timesteps 4294.
Path 316 | total_timesteps 4314.
Path 317 | total_timesteps 4325.
Path 318 | total_timesteps 4337.
Path 319 | total_timesteps 4346.
Path 320 | total_timesteps 4360.
Path 321 | total_timesteps 4368.
Path 322 | total_timesteps 4392.
Path 323 | total_timesteps 4402.
Path 324 | total_timesteps 4409.
Path 325 | total_timesteps 4419.
Path 326 | total_timesteps 4431.
Path 327 | total_timesteps 4440.
Path 328 | total_timesteps 4450.
Path 329 | total_timesteps 4464.
Path 330 | total_timesteps 4481.
Path 331 | total_timesteps 4492.
Path 332 | total_timesteps 4506.
Path 333 | total_timesteps 4528.
Path 334 | total_timesteps 4538.
Path 335 | total_timesteps 4548.
Path 336 | total_timesteps 4562.
Path 337 | total_timesteps 4571.
Path 338 | total_timesteps 4582.
Path 339 | total_timesteps 4607.
Path 340 | total_timesteps 4619.
Path 341 | total_timesteps 4629.
Path 342 | total_timesteps 4644.
Path 343 | total_timesteps 4663.
Path 344 | total_timesteps 4680.
Path 345 | total_timesteps 4690.
Path 346 | total_timesteps 4705.
Path 347 | total_timesteps 4726.
Path 348 | total_timesteps 4739.
Path 349 | total_timesteps 4753.
Path 350 | total_timesteps 4767.
Path 351 | total_timesteps 4774.
Path 352 | total_timesteps 4785.
Path 353 | total_timesteps 4803.
Path 354 | total_timesteps 4810.
Path 355 | total_timesteps 4826.
Path 356 | total_timesteps 4842.
Path 357 | total_timesteps 4848.
Path 358 | total_timesteps 4862.
Path 359 | total_timesteps 4873.
Path 360 | total_timesteps 4879.
Path 361 | total_timesteps 4897.
Path 362 | total_timesteps 4907.
Path 363 | total_timesteps 4923.
Path 364 | total_timesteps 4935.
Path 365 | total_timesteps 4946.
Path 366 | total_timesteps 4964.
Path 367 | total_timesteps 4972.
Path 368 | total_timesteps 4989.
Path 369 | total_timesteps 5000.
Path 370 | total_timesteps 5011.
Path 371 | total_timesteps 5032.
Path 372 | total_timesteps 5040.
Path 373 | total_timesteps 5053.
Path 374 | total_timesteps 5076.
Path 375 | total_timesteps 5087.
Path 376 | total_timesteps 5101.
Path 377 | total_timesteps 5114.
Path 378 | total_timesteps 5124.
Path 379 | total_timesteps 5144.
Path 380 | total_timesteps 5151.
Path 381 | total_timesteps 5164.
Path 382 | total_timesteps 5175.
Path 383 | total_timesteps 5186.
Path 384 | total_timesteps 5197.
Path 385 | total_timesteps 5207.
Path 386 | total_timesteps 5220.
Path 387 | total_timesteps 5228.
Path 388 | total_timesteps 5250.
Path 389 | total_timesteps 5260.
Path 390 | total_timesteps 5275.
Path 391 | total_timesteps 5288.
Path 392 | total_timesteps 5300.
Path 393 | total_timesteps 5316.
Path 394 | total_timesteps 5330.
Path 395 | total_timesteps 5343.
Path 396 | total_timesteps 5358.
Path 397 | total_timesteps 5368.
Path 398 | total_timesteps 5376.
Path 399 | total_timesteps 5387.
Path 400 | total_timesteps 5399.
Path 401 | total_timesteps 5412.
Path 402 | total_timesteps 5427.
Path 403 | total_timesteps 5443.
Path 404 | total_timesteps 5453.
Path 405 | total_timesteps 5460.
Path 406 | total_timesteps 5469.
Path 407 | total_timesteps 5481.
Path 408 | total_timesteps 5497.
Path 409 | total_timesteps 5511.
Path 410 | total_timesteps 5524.
Path 411 | total_timesteps 5534.
Path 412 | total_timesteps 5541.
Path 413 | total_timesteps 5551.
Path 414 | total_timesteps 5572.
Path 415 | total_timesteps 5585.
Path 416 | total_timesteps 5603.
Path 417 | total_timesteps 5613.
Path 418 | total_timesteps 5633.
Path 419 | total_timesteps 5649.
Path 420 | total_timesteps 5667.
Path 421 | total_timesteps 5678.
Path 422 | total_timesteps 5689.
Path 423 | total_timesteps 5697.
Path 424 | total_timesteps 5713.
Path 425 | total_timesteps 5724.
Path 426 | total_timesteps 5737.
Path 427 | total_timesteps 5749.
Path 428 | total_timesteps 5763.
Path 429 | total_timesteps 5774.
Path 430 | total_timesteps 5788.
Path 431 | total_timesteps 5806.
Path 432 | total_timesteps 5815.
Path 433 | total_timesteps 5827.
Path 434 | total_timesteps 5838.
Path 435 | total_timesteps 5847.
Path 436 | total_timesteps 5854.
Path 437 | total_timesteps 5864.
Path 438 | total_timesteps 5874.
Path 439 | total_timesteps 5886.
Path 440 | total_timesteps 5900.
Path 441 | total_timesteps 5907.
Path 442 | total_timesteps 5920.
Path 443 | total_timesteps 5934.
Path 444 | total_timesteps 5952.
Path 445 | total_timesteps 5969.
Path 446 | total_timesteps 5980.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.95    |
| Iteration     | 30       |
| MaximumReturn | 3.17     |
| MinimumReturn | -20.8    |
| TotalSamples  | 128215   |
----------------------------
itr #31 | 
Fitting dynamics.
Validation loss = 0.0054351575672626495
Validation loss = 0.005103978328406811
Validation loss = 0.005209116730839014
Validation loss = 0.004948184825479984
Validation loss = 0.005024871788918972
Validation loss = 0.004972315393388271
Validation loss = 0.00530895683914423
Validation loss = 0.004964541178196669
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 21.
Path 2 | total_timesteps 36.
Path 3 | total_timesteps 45.
Path 4 | total_timesteps 59.
Path 5 | total_timesteps 71.
Path 6 | total_timesteps 90.
Path 7 | total_timesteps 104.
Path 8 | total_timesteps 116.
Path 9 | total_timesteps 132.
Path 10 | total_timesteps 142.
Path 11 | total_timesteps 151.
Path 12 | total_timesteps 171.
Path 13 | total_timesteps 186.
Path 14 | total_timesteps 214.
Path 15 | total_timesteps 226.
Path 16 | total_timesteps 239.
Path 17 | total_timesteps 252.
Path 18 | total_timesteps 275.
Path 19 | total_timesteps 289.
Path 20 | total_timesteps 299.
Path 21 | total_timesteps 310.
Path 22 | total_timesteps 321.
Path 23 | total_timesteps 342.
Path 24 | total_timesteps 351.
Path 25 | total_timesteps 367.
Path 26 | total_timesteps 381.
Path 27 | total_timesteps 395.
Path 28 | total_timesteps 408.
Path 29 | total_timesteps 418.
Path 30 | total_timesteps 434.
Path 31 | total_timesteps 446.
Path 32 | total_timesteps 461.
Path 33 | total_timesteps 471.
Path 34 | total_timesteps 488.
Path 35 | total_timesteps 498.
Path 36 | total_timesteps 514.
Path 37 | total_timesteps 530.
Path 38 | total_timesteps 539.
Path 39 | total_timesteps 548.
Path 40 | total_timesteps 568.
Path 41 | total_timesteps 579.
Path 42 | total_timesteps 592.
Path 43 | total_timesteps 603.
Path 44 | total_timesteps 620.
Path 45 | total_timesteps 635.
Path 46 | total_timesteps 648.
Path 47 | total_timesteps 662.
Path 48 | total_timesteps 680.
Path 49 | total_timesteps 691.
Path 50 | total_timesteps 702.
Path 51 | total_timesteps 710.
Path 52 | total_timesteps 727.
Path 53 | total_timesteps 744.
Path 54 | total_timesteps 755.
Path 55 | total_timesteps 763.
Path 56 | total_timesteps 773.
Path 57 | total_timesteps 783.
Path 58 | total_timesteps 805.
Path 59 | total_timesteps 823.
Path 60 | total_timesteps 836.
Path 61 | total_timesteps 850.
Path 62 | total_timesteps 859.
Path 63 | total_timesteps 868.
Path 64 | total_timesteps 879.
Path 65 | total_timesteps 904.
Path 66 | total_timesteps 917.
Path 67 | total_timesteps 926.
Path 68 | total_timesteps 942.
Path 69 | total_timesteps 952.
Path 70 | total_timesteps 963.
Path 71 | total_timesteps 975.
Path 72 | total_timesteps 988.
Path 73 | total_timesteps 1000.
Path 74 | total_timesteps 1013.
Path 75 | total_timesteps 1026.
Path 76 | total_timesteps 1039.
Path 77 | total_timesteps 1058.
Path 78 | total_timesteps 1072.
Path 79 | total_timesteps 1085.
Path 80 | total_timesteps 1099.
Path 81 | total_timesteps 1108.
Path 82 | total_timesteps 1134.
Path 83 | total_timesteps 1155.
Path 84 | total_timesteps 1174.
Path 85 | total_timesteps 1181.
Path 86 | total_timesteps 1193.
Path 87 | total_timesteps 1205.
Path 88 | total_timesteps 1226.
Path 89 | total_timesteps 1235.
Path 90 | total_timesteps 1255.
Path 91 | total_timesteps 1267.
Path 92 | total_timesteps 1277.
Path 93 | total_timesteps 1294.
Path 94 | total_timesteps 1304.
Path 95 | total_timesteps 1316.
Path 96 | total_timesteps 1339.
Path 97 | total_timesteps 1351.
Path 98 | total_timesteps 1372.
Path 99 | total_timesteps 1385.
Path 100 | total_timesteps 1392.
Path 101 | total_timesteps 1413.
Path 102 | total_timesteps 1429.
Path 103 | total_timesteps 1452.
Path 104 | total_timesteps 1461.
Path 105 | total_timesteps 1476.
Path 106 | total_timesteps 1483.
Path 107 | total_timesteps 1505.
Path 108 | total_timesteps 1525.
Path 109 | total_timesteps 1532.
Path 110 | total_timesteps 1553.
Path 111 | total_timesteps 1570.
Path 112 | total_timesteps 1582.
Path 113 | total_timesteps 1600.
Path 114 | total_timesteps 1613.
Path 115 | total_timesteps 1637.
Path 116 | total_timesteps 1647.
Path 117 | total_timesteps 1664.
Path 118 | total_timesteps 1673.
Path 119 | total_timesteps 1683.
Path 120 | total_timesteps 1699.
Path 121 | total_timesteps 1712.
Path 122 | total_timesteps 1721.
Path 123 | total_timesteps 1731.
Path 124 | total_timesteps 1741.
Path 125 | total_timesteps 1755.
Path 126 | total_timesteps 1777.
Path 127 | total_timesteps 1785.
Path 128 | total_timesteps 1792.
Path 129 | total_timesteps 1813.
Path 130 | total_timesteps 1822.
Path 131 | total_timesteps 1832.
Path 132 | total_timesteps 1844.
Path 133 | total_timesteps 1856.
Path 134 | total_timesteps 1869.
Path 135 | total_timesteps 1881.
Path 136 | total_timesteps 1894.
Path 137 | total_timesteps 1905.
Path 138 | total_timesteps 1920.
Path 139 | total_timesteps 1932.
Path 140 | total_timesteps 1953.
Path 141 | total_timesteps 1969.
Path 142 | total_timesteps 1980.
Path 143 | total_timesteps 1998.
Path 144 | total_timesteps 2021.
Path 145 | total_timesteps 2042.
Path 146 | total_timesteps 2058.
Path 147 | total_timesteps 2066.
Path 148 | total_timesteps 2077.
Path 149 | total_timesteps 2097.
Path 150 | total_timesteps 2123.
Path 151 | total_timesteps 2137.
Path 152 | total_timesteps 2158.
Path 153 | total_timesteps 2170.
Path 154 | total_timesteps 2183.
Path 155 | total_timesteps 2197.
Path 156 | total_timesteps 2212.
Path 157 | total_timesteps 2230.
Path 158 | total_timesteps 2252.
Path 159 | total_timesteps 2264.
Path 160 | total_timesteps 2281.
Path 161 | total_timesteps 2295.
Path 162 | total_timesteps 2312.
Path 163 | total_timesteps 2329.
Path 164 | total_timesteps 2342.
Path 165 | total_timesteps 2356.
Path 166 | total_timesteps 2368.
Path 167 | total_timesteps 2375.
Path 168 | total_timesteps 2393.
Path 169 | total_timesteps 2406.
Path 170 | total_timesteps 2419.
Path 171 | total_timesteps 2434.
Path 172 | total_timesteps 2455.
Path 173 | total_timesteps 2473.
Path 174 | total_timesteps 2485.
Path 175 | total_timesteps 2494.
Path 176 | total_timesteps 2506.
Path 177 | total_timesteps 2523.
Path 178 | total_timesteps 2535.
Path 179 | total_timesteps 2546.
Path 180 | total_timesteps 2565.
Path 181 | total_timesteps 2572.
Path 182 | total_timesteps 2593.
Path 183 | total_timesteps 2607.
Path 184 | total_timesteps 2628.
Path 185 | total_timesteps 2641.
Path 186 | total_timesteps 2652.
Path 187 | total_timesteps 2681.
Path 188 | total_timesteps 2694.
Path 189 | total_timesteps 2703.
Path 190 | total_timesteps 2712.
Path 191 | total_timesteps 2720.
Path 192 | total_timesteps 2741.
Path 193 | total_timesteps 2754.
Path 194 | total_timesteps 2761.
Path 195 | total_timesteps 2775.
Path 196 | total_timesteps 2785.
Path 197 | total_timesteps 2796.
Path 198 | total_timesteps 2806.
Path 199 | total_timesteps 2816.
Path 200 | total_timesteps 2833.
Path 201 | total_timesteps 2846.
Path 202 | total_timesteps 2865.
Path 203 | total_timesteps 2874.
Path 204 | total_timesteps 2883.
Path 205 | total_timesteps 2893.
Path 206 | total_timesteps 2903.
Path 207 | total_timesteps 2916.
Path 208 | total_timesteps 2938.
Path 209 | total_timesteps 2950.
Path 210 | total_timesteps 2965.
Path 211 | total_timesteps 2978.
Path 212 | total_timesteps 2999.
Path 213 | total_timesteps 3011.
Path 214 | total_timesteps 3031.
Path 215 | total_timesteps 3045.
Path 216 | total_timesteps 3052.
Path 217 | total_timesteps 3066.
Path 218 | total_timesteps 3080.
Path 219 | total_timesteps 3088.
Path 220 | total_timesteps 3104.
Path 221 | total_timesteps 3116.
Path 222 | total_timesteps 3123.
Path 223 | total_timesteps 3139.
Path 224 | total_timesteps 3147.
Path 225 | total_timesteps 3157.
Path 226 | total_timesteps 3172.
Path 227 | total_timesteps 3181.
Path 228 | total_timesteps 3193.
Path 229 | total_timesteps 3206.
Path 230 | total_timesteps 3212.
Path 231 | total_timesteps 3220.
Path 232 | total_timesteps 3232.
Path 233 | total_timesteps 3246.
Path 234 | total_timesteps 3261.
Path 235 | total_timesteps 3276.
Path 236 | total_timesteps 3286.
Path 237 | total_timesteps 3311.
Path 238 | total_timesteps 3327.
Path 239 | total_timesteps 3345.
Path 240 | total_timesteps 3356.
Path 241 | total_timesteps 3378.
Path 242 | total_timesteps 3395.
Path 243 | total_timesteps 3405.
Path 244 | total_timesteps 3415.
Path 245 | total_timesteps 3428.
Path 246 | total_timesteps 3442.
Path 247 | total_timesteps 3468.
Path 248 | total_timesteps 3484.
Path 249 | total_timesteps 3497.
Path 250 | total_timesteps 3506.
Path 251 | total_timesteps 3515.
Path 252 | total_timesteps 3545.
Path 253 | total_timesteps 3554.
Path 254 | total_timesteps 3577.
Path 255 | total_timesteps 3591.
Path 256 | total_timesteps 3612.
Path 257 | total_timesteps 3620.
Path 258 | total_timesteps 3632.
Path 259 | total_timesteps 3646.
Path 260 | total_timesteps 3653.
Path 261 | total_timesteps 3679.
Path 262 | total_timesteps 3693.
Path 263 | total_timesteps 3701.
Path 264 | total_timesteps 3713.
Path 265 | total_timesteps 3727.
Path 266 | total_timesteps 3736.
Path 267 | total_timesteps 3746.
Path 268 | total_timesteps 3755.
Path 269 | total_timesteps 3766.
Path 270 | total_timesteps 3778.
Path 271 | total_timesteps 3791.
Path 272 | total_timesteps 3800.
Path 273 | total_timesteps 3816.
Path 274 | total_timesteps 3826.
Path 275 | total_timesteps 3834.
Path 276 | total_timesteps 3848.
Path 277 | total_timesteps 3864.
Path 278 | total_timesteps 3876.
Path 279 | total_timesteps 3887.
Path 280 | total_timesteps 3903.
Path 281 | total_timesteps 3913.
Path 282 | total_timesteps 3928.
Path 283 | total_timesteps 3937.
Path 284 | total_timesteps 3950.
Path 285 | total_timesteps 3963.
Path 286 | total_timesteps 3972.
Path 287 | total_timesteps 3983.
Path 288 | total_timesteps 3991.
Path 289 | total_timesteps 4001.
Path 290 | total_timesteps 4017.
Path 291 | total_timesteps 4025.
Path 292 | total_timesteps 4041.
Path 293 | total_timesteps 4057.
Path 294 | total_timesteps 4070.
Path 295 | total_timesteps 4082.
Path 296 | total_timesteps 4105.
Path 297 | total_timesteps 4118.
Path 298 | total_timesteps 4128.
Path 299 | total_timesteps 4138.
Path 300 | total_timesteps 4152.
Path 301 | total_timesteps 4171.
Path 302 | total_timesteps 4188.
Path 303 | total_timesteps 4204.
Path 304 | total_timesteps 4212.
Path 305 | total_timesteps 4224.
Path 306 | total_timesteps 4247.
Path 307 | total_timesteps 4255.
Path 308 | total_timesteps 4271.
Path 309 | total_timesteps 4280.
Path 310 | total_timesteps 4293.
Path 311 | total_timesteps 4300.
Path 312 | total_timesteps 4314.
Path 313 | total_timesteps 4327.
Path 314 | total_timesteps 4337.
Path 315 | total_timesteps 4347.
Path 316 | total_timesteps 4360.
Path 317 | total_timesteps 4374.
Path 318 | total_timesteps 4385.
Path 319 | total_timesteps 4396.
Path 320 | total_timesteps 4411.
Path 321 | total_timesteps 4420.
Path 322 | total_timesteps 4441.
Path 323 | total_timesteps 4458.
Path 324 | total_timesteps 4473.
Path 325 | total_timesteps 4486.
Path 326 | total_timesteps 4501.
Path 327 | total_timesteps 4530.
Path 328 | total_timesteps 4542.
Path 329 | total_timesteps 4552.
Path 330 | total_timesteps 4559.
Path 331 | total_timesteps 4567.
Path 332 | total_timesteps 4576.
Path 333 | total_timesteps 4588.
Path 334 | total_timesteps 4598.
Path 335 | total_timesteps 4616.
Path 336 | total_timesteps 4637.
Path 337 | total_timesteps 4652.
Path 338 | total_timesteps 4662.
Path 339 | total_timesteps 4675.
Path 340 | total_timesteps 4687.
Path 341 | total_timesteps 4698.
Path 342 | total_timesteps 4715.
Path 343 | total_timesteps 4725.
Path 344 | total_timesteps 4743.
Path 345 | total_timesteps 4755.
Path 346 | total_timesteps 4771.
Path 347 | total_timesteps 4801.
Path 348 | total_timesteps 4810.
Path 349 | total_timesteps 4826.
Path 350 | total_timesteps 4839.
Path 351 | total_timesteps 4851.
Path 352 | total_timesteps 4860.
Path 353 | total_timesteps 4883.
Path 354 | total_timesteps 4901.
Path 355 | total_timesteps 4913.
Path 356 | total_timesteps 4922.
Path 357 | total_timesteps 4932.
Path 358 | total_timesteps 4949.
Path 359 | total_timesteps 4966.
Path 360 | total_timesteps 4986.
Path 361 | total_timesteps 4994.
Path 362 | total_timesteps 5010.
Path 363 | total_timesteps 5020.
Path 364 | total_timesteps 5051.
Path 365 | total_timesteps 5072.
Path 366 | total_timesteps 5086.
Path 367 | total_timesteps 5095.
Path 368 | total_timesteps 5102.
Path 369 | total_timesteps 5116.
Path 370 | total_timesteps 5124.
Path 371 | total_timesteps 5148.
Path 372 | total_timesteps 5156.
Path 373 | total_timesteps 5164.
Path 374 | total_timesteps 5174.
Path 375 | total_timesteps 5185.
Path 376 | total_timesteps 5205.
Path 377 | total_timesteps 5213.
Path 378 | total_timesteps 5222.
Path 379 | total_timesteps 5238.
Path 380 | total_timesteps 5254.
Path 381 | total_timesteps 5266.
Path 382 | total_timesteps 5279.
Path 383 | total_timesteps 5297.
Path 384 | total_timesteps 5306.
Path 385 | total_timesteps 5318.
Path 386 | total_timesteps 5327.
Path 387 | total_timesteps 5344.
Path 388 | total_timesteps 5355.
Path 389 | total_timesteps 5375.
Path 390 | total_timesteps 5391.
Path 391 | total_timesteps 5410.
Path 392 | total_timesteps 5422.
Path 393 | total_timesteps 5437.
Path 394 | total_timesteps 5450.
Path 395 | total_timesteps 5459.
Path 396 | total_timesteps 5477.
Path 397 | total_timesteps 5483.
Path 398 | total_timesteps 5494.
Path 399 | total_timesteps 5501.
Path 400 | total_timesteps 5515.
Path 401 | total_timesteps 5526.
Path 402 | total_timesteps 5539.
Path 403 | total_timesteps 5556.
Path 404 | total_timesteps 5576.
Path 405 | total_timesteps 5586.
Path 406 | total_timesteps 5594.
Path 407 | total_timesteps 5614.
Path 408 | total_timesteps 5625.
Path 409 | total_timesteps 5638.
Path 410 | total_timesteps 5650.
Path 411 | total_timesteps 5659.
Path 412 | total_timesteps 5667.
Path 413 | total_timesteps 5677.
Path 414 | total_timesteps 5686.
Path 415 | total_timesteps 5696.
Path 416 | total_timesteps 5715.
Path 417 | total_timesteps 5729.
Path 418 | total_timesteps 5738.
Path 419 | total_timesteps 5748.
Path 420 | total_timesteps 5769.
Path 421 | total_timesteps 5782.
Path 422 | total_timesteps 5795.
Path 423 | total_timesteps 5812.
Path 424 | total_timesteps 5830.
Path 425 | total_timesteps 5846.
Path 426 | total_timesteps 5868.
Path 427 | total_timesteps 5887.
Path 428 | total_timesteps 5899.
Path 429 | total_timesteps 5911.
Path 430 | total_timesteps 5925.
Path 431 | total_timesteps 5945.
Path 432 | total_timesteps 5963.
Path 433 | total_timesteps 5980.
Path 434 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.04    |
| Iteration     | 31       |
| MaximumReturn | 1.21     |
| MinimumReturn | -21      |
| TotalSamples  | 132216   |
----------------------------
itr #32 | 
Fitting dynamics.
Validation loss = 0.0050393203273415565
Validation loss = 0.004789926111698151
Validation loss = 0.0047700973227620125
Validation loss = 0.005205737426877022
Validation loss = 0.004790400620549917
Validation loss = 0.004902902990579605
Validation loss = 0.004832176491618156
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 31.
Path 4 | total_timesteps 47.
Path 5 | total_timesteps 56.
Path 6 | total_timesteps 69.
Path 7 | total_timesteps 81.
Path 8 | total_timesteps 96.
Path 9 | total_timesteps 105.
Path 10 | total_timesteps 112.
Path 11 | total_timesteps 126.
Path 12 | total_timesteps 133.
Path 13 | total_timesteps 149.
Path 14 | total_timesteps 165.
Path 15 | total_timesteps 173.
Path 16 | total_timesteps 182.
Path 17 | total_timesteps 204.
Path 18 | total_timesteps 213.
Path 19 | total_timesteps 223.
Path 20 | total_timesteps 235.
Path 21 | total_timesteps 246.
Path 22 | total_timesteps 260.
Path 23 | total_timesteps 269.
Path 24 | total_timesteps 286.
Path 25 | total_timesteps 293.
Path 26 | total_timesteps 304.
Path 27 | total_timesteps 321.
Path 28 | total_timesteps 331.
Path 29 | total_timesteps 341.
Path 30 | total_timesteps 350.
Path 31 | total_timesteps 363.
Path 32 | total_timesteps 372.
Path 33 | total_timesteps 387.
Path 34 | total_timesteps 401.
Path 35 | total_timesteps 411.
Path 36 | total_timesteps 422.
Path 37 | total_timesteps 447.
Path 38 | total_timesteps 455.
Path 39 | total_timesteps 474.
Path 40 | total_timesteps 486.
Path 41 | total_timesteps 494.
Path 42 | total_timesteps 510.
Path 43 | total_timesteps 526.
Path 44 | total_timesteps 540.
Path 45 | total_timesteps 557.
Path 46 | total_timesteps 566.
Path 47 | total_timesteps 577.
Path 48 | total_timesteps 586.
Path 49 | total_timesteps 602.
Path 50 | total_timesteps 611.
Path 51 | total_timesteps 620.
Path 52 | total_timesteps 629.
Path 53 | total_timesteps 648.
Path 54 | total_timesteps 659.
Path 55 | total_timesteps 669.
Path 56 | total_timesteps 679.
Path 57 | total_timesteps 694.
Path 58 | total_timesteps 708.
Path 59 | total_timesteps 729.
Path 60 | total_timesteps 739.
Path 61 | total_timesteps 747.
Path 62 | total_timesteps 756.
Path 63 | total_timesteps 767.
Path 64 | total_timesteps 776.
Path 65 | total_timesteps 789.
Path 66 | total_timesteps 801.
Path 67 | total_timesteps 808.
Path 68 | total_timesteps 829.
Path 69 | total_timesteps 853.
Path 70 | total_timesteps 863.
Path 71 | total_timesteps 879.
Path 72 | total_timesteps 892.
Path 73 | total_timesteps 905.
Path 74 | total_timesteps 916.
Path 75 | total_timesteps 933.
Path 76 | total_timesteps 940.
Path 77 | total_timesteps 953.
Path 78 | total_timesteps 963.
Path 79 | total_timesteps 974.
Path 80 | total_timesteps 983.
Path 81 | total_timesteps 1001.
Path 82 | total_timesteps 1014.
Path 83 | total_timesteps 1021.
Path 84 | total_timesteps 1036.
Path 85 | total_timesteps 1055.
Path 86 | total_timesteps 1067.
Path 87 | total_timesteps 1080.
Path 88 | total_timesteps 1096.
Path 89 | total_timesteps 1106.
Path 90 | total_timesteps 1113.
Path 91 | total_timesteps 1125.
Path 92 | total_timesteps 1140.
Path 93 | total_timesteps 1150.
Path 94 | total_timesteps 1160.
Path 95 | total_timesteps 1171.
Path 96 | total_timesteps 1181.
Path 97 | total_timesteps 1188.
Path 98 | total_timesteps 1203.
Path 99 | total_timesteps 1217.
Path 100 | total_timesteps 1227.
Path 101 | total_timesteps 1244.
Path 102 | total_timesteps 1255.
Path 103 | total_timesteps 1265.
Path 104 | total_timesteps 1276.
Path 105 | total_timesteps 1290.
Path 106 | total_timesteps 1307.
Path 107 | total_timesteps 1318.
Path 108 | total_timesteps 1330.
Path 109 | total_timesteps 1350.
Path 110 | total_timesteps 1357.
Path 111 | total_timesteps 1368.
Path 112 | total_timesteps 1378.
Path 113 | total_timesteps 1386.
Path 114 | total_timesteps 1394.
Path 115 | total_timesteps 1402.
Path 116 | total_timesteps 1409.
Path 117 | total_timesteps 1421.
Path 118 | total_timesteps 1428.
Path 119 | total_timesteps 1442.
Path 120 | total_timesteps 1457.
Path 121 | total_timesteps 1472.
Path 122 | total_timesteps 1482.
Path 123 | total_timesteps 1502.
Path 124 | total_timesteps 1509.
Path 125 | total_timesteps 1516.
Path 126 | total_timesteps 1527.
Path 127 | total_timesteps 1537.
Path 128 | total_timesteps 1558.
Path 129 | total_timesteps 1566.
Path 130 | total_timesteps 1581.
Path 131 | total_timesteps 1601.
Path 132 | total_timesteps 1620.
Path 133 | total_timesteps 1636.
Path 134 | total_timesteps 1645.
Path 135 | total_timesteps 1659.
Path 136 | total_timesteps 1668.
Path 137 | total_timesteps 1680.
Path 138 | total_timesteps 1689.
Path 139 | total_timesteps 1702.
Path 140 | total_timesteps 1711.
Path 141 | total_timesteps 1724.
Path 142 | total_timesteps 1735.
Path 143 | total_timesteps 1750.
Path 144 | total_timesteps 1759.
Path 145 | total_timesteps 1775.
Path 146 | total_timesteps 1790.
Path 147 | total_timesteps 1801.
Path 148 | total_timesteps 1813.
Path 149 | total_timesteps 1822.
Path 150 | total_timesteps 1834.
Path 151 | total_timesteps 1853.
Path 152 | total_timesteps 1863.
Path 153 | total_timesteps 1872.
Path 154 | total_timesteps 1885.
Path 155 | total_timesteps 1893.
Path 156 | total_timesteps 1902.
Path 157 | total_timesteps 1914.
Path 158 | total_timesteps 1923.
Path 159 | total_timesteps 1932.
Path 160 | total_timesteps 1942.
Path 161 | total_timesteps 1955.
Path 162 | total_timesteps 1971.
Path 163 | total_timesteps 1987.
Path 164 | total_timesteps 2009.
Path 165 | total_timesteps 2020.
Path 166 | total_timesteps 2030.
Path 167 | total_timesteps 2045.
Path 168 | total_timesteps 2054.
Path 169 | total_timesteps 2063.
Path 170 | total_timesteps 2074.
Path 171 | total_timesteps 2089.
Path 172 | total_timesteps 2106.
Path 173 | total_timesteps 2121.
Path 174 | total_timesteps 2131.
Path 175 | total_timesteps 2142.
Path 176 | total_timesteps 2153.
Path 177 | total_timesteps 2166.
Path 178 | total_timesteps 2174.
Path 179 | total_timesteps 2186.
Path 180 | total_timesteps 2196.
Path 181 | total_timesteps 2210.
Path 182 | total_timesteps 2222.
Path 183 | total_timesteps 2262.
Path 184 | total_timesteps 2272.
Path 185 | total_timesteps 2283.
Path 186 | total_timesteps 2295.
Path 187 | total_timesteps 2318.
Path 188 | total_timesteps 2332.
Path 189 | total_timesteps 2342.
Path 190 | total_timesteps 2360.
Path 191 | total_timesteps 2368.
Path 192 | total_timesteps 2385.
Path 193 | total_timesteps 2401.
Path 194 | total_timesteps 2411.
Path 195 | total_timesteps 2424.
Path 196 | total_timesteps 2436.
Path 197 | total_timesteps 2446.
Path 198 | total_timesteps 2456.
Path 199 | total_timesteps 2470.
Path 200 | total_timesteps 2488.
Path 201 | total_timesteps 2496.
Path 202 | total_timesteps 2507.
Path 203 | total_timesteps 2522.
Path 204 | total_timesteps 2539.
Path 205 | total_timesteps 2547.
Path 206 | total_timesteps 2558.
Path 207 | total_timesteps 2575.
Path 208 | total_timesteps 2587.
Path 209 | total_timesteps 2603.
Path 210 | total_timesteps 2613.
Path 211 | total_timesteps 2621.
Path 212 | total_timesteps 2637.
Path 213 | total_timesteps 2648.
Path 214 | total_timesteps 2659.
Path 215 | total_timesteps 2666.
Path 216 | total_timesteps 2677.
Path 217 | total_timesteps 2688.
Path 218 | total_timesteps 2719.
Path 219 | total_timesteps 2732.
Path 220 | total_timesteps 2744.
Path 221 | total_timesteps 2779.
Path 222 | total_timesteps 2792.
Path 223 | total_timesteps 2808.
Path 224 | total_timesteps 2821.
Path 225 | total_timesteps 2833.
Path 226 | total_timesteps 2841.
Path 227 | total_timesteps 2862.
Path 228 | total_timesteps 2878.
Path 229 | total_timesteps 2892.
Path 230 | total_timesteps 2903.
Path 231 | total_timesteps 2920.
Path 232 | total_timesteps 2932.
Path 233 | total_timesteps 2942.
Path 234 | total_timesteps 2960.
Path 235 | total_timesteps 2972.
Path 236 | total_timesteps 2983.
Path 237 | total_timesteps 2994.
Path 238 | total_timesteps 3001.
Path 239 | total_timesteps 3011.
Path 240 | total_timesteps 3024.
Path 241 | total_timesteps 3045.
Path 242 | total_timesteps 3057.
Path 243 | total_timesteps 3066.
Path 244 | total_timesteps 3074.
Path 245 | total_timesteps 3098.
Path 246 | total_timesteps 3125.
Path 247 | total_timesteps 3146.
Path 248 | total_timesteps 3156.
Path 249 | total_timesteps 3167.
Path 250 | total_timesteps 3181.
Path 251 | total_timesteps 3197.
Path 252 | total_timesteps 3208.
Path 253 | total_timesteps 3217.
Path 254 | total_timesteps 3227.
Path 255 | total_timesteps 3242.
Path 256 | total_timesteps 3253.
Path 257 | total_timesteps 3263.
Path 258 | total_timesteps 3275.
Path 259 | total_timesteps 3293.
Path 260 | total_timesteps 3303.
Path 261 | total_timesteps 3311.
Path 262 | total_timesteps 3319.
Path 263 | total_timesteps 3331.
Path 264 | total_timesteps 3345.
Path 265 | total_timesteps 3362.
Path 266 | total_timesteps 3387.
Path 267 | total_timesteps 3395.
Path 268 | total_timesteps 3403.
Path 269 | total_timesteps 3412.
Path 270 | total_timesteps 3423.
Path 271 | total_timesteps 3435.
Path 272 | total_timesteps 3449.
Path 273 | total_timesteps 3462.
Path 274 | total_timesteps 3471.
Path 275 | total_timesteps 3486.
Path 276 | total_timesteps 3497.
Path 277 | total_timesteps 3507.
Path 278 | total_timesteps 3522.
Path 279 | total_timesteps 3541.
Path 280 | total_timesteps 3548.
Path 281 | total_timesteps 3563.
Path 282 | total_timesteps 3575.
Path 283 | total_timesteps 3592.
Path 284 | total_timesteps 3607.
Path 285 | total_timesteps 3614.
Path 286 | total_timesteps 3632.
Path 287 | total_timesteps 3646.
Path 288 | total_timesteps 3661.
Path 289 | total_timesteps 3674.
Path 290 | total_timesteps 3683.
Path 291 | total_timesteps 3690.
Path 292 | total_timesteps 3712.
Path 293 | total_timesteps 3722.
Path 294 | total_timesteps 3738.
Path 295 | total_timesteps 3747.
Path 296 | total_timesteps 3765.
Path 297 | total_timesteps 3783.
Path 298 | total_timesteps 3790.
Path 299 | total_timesteps 3806.
Path 300 | total_timesteps 3813.
Path 301 | total_timesteps 3823.
Path 302 | total_timesteps 3831.
Path 303 | total_timesteps 3848.
Path 304 | total_timesteps 3865.
Path 305 | total_timesteps 3890.
Path 306 | total_timesteps 3900.
Path 307 | total_timesteps 3912.
Path 308 | total_timesteps 3924.
Path 309 | total_timesteps 3933.
Path 310 | total_timesteps 3942.
Path 311 | total_timesteps 3959.
Path 312 | total_timesteps 3970.
Path 313 | total_timesteps 3993.
Path 314 | total_timesteps 4001.
Path 315 | total_timesteps 4009.
Path 316 | total_timesteps 4024.
Path 317 | total_timesteps 4034.
Path 318 | total_timesteps 4041.
Path 319 | total_timesteps 4065.
Path 320 | total_timesteps 4075.
Path 321 | total_timesteps 4087.
Path 322 | total_timesteps 4095.
Path 323 | total_timesteps 4107.
Path 324 | total_timesteps 4123.
Path 325 | total_timesteps 4133.
Path 326 | total_timesteps 4142.
Path 327 | total_timesteps 4154.
Path 328 | total_timesteps 4166.
Path 329 | total_timesteps 4188.
Path 330 | total_timesteps 4203.
Path 331 | total_timesteps 4214.
Path 332 | total_timesteps 4223.
Path 333 | total_timesteps 4232.
Path 334 | total_timesteps 4247.
Path 335 | total_timesteps 4256.
Path 336 | total_timesteps 4264.
Path 337 | total_timesteps 4273.
Path 338 | total_timesteps 4284.
Path 339 | total_timesteps 4299.
Path 340 | total_timesteps 4307.
Path 341 | total_timesteps 4321.
Path 342 | total_timesteps 4331.
Path 343 | total_timesteps 4338.
Path 344 | total_timesteps 4350.
Path 345 | total_timesteps 4358.
Path 346 | total_timesteps 4371.
Path 347 | total_timesteps 4395.
Path 348 | total_timesteps 4411.
Path 349 | total_timesteps 4423.
Path 350 | total_timesteps 4437.
Path 351 | total_timesteps 4446.
Path 352 | total_timesteps 4455.
Path 353 | total_timesteps 4465.
Path 354 | total_timesteps 4476.
Path 355 | total_timesteps 4498.
Path 356 | total_timesteps 4514.
Path 357 | total_timesteps 4523.
Path 358 | total_timesteps 4533.
Path 359 | total_timesteps 4546.
Path 360 | total_timesteps 4557.
Path 361 | total_timesteps 4571.
Path 362 | total_timesteps 4591.
Path 363 | total_timesteps 4605.
Path 364 | total_timesteps 4627.
Path 365 | total_timesteps 4638.
Path 366 | total_timesteps 4647.
Path 367 | total_timesteps 4656.
Path 368 | total_timesteps 4664.
Path 369 | total_timesteps 4670.
Path 370 | total_timesteps 4699.
Path 371 | total_timesteps 4710.
Path 372 | total_timesteps 4721.
Path 373 | total_timesteps 4738.
Path 374 | total_timesteps 4750.
Path 375 | total_timesteps 4760.
Path 376 | total_timesteps 4769.
Path 377 | total_timesteps 4783.
Path 378 | total_timesteps 4797.
Path 379 | total_timesteps 4809.
Path 380 | total_timesteps 4819.
Path 381 | total_timesteps 4837.
Path 382 | total_timesteps 4847.
Path 383 | total_timesteps 4855.
Path 384 | total_timesteps 4871.
Path 385 | total_timesteps 4878.
Path 386 | total_timesteps 4893.
Path 387 | total_timesteps 4911.
Path 388 | total_timesteps 4924.
Path 389 | total_timesteps 4943.
Path 390 | total_timesteps 4956.
Path 391 | total_timesteps 4974.
Path 392 | total_timesteps 4980.
Path 393 | total_timesteps 4989.
Path 394 | total_timesteps 5004.
Path 395 | total_timesteps 5035.
Path 396 | total_timesteps 5046.
Path 397 | total_timesteps 5067.
Path 398 | total_timesteps 5077.
Path 399 | total_timesteps 5089.
Path 400 | total_timesteps 5102.
Path 401 | total_timesteps 5111.
Path 402 | total_timesteps 5120.
Path 403 | total_timesteps 5132.
Path 404 | total_timesteps 5145.
Path 405 | total_timesteps 5161.
Path 406 | total_timesteps 5169.
Path 407 | total_timesteps 5185.
Path 408 | total_timesteps 5194.
Path 409 | total_timesteps 5204.
Path 410 | total_timesteps 5212.
Path 411 | total_timesteps 5219.
Path 412 | total_timesteps 5227.
Path 413 | total_timesteps 5247.
Path 414 | total_timesteps 5256.
Path 415 | total_timesteps 5266.
Path 416 | total_timesteps 5276.
Path 417 | total_timesteps 5289.
Path 418 | total_timesteps 5302.
Path 419 | total_timesteps 5311.
Path 420 | total_timesteps 5327.
Path 421 | total_timesteps 5340.
Path 422 | total_timesteps 5351.
Path 423 | total_timesteps 5362.
Path 424 | total_timesteps 5370.
Path 425 | total_timesteps 5380.
Path 426 | total_timesteps 5397.
Path 427 | total_timesteps 5406.
Path 428 | total_timesteps 5420.
Path 429 | total_timesteps 5427.
Path 430 | total_timesteps 5438.
Path 431 | total_timesteps 5448.
Path 432 | total_timesteps 5460.
Path 433 | total_timesteps 5483.
Path 434 | total_timesteps 5496.
Path 435 | total_timesteps 5505.
Path 436 | total_timesteps 5518.
Path 437 | total_timesteps 5526.
Path 438 | total_timesteps 5532.
Path 439 | total_timesteps 5544.
Path 440 | total_timesteps 5553.
Path 441 | total_timesteps 5569.
Path 442 | total_timesteps 5579.
Path 443 | total_timesteps 5593.
Path 444 | total_timesteps 5601.
Path 445 | total_timesteps 5620.
Path 446 | total_timesteps 5630.
Path 447 | total_timesteps 5641.
Path 448 | total_timesteps 5652.
Path 449 | total_timesteps 5674.
Path 450 | total_timesteps 5685.
Path 451 | total_timesteps 5702.
Path 452 | total_timesteps 5713.
Path 453 | total_timesteps 5727.
Path 454 | total_timesteps 5736.
Path 455 | total_timesteps 5756.
Path 456 | total_timesteps 5780.
Path 457 | total_timesteps 5791.
Path 458 | total_timesteps 5799.
Path 459 | total_timesteps 5821.
Path 460 | total_timesteps 5828.
Path 461 | total_timesteps 5841.
Path 462 | total_timesteps 5851.
Path 463 | total_timesteps 5863.
Path 464 | total_timesteps 5876.
Path 465 | total_timesteps 5890.
Path 466 | total_timesteps 5905.
Path 467 | total_timesteps 5916.
Path 468 | total_timesteps 5931.
Path 469 | total_timesteps 5950.
Path 470 | total_timesteps 5964.
Path 471 | total_timesteps 5977.
Path 472 | total_timesteps 5984.
Path 473 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.71    |
| Iteration     | 32       |
| MaximumReturn | 2.92     |
| MinimumReturn | -20.6    |
| TotalSamples  | 136219   |
----------------------------
